Troubleshooting - Puppet Failed to generate additional resources using 'eval generate: SSL connect returned=1'

Error Details

This error occurs after initiating a puppet agent run with:

[root@aleph2 /]# puppet agent --test 
info: Retrieving plugin
err: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=puppetmaster.cluster.ucsf.bkslab.org]
err: /File[/var/lib/puppet/lib]: Could not evaluate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=puppetmaster.cluster.ucsf.bkslab.org] Could not retrieve file metadata for puppet://puppetmaster.cluster.ucsf.bkslab.org/plugins: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=puppetmaster.cluster.ucsf.bkslab.org]
err: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=puppetmaster.cluster.ucsf.bkslab.org]
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
err: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=puppetmaster.cluster.ucsf.bkslab.org]

Causes: The key hint in the error messages is that "CRL is not yet valid for /CN=puppetmaster.cluster.ucsf.bkslab.org". This means that the time between the puppet agent and the puppetmaster is too different; they are out of sync. To address this, we need to ensure that time is the same.

Solution: Make sure ntp is turned on on both hosts and that the ntpd server they point to is working fine.

Example: I did this example on aleph2. Commands also shown from alpha (puppetmaster) to show the time drift

# Today is June 6, 2017, around 10:00 AM
[root@aleph2 network-scripts]# date
Wed May 31 19:27:21 PDT 2017
# That's not the right time at all!

# Let's look at alpha
-bash-4.1$ hostname
alpha.cluster.ucsf.bkslab.org
-bash-4.1$ date
Tue Jun  6 10:04:45 PDT 2017

# The time has to be corrected on aleph2
[root@aleph2 network-scripts]# service ntpd status
ntpd (pid  27743) is running...
# NTPD is working so what gives? I checked ntp.conf and saw that it was pointing to a public time server but I had no public network yet!  I had to resort to setting time manually.  
[root@aleph2 network-scripts]# date -s '2017-06-06 09:58'
Tue Jun  6 09:58:00 PDT 2017
# And now puppet works!
[root@aleph2 network-scripts]# puppet agent --test
notice: Run of Puppet configuration client already in progress; skipping