Certificates not updated automatically after 90 days - fixed

Hi guys,

I've been enjoying the LetsEncrypt SSL certificates for 3 months. They were due to automatic renewal via CRON task yesterday, but the certificates didn't get updated, so visitors to the sites were presented with a nice browser warning that the certificate has expired and the site might be malitious. Not nice.

After getting it fixed, here's how I did it. Sorry for explaining it at noob level, that's where I am and maybe it helps other noobs as well


First step: resolve websites' functionality without certificate warning (some had enforced redirects from HTTP to HTTPS).

CWP-Admin -> Apache Settings -> SSL Cert Manager -> Run Auto Renewal button.

Patience - for 8 domains, the page took a few minutes to load. This action updated all domains except for one.


Second step: fix the renewal for the domain that didn't work.

In SSL Cert Manager, check which domains' certificates are still listed as expired. Sure, you can click the Run Auto Renewal button again, but let's try another way.

SSH into the server with your favorite terminal, get root access via su command.
For each domain with expired certificate, run the command:

Code: [Select]

/root/.acme.sh/acme.sh --home /root/.acme.sh/cwp_certs --renew -d www.domainname.tld
If successful, check again in the SSL Cert Manager that the certificate is now expiring in 89 days.
If not successful, you may need to add the --force parameter to the command, or investigate more thoroughly the error message.


Third step: figure out why the CRON command didn't update the certificates automatically. Investigation hints:

- The CRON tasks exists, and looks like this:

Code: [Select]

31 0 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" > /dev/null
- The CRON log in /var/log/cron simply shows that the CRON was running the /root/.acme.sh/acme.sh script every night, but with no additional info. That's because the CRON task redirects output to /dev/null

- The ACME log in /root/.acme.sh/acme.sh.log has not been doing anything successful in the past 90 days, because it ran into the same error:

Code: [Select]

_stopRenewOnError
di='/root/.acme.sh/*.*/
Not directory, skip: /root/.acme.sh/*.*/
- Checking the list of domains and certificates from the command line did not return anything:

Code: [Select]

/root/.acme.sh/acme.sh --list
- Checking into the /root/.acme.sh/acme.sh script for the occurrence of "Not directory", I find out that it occurs when it tries to update the certificates one by one, and loops through all files/folders in the ${CERT_HOME} variable. This variable is not set, it doesn't appear anywhere in the list of environment variables.

Code: [Select]

env
- Checking where the variable is defined in the same acme.sh script, apparently if it is not defined at the system level, it takes as default the location of the LetsEncrypt home folder, which IS defined as an environment variable ${LE_CONFIG_HOME}


Putting it all together, the LetsEncrypt script acme.sh has no idea that the certificate home folder on CWP is actually in /root/.acme.sh/cwp_certs, unless I explicitly mention it in the command line (as done in previous commands where --home parameter is mentioned) or, to save some keyboard ink, set the CERT_HOME environment variable to this path.

The solution:
1: Delete the incorrect CRON entry in the CWP-Admin, and add the custom entry with correct parameters:

Code: [Select]

31 0 * * * /root/.acme.sh/acme.sh --cron --home "/root/.acme.sh/cwp_certs" > /dev/null
2: Set the environment variable CERT_HOME so that the acme.sh script works correctly even without explicitly adding the --home parameter every time:

Code: [Select]

export CERT_HOME="/root/.acme.sh/cwp_certs"
3: Add the same command to set the environment variable to the /root/.acme.sh/acme.sh.env file, under the line defining the LE_WORKING_DIR variable, so that the variable is set even after a reboot.


After all this work, I am getting correct output from

Code: [Select]

/root/.acme.sh/acme.sh --list
and from

Code: [Select]

/root/.acme.sh/acme.sh --cron
without the need of other parameters. However, I will leave the explicit certificate home folder parameter in the CRON command, just in case something borks up the variables again.

For reference, my set-up is still with CentOS 6 and CWP 0.9.8.273.

Did this help? Please feel free to comment with any corrections/improvements I may have missed.

An update: the certificates are being cached by various services, for example the mail services. You will need to remember to restart all mail services and Apache for the new certificates to be loaded and used. It just happened to me that, 24 hours after renewing the certificates, the mail server was still delivering the old (and no longer existing as a file on the server) certificate to mail clients, which complained that it was expired.

This is a life saver post -- I didn't bother to do the correct step 3, and use a insecure crontab to run it every 2 month instead

0 0 1 */2 * /root/.acme.sh/acme.sh --home /root/.acme.sh/cwp_certs --force  --renew -d fdqn.domain.name > /dev/null