Certbot support using certbot nginx plugin

HISTORY.txt

@@ -1,5 +1,16 @@
 1.3.9 unreleased
 
+- Get certbot role working for provisioning on focal.
+  Certificate issuance is not yet working there due to a hopefully soon-to-be-fixed bug.
+  [smcmahon]
+
+- Add certbot test file; this tests provisioning of certbot-nginx, but not certificate issuance.
+  [smcmahon]
+
+- Add certbot playbook, role and documentation.
+  Provides LetsEncrypt support using certbot-nginx plugin.
+  [smcmahon, stevepiercy]
+
 - Add "tcp-check connect" to haproxy tcp-check sequence.
   Fixes #123.
   Thanks, jid.

docs/certbot.rst

@@ -0,0 +1,52 @@
+Certbot options
+```````````````
+
+The certbot playbook
+~~~~~~~~~~~~~~~~~~~~
+
+As a convenience, the Plone Ansible Playbook kit includes a separate
+playbook that will install certbot-nginx and create certificates as necessary for specified hostnames.
+
+The certbot playbook currently only supports Debian-family target servers.
+
+To use the certbot playbook, edit your ``local-configure.yml`` file to add a ``certbot_hosts`` list variable containing an entry for each hostname for which you wish to get a certbot certificate:
+
+.. code-block:: yaml
+
+   certbot_hosts:
+     - one.mcsmith.org
+     - two.mcsmith.org
+
+Run the playbook as you would the main playbook, adding whatever command-line switches you need (like ``-k`` or ``-K``):
+
+.. code-block:: console
+
+    ansible-playbook -k certbot.yml
+
+This will first install ``python3-certbot-nginx`` from the certbot/certbot ppa.
+Then it will create certificates as necessary for each hostname in the ``certbot_hosts`` list.
+If a certificate already exists, it will not attempt addition.
+
+Note that ``python3-certbot-nginx`` includes an auto-renewal cronjob.
+
+
+Webserver support
+~~~~~~~~~~~~~~~~~
+
+When the nginx role creates a configuration file for a virtual host, it will check TLS hostnames against the ``certbot_hosts`` list.
+If the hostname matches, the certbot certificate/key will be used automatically (unless you override this by specifying certificate/key files).
+
+Certificate/key files for certbot are expected to be in ``/etc/letsencrypt/live/HOST_NAME`` or this mechanism will fail when nginx is reloaded after configuration.
+
+
+Why is this a separate playbook?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As with the ``firewall.yml`` playbook, we want to encourage users to think and research before using the certbot playbook.
+*Let's Encrypt* is security software and is not for everyone.
+It should be used only with knowledge and deliberation and not as an autopilot choice.
+
+Note in particular that the certbot-nginx support uses root priveleges for both certificate creation and renewal.
+Some sysadmins choosing certbot may wish to set up their own creation/renewal systems to avoid this exposure.
+
+Note that even if you never run the certbot playbook, you may still find the webserver setup support useful.

docs/index.rst

@@ -24,3 +24,4 @@ Plone's Ansible Playbook
    multiserver
    restart_script
    audit
+   certbot

docs/webserver.rst

@@ -89,6 +89,27 @@ To use files that already exist on the controlled server, use:
           key: /etc/ssl/private/ssl-cert-snakeoil.key
           crt: /etc/ssl/certs/ssl-cert-snakeoil.pem
 
+Alternatively, you can use certbot to create and renew certificates.
+Certificates are in the usual ``/etc/letsencrypt/live/HOST_NAME`` folders.
+If you specify a global ``certbot_hosts`` list variable, then certificates managed by certbot from Let's Encrypt will be used for all matching hosts.
+
+.. code-block:: yaml
+
+   certbot_hosts:
+     - one.mcsmith.org
+     - two.mcsmith.org
+
+Or if you have the ``inventory_hostname`` variable defined:
+
+.. code-block:: yaml
+
+   certbot_hosts:
+     - "{{ inventory_hostname }}"
+
+Remember, the ``certbot_hosts`` variable must be global, not part of ``webserver_virtualhosts`` list.
+Also the ``certificate`` key and items under ``webserver_virtualhosts`` takes precedence over all other certificate management methods.
+If you want to use certbot, then remove the ``certificate`` block.
+
 
 Redirections, etc.
 ~~~~~~~~~~~~~~~~~~

roles/certbot/tasks/Debian.yml

@@ -1,9 +1,20 @@
 ---
 
-- name: Add certbot repository
-  apt_repository:
-    repo: 'ppa:certbot/certbot'
-    state: present
+- name: Install software-properties-common
+  package:
+    name: "software-properties-common"
+    state: latest
+
+- name: Add universe repository
+  command: "add-apt-repository universe"
+
+- name: Check for python3-certbot-nginx
+  command: "apt-cache search python3-certbot-nginx"
+  register: pcn_present
+
+- name: Add certbot repository if not present
+  command: "add-apt-repository ppa:certbot/certbot"
+  when: pcn_present.failed
 
 - name: Update packages via apt
   when: ansible_os_family == 'Debian'
@@ -12,4 +23,4 @@
 - name: Install python3-certbot-nginx
   package:
     name: "python3-certbot-nginx"
-    state: present
+    state: latest

roles/certbot/tasks/main.yml

@@ -11,8 +11,13 @@
 - name: "Generate new certificates as necessary"
   shell: >
     certbot certonly --nginx --email '{{ admin_email }}'
+    --non-interactive
     --agree-tos -d '{{ item }}'
     {% if certbot_staging %} --staging {% endif %}
   args:
     creates: "/etc/letsencrypt/live/{{ item }}/cert.pem"
   loop: "{{ certbot_hosts }}"
+
+- name: "Test renewal with a dry run."
+  shell: >
+    certbot renew --dry-run --nginx

roles/nginx/templates/host.j2

@@ -5,6 +5,9 @@ server {
 {% if item.get('certificate') %}
   ssl_certificate {{ item.certificate.crt }};
   ssl_certificate_key {{ item.certificate.key }};
+{% elif certbot_hosts is defined and item.hostname in certbot_hosts %}
+  ssl_certificate /etc/letsencrypt/live/{{ item.hostname }}/fullchain.pem;
+  ssl_certificate_key /etc/letsencrypt/live/{{ item.hostname }}/privkey.pem;
 {% else %}
   ssl_certificate /etc/nginx/ssl/{{ item.hostname }}.crt;
   ssl_certificate_key /etc/nginx/ssl/{{ item.hostname }}.key;

tests/certbot.yml

@@ -0,0 +1,56 @@
+------------------------------------------
+Test the certbot playbook for provisioning
+------------------------------------------
+    >>> sample = 'sample-medium.yml'
+
+    >>> extras = r"""
+    ... admin_email: [email protected]
+    ... plone_initial_password: admin
+    ... additional_packages:
+    ...   - curl
+    ...   - lsof
+    ... muninnode_query_ips:
+    ...     - 127.0.0.1
+    ... certbot_hosts: []
+    ... """
+
+    >>> import subprocess
+    >>> import sys
+    >>> import time
+
+Set up local-configure.yml by copying our sample.
+Append extras.
+
+    >>> with open(sample, 'r') as f:
+    ...     with open('local-configure.yml', 'w') as g:
+    ...         g.write(f.read() + extras)
+
+Vagrant up
+
+    >>> print >> sys.stderr, "Bringing up %s" % box
+    >>> run("vagrant up %s --provision-with write_vbox_cfg" % box)
+
+Vagrant provision -- unless contraindicated.
+
+    >>> if skip_provisioning:
+    ...     print >> sys.stderr, "Skipping provisioning"
+    ... else:
+    ...     print >> sys.stderr, "Provisioning"
+    ...     run("ansible-playbook -i vbox_host.cfg certbot.yml")
+
+
+And, now run tests against the box.
+
+    >>> print >> sys.stderr, "Running tests against box"
+
+Check hostname:
+
+    >>> ssh_run('which certbot').strip()
+    '/usr/bin/certbot'
+
+    >>> ssh_run('ls -d /etc/letsencrypt').strip()
+    '/etc/letsencrypt'
+
+    >>> ssh_run('certbot --help | grep "\--nginx"')
+    '  --nginx           Use the Nginx plugin for authentication & installation\r\n'
+