DietPi-LetsEncrypt | Migrate from webroot authentication back to Apache/Nginx plugin #3111

Phil1988 · 2019-09-16T10:35:04Z

Required Information

DietPi version |
G_DIETPI_VERSION_CORE=6
G_DIETPI_VERSION_SUB=25
G_DIETPI_VERSION_RC=3
G_GITBRANCH=master
G_GITOWNER=MichaIng
Distro version | stretch
Kernel version | Linux DietPi 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64 GNU/Linux
SBC device |
Virtual Machine (x86_64)

Description

I am using dietpi on a VM with following programs installed:
pi-hole
nextcloud
letsencrypt
blynk server
LASP (blynk wanted it I think...)
LAMP

The renewal worked in the past.
DietPi-LetsEncrypt is set with
Domain
Email
Redirect [On]
HSTS [Off]
Key Size 4096

I have updated dietpi and some of the programs (I remember at least the Pi-hole update) since then.

Now the renewal isnt working any more.

I am getting:

My Nextcloud is not (and never have been) accessable on port 80. Only on Port 443.
In the past the port 80 opened the standard apache site.
One thing I discovered is, that after the pi-hole update, I see the Pihole page on port 80.

Maybe not linked to this issue but maybe yes :D

I have no idea where to search for the source of this problem.
Tell me what I need to provide and I will do my best :)

Thanks a lot!

The text was updated successfully, but these errors were encountered:

MichaIng · 2019-09-16T11:53:40Z

@Phil1988
Many thanks for your report.

Certbot requires port 80 to use the http-01 challenge. Please open/forward port 80 and retry certbot renew.

General convention to force HTTPS is to set a HTTP (port 80) => HTTPS (port 443) redirect. If you did not chose this within DietPi-LetsEncrypt before, on Apache do:

cat << _EOF_ > /etc/apache2/sites-available/000-default.conf
<VirtualHost *:80>
        Redirect permanent / https://your.domain.de/
</VirtualHost>
_EOF_
a2ensite 000-default

Having Pi-hole blocking page available on webroot is default/expected. If this is really serviced via HTTP (instead of expected HTTPS) depends on how you blocked HTTP/configured Apache.

Phil1988 · 2019-09-17T08:21:47Z

@MichaIng : Thanks a lot for your fast reply.

My port 80 is forwardeds to SSL as I have access from the internet. (and get the https:// page back).
Currently I just see the warning page from my browser, saying, that this site uses HTTP Strict Transport Security (HSTS) and that the certificate has expired (Fehlercode: SEC_ERROR_EXPIRED_CERTIFICATE).

(also a quick check if HSTS to [On] on Letsencrypt could solve the issue didnt change anything)

my /etc/apache2/sites-available/000-default.conf looks like this

<VirtualHost *:80>
ServerName mydomain.de
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www

        ErrorLog ${APACHE_LOG_DIR}/error.log
        #CustomLog ${APACHE_LOG_DIR}/access.log combined
RewriteEngine on
RewriteCond %{SERVER_NAME} =mydomain.de
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,QSA,R=permanent]
</VirtualHost>

in the letsencrypt logs (/var/log/letsencrypt/letsencrypt.log) I can see:

2019-09-17 08:07:39,104:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx

I thought it should be apache instead of nginx?

on the renewal via the dietpi-launcher -> Letsencrypt, I can see
[ OK ] DietPi-LetsEncrypt | Apache2 webserver detected

I am still having no idea why the renewal process got broken.
I can just imagine that installation of blynk server (which installed LASP) maybe has destroyed some configurations...

Could this problem also come from changing from IP4->IP6?

Could I try to "delete" the letzencrypt program and install it again via the dietpi-software?
Would that change anything or break something new? :D

MichaIng · 2019-09-17T09:24:01Z

@Phil1988
If the browser states that HSTS is enabled, then changing something in dietpi-letsencrypt has no effect anymore. However HSTS is fine as long as you do not want to access with HTTP by times with the same browsers/clients, and HSTS does not break certbot, I use it myself with Apache webserver.

Server: nginx

That is the ACME server, not your webserver 😉.

As long as you can access to your webserver by entering the bare domain (that you used for the certificate) into any web browser with http:// (+ being redirected) and https://, certbot should work as well.

Please verify the following:

ls -l /var lists www as writeable by the www-data user.
Apache is running while you try to run certbot renew.
cat /etc/letsencrypt/renewal/you.domain.de.conf shows:

authenticator = apache
installer = apache

Phil1988 · 2019-09-17T10:42:16Z

@MichaIng
Thanks for your reply

As long as you can access to your webserver by entering the bare domain (that you used for the certificate) into any web browser with http:// (+ being redirected) and https://, certbot should work as well.
This is the case. Entering with http://domain redirects me... and https:// also opens the same page

Website Text in FireFox (german)

Kein Verbindungsversuch unternommen: Mögliches Sicherheitsproblem

Firefox hat ein Problem erkannt und my.domain.de nicht aufgerufen. Entweder ist die Website falsch eingerichtet oder Datum und/oder Uhrzeit auf diesem Computer sind nicht korrekt.

Das Zertifikat der Website ist wahrscheinlich abgelaufen, weshalb Firefox keine verschlüsselte Verbindung aufbauen kann.

Was können Sie dagegen tun?

my.domain.de verwendet eine Sicherheitstechnologie namens "HTTP Strict Transport Security (HSTS)", durch welche Firefox nur über gesicherte Verbindungen mit der Website verbinden darf. Daher kann keine Ausnahme für die Website hinzugefügt werden.

Datum und Uhrzeit Ihres Computers sind eingestellt auf 17.9.2019. Überprüfen Sie, ob Datum, Uhrzeit und Zeitzone in den Systemeinstellungen korrekt gesetzt sind und laden Sie my.domain.de neu.

Falls Datum und Uhrzeit bereits korrekt sind, so ist die Website wahrscheinlich falsch eingerichtet und Sie können nichts machen, um das Problem zu beheben. Sie können den Website-Administrator über das Problem benachrichtigen.

So could it be a time settings issue?

Because for blynk server I also had to change the time settings in the dietpi settings.

Server: nginx

That is the ACME server, not your webserver wink.
good to know :D

ls -l /var lists:
drwxrwxr-x 5 www-data www-data 4096 Jul 20 03:42 www
sudo systemctl status apache2 tells me thats apache is running.
cat /etc/letsencrypt/renewal/my.domain.de.conf
tells me:

# renew_before_expiry = 30 days
version = 0.10.2
archive_dir = /etc/letsencrypt/archive/my.domain.de
cert = /etc/letsencrypt/live/my.domain.de/cert.pem
privkey = /etc/letsencrypt/live/my.domain.de/privkey.pem
chain = /etc/letsencrypt/live/my.domain.de/chain.pem
fullchain = /etc/letsencrypt/live/my.domains.de/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = XXXX
authenticator = webroot
rsa_key_size = 4096
installer = apache
[[webroot_map]]
my.domain.de = /var/www

so the authenticator = webroot and not apache...

is that a hit? If yes.. changing it to
authenticator = apache
gives me:

Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA.
Attempting to renew cert from /etc/letsencrypt/renewal/my.domain.de.conf produced an unexpected error: Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA.. Skipping.

MichaIng · 2019-09-17T10:49:35Z

@Phil1988
Ahh, I just recognise that you use a veeery outdated version of certbot and most likely of DietPi then as well?

Please consider to upgrade to current version: dietpi-update 1
Otherwise at least update your certbot packages: apt install python3-certbot-apache

The webroot authentication method should work as well, we applied this during some time since the Apache+Nginx plugins failed with the message you got as well, which is due to the usage of a protocol (TLS-SNI) that was rendered as insecure and dropped by Let's Encrypt. However on current certbot versions, those webserver specific plugins got updated to again work well.

Phil1988 · 2019-09-17T11:04:55Z

I will do so and report here back..

manwhile: the version of dietpi:

DietPi v6.25.3 : 13:04 - Di 17.09.2019
 ─────────────────────────────────────────────────────
 - Device model : Virtual Machine (x86_64)
 - CPU temp : N/A
 - LAN IP : 192.168.1.100 (eth0)
 - MOTD : If G_AGUP fails with "https://apt.sonarr.tv":
          https://github.com/MichaIng/DietPi/issues/3029

MichaIng · 2019-09-17T11:07:02Z

@Phil1988
Hmm, do you run a Debian Jessie image?

Phil1988 · 2019-09-17T11:12:37Z

Debian yes... Jessie... I dont know :D

DietPi Team     : MichaIng (lead), Daniel Knight (founder)
 Image           : DietPi Core Team (pre-image: Debian)
 Web             : https://DietPi.com | https://twitter.com/dietpi_
 Patreon Legends : PINE64 community
 Donate          : https://DietPi.com/#donate
 DietPi Hosting  : Powered by https://MyVirtualServer.com

MichaIng · 2019-09-17T11:13:35Z

Ah even there v0.28 is present, which already contains the fix. However if your image is old, certbot was probably installed from source and not via APT package that time, so /etc/certbot_scripts/certbot-auto would be the binary.

To check your Debian version: echo $G_DISTRO_NAME
To update certbot, when using a Jessie image: dietpi-software reinstall 92

Phil1988 · 2019-09-17T11:15:26Z

echo $G_DISTRO_NAME
stretch

I also did run

apt install python3-certbot-apache

but checking sudo nano /etc/letsencrypt/renewal/my.domain.de.conf
still gives me the version = 0.10.2

Edit:
I am using an image from middle of 2018.... and tried at least to keep everything up to date :)

Trying

dietpi-software reinstall 92

now

MichaIng · 2019-09-17T11:16:38Z

@Phil1988
Okay its Stretch, so the certbot package should be installed, what does certbot --version say?

Phil1988 · 2019-09-17T11:17:26Z

now certbot says 0.28.0

should I try dietpi-software reinstall 92 anyway?

MichaIng · 2019-09-17T11:19:54Z

@Phil1988
Nope, is not required then.
Please try again: certbot renew

Phil1988 · 2019-09-17T11:22:43Z

Still gives me:

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/my.domain.de.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer apache
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for my.domain.de
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (my.domain.de) from /etc/letsencrypt/renewal/my.domain.de.conf produced an unexpected error: Failed authorization procedure. my.domain.de (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching https://my.domain.de/.well-known/acme-challenge/oe1iyGiGODUdjUQDfnUHKUFo2CQO7xNh2LPCl6BQEOY: Timeout during connect (likely firewall problem). Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/my.domain.de/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/my.domain.de/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: my.domain.de
   Type:   connection
   Detail: Fetching
   https://my.domain.de/.well-known/acme-challenge/oe1iyGiGODUdjUQDfnUHKUFo2CQO7xNh2LPCl6BQEOY:
   Timeout during connect (likely firewall problem)

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

MichaIng · 2019-09-17T11:36:12Z

@Phil1988
It tries webroot authentication again. Please try: certbot renew --apache

Phil1988 · 2019-09-17T11:47:13Z

Ohhh my god.. you are godlike...
so "oh my MichaIng!"
You did it!

I changed

#authenticator = webroot
authenticator = apache

in /etc/letsencrypt/renewal/ourpicloud.spdns.de.conf

and then did

certbot renew

result:

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/ourpicloud.spdns.de/fullchain.pem (success)

Thanks a lot dude! You are awesome :)
How can I buy you a beer (or tea if you prefer? ;) )

MichaIng · 2019-09-17T12:23:53Z

@Phil1988
Great, still not sure why webroot authentication did not work, however no issue as long as the native Apache plugin finally does its job 😃.

How can I buy you a beer (or tea if you prefer? ;) )

Paypal donations are always open, see e.g. "Sponsors" link at the top of GitHub page, currently its mostly coffee 😉. However I'm always happy to help and in many cases there is some positive outcome for DietPi as well:

With v6.26 patch, assure that all users are on current certbot packages and for Nginx+Apache users, if currently webroot authentication is in use, offer to switch to Nginx/Apache plugin authentication automatically.

MichaIng · 2019-09-17T16:06:50Z

I reopen this issue to not forget implementing the above mentioned steps.

MichaIng · 2019-09-18T14:58:58Z

Done: eafdd3b

Phil1988 · 2019-09-19T08:49:48Z

Well done dude !

Who will close? :)

MichaIng · 2019-09-19T11:05:08Z

@Phil1988
I will do a final test (the migration prompt) then close.

MichaIng added the Investigating 🤔 label Sep 16, 2019

MichaIng added the Enhancement 💨 label Sep 17, 2019

MichaIng added this to the v6.26 milestone Sep 17, 2019

MichaIng removed the Investigating 🤔 label Sep 17, 2019

MichaIng changed the title ~~Certbot renew failed~~ DietPi-LetsEncrypt | Migrate from webroot authentication back to Apache/Nginx plugin Sep 17, 2019

Phil1988 closed this as completed Sep 17, 2019

MichaIng reopened this Sep 17, 2019

MichaIng added Solution available 🥂 Definite solution has been done Testing/testers required 🔽 labels Sep 18, 2019

MichaIng mentioned this issue Sep 29, 2019

Open Beta v6.26 | Please help testing and hardening the upcoming release #3131

Closed

MichaIng removed the Testing/testers required 🔽 label Oct 15, 2019

MichaIng closed this as completed Oct 15, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DietPi-LetsEncrypt | Migrate from webroot authentication back to Apache/Nginx plugin #3111

DietPi-LetsEncrypt | Migrate from webroot authentication back to Apache/Nginx plugin #3111

Phil1988 commented Sep 16, 2019 •

edited by MichaIng

Loading

MichaIng commented Sep 16, 2019

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019

MichaIng commented Sep 17, 2019

MichaIng commented Sep 17, 2019

MichaIng commented Sep 18, 2019

Phil1988 commented Sep 19, 2019 •

edited

Loading

MichaIng commented Sep 19, 2019

DietPi-LetsEncrypt | Migrate from webroot authentication back to Apache/Nginx plugin #3111

DietPi-LetsEncrypt | Migrate from webroot authentication back to Apache/Nginx plugin #3111

Comments

Phil1988 commented Sep 16, 2019 • edited by MichaIng Loading

Required Information

Description

MichaIng commented Sep 16, 2019

Phil1988 commented Sep 17, 2019 • edited Loading

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019 • edited Loading

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019 • edited Loading

MichaIng commented Sep 17, 2019 • edited Loading

Phil1988 commented Sep 17, 2019 • edited Loading

MichaIng commented Sep 17, 2019 • edited Loading

Phil1988 commented Sep 17, 2019 • edited Loading

MichaIng commented Sep 17, 2019 • edited Loading

Phil1988 commented Sep 17, 2019 • edited Loading

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019

MichaIng commented Sep 17, 2019

Phil1988 commented Sep 17, 2019

MichaIng commented Sep 17, 2019

MichaIng commented Sep 17, 2019

MichaIng commented Sep 18, 2019

Phil1988 commented Sep 19, 2019 • edited Loading

MichaIng commented Sep 19, 2019

Phil1988 commented Sep 16, 2019 •

edited by MichaIng

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

MichaIng commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 17, 2019 •

edited

Loading

Phil1988 commented Sep 19, 2019 •

edited

Loading