WATCHTOWER_NOTIFICATION_GOTIFY_TLS_SKIP_VERIFY does not work as expected

[fdw]

Describe the bug

I have a gotify server set up, behind a reverse proxy. It uses a self-signed certificate.
Now, I want watchtower to notify using that gotify instance, and I want it to connect using TLS (just skip the verification). I've set it up with

-e WATCHTOWER_NOTIFICATIONS=gotify
-e WATCHTOWER_NOTIFICATION_GOTIFY_URL="https://<my_gotify_url>/"
-e WATCHTOWER_NOTIFICATION_GOTIFY_TOKEN="<obviously secret>"
-e WATCHTOWER_NOTIFICATION_GOTIFY_TLS_SKIP_VERIFY=true

However, it fails with the error message

level=error msg="Failed to send shoutrrr notification" error="failed to send notification to Gotify: Post \"https://<my_gotify_url>/message?token=<obviously secret>\": x509: certificate signed by unknown authority" index=0 notify=no service=gotify

To Reproduce

1. Set up Gotify and reverse proxy with a self-signed certificate.
2. Set up Watchtower with the mentioned configuration
3. Watch it error out.

Expected behavior

Watchtower should ignore that the certificate was signed by an unknown authority and still send the notification over TLS.

Screenshots

Environment

ARMv7, Docker 20.10.14, Watchtower 1.4

Additional context

This seems very similar to #822, but that was closed as fixed. #866 might also be related, but the proposed fix there is to just not use TLS, which I do not want.

[github-actions]

Hi there! 👋🏼 As you're new to this repo, we'd like to suggest that you read our code of conduct as well as our contribution guidelines. Thanks a bunch for opening your first issue! 🙏

[piksel]

No, the legacy notification rewriter doesn't handle the TLS verification skipping. The plan is to deprecate skipping TLS verification and instead allowing you to point to custom certs, but that has not yet been completed. If you use shoutrrr directly it's still supported though:

-e WATCHTOWER_NOTIFICATIONS=shoutrrr
-e WATCHTOWER_NOTIFICATION_URL="gotify://<my_gotify_url>/<obviously secret>?disableTLS=yes"

[fdw]

Thank you! 🙂

I migrated to shoutrrr, but I encountered another problem: Using gotify://<my_gotify_url>/<obviously secret>?disableTLS=yes, watchtower receives a 401 from gotify, even though the token is correct.
I assumed that ?disableTLS=yes was read as part of the token, and so used gotify://<my_gotify_url>/<obviously secret>/?disableTLS=yes, which is also what the shoutrrr documentation says. But that results in invalid gotify token \"\". Any hints?

[piksel]

The slash at the end actually doesn't work correctly. The docs are wrong, or rather, it should work with the slash as well, but instead it interprets that as an empty token (with the path being the token instead).
It should work without the slash though:

Unfortunately, the error message doesn't provide much information, but since it's a 401, the message returned from the server is probably: "you need to provide a valid access token or user credentials to access this api". Does it work with:

curl -k https://<my_gotify_url>/message?token=<obviously secret> -H 'Content-Type: application/json' -d '{ "message": "Sent from CURL"}'

?

[fdw]

Yes, curl works fine. I also created a second token to use in watchtower, but that shows the same 401 result.

Unfortunately, I also can't find a way to show the token watchtower uses, neither in watchtower (with tracing) nor in gotify :(

[piksel]

I tried creating a new app token that I would not have to censor, and ran this:

❯ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock  containrrr/watchtower \
  --notification-url 'gotify://gotify.p1k.se/AvmgKXUmPLNtbfR?disableTLS=yes' \
  --trace

Output

DEBU[0000]
DEBU[0000] Sleeping for a second to ensure the docker api client has been properly initialized.
DEBU[0001] Making sure everything is sane before starting
DEBU[0001] Retrieving running containers
DEBU[0001] There are no additional watchtower containers
DEBU[0001] Watchtower HTTP API skipped.
INFO[0001] Watchtower 1.4.0
INFO[0001] Using notifications: gotify
INFO[0001] Checking all containers (except explicitly disabled with label)
INFO[0001] Scheduling first run: 2022-05-02 10:51:32 +0000 UTC
INFO[0001] Note that the first check will be performed in 23 hours, 59 minutes, 59 seconds
WARN[0001] Trace level enabled: log will include sensitive information as credentials and tokens
^CINFO[0002] Waiting for running update to be finished...

Result

Not sure if that helps, but I cannot reproduce the problem. Sorry.
There is containrrr/shoutrrr#235, which will at least show the error message returned by server (and fix the aforementioned end-slash bug).

[Petbotson]

Hey,
just wanted to drop by and ask, when we can expect this issue to be fixed?

i run into the same issue as @fdw
Running gotify with a self signed certificate and trying to send notifications from watchtower.

WATCHTOWER_NOTIFICATION_GOTIFY_TLS_SKIP_VERIFY=true
this variable does not work for me.

Thanks in advance

(also sorry if there are any mistakes. Kinda new to Github)

[KalyaSc]

@Petbotson Exactly 1 year later I encountered the same issue.

WATCHTOWER_NOTIFICATION_URL=gotify://<my_gotify_url>/?disableTLS=yes

Works 👍

[Baltimorepc]

do you have a way to do this for email as well i keep getting
Failed to send shoutrrr notification" error="error enabling StartTLS: x509: certificate signed by unknown authority" index=0 notify=no service=smtp

but it is signed with lets encrypt so its a R3