Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] chart version 3.0.0 throws "Your DNS resolver at 10.43.0.10 isn't doing DNSSEC validation; Please use another resolver or enable unbound." #144

Closed
kooskaspers opened this issue Jan 27, 2022 · 24 comments
Labels
bug Something isn't working

Comments

@kooskaspers
Copy link

kooskaspers commented Jan 27, 2022

Describe the bug
Upgraded to chart version 3.0.0 (mailu 1.9.10). I've updated my values.yaml with new variables. After upgrading, the admin throws error:
Your DNS resolver at 10.43.0.10 isn't doing DNSSEC validation; Please use another resolver or enable unbound
I'm using cloudflared as a dns over https dns service. I've switched to google (8.8.8.8) after receiving this error but to no avail. I'm curious whether more people on k8s have encountered this issues on chart version 3.0.0. Anyone succesfully upgrade to 3.0.0 as well?

Environment

  • K3OS

Additional context

@kooskaspers kooskaspers added the bug Something isn't working label Jan 27, 2022
@micw
Copy link
Contributor

micw commented Jan 27, 2022

I'm running it without problems (on bare metal with rancher2). Possible solutions are described here:
Mailu/Mailu#2135

Can you try to set for the admin pot:

      dnsConfig:
        nameservers:
        - 8.8.8.8

But I'm not sure it works since the default k8s resolver is still in /etc/rsolve.conf.

@ross-worth
Copy link

I'm hitting this issue too, on a new install. Using Helm to AKS. Tried the DNS setting above, it does not have an effect when used with dnsPolicy: ClusterFirst

If you disable dnsPolicy: ClusterFirst the the DNSSEC issue disappears but DNS requests for the REDIS service then fail.

I have tried adding the below to the admin container but without success.

containers: - env: - name: DEFER_ON_TLS_ERROR value: "false"

I am using Core-DNS within the cluster.

@kooskaspers
Copy link
Author

Indeed, tried

      dnsConfig:
        nameservers:
        - 8.8.8.8

and also other dns configurations, but then the local pods can't be resolved, and it seems its depending on redis.

I'm not seem to fix it either by setting the DEFER_ON_TLS_ERROR env value to false.
Well, let's stick to mailu 1.8 then for now..

@micw
Copy link
Contributor

micw commented Jan 29, 2022

Please see current discussion in the matrix chat. Most important result to me is:

in #144 someone mis-configured his kubernetes stack. That's not a problem with the helm chart and enabling unbound is not the solution

You should check that your upstream dns (that configured on your host) has properly working DNSSec: https://dnsinstitute.com/documentation/dnssec-guide/ch03s02.html. You should also check which upstream DNS your CoreDNS actually uses.

@kooskaspers
Copy link
Author

Hey @micw, thanks for digging into this.

I started doing some tests. I created a new pod, based on an ubuntu image.
I ran:
dig @192.168.1.1 www.isc.org. A +dnssec +multiline
Result:

; <<>> DiG 9.16.1-Ubuntu <<>> @192.168.1.1 www.isc.org. A +dnssec +multiline
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13865
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
;; QUESTION SECTION:
;www.isc.org.           IN A

;; ANSWER SECTION:
www.isc.org.            30 IN CNAME dualstack.osff2.map.fastly.net.
www.isc.org.            30 IN RRSIG CNAME 13 3 300 (
                                20220215022719 20220116021519 27566 isc.org.
                                tEcTiN76kEFGNr/RJkkPMuIvJbUePv0jR72NtUfC07UV
                                /m7PAtm/X8VB3YBUPatN0A3SaqkQFMrS7NcTH4/HWw== )
dualstack.osff2.map.fastly.net. 30 IN A 151.101.2.217
dualstack.osff2.map.fastly.net. 30 IN A 151.101.66.217
dualstack.osff2.map.fastly.net. 30 IN A 151.101.130.217
dualstack.osff2.map.fastly.net. 30 IN A 151.101.194.217

;; Query time: 24 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sat Jan 29 13:39:54 CET 2022
;; MSG SIZE  rcvd: 393

It contains the DNSSEC OK (do) flag indicating the recursive server is DNSSEC-aware.
This is my CoreDNS configuration:

apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
        }
        hosts /etc/coredns/NodeHosts {
          ttl 60
          reload 15s
          fallthrough
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }
  NodeHosts: |
    192.168.1.51 kubernetes
kind: ConfigMap

"forward . /etc/resolv.conf" means it forces all non-cluster DNS lookups to go through a specific nameserver at /etc/resolv.conf
/etc/resolv.conf contains:
nameserver 192.168.1.1, thats my router. By adjusting my router configuration, i've been testing multiple DNS servers, like 8.8.8.8, 8.8.4.4, and 1.1.1.1 and 1.1.0.0, but to no avail.
Any other ideas you might have? Can I suppress the check in some way?

@kooskaspers
Copy link
Author

kooskaspers commented Jan 29, 2022

Another test:
I omitted the '@192.168.1.1' part in the dig command; to enforce the use of the default dns resolver I guess. Again, I do see the "do" flag:
dig www.isc.org. A +dnssec +multiline
result:

dig www.isc.org. A +dnssec +multiline

; <<>> DiG 9.16.1-Ubuntu <<>> www.isc.org. A +dnssec +multiline
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12016
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: 4cf78fbc45c2caaf (echoed)
;; QUESTION SECTION:
;www.isc.org.           IN A

;; ANSWER SECTION:
www.isc.org.            30 IN CNAME dualstack.osff2.map.fastly.net.
www.isc.org.            30 IN RRSIG CNAME 13 3 300 (
                                20220215022719 20220116021519 27566 isc.org.
                                tEcTiN76kEFGNr/RJkkPMuIvJbUePv0jR72NtUfC07UV
                                /m7PAtm/X8VB3YBUPatN0A3SaqkQFMrS7NcTH4/HWw== )
dualstack.osff2.map.fastly.net. 30 IN A 151.101.194.217
dualstack.osff2.map.fastly.net. 30 IN A 151.101.2.217
dualstack.osff2.map.fastly.net. 30 IN A 151.101.130.217
dualstack.osff2.map.fastly.net. 30 IN A 151.101.66.217

;; Query time: 12 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Sat Jan 29 13:48:37 CET 2022
;; MSG SIZE  rcvd: 405

Maybe the mailu-v1.9 code regarding the check whether my DNS server supports DNSSEC is not 100% correct?

@kooskaspers
Copy link
Author

kooskaspers commented Jan 29, 2022

Another test, this time I'm making use of the admin-v1.9 container itself.

I've installed dig, using 'apk', since the admin container seem to be based on the alpine image:
apk add bind-tools
I Ran:
dig A www.broadcom.com +dnssec
Result:

; <<>> DiG 9.16.20 <<>> A www.broadcom.com +dnssec
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37565
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; COOKIE: 61626f6aa71a8e91 (echoed)
;; QUESTION SECTION:
;www.broadcom.com.              IN      A

;; ANSWER SECTION:
www.broadcom.com.       30      IN      CNAME   cdn.broadcom.com.
cdn.broadcom.com.       30      IN      CNAME   www.broadcom.com.cdn.cloudflare.net.
www.broadcom.com.       30      IN      RRSIG   CNAME 8 3 300 20220202130815 20220129122802 60666 broadcom.com. aQFiHACl+IyHc/Jd5dL44Pb8nZPf4ImAfpm87eVt5sWBnHCPKHObJz4g
QiOFppxgtWxA6EAmgwObMZQHIqM7yLtbyA5WV5AgZNWgKuD34AWCL+sn 3sOfud+DGyWkjOFQ4mtiStPE/wHuuc3x7Ms6AuRAvxmZ4DvfAgEJYjsI xr2IfPcmJ+YfjEsIZW8GmAuXTG2mJwJHSYLGbhMLQNaUlU9xARZwbw6r OjniEUGPfpQjXX/4/dw+3faJnQ8WshFbWix6z7v7WWpVn4proEuod256 Wr7lpnWh3C2lrjGNLe53IAacq3lkSbBD2wiomfVEzwsr/191yHqN41EY MGl5fg==
cdn.broadcom.com.       30      IN      RRSIG   CNAME 8 3 3600 20220202124900 20220129120749 60666 broadcom.com. DKI4dYX4bylKwCZLWVCrfPbedHblCIpxknb+duG2rzyR2jR7xQU0cPte O3VeIGTr/HvQtyTZ2n5WU0qXKRUQVkDFPJ94OFTb4yeKHClDZXmt9dVO qRKGlO37V5VXWE+jkfmYywqscH2Fz8OY9nkyJcXwahImGiMYh6DQnkGw +CLoGme5Y4bcRTJPcM+NMXIOTz33oAk0byJi5Xtmzl4ts6MRBqTQbmjZ Svts/X/dHpseHF1EpjbFtXx+GjRX0gJeZ+qohNN5N1UiPidp7+WLvQGx FByleaRGyZBTtJqJpgyT2XqM9+bwRD3otjHWpYt+i/A8vW0ekTxUSDDq wlSI9w==
www.broadcom.com.cdn.cloudflare.net. 30 IN RRSIG A 13 6 300 20220130150658 20220128130658 34505 cloudflare.net. FBGQNaE3+Ogiv+3Se8Kzqkzj92hKMcIfP2Zvj6eTyDY5vHtHiLkHXmMs
M8fDgjQ9Tejm1vf55Flq4fRWw6sZyQ==
www.broadcom.com.cdn.cloudflare.net. 30 IN A    104.18.4.158
www.broadcom.com.cdn.cloudflare.net. 30 IN A    104.18.5.158

;; Query time: 12 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Sat Jan 29 15:07:07 CET 2022
;; MSG SIZE  rcvd: 1047

Conclusion:

  1. It shows the 'DO' flag again, which should mean the recursive server is DNSSEC-aware.
  2. Based on this page, we can also conclude based on the presence of the 'RRSIG' line, that I should be fine:

The interesting entry here will be the RSSIG line. If that line is there, and on a well configured setup actually shows up, they you're good to go.
If the RSSIG line does not show up, chances are your local resolver does not check the RSSIG. However, as soon as you will use that domain inside Kubernetes or Openshift using coredns, all requests for that domain will return a SERVFAIL and your deployed services will not run.

@GreatWhiteDan
Copy link

I was facing the same issue yesterday, and although all the checks I tested worked, with the right flags, in dig I could not get it to work, so I deployed unbound on a pod and it is all running great.

The default unbound config from mailu will not work since coredns does not support dnssec, so here is what I used in a configmap mapping into to the unbond.conf:

apiVersion: v1
kind: ConfigMap
metadata:
name: unbound-conf
namespace: services
data:
unbound.conf: |
server:
verbosity: 1
interface: 0.0.0.0
interface: ::0
logfile: /dev/stdout
do-ip4: yes
do-ip6: yes
do-udp: yes
do-tcp: yes
do-daemonize: no
access-control: allow
access-control: allow
directory: "/etc/unbound"
username: unbound
auto-trust-anchor-file: trusted-key.key
root-hints: "/etc/unbound/root.hints"
harden-dnssec-stripped: no
disable-dnssec-lame-check: yes
domain-insecure: "cluster.local"
hide-identity: yes
hide-version: yes
cache-min-ttl: 300
private-domain: "cluster.local"
local-zone: ".172.in-addr.arpa." nodefault
stub-zone:
name: "cluster.local"
stub-addr: @53
forward-zone:
name: "."
forward-addr:

@micw
Copy link
Contributor

micw commented Jan 29, 2022

@GreatWhiteDan thank you. I'm open to add this to the chart. Can you explain how a pod can use this and still resolve the internal addresses?
Edit: Oh, and please use code formatting markdown.

@kooskaspers
Copy link
Author

Great to hear @GreatWhiteDan. I'm also curious how the admin-pod makes use of the unbound container. And you're saying 'coredns does not support dnssec', can you elaborate on this?

@GreatWhiteDan
Copy link

Sure, you can add this to your deployment for the admin pod:
Change dnsPolicy to None and then add the rest with the IP being you service of unbound. You could use either a static cluster IP or an external IP for the service so that you can connect to unbound from the admin pod. The searches and options below are required or it will not work.

 dnsPolicy: None
  dnsConfig:
    nameservers:
      - 172.23.1.41
    searches:
      - services.svc.cluster.local
      - svc.cluster.local
      - cluster.local
    options:
      - name: ndots
        value: '5'

@nextgens
Copy link

nextgens commented Jan 30, 2022

@kooskaspers I'm afraid you're running the dig query wrong. Mailu requires a DNSSEC resolver that's doing validation as neither the admin app nor postfix can do it on their own. The AD flag says that the response data has been authenticated: that's what the admin container checks (see https://github.com/Mailu/Mailu/blob/2c1d1e995d0ff7e52684c67d72265ff3b4439341/core/admin/start.py#L42 and https://datatracker.ietf.org/doc/html/rfc6840#section-5.7 if you're interested in the specifics).

Being DNSSEC "aware" is not enough.

Please post the output of those three commands:

dig www.example.org. A +adflag
dig @IP_OF_COREDNS www.example.org. A +adflag 
dig @8.8.8.8 www.example.org. A +adflag
dig dnssec-failed.org. A
dig @IP_OF_COREDNS dnssec-failed.org. A
dig @8.8.8.8 dnssec-failed.org. A
dig @8.8.8.8 dnssec-failed.org. A +cd

None of the snippets you've pasted so far feature the AD flag, suggesting that your resolver isn't doing DNSSEC validation; This should become obvious when with the queries above and show that Mailu is right in refusing to let you use your DNS resolver as currently configured. I would expect your resolver to give you answers for dnssec-failed.org even when the CD (CheckDisabled) flag is missing.

@nextgens
Copy link

nextgens commented Jan 30, 2022

You could use (either a static cluster IP or) an external IP for the service so that you can connect to unbound from the admin pod.

@GreatWhiteDan Please be mindful that with the configuration you have posted above, ACLs are disabled and you may end up providing recursive DNS resolution services to the WWW (which will most likely make you become a DDoS relay).

access-control: allow is hardly ever a good idea.

@kooskaspers
Copy link
Author

kooskaspers commented Jan 30, 2022

@kooskaspers I'm afraid you're running the dig query wrong. Mailu requires a DNSSEC resolver that's doing validation as neither the admin app nor postfix can do it on their own. The AD flag says that the response data has been authenticated: that's what the admin container checks (see https://github.com/Mailu/Mailu/blob/2c1d1e995d0ff7e52684c67d72265ff3b4439341/core/admin/start.py#L42 and https://datatracker.ietf.org/doc/html/rfc6840#section-5.7 if you're interested in the specifics).

Well first of all, thank you for bringing me a bit up to speed regarding the topic; appreciated.

Please post the output of those three commands:

dig www.example.org. A +adflag
dig @IP_OF_COREDNS www.example.org. A +adflag 
dig @8.8.8.8 www.example.org. A +adflag
dig dnssec-failed.org. A
dig @IP_OF_COREDNS dnssec-failed.org. A
dig @8.8.8.8 dnssec-failed.org. A
dig @8.8.8.8 dnssec-failed.org. A +cd

Here we go:

bash-5.1# dig www.example.org. A +adflag

; <<>> DiG 9.16.20 <<>> www.example.org. A +adflag
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14379
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 73772f30dda1e3c6 (echoed)
;; QUESTION SECTION:
;www.example.org.               IN      A

;; ANSWER SECTION:
www.example.org.        30      IN      A       93.184.216.34

;; Query time: 92 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Sun Jan 30 15:00:00 CET 2022
;; MSG SIZE  rcvd: 87
bash-5.1# dig @10.43.0.10 www.example.org. A +adflag

; <<>> DiG 9.16.20 <<>> @10.43.0.10 www.example.org. A +adflag
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16879
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 9c759960a3792210 (echoed)
;; QUESTION SECTION:
;www.example.org.               IN      A

;; ANSWER SECTION:
www.example.org.        30      IN      A       93.184.216.34

;; Query time: 4 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Sun Jan 30 15:02:42 CET 2022
;; MSG SIZE  rcvd: 87
bash-5.1# dig @8.8.8.8 www.example.org. A +adflag

; <<>> DiG 9.16.20 <<>> @8.8.8.8 www.example.org. A +adflag
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54973
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.example.org.               IN      A

;; ANSWER SECTION:
www.example.org.        20785   IN      A       93.184.216.34

;; Query time: 4 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Jan 30 15:03:38 CET 2022
;; MSG SIZE  rcvd: 60
bash-5.1# dig dnssec-failed.org. A

; <<>> DiG 9.16.20 <<>> dnssec-failed.org. A
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 24841
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 5f1a600b912a1e11 (echoed)
;; QUESTION SECTION:
;dnssec-failed.org.             IN      A

;; Query time: 300 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Sun Jan 30 15:04:23 CET 2022
;; MSG SIZE  rcvd: 58
bash-5.1# dig @8.8.8.8 dnssec-failed.org. A

; <<>> DiG 9.16.20 <<>> @8.8.8.8 dnssec-failed.org. A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 34444
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;dnssec-failed.org.             IN      A

;; Query time: 212 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Jan 30 15:05:18 CET 2022
;; MSG SIZE  rcvd: 46
bash-5.1# dig @8.8.8.8 dnssec-failed.org. A +cd

; <<>> DiG 9.16.20 <<>> @8.8.8.8 dnssec-failed.org. A +cd
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53078
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;dnssec-failed.org.             IN      A

;; ANSWER SECTION:
dnssec-failed.org.      7200    IN      A       69.252.80.75

;; Query time: 100 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Jan 30 15:05:52 CET 2022
;; MSG SIZE  rcvd: 62

The AD flag is indeed missing when quering with the local cluster resolver, CoreDNS.

@nextgens
Copy link

I was expecting dig dnssec-failed.org. A to return an answer (like dig @8.8.8.8 dnssec-failed.org. A +cd). The fact that it didn't suggests that one of your upstream resolvers is doing the DNSSEC validation somewhere... but when querying a domain for which the validation should work the AD flags gets stripped out...

I think that Mailu behaves like expected: it warns you that something is off.

@kooskaspers
Copy link
Author

kooskaspers commented Jan 30, 2022

I was expecting dig dnssec-failed.org. A to return an answer (like dig @8.8.8.8 dnssec-failed.org. A +cd). The fact that it didn't suggests that one of your upstream resolvers is doing the DNSSEC validation somewhere... but when querying a domain for which the validation should work the AD flags gets stripped out...

I think that Mailu behaves like expected: it warns you that something is off.

Found this setting in my upstream dns-resolver:
image
I'm making use of pihole, combined with cloudflared dns-over-https (server on 192.168.1.51). After enabling this setting, I do receive the AD flag when quering directly on 192.168.1.51:

dig @192.168.1.51 www.example.org. A +adflag

; <<>> DiG 9.16.1-Ubuntu <<>> @192.168.1.51 www.example.org. A +adflag
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35066
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.example.org.               IN      A

;; ANSWER SECTION:
www.example.org.        3081    IN      A       93.184.216.34

;; Query time: 0 msec
;; SERVER: 192.168.1.51#53(192.168.1.51)
;; WHEN: Sun Jan 30 18:29:03 CET 2022
;; MSG SIZE  rcvd: 60

Now the thing is: my router forwards the DNS requests to my dns server on 192.168.1.51. So my clients (including the kubernetes cluster) have 192.168.1.1 set as their upstream DNS server. When I query 192.168.1.1 via dig, the AD flag is gone:

dig @192.168.1.1 www.example.org. A +adflag

; <<>> DiG 9.16.1-Ubuntu <<>> @192.168.1.1 www.example.org. A +adflag
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46167
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.example.org.               IN      A

;; ANSWER SECTION:
www.example.org.        85447   IN      A       93.184.216.34

;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sun Jan 30 18:30:39 CET 2022
;; MSG SIZE  rcvd: 60

This is where I configure the upstream DNS server in my router (Ubiquiti Edgerouter):
image
I'll have a try to reconfigure my clients, including kubernetes / CoreDNS to make directly use of 192.168.1.51, instead of proxying via my router on 192.168.1.1; let's see what happens then.

@nextgens
Copy link

@kooskaspers
Copy link
Author

Yep, I got it working now. Enabling DNSSEC in pihole, even though I route the DNS requests through my router to this pihole server, has resolved the problem!
Thanks everybody for pointing in the right direciton @nextgens @micw @GreatWhiteDan @ross-worth.

@lknite
Copy link

lknite commented Feb 18, 2022

Note to anyone who comes here due to a critical outage and you just need to get things back up and working. You can always use the previous version 0.2.0 for now so you can get this whole DNSSEC thing working when you have a little more time.

@jemand771
Copy link

jemand771 commented Feb 26, 2022

another note for anyone using k3s and getting this: mailuVersion is currently set to 1.9.10, manually bumping it to 1.9.20 fixes the error for me. (alternatively, just bump admin.image.tag, which defaults to mailuVersion)
coredns PR that broke it: coredns/coredns#4736
relevant coredns issue: coredns/coredns#5189
mailu PR that fixes it (workaround) Mailu/Mailu#2253

thanks to everyone that helped analyzing this!

@carldjohnston
Copy link

@kooskaspers I had to enable the proxy-dnssec option in on my edgeos router with set service dns forwarding options proxy-dnssec. You can supply dnsmasq options individually this way.

@kooskaspers
Copy link
Author

another note for anyone using k3s and getting this: mailuVersion is currently set to 1.9.10, manually bumping it to 1.9.20 fixes the error for me. (alternatively, just bump admin.image.tag, which defaults to mailuVersion) coredns PR that broke it: coredns/coredns#4736 relevant coredns issue: coredns/coredns#5189 mailu PR that fixes it (workaround) Mailu/Mailu#2253

thanks to everyone that helped analyzing this!

Thanks mate. Came back to my own topic here, after having troubles with the same error I receiver earlier!
It absolutely wasn't the DNS configuration this time, my upstream dns server had no issues handling DNSSEC. Your tip solved it. Whats the diff between 1.9.10 and 1.9.20 according DNSSEC?

@jemand771
Copy link

@kooskaspers I don't know what the workaround is (as in, how it works), but you can check out the related PR or the complete 1.9.10..1.9.20 diff

@nextgens
Copy link

nextgens commented Apr 4, 2022

The workaround is to set the DO flag (that shouldn't be required) on top of the AD flag.

Both of those commands should return responses with the AD flag ... but only the later does in buggy versions of coredns.

dig www.example.org. A +adflag @ip
dig www.example.org. A +dnssec @ip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

8 participants