AdGuard Home sending out excessive PTR requests in its CIDR range

[state-of-affairs]

Background
I have two Raspberry Pi 4B units running the latest version of Raspberry Pi OS. Each unit has AdGuard Home installed on it and nothing more. AdGuard Home is latest version (0.106.2) and is configured identically for each unit.

The presence of two units allows one unit to provide DNS service while the other is offline (e.g., to install software patches). The units are part of 10.0.10.0/24 and have respective IP addresses of 10.0.10.6 and 10.0.10.7. DHCP functionality is disabled for both units. (DHCP is managed by my firewall.)

Problem
Both units send excess PTR requests for other clients in the 10.0.10.0/24 range. The response is always SERVFAIL. For example, AdGuard Home at 10.0.10.6 may send out a PTR request of 1.10.0.10.in-addr.arpa for which the response is SERVFAIL. The unit may also query their own host names, e.g., AdGuard Home at 10.0.10.7 may send out a PTR request of 7.10.0.10.in-addr.arpa. How do I stop this behavior? Is this behavior from AdGuard Home, or could it be from the underlying Raspberry Pi OS?

Attempted Solution
There is a box to “Enable reverse resolving of clients’ IP addresses” under Settings --> DNS Settings. I have checked and then unchecked this box to ensure this feature is off. However, an unchecked box does not stop this behavior.

Comments
I did not have this problem before the recent upgrade to the version 106.x series. However, since the 106.x series has been released, I have also patched the Raspberry Pi OS a couple of times. Unfortunately, the PTR requests are so excessive that they populate the first few slots of the “Top Queried Domains”. The AdGuard Home units also occupy the first and second slots of the “Top clients”.

[ameshkov]

To troubleshoot this issue we need to see AdGuard Home logs.

1. Configure AdGuard Home to write verbose-level log.
2. Reproduce the issue.
3. Post the log file here.

[ainar-g]

Hello. Have you tried setting the “Private DNS servers” field to the address of your router?

[ainar-g]

Hello. Is there any update on this? We would like to have some more info before releasing v0.106.3.

[state-of-affairs]

Thank you for your prompt messages. I have been unexpectedly busy yesterday and today. But I will provide further details later this evening (my time).

[N1ghtshad0w]

I upgraded to v0.106.3 the initial issue #3064 is solved, but still why does it send PTR requests to all my clients though my router is listed under “Private DNS servers” and "Upstream DNS servers" → "[/x.x.in-addr.arpa/]x.x.x.x"?

below is an image of the top queried domains over a period of 29 hours:

My router is setup to forward DNS requests to AGH and AGH is setup to use my router for PTR queries.
So shouldn't the normal behavior for AGH is to send PTR queries to the router only and once resolved it should be cached locally for further use without querying the router about it again to minimize unnecessary traffic?

[state-of-affairs]

"Hello. Have you tried setting the “Private DNS servers” field to the address of your router?"

I did this, but I am still seeing reverse look-ups in my query logs. However, instead of processing at > 1000 ms with SERVFAIL they are now reported at < 1 ms with NXDOMAIN. But I do not believe setting the private DNS server to a value (e.g., the IP address of my router) solves anything. In my default configuration “Enable reverse resolving of clients’ IP addresses” is disabled. So both AdGuard Home units should be ignoring (or refusing) PTR queries. Yet here, the PTR queries are coming from the units themselves! However, I am not sure if the PTR queries are generated by AdGuard Home. Alternatively, perhaps the Raspberry Pi OS is generating the PTR queries as the IP address is statically assigned to the hardware by the router (i.e., AdGuard Home and Raspberry Pi OS share a common IP address).

[state-of-affairs]

I have updated to 106.3 and cleared my query logs and statistics. I will see what happens over the next day or two and get back to you.

[ainar-g]

@state-of-affairs Hello. Unchecking “Enable reverse resolving of clients” does not make AGH ignore PTR or disable private DNS servers functionality, it only disables AGH looking up its clients. So, if you see some PTRs, that's perfectly fine, as long as they don't overflow the real traffic.

[EugeneOne1]

@state-of-affairs, does the issue recur?

[state-of-affairs]

@EugeneOne1

Yes, it does. Last weekend I finally spent some time trying to solve this problem. I will detail it in a post below.

[state-of-affairs]

@ainar-g

Okay. I finally found some time to try and troubleshoot this problem, and I believe I have made some progress.

In my original post I wondered if the underlying OS (i.e., the Raspberry Pi OS) was the source of the PTR requests. So I attempted three tests.

Test 1
For the Raspberry Pi 4B unit at 10.0.10.7, I wiped the SD card and installed a fresh copy of Ubuntu Server. I then installed the latest version of AdGuard Home. After configuring AdGuard Home, I reviewed the query logs of both AdGuard Home installations. Each log continued to showed PTR requests.

Test 2
I returned the unit at 10.0.10.7 to its previous configuration, i.e., with the latest version of AdGuard Home installed on the latest version of the Raspberry Pi OS. I then powered this unit off. The PTR requests stopped immediately (i.e., on the query log for the unit at 10.0.10.6). I reset the query log and let AdGuard Home run for two days. There were no further PTR requests on the query log for the unit at 10.0.10.6.

Test 3
I turned the unit at 10.0.10.7 back on but stopped its AdGuard Home service (i.e., with the command “sudo ./AdGuardHome -s stop”). There are no PTR requests even though the Raspberry Pi OS was still running.

Summary
The problem is not the Raspberry Pi OS. Instead, it seems as if AdGuard Home has problems when another instance of AdGuard Home is running on the same subnet. This behavior is very strange because “Enable reverse resolving of clients’ IP addresses” is ALWAYS UNCHECKED.

[state-of-affairs]

@ainar-g

"Hello. Unchecking “Enable reverse resolving of clients” does not make AGH ignore PTR or disable private DNS servers functionality, it only disables AGH looking up its clients. So, if you see some PTRs, that's perfectly fine, as long as they don't overflow the real traffic."

Understood. But look at the query log posted above. You can see that AdGuard Home at 10.0.10.6 is the source of the PTR requests. In other words, AdGuard Home at 10.0.10.6 is generating PTR requests that are sent to AdGuard Home at 10.0.10.7. Moreover, as I have shown in my other post, it is not the Raspberry Pi OS generating these requests.

I have also seen PTR requests where AdGuard Home at 10.0.10.6 queries its own host name, i.e., AdGuard Home at 10.0.10.6 sends out a PTR request of 6.10.0.10.in-addr.arpa to itself. This behavior is very strange.

Lastly, I would like to point out that my firewall uses AdGuard Home at 10.0.10.6 as the primary DNS server and AdGuard home 10.0.10.7 as the secondary DNS server. So AdGuard Home at 10.0.10.6 gets the vast majority of DNS queries. However, unless I leave AdGuard Home at 10.0.10.7 disabled (or powered off), the query log of AdGuard Home at 10.0.10.6 gets flooded with PTR requests. They are quite significant and can be up to 50% of my DNS traffic.

[state-of-affairs]

@ameshkov

I followed the instructions provided in your link to generate a verbose log file, but am having problems getting AdGuard Home to run. On the Raspberry Pi OS, AdGuard Home is installed in "/home/pi/AdGuardHome". So I created a sub-directory called "log" for the verbose log file. However, when I edit AdGuardHome.yaml with the following:

log_file: "/home/pi/AdGuardHome/log"

AdGuard Home stops responding through its web interface. What is the syntax expected by the *.yaml file?

[EugeneOne1]

@state-of-affairs, the value of log_file field should be a file, not a directory. You should either change it to something like /home/pi/AdGuardHome/log/agh.log or delete the directory to let AdGuard Home create the file itself.

[state-of-affairs]

@EugeneOne1

Thanks for providing this clarification. It looks like AdGuard is close releasing Ver. 0.107, which looks to address certain PTR behavior. I am going to wait until this version is released and see if it fixes my problem. If it does, I will post a message to this thread and request this ticket be closed. If not, I will collect a log and submit its contents to the AdGuard team for review.

[MartenBE]

I also have this problem. The PTR requests take up at least 11+% of my DNS traffic. I only have a single Adguard instance on my network:

It looks like the adguard instance tries to reverse lookup itself with 5 attempts every 10 minutes?

[state-of-affairs]

@MartenBE

I am not sure if the developers are monitoring this thread anymore. However, I was able to substantially reduce the frequency of PTR requests by simply turning off one of my two Raspberry Pi's. (One served as a primary instance of AdGuard Home, the other a secondary instance or back-up.) The frequency of PTR requests on my remaining Raspberry Pi is now less than 0.15%. Of course, that doesn't really help you.

I am hoping that when the developers release Ver. 0.107, this issue will be resolved and I can return to using two AdGuard Home instances. You might try posting your problem to the user group on Reddit (i.e., r/Adguard). They might be able to help you solve your problem. I would also be interested in seeing any suggestions they make.

[timkgh]

I think I'm seeing the same issue, it really wants to get a name for the WAN IP which doesn't have a PTR record.

[state-of-affairs]

@timkgh

For me, AdGuard Home is generating PTR requests for every IP address in its subnet. Oddly, it also has started generating a PTR address for one IP address in another subnet. (I run multiple subnets in my network.) I am hoping Ver. 0.107 addresses this issue. It seem like the developers are no longer paying attention to this thread.

[timkgh]

Is it possible it's some other process sending the PTR requests? I run AGH on my router, it could be that other processes running on the router are trying to resolve IP addresses, but not sure how to tell. I see PTR requests for the entire local subnet + WAN IP.

[state-of-affairs]

@timkgh

I was wondering about other processes as well. In my case, I wondered if something in the Raspberry Pi OS was sending out the PTR requests. If so, that might explain why the query log would list the IP address of my Raspberry Pi unit even if AdGuard Home was not the generating source. So I wiped the SD Card, installed Ubuntu Server, and still installed AdGuard Home. I still had the same behavior.

[agneevX]

Yes, it's the host. Happens at 10 min intervals but not always.

v0.108.0-a.72+b43aa86c

[EugeneOne1]

@state-of-affairs, @agneevX, @timkgh, hello there again and apologies for the late response. Could you please answer a couple of questions:

1. If it reproduces on v0.107.x versions?
2. What are the DNS configurations of both AGHs? Which upstream servers are used for common and private resolving?
3. Do you use another resolvers, e.g. Unbound?

[agneevX]

It does... every 10 mins on v0.107.5. Unbound is used as upstream.

[EugeneOne1]

@agneevX, thanks. Is it used for both common and private reverse resolving?

[agneevX]

That field is left blank and the two checkboxes below are also unchecked.

AGH acts as DHCP and it's used only in the LAN, so there's no further need for rDNS recursion.

[EugeneOne1]

@agneevX, do you use systemd-resolved? Also could you please try to exclude Unbound from the DNS resolving chain and check if the PTRs are still sent?

[agneevX]

do you use systemd-resolved?

Yes, the server runs Ubuntu with default network config aside from a static IP.

Also could you please try to exclude Unbound from the DNS resolving chain and check if the PTRs are still sent?

Are you sure the upstream is to blame? The rDNS queries do not even go there.

[EugeneOne1]

@agneevX, shortly, PTR requests for local addresses may come not from AGH but from the same machine. Am I right that 10.0.0.10 is the host-machine of AGH? Where is Unbound ran?

Also, Unbound may change the message's ID that spoils detecting the recursion. If excluding Unbound even temporarily is somehow impossible, could you please try to set the 127.0.0.1 into Private reverse DNS servers?

[subnut]

@EugeneOne1 I am neither using systemd-resolved nor Unbound. I still face this issue.

But, I also have my 10.500.500.500 router in both Upstream and Private DNS server fields.

What troubleshooting steps should I take?

[EugeneOne1]

@subnut, hello. Firstly, your verbose log would be rather helpful. You may send it to [email protected] with the subject like "Issue 3157".

Then, several questions to answer:

1. Do you use AGH's DHCP or some another one?
2. Which of Use private reverse DNS resolvers and Enable reverse resolving of clients' IP addresses have you got enabled?
3. Are these requests appearing in the query log? If so, who requests those?

And finally, what version of AGH do you use?

[agneevX]

Am I right that 10.0.0.10 is the host-machine of AGH? Where is Unbound ran?

You are indeed. Unbound is listening on localhost:5053/udp.

could you please try to set the 127.0.0.1 into Private reverse DNS servers?

AGH only listens for DNS on 10.0.0.10:53, so the system DNS used in the server is 127.0.0.53, which I guess eventually lands up at 10.0.0.10.

[EugeneOne1]

@agneevX, honestly, I don't believe AGH is the source of these requests directly. There is only one place in the entire source code that uses value of 10 minutes, and it's ARP clients collector which shouldn't affect DNS at all.

What does Unbound handles in your setup? It would be rather helpful if you'd set its outgoing-port-permit setting to some value and check the logs for the incoming requests with an appropriate port number. Thanks for the time you're contributing.

[agneevX]

Unbound handles domain-based forwarding for the most part.

AdGuard Home listens for DNS on 10.0.0.10:53, which is the eth0 interface. Running tcpdump on that interface did not turn up anything relevant.

So I tried the loopback interface (tcpdump -i lo udp port 53 -vvn):

15:30:26.109350 IP (tos 0x0, ttl 64, id 23039, offset 0, flags [DF], proto UDP (17), length 68)
    127.0.0.1.47352 > 127.0.0.53.53: [bad udp cksum 0xfe77 -> 0x28c7!] 18347+ PTR? 60.0.0.10.in-addr.arpa. (40)
15:30:26.109829 IP (tos 0x0, ttl 64, id 36989, offset 0, flags [DF], proto UDP (17), length 68)
    10.0.0.10.34621 > 10.0.0.10.53: [bad udp cksum 0x1455 -> 0xf299!] 39350+ PTR? 60.0.0.10.in-addr.arpa. (40)
15:30:26.110220 IP (tos 0x0, ttl 64, id 36990, offset 0, flags [DF], proto UDP (17), length 92)
    10.0.0.10.53 > 10.0.0.10.34621: [bad udp cksum 0x146d -> 0xbca0!] 39350 q: PTR? 60.0.0.10.in-addr.arpa. 1/0/0 60.0.0.10.in-addr.arpa. PTR bedroom-tv. (64)
15:30:26.110546 IP (tos 0x0, ttl 64, id 56870, offset 0, flags [DF], proto UDP (17), length 92)
    127.0.0.53.53 > 127.0.0.1.47352: [bad udp cksum 0xfe8f -> 0xf2cd!] 18347 q: PTR? 60.0.0.10.in-addr.arpa. 1/0/0 60.0.0.10.in-addr.arpa. PTR bedroom-tv. (64)

[EugeneOne1]

@agneevX, @state-of-affairs, @timkgh, @subnut, hello again. We've just pushed the newest edge build that should fix that. Shortly, we assume fetching the ARP tables causes these PTRs on some OSes since it tries to resolve the retrieved addresses. Could you please check if it fixes the issue?

[agneevX]

Thanks @EugeneOne1! I can confirm that the issue is resolved as of the latest edge image.

[EugeneOne1]

@agneevX, that's great! Thanks for your effort.

[EugeneOne1]

@state-of-affairs, @timkgh, @subnut, I'll close the issue for now. Feel free to reopen it if the issue reoccurs.

[lordraiden]

I have 2 instances of Adguard they are not related each other by config in any way but for some reason one tries to call the other every 10 minutes over port 53 why is this happening?

@EugeneOne1

[agneevX]

This is out of topic. Please create a new issue if needed @lordraiden.