Large stats.db causes slowdowns in DNS lookups

[PLight14]

I have Adguard home running on a Synology NAS (HDD, no SSD). The switch from 105.2 to 106 caused dns lookup delays that did not appear in the lookup times in the query log. Websites would hang on DNS lookup for a few second. However, after clearing the history (deleting the 25mb stats.db) the delays were gone again.

This probably has something to do with a relatively slow insert (or equivalent) in the SQLite DB when it increases in size.

[ainar-g]

Hello. The file isn't an SQLite database, unfortunately, but a BoltDB one. This is tangentially related to #2290, as we've been struggling to fine a performant, portable, reliable, and pure-Go database format. We have some experiments in the works, but unfortunately I cannot give any guarantees for now.

In the mean time, we'll probably need to be smarter about how we update statistics and its impact on the overall performance. Perhaps the DB's schema needs some rethinking as well.