Add reliable large-scale scanning integration test #374
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR #370 attempted to add a large-scale integration test where we'd scan the top 100 domains and attempt to verify they were actually hosted at that IP address. This turned out to be quite flaky and unreliable and was removed.
This PR is to bring a more reproducible test into ZDNS. It also runs in only 20 seconds, so it's much more efficient than the old test.
From the
README.md
documenting this test:Large Scan Integration Test
This script runs a large scan integration test for the A and ALOOKUP modules of zdns.
10,000 domains were set up in the zdns-testing.com namespace, 5k for
subdomain0.zdns-testing.com
and 5k forsubdomain1.subdomain0.zdns-testing.com
, with the following structure:subdomain1.subdomain0
andsubdomain0
) have unique nameservers compared to eachother and tozdns-testing.com
zdns-testing.com
->subdomain0.zdns-testing.com
andsubdomain0.zdns-testing.com
->subdomain1.subdomain0.zdns-testing.com
.This should enable us to test the iterative resolution of zdns in a reproducible and reliable manner.
Changes
domains.csv
which should have been cleaned up when I removed the old test