Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass domains as arguments similar to dig #418

Closed
wants to merge 25 commits into from

Conversation

phillip-stephens
Copy link
Contributor

@phillip-stephens phillip-stephens commented Aug 12, 2024

Closes #266

Description

Let's the user provide domains as additional arguments to ZDNS, ex: ./zdns --module=A google.com yahoo.com as long as they provide the module with --module.

  • if users want to pass domains as arguments, they'll need to specify the type with --module
  • existing usage of the sole argument being a module still works (echo "Cloudflare.com" | ./zdns A)
  • Does not allow the user to use multiple modules, that'll be addressed with Perform multiple queries in the same run #353
  • Added integration test for this usage
  • added integration test for --name-server-mode
  • added new text to the --help to show available modules
~/zdns on  phillip/266-new! ⌚ 15:15:07
$ ./zdns --help

ZDNS is a library and CLI tool for making very fast DNS requests. It's built upon
https://github.com/zmap/dns (and in turn https://github.com/miekg/dns) for constructing
and parsing raw DNS packets.

ZDNS also includes its own recursive resolution and a cache to further optimize performance.

ZDNS can take input (usually domains and a module name) in the following ways:
- file (./zdns A --input-file=domains.txt)
- stream (echo "example.com" | ./zdns A)
- as arguments (./zdns --module=A example.com google.com).

Available modules:
A             AAAA          AFSDB         ANY           ATMA          AVC
AXFR          BINDVERSION   CAA           CDNSKEY       CDS           CERT
CNAME         CSYNC         DHCID         DMARC         DNAME         DNSKEY
DS            EID           EUI48         EUI64         GID           GPOS
HINFO         HIP           HTTPS         ISDN          KEY           KX
L32           L64           LOC           LP            MB            MD
MF            MG            MR            MX            NAPTR         NID
NIMLOC        NINFO         NS            NSAPPTR       NSEC          NSEC3
NSEC3PARAM    NULL          NXT           OPENPGPKEY    PTR           PX
RP            RRSIG         RT            SMIMEA        SOA           SPF
SRV           SSHFP         SVCB          TALINK        TKEY          TLSA
TXT           UID           UINFO         UNSPEC        URI

Usage:
  zdns [flags]
  zdns [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  mxlookup    Run a more exhaustive mxlookup
  nslookup    Run a more exhaustive nslookup         

Testing

Old Behavior Unchanged

$ echo "example.com" | ./zdns A
{"data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":4096,"version":0}],"answers":[{"answer":"93.184.215.14","class":"IN","name":"example.com","ttl":2668,"type":"A"}],"protocol":"udp","resolver":"192.168.1.1:53"},"duration":0.01300825,"name":"example.com","status":"NOERROR","timestamp":"2024-08-14T16:34:10-04:00"}

Module can be specified with --module

$ echo "example.com" | ./zdns --module=A
{"data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":4096,"version":0}],"answers":[{"answer":"93.184.215.14","class":"IN","name":"example.com","ttl":2656,"type":"A"}],"protocol":"udp","resolver":"192.168.1.1:53"},"duration":0.011599709,"name":"example.com","status":"NOERROR","timestamp":"2024-08-14T16:34:22-04:00"}

User can provide domains with a module

$ ./zdns --module=A google.com example.com
{"data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":4096,"version":0}],"answers":[{"answer":"93.184.215.14","class":"IN","name":"example.com","ttl":2644,"type":"A"}],"protocol":"udp","resolver":"192.168.1.1:53"},"duration":0.011241459,"name":"example.com","status":"NOERROR","timestamp":"2024-08-14T16:34:34-04:00"}
{"data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":512,"version":0}],"answers":[{"answer":"142.250.191.238","class":"IN","name":"google.com","ttl":79,"type":"A"}],"protocol":"udp","resolver":"192.168.1.1:53"},"duration":0.034962042,"name":"google.com","status":"NOERROR","timestamp":"2024-08-14T16:34:34-04:00"}

--name-server-mode

$ echo "1.1.1.1\n8.8.8.8" | ./zdns --module=A --name-server-mode --override-name="google.com"
{"altered_name":"google.com","data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":1232,"version":0}],"answers":[{"answer":"142.251.165.101","class":"IN","name":"google.com","ttl":165,"type":"A"},{"answer":"142.251.165.102","class":"IN","name":"google.com","ttl":165,"type":"A"},{"answer":"142.251.165.139","class":"IN","name":"google.com","ttl":165,"type":"A"},{"answer":"142.251.165.138","class":"IN","name":"google.com","ttl":165,"type":"A"},{"answer":"142.251.165.100","class":"IN","name":"google.com","ttl":165,"type":"A"},{"answer":"142.251.165.113","class":"IN","name":"google.com","ttl":165,"type":"A"}],"protocol":"udp","resolver":"1.1.1.1:53"},"duration":0.023474625,"status":"NOERROR","timestamp":"2024-08-15T13:05:11-04:00"}
{"altered_name":"google.com","data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":512,"version":0}],"answers":[{"answer":"142.250.190.78","class":"IN","name":"google.com","ttl":102,"type":"A"}],"protocol":"udp","resolver":"8.8.8.8:53"},"duration":0.044990458,"status":"NOERROR","timestamp":"2024-08-15T13:05:11-04:00"}

--name-server-mode with dig-style inputs

./zdns 1.1.1.1 8.8.8.8 --module=A --name-server-mode --override-name="google.com"
{"altered_name":"google.com","data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":1232,"version":0}],"answers":[{"answer":"142.251.165.113","class":"IN","name":"google.com","ttl":163,"type":"A"},{"answer":"142.251.165.139","class":"IN","name":"google.com","ttl":163,"type":"A"},{"answer":"142.251.165.100","class":"IN","name":"google.com","ttl":163,"type":"A"},{"answer":"142.251.165.101","class":"IN","name":"google.com","ttl":163,"type":"A"},{"answer":"142.251.165.102","class":"IN","name":"google.com","ttl":163,"type":"A"},{"answer":"142.251.165.138","class":"IN","name":"google.com","ttl":163,"type":"A"}],"protocol":"udp","resolver":"1.1.1.1:53"},"duration":0.026741375,"status":"NOERROR","timestamp":"2024-08-15T13:05:05-04:00"}
{"altered_name":"google.com","data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":512,"version":0}],"answers":[{"answer":"142.250.191.174","class":"IN","name":"google.com","ttl":8,"type":"A"}],"protocol":"udp","resolver":"8.8.8.8:53"},"duration":0.043545375,"status":"NOERROR","timestamp":"2024-08-15T13:05:05-04:00"}

Arg. Order Doesn't Matter

$ ./zdns google.com --module=A example.com
{"data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":4096,"version":0}],"answers":[{"answer":"142.250.191.238","class":"IN","name":"google.com","ttl":44,"type":"A"}],"protocol":"udp","resolver":"192.168.1.1:53"},"duration":0.013578542,"name":"google.com","status":"NOERROR","timestamp":"2024-08-14T16:35:09-04:00"}
{"data":{"additionals":[{"flags":"","type":"EDNS0","udpsize":4096,"version":0}],"answers":[{"answer":"93.184.215.14","class":"IN","name":"example.com","ttl":2609,"type":"A"}],"protocol":"udp","resolver":"192.168.1.1:53"},"duration":0.013296708,"name":"example.com","status":"NOERROR","timestamp":"2024-08-14T16:35:09-04:00"}

Error States

User must specify type

echo "google.com" | ./zdns --iterative
$ echo "google.com" | ./zdns --iterative
FATA[0000] No valid DNS lookup module specified. Please provide a module to run. 
$ ./zdns google.com
FATA[0000] could not parse arguments: invalid lookup module specified - google.com. ex: zdns A or zdns --module=A. See zdns --avail-modules for more 
echo "google.com" | ./zdns
FATA[0000] No valid DNS lookup module specified. Please provide a module to run. 

Multiple types not allowed

./zdns --module=A,AAAA google.com
FATA[0000] could not parse arguments: invalid lookup module specified - A,AAAA. ex: zdns A or zdns --module=A. See zdns --avail-modules for more 

@phillip-stephens phillip-stephens changed the title Phillip/266 new Pass domains as arguments similar to dig Aug 12, 2024
@zakird
Copy link
Member

zakird commented Aug 12, 2024

It really feels offhand like the module should come first. It's very confusing to me to allow it to be in the middle or end.

@phillip-stephens
Copy link
Contributor Author

phillip-stephens commented Aug 12, 2024

@zakird
I can enforce that for sure, I was just following the dig pattern where you can specify the query type whenever:
dig google.com -t A

I don't think I have strong preferences either way. A lookup module name will never be confused with a domain name, (A is not a valid domain, for example) so I'm not worried about difficulties in parsing it or mistaking a module name for a domain or vice versa.

@phillip-stephens phillip-stephens marked this pull request as ready for review August 12, 2024 19:55
@phillip-stephens phillip-stephens requested a review from a team as a code owner August 12, 2024 19:55
src/cli/cli.go Outdated
"MR", "MX", "MXLOOKUP", "NAPTR", "NID", "NIMLOC", "NINFO", "NS", "NSAPPTR", "NSEC", "NSEC3", "NSEC3PARAM",
"NSLOOKUP", "NULL", "NXT", "OPENPGPKEY", "PTR", "PX", "RP", "RRSIG", "RT", "SMIMEA", "SOA", "SPF", "SRV",
"SSHFP", "SVCB", "TALINK", "TKEY", "TLSA", "TXT", "UID", "UINFO", "UNSPEC", "URI",
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variables get initialized and then init() gets run (where modules are registered), so we'll have to hardcode the list.

Why can't we just populate the variable at the time of init() running. I don't follow why this needs to be in the variable declaration. This feels like it's absolutely going to end up out of date.

Copy link
Contributor Author

@phillip-stephens phillip-stephens Aug 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to say earlier when I replied to the other one, I took another look at this and found another way to go about it that doesn't require this. I agree that the hard-coded list was asking for trouble.

"MXLOOKUP": {},
"NSLOOKUP": {},
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why they shouldn't be printed as part of modules. It seems like we want the user to know they exist? They are modules.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're modules in the way we mean it with ZDNS but in terms of Cobra/the CLI, they're commands. We register them as "commands" and they get printed by Cobra here:

$ ./zdns --help
...
Usage:
  zdns [flags]
  zdns [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  mxlookup    Run a more exhaustive mxlookup
  nslookup    Run a more exhaustive nslookup

So they must be called with ./zdns mxlookup., or ./zdns_binary CMD_NAME We can't have ./zdns --module=mxlookup since Cobra only initializes the command specific flags (--mx-cache-size here) if the command is used as cobra expects.

So I'd say they are modules in the ZDNS sense but we must call them like Cobra expects commands to be called. Cobra will print out mxlookup and nslookup in the Available Commands section, and we'll print out the other ZDNS modules in the new section of --help, Available modules:

Available modules:
A             AAAA          AFSDB         ANY           ATMA          AVC           
AXFR          BINDVERSION   CAA           CDNSKEY       CDS           CERT          
CNAME         CSYNC         DHCID         DMARC         DNAME         DNSKEY        
DS            EID           EUI48         EUI64         GID           GPOS          
HINFO         HIP           HTTPS         ISDN          KEY           KX            
L32           L64           LOC           LP            MB            MD            
MF            MG            MR            MX            NAPTR         NID           
NIMLOC        NINFO         NS            NSAPPTR       NSEC          NSEC3         
NSEC3PARAM    NULL          NXT           OPENPGPKEY    PTR           PX            
RP            RRSIG         RT            SMIMEA        SOA           SPF           
SRV           SSHFP         SVCB          TALINK        TKEY          TLSA          
TXT           UID           UINFO         UNSPEC        URI

In this way, the user will know about all available options.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to not just define our own --help that patches this up? Does cobra allow us to define that ourselves?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure how that would solve things.

func init() {
	rootCmd.AddCommand(mxlookupCmd)

	mxlookupCmd.PersistentFlags().Bool("ipv4-lookup", false, "perform A lookups for each MX server")
	mxlookupCmd.PersistentFlags().Bool("ipv6-lookup", false, "perform AAAA record lookups for each MX server")
	mxlookupCmd.PersistentFlags().Int("mx-cache-size", 1000, "number of records to store in MX -> A/AAAA cache")

	util.BindFlags(mxlookupCmd, viper.GetViper(), util.EnvPrefix)
}

The --ipv4-lookup and --ipv6-lookup don't matter too much because we also define those on the rootCmd, but the mx-cache-size is only defined here.

Cobra will only initialize that flag if the mxlookup command is used as a "Cobra command", ie. zdns mxlookup.

Unless we define --mx-cache-size on the root command and let it be used by all modules, we must treat it separately because Cobra handles it differently.

@phillip-stephens
Copy link
Contributor Author

Closing, will re-work post-ZFlags merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ZDNS should accept dig-like input
2 participants