-
Notifications
You must be signed in to change notification settings - Fork 681
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Crawl the source map files #413
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm! - Question: By unmarshaling the json, we might detect more decoded endpoints, as many characters are escaped in the marshaled json and might not match the regexes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm !
suggesting minor changes
@Mzack9999 , a sample sourcemap file is available at below url curl https://wac-cdn.atlassian.com/static/master/4466/assets/build/js/chunks/9b24561751aa1c.js.map | jq . as you can tell running Endpoint regex and other regex on sourcemap content already generates lot of false positives this is due to
by unmarshalling and removing ideal solution would be to also add a new regex since it is quite rare to find a url like |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
./katana -u https://wac-cdn.atlassian.com/static/master/4466/assets/build/js/chunks/9b24561751aa1c.js.map -jc -ns
__ __
/ /_____ _/ /____ ____ ___ _
/ '_/ _ / __/ _ / _ \/ _ /
/_/\_\\_,_/\__/\_,_/_//_/\_,_/
projectdiscovery.io
[INF] Current katana version v1.0.2-dev (dev)
[INF] Started standard crawling for => https://wac-cdn.atlassian.com/static/master/4466/assets/build/js/chunks/9b24561751aa1c.js.map
https://wac-cdn.atlassian.com/static/master/4466/assets/build/js/chunks/9b24561751aa1c.js.map
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
./katana -u https://bitbucket.org -jc -ns -v -proxy http://127.0.0.1:8080
__ __
/ /_____ _/ /____ ____ ___ _
/ '_/ _ / __/ _ / _ \/ _ /
/_/\_\\_,_/\__/\_,_/_//_/\_,_/
projectdiscovery.io
[INF] Current katana version v1.0.4 (latest)
[INF] Started standard crawling for => https://bitbucket.org
[GET] https://bitbucket.org
[html] [GET] http://www.w3.org/2000/svg
[script] [GET] https://atl-global.atlassian.com/js/atl-global.min.js
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x10390b1fc]
goroutine 31 [running]:
github.com/projectdiscovery/utils/url.(*OrderedParams).Clone(0x0)
/Users/geekboy/go/pkg/mod/github.com/projectdiscovery/[email protected]/url/orderedparams.go:157 +0x6c
github.com/projectdiscovery/utils/url.(*URL).Clone(0x1400128fd50)
/Users/geekboy/go/pkg/mod/github.com/projectdiscovery/[email protected]/url/url.go:90 +0x204
github.com/projectdiscovery/katana/pkg/engine/parser.scriptJSFileRegexParser(0x140012a2790)
/Users/geekboy/Github/katana/pkg/engine/parser/parser.go:674 +0x29c
github.com/projectdiscovery/katana/pkg/engine/parser.ParseResponse(0x140012a2790)
/Users/geekboy/Github/katana/pkg/engine/parser/parser.go:101 +0x244
github.com/projectdiscovery/katana/pkg/engine/common.(*Shared).Do.func1()
/Users/geekboy/Github/katana/pkg/engine/common/base.go:246 +0x174
created by github.com/projectdiscovery/katana/pkg/engine/common.(*Shared).Do in goroutine 61
/Users/geekboy/Github/katana/pkg/engine/common/base.go:214 +0x250
go run . -u https://bitbucket.org -jc -ns -v -proxy http://127.0.0.1:8080
__ __
/ /_____ _/ /____ ____ ___ _
/ '_/ _ / __/ _ / _ \/ _ /
/_/\_\\_,_/\__/\_,_/_//_/\_,_/
projectdiscovery.io
[INF] Current katana version v1.0.4 (latest)
[INF] Started standard crawling for => https://bitbucket.org
[GET] https://bitbucket.org
[script] [GET] https://atl-global.atlassian.com/js/atl-global.min.js
[script] [GET] https://bitbucket.org/bower_components/
... |
Closing this due to lost context. We'll review the issue again. |
Example 1
Example 2