-
Notifications
You must be signed in to change notification settings - Fork 129
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add IgnoreNonJSONOutputLines, --ignore-non-json-output-lines
#194
Conversation
--ignore-non-json-output-lines causes gotestsum to write non-JSON test output lines to os.Stderr instead of failing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the PR! This looks great. Left a couple small suggestions.
I think a test case would also be a good addition. TestScanOutput_WithMissingEvents
might work as an example. If the Stdout
reader passes both json lines and non-json lines, and the Handler
could capture the errors.
Also the update to the README mentioned in one of the comments would be great.
I'm also happy to make those changes in the next few days if you don't have the time.
flags.BoolVar(&opts.ignoreNonJSONOutputLines, "ignore-non-json-output-lines", false, | ||
"write non-JSON 'go test' output lines to stderr instead of failing") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if we should hide this flag (similar to no-summary
below). My thinking is that this flag should only be necessary in very rare cases (possibly only when running tests for the Go project itself), and by hiding it we prevent misuse of the flag in cases where there are better options. It would also remove a potential distraction when vieweing the --help
output.
We can still include it in the README in the Custom go test
command section. There's already a note there about the previous limitation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hid (and updated the test baseline).
When I tried to put this in the readme, it seemed like that paragraph was getting long and the difference between the script's stderr vs. gotestsum
's stderr seemed muddy. So, I split the existing bit into two bullet points to try to make it a bit clearer what the new sentence is talking about. Otherwise kept it mostly the same:
Note: when using
--raw-command
, the script must follow a few rules about stdout and stderr output:
- The stdout produced by the script must only contain the
test2json
output, orgotestsum
will fail. If it isn't possible to change the script to avoid non-JSON output, you can use--ignore-non-json-output-lines
to ignore
non-JSON lines and write them togotestsum
's stderr instead.- Any stderr produced by the script will be considered an error (this behaviour is necessary because package build errors are only reported by writting to stderr, not the
test2json
stdout). Any stderr produced by tests is not considered an error (it will be in thetest2json
stdout).
Thanks for the review! Agreed on all points, I'll try to get it done today. |
Co-authored-by: Daniel Nephin <[email protected]>
Use t.Run to separate the cases, use DeepEqual to check the slices removing the need for a separate length check, and wrap a line to make the linter happy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! Looks great!
I made some minor changes to the test in the latest commit to fix the lint error and improve the failure messages in case the test ever fails.
Adds a
ScanConfig
fieldIgnoreNonJSONOutputLines
, enabled from the command line by--ignore-non-json-output-lines
. It causes gotestsum to write non-JSON 'go test' output lines toos.Stderr
instead of failing.(For #193. This is useful to handle underlying testing tools that mix
test2json
lines with other output and can't easily be changed to purely JSON.)