option to disable HTTP logs appearing in reports on demand

[ptrthomas]

a bunch of asks coming in:

• option to hide steps which are just "noise", and only have method, match and print shown
• option to switch on and off logs from within a running test

[ptrthomas]

good to have:

• declare list of headers that need to be masked for all requests. else even if you hide the "sign-in" flow, all subsequent "business" flows would still have sensitive data.
• if we could declare JsonPath and XPath expressions of request and response fields that need to be masked, that would be super

[svarunbe]

@ptrthomas

We can hide the steps starting with '*' in the report, so that all the noise will be reduced and user can have control over which step to be displayed in the report and which is not.
In the generated json files, there is a parameter called 'Keyword' and the same can be used to handle this.

But in case of a failure in the hidden step, it should be displayed in the report.

My suggestion is that hide the steps when Keyword is '*' and result is 'Passed'.

Thanks,

[ptrthomas]

@svarunbe I like this idea. we can have this behavior switched on via a * configure report = foo setting so the existing behavior can be the default.

thinking aloud, the configure report RHS should be a JSON to be extensible / future proof, this is already a pretty confusing area and I am seeing the following requirements:

• configure report = true - re-sets any fine-grained config (see below)
• configure report = false - disables all logging useful for those common / shared features where you don't want to log boring sign-in etc.
• configure report = { showAllSteps: true } - this is the default. below we talk about only the keys similar to / same level as showAllSteps
• if showAllSteps is false, any step with the Cucumber keyword / prefix * will be hidden from HTML / JSON reports. in other words only steps that begin with Given, When etc would be logged
• an exception to the above rule is the print step. I'm thinking even method, match and assert need to be prefixed with When, Then, And etc. should be reasonable !
• maskRequestHeaders: ['Authorization', 'SomeOtherSecretHeader'] - header will still be shown but the content replaced with ***
• maskResponseHeaders - like above
• maskRequestPaths: ['$.password', '$..ssnId'] - this will replace the payload values with ***
• maskResponsePaths - like above
• any step that fails would over-ride all rules and be logged / printed
• of course, mask rules are always honored

[svarunbe]

@ptrthomas

It is clean and neat.
If possible, please provide a flag to enable and disable visibility of Doc String in the report.
So that high level report can be shared across the organisation.

Thanks,

[ptrthomas]

speaking of DocString - just realized there are 2 distinct concepts:

1. logs - by default controlled by logback-test.xml- this controls what is logged to the console and the target/karate.log (by default). HTTP traffic is logged to the console / log-file only if the log-level is DEBUG (default)

2. reports - in current state, the HTTP request and responses are inserted into the cucumber JSON / HTML ONLY IF the log-level is DEBUG. I'm proposing we de-couple this. so what appears in the JSON / HTML should be independent of the console / log-file log-level. and controlled via configure

so in other words configure report = { showRequest: false, showResponse: false } can disable HTTP traffic ending up as DocString in the HTML report.

there is this configure logPrettyRequest and configure logPrettyResponse that will still work unchanged. I don't think many people use this though, since print can do the same on demand.

[ptrthomas]

this ticket ended up only implementing the ability to show / hide logs on demand. see above history for the 2 tickets split-opened for the remaining enhancements

[ptrthomas]

released 0.8.0

[djangofan]

I'm running 0.9.4.RC5 . Two problems I see relating to this . (reproducible in my repo: https://github.com/djangofan/karate-test-prime-example ):

1. If I try to configure showRequest=false or showResponse=false at the start of a test, the html cucumber-report still shows the request output. The only way I can suppress it is by turning off the http logging completely via the logback.xml
2. showAllSteps=false works but steps that call other feature files will still show on the report (i'll add this example to the above project)

I would say this issue should be re-opened in some fashion.

[ptrthomas]

@djangofan please don't use an old discussion as a reference, go with the readme: https://github.com/intuit/karate#report-verbosity - and when you open an issue, follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue - including steps to replicate etc