feat(json-reporter): added parallelIndex to TestResult #34740
Conversation
Co-authored-by: Neel Deshmukh <[email protected]> Co-authored-by: Marcelo Villalobos Diaz <[email protected]>
This comment has been minimized.
This comment has been minimized.
|
@microsoft-github-policy-service agree |
mxschmitt
changed the title
mxschmitt
changed the title
| @@ -291,6 +291,7 @@ export interface JSONReportError { | |||
|
|
|||
| export interface JSONReportTestResult { | |||
| workerIndex: number; | |||
| parallelIndex: number; | |||
There was a problem hiding this comment.
this file is actually generated by this file - so you need to add it there as well.
There was a problem hiding this comment.
Oh we missed that, I'll be sure to relay with my co-authors and get those changes made.
mxschmitt
changed the title
for the test, would you prefer to have multiple tests ran in parallel or just that parallelIndex is accessible? |
|
Ideally one test which runs e.g. 3 tests (pass, fail, pass) inside with 1 worker. The json report you'll assert then that the parallelIndex and workerIndex look correct. Why pass/fail/pass? Because we restart the worker if it fails, so test 3 gets a different workerIndex compared to test 2. |
Thank you for such a concise example and explanation as it aids us in better understanding the distinction between workerIndex and parallelIndex. So to my understanding the expected behavior for the above example could look something like this?
|
This comment has been minimized.
This comment has been minimized.
|
Yes. Maybe it would be neat to say |
Thanks so much for the extra input! My co-authors will be meeting tonight and over the weekend, I think we can approach the first example test(and get it working) then, look towards modifying the test to handle 2 workers. |
Test results for "tests 1"12 flaky38542 passed, 793 skipped Merge workflow run. |
…ts and added initial test to reporter-json.spec.ts Co-authored-by: Neel Deshmukh <[email protected]> Co-authored-by: Marcelo Villalobos Diaz <[email protected]>
There was a problem hiding this comment.
addition of parallelIndex property to overrides-testReporter.d.ts
There was a problem hiding this comment.
test for addition of parallelIndex to TestResult for 1 worker and 3 tests, in the order of pass/fail/pass.
However, on building the test we had an unexpected value for workerIndex where the range begins at 0 instead of 1?
We were under the impression that workerIndex should start at 1, then increment upon test failure(incrementing does happen upon failure of test 2). Is this behavior normal across the application or do we need to look more into this?
added parallelIndex to JSONReportTestResult and serializeTestResult
Fixes #33036