Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add junit reporter #235

Merged
merged 12 commits into from
Mar 4, 2021
Merged

feat: add junit reporter #235

merged 12 commits into from
Mar 4, 2021

Conversation

vigneshshanmugam
Copy link
Member

@vigneshshanmugam vigneshshanmugam commented Mar 2, 2021 •
edited
Loading 

npx @elastic/synthetics suites --reporter=junit
// OR
npx @elastic/synthetics suites --reporter junit
<testsuites name=\\"\\" tests=\\"2\\" failures=\\"1\\" skipped=\\"1\\" errors=\\"0\\" time=\\"0\\">
   <testsuite name=\\"j1\\" tests=\\"2\\" failures=\\"1\\" skipped=\\"1\\" errors=\\"0\\">
      <testcase name=\\"s1\\" classname=\\"j1 s1\\" time=\\"1\\">
         <failure message=\\"Boom\\" type=\\"Error\\">
         Error: Boom
         </failure>
      </testcase>
      <testcase name=\\"s2\\" classname=\\"j1 s2\\" time=\\"1\\">
         <skipped message=\\"previous step failed\\">
         </skipped>
      </testcase>
   </testsuite>
</testsuites>
  • Extract the test case to a separate file using the env flag
SYNTHETICS_JUNIT_FILE='/path/to/junit.xml' npx @elastic/synthetics suites --reporter junit

@apmmachine
Copy link

apmmachine commented Mar 2, 2021 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #235 updated

  • Start Time: 2021-03-04T12:35:20.802+0000

  • Duration: 17 min 55 sec

  • Commit: 921f8c0

Test stats 🧪

Test Results
Failed 0
Passed 73
Skipped 0
Total 73

Trends 🧪

Image of Build Times

Image of Tests

vigneshshanmugam
vigneshshanmugam commented Mar 3, 2021
View reviewed changes
src/parse_args.ts
@@ -33,6 +38,10 @@ program
.option('-s, --suite-params <jsonstring>', 'Variables', '{}')
.option('-e, --environment <envname>', 'e.g. production', 'development')
.option('-j, --json', 'output newline delimited JSON')
Copy link
Member Author

@vigneshshanmugam vigneshshanmugam Mar 3, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will remove --json in next major. We can start using --reporter json from heartbeat

v1v
v1v reviewed Mar 3, 2021
View reviewed changes
__tests__/e2e/scripts/setup.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
set -xe
set -e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

less verbose output

__tests__/e2e/docker-compose.yml
@@ -28,7 +28,7 @@ services:
- elasticsearch
- synthetic
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:5601/api/status"]
test: ["CMD", "curl", "-s", "-f", "http://localhost:5601/api/status"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

less verbose output when running docker inspect to debug what's going on

__tests__/e2e/scripts/test.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
set -xe
set -eo pipefail
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Populate the error when using pipes

__tests__/e2e/scripts/test.sh
@@ -14,7 +14,7 @@ fi
##################################################
echo "" # newline
echo "${bold}Waiting for synthetics docker to start...${normal}"
until [ "$(docker inspect -f {{.State.Running}} synthetics)" == "true" ]; do
until [ "$(docker inspect -f '{{.State.Running}}' synthetics)" == "true" ]; do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cosmetic

__tests__/e2e/scripts/test.sh Outdated
@@ -23,5 +23,13 @@ echo "✅ Setup completed successfully. Running e2e tests..."
#
# run e2e tests journey
##################################################
set +e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enable error to trap the errorlevel to manipulate the junit report

This could be moved to the reporter junit but while it's not there this is the approach to run this in the CI

__tests__/e2e/scripts/test.sh Outdated Show resolved Hide resolved
v1v
v1v reviewed Mar 3, 2021
View reviewed changes
.ci/Jenkinsfile Outdated Show resolved Hide resolved
shahzad31
shahzad31 approved these changes Mar 4, 2021
View reviewed changes
Copy link
Contributor

@shahzad31 shahzad31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Great work !!

v1v
v1v reviewed Mar 4, 2021
View reviewed changes
.ci/Jenkinsfile Outdated Show resolved Hide resolved
v1v
v1v reviewed Mar 4, 2021
View reviewed changes
__tests__/e2e/scripts/test.sh Outdated Show resolved Hide resolved
__tests__/e2e/scripts/test.sh Outdated Show resolved Hide resolved
__tests__/e2e/scripts/test.sh Outdated Show resolved Hide resolved
v1v
v1v reviewed Mar 4, 2021
View reviewed changes
__tests__/e2e/scripts/test.sh Outdated Show resolved Hide resolved
@vigneshshanmugam
Copy link
Member Author

@v1v Do you want to make any further changes to the PR? Otherwise, I will go ahead and merge this and create a new release.

@vigneshshanmugam vigneshshanmugam merged commit 3129d3c into elastic:master Mar 4, 2021
@vigneshshanmugam vigneshshanmugam deleted the junit branch March 4, 2021 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shahzad31 shahzad31 shahzad31 approved these changes

@v1v v1v v1v approved these changes

@andrewvc andrewvc Awaiting requested review from andrewvc

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add support for junit file output
4 participants
@vigneshshanmugam @apmmachine @v1v @shahzad31