Skip to content
This repository has been archived by the owner on Jan 13, 2025. It is now read-only.

chore(infrastructure): Report screenshot test results on GitHub PR #3080

Merged
merged 40 commits into from
Jul 16, 2018

Conversation

acdvorak
Copy link
Contributor

@acdvorak acdvorak commented Jul 15, 2018

What it does

  • Posts screenshot test statuses and diff report links on every PR
    • The "Details" link takes you to the Travis CI job A) while the test is running, and B) when it fails with an error
    • The "Details" link takes you to the report page when the screenshot test completes
    • Uses the GItHub commit status API and a bot account: @mdc-web-bot
  • Adds an encrypted credential file for the GitHub API
  • Adds color and collapsible sections to Travis CI log output to make it easier to read

Notes:

  • Travis CI screenshot tests currently diff against the PR's golden.json file, so the tests will pass if the browsers render what the PR says they do. It's still up to the PR author and reviewer to diff against origin/master's golden.json and make sure the change makes sense.
  • We should create a follow up PR that links to a diff against master's golden.json instead.

How to test it

  1. Create a PR that modifies a file in packages/ or test/screenshot/ (such as this one).
  2. Scroll to the bottom of the PR in GItHub.
  3. After about 2 minutes, a screenshot-test/butter-bot status should appear - like magic.

Example output

Pending - Running screenshot tests:

01 - pending

Passed - No diffs:

02 - passed

Failed - 4 Diffs found:

03 - failed

Full page:

image

@codecov-io
Copy link

codecov-io commented Jul 15, 2018

Codecov Report

Merging #3080 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #3080   +/-   ##
======================================
  Coverage    98.3%   98.3%           
======================================
  Files         101     101           
  Lines        4368    4368           
  Branches      564     564           
======================================
  Hits         4294    4294           
  Misses         74      74

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d7ad8d...c3650b4. Read the comment docs.

@acdvorak acdvorak merged commit e73c86e into master Jul 16, 2018
@acdvorak acdvorak deleted the chore/infra/github-integration branch July 16, 2018 07:58
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants