Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: more descriptive slack alert #9739

Merged
merged 14 commits into from
Nov 6, 2024
80 changes: 35 additions & 45 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -937,7 +937,6 @@ jobs:
merge-check:
runs-on: ubuntu-20.04
needs:
# must be kept in sync with rerun-check
- setup
- configure
- build
Expand Down Expand Up @@ -966,6 +965,7 @@ jobs:
- yarn-project-test
- prover-client-test
- network-test
- kind-network-smoke
- kind-network-test
- l1-contracts-test
- docs-preview
Expand All @@ -974,15 +974,30 @@ jobs:
- boxes-test
# - protocol-circuits-gates-report # non-blocking
if: always()
outputs:
failure: ${{ steps.set_failed_jobs.outputs.failure }}
failed_jobs: ${{ steps.set_failed_jobs.outputs.failed_jobs }}
steps:
- name: Check for Failures and Set Output
id: set_failed_jobs
env:
# Collect failed jobs from needs
FAILED_JOBS_JSON: ${{ toJson(needs) }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these actually the failed jobs or just all jobs that this one "needs"? If the latter then the comment and variable name are misleading.

Based on the run rule below, it looks like it's the latter, and the run rule extracts the ones that failed. But also I don't really know what I'm looking at!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, this is poorly named

run: |
echo "Processing failed jobs..."
failed_jobs=$(echo "$FAILED_JOBS_JSON" | jq -r 'to_entries[] | select(.value.result == "failure") | .key' | paste -sd "," -)
echo "$failed_jobs" > .failed
echo "failure=${{contains(needs.*.result, 'failure')}}" >> $GITHUB_OUTPUT
echo "failed_jobs=$failed_jobs" >> $GITHUB_OUTPUT

- name: Report overall success (non-draft)
if: github.event.pull_request.draft == false
env:
# We treat any skipped or failing jobs as a failure for the workflow as a whole.
FAIL: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
run: |
if [[ $FAIL == true ]]; then
echo "At least one job failed (or cancelled), merging not allowed."
echo "Jobs failed: $(cat .failed), merging not allowed."
exit 1
else
echo "All jobs succeeded, merge allowed."
Expand All @@ -997,69 +1012,44 @@ jobs:
permissions:
actions: write
needs:
# must be kept in sync with merge-check
- setup
- configure
- build
- e2e
# - bench-e2e # non-blocking
# - acir-bench # non-blocking
# - bench-summary # non-blocking
- bb-gcc
- bb-native-tests
- bb-js-test
- noir-build-acir-tests
- bb-acir-tests-bb
- bb-acir-tests-bb-ultra-plonk
- bb-acir-tests-bb-ultra-honk
- bb-acir-tests-bb-mega-honk
- bb-acir-tests-sol
- bb-acir-tests-sol-honk
- bb-acir-tests-bb-js
- noir-format
- noir-test
- noir-examples
- noir-packages-test
- noir-projects
- avm-format
- yarn-project-formatting
- yarn-project-test
- prover-client-test
- network-test
- kind-network-test
- l1-contracts-test
- docs-preview
# - bb-bench # non-blocking
- boxes
- boxes-test
# - protocol-circuits-gates-report # non-blocking
- merge-check
if: github.event.pull_request.draft == false && !cancelled()
steps:
- name: Check for Rerun
env:
# We treat any skipped or failing jobs as a failure for the workflow as a whole.
HAD_FAILURE: ${{ contains(needs.*.result, 'failure') }}
GH_REPO: ${{ github.repository }}
GH_TOKEN: ${{ github.token }}
run: |
if [[ $HAD_FAILURE == true ]] && [[ $RUN_ATTEMPT -lt 2 ]] ; then
if [[ ${{ needs.merge-check.outputs.failure }} == true ]] && [[ $RUN_ATTEMPT -lt 2 ]] ; then
echo "Retrying first workflow failure. This is a stop-gap until things are more stable."
gh workflow run rerun.yml -F run_id=${{ github.run_id }}
fi

# NOTE: we only notify failures after a rerun has occurred
notify:
runs-on: ubuntu-20.04
needs:
- merge-check
runs-on: ubuntu-20.04
if: github.event.pull_request.draft == false && github.ref == 'refs/heads/master' && failure() && github.run_attempt >= 2
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Get Authors of Recent Commit
id: get_authors
run: |
git fetch --depth=1 origin ${{ github.sha }}
authors=$(git log -1 --pretty=format:'%an <%ae>' ${{ github.sha }})
echo "authors=${authors}" >> $GITHUB_OUTPUT

- name: Send notification to aztec3-ci channel if workflow failed on master
uses: slackapi/[email protected]
with:
payload: |
{
"url": "https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
"text": "Master Github Actions failure",
"url": "https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}",
"authors": "${{ steps.get_authors.outputs.authors }}",
"failed_jobs": "${{ needs.merge-check.outputs.failed_jobs }}"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_NOTIFY_WORKFLOW_TRIGGER_URL }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_NOTIFY_WORKFLOW_TRIGGER_URL2 }}