Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Require special approval for intentional regs in consecutive releases. #2262

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

dgoodwin
Copy link
Contributor

Surface the problem via a unit test which will fail PRs adding an
intentional regression for two releases in a row, for the same test.

To work past this, a new overrides.json file is added in a new directory
with very limited approvers. Test output directs to a README in this
directory signalling how to add to the overrides.json, and reach out to
get approval.

@dgoodwin dgoodwin changed the title Require special approval for intentional regs in consecutive releases. WIP: Require special approval for intentional regs in consecutive releases. Jan 16, 2025
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 16, 2025
@dgoodwin
Copy link
Contributor Author

First run will intentionally fail so we can see what it looks like.

@openshift-ci openshift-ci bot requested review from cuppett and deads2k January 16, 2025 12:05
"TestID": "openshift-tests:d6b41cee7afca1c2a0b52f9e6975425f",
"TestName": "[bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above info",
"JiraBug": "https://issues.redhat.com/browse/OCPBUGS-42083",
"ReasonToAllowInsteadOfFix": "Degraded enough to trip test near the end of stabilization. Still no root cause determined, and the team is still actively investigating. We are marking this as not a blocker because we did not see signals indicating that this does not stabilize after install is completed, although that has not yet been confirmed.",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a straight dupe of 4.17 to trigger the failure, i'll remove once we see it work.

@dgoodwin dgoodwin force-pushed the block-consec-reg-allowance branch from a9e013f to 5eac98f Compare January 16, 2025 15:18
@dgoodwin dgoodwin changed the title WIP: Require special approval for intentional regs in consecutive releases. Require special approval for intentional regs in consecutive releases. Jan 16, 2025
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 16, 2025
@stbenjam
Copy link
Member

Nit, could we get it to produce junits so it's more obvious in spyglass? gotestsum can do it

@stbenjam
Copy link
Member

Nit, could we get it to produce junits so it's more obvious in spyglass? gotestsum can do it

It can also be a separate thing to do, it's not exactly related to this.

/lgtm

@openshift-ci openshift-ci bot added lgtm Indicates that a PR is ready to be merged. and removed lgtm Indicates that a PR is ready to be merged. labels Jan 16, 2025
@dgoodwin
Copy link
Contributor Author

dgoodwin commented Jan 16, 2025

Nit, could we get it to produce junits so it's more obvious in spyglass? gotestsum can do it

Pushed a commit to try, I fear it wont be in the src image we use to run make test

@sosiouxme
Copy link
Member

yup...
make: gotestsum: No such file or directory
it works locally though

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 16, 2025
Surface the problem via a unit test which will fail PRs adding an
intentional regression for two releases in a row, for the same test.

To work past this, a new overrides.json file is added in a new directory
with very limited approvers. Test output directs to a README in this
directory signalling how to add to the overrides.json, and reach out to
get approval.
@dgoodwin dgoodwin force-pushed the block-consec-reg-allowance branch from aefc1db to 3eb2d7b Compare January 16, 2025 23:00
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 16, 2025
@dgoodwin dgoodwin force-pushed the block-consec-reg-allowance branch from 3eb2d7b to 41c82f4 Compare January 17, 2025 12:00
@dgoodwin
Copy link
Contributor Author

Better failure now: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_sippy/2262/pull-ci-openshift-sippy-master-unit/1880027527498436608

PR updated to remove the forced failure now and should be ready for review.

Copy link
Contributor

openshift-ci bot commented Jan 17, 2025

@dgoodwin: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@stbenjam
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 17, 2025
Copy link
Contributor

openshift-ci bot commented Jan 17, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dgoodwin, stbenjam
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants