Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add e2e tests for multi-tenancy #145

Closed
wants to merge 7 commits into from

Conversation

binbin-li
Copy link
Owner

Description

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

  • Test A
  • Test B

Checklist:

  • Does the affected code have corresponding tests?
  • Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
  • Does this introduce breaking changes that would require an announcement or bumping the major version?
  • Do all new files have appropriate license header?

Post Merge Requirements

  • MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

Sorry, something went wrong.

@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch 6 times, most recently from c459710 to 7493545 Compare May 1, 2024 00:40
@binbin-li binbin-li force-pushed the multi-tenancy-pr-11 branch from 641a76a to f977961 Compare May 1, 2024 01:56
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch 5 times, most recently from 065e9fd to c680f5a Compare May 1, 2024 05:26
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch from c680f5a to 6d0c98d Compare May 1, 2024 06:34
@binbin-li binbin-li closed this May 1, 2024
@binbin-li binbin-li reopened this May 1, 2024
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch from 6d0c98d to 3669963 Compare May 1, 2024 07:38
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch 3 times, most recently from 830c01d to f0f77f7 Compare May 1, 2024 12:07
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch from f0f77f7 to 1d73209 Compare May 1, 2024 12:37
@@ -206,6 +206,69 @@ SLEEP_TIME=1
assert_success
}

@test "namespaced sbom/notary/cosign/licensechecker/schemavalidator verifiers test" {
Copy link

@akashsinghal akashsinghal May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a note that the other all-in-one test is already very flaky. The concurrent oras store issue has a higher chance of being triggered.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the note! It did fail quite often recently, I added namespaced test to all-in-one setup so that we can cover most namespaced resources to increase our confidence on multi-tenancy feature.

Copy link

@akashsinghal akashsinghal May 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@binbin-li I'm wondering if you think it's necessary to run an all-in-one test for this scenario. I'm concerned adding another all-in-one test will increase chances of test failures. Is there a part of all-in-one test unique to multi-tenancy that cannot be covered by a single verifier test?

@binbin-li binbin-li force-pushed the multi-tenancy-pr-11 branch from f977961 to 80c51d1 Compare May 2, 2024 11:48
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch from 1d73209 to a6d87b7 Compare May 2, 2024 12:17
@binbin-li binbin-li force-pushed the multi-tenancy-pr-11 branch from ce2415f to 758e479 Compare May 3, 2024 14:37
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch from a6d87b7 to 170a078 Compare May 3, 2024 14:38
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch from 170a078 to f0df760 Compare May 4, 2024 00:16
binbin-li and others added 5 commits May 4, 2024 08:46

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
…t#1433)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…-project#1440)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@binbin-li binbin-li force-pushed the multi-tenancy-pr-12 branch from 4726003 to 2b6c557 Compare May 6, 2024 05:51
@binbin-li binbin-li closed this Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants