test: add e2e tests for multi-tenancy [multi-tenancy PR 12]

[binbin-li]

Description

What this PR does / why we need it:

This is the 12th PR for multi-tenancy support. Check the diff between PR 11 and PR 12 at: binbin-li#145

1. Added namespace to the logger values for easier debugging. e.g.
   
2. Added constraint templates for multi-tenancy scenarios and updated all references to CT in bats files. To use multi-tenancy in Ratify, constraint template should pass keys in format: [namespace]image-reference instead of single image-reference.
3. Added a e2e test for using namespaced CRs to validate images.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

• Test A
• Test B

Checklist:

• Does the affected code have corresponding tests?
• Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
• Does this introduce breaking changes that would require an announcement or bumping the major version?
• Do all new files have appropriate license header?

Post Merge Requirements

• MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.74%. Comparing base (1bd347c) to head (9223d64).
Report is 8 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1429      +/-   ##
==========================================
- Coverage   66.76%   66.74%   -0.03%     
==========================================
  Files         116      119       +3     
  Lines        6030     6134     +104     
==========================================
+ Hits         4026     4094      +68     
- Misses       1620     1659      +39     
+ Partials      384      381       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[akashsinghal]

@binbin-li cross posting discussion we were having in your fork's PR. Left another comment: https://github.com/binbin-li/ratify/pull/145/files#r1589761430

[binbin-li]

@binbin-li cross posting discussion we were having in your fork's PR. Left another comment: https://github.com/binbin-li/ratify/pull/145/files#r1589761430

Following on the main PR. That's valid concern on the test failure. I'll add the basic test case covering the multi-tenancy scenario. Thanks for calling it out!

[binbin-li]

@binbin-li cross posting discussion we were having in your fork's PR. Left another comment: https://github.com/binbin-li/ratify/pull/145/files#r1589761430

Following on the main PR. That's valid concern on the test failure. I'll add the basic test case covering the multi-tenancy scenario. Thanks for calling it out!

Added a test covering notation and cosign verifiers which will cover both cert/key fetching in KMP. And we run the validation for notation and cosign separately which could avoid the testing failure.

[akashsinghal]

Thanks for the updates. lgtm

[akashsinghal]

@binbin-li Side question: What is the guidance moving forward when we need to add e2e tests. Do we target clustered or namespaced scenarios or do we have to do both?

[binbin-li]

@binbin-li Side question: What is the guidance moving forward when we need to add e2e tests. Do we target clustered or namespaced scenarios or do we have to do both?

good question. I think we can keep focusing on cluster-wide scenarios. But if the change involves multi-tenancy or namespace, then we should add or update namespaced tests.