Deploy and test each release after it's done

[afrittoli]

We should test releases after they're done (including nightly ones) and track the results.
This should use the work done in tektoncd/plumbing#75.

Ideally this should be automated E2E, and we should not use prow to trigger the pipeline, so that we can benefit use the latest Tekton syntax in the pipeline definition.

[afrittoli]

We're setting up a new cluster for dogfooding. We plan to CD tekton there on a daily basis (at least) and we can use that same Tekton to run this test.

[vdemeester]

@afrittoli related to #1121 right ? (at least this is the foundation for it)

[afrittoli]

@afrittoli related to #1121 right ? (at least this is the foundation for it)

Yeah, even though #1121 is about testing code before releasing, while this one is to test a release after it is made to ensure the build and release process was successful.

[afrittoli]

I've implemented nightly release automation with tektoncd/triggers in the dogfooding cluster. Now there are three options for triggering testing automatically:

1. trigger a cloudevent using the CloudEventPipelineResource at the end of the release pipeline. Use a dedicated event listener as a sink. There are limitations in the way binding can extract info from the POST body, specifically filtering dicts from a list based on keys is not yet possible, so it would not be possible to extract the info required to trigger the testing pipeline. An alternative here would be to build an interceptor service that prepares a simple json body for the binding to use
2. use something https://cloud.google.com/storage/docs/object-change-notification to receive the trigger from the bucket directly. The notification from the storage service would include a link to the storage, which would include the version tag (in the folder), the bucket URL and the path to the release file to be installed and tested, which should be enough info for the test pipeline to run
3. add a task to the release pipeline that issues a POST to the event listener with only the info required

(1) Is the option I prefer, because it doesn't depend on a specific cloud provider like (2) and because it's less coupled to the sink like (3).

[vdemeester]

/assign @afrittoli

[bobcatfish]

Related: #1372

[bobcatfish]

• These days dogfooding is in full swing and most issues are in https://github.com/tektoncd/plumbing
• This is the project we are using to track our current dogfooding efforts: https://github.com/orgs/tektoncd/projects/1
• This document contains our dogfooding plans