diff --git a/teps/0084-endtoend-provenance-collection.md b/teps/0084-endtoend-provenance-collection.md new file mode 100644 index 000000000..bbdb88547 --- /dev/null +++ b/teps/0084-endtoend-provenance-collection.md @@ -0,0 +1,297 @@ +--- +status: proposed +title: end-to-end provenance collection +creation-date: '2021-09-16' +last-updated: '2021-09-16' +authors: +- '@nadgowdas' +--- + +# TEP-0084: end-to-end provenance collection + + + + + + + + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) + - [Use Cases (optional)](#use-cases-optional) +- [Requirements](#requirements) +- [Proposal](#proposal) + - [Notes/Caveats (optional)](#notescaveats-optional) + - [Risks and Mitigations](#risks-and-mitigations) + - [User Experience (optional)](#user-experience-optional) + - [Performance (optional)](#performance-optional) +- [Design Details](#design-details) +- [Test Plan](#test-plan) +- [Design Evaluation](#design-evaluation) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Infrastructure Needed (optional)](#infrastructure-needed-optional) +- [Upgrade & Migration Strategy (optional)](#upgrade--migration-strategy-optional) +- [Implementation Pull request(s)](#implementation-pull-request-s) +- [References (optional)](#references-optional) + + +## Summary + +As we are designing and building supply chain security solutions, one of the critical requirement is to be able to to capture attestable provenance for every action from code --> container. And our CICD pipelines are biggest part within this spectrum of code --> container. In this proposal, we are presenting some ideas around achieving them in comprehensive manner. + +There is some existing great work being done with ["tektoncd/chains"](https://github.com/tektoncd/chains). The objective in this proposal is to build technologies that complement and extends "chains". + + +## Motivation + +Let's consider a simple CI workflow shown below: + +![](images/0084-endtoend-prov.png) + +With "chains", we are able to capture the signed provenance for individual `taskruns`, that includes input parameters, image/command used for execution and output results. If we query provenance for the output `image`, we only get the record for `image-build` task with input parameter telling us that `image` was build from `clone-dir` path. But, we do not get or link provenance across multiple tasks in the pipeline, such that we can attest end-to-end. + + +### Goals + +* Allow automated attestation of pipeline execution from event-trigger to completion +* Attestation records are captured in popular formats like "in-toto" +* These record(s) are automatically signed and pushed to different storage backends (same techniques that chains employed) +* The attestation record(s) for pipeline execution are self-contained to perform independent provenance audit +* The attestation process is transparent to the user, in the sense user do not need to change their pipeline. + +### Non-Goals + +* Ensure performance impact on pipeline execution is minimum +* Any failure in the controller does not impact pipeline execution +* All the verifications/enforcements during pipeline execution are currently out-of-scope (possibly should be addressed by seperate pipeline admission controller) + + + + +### Use Cases (optional) + + + +## Requirements + + + +## Proposal + +In this proposal, we are suggesting following extensions for an end-to-end and comprehensive provenance: + +##### 1. Event Signing Interface +As the event-payload is received by the `EventListener`, we need an interface to introspect/query payload so we can sign it from the controller. + +##### 2. Provenance for pipelinerun +In addition to individual `taskruns`, we collect the attestation for `pipelineruns`. This would allow us to attest: (a) list of all tasks executed in the pipelines (b) order in which these tasks were executed ( c) list of shared resources created/shared across tasks + +##### 3. Attestation Format +(As an optimizion option) Instead of creating seperate attestation records for `taskrun`, `pipelinerun`, `event-payload`, create a single attestation record at the "end" of a `pipelinerun` that includes everything. + +In our running example above, with this changes, for a given `image` we can attest that: +1. A pipeline was trigger by event with attested payload +2. In the pipeline, 3 tasks were execure in this order: "git-clone" --> "security-scan" --> "image-build". +3. The "image" was built from "clone-repo" dirpath, which was populated by "git-clone" task from {repo-url, revision} which match the signed event-payload. + +These attestations help audit/validate our pipeline executions for: + +1. A pipeline was trigger by authorized event +2. The source of input parameters to our tasks. In our example the source was an event-payload, but it could be configuration resources as well. +3. List and order of all tasks performed in the pipeline + +### Notes/Caveats (optional) + + + +### Risks and Mitigations + + + +### User Experience (optional) + + + +### Performance (optional) + + + +## Design Details + + + +## Test Plan + + + +## Design Evaluation + + +## Drawbacks + + + +## Alternatives + + + +## Infrastructure Needed (optional) + + + +## Upgrade & Migration Strategy (optional) + + + +## Implementation Pull request(s) + + + +## References (optional) + + diff --git a/teps/README.md b/teps/README.md index 2ff541829..b74cfc6fe 100644 --- a/teps/README.md +++ b/teps/README.md @@ -222,3 +222,4 @@ This is the complete list of Tekton teps: |[TEP-0072](0072-results-json-serialized-records.md) | Results: JSON Serialized Records | implementable | 2021-07-26 | |[TEP-0073](0073-simplify-metrics.md) | Simplify metrics | proposed | 2021-06-23 | |[TEP-0080](0080-support-domainscoped-parameterresult-names.md) | Support domain-scoped parameter/result names | implemented | 2021-08-19 | +|[TEP-0084](0084-endtoend-provenance-collection.md) | end-to-end provenance collection | proposed | 2021-09-16 | diff --git a/teps/images/0084-endtoend-prov.png b/teps/images/0084-endtoend-prov.png new file mode 100644 index 000000000..eb4ec3b69 Binary files /dev/null and b/teps/images/0084-endtoend-prov.png differ