TEP-0087: Custom Tasks Graduation

This change adds the Tekton Enhancement Proposal for `Custom Tasks` Graduation.

Today, `Custom Tasks` shared by the Tekton community are all *experimental* and we haven't
defined a process to promote them beyond *experimental*.

As such:
- Users can't depend on the `Custom Tasks` because they not *stable* (they can change any time).
- Contributors don't have a process to stabilize their `Custom Tasks` or integrate them natively
to the *Tekton Pipelines API*.

In this TEP, we aim to:
- Define the graduation path for `Custom Tasks` from *experimental* to *stable* to *integrated*.
- Provide infrastructure for *stable* `Custom Tasks` that are officially supported by Tekton.

`Custom Tasks` will have three levels in their graduation path:

1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on
them.
2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
3. *Integrated*: `Custom Tasks`' functionalities are natively supported in the *Tekton Pipelines API*.

See also [TEP-0002: Enable Custom Tasks](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md).

[api-policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#alpha-beta-and-ga

teps/0087-custom-tasks-graduation.md

@@ -0,0 +1,309 @@
+---
+status: proposed
+title: Custom Tasks Graduation
+creation-date: '2021-09-28'
+last-updated: '2021-09-30'
+authors:
+- '@jerop'
+see-also:
+- TEP-0002
+---
+
+# TEP-0087: Custom Tasks Graduation
+
+<!-- toc -->
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+  - [Use Cases](#use-cases)
+  - [Requirements](#requirements)
+- [Proposal](#proposal)
+  - [Experimental](#experimental)
+    - [Admission Requirements](#admission-requirements)
+    - [Admission Process](#admission-process)
+  - [Stable](#stable)
+    - [Graduation Requirements](#graduation-requirements)
+    - [Graduation Process](#graduation-process)
+  - [Integrated](#integrated)
+    - [Graduation Requirements](#graduation-requirements-1)
+    - [Graduation Process](#graduation-process-1)
+- [Design Evaluation](#design-evaluation)
+    - [Simplicity](#simplicity)
+    - [Reusability](#reusability)
+    - [Flexibility](#flexibility)
+- [Alternatives](#alternatives)
+    - [Experimental to Integrated](#experimental-to-integrated)
+        - [Pros](#pros)
+        - [Cons](#cons)
+    - [Experimental and Stable in One Repository](#experimental-and-stable-in-one-repository)
+        - [Pros](#pros-1)
+        - [Cons](#cons-1)
+- [Infrastructure Needed](#infrastructure-needed)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Today, `Custom Tasks` shared by the Tekton community are all *experimental* and we haven't 
+defined a process to promote them beyond *experimental*. 
+
+As such:
+- Users can't depend on the `Custom Tasks` because they not *stable* (they can change any time).
+- Contributors don't have a process to stabilize their `Custom Tasks` or integrate them natively 
+to the *Tekton Pipelines API*.
+
+In this TEP, we aim to:
+- Define the graduation path for `Custom Tasks` from *experimental* to *stable* to *integrated*.
+- Provide infrastructure for *stable* `Custom Tasks` that are officially supported by Tekton.
+
+`Custom Tasks` will have three levels in their graduation path:
+
+1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on 
+them.
+2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
+3. *Integrated*: `Custom Tasks`' functionalities are natively supported in the *Tekton Pipelines API*.
+
+## Motivation
+
+As described in [TEP-0002: Enable Custom Tasks][TEP-0002], we introduced `Custom Tasks` that are defined as CRDs that 
+are executed using [`Runs`][runs]. These `Custom Tasks` have reconciling controllers that watch for `Runs` referencing 
+their types and updates their status. `Custom Tasks` enable extensibility in Tekton - users can implement functionality 
+that's not supported natively in *Tekton Pipelines API*. 
+
+Hitherto, we have implemented several `Custom Tasks` in the [*Tekton Experimental*][experimental-repo] repository:
+- [Common Expression Language Custom Tasks][cel-ct] - provides support for Common Expression Language in *Tekton Pipelines*.
+- [Wait Custom Tasks][wait-ct] - enables waiting for some amount of time between `Tasks`.
+- [Task Looping Custom Tasks][tl-ct] - enables running a `Task` in a loop with varying `Parameter` values.
+- [Pipelines in Pipelines Custom Tasks][pip-ct] - provide support for composing and executing `Pipelines` in `Pipelines`.
+
+These `Custom Tasks` implementations are all *experimental*. Notwithstanding, they are in use in critical projects, such 
+as [Kubeflow Pipelines on Tekton][kubeflow]. Providing stability levels and progression for `Custom Tasks` will enable 
+users to rely on them and empower contributors to safely update them. 
+
+We may decide to promote some `Custom Tasks` to top level such that their functionality is supported natively in the 
+*Tekton Pipelines API*. On the other hand, we may decide not to natively support some functionalities provided in some 
+`Custom Tasks` but we need to provide stability guarantees. Thus, we need to provide an incremental graduation path and 
+stability levels for `Custom Tasks`. 
+
+### Goals
+
+1. Provide a graduation path for `Custom Tasks` from *experimental* to *stable* and *integrated*.
+2. Provide infrastructure for *stable* `Custom Tasks` that are officially supported by Tekton.
+
+### Non-Goals
+
+1. Promote `Custom Tasks` feature itself from `alpha` to `beta` - we will address this separately soon.
+2. Provide CLI support for `Custom Tasks` provided as *stable* integrations - we can explore this later.
+
+### Use Cases
+
+1. As a contributor, I want to promote my `Custom Tasks` from *experimental* to *stable* and, possibly, *integrated* so 
+I need a process that I can follow with clear requirements for graduation to the next level.
+2. As a user, I need to use *stable* `Custom Tasks` that I can rely on for critical projects.
+
+### Requirements
+
+- Define graduation requirements and path for `Custom Tasks` from *experimental* to *stable* and *integrated*.
+- Provide a catalog of `Custom Tasks` that are *stable* integrations with necessary infrastructure and processes, such 
+as thorough testing and frequent releases.
+
+
+## Proposal
+
+`Custom Tasks` will have three levels in their graduation path:
+
+1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on
+   them.
+2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
+3. *Integrated*: `Custom Tasks`' functionalities are natively supported in the *Tekton Pipelines API*.
+
+### Experimental
+
+`Custom Tasks` will start as *experimental* to keep the barrier of sharing integrations low. 
+
+The [*alpha* API policy][api-policy] applies to the *experimental* `Custom Tasks`, meaning their Owners can make any
+changes at any time as they iterate on the `Custom Tasks`.
+
+#### Admission Requirements
+
+We can consider admitting an *experimental* `Custom Task` *stable* if:
+
+1. A `Custom Task` is needed to provide a specific functionality to solve common Continuous Delivery use cases.
+2. At least two contributors are interested in owning a `Custom Task` that provides that functionality.
+
+#### Admission Process
+
+As described in [Tekton's community process][proposing-projects] for proposing new projects, the `Custom Tasks` Owners 
+would:
+
+1. Contributors propose the experimental `Custom Task` in the [Tekton API working group meeting][api-wg].
+2. Contributors file an issue in the [Tekton Community repository][community-repo] describing the problem the 
+`Custom Task` would solve and who will own it.
+3. When at least two *Tekton Pipelines* Owners approve the issue, the contributors can add the `Custom Task` to the 
+[*Tekton Experimental*][experimental-repo] repository (N/B: other experimental projects need Governing Board approval).
+
+### Stable
+
+Tekton will add a new GitHub repository - *Tekton Custom Tasks* - that would contain high quality `Custom Tasks`. These are 
+extensions that users can rely on to use functionality that's not provided in the *Tekton Pipelines API* directly. The 
+Tekton Pipelines Owners will be the Owners of this repository, and each `Custom Task` will have its own Owners. 
+
+The [*beta* API policy][api-policy] applies to the *stable* `Custom Tasks`, meaning that:
+- Any [backwards incompatible][backwards] changes must be introduced in a backwards compatible manner first, with a 
+deprecation warning in the release notes and migration instructions.
+- Users will be given at least 9 months to migrate before a backward incompatible change is made.
+- Backwards incompatible changes have to be approved by more than half of the `Custom Task`'s Owners.
+
+The *Tekton Custom Tasks* repository will provide the testing infrastructure needed by stable *Custom Tasks*. We will make 
+monthly releases of `Custom Tasks` that have had changes since their latest release, which we can automate. Moreover, 
+we will have nightly releases of each `Custom Task` to catch any failures. Each *stable* `Custom Task` will have 
+detailed documentation and multiple examples, and be used in dogfooding if we have use cases for it.
+
+We can explore a bundling releases of *Tekton Pipelines* and its latest compatible *stable* `Custom Tasks`, such that 
+users can install *Tekton Pipelines* and `Custom Tasks` together.
+
+#### Graduation Requirements
+
+We can consider graduating a given *experimental* `Custom Task` to *stable* if it meets the following requirements:
+
+1. It is thoroughly tested.
+2. It has nightly releases.
+3. It provides several examples or samples.
+4. It has detailed documentation for installation and usage.
+5. It is used in dogfooding if we have use cases for it.
+6. It has received feedback from the community (solicited in working group, mailing list or other channels).
+
+#### Graduation Process
+
+The graduation process from *experimental* to *stable* for a given `Custom Task` would be:
+
+1. Its Owners validate that the *experimental* `Custom Task` meets the above requirements.
+2. Its Owners open a Tekton Enhancement Proposal (TEP) for the functionality provided by the `Custom Task` with:
+   1. A nomination for graduation from *experimental* to *stable*.
+   2. The motivation (goals, use cases and requirements).
+   3. The proposal (syntax, design details, design evaluation and alternatives).
+3. Its Owners nominate the `Custom Task` for graduation from *experimental* to *stable*.
+4. When the *Tekton Pipelines* Owners have approved the TEP as implementable, the `Custom Task`'s Owners will:
+   1. Migrate the `Custom Task` from the *Tekton Experimental* repository to the *Tekton Custom Tasks* repository.
+   2. Mark the *experimental* to *stable* TEP as *implemented* 
+
+### Integrated
+
+When a given *stable* `Custom Task` is actively used and needed for common Continuous Delivery use cases, we can 
+consider making it *integrated* - meaning that its functionality is natively supported in the *Tekton Pipelines API*. 
+Not all *stable* `Custom Tasks` have to be *integrated*; a *stable* `Custom Task` that improves user experience but
+is not really necessary in *Tekton Pipelines API* can stay as a *stable* `Custom Task` in the long term. 
+
+#### Graduation Requirements
+
+We can consider graduating a given *stable* `Custom Task` to *integrated* if it meets the following requirements:
+
+1. Its functionality is critical to common Continuous Delivery use cases.
+2. It's actively used by the community and in dogfooding (if we have use cases for it).
+
+#### Graduation Process
+
+The graduation process from *stable* to *integrated* for a given `Custom Task` would be:
+
+1. Its Owners validate that the *stable* `Custom Task` meets the above requirements.
+2. Its Owners open a Tekton Enhancement Proposal (TEP) for the integrating the functionality provided by the 
+`Custom Task` directly to the *Tekton Pipelines API* with: 
+   1. A nomination for graduation from *stable* to *integrated*.
+   2. The motivation (goals, use cases and requirements) - a discussion of how it's been useful and why it's necessary 
+      to natively support the functionality in the *Tekton Pipelines API*.
+   3. The proposal (syntax, design details, design evaluation and alternatives) for its integration to *Tekton Pipelines*. 
+3. When the *Tekton Pipelines* Owners have approved the TEP as implementable, the `Custom Task`'s Owners will:
+   1. Add its functionality directly to the *Tekton Pipelines API* as an *alpha* feature.
+   2. Deprecate the *stable* `Custom Task`.
+   3. Mark the *stable* to *integrated* TEP as *implemented*
+   4. Mark the *experimental* to *stable* TEP as *superseded-by* the *stable* to *integrated* TEP
+4. When the *alpha* feature is promoted to *beta*, the `Custom Task`'s Owners will remove it from the *Tekton Custom Tasks* 
+repository.
+
+## Design Evaluation
+
+#### Simplicity
+
+By providing *stable* `Custom Tasks`, we provide an intermediary stability tier that provides the reliability needed by
+users without making unnecessary changes directly in the *Tekton Pipelines API*. This ensures that the *Tekton Pipelines* 
+API has the bare minimum features needed to solve most Continuous Delivery use cases.
+
+#### Reusability
+
+By providing *stable* `Custom Tasks`, we enable users to reuse extensions that they can rely on. Moreover, having both 
+*experimental* and *stable* `Custom Tasks` allows contributors to share reusable `Custom Tasks` across the community. 
+
+#### Flexibility
+
+The *experimental* and *stable* `Custom Tasks` provide a mechanism to provide functionality that's not directly
+available in the *Tekton Pipelines API*. It empowers users to extend *Tekton Pipelines* to solve their bespoke Continuous 
+Delivery use cases. 
+
+The `Custom Tasks` graduation path and process in this TEP gives us flexibility to safely iterate on functionality 
+provided through the `Custom Tasks` as we make progress through each stage.
+
+## Alternatives
+
+#### Experimental to Integrated
+
+We could remove the *stable* `Custom Tasks` and promote from *experimental* to *integrated* directly.
+
+###### Pros
+
+- In the best case, speeds up the graduation process without intermediary stage.
+- Simplifies the graduation path.
+
+###### Cons
+
+- In the worst case, slows down the graduation process because of higher bar of approval for graduation from
+  *experimental* to *integrated* directly.
+- Does not provide a way to discover `Custom Tasks` whose quality and reliability are validated by Tekton.
+- Reduces the time to iterate on the functionality provided by `Custom Tasks` before integrating it to *Tekton Pipelines*.
+
+#### Experimental and Stable in One Repository
+
+We could have the *experimental* and *stable* `Custom Tasks` in one repository, and indicate their stability levels
+by other means (such as documentation). 
+
+###### Pros
+
+- Consolidates `Custom Tasks` in one place, making them more easily discoverable.
+- Simplifies the change needed to migrate from *experimental* to *stable*.
+
+###### Cons
+
+- Reduces the separation between the *experimental* and *stable* `Custom Tasks`, taking more effort to distinguish them.
+- Makes it harder to enforce quality requirements for *stable* `Custom Tasks` (we're considering separating official 
+resources in the *Tekton Catalog* for this reason).
+
+
+## Infrastructure Needed
+
+We need a GitHub repository for *stable* `Custom Tasks`.
+
+## References
+
+- [TEP-0002: Enable Custom Tasks][TEP-0002]
+- [TEP-0056: Pipelines in Pipelines][TEP-0056]
+- [Pipelines in Pipelines Custom Tasks][pip-ct]
+- [Common Expression Language Custom Tasks][cel-ct]
+- [Wait Custom Tasks][wait-ct]
+- [Task Looping Custom Tasks][tl-ct]
+- [Kubeflow Pipelines on Tekton][kubeflow]
+- [Tekton Pipelines API Policy][api-policy]
+
+[experimental-repo]: https://github.com/tektoncd/experimental
+[community-repo]: https://github.com/tektoncd/community
+[api-wg]: https://github.com/tektoncd/community/blob/main/working-groups.md#api
+[TEP-0002]: 0002-custom-tasks.md
+[TEP-0056]: 0056-pipelines-in-pipelines.md
+[pip-ct]: https://github.com/tektoncd/experimental/tree/main/pipelines-in-pipelines
+[cel-ct]: https://github.com/tektoncd/experimental/tree/main/cel
+[wait-ct]: https://github.com/tektoncd/experimental/tree/main/wait-task
+[tl-ct]: https://github.com/tektoncd/experimental/tree/main/task-loops
+[kubeflow]: https://developer.ibm.com/blogs/kubeflow-pipelines-and-tekton-advances-data-workloads
+[proposing-projects]: https://github.com/tektoncd/community/blob/main/process.md#proposing-projects
+[runs]: https://github.com/tektoncd/pipeline/blob/main/docs/runs.md
+[api-policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#alpha-beta-and-ga
+[backwards]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#backwards-incompatible-changes

teps/README.md

@@ -223,3 +223,4 @@ This is the complete list of Tekton teps:
 |[TEP-0073](0073-simplify-metrics.md) | Simplify metrics | proposed | 2021-06-23 |
 |[TEP-0080](0080-support-domainscoped-parameterresult-names.md) | Support domain-scoped parameter/result names | implemented | 2021-08-19 |
 |[TEP-0084](0084-endtoend-provenance-collection.md) | end-to-end provenance collection | proposed | 2021-09-16 |
+|[TEP-0087](0087-custom-tasks-graduation.md) | Custom Tasks Graduation | proposed | 2021-09-30 |