TEP-0087: Custom Tasks Graduation

This change adds the Tekton Enhancement Proposal for `Custom Tasks` Graduation.

Today, `Custom Tasks` shared by the Tekton community are all *experimental* and we haven't
defined a process to promote them beyond *experimental*.

As such:
- users can't depend on the `Custom Tasks` because they not *stable* (they can change any time).
- contributors don't have a process to stabilize their `Custom Tasks` or integrate them natively
to the Tekton Pipelines API.

In this TEP, we aim to:
- define the graduation path for `Custom Tasks` from *experimental* to *stable* to *integrated*.
- provide infrastructure for *stable* `Custom Tasks` that are officially supported by Tekton.

`Custom Tasks` will have three levels in their graduation path:

1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we're iterating on it.
2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
3. *Integrated*: `Custom Tasks`' functionality are natively supported in Tekton Pipelines API.

See also [TEP-0002: Enable Custom Tasks](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md).

teps/0087-custom-tasks-graduation.md

@@ -0,0 +1,339 @@
+---
+status: proposed
+title: Custom Tasks Graduation
+creation-date: '2021-09-28'
+last-updated: '2021-09-28'
+authors:
+- '@jerop'
+see-also:
+- TEP-0002
+---
+
+# TEP-0087: Custom Tasks Graduation
+
+<!--
+**Note:** When your TEP is complete, all of these comment blocks should be removed.
+
+To get started with this template:
+
+- [ ] **Fill out this file as best you can.**
+  At minimum, you should fill in the "Summary", and "Motivation" sections.
+  These should be easy if you've preflighted the idea of the TEP with the
+  appropriate Working Group.
+- [ ] **Create a PR for this TEP.**
+  Assign it to people in the SIG that are sponsoring this process.
+- [ ] **Merge early and iterate.**
+  Avoid getting hung up on specific details and instead aim to get the goals of
+  the TEP clarified and merged quickly.  The best way to do this is to just
+  start with the high-level sections and fill out details incrementally in
+  subsequent PRs.
+
+Just because a TEP is merged does not mean it is complete or approved.  Any TEP
+marked as a `proposed` is a working document and subject to change.  You can
+denote sections that are under active debate as follows:
+
+```
+<<[UNRESOLVED optional short context or usernames ]>>
+Stuff that is being argued.
+<<[/UNRESOLVED]>>
+```
+
+When editing TEPS, aim for tightly-scoped, single-topic PRs to keep discussions
+focused.  If you disagree with what is already in a document, open a new PR
+with suggested changes.
+
+If there are new details that belong in the TEP, edit the TEP.  Once a
+feature has become "implemented", major changes should get new TEPs.
+
+The canonical place for the latest set of instructions (and the likely source
+of this file) is [here](/teps/NNNN-TEP-template/README.md).
+
+-->
+
+<!--
+This is the title of your TEP.  Keep it short, simple, and descriptive.  A good
+title can help communicate what the TEP is and should be considered as part of
+any review.
+-->
+
+<!--
+A table of contents is helpful for quickly jumping to sections of a TEP and for
+highlighting any additional information provided beyond the standard TEP
+template.
+
+Ensure the TOC is wrapped with
+  <code>&lt;!-- toc --&rt;&lt;!-- /toc --&rt;</code>
+tags, and then generate with `hack/update-toc.sh`.
+-->
+
+<!-- toc -->
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+  - [Use Cases](#use-cases)
+- [Requirements](#requirements)
+- [Proposal](#proposal)
+  - [Experimental](#experimental)
+        - [Admission Requirements](#admission-requirements)
+        - [Admission Process](#admission-process)
+  - [Stable](#stable)
+        - [Graduation Requirements](#graduation-requirements)
+        - [Graduation Process](#graduation-process)
+  - [Integrated](#integrated)
+        - [Graduation Requirements](#graduation-requirements-1)
+        - [Graduation Process](#graduation-process-1)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+<!--
+This section is incredibly important for producing high quality user-focused
+documentation such as release notes or a development roadmap.  It should be
+possible to collect this information before implementation begins in order to
+avoid requiring implementors to split their attention between writing release
+notes and implementing the feature itself.
+
+A good summary is probably at least a paragraph in length.
+
+Both in this section and below, follow the guidelines of the [documentation
+style guide]. In particular, wrap lines to a reasonable length, to make it
+easier for reviewers to cite specific portions, and to minimize diff churn on
+updates.
+
+[documentation style guide]: https://github.com/kubernetes/community/blob/master/contributors/guide/style-guide.md
+-->
+
+Today, `Custom Tasks` shared by the Tekton community are all *experimental* and we haven't 
+defined a process to promote them beyond *experimental*. 
+
+As such:
+- users can't depend on the `Custom Tasks` because they not *stable* (they can change any time)
+- contributors don't have a process to stabilize their `Custom Tasks` or integrate them natively 
+to the Tekton Pipelines API
+
+In this TEP, we aim to:
+- define the graduation path for `Custom Tasks` from *experimental* to *stable* to *integrated*
+- provide infrastructure for *stable* `Custom Tasks` that are officially supported by Tekton
+
+## Motivation
+
+<!--
+This section is for explicitly listing the motivation, goals and non-goals of
+this TEP.  Describe why the change is important and the benefits to users.  The
+motivation section can optionally provide links to [experience reports][] to
+demonstrate the interest in a TEP within the wider Tekton community.
+
+[experience reports]: https://github.com/golang/go/wiki/ExperienceReports
+-->
+
+As described in [TEP-0002: Enable Custom Tasks][TEP-0002], we introduced `Custom Tasks` that are defined as CRDs that 
+are executed using [`Runs`][runs]. These `Custom Tasks` have reconciling controllers that watch for `Runs` referencing 
+their types and updates their status. `Custom Tasks` enable extensibility in Tekton - users can implement functionality 
+that's not supported natively in Tekton Pipelines. 
+
+Hitherto, we have implemented several `Custom Tasks` in [Tekton Experimental Repository][experimental-repo]:
+- [Common Expression Language Custom Tasks][cel-ct] - provides support for Common Expression Language in Tekton Pipelines
+- [Wait Custom Tasks][wait-ct] - enables waiting for some amount of time between `Tasks`
+- [Task Looping Custom Tasks][tl-ct] - enables running a `Task` in a loop with varying `Parameter` values
+- [Pipelines in Pipelines Custom Tasks][pip-ct] - provide support for composing and executing `Pipelines` in `Pipelines`
+
+These `Custom Tasks` implementations are all *experimental*. Notwithstanding, they are in use in critical projects, such 
+as [Kubeflow Pipelines on Tekton][kubeflow]. Providing stability levels and progression for `Custom Tasks` will enable 
+users to rely on them and empower contributors to safely update them. 
+
+We may decide to promote some `Custom Tasks` to top level such that their functionality is supported natively in the 
+Tekton Pipelines API. On the other hand, we may decide not to natively support some functionalities provided in some 
+`Custom Tasks` but we need to provide stability guarantees. Thus, we need to provide an incremental graduation path and 
+stability levels for `Custom Tasks`. 
+
+### Goals
+
+<!--
+List the specific goals of the TEP.  What is it trying to achieve?  How will we
+know that this has succeeded?
+-->
+
+1. Provide a graduation path for `Custom Tasks` from *experimental* to *stable* and *integrated*
+2. Provide infrastructure for *stable* `Custom Tasks` that are officially supported by Tekton
+
+### Non-Goals
+
+<!--
+What is out of scope for this TEP?  Listing non-goals helps to focus discussion
+and make progress.
+-->
+
+1. Promote `Custom Tasks` feature itself from `alpha` to `beta` - we will address this separately soon
+2. Provide CLI support for `Custom Tasks` provided as *stable* integrations - we can explore this later
+
+### Use Cases
+
+<!--
+Describe the concrete improvement specific groups of users will see if the
+Motivations in this doc result in a fix or feature.
+
+Consider both the user's role (are they a Task author? Catalog Task user?
+Cluster Admin? etc...) and experience (what workflows or actions are enhanced
+if this problem is solved?).
+-->
+
+1. As a contributor, I want to promote my `Custom Tasks` from *experimental* to *stable* and, possibly, *integrated* so 
+I need a process that I can follow with clear requirements for graduation to the next level
+2. As a user, I need to use *stable* `Custom Tasks` that I can rely on for critical projects
+
+## Requirements
+
+<!--
+Describe constraints on the solution that must be met. Examples might include
+performance characteristics that must be met, specific edge cases that must
+be handled, or user scenarios that will be affected and must be accommodated.
+-->
+
+- Define graduation requirements and path for `Custom Tasks` from *experimental* to *stable* and *integrated*.
+- Provide a catalog of `Custom Tasks` that are *stable* integrations with necessary infrastructure and processes, such 
+as thorough testing and frequent releases.
+
+
+## Proposal
+
+<!--
+This is where we get down to the specifics of what the proposal actually is.
+This should have enough detail that reviewers can understand exactly what
+you're proposing, but should not include things like API designs or
+implementation.  The "Design Details" section below is for the real
+nitty-gritty.
+-->
+
+`Custom Tasks` will have three levels in their graduation path:
+
+1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we're iterating on it.
+2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
+3. *Integrated*: `Custom Tasks`' functionality are natively supported in Tekton Pipelines API.
+
+### Experimental
+
+`Custom Tasks` will start as *experimental* to keep the barrier of sharing integrations low. 
+
+The [*alpha* API policy][api-policy] applies to the *experimental* `Custom Tasks`, meaning their Owners can make any
+changes at any time as they iterate on the `Custom Tasks`.
+
+###### Admission Requirements
+
+We can consider admitting an *experimental* `Custom Task` *stable* if:
+
+1. A `Custom Task` is needed to provide a specific functionality to solve common Continuous Delivery use cases
+2. At least two contributors are interested in owning a `Custom Task` that provides that functionality
+
+###### Admission Process
+
+As described in [Tekton's community process][proposing-projects] for proposing new projects, the `Custom Tasks` Owners 
+would:
+
+1. Contributors propose the experimental `Custom Task` in the [Tekton API working group meeting][api-wg]
+2. Contributors file an issue in the [Tekton Community repository][community-repo] describing the problem the 
+`Custom Task` would solve and who will own it
+3. When at least two Tekton Pipelines Owners approve the issue, the contributors can add the `Custom Task` to the 
+[Tekton Experimental repository][experimental-repo] (N/B: other experimental projects need Governing Board approval)
+
+### Stable
+
+Tekton will add a new GitHub repository - *custom-tasks* - that would contain high quality `Custom Tasks`. These are 
+extensions that users can rely on to use functionality that's not provided in the Tekton Pipelines API directly. The 
+Tekton Pipelines Owners will be the Owners of this repository, and each `Custom Task` will have its own Owners. 
+
+The [*beta* API policy][api-policy] applies to the *stable* `Custom Tasks`, meaning that:
+- Any [backwards incompatible][backwards] changes must be introduced in a backwards compatible manner first, with a 
+deprecation warning in the release notes and migration instructions
+- Users will be given at least 9 months to migrate before a backward incompatible change is made
+- Backwards incompatible changes have to be approved by more than half of the `Custom Task`'s Owners
+
+The *custom-tasks* repository will provide the testing infrastructure needed by stable *Custom Tasks*. We will make 
+monthly releases of `Custom Tasks` that have had changes since their latest release, which we can automate. Moreover, 
+we will have nightly releases of each `Custom Task` to catch any failures. Each *stable* `Custom Task` will be used in 
+dogfooding, and will have detailed documentation and multiple examples.
+
+###### Graduation Requirements
+
+We can consider graduating a given *experimental* `Custom Task` to *stable* if it meets the following requirements:
+
+1. It is thoroughly tested
+2. It has nightly releases
+3. It is used in dogfooding
+4. It provides several examples or samples 
+5. It has detailed documentation for installation and usage
+6. It has received feedback from the community (solicited in working group, mailing list or other channels)
+
+###### Graduation Process
+
+The graduation process from *experimental* to *stable* for a given `Custom Task` would be:
+
+1. Its Owners validate that the *experimental* `Custom Task` meets the above requirements
+2. Its Owners open a Tekton Enhancement Proposal (TEP) for the functionality provided by the `Custom Task` with its 
+motivation (goals, use cases and requirements), proposal (with design details and evaluation), and alternatives
+3. Its Owners nominate the `Custom Task` for graduation from *experimental* to *stable*
+4. When the Tekton Pipelines Owners have approved the TEP as implementable, the `Custom Task`'s Owners can migrate it to
+the *custom-tasks* repository
+
+### Integrated
+
+When a given *stable* `Custom Task` is actively used and needed for common Continuous Delivery use cases, we can 
+consider making it *integrated* - meaning that its functionality is natively supported in the Tekton Pipelines API. 
+Not all *stable* `Custom Tasks` have to be *integrated*, a *stable* `Custom Task` that improves user experience but
+is not really necessary in Tekton Pipelines API can stay as a *stable* `Custom Task` in the long term. 
+
+###### Graduation Requirements
+
+We can consider graduating a given *stable* `Custom Task` to *integrated* if it meets the following requirements:
+
+1. Its functionality is critical to common Continuous Delivery use cases
+2. It's actively used by the community and in dogfooding
+
+###### Graduation Process
+
+The graduation process from *stable* to *integrated* for a given `Custom Task` would be:
+
+1. Its Owners validate that the *stable* `Custom Task` meets the above requirements
+2. Its Owners update the `Custom Task`'s TEP with:
+   1. a nomination for graduation from *stable* to *integrated*
+   2. a discussion of how its been useful and why it's necessary to natively support the functionality in the 
+   Tekton Pipelines API
+   3. proposal and alternatives for its syntax in the Tekton Pipelines API
+3. When the Tekton Pipelines Owners have approved the TEP as implementable, the `Custom Task`'s Owners will add its 
+functionality directly to the Tekton Pipelines API as an *alpha* feature and deprecate the *stable* `Custom Task`
+4. When the *alpha* feature is promoted to *beta*, the `Custom Task`'s Owners can remove it from the *custom-tasks* repo
+
+## References
+
+<!--
+Use this section to add links to GitHub issues, other TEPs, design docs in Tekton
+shared drive, examples, etc. This is useful to refer back to any other related links
+to get more details.
+-->
+
+- [TEP-0002: Enable Custom Tasks][TEP-0002]
+- [TEP-0056: Pipelines in Pipelines][TEP-0056]
+- [Pipelines in Pipelines Custom Tasks][pip-ct]
+- [Common Expression Language Custom Tasks][cel-ct]
+- [Wait Custom Tasks][wait-ct]
+- [Task Looping Custom Tasks][tl-ct]
+- [Kubeflow Pipelines on Tekton][kubeflow]
+- [Tekton Pipelines API Policy][api-policy]
+
+
+
+[experimental-repo]: https://github.com/tektoncd/experimental
+[community-repo]: https://github.com/tektoncd/community
+[api-wg]: https://github.com/tektoncd/community/blob/main/working-groups.md#api
+[TEP-0002]: 0002-custom-tasks.md
+[TEP-0056]: 0056-pipelines-in-pipelines.md
+[pip-ct]: https://github.com/tektoncd/experimental/tree/main/pipelines-in-pipelines
+[cel-ct]: https://github.com/tektoncd/experimental/tree/main/cel
+[wait-ct]: https://github.com/tektoncd/experimental/tree/main/wait-task
+[tl-ct]: https://github.com/tektoncd/experimental/tree/main/task-loops
+[kubeflow]: https://developer.ibm.com/blogs/kubeflow-pipelines-and-tekton-advances-data-workloads
+[proposing-projects]: https://github.com/tektoncd/community/blob/main/process.md#proposing-projects
+[runs]: https://github.com/tektoncd/pipeline/blob/main/docs/runs.md
+[api-policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#alpha-beta-and-ga
+[backwards]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#backwards-incompatible-changes

teps/README.md

@@ -223,3 +223,4 @@ This is the complete list of Tekton teps:
 |[TEP-0073](0073-simplify-metrics.md) | Simplify metrics | proposed | 2021-06-23 |
 |[TEP-0080](0080-support-domainscoped-parameterresult-names.md) | Support domain-scoped parameter/result names | implemented | 2021-08-19 |
 |[TEP-0084](0084-endtoend-provenance-collection.md) | end-to-end provenance collection | proposed | 2021-09-16 |
+|[TEP-0087](0087-custom-tasks-graduation.md) | Custom Tasks Graduation | proposed | 2021-09-28 |