Skip to content

Commit

Permalink
TEP-0075(object params and results) promoted to beta
Browse files Browse the repository at this point in the history 
 Pipeline 0.38 introduced object params and results. This commit is
 promoting TEP-0075 to beta such that these features can be used by
 the task authors and pipeline authors in a cluster when enable-api-fields
 is either set to beta or alpha.

 Signed-off-by: Yongxuan Zhang [email protected]
  • Loading branch information
@Yongxuanzhang @tekton-robot
Yongxuanzhang authored and tekton-robot committed Mar 16, 2023
1 parent d89850f commit 03512d8
Show file tree
Hide file tree
Showing 17 changed files with 44 additions and 42 deletions.
12 changes: 5 additions & 7 deletions docs/pipelines.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -923,17 +923,15 @@ Sharing `Results` between `Tasks` in a `Pipeline` happens via
a `Result` and another receives it as a `Parameter` with a variable such as
`$(tasks.<task-name>.results.<result-name>)`. Pipeline support two new types of
results and parameters: array `[]string` and object `map[string]string`.
Array result is a beta feature and can be enabled by setting `enable-api-fields` to `beta`
and is also supported with `enable-api-fields` set to `alpha`.
Object result is an alpha feature and can be enabled by setting `enable-api-fields` to `alpha`.
Array and Object result is a beta feature and can be enabled by setting `enable-api-fields` to `alpha` or `beta`.

| Result Type | Parameter Type | Specification | `enable-api-fields` |
|-------------|----------------|--------------------------------------------------|---------------------|
| string | string | `$(tasks.<task-name>.results.<result-name>)` | stable |
| array | array | `$(tasks.<task-name>.results.<result-name>[*])` | alpha or beta |
| array | string | `$(tasks.<task-name>.results.<result-name>[i])` | alpha or beta |
| object | object | `$(tasks.<task-name>.results.<result-name>[*])` | alpha |
| object | string | `$(tasks.<task-name>.results.<result-name>.key)` | alpha |
| object | object | `$(tasks.<task-name>.results.<result-name>[*])` | alpha or beta |
| object | string | `$(tasks.<task-name>.results.<result-name>.key)` | alpha or beta |

**Note:** Whole Array and Object `Results` (using star notation) cannot be referred in `script`.

Expand Down Expand Up @@ -1022,8 +1020,8 @@ results:

For an end-to-end example, see [`Results` in a `PipelineRun`](../examples/v1beta1/pipelineruns/pipelinerun-results.yaml).

Object result is supported as alpha feature and array result is a beta feature,
see [`Array and Object Results` in a `PipelineRun`](../examples/v1beta1/pipelineruns/alpha/pipeline-emitting-results.yaml).
Object result and array result are beta features,
see [`Array and Object Results` in a `PipelineRun`](../examples/v1beta1/pipelineruns/beta/pipeline-emitting-results.yaml).

```yaml
results:
Expand Down
18 changes: 9 additions & 9 deletions docs/tasks.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -528,7 +528,7 @@ For example, `foo.Is-Bar_` is a valid parameter name for string or array type, b
> 2. If a parameter name contains dots (.), it must be referenced by using the [bracket notation](#substituting-parameters-and-resources) with either single or double quotes i.e. `$(params['foo.bar'])`, `$(params["foo.bar"])`. See the following example for more information.

#### Parameter type
Each declared parameter has a `type` field, which can be set to `string`, `array` or `object` (alpha feature).
Each declared parameter has a `type` field, which can be set to `string`, `array` (beta feature) or `object` (beta feature).

##### `object` type

Expand All @@ -548,11 +548,11 @@ spec:

Refer to the [TaskRun example](../examples/v1beta1/taskruns/alpha/object-param-result.yaml) and the [PipelineRun example](../examples/v1beta1/pipelineruns/alpha/pipeline-object-param-and-result.yaml) in which `object` parameters are demonstrated.

> NOTE:
> - `object` param is an `alpha` feature and gated by the `alpha` feature flag.
> NOTE:
> - `object` param is a `beta` feature and gated by the `alpha` or `beta` feature flag.
> - `object` param must specify the `properties` section to define the schema i.e. what keys are available for this object param. See how to define `properties` section in the following example and the [TEP-0075](https://github.com/tektoncd/community/blob/main/teps/0075-object-param-and-result-types.md#defaulting-to-string-types-for-values).
> - When providing value for an `object` param, one may provide values for just a subset of keys in spec's `default`, and provide values for the rest of keys at runtime ([example](../examples/v1beta1/taskruns/alpha/object-param-result.yaml)).
> - When using object in variable replacement, users can only access its individual key ("child" member) of the object by its name i.e. `$(params.gitrepo.url)`. Using an entire object as a value is only allowed when the value is also an object like [this example](https://github.com/tektoncd/pipeline/blob/55665765e4de35b3a4fb541549ae8cdef0996641/examples/v1beta1/pipelineruns/alpha/pipeline-object-param-and-result.yaml#L64-L65). See more details about using object param from the [TEP-0075](https://github.com/tektoncd/community/blob/main/teps/0075-object-param-and-result-types.md#using-objects-in-variable-replacement).
> - When providing value for an `object` param, one may provide values for just a subset of keys in spec's `default`, and provide values for the rest of keys at runtime ([example](../examples/v1beta1/taskruns/beta/object-param-result.yaml)).
> - When using object in variable replacement, users can only access its individual key ("child" member) of the object by its name i.e. `$(params.gitrepo.url)`. Using an entire object as a value is only allowed when the value is also an object like [this example](https://github.com/tektoncd/pipeline/blob/55665765e4de35b3a4fb541549ae8cdef0996641/examples/v1beta1/pipelineruns/beta/pipeline-object-param-and-result.yaml#L64-L65). See more details about using object param from the [TEP-0075](https://github.com/tektoncd/community/blob/main/teps/0075-object-param-and-result-types.md#using-objects-in-variable-replacement).

##### `array` type

Expand Down Expand Up @@ -928,7 +928,7 @@ At present, the limit is ["4096 bytes"](https://github.com/kubernetes/kubernetes
This also means that the number of Steps in a Task affects the maximum size of a Result,
as each Step is implemented as a container in the TaskRun's pod.
The more containers we have in our pod, *the smaller the allowed size of each container's
message*, meaning that the **more steps you have in a Task, the smaller the result for each step can be**.
message*, meaning that the **more steps you have in a Task, the smaller the result for each step can be**.
For example, if you have 10 steps, the size of each step's Result will have a maximum of less than 1KB.

If your `Task` writes a large number of small results, you can work around this limitation
Expand All @@ -945,7 +945,7 @@ As a general rule-of-thumb, if a result needs to be larger than a kilobyte, you

This is an experimental feature. The `results-from` feature flag must be set to `"sidecar-logs"`](./install.md#enabling-larger-results-using-sidecar-logs).

Instead of using termination messages to store results, the taskrun controller injects a sidecar container which monitors the results of all the steps. The sidecar mounts the volume where results of all the steps are stored. As soon as it finds a new result, it logs it to std out. The controller has access to the logs of the sidecar container (Caution: we need you to enable access to [kubernetes pod/logs](./install.md#enabling-larger-results-using-sidecar-logs).
Instead of using termination messages to store results, the taskrun controller injects a sidecar container which monitors the results of all the steps. The sidecar mounts the volume where results of all the steps are stored. As soon as it finds a new result, it logs it to std out. The controller has access to the logs of the sidecar container (Caution: we need you to enable access to [kubernetes pod/logs](./install.md#enabling-larger-results-using-sidecar-logs).

This feature allows users to store up to 4 KB per result by default. Because we are not limited by the size of the termination messages, users can have as many results as they require (or until the CRD reaches its limit). If the size of a result exceeds this limit, then the TaskRun will be placed into a failed state with the following message: `Result exceeded the maximum allowed limit.`

Expand Down Expand Up @@ -1008,7 +1008,7 @@ steps:
---
# The secret 'test' part data is as follows.
data:
# The decoded value of 'cHJpdmF0ZQo=' is 'private'.
# The decoded value of 'cHJpdmF0ZQo=' is 'private'.
token: "cHJpdmF0ZQo="
```

Expand Down Expand Up @@ -1247,7 +1247,7 @@ Study the following code examples to better understand how to configure your `Ta
- [Using a `Secret` as an environment source](#using-a-secret-as-an-environment-source)
- [Using a `Sidecar` in a `Task`](#using-a-sidecar-in-a-task)

_Tip: See the collection of Tasks in the
_Tip: See the collection of Tasks in the
[Tekton community catalog](https://github.com/tektoncd/catalog) for
more examples.

Expand Down
0 ...pha/pipeline-object-param-and-result.yaml → ...eta/pipeline-object-param-and-result.yaml
View file
File renamed without changes.
0 ...neruns/alpha/pipeline-object-results.yaml → ...ineruns/beta/pipeline-object-results.yaml
View file
File renamed without changes.
0 ...1/taskruns/alpha/object-param-result.yaml → ...a1/taskruns/beta/object-param-result.yaml
View file
File renamed without changes.
8 changes: 5 additions & 3 deletions pkg/apis/pipeline/v1/pipelineref_validation_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ func TestPipelineRef_Invalid(t *testing.T) {
wantErr: apis.ErrMultipleOneOf("name", "params").Also(apis.ErrMissingField("resolver")),
withContext: config.EnableBetaAPIFields,
}, {
name: "pipelineref param object not allowed",
name: "pipelineref param object not allowed in stable",
ref: &v1.PipelineRef{
ResolverRef: v1.ResolverRef{
Resolver: "some-resolver",
Expand All @@ -102,8 +102,10 @@ func TestPipelineRef_Invalid(t *testing.T) {
}},
},
},
wantErr: apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" but it is \"beta\""),
withContext: config.EnableBetaAPIFields,
wantErr: apis.ErrGeneric("resolver requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"").Also(
apis.ErrGeneric("resolver params requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"")).Also(
apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"")),
withContext: config.EnableStableAPIFields,
}}

for _, tc := range tests {
Expand Down
4 changes: 2 additions & 2 deletions pkg/apis/pipeline/v1/result_validation.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,10 @@ func (tr TaskResult) Validate(ctx context.Context) (errs *apis.FieldError) {
}

switch {
// Object are alpha features
// Object results is beta feature - check if the feature flag is set to "beta" or "alpha"
case tr.Type == ResultsTypeObject:
errs := validateObjectResult(tr)
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "results type", config.AlphaAPIFields))
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "results type", config.BetaAPIFields))
return errs
// Array results is a beta feature - check if the feature flag is set to "beta" or "alpha"
case tr.Type == ResultsTypeArray:
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/pipeline/v1/result_validation_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ func TestResultsValidateError(t *testing.T) {
},
apiFields: "stable",
expectedError: apis.FieldError{
Message: "results type requires \"enable-api-fields\" feature gate to be \"alpha\" but it is \"stable\"",
Message: "results type requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"",
},
}, {
name: "invalid object properties type",
Expand Down
6 changes: 3 additions & 3 deletions pkg/apis/pipeline/v1/task_validation.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -295,9 +295,9 @@ func validateStep(ctx context.Context, s Step, names sets.String) (errs *apis.Fi
func ValidateParameterTypes(ctx context.Context, params []ParamSpec) (errs *apis.FieldError) {
for _, p := range params {
if p.Type == ParamTypeObject {
// Object type parameter is an alpha feature and will fail validation if it's used in a task spec
// when the enable-api-fields feature gate is not "alpha".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.AlphaAPIFields))
// Object type parameter is a beta feature and will fail validation if it's used in a task spec
// when the enable-api-fields feature gate is not "alpha" or "beta".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.BetaAPIFields))
}
errs = errs.Also(p.ValidateType(ctx))
}
Expand Down
8 changes: 5 additions & 3 deletions pkg/apis/pipeline/v1/taskref_validation_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ func TestTaskRef_Invalid(t *testing.T) {
wantErr: apis.ErrMultipleOneOf("name", "params").Also(apis.ErrMissingField("resolver")),
wc: config.EnableBetaAPIFields,
}, {
name: "taskref param object requires alpha",
name: "taskref param object requires beta",
taskRef: &v1.TaskRef{
ResolverRef: v1.ResolverRef{
Resolver: "some-resolver",
Expand All @@ -150,8 +150,10 @@ func TestTaskRef_Invalid(t *testing.T) {
}},
},
},
wc: config.EnableBetaAPIFields,
wantErr: apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" but it is \"beta\""),
wc: config.EnableStableAPIFields,
wantErr: apis.ErrGeneric("resolver requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"").Also(
apis.ErrGeneric("resolver params requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"")).Also(
apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"")),
}}
for _, ts := range tests {
t.Run(ts.name, func(t *testing.T) {
Expand Down
6 changes: 3 additions & 3 deletions pkg/apis/pipeline/v1/taskrun_validation.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -242,9 +242,9 @@ func ValidateParameters(ctx context.Context, params []Param) (errs *apis.FieldEr
var names []string
for _, p := range params {
if p.Value.Type == ParamTypeObject {
// Object type parameter is an alpha feature and will fail validation if it's used in a taskrun spec
// when the enable-api-fields feature gate is not "alpha".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.AlphaAPIFields))
// Object type parameter is a beta feature and will fail validation if it's used in a taskrun spec
// when the enable-api-fields feature gate is not "alpha" or "beta".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.BetaAPIFields))
}
names = append(names, p.Name)
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/pipeline/v1beta1/pipelineref_validation_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ func TestPipelineRef_Invalid(t *testing.T) {
}},
},
},
wantErr: apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" but it is \"stable\""),
wantErr: apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\""),
}}

for _, tc := range tests {
Expand Down
4 changes: 2 additions & 2 deletions pkg/apis/pipeline/v1beta1/result_validation.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ func (tr TaskResult) Validate(ctx context.Context) (errs *apis.FieldError) {
}

switch {
// Object are alpha features
// Object results is a beta feature - make sure the feature flag is set to "beta"
case tr.Type == ResultsTypeObject:
errs := validateObjectResult(tr)
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "results type", config.AlphaAPIFields))
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "results type", config.BetaAPIFields))
return errs
// Array results is a beta feature - make sure the feature flag is set to "beta"
case tr.Type == ResultsTypeArray:
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/pipeline/v1beta1/result_validation_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ func TestResultsValidateError(t *testing.T) {
},
apiFields: "stable",
expectedError: apis.FieldError{
Message: "results type requires \"enable-api-fields\" feature gate to be \"alpha\" but it is \"stable\"",
Message: "results type requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\"",
},
}, {
name: "invalid object properties type",
Expand Down
6 changes: 3 additions & 3 deletions pkg/apis/pipeline/v1beta1/task_validation.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -295,9 +295,9 @@ func validateStep(ctx context.Context, s Step, names sets.String) (errs *apis.Fi
func ValidateParameterTypes(ctx context.Context, params []ParamSpec) (errs *apis.FieldError) {
for _, p := range params {
if p.Type == ParamTypeObject {
// Object type parameter is an alpha feature and will fail validation if it's used in a task spec
// when the enable-api-fields feature gate is not "alpha".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.AlphaAPIFields))
// Object type parameter is a beta feature and will fail validation if it's used in a task spec
// when the enable-api-fields feature gate is not "alpha" or "beta".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.BetaAPIFields))
}
errs = errs.Also(p.ValidateType(ctx))
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/pipeline/v1beta1/taskref_validation_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ func TestTaskRef_Invalid(t *testing.T) {
}},
},
},
wantErr: apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" but it is \"stable\""),
wantErr: apis.ErrGeneric("object type parameter requires \"enable-api-fields\" feature gate to be \"alpha\" or \"beta\" but it is \"stable\""),
}}
for _, ts := range tests {
t.Run(ts.name, func(t *testing.T) {
Expand Down
6 changes: 3 additions & 3 deletions pkg/apis/pipeline/v1beta1/taskrun_validation.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -241,9 +241,9 @@ func ValidateParameters(ctx context.Context, params []Param) (errs *apis.FieldEr
var names []string
for _, p := range params {
if p.Value.Type == ParamTypeObject {
// Object type parameter is an alpha feature and will fail validation if it's used in a taskrun spec
// when the enable-api-fields feature gate is not "alpha".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.AlphaAPIFields))
// Object type parameter is a beta feature and will fail validation if it's used in a taskrun spec
// when the enable-api-fields feature gate is not "alpha" or "beta".
errs = errs.Also(version.ValidateEnabledAPIFields(ctx, "object type parameter", config.BetaAPIFields))
}
names = append(names, p.Name)
}
Expand Down

0 comments on commit 03512d8

Please sign in to comment.