Add matrix support for using references to entire PipelineRun array parameters

[EmmaMunley]

Changes

This PR adds support for using whole array PipelineRun parameters as matrix parameter values by moving the validation for checking matrix parameter types to after parameter substitution. This is necessary for allowing whole array substitutions using the syntax $(params.foo[*]), rather than an explicit YAML array, to be used as a matrix parameter. Prior to this, validation fails because a string referencing an array type was considered a string type and this was done before the applying array replacements.

Note: Whole array replacements for matrix results has not yet been implemented and will be added in a subsequent PR.

This issue is described in more detail here: #6056 (comment)

Thanks to @abayer for investigating and co-authoring this PR: #6056 😄

/kind feature

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

• Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
• Has Tests included if any functionality added or changed
• Follows the commit message standard
• Meets the Tekton contributor standards (including functionality, content, code)
• Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
• Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

NONE

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/param_types.go	99.1%	98.7%	-0.4
pkg/apis/pipeline/v1beta1/param_types.go	99.1%	98.6%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/param_types.go	99.1%	98.7%	-0.4
pkg/apis/pipeline/v1beta1/param_types.go	99.1%	98.6%	-0.4

[EmmaMunley]

/assign @jerop @lbernick

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.4%	89.0%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.4%	89.0%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.4%	89.0%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.4%	89.0%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.4%	89.0%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.4%	89.0%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.3%	88.9%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.3%	88.9%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.3%	88.9%	-0.4

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	89.3%	88.9%	-0.4

[EmmaMunley]

/test pull-tekton-pipeline-alpha-integration-tests

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[Yongxuanzhang]

I wonder if we could use for loop to create the expectedTaskRuns to reduce the size of the test code? The difference between each case seem to be the params, we could use %s in the mustParseTaskRunWithObjectMeta to pass the params?
e.g.

pipeline/pkg/reconciler/pipelinerun/pipelinerun_test.go

Line 3487 in 50fa5f1

expectedTaskRun := mustParseTaskRunWithObjectMeta(t,

[Yongxuanzhang]

We could ignore this comment in this PR, I just feel like it would be helpful if we could clean up it.
The suggestion in my mind is something like:

 data := []struct {
        name     string
        browser  string
        platform string
    }{{
        name:     "pr-platforms-and-browsers-0",
        browser:  "chrome",
        platform: "linux",
    }, {
        name:     "pr-platforms-and-browsers-1",
        browser:  "chrome",
        platform: "mac",
    }}
    expectedTaskRuns := []*v1beta1.TaskRun{}
    for _, d := range data {
        expectedTaskRuns1 = append(expectedTaskRuns,

            mustParseTaskRunWithObjectMeta(t,
                taskRunObjectMeta(d.name, "foo",
                    "pr", "p-dag", "platforms-and-browsers", false),
                fmt.Sprintf(`
            spec:
            params:
            - name: browser
                value: %s
            - name: platform
                value: %s
            serviceAccountName: test-sa
            taskRef:
                name: mytask
                kind: Task
            labels:
                tekton.dev/memberOf: tasks
                tekton.dev/pipeline: p-dag
            `, d.browser, d.platform)),
        )
    }

[EmmaMunley]

Thanks for the suggestion @Yongxuanzhang. I updated this here and think in the future we could refactor all of the tests to use this and perhaps make a createExpectedTaskRuns() function that can abstract some of this even more from a mapping of parameter names and values.

I also think there's opportunity to create more variables (pr name, namespace, pipelineTaskName, etc that can be passed between the specs to avoid some of the manual errors and pain points around formatting the yaml syntax.

cc: @jerop @lbernick

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[Yongxuanzhang]

/lgtm

[EmmaMunley]

/test pull-tekton-pipeline-alpha-integration-tests

[lbernick]

This is a known flake: #4567

[EmmaMunley]

/test pull-tekton-pipeline-go-coverage-df

[tekton-robot]

@EmmaMunley: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

• /test pull-tekton-pipeline-alpha-integration-tests
• /test pull-tekton-pipeline-beta-integration-tests
• /test pull-tekton-pipeline-build-tests
• /test pull-tekton-pipeline-integration-tests
• /test tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

• /test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test pull-tekton-pipeline-go-coverage-df

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[EmmaMunley]

/test pull-tekton-pipeline-go-coverage

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.1%	89.2%	0.1