Design and implement Pipeline Results for pipeline with finally tasks

[pritidesai]

A new field finally is being introduced to PipelineSpec at the same level as Tasks, Params, and Workspaces. The finally field is a list of one or more PipelineTasks similar to Tasks that provide a guarantee that they are executed only once all PipelineTasks under tasks have completed regardless if that is because of success or error.

This new type is getting added with PR #2650 and functionality with PR #2661

While designing and implementing finally, task results were deferred to next iteration. Issue #2557 is tracking task results and in this issue, we will keep track of adding pipeline results to the pipeline with finally.

[pritidesai]

/kind feature

[bobcatfish]

@pritidesai has created a WIP design at Task Results with finally

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vdemeester]

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

[tekton-robot]

@vdemeester: Reopened this issue.

In response to this:

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[vdemeester]

/remove-lifecycle stale
/cc @pritidesai @jerop

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[pritidesai]

Reopening as this feature is not implemented yet. And will be a good feature to have finally task produce some kind of result and have it as part of the pipeline.

[pritidesai]

/remove-lifecycle stale

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[jerop]

/lifecycle frozen

may be useful even for pipelines in pipelines for the results from finally tasks to be passed to pipeline results

[jerop]

Is the idea here that a Pipeline's Result can reference a Finally Task's Result using a variable $(finally.<pipeline-task-name>.results.<result-name>)?

Was curious to see what happens if a Pipeline's Result references a Finally Task's Result using a variable $(tasks.<pipeline-task-name>.results.<result-name>) - and found that it worked, is that expected behavior? If so, can we document this and close this issue? If not, can we fix it and provide the finally results in pipeline level, or do users depend on the existing behavior?

Example

Modified the Pipeline in the provided example as such:

# Pipeline to clone repo into shared workspace and cleanup the workspace after done
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: clone-cleanup-workspace
spec:
  results:
    - name: initialized
      value: $(tasks.check-git-commit.results.init)
  workspaces:
    # common workspace where git repo is cloned and needs to be cleanup after done
    - name: git-source
  tasks:
    # Clone app repo to workspace
    - name: clone-app-repo
      taskRef:
        name: git-clone-from-catalog
      params:
        - name: url
          value: https://github.com/tektoncd/community.git
        - name: subdirectory
          value: application
      workspaces:
        - name: output
          workspace: git-source
  finally:
    # Cleanup workspace
    - name: cleanup
      taskRef:
        name: cleanup-workspace
      workspaces:
        - name: source
          workspace: git-source
    - name: check-git-commit
      params:
        - name: commit
          value: $(tasks.clone-app-repo.results.commit)
      taskSpec:
        params:
          - name: commit
        results:
          - name: init
        steps:
          - name: check-commit-initialized
            image: alpine
            script: |
              if [[ ! $(params.commit) ]]; then
                exit 1
              fi
              printf true | tee /tekton/results/init

And this was the created PipelineRun, with the Finally Task's Result passed on to the Pipeline:

$ tkn pr describe -L
Name:              write-and-cleanup-workspace
Namespace:         default
Pipeline Ref:      clone-cleanup-workspace
Service Account:   default
Timeout:           1h0m0s
Labels:
 tekton.dev/pipeline=clone-cleanup-workspace

🌡️  Status

STARTED          DURATION     STATUS
42 seconds ago   26 seconds   Succeeded

📦 Resources

 No resources

⚓ Params

 No params

📝 Results

 NAME            VALUE
 ∙ initialized   true

📂 Workspaces

 NAME           SUB PATH   WORKSPACE BINDING
 ∙ git-source   ---        VolumeClaimTemplate

🗂  Taskruns

 NAME                                             TASK NAME          STARTED          DURATION     STATUS
 ∙ write-and-cleanup-workspace-check-git-commit   check-git-commit   24 seconds ago   7 seconds    Succeeded
 ∙ write-and-cleanup-workspace-cleanup            cleanup            24 seconds ago   8 seconds    Succeeded
 ∙ write-and-cleanup-workspace-clone-app-repo     clone-app-repo     42 seconds ago   18 seconds   Succeeded

⏭️  Skipped Tasks

 No Skipped Tasks

[vsinghai]

/assign @vsinghai

[vsinghai]

This feature has been proposed in TEP-0116: Referencing Finally Task Results in Pipeline Results.

[jerop]

This was fixed by @vsinghai in https://github.com/tektoncd/community/blob/main/teps/0116-referencing-finally-task-results-in-pipeline-results.md - thank you!