Skip to content

Commit

Permalink
Rename providedBy to from
Browse files Browse the repository at this point in the history
`providedBy` was a way for users to indicate when a Tasks's input should
come from the output of another Task - including any changes that Task
had made to it (e.g. adding files to a checkout of a git repo by
building a binary). We have struggled with an intuitive way to describe
it - orginally this was called `passedConstraint`. In the design doc for tektoncd#320,
@sebgoa suggested we call this `from` which seems like the clearest name
yet!

Fixes tektoncd#320 (remaining item in tektoncd#320 to add `runAfter` will be moved to
another issue)
  • Loading branch information
bobcatfish authored and knative-prow-robot committed Jan 25, 2019
1 parent a561dd3 commit 405702f
Show file tree
Hide file tree
Showing 21 changed files with 118 additions and 119 deletions.
2 changes: 1 addition & 1 deletion docs/developers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ on path `/pvc` by PipelineRun.
adds a step to copy each output resource to the directory path
`/pvc/task_name/resource_name`.

- If an input resource includes `providedBy` condition then the `TaskRun`
- If an input resource includes `from` condition then the `TaskRun`
controller adds a step to copy from PVC to directory path
`/pvc/previous_task/resource_name`.

Expand Down
18 changes: 9 additions & 9 deletions docs/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,9 +331,9 @@ resource definition.
# Pipeline

A [`Pipeline`](concepts.md#pipelines) defines a list of tasks to execute in
order, while also indicating if any outputs should be used as inputs of a
following task by using [the `providedBy` field](using.md#providedby). The same
templating you used in tasks is also available in pipeline.
order, while also indicating if any outputs should be used as inputs
of a following task by using [the `from` field](using.md#from).
The same templating you used in tasks is also available in pipeline.

For example:

Expand Down Expand Up @@ -369,12 +369,12 @@ spec:
name: demo-deploy-kubectl
resources:
inputs:
- name: workspace
resource: source-repo
- name: image
resource: web-image
providedBy:
- build-skaffold-web
- name: workspace
resource: source-repo
- name: image
resource: web-image
from:
- build-skaffold-web
params:
- name: path
value: /workspace/examples/microservices/leeroy-web/kubernetes/deployment.yaml
Expand Down
41 changes: 20 additions & 21 deletions docs/using.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@
specific to your project (e.g. running your particular set of unit tests).
2. Create a `Pipeline` which expresses the Tasks you would like to run and what
[PipelineResources](#resources-in-a-pipeline) the Tasks need. Use
[`providedBy`](#providedBy) to express when the input of a `Task` should come
[`from`](#from) to express when the input of a `Task` should come
from the output of a previous `Task`.

See [the example Pipeline](../examples/pipeline.yaml).

### PipelineResources in a Pipeline

In order for a `Pipeline` to execute, it will probably need
[`PipelineResources`](#creating-pipelineresources) which will be provided to
[`PipelineResources`](#creating-pipelineresources) which will be given to
`Tasks` as inputs and outputs.

Your `Pipeline` must declare the `PipelineResources` it needs in a `resources`
Expand All @@ -42,7 +42,7 @@ spec:
type: image
```
These `PipelineResources` can then be provided to `Task`s in the `Pipeline` as
These `PipelineResources` can then be given to `Task`s in the `Pipeline` as
inputs and outputs, for example:

```yaml
Expand All @@ -61,21 +61,21 @@ spec:
resource: my-image
```

### ProvidedBy
### From

Sometimes you will have `Tasks` that need to take as input the output of a
previous `Task`, for example, an image built by a previous `Task`.

Express this dependency by adding `providedBy` on `Resources` that your `Tasks`
Express this dependency by adding `from` on `Resources` that your `Tasks`
need.

- The (optional) `providedBy` key on an `input source` defines a set of previous
- The (optional) `from` key on an `input source` defines a set of previous
`PipelineTasks` (i.e. the named instance of a `Task`) in the `Pipeline`
- When the `providedBy` key is specified on an input source, the version of the
resource that is provided by the defined list of tasks is used
- The `providedBy` can support fan in and fan out
- When the `from` key is specified on an input source, the version of
the resource that is from the defined list of tasks is used
- `from` can support fan in and fan out
- The name of the `PipelineResource` must correspond to a `PipelineResource`
from the `Task` that the referenced `PipelineTask` provides as an output
from the `Task` that the referenced `PipelineTask` gives as an output

For example see this `Pipeline` spec:

Expand All @@ -93,13 +93,13 @@ For example see this `Pipeline` spec:
resources:
inputs:
- name: my-image
providedBy:
from:
- build-app
```

The resource `my-image` is expected to be provided to the `deploy-app` `Task`
from the `build-app` `Task`. This means that the `PipelineResource` `my-image`
must also be declared as an output of `build-app`.
The resource `my-image` is expected to be given to the `deploy-app` `Task` from
the `build-app` `Task`. This means that the `PipelineResource` `my-image` must also
be declared as an output of `build-app`.

For implementation details, see [the developer docs](docs/developers/README.md).

Expand Down Expand Up @@ -184,9 +184,9 @@ configure that by edit the `image`'s value in a configmap named
### Resource sharing between tasks

Pipeline `Tasks` are allowed to pass resources from previous `Tasks` via the
[`providedBy`](#providedby) field. This feature is implemented using Persistent
Volume Claims under the hood but however has an implication that tasks cannot
have any volume mounted under path `/pvc`.
[`from`](#from) field. This feature is implemented using
Persistent Volume Claims under the hood but however has an implication
that tasks cannot have any volume mounted under path `/pvc`.

### Outputs

Expand Down Expand Up @@ -338,10 +338,9 @@ In order to run a Pipeline, you will need to provide:
2. The `PipelineResources` to use with this Pipeline.

On its own, a `Pipeline` declares what `Tasks` to run, and dependencies between
`Task` inputs and outputs via [`providedBy`](#providedby). When running a
`Pipeline`, you will need to specify the `PipelineResources` to use with it. One
`Pipeline` may need to be run with different `PipelineResources` in cases such
as:
`Task` inputs and outputs via [`from`](#from). When running a `Pipeline`, you
will need to specify the `PipelineResources` to use with it. One `Pipeline` may
need to be run with different `PipelineResources` in cases such as:

- When triggering the run of a `Pipeline` against a pull request, the triggering
system must specify the commitish of a git `PipelineResource` to use
Expand Down
6 changes: 3 additions & 3 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,15 @@ The [run](./run/) directory contains an example
### Pipeline with outputs

[The Pipeline with outputs](output-pipeline.yaml) contains a Pipeline that
demonstrates how the outputs of a `Task` can be provided as inputs to the next
demonstrates how the outputs of a `Task` can be given as inputs to the next
`Task`. It does this by:

1. Running a `Task` that writes to a `PipelineResource`
2. Running a `Task` that reads the written value from the `PipelineResource`

The [`Output`](../docs/Concepts.md#outputs) of the first `Task` is provided as
The [`Output`](../docs/Concepts.md#outputs) of the first `Task` is given as
an [`Input`](../docs/Concepts.md#inputs) to the next `Task` thanks to the
[`providedBy`](../docs/using.md#providedby) clause.
[`from`](../docs/using.md#from) clause.

#### Output Tasks

Expand Down
2 changes: 1 addition & 1 deletion examples/output-pipeline.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ spec:
inputs:
- name: workspace
resource: source-repo
providedBy: [first-create-file]
from: [first-create-file]
4 changes: 2 additions & 2 deletions examples/pipeline.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ spec:
resource: source-repo
- name: image
resource: app-image
providedBy:
from:
- build-skaffold-app
params:
- name: path
Expand All @@ -68,7 +68,7 @@ spec:
resource: source-repo
- name: image
resource: web-image
providedBy:
from:
- build-skaffold-web
params:
- name: path
Expand Down
4 changes: 2 additions & 2 deletions pkg/apis/pipeline/v1alpha1/pipeline_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,9 @@ type PipelineTaskInputResource struct {
Name string `json:"name"`
// Resource is the name of the DeclaredPipelineResource to use.
Resource string `json:"resource"`
// ProvidedBy is the list of PipelineTask names that the resource has to come from.
// From is the list of PipelineTask names that the resource has to come from.
// +optional
ProvidedBy []string `json:"providedBy,omitempty"`
From []string `json:"from,omitempty"`
}

// PipelineTaskOutputResource maps the name of a declared PipelineResource output
Expand Down
18 changes: 9 additions & 9 deletions pkg/apis/pipeline/v1alpha1/pipeline_validation.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,30 +66,30 @@ func isOutput(task PipelineTask, resource string) bool {
return false
}

// validateProvidedBy ensures that the `providedBy` values make sense: that they rely on values from Tasks
// validateFrom ensures that the `from` values make sense: that they rely on values from Tasks
// that ran previously, and that the PipelineResource is actually an output of the Task it should come from.
// TODO(#168) when pipelines don't just execute linearly this will need to be more sophisticated
func validateProvidedBy(tasks []PipelineTask) error {
func validateFrom(tasks []PipelineTask) error {
for i, t := range tasks {
if t.Resources != nil {
for _, rd := range t.Resources.Inputs {
for _, pb := range rd.ProvidedBy {
for _, pb := range rd.From {
if i == 0 {
return fmt.Errorf("first Task in Pipeline can't depend on anything before it (b/c there is nothing)")
}
found := false
// Look for previous Task that satisfies constraint
for j := i - 1; j >= 0; j-- {
if tasks[j].Name == pb {
// The input resource must actually be an output of the providedBy tasks
// The input resource must actually be an output of the from tasks
if !isOutput(tasks[j], rd.Resource) {
return fmt.Errorf("the resource %s provided by %s must be an output but is an input", rd.Resource, pb)
return fmt.Errorf("the resource %s from %s must be an output but is an input", rd.Resource, pb)
}
found = true
}
}
if !found {
return fmt.Errorf("expected resource %s to be provided by task %s, but task %s doesn't exist", rd.Resource, pb, pb)
return fmt.Errorf("expected resource %s to be from task %s, but task %s doesn't exist", rd.Resource, pb, pb)
}
}
}
Expand Down Expand Up @@ -120,9 +120,9 @@ func (ps *PipelineSpec) Validate() *apis.FieldError {
return apis.ErrInvalidValue(err.Error(), "spec.resources")
}

// The providedBy values should make sense
if err := validateProvidedBy(ps.Tasks); err != nil {
return apis.ErrInvalidValue(err.Error(), "spec.tasks.resources.inputs.providedBy")
// The from values should make sense
if err := validateFrom(ps.Tasks); err != nil {
return apis.ErrInvalidValue(err.Error(), "spec.tasks.resources.inputs.from")
}
return nil
}
18 changes: 9 additions & 9 deletions pkg/apis/pipeline/v1alpha1/pipeline_validation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,28 +36,28 @@ func TestPipelineSpec_Validate_Error(t *testing.T) {
)),
},
{
name: "providedby is on first task",
name: "from is on first task",
p: tb.Pipeline("pipeline", "namespace", tb.PipelineSpec(
tb.PipelineDeclaredResource("great-resource", v1alpha1.PipelineResourceTypeGit),
tb.PipelineTask("foo", "foo-task",
tb.PipelineTaskInputResource("the-resource", "great-resource", tb.ProvidedBy("bar"))),
tb.PipelineTaskInputResource("the-resource", "great-resource", tb.From("bar"))),
)),
},
{
name: "providedby task doesnt exist",
name: "from task doesnt exist",
p: tb.Pipeline("pipeline", "namespace", tb.PipelineSpec(
tb.PipelineDeclaredResource("great-resource", v1alpha1.PipelineResourceTypeGit),
tb.PipelineTask("baz", "baz-task"),
tb.PipelineTask("foo", "foo-task",
tb.PipelineTaskInputResource("the-resource", "great-resource", tb.ProvidedBy("bar"))),
tb.PipelineTaskInputResource("the-resource", "great-resource", tb.From("bar"))),
)),
},
{
name: "providedby task is afterward",
name: "from task is afterward",
p: tb.Pipeline("pipeline", "namespace", tb.PipelineSpec(
tb.PipelineDeclaredResource("great-resource", v1alpha1.PipelineResourceTypeGit),
tb.PipelineTask("foo", "foo-task",
tb.PipelineTaskInputResource("the-resource", "great-resource", tb.ProvidedBy("bar"))),
tb.PipelineTaskInputResource("the-resource", "great-resource", tb.From("bar"))),
tb.PipelineTask("bar", "bar-task",
tb.PipelineTaskOutputResource("the-resource", "great-resource")),
)),
Expand Down Expand Up @@ -90,14 +90,14 @@ func TestPipelineSpec_Validate_Error(t *testing.T) {
)),
},
{
name: "providedBy resource isn't output by task",
name: "from resource isn't output by task",
p: tb.Pipeline("pipeline", "namespace", tb.PipelineSpec(
tb.PipelineDeclaredResource("great-resource", v1alpha1.PipelineResourceTypeGit),
tb.PipelineDeclaredResource("wonderful-resource", v1alpha1.PipelineResourceTypeImage),
tb.PipelineTask("bar", "bar-task",
tb.PipelineTaskInputResource("some-workspace", "great-resource")),
tb.PipelineTask("foo", "foo-task",
tb.PipelineTaskInputResource("wow-image", "wonderful-resource", tb.ProvidedBy("bar"))),
tb.PipelineTaskInputResource("wow-image", "wonderful-resource", tb.From("bar"))),
)),
},
}
Expand Down Expand Up @@ -141,7 +141,7 @@ func TestPipelineSpec_Validate_Valid(t *testing.T) {
tb.PipelineTaskInputResource("some-workspace", "great-resource"),
tb.PipelineTaskOutputResource("some-image", "wonderful-resource")),
tb.PipelineTask("foo", "foo-task",
tb.PipelineTaskInputResource("wow-image", "wonderful-resource", tb.ProvidedBy("bar"))),
tb.PipelineTaskInputResource("wow-image", "wonderful-resource", tb.From("bar"))),
)),
},
}
Expand Down
4 changes: 2 additions & 2 deletions pkg/apis/pipeline/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions pkg/reconciler/v1alpha1/pipeline/resources/dag.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,11 +103,11 @@ func Build(p *v1alpha1.Pipeline) (*DAG, error) {
return nil, errors.NewDuplicatePipelineTask(p, pt.Name)
}
}
// Process all providedBy constraints to add task dependency
// Process all from constraints to add task dependency
for _, pt := range p.Spec.Tasks {
if pt.Resources != nil {
for _, rd := range pt.Resources.Inputs {
for _, constraint := range rd.ProvidedBy {
for _, constraint := range rd.From {
// We need to add dependency from constraint to node n
prev, ok := d.Nodes[constraint]
if !ok {
Expand Down
16 changes: 8 additions & 8 deletions pkg/reconciler/v1alpha1/pipeline/resources/dag_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,37 +31,37 @@ func TestBuild(t *testing.T) {
xDependsOnA := v1alpha1.PipelineTask{
Name: "x",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"a"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"a"}}},
},
}
yDependsOnAB := v1alpha1.PipelineTask{
Name: "y",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"b", "a"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"b", "a"}}},
},
}
zDependsOnX := v1alpha1.PipelineTask{
Name: "z",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"x"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"x"}}},
},
}
aDependsOnZ := v1alpha1.PipelineTask{
Name: "a",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"z"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"z"}}},
},
}
selfLink := v1alpha1.PipelineTask{
Name: "a",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"a"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"a"}}},
},
}
invalidTask := v1alpha1.PipelineTask{
Name: "a",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"none"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"none"}}},
},
}
nodeX := &Node{Task: xDependsOnA, Prev: []*Node{{Task: a}}}
Expand Down Expand Up @@ -149,13 +149,13 @@ func TestGetPrevTasks(t *testing.T) {
x := v1alpha1.PipelineTask{
Name: "x",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"a"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"a"}}},
},
}
y := v1alpha1.PipelineTask{
Name: "y",
Resources: &v1alpha1.PipelineTaskResources{
Inputs: []v1alpha1.PipelineTaskInputResource{{ProvidedBy: []string{"x", "a"}}},
Inputs: []v1alpha1.PipelineTaskInputResource{{From: []string{"x", "a"}}},
},
}
p := v1alpha1.Pipeline{
Expand Down
Loading

0 comments on commit 405702f

Please sign in to comment.