Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEP-0133]: Configure Default Resolver #6317

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions config/config-defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -82,3 +82,7 @@ data:
# default-forbidden-env contains comma seperated environment variables that cannot be
# overridden by podTemplate.
default-forbidden-env:

# default-resolver-type contains the default resolver type to be used in the cluster,
# no default-resolver-type is specified by default
default-resolver-type:
7 changes: 5 additions & 2 deletions docs/additional-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ _In the above example the environment variable `TEST_TEKTON` will not be overrid

## Customizing basic execution parameters

You can specify your own values that replace the default service account (`ServiceAccount`), timeout (`Timeout`), and Pod template (`PodTemplate`) values used by Tekton Pipelines in `TaskRun` and `PipelineRun` definitions. To do so, modify the ConfigMap `config-defaults` with your desired values.
You can specify your own values that replace the default service account (`ServiceAccount`), timeout (`Timeout`), resolver (`Resolver`), and Pod template (`PodTemplate`) values used by Tekton Pipelines in `TaskRun` and `PipelineRun` definitions. To do so, modify the ConfigMap `config-defaults` with your desired values.

The example below customizes the following:

Expand All @@ -156,9 +156,10 @@ The example below customizes the following:
- the default `app.kubernetes.io/managed-by` label is applied to all Pods created to execute `TaskRuns`.
- the default Pod template to include a node selector to select the node where the Pod will be scheduled by default. A list of supported fields is available [here](https://github.com/tektoncd/pipeline/blob/main/docs/podtemplates.md#supported-fields).
For more information, see [`PodTemplate` in `TaskRuns`](./taskruns.md#specifying-a-pod-template) or [`PodTemplate` in `PipelineRuns`](./pipelineruns.md#specifying-a-pod-template).
- the default `Workspace` configuration can be set for any `Workspaces` that a Task declares but that a TaskRun does not explicitly provide
- the default `Workspace` configuration can be set for any `Workspaces` that a Task declares but that a TaskRun does not explicitly provide.
- the default maximum combinations of `Parameters` in a `Matrix` that can be used to fan out a `PipelineTask`. For
more information, see [`Matrix`](matrix.md).
- the default resolver type to `git`.

```yaml
apiVersion: v1
Expand All @@ -175,6 +176,7 @@ data:
default-task-run-workspace-binding: |
emptyDir: {}
default-max-matrix-combinations-count: "1024"
default-resolver-type: "git"
```

**Note:** The `_example` key in the provided [config-defaults.yaml](./../config/config-defaults.yaml)
Expand Down Expand Up @@ -288,6 +290,7 @@ Features currently in "alpha" are:
| [Trusted Resources](./trusted-resources.md) | [TEP-0091](https://github.com/tektoncd/community/blob/main/teps/0091-trusted-resources.md) | N/A | `resource-verification-mode` |
| [`Provenance` field in Status](pipeline-api.md#provenance) | [issue#5550](https://github.com/tektoncd/pipeline/issues/5550) | N/A | `enable-provenance-in-status` |
| [Larger Results via Sidecar Logs](#enabling-larger-results-using-sidecar-logs) | [TEP-0127](https://github.com/tektoncd/community/blob/main/teps/0127-larger-results-via-sidecar-logs.md) | [v0.43.0](https://github.com/tektoncd/pipeline/releases/tag/v0.43.0) | `results-from` |
| [Configure Default Resolver](./resolution.md#configuring-built-in-resolvers) | [TEP-0133](https://github.com/tektoncd/community/blob/main/teps/0133-configure-default-resolver.md) | N/A | |

### Beta Features

Expand Down
2 changes: 2 additions & 0 deletions docs/resolution.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ For new users getting started with Tekton Pipeline remote resolution, check out
These resolvers are enabled by setting the appropriate feature flag in the `resolvers-feature-flags`
ConfigMap in the `tekton-pipelines-resolvers` namespace. See the [section in install.md](install.md#configuring-built-in-remote-task-and-pipeline-resolution) for details.

The default resolver type can be configured by the `default-resolver-type` field in the `config-defaults` ConfigMap (`alpha` feature). See [additional-configs.md](./additional-configs.md) for details.

## Developer Howto: Writing a Resolver From Scratch

For a developer getting started with writing a new Resolver, see
Expand Down
10 changes: 10 additions & 0 deletions pkg/apis/config/default.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ const (
DefaultCloudEventSinkValue = ""
// DefaultMaxMatrixCombinationsCount is used when no max matrix combinations count is specified.
DefaultMaxMatrixCombinationsCount = 256
// DefaultResolverTypeValue is used when no default resolver type is specified
DefaultResolverTypeValue = ""

defaultTimeoutMinutesKey = "default-timeout-minutes"
defaultServiceAccountKey = "default-service-account"
Expand All @@ -54,6 +56,7 @@ const (
defaultTaskRunWorkspaceBinding = "default-task-run-workspace-binding"
defaultMaxMatrixCombinationsCountKey = "default-max-matrix-combinations-count"
defaultForbiddenEnv = "default-forbidden-env"
defaultResolverTypeKey = "default-resolver-type"
)

// Defaults holds the default configurations
Expand All @@ -68,6 +71,7 @@ type Defaults struct {
DefaultTaskRunWorkspaceBinding string
DefaultMaxMatrixCombinationsCount int
DefaultForbiddenEnv []string
DefaultResolverType string
}

// GetDefaultsConfigName returns the name of the configmap containing all
Expand Down Expand Up @@ -97,6 +101,7 @@ func (cfg *Defaults) Equals(other *Defaults) bool {
other.DefaultCloudEventsSink == cfg.DefaultCloudEventsSink &&
other.DefaultTaskRunWorkspaceBinding == cfg.DefaultTaskRunWorkspaceBinding &&
other.DefaultMaxMatrixCombinationsCount == cfg.DefaultMaxMatrixCombinationsCount &&
other.DefaultResolverType == cfg.DefaultResolverType &&
reflect.DeepEqual(other.DefaultForbiddenEnv, cfg.DefaultForbiddenEnv)
}

Expand All @@ -108,6 +113,7 @@ func NewDefaultsFromMap(cfgMap map[string]string) (*Defaults, error) {
DefaultManagedByLabelValue: DefaultManagedByLabelValue,
DefaultCloudEventsSink: DefaultCloudEventSinkValue,
DefaultMaxMatrixCombinationsCount: DefaultMaxMatrixCombinationsCount,
DefaultResolverType: DefaultResolverTypeValue,
}

if defaultTimeoutMin, ok := cfgMap[defaultTimeoutMinutesKey]; ok {
Expand Down Expand Up @@ -166,6 +172,10 @@ func NewDefaultsFromMap(cfgMap map[string]string) (*Defaults, error) {
tc.DefaultForbiddenEnv = tmpString.List()
}

if defaultResolverType, ok := cfgMap[defaultResolverTypeKey]; ok {
tc.DefaultResolverType = defaultResolverType
}

return &tc, nil
}

Expand Down
1 change: 1 addition & 0 deletions pkg/apis/config/default_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ func TestNewDefaultsFromConfigMap(t *testing.T) {
DefaultServiceAccount: "tekton",
DefaultManagedByLabelValue: "something-else",
DefaultMaxMatrixCombinationsCount: 256,
DefaultResolverType: "git",
},
fileName: config.GetDefaultsConfigName(),
},
Expand Down
1 change: 1 addition & 0 deletions pkg/apis/config/testdata/config-defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ data:
default-timeout-minutes: "50"
default-service-account: "tekton"
default-managed-by-label-value: "something-else"
default-resolver-type: "git"
5 changes: 5 additions & 0 deletions pkg/apis/pipeline/v1/pipeline_defaults.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package v1
import (
"context"

"github.com/tektoncd/pipeline/pkg/apis/config"
"knative.dev/pkg/apis"
)

Expand All @@ -31,6 +32,7 @@ func (p *Pipeline) SetDefaults(ctx context.Context) {

// SetDefaults sets default values for the PipelineSpec's Params, Tasks, and Finally
func (ps *PipelineSpec) SetDefaults(ctx context.Context) {
cfg := config.FromContextOrDefaults(ctx)
for i := range ps.Params {
ps.Params[i].SetDefaults(ctx)
}
Expand All @@ -40,6 +42,9 @@ func (ps *PipelineSpec) SetDefaults(ctx context.Context) {
if pt.TaskRef.Kind == "" {
pt.TaskRef.Kind = NamespacedTaskKind
}
if pt.TaskRef.Name == "" && pt.TaskRef.Resolver == "" {
pt.TaskRef.Resolver = ResolverName(cfg.Defaults.DefaultResolverType)
}
}
if pt.TaskSpec != nil {
pt.TaskSpec.SetDefaults(ctx)
Expand Down
59 changes: 56 additions & 3 deletions pkg/apis/pipeline/v1/pipeline_defaults_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (
"testing"

"github.com/google/go-cmp/cmp"
dfttesting "github.com/tektoncd/pipeline/pkg/apis/config/testing"
v1 "github.com/tektoncd/pipeline/pkg/apis/pipeline/v1"
"github.com/tektoncd/pipeline/test/diff"
)
Expand All @@ -37,9 +38,10 @@ func TestPipeline_SetDefaults(t *testing.T) {

func TestPipelineSpec_SetDefaults(t *testing.T) {
cases := []struct {
desc string
ps *v1.PipelineSpec
want *v1.PipelineSpec
desc string
ps *v1.PipelineSpec
want *v1.PipelineSpec
defaults map[string]string
}{{
desc: "empty pipelineSpec must not change after setting defaults",
ps: &v1.PipelineSpec{},
Expand Down Expand Up @@ -120,6 +122,54 @@ func TestPipelineSpec_SetDefaults(t *testing.T) {
},
}},
},
}, {
desc: "pipeline task with taskRef - with default resolver",
ps: &v1.PipelineSpec{
Tasks: []v1.PipelineTask{{
Name: "foo",
TaskRef: &v1.TaskRef{},
}},
},
want: &v1.PipelineSpec{
Tasks: []v1.PipelineTask{{
Name: "foo",
TaskRef: &v1.TaskRef{
Kind: v1.NamespacedTaskKind,
ResolverRef: v1.ResolverRef{
Resolver: "git",
},
},
}},
},
defaults: map[string]string{
"default-resolver-type": "git",
},
}, {
desc: "pipeline task with taskRef - user-provided resolver overwrites default resolver",
ps: &v1.PipelineSpec{
Tasks: []v1.PipelineTask{{
Name: "foo",
TaskRef: &v1.TaskRef{
ResolverRef: v1.ResolverRef{
Resolver: "custom resolver",
},
},
}},
},
want: &v1.PipelineSpec{
Tasks: []v1.PipelineTask{{
Name: "foo",
TaskRef: &v1.TaskRef{
Kind: v1.NamespacedTaskKind,
ResolverRef: v1.ResolverRef{
Resolver: "custom resolver",
},
},
}},
},
defaults: map[string]string{
"default-resolver-type": "git",
},
}, {
desc: "final pipeline task with taskSpec - default param type must be " + string(v1.ParamTypeString),
ps: &v1.PipelineSpec{
Expand Down Expand Up @@ -149,6 +199,9 @@ func TestPipelineSpec_SetDefaults(t *testing.T) {
for _, tc := range cases {
t.Run(tc.desc, func(t *testing.T) {
ctx := context.Background()
if len(tc.defaults) > 0 {
ctx = dfttesting.SetDefaults(context.Background(), t, tc.defaults)
}
tc.ps.SetDefaults(ctx)
if d := cmp.Diff(tc.want, tc.ps); d != "" {
t.Errorf("Mismatch of pipelineSpec after setting defaults: %s", diff.PrintWantGot(d))
Expand Down
3 changes: 3 additions & 0 deletions pkg/apis/pipeline/v1/pipelinerun_defaults.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ func (pr *PipelineRun) SetDefaults(ctx context.Context) {
// SetDefaults implements apis.Defaultable
func (prs *PipelineRunSpec) SetDefaults(ctx context.Context) {
cfg := config.FromContextOrDefaults(ctx)
if prs.PipelineRef != nil && prs.PipelineRef.Name == "" && prs.PipelineRef.Resolver == "" {
prs.PipelineRef.Resolver = ResolverName(cfg.Defaults.DefaultResolverType)
}

if prs.Timeouts == nil || prs.Timeouts.Pipeline == nil {
prs.Timeouts = &TimeoutFields{
Expand Down
50 changes: 50 additions & 0 deletions pkg/apis/pipeline/v1/pipelinerun_defaults_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,56 @@ func TestPipelineRunDefaulting(t *testing.T) {
"default-service-account": "tekton",
"default-pod-template": "nodeSelector: { 'label': 'value' }\nhostNetwork: true",
},
}, {
name: "PipelineRef uses default resolver",
in: &v1.PipelineRun{Spec: v1.PipelineRunSpec{PipelineRef: &v1.PipelineRef{}}},
want: &v1.PipelineRun{
Spec: v1.PipelineRunSpec{
TaskRunTemplate: v1.PipelineTaskRunTemplate{
ServiceAccountName: config.DefaultServiceAccountValue,
},
Timeouts: &v1.TimeoutFields{
Pipeline: &metav1.Duration{Duration: config.DefaultTimeoutMinutes * time.Minute},
},
PipelineRef: &v1.PipelineRef{
ResolverRef: v1.ResolverRef{
Resolver: "git",
},
},
},
},
defaults: map[string]string{
"default-resolver-type": "git",
},
}, {
name: "PipelineRef user-provided resolver overwrites default resolver",
in: &v1.PipelineRun{
Spec: v1.PipelineRunSpec{
PipelineRef: &v1.PipelineRef{
ResolverRef: v1.ResolverRef{
Resolver: "hub",
},
},
},
},
want: &v1.PipelineRun{
Spec: v1.PipelineRunSpec{
TaskRunTemplate: v1.PipelineTaskRunTemplate{
ServiceAccountName: config.DefaultServiceAccountValue,
},
Timeouts: &v1.TimeoutFields{
Pipeline: &metav1.Duration{Duration: config.DefaultTimeoutMinutes * time.Minute},
},
PipelineRef: &v1.PipelineRef{
ResolverRef: v1.ResolverRef{
Resolver: "hub",
},
},
},
},
defaults: map[string]string{
"default-resolver-type": "git",
},
}}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
Expand Down
9 changes: 7 additions & 2 deletions pkg/apis/pipeline/v1/taskrun_defaults.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,13 @@ func (tr *TaskRun) SetDefaults(ctx context.Context) {
// SetDefaults implements apis.Defaultable
func (trs *TaskRunSpec) SetDefaults(ctx context.Context) {
cfg := config.FromContextOrDefaults(ctx)
if trs.TaskRef != nil && trs.TaskRef.Kind == "" {
trs.TaskRef.Kind = NamespacedTaskKind
if trs.TaskRef != nil {
if trs.TaskRef.Kind == "" {
trs.TaskRef.Kind = NamespacedTaskKind
}
if trs.TaskRef.Name == "" && trs.TaskRef.Resolver == "" {
trs.TaskRef.Resolver = ResolverName(cfg.Defaults.DefaultResolverType)
}
}

if trs.Timeout == nil {
Expand Down
54 changes: 54 additions & 0 deletions pkg/apis/pipeline/v1/taskrun_defaults_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,60 @@ func TestTaskRunDefaulting(t *testing.T) {
"default-service-account": "tekton",
"default-pod-template": "nodeSelector: { 'label': 'value' }",
},
}, {
name: "TaskRef with default resolver",
in: &v1.TaskRun{
Spec: v1.TaskRunSpec{
TaskRef: &v1.TaskRef{},
},
},
want: &v1.TaskRun{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{"app.kubernetes.io/managed-by": "tekton-pipelines"},
},
Spec: v1.TaskRunSpec{
TaskRef: &v1.TaskRef{
Kind: "Task",
ResolverRef: v1.ResolverRef{
Resolver: "git",
},
},
Timeout: &metav1.Duration{Duration: time.Hour},
ServiceAccountName: "default",
},
},
defaults: map[string]string{
"default-resolver-type": "git",
},
}, {
name: "TaskRef user-provided resolver overwrites default resolver",
in: &v1.TaskRun{
Spec: v1.TaskRunSpec{
TaskRef: &v1.TaskRef{
ResolverRef: v1.ResolverRef{
Resolver: "custom resolver",
},
},
},
},
want: &v1.TaskRun{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{"app.kubernetes.io/managed-by": "tekton-pipelines"},
},
Spec: v1.TaskRunSpec{
TaskRef: &v1.TaskRef{
Kind: "Task",
ResolverRef: v1.ResolverRef{
Resolver: "custom resolver",
},
},
Timeout: &metav1.Duration{Duration: time.Hour},
ServiceAccountName: "default",
},
},
defaults: map[string]string{
"default-resolver-type": "git",
},
}}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
Expand Down
Loading