Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New runners without default labels #249

Merged
merged 5 commits into from
May 22, 2024
Merged

New runners without default labels #249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1d14a26
feat: garm pools do not force default labels
bavarianbidi May 21, 2024
40cdf5b
doc: remove self-hosted label from docs
bavarianbidi May 21, 2024
c4f023b
doc: add some notes about default labels
bavarianbidi May 21, 2024
27e081e
remove required tags during update
bavarianbidi May 22, 2024
debb969
docs: document released provider-common version
bavarianbidi May 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion database/sql/enterprise_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ func (s *EnterpriseTestSuite) SetupTest() {
Flavor: "test-flavor",
OSType: "linux",
OSArch: "amd64",
Tags: []string{"self-hosted", "arm64", "linux"},
Tags: []string{"amd64-linux-runner"},
},
CreateInstanceParams: params.CreateInstanceParams{
Name: "test-instance-name",
Expand Down
2 changes: 1 addition & 1 deletion database/sql/instances_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ func (s *InstancesTestSuite) SetupTest() {
Image: "test-image",
Flavor: "test-flavor",
OSType: "linux",
Tags: []string{"self-hosted", "amd64", "linux"},
Tags: []string{"amd64", "linux"},
}
entity, err := org.GetEntity()
s.Require().Nil(err)
Expand Down
2 changes: 1 addition & 1 deletion database/sql/organizations_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ func (s *OrgTestSuite) SetupTest() {
Flavor: "test-flavor",
OSType: "linux",
OSArch: "amd64",
Tags: []string{"self-hosted", "arm64", "linux"},
Tags: []string{"amd64-linux-runner"},
},
CreateInstanceParams: params.CreateInstanceParams{
Name: "test-instance-name",
Expand Down
2 changes: 1 addition & 1 deletion database/sql/pools_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ func (s *PoolsTestSuite) SetupTest() {
Image: fmt.Sprintf("test-image-%d", i),
Flavor: "test-flavor",
OSType: "linux",
Tags: []string{"self-hosted", "amd64", "linux"},
Tags: []string{"amd64-linux-runner"},
},
)
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion database/sql/repositories_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ func (s *RepoTestSuite) SetupTest() {
Flavor: "test-flavor",
OSType: "windows",
OSArch: "amd64",
Tags: []string{"self-hosted", "arm64", "windows"},
Tags: []string{"arm64-windows-runner"},
},
CreateInstanceParams: params.CreateInstanceParams{
Name: "test-instance",
Expand Down
3 changes: 0 additions & 3 deletions doc/external_provider.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,9 +188,6 @@ Here is a sample of that:
"image": "8ed8a690-69b6-49eb-982f-dcb466895e2d",
"labels": [
"ubuntu",
"self-hosted",
"x64",
"linux",
"openstack",
"runner-controller-id:f9286791-1589-4f39-a106-5b68c2a18af4",
"runner-pool-id:9dcf590a-1192-4a9c-b3e4-e0902974c2c0"
Expand Down
15 changes: 15 additions & 0 deletions doc/labels.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Labels and Tags

Github Runners can be tagged with labels. These labels can be used to restrict the jobs that can run on a runner. For example, you can have a runner with the label `linux` and another with the label `windows`. You can then restrict a job to run only on a runner with the label `linux`.

Whenever a new runner register themselves on Github, the runner knows its own labels as the labels are defined in the pool specification as tags.

Beside the custom labels, Github also has some predefined labels that are appended by the runner script per default.
These are:
```yaml
[ 'self-hosted', '$OS_TYPE', '$OS_ARCH' ]
```

With Version `v0.1.2` of `garm-provider-common`, the runner script will register themselves with a new command line flag, called `--no-default-labels`. If this flag is set, the runner will not append any default label.

As all labels can be defined in the pool specification, it's still possible to add the default labels manually.
14 changes: 7 additions & 7 deletions doc/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -521,7 +521,7 @@ gabriel@rossak:~$ garm-cli pool add \
| Max Runners | 5 |
| Min Idle Runners | 0 |
| Runner Bootstrap Timeout | 20 |
| Tags | self-hosted, amd64, Linux, ubuntu, generic |
| Tags | ubuntu, generic |
| Belongs to | gsamfira/scripts |
| Level | repo |
| Enabled | true |
Expand All @@ -535,11 +535,11 @@ If we list the pool we should see it:

```bash
gabriel@rock:~$ garm-cli pool ls -a
+--------------------------------------+---------------------------+--------------+-----------------------------------------+------------------+-------+---------+---------------+----------+
| ID | IMAGE | FLAVOR | TAGS | BELONGS TO | LEVEL | ENABLED | RUNNER PREFIX | PRIORITY |
+--------------------------------------+---------------------------+--------------+-----------------------------------------+------------------+-------+---------+---------------+----------+
| 344e4a72-2035-4a18-a3d5-87bd3874b56c | ubuntu:22.04 | default | self-hosted amd64 Linux ubuntu generic | gsamfira/scripts | repo | true | garm | 0 |
+--------------------------------------+---------------------------+--------------+-----------------------------------------+------------------+-------+---------+---------------+----------+
+--------------------------------------+---------------------------+--------------+-----------------+------------------+-------+---------+---------------+----------+
| ID | IMAGE | FLAVOR | TAGS | BELONGS TO | LEVEL | ENABLED | RUNNER PREFIX | PRIORITY |
+--------------------------------------+---------------------------+--------------+-----------------+------------------+-------+---------+---------------+----------+
| 344e4a72-2035-4a18-a3d5-87bd3874b56c | ubuntu:22.04 | default | ubuntu generic | gsamfira/scripts | repo | true | garm | 0 |
+--------------------------------------+---------------------------+--------------+-----------------+------------------+-------+---------+---------------+----------+
```

This pool is enabled, but the `min-idle-runners` option is set to 0. This means that it will not create any lingering runners. It will only create runners when a job is started. If your provider is slow to boot up new instances, you may want to set this to a value higher than 0.
Expand Down Expand Up @@ -573,7 +573,7 @@ gabriel@rossak:~$ garm-cli pool update 344e4a72-2035-4a18-a3d5-87bd3874b56c --mi
| Max Runners | 5 |
| Min Idle Runners | 1 |
| Runner Bootstrap Timeout | 20 |
| Tags | self-hosted, amd64, Linux, ubuntu, generic |
| Tags | ubuntu, generic |
| Belongs to | gsamfira/scripts |
| Level | repo |
| Enabled | true |
Expand Down
18 changes: 9 additions & 9 deletions doc/using_garm.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,7 @@ garm-cli pool add \
| Max Runners | 5 |
| Min Idle Runners | 1 |
| Runner Bootstrap Timeout | 20 |
| Tags | self-hosted, x64, Linux, ubuntu, incus |
| Tags | ubuntu, incus |
| Belongs to | gabriel-samfira/garm |
| Level | repo |
| Enabled | false |
Expand Down Expand Up @@ -328,11 +328,11 @@ To list pools created for a repository you can run:

```bash
ubuntu@garm:~$ garm-cli pool list --repo=be3a0673-56af-4395-9ebf-4521fea67567
+--------------------------------------+---------------------------+---------+------------------------------------+------------+-------+---------+---------------+
| ID | IMAGE | FLAVOR | TAGS | BELONGS TO | LEVEL | ENABLED | RUNNER PREFIX |
+--------------------------------------+---------------------------+---------+------------------------------------+------------+-------+---------+---------------+
| 9daa34aa-a08a-4f29-a782-f54950d8521a | images:ubuntu/22.04/cloud | default | self-hosted x64 Linux ubuntu incus | | | false | garm |
+--------------------------------------+---------------------------+---------+------------------------------------+------------+-------+---------+---------------+
+--------------------------------------+---------------------------+---------+--------------+------------+-------+---------+---------------+
| ID | IMAGE | FLAVOR | TAGS | BELONGS TO | LEVEL | ENABLED | RUNNER PREFIX |
+--------------------------------------+---------------------------+---------+--------------+------------+-------+---------+---------------+
| 9daa34aa-a08a-4f29-a782-f54950d8521a | images:ubuntu/22.04/cloud | default | ubuntu incus | | | false | garm |
+--------------------------------------+---------------------------+---------+--------------+------------+-------+---------+---------------+
```

If you want to list pools for an organization or enterprise, you can use the `--org` or `--enterprise` options respectively.
Expand All @@ -355,7 +355,7 @@ ubuntu@garm:~$ garm-cli pool show 9daa34aa-a08a-4f29-a782-f54950d8521a
| Max Runners | 5 |
| Min Idle Runners | 1 |
| Runner Bootstrap Timeout | 20 |
| Tags | self-hosted, x64, Linux, ubuntu, incus |
| Tags | ubuntu, incus |
| Belongs to | gabriel-samfira/garm |
| Level | repo |
| Enabled | false |
Expand Down Expand Up @@ -385,7 +385,7 @@ ubuntu@garm:~$ garm-cli pool update 9daa34aa-a08a-4f29-a782-f54950d8521a --enabl
| Max Runners | 5 |
| Min Idle Runners | 1 |
| Runner Bootstrap Timeout | 20 |
| Tags | self-hosted, x64, Linux, ubuntu, incus |
| Tags | ubuntu, incus |
| Belongs to | gabriel-samfira/garm |
| Level | repo |
| Enabled | false |
Expand Down Expand Up @@ -419,7 +419,7 @@ ubuntu@garm:~$ garm-cli pool update 9daa34aa-a08a-4f29-a782-f54950d8521a --enabl
| Max Runners | 5 |
| Min Idle Runners | 1 |
| Runner Bootstrap Timeout | 20 |
| Tags | self-hosted, x64, Linux, ubuntu, incus |
| Tags | ubuntu, incus |
| Belongs to | gabriel-samfira/garm |
| Level | repo |
| Enabled | true |
Expand Down
2 changes: 1 addition & 1 deletion runner/enterprises_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ func (s *EnterpriseTestSuite) SetupTest() {
Flavor: "test",
OSType: "linux",
OSArch: "arm64",
Tags: []string{"self-hosted", "arm64", "linux"},
Tags: []string{"arm64-linux-runner"},
RunnerBootstrapTimeout: 0,
},
CreateInstanceParams: params.CreateInstanceParams{
Expand Down
2 changes: 1 addition & 1 deletion runner/organizations_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ func (s *OrgTestSuite) SetupTest() {
Flavor: "test",
OSType: "linux",
OSArch: "arm64",
Tags: []string{"self-hosted", "arm64", "linux"},
Tags: []string{"arm64-linux-runner"},
RunnerBootstrapTimeout: 0,
},
CreateInstanceParams: params.CreateInstanceParams{
Expand Down
8 changes: 0 additions & 8 deletions runner/pools.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,14 +99,6 @@ func (r *Runner) UpdatePoolByID(ctx context.Context, poolID string, param params
return params.Pool{}, runnerErrors.NewBadRequestError("min_idle_runners cannot be larger than max_runners")
}

if param.Tags != nil && len(param.Tags) > 0 {
newTags, err := r.processTags(string(pool.OSArch), pool.OSType, param.Tags)
if err != nil {
return params.Pool{}, errors.Wrap(err, "processing tags")
}
param.Tags = newTags
}

entity, err := pool.GithubEntity()
if err != nil {
return params.Pool{}, errors.Wrap(err, "getting entity")
Expand Down
5 changes: 4 additions & 1 deletion runner/pools_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ func (s *PoolTestSuite) SetupTest() {
Image: fmt.Sprintf("test-image-%d", i),
Flavor: "test-flavor",
OSType: "linux",
Tags: []string{"self-hosted", "amd64", "linux"},
Tags: []string{"amd64-linux-runner"},
RunnerBootstrapTimeout: 0,
},
)
Expand All @@ -113,6 +113,9 @@ func (s *PoolTestSuite) SetupTest() {
MinIdleRunners: &minIdleRunners,
Image: "test-images-updated",
Flavor: "test-flavor-updated",
Tags: []string{
"amd64-linux-runner",
},
},
CreateInstanceParams: params.CreateInstanceParams{
Name: "test-instance-name",
Expand Down
4 changes: 2 additions & 2 deletions runner/repositories_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ func (s *RepoTestSuite) SetupTest() {
Flavor: "test",
OSType: "linux",
OSArch: "arm64",
Tags: []string{"self-hosted", "arm64", "linux"},
Tags: []string{"arm64-linux-runner"},
RunnerBootstrapTimeout: 0,
},
CreateInstanceParams: params.CreateInstanceParams{
Expand Down Expand Up @@ -177,7 +177,7 @@ func (s *RepoTestSuite) TestCreateRepository() {
s.Require().Equal(params.PoolBalancerTypeRoundRobin, repo.PoolBalancerType)
}

func (s *RepoTestSuite) TestCreareRepositoryPoolBalancerTypePack() {
func (s *RepoTestSuite) TestCreateRepositoryPoolBalancerTypePack() {
// setup mocks expectations
s.Fixtures.PoolMgrMock.On("Start").Return(nil)
s.Fixtures.PoolMgrCtrlMock.On("CreateRepoPoolManager", s.Fixtures.AdminContext, mock.AnythingOfType("params.Repository"), s.Fixtures.Providers, s.Fixtures.Store).Return(s.Fixtures.PoolMgrMock, nil)
Expand Down
40 changes: 0 additions & 40 deletions runner/runner.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -786,49 +786,9 @@ func (r *Runner) appendTagsToCreatePoolParams(param params.CreatePoolParams) (pa
return params.CreatePoolParams{}, runnerErrors.NewBadRequestError("no such provider %s", param.ProviderName)
}

newTags, err := r.processTags(string(param.OSArch), param.OSType, param.Tags)
if err != nil {
return params.CreatePoolParams{}, fmt.Errorf("failed to process tags: %w", err)
}

param.Tags = newTags

return param, nil
}

func (r *Runner) processTags(osArch string, osType commonParams.OSType, tags []string) ([]string, error) {
// github automatically adds the "self-hosted" tag as well as the OS type (linux, windows, etc)
// and architecture (arm, x64, etc) to all self hosted runners. When a workflow job comes in, we try
// to find a pool based on the labels that are set in the workflow. If we don't explicitly define these
// default tags for each pool, and the user targets these labels, we won't be able to match any pools.
// The downside is that all pools with the same OS and arch will have these default labels. Users should
// set distinct and unique labels on each pool, and explicitly target those labels, or risk assigning
// the job to the wrong worker type.
ghArch, err := util.ResolveToGithubArch(osArch)
if err != nil {
return nil, errors.Wrap(err, "invalid arch")
}

ghOSType, err := util.ResolveToGithubTag(osType)
if err != nil {
return nil, errors.Wrap(err, "invalid os type")
}

labels := []string{
"self-hosted",
ghArch,
ghOSType,
}

for _, val := range tags {
if val != "self-hosted" && val != ghArch && val != ghOSType {
labels = append(labels, val)
}
}

return labels, nil
}

func (r *Runner) GetInstance(ctx context.Context, instanceName string) (params.Instance, error) {
if !auth.IsAdmin(ctx) {
return params.Instance{}, runnerErrors.ErrUnauthorized
Expand Down
Loading