[CI] one shot workers #22626

v1v · 2020-11-17T16:13:22Z

What does this PR do?

Use dynamic labels to assign one shot workers

Why is it important?

To avoid any environmental issues if any with the provisioner.

Test

I'll run a few times this PR to validate it works as expected

elasticmachine · 2020-11-17T17:46:20Z

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Branch indexing
Start Time: 2020-12-17T09:28:44.443+0000
Duration: 168 min 14 sec

Test stats 🧪

Test	Results
Failed	0
Passed	37491
Skipped	3117
Total	40608

Steps errors

Expand to view the steps failures

`Recursively delete the current directory from the workspace`

Took 3 min 48 sec . View more details on here

`Recursively delete the current directory from the workspace`

Took 3 min 59 sec . View more details on here

`Recursively delete the current directory from the workspace`

Took 3 min 52 sec . View more details on here

`Recursively delete the current directory from the workspace`

Took 3 min 0 sec . View more details on here

`Terraform Apply on x-pack/metricbeat/module/aws`

Took 0 min 14 sec . View more details on here
Description: terraform apply -auto-approve

`Terraform Apply on x-pack/metricbeat/module/aws`

Took 0 min 15 sec . View more details on here
Description: terraform apply -auto-approve

`Recursively delete the current directory from the workspace`

Took 3 min 57 sec . View more details on here

`Recursively delete the current directory from the workspace`

Took 4 min 59 sec . View more details on here

Log output

Expand to view the last 100 lines of log output

[2020-12-17T12:15:26.743Z] Versions match. No need to install Go. Exiting.
[2020-12-17T12:15:26.743Z] + exit 0
[2020-12-17T12:15:26.743Z] + .ci/scripts/install-docker-compose.sh
[2020-12-17T12:15:26.743Z] + MSG='environment variable missing: DOCKER_COMPOSE_VERSION.'
[2020-12-17T12:15:26.743Z] + DOCKER_COMPOSE_VERSION=1.21.0
[2020-12-17T12:15:26.743Z] + HOME=/var/lib/jenkins/workspace/Beats_beats_PR-22626
[2020-12-17T12:15:26.743Z] + command -v docker-compose
[2020-12-17T12:15:26.743Z] /usr/local/bin/docker-compose
[2020-12-17T12:15:26.743Z] + echo 'Found docker-compose. Checking version..'
[2020-12-17T12:15:26.743Z] Found docker-compose. Checking version..
[2020-12-17T12:15:26.743Z] ++ docker-compose --version
[2020-12-17T12:15:26.743Z] ++ awk '{print $3}'
[2020-12-17T12:15:26.743Z] ++ sed s/,//
[2020-12-17T12:15:27.361Z] + FOUND_DOCKER_COMPOSE_VERSION=1.23.2
[2020-12-17T12:15:27.361Z] + '[' 1.23.2 == 1.21.0 ']'
[2020-12-17T12:15:27.361Z] + echo 'UNMET DEP: Installing docker-compose'
[2020-12-17T12:15:27.361Z] UNMET DEP: Installing docker-compose
[2020-12-17T12:15:27.361Z] + DC_CMD=/var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/docker-compose
[2020-12-17T12:15:27.361Z] + mkdir -p /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin
[2020-12-17T12:15:27.361Z] ++ uname -s
[2020-12-17T12:15:27.361Z] ++ uname -m
[2020-12-17T12:15:27.361Z] + curl -sSLo /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/docker-compose https://github.com/docker/compose/releases/download/1.21.0/docker-compose-Linux-x86_64
[2020-12-17T12:15:28.047Z] + chmod +x /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/docker-compose
[2020-12-17T12:15:28.047Z] + .ci/scripts/install-terraform.sh
[2020-12-17T12:15:28.047Z] + MSG='environment variable missing.'
[2020-12-17T12:15:28.047Z] + TERRAFORM_VERSION=0.12.24
[2020-12-17T12:15:28.047Z] + HOME=/var/lib/jenkins/workspace/Beats_beats_PR-22626
[2020-12-17T12:15:28.047Z] + TERRAFORM_CMD=/var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/terraform
[2020-12-17T12:15:28.047Z] ++ uname -s
[2020-12-17T12:15:28.047Z] ++ tr '[:upper:]' '[:lower:]'
[2020-12-17T12:15:28.047Z] + OS=linux
[2020-12-17T12:15:28.047Z] + command -v terraform
[2020-12-17T12:15:28.047Z] + echo 'UNMET DEP: Installing Terraform'
[2020-12-17T12:15:28.047Z] UNMET DEP: Installing Terraform
[2020-12-17T12:15:28.047Z] + mkdir -p /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin
[2020-12-17T12:15:28.047Z] + curl -sSLo - https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip
[2020-12-17T12:15:28.618Z] ++ dirname /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/terraform
[2020-12-17T12:15:28.618Z] + unzip -o /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/terraform.zip -d /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin
[2020-12-17T12:15:28.618Z] Archive:  /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/terraform.zip
[2020-12-17T12:15:29.187Z]   inflating: /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/terraform  
[2020-12-17T12:15:29.187Z] + rm /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/terraform.zip
[2020-12-17T12:15:29.187Z] + chmod +x /var/lib/jenkins/workspace/Beats_beats_PR-22626/bin/terraform
[2020-12-17T12:15:29.187Z] + make mage
[2020-12-17T12:15:29.187Z] Installing mage v1.10.0.
[2020-12-17T12:15:30.295Z] go: downloading github.com/magefile/mage v1.10.0
[2020-12-17T12:15:31.245Z] /var/lib/jenkins/workspace/Beats_beats_PR-22626/.magefile cleaned
[2020-12-17T12:15:31.797Z] + git config --get user.email
[2020-12-17T12:15:31.797Z] + [ -z  ]
[2020-12-17T12:15:31.797Z] + git config --global user.email [email protected]
[2020-12-17T12:15:31.797Z] + git config --global user.name beatsmachine
[2020-12-17T12:15:32.194Z] + go mod download
[2020-12-17T12:15:48.468Z] + .ci/scripts/terraform-cleanup.sh x-pack/metricbeat
[2020-12-17T12:15:48.468Z] + DIRECTORY=x-pack/metricbeat
[2020-12-17T12:15:48.468Z] + FAILED=0
[2020-12-17T12:15:48.468Z] ++ find x-pack/metricbeat -name terraform.tfstate
[2020-12-17T12:15:48.468Z] + for tfstate in $(find $DIRECTORY -name terraform.tfstate)
[2020-12-17T12:15:48.468Z] ++ dirname x-pack/metricbeat/module/aws/terraform.tfstate
[2020-12-17T12:15:48.468Z] + cd x-pack/metricbeat/module/aws
[2020-12-17T12:15:48.468Z] + terraform destroy -auto-approve
[2020-12-17T12:15:51.790Z] 
[2020-12-17T12:15:51.790Z] Destroy complete! Resources: 0 destroyed.
[2020-12-17T12:15:51.790Z] + cd -
[2020-12-17T12:15:51.790Z] /var/lib/jenkins/workspace/Beats_beats_PR-22626/src/github.com/elastic/beats/src/github.com/elastic/beats
[2020-12-17T12:15:51.790Z] + exit 0
[2020-12-17T12:15:52.896Z] Client: Docker Engine - Community
[2020-12-17T12:15:52.896Z]  Version:           19.03.14
[2020-12-17T12:15:52.896Z]  API version:       1.40
[2020-12-17T12:15:52.896Z]  Go version:        go1.13.15
[2020-12-17T12:15:52.896Z]  Git commit:        5eb3275d40
[2020-12-17T12:15:52.896Z]  Built:             Tue Dec  1 19:20:17 2020
[2020-12-17T12:15:52.896Z]  OS/Arch:           linux/amd64
[2020-12-17T12:15:52.896Z]  Experimental:      false
[2020-12-17T12:15:52.896Z] 
[2020-12-17T12:15:52.896Z] Server: Docker Engine - Community
[2020-12-17T12:15:52.896Z]  Engine:
[2020-12-17T12:15:52.896Z]   Version:          19.03.14
[2020-12-17T12:15:52.896Z]   API version:      1.40 (minimum version 1.12)
[2020-12-17T12:15:52.896Z]   Go version:       go1.13.15
[2020-12-17T12:15:52.896Z]   Git commit:       5eb3275d40
[2020-12-17T12:15:52.896Z]   Built:            Tue Dec  1 19:18:45 2020
[2020-12-17T12:15:52.896Z]   OS/Arch:          linux/amd64
[2020-12-17T12:15:52.896Z]   Experimental:     false
[2020-12-17T12:15:52.896Z]  containerd:
[2020-12-17T12:15:52.896Z]   Version:          1.3.9
[2020-12-17T12:15:52.896Z]   GitCommit:        ea765aba0d05254012b0b9e595e995c09186427f
[2020-12-17T12:15:52.896Z]  runc:
[2020-12-17T12:15:52.896Z]   Version:          1.0.0-rc10
[2020-12-17T12:15:52.896Z]   GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
[2020-12-17T12:15:52.896Z]  docker-init:
[2020-12-17T12:15:52.896Z]   Version:          0.18.0
[2020-12-17T12:15:52.896Z]   GitCommit:        fec3683
[2020-12-17T12:16:31.356Z] Stage "Packaging" skipped due to earlier failure(s)
[2020-12-17T12:16:31.975Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-22626/src/github.com/elastic/beats
[2020-12-17T12:16:56.588Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-22626
[2020-12-17T12:16:56.784Z] [INFO] getVaultSecret: Getting secrets
[2020-12-17T12:16:56.900Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-12-17T12:16:58.532Z] + chmod 755 generate-build-data.sh
[2020-12-17T12:16:58.532Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-22626/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-22626/runs/1 FAILURE 10093797
[2020-12-17T12:16:59.084Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-22626/runs/1/steps/?limit=10000 -o steps-info.json
[2020-12-17T12:17:37.674Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-22626/runs/1/tests/?status=FAILED -o tests-errors.json

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	37491
Skipped	3117
Total	40608

v1v · 2020-11-17T21:25:04Z

jenkins run the tests please

v1v · 2020-11-18T09:06:43Z

jenkins run the tests please

v1v · 2020-11-18T13:25:52Z

jenkins run the tests please

v1v · 2020-11-18T17:50:51Z

jenkins run the tests please

v1v · 2020-11-19T14:53:02Z

jenkins run the tests please

v1v · 2020-11-20T12:01:35Z

jenkins run the tests please

v1v and others added 3 commits November 17, 2020 16:07

[CI] ensure one shot-worker (elastic#22588)

7aca472

[CI] filter static versus ephemeral workers (elastic#22607)

4e32302

Add logs

35d14df

v1v added macOS Enable builds in the CI for darwin testing arm Enable builds in the CI for ARM testing windows-2016 Enable builds in the CI for windows-2016 windows-2012 Enable builds in the CI for windows-2012 labels Nov 17, 2020

v1v self-assigned this Nov 17, 2020

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Nov 17, 2020

andresrc added the Team:Automation Label for the Observability productivity team label Nov 17, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Nov 17, 2020

v1v added the windows-2008 Enable builds in the CI for windows-2008 label Nov 18, 2020

v1v mentioned this pull request Nov 18, 2020

[CI] retry if environmental issues #22662

Closed

v1v closed this Jan 8, 2021

v1v mentioned this pull request Mar 9, 2021

[CI] enable one shoot workers #24435

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] one shot workers #22626

[CI] one shot workers #22626

v1v commented Nov 17, 2020

elasticmachine commented Nov 17, 2020 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`

`Terraform Apply on x-pack/metricbeat/module/aws`

`Terraform Apply on x-pack/metricbeat/module/aws`

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`

Test stats 🧪

v1v commented Nov 17, 2020

v1v commented Nov 18, 2020

v1v commented Nov 18, 2020

v1v commented Nov 18, 2020

v1v commented Nov 19, 2020

v1v commented Nov 20, 2020

[CI] one shot workers #22626

[CI] one shot workers #22626

Conversation

v1v commented Nov 17, 2020

What does this PR do?

Why is it important?

Test

elasticmachine commented Nov 17, 2020 • edited by jenkins-beats-ci bot Loading

💔 Build Failed

Build stats

Test stats 🧪

Steps errors

Recursively delete the current directory from the workspace

Recursively delete the current directory from the workspace

Recursively delete the current directory from the workspace

Recursively delete the current directory from the workspace

Terraform Apply on x-pack/metricbeat/module/aws

Terraform Apply on x-pack/metricbeat/module/aws

Recursively delete the current directory from the workspace

Recursively delete the current directory from the workspace

Log output

💚 Flaky test report

Test stats 🧪

v1v commented Nov 17, 2020

v1v commented Nov 18, 2020

v1v commented Nov 18, 2020

v1v commented Nov 18, 2020

v1v commented Nov 19, 2020

v1v commented Nov 20, 2020

elasticmachine commented Nov 17, 2020 •

edited by jenkins-beats-ci bot

Loading

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`

`Terraform Apply on x-pack/metricbeat/module/aws`

`Terraform Apply on x-pack/metricbeat/module/aws`

`Recursively delete the current directory from the workspace`

`Recursively delete the current directory from the workspace`