Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #20627 to 7.x: Ensure kind cluster has RFC1123 compliant name #20893

Merged
merged 1 commit into from
Sep 22, 2020

Conversation

blakerouse
Copy link
Contributor

@blakerouse blakerouse commented Sep 1, 2020

Cherry-pick of PR #20627 to 7.x branch. Original message:

What does this PR do?

This patch:

  • Modifies the cluster name generation function to produce - where it previously produced _
  • Adds run-time validation that the generated name is compliant
  • Renames the function from kubernetesPodName to kubernetesClusterName, reflecting its purpose

Why is it important?

When creating a kind cluster, we must use a cluster name that is a valid Kubernetes resource name, and by extension, a valid DNS name. If not, kind cluster provisioning can fail when kind tries to create resources with invalid names.

For example, if trying to create a cluster called "a_b" (underscores are not permitted), control-plane provisioning will fail with:

host 'a_b-control-plane' must be [...] a valid RFC-1123 DNS subdomain

With this patch, the kubernetes Metricbeat module integration tests are able to run to completion on my system. Without it, kind fails in this manner:

$ kind create cluster --name metricbeat-8_0_0-a34234bc-master
Creating cluster "metricbeat-8_0_0-a34234bc-master" ...
 ✓ Ensuring node image (kindest/node:v1.18.2) 🖼 
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✗ Starting control-plane 🕹️ 
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged metricbeat-8_0_0-a34234bc-master-control-plane kubeadm init --ignore-preflight-errors=all --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I0817 07:52:20.014341     163 initconfiguration.go:200] loading configuration from "/kind/kubeadm.conf"
[config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta2, Kind=JoinConfiguration
I0817 07:52:20.018540     163 interface.go:400] Looking for default routes with IPv4 addresses
I0817 07:52:20.018553     163 interface.go:405] Default route transits interface "eth0"
I0817 07:52:20.018623     163 interface.go:208] Interface eth0 is up
I0817 07:52:20.018673     163 interface.go:256] Interface "eth0" has 3 addresses :[172.20.0.2/16 fc00:f853:ccd:e793::2/64 fe80::42:acff:fe14:2/64].
I0817 07:52:20.018689     163 interface.go:223] Checking addr  172.20.0.2/16.
I0817 07:52:20.018696     163 interface.go:230] IP found 172.20.0.2
I0817 07:52:20.018702     163 interface.go:262] Found valid IPv4 address 172.20.0.2 for interface "eth0".
I0817 07:52:20.018706     163 interface.go:411] Found active IP 172.20.0.2 
hostport metricbeat-8_0_0-a34234bc-master-control-plane:6443: host 'metricbeat-8_0_0-a34234bc-master-control-plane' must be a valid IP address or a valid RFC-1123 DNS subdomain

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Run the integ tests for the kubernetes Metricbeat module with Mage in verbose mode. kind will be seen to run correctly. Without the patch kind will fail to provision the control-plane.

MODULE=kubernetes mage -v integtest

With Mage not in verbose mode, the kubernetes integ test is not run, and no output is seen (because it dies at setup time).

When creating a kind cluster, we must use a cluster name that is a valid
Kubernetes resource name, and by extension, a valid DNS name. If not,
kind cluster provisioning can fail when kind tries to create resources
with invalid names.

For example, if trying to create a cluster called "a_b" (underscores
are not permitted), control-plane provisioning will fail with:

  host 'a_b-control-plane' must be [...] a valid RFC-1123 DNS subdomain

(cherry picked from commit 345e045)
@blakerouse blakerouse added [zube]: In Review backport Team:Platforms Label for the Integrations - Platforms team labels Sep 1, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 1, 2020
@blakerouse blakerouse self-assigned this Sep 1, 2020
@elasticmachine
Copy link
Collaborator

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20893 opened]

  • Start Time: 2020-09-01T11:55:05.739+0000

  • Duration: 92 min 12 sec

Test stats 🧪

Test Results
Failed 4
Passed 19569
Skipped 1856
Total 21429

Test errors

Expand to view the tests failures

  • Name: Build and Test / Elastic Agent x-pack / TestFleetGateway – application

    • Age: 1
    • Duration: 0
    • Error Details: Failed
  • Name: Build and Test / Elastic Agent x-pack / TestFleetGateway/send_no_event_and_receive_no_action – application

    • Age: 1
    • Duration: 0
    • Error Details: Failed
  • Name: Build and Test / Elastic Agent x-pack / TestFleetGateway/Successfully_connects_and_receives_a_series_of_actions – application

    • Age: 1
    • Duration: 0
    • Error Details: Failed
  • Name: Build and Test / Elastic Agent x-pack / TestFleetGateway/Periodically_communicates_with_Fleet – application

    • Age: 1
    • Duration: 0
    • Error Details: Failed

Steps errors

Expand to view the steps failures

  • Name: Mage build test
    • Description: mage build test

    • Duration: 11 min 9 sec

    • Start Time: 2020-09-01T12:19:18.168+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-09-01T13:26:44.455Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats
[2020-09-01T13:26:44.504Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Lint
[2020-09-01T13:26:44.664Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Winlogbeat-oss
[2020-09-01T13:26:44.823Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-09-01T13:26:44.984Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/dockerlogbeat
[2020-09-01T13:26:45.142Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-09-01T13:26:45.300Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Journalbeat
[2020-09-01T13:26:45.464Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-09-01T13:26:45.627Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Packetbeat-Linux
[2020-09-01T13:26:45.787Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-09-01T13:26:45.957Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-09-01T13:26:46.124Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Heartbeat-Windows
[2020-09-01T13:26:46.298Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Heartbeat-oss
[2020-09-01T13:26:46.473Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-09-01T13:26:46.644Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-09-01T13:26:46.822Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-09-01T13:26:46.995Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-09-01T13:26:47.166Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-09-01T13:26:47.332Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-09-01T13:26:47.507Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-09-01T13:26:47.679Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Filebeat-Windows
[2020-09-01T13:26:47.857Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Functionbeat-Windows
[2020-09-01T13:26:48.022Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-09-01T13:26:48.184Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Libbeat-x-pack
[2020-09-01T13:26:48.352Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Packetbeat-Windows
[2020-09-01T13:26:48.520Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-09-01T13:26:48.690Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-Windows
[2020-09-01T13:26:48.861Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-09-01T13:26:49.031Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-09-01T13:26:49.199Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-09-01T13:26:49.369Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-09-01T13:26:49.537Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-09-01T13:26:49.705Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Filebeat-x-pack
[2020-09-01T13:26:49.871Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-09-01T13:26:50.034Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Filebeat-oss
[2020-09-01T13:26:50.199Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-09-01T13:26:50.369Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-09-01T13:26:50.538Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Libbeat-oss
[2020-09-01T13:26:50.711Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests
[2020-09-01T13:26:50.880Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-09-01T13:26:51.052Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-09-01T13:26:51.220Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Packetbeat-Mac-OS-X
[2020-09-01T13:26:51.388Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-OSS-Python-Integration-tests
[2020-09-01T13:26:51.556Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-09-01T13:26:51.728Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-09-01T13:26:51.899Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-09-01T13:26:52.067Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-09-01T13:26:52.240Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-09-01T13:26:52.410Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-09-01T13:26:52.948Z] + cat
[2020-09-01T13:26:52.948Z] + /usr/local/bin/runbld ./runbld-script --job-name elastic+beats+pull-request
[2020-09-01T13:26:52.948Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-09-01T13:26:59.561Z] runbld>>> runbld started
[2020-09-01T13:26:59.561Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-09-01T13:27:00.950Z] runbld>>> The following profiles matched the job 'elastic+beats+pull-request' in order of occurrence in the config (last value wins).
[2020-09-01T13:27:00.950Z] runbld>>> Matches in the system config:
[2020-09-01T13:27:00.950Z] runbld>>> - Matched ^elastic\+beats
[2020-09-01T13:27:00.950Z] runbld>>> - Matched ^elastic\+beats\+pull-request
[2020-09-01T13:27:02.340Z] runbld>>> Debug logging enabled.
[2020-09-01T13:27:02.340Z] runbld>>> Storing result
[2020-09-01T13:27:02.340Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-09-01T13:27:02.340Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200901132702-36AE9D11
[2020-09-01T13:27:02.340Z] runbld>>> Adding system facts.
[2020-09-01T13:27:03.288Z] runbld>>> Adding vcs info for the latest commit:  49d438c5e2c6cdac47c8a3ac91feaa9d3ffce146
[2020-09-01T13:27:03.288Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-09-01T13:27:03.288Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-09-01T13:27:03.550Z] Processing JUnit reports with runbld...
[2020-09-01T13:27:03.550Z] + echo 'Processing JUnit reports with runbld...'
[2020-09-01T13:27:03.810Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-09-01T13:27:03.810Z] runbld>>> DURATION: 20ms
[2020-09-01T13:27:03.810Z] runbld>>> STDOUT: 40 bytes
[2020-09-01T13:27:03.810Z] runbld>>> STDERR: 49 bytes
[2020-09-01T13:27:03.810Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-09-01T13:27:03.810Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats_PR-20893
[2020-09-01T13:27:04.757Z] runbld>>> Storing build metadata: 
[2020-09-01T13:27:04.757Z] runbld>>> Adding test report.
[2020-09-01T13:27:04.757Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats
[2020-09-01T13:27:05.702Z] runbld>>> Found 140 test output files
[2020-09-01T13:27:08.254Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-09-01T13:27:08.254Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-09-01T13:27:08.254Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-09-01T13:27:08.255Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-09-01T13:27:08.255Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-09-01T13:27:08.829Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-09-01T13:27:08.830Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20893/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-09-01T13:27:09.091Z] runbld>>> Test output logs contained: Errors: 0 Failures: 4 Tests: 21278 Skipped: 1575
[2020-09-01T13:27:09.352Z] runbld>>> Storing result
[2020-09-01T13:27:09.352Z] runbld>>> FAILURES: 4
[2020-09-01T13:27:10.297Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-09-01T13:27:10.297Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200901132702-36AE9D11
[2020-09-01T13:27:10.297Z] runbld>>> Email notification disabled by environment variable.
[2020-09-01T13:27:10.297Z] runbld>>> Slack notification disabled by environment variable.
[2020-09-01T13:27:16.409Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-20893
[2020-09-01T13:27:16.601Z] [INFO] getVaultSecret: Getting secrets
[2020-09-01T13:27:16.683Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-01T13:27:17.809Z] + chmod 755 generate-build-data.sh
[2020-09-01T13:27:17.809Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20893/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20893/runs/1 FAILURE 5531791
[2020-09-01T13:27:17.809Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20893/runs/1/steps/?limit=10000 -o steps-info.json
[2020-09-01T13:27:20.292Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20893/runs/1/tests/?status=FAILED -o tests-errors.json
[2020-09-01T13:27:20.292Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20893/runs/1/log/ -o pipeline-log.txt

@blakerouse
Copy link
Contributor Author

CI failures are un-related to this backport.

@ph ph requested review from jsoriano and exekias September 21, 2020 16:10
@blakerouse blakerouse merged commit f0bd27a into elastic:7.x Sep 22, 2020
@blakerouse blakerouse deleted the backport_20627_7.x branch September 22, 2020 12:46
@zube zube bot removed the [zube]: Done label Dec 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:Platforms Label for the Integrations - Platforms team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants