7.4.0 Release #307

jmlrt · 2019-10-01T14:55:26Z

do what it says on the tin

mgreau

LGTM

jmlrt · 2019-10-01T15:33:34Z

jenkins test this please

jmlrt · 2019-10-01T16:19:52Z

jenkins test this please

jmlrt · 2019-10-01T17:52:34Z

jenkins test this please

jmlrt · 2019-10-02T16:55:30Z

This is failing on metricbeat-oss and metricbeat-security integration tests with the following error:

metricbeat-oss:

HTTP: http://oss-master:9200/_search?q=metricset.name:state_deployment: status: matches expectation: [200]
HTTP: http://oss-master:9200/_search?q=metricset.name:state_deployment: Body: patterns not found: [metricbeat-7.4.0]

Failures/Skipped:

HTTP: http://oss-master:9200/_search?q=metricset.name:state_deployment: Body: patterns not found: [metricbeat-7.4.0]

Total Duration: 0.085s
Count: 15, Failed: 1, Skipped: 0

metricbeat-security:

HTTP: https://security-master:9200/_search?q=metricset.name:state_deployment: status: matches expectation: [200]
HTTP: https://security-master:9200/_search?q=metricset.name:state_deployment: Body: patterns not found: [metricbeat-7.4.0]
HTTP: https://security-master:9200/_cat/indices: status: matches expectation: [200]
HTTP: https://security-master:9200/_cat/indices: Body: matches expectation: [metricbeat-7.4.0]
Command: cd /usr/share/metricbeat && metricbeat test output: exit-status: matches expectation: [0]
Command: cd /usr/share/metricbeat && metricbeat test output: stdout: matches expectation: [elasticsearch: https://security-master:9200 version: 7.4.0]

Failures/Skipped:

HTTP: https://security-master:9200/_search?q=metricset.name:state_deployment: Body: patterns not found: [metricbeat-7.4.0]

Total Duration: 0.144s
Count: 15, Failed: 1, Skipped: 0

However I tried to reproduce it locally but test were OK on my local environment.

jmlrt · 2019-10-02T16:56:30Z

jenkins test this please

jmlrt · 2019-10-02T18:27:32Z

When running metricbeat-oss test locally (using docker-for-mac k8s), it is OK:

# Deploying elasticsearch on docker-for-mac (required to test metricbeat chart)
$ cd helm-charts/elasticsearch/examples/docker-for-mac
# add es helm config required for oss test:
$ cat << EOF >> values.yaml
clusterName: "oss"
image: "docker.elastic.co/elasticsearch/elasticsearch-oss"
EOF
$ make install
...
# Deploying metricbeat-oss and running integration tests
$ cd ../../../metricbeat/examples/oss
$ make test
...
Testing with pod: helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-dbx8f
File: /usr/share/metricbeat/metricbeat.yml: exists: matches expectation: [true]
File: /usr/share/metricbeat/metricbeat.yml: contains: matches expectation: [add_kubernetes_metadata output.elasticsearch]
User: metricbeat: exists: matches expectation: [true]
User: metricbeat: uid: matches expectation: [1000]
User: metricbeat: gid: matches expectation: [1000]
Mount: /usr/share/metricbeat/kube-state-metrics-metricbeat.yml: exists: matches expectation: [true]
Mount: /usr/share/metricbeat/kube-state-metrics-metricbeat.yml: opts: matches expectation: [["ro"]]
Port: tcp:5066: listening: matches expectation: [true]
Port: tcp:5066: ip: matches expectation: [["127.0.0.1"]]
Command: cd /usr/share/metricbeat && metricbeat test output: exit-status: matches expectation: [0]
Command: cd /usr/share/metricbeat && metricbeat test output: stdout: matches expectation: [elasticsearch: http://oss-master:9200 version: 7.4.0]
HTTP: http://oss-master:9200/_cat/indices: status: matches expectation: [200]
HTTP: http://oss-master:9200/_cat/indices: Body: matches expectation: [metricbeat-7.4.0]
HTTP: http://oss-master:9200/_search?q=metricset.name:state_deployment: status: matches expectation: [200]
HTTP: http://oss-master:9200/_search?q=metricset.name:state_deployment: Body: matches expectation: [metricbeat-7.4.0]

Total Duration: 0.317s
Count: 15, Failed: 0, Skipped: 0
...
# Querying http://oss-master:9200/_search?q=metricset.name:state_deployment
$ curl http://oss-master:9200/_search?q=metricset.name:state_deployment
{"took":26,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0},"hits":{"total":{"value":3331,"relation":"eq"},"max_score":2.3524396,"hits":[{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"7oJWjW0BLxXdZczHwjWG","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:38:55.379Z","host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"agent":{"type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0"},"kubernetes":{"namespace":"default","labels":{"heritage":"Tiller","release":"helm-metricbeat-oss","app":"kube-state-metrics","chart":"kube-state-metrics-1.6.0"},"deployment":{"name":"helm-metricbeat-oss-kube-state-metrics","replicas":{"desired":1,"unavailable":0,"available":1,"updated":1},"paused":false}},"event":{"dataset":"kubernetes.deployment","module":"kubernetes","duration":8310277},"metricset":{"name":"state_deployment","period":10000},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"ecs":{"version":"1.1.0"}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"74JWjW0BLxXdZczHwjWG","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:38:55.379Z","event":{"dataset":"kubernetes.deployment","module":"kubernetes","duration":8316726},"metricset":{"name":"state_deployment","period":10000},"ecs":{"version":"1.1.0"},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"agent":{"type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0"},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"kubernetes":{"namespace":"default","labels":{"chart":"metricbeat-7.4.0","heritage":"Tiller","release":"helm-metricbeat-oss","app":"helm-metricbeat-oss-metricbeat-metrics"},"deployment":{"name":"helm-metricbeat-oss-metricbeat-metrics","replicas":{"updated":1,"desired":1,"unavailable":0,"available":1},"paused":false}}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"8IJWjW0BLxXdZczHwjWG","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:38:55.379Z","event":{"dataset":"kubernetes.deployment","module":"kubernetes","duration":8319213},"metricset":{"name":"state_deployment","period":10000},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"kubernetes":{"labels":{"com_docker_image-tag":"v0.4.23","com_docker_default-service-type":"","com_docker_deploy-namespace":"docker","com_docker_fry":"compose"},"deployment":{"replicas":{"desired":1,"unavailable":0,"available":1,"updated":1},"paused":false,"name":"compose"},"namespace":"docker"},"ecs":{"version":"1.1.0"},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"agent":{"id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0","type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"8YJWjW0BLxXdZczHwjWG","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:38:55.379Z","event":{"duration":8346502,"dataset":"kubernetes.deployment","module":"kubernetes"},"metricset":{"name":"state_deployment","period":10000},"service":{"type":"kubernetes","address":"10.102.190.15:8080"},"kubernetes":{"namespace":"docker","labels":{"com_docker_image-tag":"v0.4.23","com_docker_deploy-namespace":"docker","com_docker_fry":"compose.api"},"deployment":{"name":"compose-api","replicas":{"available":1,"updated":1,"desired":1,"unavailable":0},"paused":false}},"agent":{"type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0"},"ecs":{"version":"1.1.0"},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"8oJWjW0BLxXdZczHwjWH","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:38:55.379Z","agent":{"version":"7.4.0","type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05"},"ecs":{"version":"1.1.0"},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"event":{"duration":8348954,"dataset":"kubernetes.deployment","module":"kubernetes"},"metricset":{"period":10000,"name":"state_deployment"},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"kubernetes":{"namespace":"kube-system","labels":{"k8s-app":"kube-dns"},"deployment":{"name":"coredns","replicas":{"updated":2,"desired":2,"unavailable":0,"available":2},"paused":false}}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"84JWjW0BLxXdZczHwjWH","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:38:55.379Z","event":{"dataset":"kubernetes.deployment","module":"kubernetes","duration":8351097},"metricset":{"name":"state_deployment","period":10000},"kubernetes":{"deployment":{"name":"tiller-deploy","replicas":{"unavailable":0,"available":1,"updated":1,"desired":1},"paused":false},"namespace":"kube-system","labels":{"app":"helm","name":"tiller"}},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"ecs":{"version":"1.1.0"},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"agent":{"type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0"}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"RYJXjW0BLxXdZczH0zfK","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:40:05.337Z","agent":{"type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0"},"event":{"dataset":"kubernetes.deployment","module":"kubernetes","duration":6752791},"metricset":{"name":"state_deployment","period":10000},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"kubernetes":{"namespace":"kube-system","labels":{"app":"helm","name":"tiller"},"deployment":{"replicas":{"updated":1,"desired":1,"available":1,"unavailable":0},"paused":false,"name":"tiller-deploy"}},"ecs":{"version":"1.1.0"},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"RoJXjW0BLxXdZczH0zfK","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:40:05.337Z","kubernetes":{"namespace":"default","labels":{"app":"kube-state-metrics","chart":"kube-state-metrics-1.6.0","heritage":"Tiller","release":"helm-metricbeat-oss"},"deployment":{"replicas":{"available":1,"unavailable":0,"updated":1,"desired":1},"paused":false,"name":"helm-metricbeat-oss-kube-state-metrics"}},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"agent":{"type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0"},"ecs":{"version":"1.1.0"},"metricset":{"name":"state_deployment","period":10000},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"event":{"dataset":"kubernetes.deployment","module":"kubernetes","duration":6787791}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"R4JXjW0BLxXdZczH0zfK","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:40:05.337Z","metricset":{"name":"state_deployment","period":10000},"service":{"address":"10.102.190.15:8080","type":"kubernetes"},"kubernetes":{"namespace":"default","labels":{"release":"helm-metricbeat-oss","app":"helm-metricbeat-oss-metricbeat-metrics","chart":"metricbeat-7.4.0","heritage":"Tiller"},"deployment":{"name":"helm-metricbeat-oss-metricbeat-metrics","replicas":{"available":1,"unavailable":0,"updated":1,"desired":1},"paused":false}},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"agent":{"hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0","type":"metricbeat","ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812"},"ecs":{"version":"1.1.0"},"event":{"duration":6791688,"dataset":"kubernetes.deployment","module":"kubernetes"}}},{"_index":"metricbeat-7.4.0-2019.10.02","_type":"_doc","_id":"SIJXjW0BLxXdZczH0zfK","_score":2.3524396,"_source":{"@timestamp":"2019-10-02T16:40:05.337Z","kubernetes":{"namespace":"docker","labels":{"com_docker_fry":"compose","com_docker_image-tag":"v0.4.23","com_docker_default-service-type":"","com_docker_deploy-namespace":"docker"},"deployment":{"name":"compose","replicas":{"desired":1,"available":1,"unavailable":0,"updated":1},"paused":false}},"ecs":{"version":"1.1.0"},"host":{"name":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw"},"agent":{"ephemeral_id":"dda4b994-619b-44a2-aa10-eda46b4db812","hostname":"helm-metricbeat-oss-metricbeat-metrics-74b5bf7854-v2scw","id":"91ef4614-90f8-4852-8d59-59ae1d0a0e05","version":"7.4.0","type":"metricbeat"},"event":{"duration":6821899,"dataset":"kubernetes.deployment","module":"kubernetes"},"metricset":{"name":"state_deployment","period":10000},"service":{[root@helm-metricbeat-oss-metricbeat-4ndfd metricbeat]#

When running on CI job it is KO (see metricbeat-oss logs)

# Querying http://oss-master:9200/_search?q=metricset.name:state_deployment
$ curl http://oss-master:9200/_search?q=metricset.name:state_deployment
{"took":7,"timed_out":false,"_shards":{"total":2,"successful":2,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}

cc @Crazybus

Crazybus · 2019-10-03T06:34:16Z

While it does work locally I was able to reproduce this on an existing GKE cluster. After rebasing on #310 it is working again though. So I think we should merge that one in and add it to this release.

Crazybus

LGTM

jmlrt requested review from mgreau, Conky5 and Crazybus October 1, 2019 14:55

mgreau previously approved these changes Oct 1, 2019

View reviewed changes

jmlrt mentioned this pull request Oct 2, 2019

Helm Charts template testing is failing with TypeError: attrib() got an unexpected keyword argument 'convert' #308

Closed

jmlrt dismissed mgreau’s stale review via e81d3a8 October 2, 2019 08:17

jmlrt force-pushed the 7.4.0 branch from d907ad2 to e81d3a8 Compare October 2, 2019 08:17

jmlrt requested a review from mgreau October 2, 2019 08:38

7.4.0 Release

73c25d1

jmlrt force-pushed the 7.4.0 branch from e81d3a8 to 73c25d1 Compare October 3, 2019 06:55

[metricbeat] update changelog with cluster role configurable pr

0ab91ee

jmlrt force-pushed the 7.4.0 branch from d9b181b to 0ab91ee Compare October 3, 2019 07:37

Crazybus approved these changes Oct 3, 2019

View reviewed changes

jmlrt merged commit 313ee8c into elastic:master Oct 3, 2019

jmlrt deleted the 7.4.0 branch October 3, 2019 08:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

7.4.0 Release #307

7.4.0 Release #307

jmlrt commented Oct 1, 2019

mgreau left a comment

jmlrt commented Oct 1, 2019

jmlrt commented Oct 1, 2019

jmlrt commented Oct 1, 2019

jmlrt commented Oct 2, 2019

jmlrt commented Oct 2, 2019

jmlrt commented Oct 2, 2019

Crazybus commented Oct 3, 2019

Crazybus left a comment

7.4.0 Release #307

7.4.0 Release #307

Conversation

jmlrt commented Oct 1, 2019

mgreau left a comment

Choose a reason for hiding this comment

jmlrt commented Oct 1, 2019

jmlrt commented Oct 1, 2019

jmlrt commented Oct 1, 2019

jmlrt commented Oct 2, 2019

jmlrt commented Oct 2, 2019

jmlrt commented Oct 2, 2019

Crazybus commented Oct 3, 2019

Crazybus left a comment

Choose a reason for hiding this comment