Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement basic CI metric reporting #64263

Merged
merged 8 commits into from
Apr 27, 2020

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Apr 22, 2020

This PR implements some API calls and Jenkins pipeline changes that attempt to do two things:

  • store some information about builds as they are running in an external service
  • store the amount of time the @kbn/optimizer runs triggered by our two distributable builds take, in the same service, using the CiStatsReporter.

This infrastructure will be expanded to collect additional metrics, and eventually will be used to report to PR authors how they are impacting key metrics with their PR.

Included some basic docs: https://github.com/spalger/kibana/blob/implement/ci-stats-build-creation/packages/kbn-dev-utils/src/ci_stats_reporter/README.md

@spalger spalger force-pushed the implement/ci-stats-build-creation branch 14 times, most recently from 4c08bea to d6f28f5 Compare April 24, 2020 05:06
@spalger spalger force-pushed the implement/ci-stats-build-creation branch from 0126a6f to a226114 Compare April 24, 2020 23:30
@spalger spalger marked this pull request as ready for review April 25, 2020 00:37
@spalger spalger requested a review from a team as a code owner April 25, 2020 00:37
@spalger spalger added release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v6.8.9 v7.7.1 v7.8.0 v8.0.0 labels Apr 25, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@spalger spalger changed the title create builds in ci-stats service as builds are running Implement basic CI metric reporting Apr 26, 2020
@spalger
Copy link
Contributor Author

spalger commented Apr 26, 2020

@elasticmachine merge upstream

}

async metric(name: string, subName: string, value: number) {
if (!this.config) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small thing, should this be !this.isEnabled()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use that, but it wouldn't teach TypeScript that this.config should be considered non-null below.

vars/workers.groovy Outdated Show resolved Hide resolved
@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / kibana-xpack-agent / X-Pack API Integration Tests.x-pack/test/api_integration/apis/management/index_management/indices·js.apis management index management indices list should list all the indices with the expected properties and data enrichers

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://github.com/elastic/kibana/issues/64473

[00:00:00]       │
[00:00:00]         └-: apis
[00:00:00]           └-> "before all" hook
[00:05:19]           └-: management
[00:05:19]             └-> "before all" hook
[00:05:42]             └-: index management
[00:05:42]               └-> "before all" hook
[00:05:42]               └-: indices
[00:05:42]                 └-> "before all" hook
[00:05:44]                 └-: list
[00:05:44]                   └-> "before all" hook
[00:05:44]                   └-> should list all the indices with the expected properties and data enrichers
[00:05:44]                     └-> "before each" hook: global before each
[00:05:44]                     └- ✖ fail: "apis management index management indices list should list all the indices with the expected properties and data enrichers"
[00:05:44]                     │

Stack Trace

{ Error: expected [ 'health',
  'status',
  'name',
  'uuid',
  'primary',
  'replica',
  'documents',
  'size',
  'isFrozen',
  'aliases',
  'ilm',
  'isFollowerIndex',
  'isRollupIndex' ] to sort of equal [ 'health',
  'status',
  'name',
  'uuid',
  'primary',
  'replica',
  'documents',
  'size',
  'isFrozen',
  'aliases',
  'isFollowerIndex',
  'ilm',
  'isRollupIndex' ]
    at Assertion.assert (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:244:8)
    at Context.<anonymous> (test/api_integration/apis/management/index_management/indices.js:201:41)
  actual:
   '[\n  "health"\n  "status"\n  "name"\n  "uuid"\n  "primary"\n  "replica"\n  "documents"\n  "size"\n  "isFrozen"\n  "aliases"\n  "ilm"\n  "isFollowerIndex"\n  "isRollupIndex"\n]',
  expected:
   '[\n  "health"\n  "status"\n  "name"\n  "uuid"\n  "primary"\n  "replica"\n  "documents"\n  "size"\n  "isFrozen"\n  "aliases"\n  "isFollowerIndex"\n  "ilm"\n  "isRollupIndex"\n]',
  showDiff: true }

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@spalger spalger merged commit 00ec971 into elastic:master Apr 27, 2020
spalger pushed a commit to spalger/kibana that referenced this pull request Apr 27, 2020
spalger pushed a commit to spalger/kibana that referenced this pull request Apr 27, 2020
# Conflicts:
#	Jenkinsfile
#	packages/kbn-optimizer/src/cli.ts
#	packages/kbn-pm/dist/index.js
spalger pushed a commit to spalger/kibana that referenced this pull request Apr 27, 2020
# Conflicts:
#	Jenkinsfile
#	packages/kbn-optimizer/src/cli.ts
#	packages/kbn-optimizer/src/index.ts
#	packages/kbn-optimizer/src/optimizer/handle_optimizer_completion.ts
#	packages/kbn-pm/dist/index.js
#	src/dev/build/tasks/build_kibana_platform_plugins.js
@spalger spalger removed the v6.8.9 label Apr 27, 2020
@spalger spalger deleted the implement/ci-stats-build-creation branch April 28, 2020 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v7.7.1 v7.8.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants