[8.0] remove `kibana.index` config property #112773

pgayvallet · 2021-09-22T06:29:43Z

Summary

Currently blocked by https://github.com/elastic/observability-test-environments/issues/915

Related PR

cloud-assets: https://github.com/elastic/cloud-assets/pull/778
cloud: https://github.com/elastic/cloud/pull/89666

Part of #101964

Remove kibana.index and kibana.enabled from config properties
Remove kibana from the shared/legacy global config
Add a new getKibanaIndex API to the savedObject service's setup contract
Adapt all plugins usages of globalConfig.kibana.index to use core.savedObjects.getKibanaIndex instead

Checklist

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

Release Note

Changing kibana.index is no longer supported starting in 8.0. See https://ela.st/kbn-remove-legacy-multitenancy for more details

…-kibana-dot-index

lukeelmers · 2021-09-28T04:13:00Z

I believe Cloud is still configuring kibana.index, so we will need to make sure to remove it there as well.

…-kibana-dot-index

cqliu1

Canvas changes LGTM 👍

lukasolson

AppServices changes LGTM (code review only)

stratoula

Kibana VisEditors team code changes LGTM!

EricDavisX · 2021-09-29T17:33:02Z

Hi - Along with the Observability Robots team ticket cited in the description, the Security Engineering Productivity team has a ticket that they need to complete to support the similar Security relating team environments, i put that in to our private repo elastic/security-team # 1783 @MindyRS fyi

…-kibana-dot-index

pgayvallet · 2021-10-18T14:26:28Z

retest

…-kibana-dot-index

kibanamachine · 2021-10-19T08:30:11Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: df11842
Storybooks Preview
Documentation Changes
Flaky suites:
- xpack-kibana-ciGroup6

Test Failures

Kibana Pipeline / general / X-Pack API Integration Tests.x-pack/test/api_integration/apis/ml/jobs/categorization_field_examples·ts.apis Machine Learning jobs Categorization example endpoint - invalid, too many tokens.

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]     │
[00:00:00]       └-: apis
[00:00:00]         └-> "before all" hook in "apis"
[00:10:36]         └-: Machine Learning
[00:10:36]           └-> "before all" hook in "Machine Learning"
[00:10:36]           └-> "before all" hook in "Machine Learning"
[00:10:36]             │ debg creating role ft_ml_source
[00:10:36]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_source]
[00:10:36]             │ debg creating role ft_ml_source_readonly
[00:10:36]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_source_readonly]
[00:10:36]             │ debg creating role ft_ml_dest
[00:10:36]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_dest]
[00:10:36]             │ debg creating role ft_ml_dest_readonly
[00:10:36]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_dest_readonly]
[00:10:36]             │ debg creating role ft_ml_ui_extras
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_ui_extras]
[00:10:37]             │ debg creating role ft_default_space_ml_all
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space_ml_all]
[00:10:37]             │ debg creating role ft_default_space1_ml_all
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space1_ml_all]
[00:10:37]             │ debg creating role ft_all_spaces_ml_all
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_all_spaces_ml_all]
[00:10:37]             │ debg creating role ft_default_space_ml_read
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space_ml_read]
[00:10:37]             │ debg creating role ft_default_space1_ml_read
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space1_ml_read]
[00:10:37]             │ debg creating role ft_all_spaces_ml_read
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_all_spaces_ml_read]
[00:10:37]             │ debg creating role ft_default_space_ml_none
[00:10:37]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space_ml_none]
[00:10:37]             │ debg creating user ft_ml_poweruser
[00:10:37]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser]
[00:10:37]             │ debg created user ft_ml_poweruser
[00:10:37]             │ debg creating user ft_ml_poweruser_spaces
[00:10:37]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser_spaces]
[00:10:37]             │ debg created user ft_ml_poweruser_spaces
[00:10:37]             │ debg creating user ft_ml_poweruser_space1
[00:10:37]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser_space1]
[00:10:37]             │ debg created user ft_ml_poweruser_space1
[00:10:37]             │ debg creating user ft_ml_poweruser_all_spaces
[00:10:37]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser_all_spaces]
[00:10:37]             │ debg created user ft_ml_poweruser_all_spaces
[00:10:37]             │ debg creating user ft_ml_viewer
[00:10:37]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer]
[00:10:37]             │ debg created user ft_ml_viewer
[00:10:37]             │ debg creating user ft_ml_viewer_spaces
[00:10:37]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer_spaces]
[00:10:37]             │ debg created user ft_ml_viewer_spaces
[00:10:37]             │ debg creating user ft_ml_viewer_space1
[00:10:37]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer_space1]
[00:10:37]             │ debg created user ft_ml_viewer_space1
[00:10:37]             │ debg creating user ft_ml_viewer_all_spaces
[00:10:38]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer_all_spaces]
[00:10:38]             │ debg created user ft_ml_viewer_all_spaces
[00:10:38]             │ debg creating user ft_ml_unauthorized
[00:10:38]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_unauthorized]
[00:10:38]             │ debg created user ft_ml_unauthorized
[00:10:38]             │ debg creating user ft_ml_unauthorized_spaces
[00:10:38]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_unauthorized_spaces]
[00:10:38]             │ debg created user ft_ml_unauthorized_spaces
[00:14:45]           └-: jobs
[00:14:45]             └-> "before all" hook in "jobs"
[00:14:45]             └-: Categorization example endpoint - 
[00:14:45]               └-> "before all" hook for "valid with good number of tokens"
[00:14:45]               └-> "before all" hook for "valid with good number of tokens"
[00:14:45]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Loading "mappings.json"
[00:14:45]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Loading "data.json.gz"
[00:14:45]                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [ft_categorization] creating index, cause [api], templates [], shards [1]/[0]
[00:14:45]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Created index "ft_categorization"
[00:14:45]                 │ debg [x-pack/test/functional/es_archives/ml/categorization] "ft_categorization" settings {"index":{"number_of_replicas":"0","number_of_shards":"1"}}
[00:14:46]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Indexed 1501 docs into "ft_categorization"
[00:14:46]                 │ debg applying update to kibana config: {"dateFormat:tz":"UTC"}
[00:14:47]               └-> valid with good number of tokens
[00:14:47]                 └-> "before each" hook: global before each for "valid with good number of tokens"
[00:14:47]                 └- ✓ pass  (143ms)
[00:14:47]               └-> invalid, too many tokens.
[00:14:47]                 └-> "before each" hook: global before each for "invalid, too many tokens."
[00:14:47]                 │ info [r.suppressed] [node-01] path: /_analyze, params: {}
[00:14:47]                 │      org.elasticsearch.transport.RemoteTransportException: [node-01][127.0.0.1:63181][indices:admin/analyze[s]]
[00:14:47]                 │      Caused by: java.lang.IllegalStateException: The number of tokens produced by calling _analyze has exceeded the allowed maximum of [10000]. This limit can be set by changing the [index.analyze.max_token_count] index level setting.
[00:14:47]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction$TokenCounter.increment(TransportAnalyzeAction.java:397) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.simpleAnalyze(TransportAnalyzeAction.java:229) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.analyze(TransportAnalyzeAction.java:204) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.analyze(TransportAnalyzeAction.java:122) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.shardOperation(TransportAnalyzeAction.java:110) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.shardOperation(TransportAnalyzeAction.java:62) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.lambda$asyncShardOperation$0(TransportSingleShardAction.java:99) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:47) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:14:47]                 │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
[00:14:47]                 │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
[00:14:47]                 │      	at java.lang.Thread.run(Thread.java:833) [?:?]
[00:14:47]                 └- ✖ fail: apis Machine Learning jobs Categorization example endpoint -  invalid, too many tokens.
[00:14:47]                 │       Error: expected 'partially_valid' to sort of equal 'invalid'
[00:14:47]                 │       + expected - actual
[00:14:47]                 │ 
[00:14:47]                 │       -partially_valid
[00:14:47]                 │       +invalid
[00:14:47]                 │       
[00:14:47]                 │       at Assertion.assert (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:14:47]                 │       at Assertion.eql (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/expect/expect.js:244:8)
[00:14:47]                 │       at Context.<anonymous> (test/api_integration/apis/ml/jobs/categorization_field_examples.ts:302:44)
[00:14:47]                 │       at runMicrotasks (<anonymous>)
[00:14:47]                 │       at processTicksAndRejections (node:internal/process/task_queues:96:5)
[00:14:47]                 │       at Object.apply (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:14:47]                 │ 
[00:14:47]                 │

Stack Trace

Error: expected 'partially_valid' to sort of equal 'invalid'
    at Assertion.assert (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/expect/expect.js:244:8)
    at Context.<anonymous> (test/api_integration/apis/ml/jobs/categorization_field_examples.ts:302:44)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16) {
  actual: 'partially_valid',
  expected: 'invalid',
  showDiff: true
}

Metrics [docs]

Unknown metric groups

API count

id	before	after	diff
`core`	2304	2305	+1

History

💔 Build #162099 failed 2888c5e
💔 Build #162025 failed 2888c5e
💔 Build #161909 failed 1c01ac6
💔 Build #161857 failed 3fdb550
💚 Build #159332 succeeded 627ea46

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…-kibana-dot-index

kibanamachine · 2021-10-25T17:17:11Z

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

API count

id	before	after	diff
`core`	2304	2305	+1

History

💚 Build #1258 succeeded ee793e6
💛 Build #818 was flaky f2c38d4
💔 Build #809 failed 277a3ad
💔 Build #803 failed 72ed91a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

pgayvallet added 9 commits September 22, 2021 08:28

remove kibana config

8b51e51

remove kibanaConfig usages

dc8f78b

prettier fix

c766a2b

fix some globalConfig.kibana.index access

5791071

fix xpack_legacy globalConfig usage

954217f

fix home globalConfig usage

19c916d

fix canvas globalConfig usage

d2af27d

fix action globalConfig usage

dfddcc8

fix (all?) remaining usages

43eb4fc

pgayvallet changed the title ~~remove kibana.index and kibana.enabled config properties~~ remove kibana.index config property Sep 22, 2021

pgayvallet added 2 commits September 22, 2021 17:34

fix more plugins

d4e6ef7

fix more plugins bis

c2409c1

pgayvallet added v8.0.0 Breaking Change Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc labels Sep 22, 2021

pgayvallet changed the title ~~remove kibana.index config property~~ [8.0 only] remove kibana.index config property Sep 22, 2021

pgayvallet added the backport:skip This commit does not require backporting label Sep 22, 2021

pgayvallet added 5 commits September 22, 2021 20:21

yet more usages

6dc9a7d

fix ml usages

0632b83

fix security_solution

bedf656

fix lens

bb48eb0

fix monitoring

9920a84

pgayvallet added the release_note:breaking label Sep 23, 2021

pgayvallet added 7 commits September 23, 2021 08:34

remove from settings docs

d72fe95

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

566f266

…-kibana-dot-index

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

057d9e4

…-kibana-dot-index

move doc update

fecb83b

fix unit tests

c6470cd

update generated doc

7efd28c

improve test

723312d

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

a098cd2

…-kibana-dot-index

cqliu1 approved these changes Sep 28, 2021

View reviewed changes

neptunian approved these changes Sep 28, 2021

View reviewed changes

lukasolson approved these changes Sep 28, 2021

View reviewed changes

jloleysens approved these changes Sep 29, 2021

View reviewed changes

stratoula approved these changes Sep 29, 2021

View reviewed changes

pgayvallet added 2 commits September 30, 2021 11:37

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

f068d13

…-kibana-dot-index

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

c46ae22

…-kibana-dot-index

pgayvallet mentioned this pull request Oct 5, 2021

Remove kibana.index, xpack.reporting.index, xpack.task_manager.index settings #101964

Closed

3 tasks

pgayvallet added 2 commits October 6, 2021 07:55

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

4d443c4

…-kibana-dot-index

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

627ea46

…-kibana-dot-index

pgayvallet added 3 commits October 18, 2021 08:11

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

3fdb550

…-kibana-dot-index

adapt new usage in security_solution

1c01ac6

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

2888c5e

…-kibana-dot-index

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

df11842

…-kibana-dot-index

pgayvallet added 6 commits October 21, 2021 09:50

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

7e95da3

…-kibana-dot-index

fix security_solution config

72ed91a

fix createConfig, again

277a3ad

fix mock config

f2c38d4

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

ee793e6

…-kibana-dot-index

Merge remote-tracking branch 'upstream/master' into kbn-101964-remove…

f112393

…-kibana-dot-index

kobelb merged commit 1061835 into elastic:master Oct 25, 2021

Desvelao mentioned this pull request Nov 7, 2022

Compatibility with Elastic 8 wazuh/wazuh-dashboard-plugins#4567

Closed

8 tasks

romain-chanu mentioned this pull request Mar 7, 2023

Provide a clear migration path when removing kibana.index #152792

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[8.0] remove `kibana.index` config property #112773

[8.0] remove `kibana.index` config property #112773

pgayvallet commented Sep 22, 2021 •

edited

Loading

lukeelmers commented Sep 28, 2021

cqliu1 left a comment

lukasolson left a comment

stratoula left a comment

EricDavisX commented Sep 29, 2021

pgayvallet commented Oct 18, 2021

kibanamachine commented Oct 19, 2021

Standard Out

Stack Trace

API count

kibanamachine commented Oct 25, 2021

API count

[8.0] remove kibana.index config property #112773

[8.0] remove kibana.index config property #112773

Conversation

pgayvallet commented Sep 22, 2021 • edited Loading

Summary

Checklist

Release Note

lukeelmers commented Sep 28, 2021

cqliu1 left a comment

Choose a reason for hiding this comment

lukasolson left a comment

Choose a reason for hiding this comment

stratoula left a comment

Choose a reason for hiding this comment

EricDavisX commented Sep 29, 2021

pgayvallet commented Oct 18, 2021

kibanamachine commented Oct 19, 2021

💛 Build succeeded, but was flaky

Test Failures

Standard Out

Stack Trace

Metrics [docs]

API count

History

kibanamachine commented Oct 25, 2021

💚 Build Succeeded

Metrics [docs]

API count

History

[8.0] remove `kibana.index` config property #112773

[8.0] remove `kibana.index` config property #112773

pgayvallet commented Sep 22, 2021 •

edited

Loading