Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[POC] SO Metadata events streams #130976

Closed
wants to merge 56 commits into from

Conversation

sebelga
Copy link
Contributor

@sebelga sebelga commented Apr 26, 2022

POC for the content management "metadata events".

Issue related: #9202

@@ -10,6 +10,7 @@

import { i18n } from '@kbn/i18n';
import { SavedObject } from '@kbn/core/server';
import { defaultUserContentAttributes } from '@kbn/user-content-plugin/server';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having the home plugin knowing about the user_content plugin and do this import we probably want to put an extension point in place where plugins can add modifiers to the sample data before it is loaded.

In this case the user_content plugin would register a modifier in the home plugin to add the defaultUserContentAttributes.

@@ -98,6 +100,7 @@ export function createSavedDashboardClass(
panelsJSON: { type: 'text' },
},
},
...userContentCommonMappings,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure why we have duplicate mappings definition with the server. This would probably need to be refactored to have a single mappings declaration

@@ -35,13 +38,14 @@ export class DashboardPlugin
public setup(core: CoreSetup, plugins: SetupDeps) {
this.logger.debug('dashboard: Setup');

core.savedObjects.registerType(
createDashboardSavedObjectType({
plugins.userContent.registerContent({
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swap savedObjects.registerType with userContent.registerContent and we start gathering metadata information for this saved object type.

router: IRouter,
{ metadataEventsService }: RouteDependencies
) => {
router.post(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This route is used for dev purpose. This logic should be called daily by a Cron Job to update the counters.

@sebelga sebelga force-pushed the poc/saved-object-events-stream branch from f9b9314 to 76a4849 Compare May 16, 2022 08:09
@kibana-ci
Copy link
Collaborator

kibana-ci commented May 20, 2022

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #1 / after fetch initialFilter
  • [job] [logs] Jest Tests #1 / after fetch renders all table rows
  • [job] [logs] Jest Tests #1 / after fetch renders call to action when no dashboards exist
  • [job] [logs] Jest Tests #1 / after fetch renders call to action with continue when no dashboards exist but one is in progress
  • [job] [logs] Jest Tests #1 / after fetch renders warning when listingLimit is exceeded
  • [job] [logs] Jest Tests #1 / after fetch showWriteControls
  • [job] [logs] Jest Tests #1 / after fetch When given a title that matches multiple dashboards, filter on the title
  • [job] [logs] FTR Configs #32 / apis Maps endpoints getTile should return ES vector tile containing documents and metadata
  • [job] [logs] FTR Configs #32 / apis Maps endpoints getTile should return ES vector tile containing documents and metadata
  • [job] [logs] Jest Integration Tests #2 / POST /api/saved_objects/{type} calls upon savedObjectClient.create
  • [job] [logs] Jest Integration Tests #2 / POST /api/saved_objects/{type} can specify an id
  • [job] [logs] Jest Tests #6 / SavedObjectsClient #create makes HTTP call with ID
  • [job] [logs] Jest Tests #6 / SavedObjectsClient #create makes HTTP call without ID
  • [job] [logs] Jest Tests #6 / SavedObjectsRepository#createRepository should create a repository with a unique list of hidden types
  • [job] [logs] Jest Tests #6 / SavedObjectsRepository#createRepository should create a repository without hidden types
  • [job] [logs] Jest Tests #6 / SavedObjectsService #start() #createInternalRepository creates a repository including hidden types when specified
  • [job] [logs] Jest Tests #6 / SavedObjectsService #start() #createInternalRepository creates a repository using the admin user
  • [job] [logs] Jest Tests #6 / SavedObjectsService #start() #createScopedRepository creates a repository including hidden types when specified
  • [job] [logs] Jest Tests #6 / SavedObjectsService #start() #createScopedRepository creates a repository scoped to the user
  • [job] [logs] Jest Tests #3 / TableListView render list view

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
userContent - 10 +10

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
core 977 987 +10
kibanaReact 204 217 +13
metadataEventsStreams - 24 +24
savedObjects 151 152 +1
userContent - 31 +31
total +79

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
userContent - 4 +4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
dashboard 304.2KB 304.3KB +120.0B
kibanaReact 209.5KB 209.5KB +2.0B
total +122.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
core 33 35 +2
userContent - 4 +4
total +6

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
core 300.2KB 300.3KB +121.0B
dashboard 65.9KB 66.1KB +218.0B
kibanaReact 59.5KB 61.1KB +1.6KB
savedObjects 29.0KB 29.1KB +160.0B
userContent - 3.6KB +3.6KB
total +5.8KB

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/development-plugin-saved-objects.html#_mappings

id before after diff
dashboard 22 29 +7
Unknown metric groups

API count

id before after diff
core 2539 2557 +18
kibanaReact 240 253 +13
metadataEventsStreams - 24 +24
savedObjects 192 193 +1
userContent - 36 +36
total +92

ESLint disabled line counts

id before after diff
metadataEventsStreams - 1 +1
userContent - 6 +6
total +7

Total ESLint disabled count

id before after diff
metadataEventsStreams - 1 +1
userContent - 6 +6
total +7

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@sebelga
Copy link
Contributor Author

sebelga commented May 30, 2022

Closing in favor of #133022

@sebelga sebelga closed this May 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants