Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate es-archiver documents data from ES to SO format and provide associated tooling #56727

Closed
pgayvallet opened this issue Feb 4, 2020 · 2 comments
Labels
Feature:New Platform Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@pgayvallet
Copy link
Contributor

Now that SavedObjects types can be registered from NP, the current implementation of the es-archiver has no way to be aware of all the registered SO types, and therefor can no longer instantiate a KibanaMigrator on it's own to perform the migration of it's documents after a load action.

In #55860, we decided to implements a 'temporary' /migrate savedobject endpoint to replace the migration that was done in src/es_archiver/lib/indices/kibana_index.ts.

The consensus on the long term solution would be to keep the archiver data up to date and convert it to saved object format to avoid using raw ES apis, and load them using the SO endpoints (create) instead.

Do achieve this, we need to:

  • Converts all the existing document data used by the es-archiver (example: test/functional/fixtures/es_archiver) from ES raw format to SO format
  • Adapt the es-archiver load action to properly handle the new format, and use the kbnClient.savedObjects to load the SO into elastic.
  • get rid of the manual migration
  • Provides tooling to allow plugin developers to create, migrate and maintains any additional test data they would need to use for testing.
@pgayvallet pgayvallet added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Feature:New Platform labels Feb 4, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-platform (Team:Platform)

@pgayvallet
Copy link
Contributor Author

We now have the kbnArchiver, and QA has been working on migrating the data sets from esArchiver to this new tool, so I'll consider this done or superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:New Platform Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
Status: Done (7.13)
Development

No branches or pull requests

2 participants