Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Change the auditor to write via an alias #120064

Merged
merged 20 commits into from
Jan 29, 2025

Conversation

davidkyle
Copy link
Member

ML and Transforms have an Auditor that write significant operational messages to an index. For v9 upgrades the notifications indices where the messages are stored may need upgrading, the first step of upgrade is to rollover the index which requires a alias to the index.

Previously the auditor wrote to the concrete index directly, this PR switches to using an alias

  • If the notifications index does not exist then create the index and the alias
  • If the index exists then create the alias if it does not exist (this scenario is covered in the upgrade test MlMappingsUpgradeIT)
  • Added a version number comparison when checking that the index templates is installed. If the template changes (as has happened here) the new one needs to be installed

@davidkyle davidkyle added >upgrade :ml Machine learning auto-backport Automatically create backport pull requests when merged v9.0.0 v8.18.0 labels Jan 13, 2025
@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Jan 13, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@elasticsearchmachine
Copy link
Collaborator

Hi @davidkyle, I've created a changelog YAML for you.

Copy link
Contributor

@przemekwitek przemekwitek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davidkyle davidkyle merged commit a52c26a into elastic:main Jan 29, 2025
15 of 16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 120064

davidkyle added a commit to davidkyle/elasticsearch that referenced this pull request Jan 29, 2025
Changes the ml and transform auditor classes to write through an alias.
The alias is required to rollover the index which required for upgrades
# Conflicts:
#	muted-tests.yml
#	x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MachineLearning.java
#	x-pack/qa/rolling-upgrade/src/test/java/org/elasticsearch/upgrades/MlMappingsUpgradeIT.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged backport pending :ml Machine learning Team:ML Meta label for the ML team >upgrade v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants