[Entity Analytics] API changes for right placement of deleting the old component template

[abhishekbhatia1710]

Summary

• Delete the old component template after the index template has referenced to the new component template
• Test cases for the same flow

# Let's assume this is 8.15.3
# Create the component template when Risk Score engine is initialised
# Create the index template which references the created component template

PUT /_component_template/.risk-score-mappings
{
  "template": {
    "settings": {
      "number_of_shards": 1
    },
    "mappings": {
      "properties": {
        "timestamp": {
          "type": "date"
        },
        "user": {
          "properties": {
            "id": {
              "type": "keyword"
            },
            "name": {
              "type": "text"
            }
          }
        }
      }
    }
  },
  "version": 1
}

PUT /_index_template/.risk-score.risk-score-default-index-template
{
  "index_patterns": [".risk-score.risk-score-default-index-template"],
  "template": {
    "settings": {
      "number_of_replicas": 1
    }
  },
  "composed_of": [".risk-score-mappings"],
  "priority": 100,
  "version": 1,
  "_meta": {
    "description": "Index template for indices with the pattern my_index-*"
  }
}

# The deployment is updated to 8.16
# User tries to enable the Entity store which init's the Risk Score engine (again!!) 
# Fails, but creates the component template and cannot update the index template to reference the new component template due to the error

PUT /_component_template/.risk-score-mappings-default
{
  "template": {
    "settings": {
      "number_of_shards": 1
    },
    "mappings": {
      "properties": {
        "timestamp": {
          "type": "date"
        },
        "user": {
          "properties": {
            "id": {
              "type": "keyword"
            },
            "name": {
              "type": "text"
            }
          }
        }
      }
    }
  },
  "version": 1
}

GET /_component_template?filter_path=component_templates.name&name=.risk-score-mappings*


DELETE /_component_template/.risk-score-mappings

# Fails 

# changed flow
PUT /_index_template/.risk-score.risk-score-default-index-template
{
  "index_patterns": [".risk-score.risk-score-default-index-template"],
  "template": {
    "settings": {
      "number_of_replicas": 1
    }
  },
  "composed_of": [".risk-score-mappings-default"],
  "priority": 100,
  "version": 1,
  "_meta": {
    "description": "Index template for indices with the pattern my_index-*"
  }
}

DELETE /_component_template/.risk-score-mappings

# Succeeds

#########

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios
• Flaky Test Runner was used on any tests changed

For maintainers

• This was checked for breaking API changes and was labeled appropriately
• This will appear in the Release Notes and follow the guidelines

[elasticmachine]

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

[abhishekbhatia1710]

Question:

Should we really modify the UI modal to avoid calling the init Risk Engine API if it's already initialised?

In my opinion, the init Risk Score Engine API updates the component and index templates as needed, ensuring they remain updated with the correct space name in any flow (upgrade or standard)

@jaredburgettelastic @hop-dev

[machadoum]

@abhishekbhatia1710 This PR looks good to me. But let's address your suggestion.

Should we really modify the UI modal to avoid calling the init Risk Engine API if it's already initialised?

I agree 100% with this. But I thought we already had this check in place.
@tiansivive Is moving the enablement to BE. So we should make sure it is disabled before calling init.

[machadoum]

The code LGTM!

I don't have time today to desk test it.

Are you going to address this comment before merging the PR?
#199734 (comment)

[abhishekbhatia1710]

The code LGTM!
I don't have time today to desk test it.
Are you going to address this comment before merging the PR?
#199734 (comment)

No, this scenario is already handled. Joe mentioned it in his comment (https://elastic.slack.com/archives/C05V9NE6X4K/p1731615750750869). The toggle button is disabled if the risk engine is already installed, preventing the user from enabling it through the UI.

[kibanamachine]

Starting backport for target branches: 8.16, 8.x

https://github.com/elastic/kibana/actions/runs/11895247426

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 778782d

Metrics [docs]

✅ unchanged

History

• 💛 Build #250198 was flaky 913f92c

cc @abhishekbhatia1710

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.16	Backport failed because of merge conflicts
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 199734

Questions ?

Please refer to the Backport tool documentation

[abhishekbhatia1710]

💔 Some backports could not be created

Status	Branch	Result
❌	8.x	An unhandled error occurred. Please see the logs for details
✅	8.16

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 199734

Questions ?

Please refer to the Backport tool documentation

[abhishekbhatia1710]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

[muskangulati-qasource]

Hi @abhishekbhatia1710 ,

We have tested this ticket on the latest 8.16.1 BC2 and below are our observations for the same.

Please find below the testing details:

Build details

VERSION: 8.16.1
BUILD: 79724
COMMIT: c8b46e87c4d61de4fe046ce5ea0a0b68aad5acf9

Screen Recordings

• Entity risk score and entity store are successfully enabled without any errors

1.mp4

• Entity store is enabled from the dashboards without any errors

2.mp4

• The 'enable' button is enabled even when nothing is selected to be enabled

3.mp4

Please let us know if anything is missing from our end.

cc: @MadameSheema
Thanks!!

[abhishekbhatia1710]

Thanks, @muskangulati-qasource for testing this feature out and it seems to be fine.

As discussed in our Slack thread, this appears to follow the "standard" flow where the build is freshly created with version 8.16.1.

Another scenario to consider is when a user upgrades to 8.16.1 from any of the earlier 8.15.x versions.

It would be a good idea to test that flow as well. Thanks!

[muskangulati-qasource]

Hi @abhishekbhatia1710,

Thank you for sharing the details. Please find below the testing details for upgrade testing performed:

Build details

• 8.15.3:

VERSION: 8.15.3
BUILD: 76711
COMMIT: 3933429968aafb1ba31319fc38649d0f974044bf

• 8.16.1:

VERSION: 8.16.1
BUILD: 79724
COMMIT: c8b46e87c4d61de4fe046ce5ea0a0b68aad5acf9

Screenshots and Recordings for observations

• 8.15.3:
  

• Upgrade performed:
  

• 8.16.1:

• Entity risk score and entity store are successfully enabled without any errors

1.mp4

• Entity store is enabled from the dashboards without any errors

2.mp4

• The 'enable' button is enabled even when nothing is selected to be enabled

3.mp4

Thank you!

[abhishekbhatia1710]

Awesome @muskangulati-qasource , thanks a lot for testing the upgrade flow as well.

All looks to be working as expected!

[kibanamachine]

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

[kibanamachine]

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

[kibanamachine]

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.