Fix template not equals when setting does not start with "index."

[weizijun]

When create template like this:

PUT _index_template/test
{
  "template": {
    "settings": {
      "number_of_shards": 1
    }
  }
}

The setting does not start with "index.". But ClusterState will add the "index." prefix to the settings.
So the template don't equal the inner template.

In MetadataIndexTemplateServiceTests.testAddComponentTemplate, modify the setting like:

        Template template = new Template(Settings.builder().put("number_of_shards", 1).build(), new CompressedXContent("""
            {"properties":{"@timestamp":{"type":"date"}}}
            """), ComponentTemplateTests.randomAliases());

It will cause the case failed.

I improve the Settings.equals, add the normalizePrefix to the settings before compare.

I fixed two equals case in #77008 and #80864.
This PR is a new case.

[weizijun]

hi, @martijnvg, @dakrone can you help to review this PR?

[elasticmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[dakrone]

I'm not sure about this—on one hand the settings are functionally the same, however, when it comes to the actual structure of JSON and key names, they are not the same. Internally where is this template comparison causing problems? Can you explain where this pops up a bit more?

[weizijun]

I'm not sure about this—on one hand the settings are functionally the same, however, when it comes to the actual structure of JSON and key names, they are not the same. Internally where is this template comparison causing problems? Can you explain where this pops up a bit more?

It's the template equals case in our internal plugin, I found that the template I create is not equal to the template that I get from ClusterState.

The test case: MetadataIndexTemplateServiceTests.testAddComponentTemplate is the case, I add a setting, and the test will failed.

If it is not safe to modify the equals in the setting, I can add a settingsEquals in Template, like mappingsEquals.

[dakrone]

It's the template equals case in our internal plugin, I found that the template I create is not equal to the template that I get from ClusterState.

I don't think we should set this precedent, our internal tools should always be allowed to transform things as needed when they are added through the API. We have no guarantees for internal equals usage from plugins.

If it is not safe to modify the equals in the setting, I can add a settingsEquals in Template, like mappingsEquals.

I think doing something like this in your plugin would be a better solution. Changing this behavior in the settings themselves is dangerous and I think will cause problems later down the road. We don't need an additional equals method in Template because they should actually be different when the Settings differ, this is part of why the MetadataIndexTemplateService does its own normalization (changing potentially settings and mappings) prior to doing a comparison to test whether a template has been changed.

[weizijun]

I think doing something like this in your plugin would be a better solution. Changing this behavior in the settings themselves is dangerous and I think will cause problems later down the road. We don't need an additional equals method in Template because they should actually be different when the Settings differ, this is part of why the MetadataIndexTemplateService does its own normalization (changing potentially settings and mappings) prior to doing a comparison to test whether a template has been changed.

I change the logic from Settings.equals to Template.settingsEquals.
As Settings don't use only in index, it also used in cluster settings, node settings. It's not suit the change the internal equals method.
I think Template is only used for index, adding a settingsEquals method is reasonable. As ElasticSearch server add the index. prefix internal. This makes the templates different between input and output.

[weizijun]

@elasticmachine update branch

[dakrone]

I change the logic from Settings.equals to Template.settingsEquals.

I still don't think we should do this in the template, as this is only a problem for plugins and does not affect the API.

Out of curiosity, why can you not normalize/fix the index settings themselves before comparing templates in the first place?

[weizijun]

Out of curiosity, why can you not normalize/fix the index settings themselves before comparing templates in the first place?

Yeah, we can do the same logic before execute put template action. But the create template logic change the setting of user input, and cause not equals problem. If it's not fixed now, we can see if the feature is needed later.