[Logs+] Add support for logs@custom component template

[eyalkoren]

Closes #95469

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[elasticsearchmachine]

Hi @eyalkoren, I've created a changelog YAML for you.

[felixbarny]

approved by accident

[felixbarny]

For the sake of having dedicated release notes it's probably worth splitting the PR into two parts.

[eyalkoren]

This issue is currently blocked on #95527

[felixbarny]

Hey @joshdover the reason I asked for a PR review on this was to make sure we align with the Fleet naming conventions for the custom component template and to make sure this aligns with your visions for this issue:

• [Fleet] Add support for customizing integration data streams at more levels of granularity kibana#149484

Once we've added the logs@custom component template to the logs-*-* index template, we should probably do the same for Fleet-managed logs data streams as well. WDYT?

cc @kpollich

[dakrone]

LGTM

[ruflin]

We should follow up with documentation around how this "feature" should be used. Not sure where the best place is in Elasticsearch docs for this.

[eyalkoren]

@dakrone shouldn't have we increment the stack template registry version with this change, as well as the changes made through #95329?

[dakrone]

@dakrone shouldn't have we increment the stack template registry version with this change, as well as the changes made through #95329?

@eyalkoren yes, we should increment it once (per release, it's not necessary to increment it for each change) for all the changes that are being made to the template(s).

[eyalkoren]

OK, I will.
Out of curiosity, what is the reason for not using something less manual, for example- relying on the release <major>.<minor> version for these upgrade decisions? Is it because of overhead related to unnecessary upgrades?

[dakrone]

Out of curiosity, what is the reason for not using something less manual, for example- relying on the release . version for these upgrade decisions? Is it because of overhead related to unnecessary upgrades?

Yeah, we'd like to not have to reinstall the templates if there have been no changes between rolling versions. Additionally, there have been occasions where we want to make changes in the "make this change if the template doesn't already exist, or gets deleted, but don't make it if it already exists" vein.