Inconsistencies on filebeat modules configuration load

[jsoriano]

Filebeat modules configuration load has some inconsistencies with other beats, and between different ways of configuring it. These differences make filebeat modules configuration to be error prone with some custom configurations, or when moving from one configuration method to other.

• It is not possible to configure the same module several times in the main file (under filebeat.modules).
  • If the same module is defined several times in filebeat.modules, only the last configuration seen is applied.
  • It is possible to configure the same modules several times with autodiscover, or using filebeat.config.modules. With filebeat.config.modules it works both when the definitions are in different files or in the same one.
• Not used filesets have to be explicitly disabled if the module is used. This happens with all configuration methods in filebeat but it doesn't happen in other beats, where only the listed datasets are enabled.

We might need to refactor filebeat modules configuration in order to address these inconsistencies.

[jsoriano]

It is not possible to configure the same module several times in the main file (under filebeat.modules).

To fix this we could have in filebeat.modules the same behaviour we have when defining multiple modules in a single file included from filebeat.config.modules.path.

Not used filesets have to be explicitly disabled if the module is used. This happens with all configuration methods in filebeat but it doesn't happen in other beats, where only the listed datasets are enabled.

An option to implement this in a backwards-compatible way could be to add a filesets option to filebeat modules, if not set it keeps current behaviour of loading all filesets, if it is set it only loads the listed filesets.

[jsoriano]

* Not used filesets have to be explicitly disabled if the module is used. This happens with all configuration methods in filebeat but it doesn't happen in other beats, where only the listed datasets are enabled.

Solution proposed in #17256.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[faec]

Looks like this was resolved by @adriansr in #27526.