TLS options for integration package

[ph]

Design issue: elastic/kibana#72718

A lot of inputs in Metricbeat and Filebeat support TLS options, we should allow a developer to enable TLS options for specific inputs so we can create a custom UI on top of it.

[ruflin]

I think for now we should solve it at the package level. It means every package needs to repeat its config. What should happen on the creation side, for example in the integrations repo, is that it is defined in a reusable way. But the final package does contain the full configs. @exekias @urso Thoughts?

[ph]

@ruflin This works for most of the fields like verification_mode, but we would probably still need some logic in the EPM to propose a browse files for the Certificate/Certificate Authorities (Can be multiple) and Private key.

cc @nchaulet @jen-huang

[ph]

Note that I believe that CA,Certificate and Private keys will just become blob of bytes.

[urso]

TLS settings are more or less standardized between stack products. For package authors it would be nice to mark setting as "SSL" and have kibana provide a common wizard. Integrations would then provide the SSL settings as an object which we can just put in place. The value for the template could be a JSON object for example.

[ph]

@urso Yes, I think that will work... How many times we had typos from a user with TLS options, I presume it will be the same for integration author ;)

[exekias]

++ on having current files as something we could pass inline (blob of bytes). I guess that would improve the story when using fleet?

[exekias]

I think for now we should solve it at the package level. It means every package needs to repeat its config. What should happen on the creation side, for example in the integrations repo, is that it is defined in a reusable way. But the final package does contain the full configs. @exekias @urso Thoughts?

I'm not a big fan of introducing artifacts that will translate the package from a intermediate construct in the integrations repo into a final state. We should aim at having (almost) the same format in integrations and the registry.

For now I think it should be ok if we just copy the same settings until we have a good solution in Kibana?

[ph]

++ on having current files as something we could pass inline (blob of bytes). I guess that would improve the story when using fleet?

Yes, the trust is already established between elastic-agent <-> fleet, and adding them as part of the configuration would just simplify things. Actually the "blob" feature would be part of libbeat and I think it could be useful today. beats issue: elastic/beats#19504

[ph]

@exekias The problem are TLS options are already a reusable concept in Beats, define struct once and reuse everywhere, there is a lot of options that give more chance to get it wrong.

• Certificate authorities (common): when authenticating the server.
• Certificate: required when doing mutual authentication.
• Certificate private key: required when doing mutual authentication.
• key_passphrase: used to decode the private key.
• verification_mode (advanced): Control how the client verifies the server.
• supported protocol (advanced): define which version of TLS to use.
• cipher_suites (advanced): Define which cipher is supported, required in some lock down environment.
• curve_types (advanced): Define which ECDH curve to use.
• renegotiation (advanced): Configure TLS renegotiation
• client_authentication (advanced): Configure client authentication.
• ca_sha256 (advanced): This configures a certificate pin that you can use to ensure that a specific certificate is part of the verified chain.

[ruflin]

I'm not sure if having these predefined config blocks in Kibana is the way to go. Lets assume we have a breaking change on these config options from 7.x to 8.x. 7.x agents will be able to ship data to 8.0 and they will also receive configs. This will mean, 8.x Kibana will need to contain the configs for 7.x and 8.0 instead of just taking it from the package.

[exekias]

I thought that having the agent at least in the version of the stack was a requirement?

[ruflin]

@exekias There is an exception for the latest minor that must work with the first minor of the next master. In the past, 6.8 with 7.0. Otherwise there is no upgrade path.

[exekias]

Understood, I guess that's something we could take into account

[ruflin]

Yes, it could also be that at some stage we reintroduce some reusable components in the registry to make things easier for all contributors. Lets start with duplicating content and iterate from there.

[ruflin]

To move forward here, I suggest for now we make it part of each package and follow up on finding a more generic solution when we get to it.

[ph]

@ruflin Agree, initially they could reference a local file. (CA or key).

[mtojek]

@akshay-saraswat @ruflin Shouldn't it be part of the Integrations repository?

[akshay-saraswat]

Yes, you are right it should be part of integrations. This has nothing to do with the package registry. I'll move it to the integrations repo tomorrow if there are no objections raised until then.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[botelastic]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!