[Fleet] Add support for custom Certificate Authorities, Certificate and Private keys.

[ph]

Design

• Design Custom CAs and SSL options [discuss][Fleet] Custom CAs and SSL options #72718

Ingest Manager

• Add support for custom CA for outputs [Fleet] Add support for custom Certificate Authorities, Certificate and Private keys for defined outputs. #73487
• Add support for Custom CA for integration packages [Ingest Manager] Add support for Custom CA and TLS options when configuring the integration. #73489
• Allow the checkin API to return an embedded certificate. [Ingest Manager] The Checkin API should return Certificate authorities, Certificate and Private keys as data. #73492

Integration Package

• Add support for TLS configurations option. https://github.com/elastic/package-registry/issues/611

Beats

• Add support for embedding the Certificate Authorities, Certificate and Private keys as pure data in the TLSConfig option. [Elastic Agent] Support custom certificate authorities  beats#19504
• Allow the Keystore to use embedded CA files, Certificate and Private keys. Keystore: Store PEM information directly in the keystore beats#6377

Endpoint security

• Add support for embedded CA, Certificate and private key. @ferullo

Referenced issues

• [Agent] Allow Fleet to bundle certificate for the Agent or any runnings beats. beats#14457

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[ph]

@nchaulet Added you as the owner of this feature.

[ph]

@nchaulet We might not have complete design ready that feature in this iteration, lets collaborate with @hbharding to see how we can implements it before having the complete design.

[jamiesmith]

I ran into this when trying to set up against the APM Integration Test env which uses self-signed certs and runs a fully-generated/simulated environment in Docker containers.

To get it to enroll I had to add the --insecure flag, and to get it to run I had to add a line to the generated action_store.yml file:

  outputs:
    default:
      api_key: 7_uoA.....
      hosts:
      - https://elasticsearch:9200
      type: elasticsearch
      ssl.certificate_authorities: ["/home/jamie/Projects/GitRepo/apm-integration-testing/scripts/tls/ca/ca.crt"]

I also had to add an entry to my /etc/hosts to alias elasticsearch because I was running agent outside the docker network-- so a way to do something like --insecure when running would be helpful.

[nicpenning]

Updating the action_store.yml did not seem to allow the elastic-agent to connect to ElasticSearch. I seem to see errors now at least:

I have this for the output:

    default:
      api_key: Iu...
      hosts:
      - https://192.168.4.79:9200
      - https://192.168.5.67:9200
      - https://192.168.6.114:9200
      type: elasticsearch
      ssl.certificate_authorities: ["C:\\Program Files\\elastic-agent-7.9.0-windows-x86_64\\ca.crt"]
  revision: 2

[nicpenning]

Ha, I stand corrected! I put the wrong CA in, after putting the right CA in, we are good!

[francescouk]

Ha, I stand corrected! I put the wrong CA in, after putting the right CA in, we are good!

Let me ask you something, what about the elastic endpoint? Is sending correctly? Coz following your instructions I could not make to work as we get the same certificate problem. Looks like the elastic endpoint is trying to send directly to elasticsearch without the certificate.

[nicpenning]

That is correct @francescouk , the elastic endpoint doesn't appear to connect to ElasticSearch.

[LANopop]

as mentioned in https://discuss.elastic.co/t/ingest-management-use-insecure-elasticsearch-output-managed-in-fleet-mode-for-elastic-agent/246022/10?u=lanopop please dont forget to build in some kind of fail-safe for whenever a certificate expires (this can be ca certificate or client certificate). We had this already happen to us that the ca certificate expired and therefor all the endpoints didnt connect to the manager anymore and had to manually reconfigure all of them to connect again.

[cakeben]

I am also having this or a very similar issue. I install and enroll elastic-agent using the --insecure mode but no data appears in datasets(presumably because there's a certificate error connecting to ES I can't see) is there a way to change the fleet config to not verify the ssl?

[numbfx]

I think I'm having a similar issue, I've set up Elasticsearch, Kibana, and now Elastic Agent using a self signed certificate, While getting Elastic Agent working I was getting the error that the certificate was signed by an unknown authority when I tried to add the agent to ingest manager. I modified the Elastic Agent fleet.yml to change protocol: http to protocol: https and include the line certificate_authorities: ["/etc/elastic-agent/ca.pem"] under the section ssl which allowed Elastic Agent to connect to Kibana and enroll (wasn't successfully enrolling previously). Now that it's connected, the logs are not being sent. Not sure how also encrypt/send the log data.

[sej7278]

don't set the insecure flag or edit the yaml, just put your ca.pem into your client's truststore by copying it to /etc/ssl/certs/ then run update-ca-trust or update-ca-certificates (distro-dependant).

[numbfx]

I did what you said. works, thank you. have Elasticsearch, Kibana, and Agent running encrypted all using same cert on one ubuntu machine, will try networking with windows next

[CyberAbwehr]

I have test it now on windows, it works.
You need also import the certificate as root trusted certificate.

[pkward]

I have test it now on windows, it works.
You need also import the certificate as root trusted certificate.

Could you provide the steps you took to get it working on Windows?

[CyberAbwehr]

Hi,

Here I found a good description for you.
https://www.thewindowsclub.com/manage-trusted-root-certificates-windows

If you have any questions please let me know.
Best regards

[endorama]

The same fix applied in comment above is proposed in a Discuss thread related to this same issue.

[joshdover]

This issue is significantly out of date and needs to be re-evaluated since the re-architecture of adding the Fleet Server component. It's possible the only remaining work here is #73487 to provide a UI for specifying custom CAs and including those CAs in the enroll command given to Elastic Agent. This feature also has some overlap with the support for self-signed CA fingerprints we added in 8.0, which we may be able to reuse/extend.

[jasonslater2000]

@joshdover following up on this issue, as it pertains to putting a reverse proxy in front of an Elastic Cloud cluster so that elastic-agent deployments would need to authenticate with proxy and elastic-agents would never know the true IP of the Elastic Cloud cluster behind the proxy. Is this something we resolved in 8.0? Are there instructions on our website?