TLS connection from fluent-bit to Fluentd 0.14.21 not working

[mpeltonen]

TLS connection from fluent-bit when using fluentd-generated server certificate does not work. It seems that the problem is in fluentd certificate generation. See discussion and suggested patch in fluent/fluent-bit#350.

Steps to reproduce:

Create certificate:
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365
... accept defaults, set passphrase to fluentd...

Create fluentd.conf:

<source>
  @type forward
  port 24224
  <transport tls>
     ca_cert_path /fluentd/etc/cert.pem
     ca_private_key_path /fluentd/etc/key.pem
     ca_private_key_passphrase fluentd
  </transport>
  <security>
    self_hostname localhost
    shared_key fluentd
  </security>
</source>

<match **>
  @type stdout
  <buffer>
    flush_interval 1s
  </buffer>
</match>

Run fluentd:
docker run -p 24224:24224 -v "$(pwd)":/fluentd/etc -e FLUENTD_CONF="fluentd.conf" fluent/fluentd:v0.14.21

Run fluent-bit:
docker run --network=host -v "$(pwd)":/certs fluent/fluent-bit:0.12.5 /fluent-bit/bin/fluent-bit -i random -o forward://127.0.0.1:24224 -p tls=on -p tls.verify=off -p tls.ca_file=/certs/cert.pem -p shared_key=fluentd -f 1

Applying the patch in fluent/fluent-bit#350 seems to make this work.

[edsiper]

cc: @tagomoris @repeatedly

[repeatedly]

From nahi's comment, the patch itself is good but need more update: fluent/fluent-bit#350 (comment)

In addition to it, my suggestions;

• serial must be unique. Use random?
• It also should have keyUsage and extendedKeyUsage.

cf. https://github.com/nahi/ruby-crypt/blob/master/CA.rb/gen_cert.rb#L61-L115

[repeatedly]

@mpeltonen Could you send a patch to master?

[mpeltonen]

OK, I will do a PR later today or tomorrow.

[jokester]

In case one needs to generate certificate manually (with openssl I suppose), is there any hint on command to use? 👍