Certificate Rotation

[tgrosinger]

Versions

etcd Version: 3.1.3
etcd Version SHA: 21fdcc6

clientv3 Version SHA: 350d0cd

Issue

I would like to start using --client-cert-auth and --peer-client-cert-auth to control access to my etcd cluster, however for security reasons I need to rotate these certificates fairly frequently.

1. Assuming I change the certificate in the file etcd is reading from, is there a way to ask etcd to reload the file?

2. If not, how bad of an idea is it to be restarting the etcd instances in a cluster frequently? For example, in a 3 node cluster having each instance restart once an hour?

3. And finally, any pointers on where I should start looking if I wanted to contribute this functionality to etcd?

Thank you.

[gyuho]

This is being discussed at etcd-operator coreos/etcd-operator#224.

/cc @hongchaodeng @colhom

[simonswine]

I am also looking for away way to reload server/client/peer certificates.

I guess this is where the server certificates are loaded:

https://github.com/coreos/etcd/blob/4fcea334adfc4a0fc73682aae3e62feec5b653b9/pkg/transport/listener.go#L160

And here the client certificates:

https://github.com/coreos/etcd/blob/408de4124b7e86d9c6e24e4b09633ecfc7f8837a/clientv3/yaml/config.go

I am suggesting reacting to SIGHUP signals and reload the certificates object / client config in that case. I would assume we need some locking around the config reload. Not too familar with etcd's codebase, but could have a try to get something PoC style implemented within the next weeks

[tgrosinger]

I have written a patch which forces etcd to load the certificate files from disk every time they are needed. It's not the final solution I am looking for, but it's a step in that direction. Let me know if anyone is interested in seeing that patch.

[heyitsanthony]

@tgrosinger sure, might as well post it as PR if the code's already working. If the only problem is it hits the fs when establishing every connection, it could possibly be gated so it only reloads on SIGHUP