WIP: Templates

[Radvendii]

implements #95

updates, and thus depends on #87

[Radvendii]

I just realized there's an issue with this: what if we try to derive a template secret before its dependency is in place?

Should we just decode all template secrets after all normal secrets? What about root secrets vs non-root secrets?

I also think a good change to make would be to make a separate attribute: age.derived.foo or age.templates.foo instead of age.secrets.foo. That way we don't have the confusion of file and template existing on the same secret.

Will work on those later.

[Radvendii]

Okay, I implemented all that. The code is more complex than I'd hoped...

[ryantm]

@Radvendii Thank you for your interest in improving agenix. I am not convinced that this template feature needs to be tightly integrated with agenix and suggest that you implement a separate (and general) templating module project that you can use activationScript deps to make run after agenix runs.

[Radvendii]

There are three reasons I thought it best to include in agenix itself:

1. The only possible use-case I can think of for a runtime templating system is as part of a secret system. Nothing else exists at runtime but not at build time. And if you have it at build time it's much more convenient to do templating with Nix itself.
2. The files resulting from applying the templates are also secrets, which means they need the same security options (permissions, locking the directory behind they "keys" group, etc), as well as the restarting services option that I added in add age.secrets.*.{action,service} #87
3. This is also, by my (very rough) estimation, a common thing to want to do with secrets. There are many configuration files with "password" as a field, and it makes sense to separate the sensitive information from non-sensitive. This has come up while talking with a couple different people about agenix, and I don't know that many people who use agenix.

I won't push the matter any further though. If you think the added complexity is not worth the benefits for this project, I'll implement it separately. If you think it's somewhere in the range of reasonable, I'm happy to keep working at making the addition simpler.

[ryantm]

@Radvendii You're arguments are pretty compelling. I've also noticed a need for templating with secrets, but I've usually get around it by either moving the entire configuration file into the secret or using some configuration include option. I reopened the issue you made for this so we can think about it more.

[Radvendii]

Yeah, that's the way I've done it also. Unfortunately, not all service's configurations have a way to redirect passwords to another file (two that I've encountered are msmtp and spotify passwords in mopidy). I would definitely say this is a deficiency of those projects, but it does seem to be somewhat common.

And then putting the whole file in a secret does work. That's how I've been doing things. But it ends up with a lot of extra configuration being encrypted.

The problem that someone else had recently that prompted me to actually try to code up a solution was that they had the same password used in two different configuration files. Right now they have to have two different secret files. Not the end of the world, but it would be nice to see an elegant solution to all these small issues.

I'll think more about a more elegant design for this.

[Radvendii]

So another way to structure this, which covers some of the use-cases but not all, is to have a postDecryptHook which runs after the secret gets decrypted, and allows you to apply a template. Or just add a template field to secrets that has one slot for the secret itself.

Pros:

• Much simpler implementation.
• No separate "derived secrets" from "normal secrets"

Cons:

• you can't include more than one secret in a derived secret

[Radvendii]

We could

• Let any secret be either encrypted or not encrypted (if not encrypted it just gets copied into the result dir)
• Let any secret contain references to other secrets
  • @secret-name@ (pro: can be used from non-nix files)
  • /nix/store/<hash>-secret (pro: can be referenced as ${age.secrets.blah.file} from nix code)

Pros:

• No special types of secrets, just different ways for a secret to be.

Cons:

• Someone might accidentally refer to a secret from another secret
• Have to figure out how to order unpacking the secrets.
• Someone might forget to encrypt a secret, and then it silently goes in the nix store

[rcambrj]

This seems to have gone stale :(

The approach taken seems to be different to sops-nix's templating, but it looks like @Radvendii arrived at this alternative:

to have a postDecryptHook which runs after the secret gets decrypted, and allows you to apply a template

Which feels a lot closer to what sops-nix does.

IMO the biggest practical differences between sops-nix and agenix are:

• sops-nix has templating
• agenix supports nix-darwin properly

So it would be great to bring secret templating to nix-darwin!

I imagine that the approach in this PR is unwanted, given the resistance, and by now it's likely wildly outdated. I'd be happy to spend some time on a PR to introduce templating, would just appreciate some pointers on what would be an agreeable approach, where to start, and any potential gotchas.

edit: I was too quick to comment. I see that @jhillyerd's implementation as mentioned in another comment achieves this. Perhaps this could be incorporated into ryantm/agenix ?

[jhillyerd]

Feel free to use any of my implementation, it's been working fine -- the potential downside being that it's only lightly used in a homelab setting, and I haven't written any tests for it.

IMO, it would be best for the nix community if sops and agenix used very similar template syntax, but they don't share the same license so copy-pasting would probably be a bad idea.

[rcambrj]

@jhillyerd thanks for your quick response 🙂

I think there would be significant value in splitting your implementation out into its own flake, alongside a good README which would hopefully come up in search results for agenix templates. Ideally also would be linked from this repository's README. I'm sure the bug reports will write themselves if enough people use it, and the tests can come later.

What do you say? If you're not able to put in the time, I'd be happy to do so.

[rcambrj]

@jhillyerd have you considered my proposition? In the meantime, I've been using your module with great success, having copy pasted it, just would be nice to not need to copy paste :)

[jhillyerd]

Yeah, I'll take a shot at it in the next week or two

[jhillyerd]

@rcambrj it's up https://github.com/jhillyerd/agenix-template

[rcambrj]

@jhillyerd yaay! the internet thanks you ❤️