consul-template does not render an empty file

[lneva-fastly]

Consul Template version

consul-template v0.25.0 (99efa642)

Problem Description

If a template produces no content (totally empty file), consul-template will not render an empty file to disk. consul-template will act as if it has rendered the template but there will be no empty file on disk.

To reproduce, create a template that produces no content, a totally empty file (no whitespace). Ensure that the destination file is empty as well. Run consul-template in debug mode.

You will see a "rendering ..." line but no "rendered ..." line.

Expected behavior

Consul-template should do what I told it to do: render an empty file.

Actual behavior

If the destination file does not exist, no empty file is rendered. If the destination file exists and is not empty, it is replaced with an empty file.

Discussion

Consul-template checks to see if the destination file exists with the desired content before rendering. A nonexistent file is treated as if it were an existing, empty file, and therefore CT decides that it does not need to render the file since the content has not changed.

Why would I care about this? Well, I can think of scenarios in which I would want an empty file to be rendered as a placeholder until content is available in consul KV or somesuch.

In my case though, I accidentally had an empty entry in Vault, which I was unaware of. I was incredibly confused as to why consul-template silently failed to render this one file and spent a lot of time debugging consul-template itself, tried upgrading, etc. If CT had rendered the file as empty (as I ostensibly told it to) that would have helped me narrow in on my problem much sooner.

Proposed Solution

Always render the template to disk if the target file does not exist. Skip the byte equality check in the Render function.

[eikenb]

Hey @lneva-fastly, thanks for reporting.

I'm debating if this is an enhancement or a bug. The reason it does this is that the code that renders the file to disk only does that if the contents are different and it also doesn't error on file not existing... so it compares the returned value from a non-existing file, an empty byte slice ([]byte), with the returned value form the empty eval'd template. They are they same, both empty, and it skips it.
(Note I saw that you mentioned the equality check, I put this in here as context for others.)

The behavior is not completely explicit, but pretty obvious. Though it isn't mentioned in the commit message that added the check as it only mentioned the side effect of not running the commands.

Basically we need to decide on the importance of the current behavior. If the current behavior seems like something that is relied upon then we'd need to fix this in a backwards compatible way (enhancement) but if we don't think it is relied upon much (bug) we could just fix it with a notice in the changelog.

Thoughts?

[lneva-fastly]

Thanks! Great analysis of the challenge here.

Here's my take as a CT user. Let's say I'm using CT to populate an allow-list file for some service. I'll point the consuming service at the file that CT will generate and tell it to watch for updates. Sometimes the allow list is empty. That might be represented as an empty file or as a nonexistent file. Many consuming services, when faced with a nonexistent config file, will crash rather than treating it as an empty file.

This will be especially confusing for me as a user because the file will sometimes exist and be empty, but sometimes it will not exist. I might find that, say, when provisioning a new environment, the consuming service crashes, but existing environments work fine. It's inconsistent: an empty list is sometimes represented as an empty file but sometimes as a nonexistent file.

I'm having a really hard time coming up with a hypothetical example in which I actually want this inconsistent behavior. This seems more like an implementation detail in CT than a case that I'm going to purposefully expect and plan for in my consuming service. I think the likelihood of breaking anyone's setup is very low.

So we have a very low likelyhood of this being a breaking change, and it saves a lot of debugging by removing a confusing behavior that's fairly opaque to the user. I vote you treat it as a bug and fix it.

Then again, there's always... https://xkcd.com/1172/

[eikenb]

I'm definitely leaning towards the bug side as it really does seem like unexpected behavior and if you have a template that is built in a way that it could be empty, seems like you'd want that empty file.

[lneva-fastly]

Incredible! Thanks for the speedy turnaround!