Watchtower keeps old default environment variables after updating

[reconman]

Describe the bug

Many containers have default values for environment variables, which change for each image version. For example, pymedusa/medusa includes the commit hash and branch.

Updating the container via watchtower keeps the old commit hash value. This leads to some wrong information in the app itself.

I think the same is also true for labels.

To Reproduce

1. Build your own docker image with a default value for an environment variable
2. Create a docker container from that image
3. Build a new image version with the same tag, but change the default value of the environment variable
4. Run watchtower
5. Check the current environment variables of the updated container. The environment variable will have the old default value.

Expected behavior

If a container is updated, any environment variable default values should be replaced with the new default values.

Maybe this can be achieved by checking the default environment variables of the old image and removing them from the new docker run command.

Environment
Platform: Synology
Architecture: amd64
Docker version: 20.10.3

[github-actions]

Hi there! 👋🏼 As you're new to this repo, we'd like to suggest that you read our code of conduct as well as our contribution guidelines. Thanks a bunch for opening your first issue! 🙏

[simskij]

Describe the bug

Many containers have default values for environment variables, which change for each image version. For example, pymedusa/medusa includes the commit hash and branch.

Updating the container via watchtower keeps the old commit hash value. This leads to some wrong information in the app itself.

I think the same is also true for labels.

To Reproduce

1. Build your own docker image with a default value for an environment variable
2. Create a docker container from that image
3. Build a new image version with the same tag, but change the default value of the environment variable
4. Run watchtower
5. Check the current environment variables of the updated container. The environment variable will have the old default value.

Expected behavior

If a container is updated, any environment variable default values should be replaced with the new default values.

Maybe this can be achieved by checking the default environment variables of the old image and removing them from the new docker run command.

Environment Platform: Synology Architecture: amd64 Docker version: 20.10.3

I don't think this is achievable in a reliable way. How would watchtower know whether the value was put there intentionally or if it came from a default?

[reconman]

docker inspect --format "{{.Config.Env}}" pymedusa/medusa:develop lists the default environment variables of the image.

docker inspect --format "{{.Config.Env}}" medusa lists the current environment variables of the container called "medusa".

Remove the first list from the second and only the environment variables set by the user are left.

Tried it in bash:

image_output=$(docker inspect --format "{{range .Config.Env}}{{println .}}{{end}}" pymedusa/medusa:develop)
container_output=$(docker inspect --format "{{range .Config.Env}}{{println .}}{{end}}" medusa)
echo ${image_output[@]} ${container_output[@]} | tr ' ' '\n' | sort | uniq -u

Result:

PGID=65537
PUID=1034
TZ=Europe/Berlin

That's exactly what I set.

[reconman]

I searched a bit and found that there's already code for subtracting the environment variables present: https://github.com/containrrr/watchtower/blob/main/pkg/container/container.go#L254

But it doesn't seem to work for me.

[simskij]

docker inspect --format "{{.Config.Env}}" pymedusa/medusa:develop lists the default environment variables of the image.

docker inspect --format "{{.Config.Env}}" medusa lists the current environment variables of the container called "medusa".

Remove the first list from the second and only the environment variables set by the user are left.

Tried it in bash:

image_output=$(docker inspect --format "{{range .Config.Env}}{{println .}}{{end}}" pymedusa/medusa:develop)
container_output=$(docker inspect --format "{{range .Config.Env}}{{println .}}{{end}}" medusa)
echo ${image_output[@]} ${container_output[@]} | tr ' ' '\n' | sort | uniq -u

Result:

PGID=65537
PUID=1034
TZ=Europe/Berlin

That's exactly what I set.

My question remains however, what happen if the user sets an environment variable to the very same value as the default, and then the image maintainer decides to change that default? Should the value be kept? Removed?

[reconman]

My question remains however, what happen if the user sets an environment variable to the very same value as the default, and then the image maintainer decides to change that default? Should the value be kept? Removed?

You can't detect that case, so you have to treat it as "the value was set by the image".

[simskij]

My question remains however, what happen if the user sets an environment variable to the very same value as the default, and then the image maintainer decides to change that default? Should the value be kept? Removed?

You can't detect that case, so you have to treat it as "the value was set by the image".

Which is why I'm hesitant to add this feature. @piksel, what's your take on this?

[reconman]

Like I said, it's a bugfix for a broken feature, not a new one.

[Chri-s]

Just an idea: How about solving this with a label. For example com.centurylinklabs.watchtower.env.keep.<env-name>=true for environment variables that should not be updated?

[JohannesFleischer]

Just an idea: How about solving this with a label. For example com.centurylinklabs.watchtower.env.keep.<env-name>=true for environment variables that should not be updated?

That would be great, because for example secrets that are hold solely as an env var in memory have to be kept because the container will not work without them.
But I think that a pattern would be even better so that if there are many secrets they can be named in a way that only one line is needed to keep them all.