Digest immutability issue with quay.io/nginx/nginx-unprivileged image

[aydosman]

Describe the bug

When using the NGINX Unprivileged Docker image from quay.io, it appears that during releases the digests of all tags are updated. As a result even when attempting to use a specific pinned version (e.g., quay.io/nginx/nginx-unprivileged:1.27.3-alpine@sha256:3092a71e4222a73893547dad52bbcf259582a6dd40c8b616e5bf44bcca3f01ff), the digest changes, making it impossible to pull the original image.

Expected behavior
The digest for a pinned version should remain immutable and unchanged, allowing users to reliably pull specific versions of the image regardless of subsequent releases.

Additional context
We observed that other related images such as the following, behave as expected with tags and digests remaining consistent on release:

nginx-prometheus-exporter
nginx-ingress
nginx-ingress-operator

Is there a reason why the digests for the nginx-unprivileged image change across releases? Or is this an oversight? This behaviour significantly impacts our environments.

[aydosman]

@alessfg -Any updates on this? I am asking you directly because I see that you are running the GitHub Action.

[mahesh-hegde]

I am observing this as well. Furthermore, I cannot search the old digest in the "untagged images" section of the ghcr because of this workflow. So this prevents me from even referring to this digest at all (like ghcr.io/nginxinc/nginx-unprivileged:1.27.3-bookworm@sha256:3092a71e4222a73893547dad52bbcf259582a6dd40c8b616e5bf44bcca3f01ff).

So it's not ideal for reproducibility.

I think it's inevitable the SHA would change because docker cache would not always be available and some OS dependent details (like the OS packages themselves, or anything that is randomly generated) may upgrade.

However is it required that you need to release image for a given nginx version eg: 1.27.3 multiple times and overwrite the existing package?

Or can we keep older digests?

Thanks

[alessfg]

I really don't have a good solution for this issue right now. Images are rebuilt on a weekly basis to avoid potential security issues and keep the base image updated.
If I update the image, the digest is also going to change no matter what. I will note that this is also the case with our core NGINX image and probably almost every other image out there, but they just happen at a much lower rate than what you see in this image.

Re the second issue mentioned here -- once upon a time I didn't cleanup untagged images, but that lead to issues with the image repositories getting full. I guess I could explore only deleting untagged images that are "x" time old, but I don't really have a timeline for when I could start looking into it. PRs are always welcome if any of you have any ideas!