Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The file '/etc' already exists in the tar with perms when maxLayers is set to 1 #140

Open
gytis-ivaskevicius opened this issue Jul 2, 2024 · 3 comments

Comments

@gytis-ivaskevicius
Copy link

When multiple derivations create the same directory and maxLayers=1 this error is thrown:

layers.json> The file '/etc' already exists in the tar with perms []types.Perm{types.Perm{Regex:"/etc/nix/nix.conf", Mode:"", Uid:0, Gid:0, Uname:"", Gname:""}, types.Perm{Regex:"/tmp", Mode:"1777", Uid:0, Gid:0, Uname:"", Gname:""}} but is overridden with perms []types.Perm(nil)
error: builder for '/nix/store/bryxlb2592i7zqavkb6j6m7rv59xvx35-layers.json.drv' failed with exit code 1;
       last 1 log lines:
       > The file '/etc' already exists in the tar with perms []types.Perm{types.Perm{Regex:"/etc/nix/nix.conf", Mode:"", Uid:0, Gid:0, Uname:"", Gname:""}, types.Perm{Regex:"/tmp", Mode:"1777", Uid:0, Gid:0, Uname:"", Gname:""}} but is overridden with perms []types.Perm(nil)
       For full logs, run 'nix log /nix/store/bryxlb2592i7zqavkb6j6m7rv59xvx35-layers.json.drv'.

If I set maxLayers=128 - its all good. It would be really nice to improve layers merging logic, if permissions are in fact different (I am not sure if that is the case) - set higher perms. And if file/dir and perms are identical - just keep one of them, ignore the error

If you'd like I could create a minimal example of this issue

@dr-housemd
Copy link

dr-housemd commented Jul 12, 2024

I've run into a similar issue where I've coreutils as one layer and tzdata as another.
This isn't an issue if I add both packages to a single layer.

 > The file '/bin' already exists in the graph with mode 'Lrwxrwxrwx' from '/nix/store/8kznh5nc5lmsqmkqlfkwsvd6ianwap2a-runtimeenv/bin' while it is added again with mode 'dr-xr-xr-x' by '/nix/store/7g55zq3h09rg244d343ljxijpi6hz6xb-devenv/bin'

Looks like this is where we're running into it.
https://github.com/nlewo/nix2container/blob/master/nix/graph.go#L85

Happy to help fix this issue if we can decide on what the fix should be..

@dr-housemd
Copy link

One way I can think of is to compute the nar hash of the directories/files. If they're the same, pick one with higher permissions.
If they're different and it is a directory,

  • we need to walk the directory
  • check if they're common files
  • if files with same name in both layers exist, compute nar hash. Only fail if there's a hash mismatch.
  • add non-overlapping files

@nlewo
Copy link
Owner

nlewo commented Jul 13, 2024

If you'd like I could create a minimal example of this issue

Yes, that would be great!

(I don't exactly understand which situations lead to this issue and an example could help me.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants