Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

git-lfs: remove leaveDotGit=true #180079

Merged
merged 1 commit into from Jul 27, 2022
Merged

git-lfs: remove leaveDotGit=true #180079

merged 1 commit into from Jul 27, 2022

Conversation

ghost
Copy link

@ghost ghost commented Jul 4, 2022

Description of changes

Since commit 0b0119f on 2022-Jun-04, git-lfs no longer builds with --option substituters "":

error: hash mismatch in fixed-output derivation '/nix/store/2g1jwczzld8l190s2apc0ihffmil385f-source.drv':
         specified: sha256-3gVUPfZs5GViEA3D7Zm5NdxhuEz9DhwPLoQqHFdGCrI=
            got:    sha256-D7ZA04HZOG8DcejW+S91c5HjVIW+SLMzXZHylpARwrQ=
note: keeping build directory '/tmp/nix-build-source.drv-3'
error: 1 dependencies of derivation '/nix/store/jzl8ypirdrii6aik2pss84df7sds7vqr-git-lfs-3.2.0.drv' failed to build

It appears that the hash sha256-3gVUPfZs5GViEA3D7Zm5NdxhuEz9DhwPLoQqHFdGCrI= was calculated based on fetchDotGit=false, and was in cachix, so nobody noticed that the hash doesn't match the fetcher expression.

This is yet another case of an ongoing problem with nixpkgs: we have no way of noticing when a fetcher expression has the wrong hash if that incorrect hash happens to be the hash of something that is in cachix. Apparently nobody (else) is checking that nixpkgs works with --option substituters "".

Things done
  • Built on platform(s)
    • powerpc64le-linux
    • aarch64-linux
    • x86_64-linux
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

Since commit 0b0119f on 2022-Jun-04,
`git-lfs` no longer builds with `--option substituters ""`:

```
error: hash mismatch in fixed-output derivation '/nix/store/2g1jwczzld8l190s2apc0ihffmil385f-source.drv':
         specified: sha256-3gVUPfZs5GViEA3D7Zm5NdxhuEz9DhwPLoQqHFdGCrI=
            got:    sha256-D7ZA04HZOG8DcejW+S91c5HjVIW+SLMzXZHylpARwrQ=
note: keeping build directory '/tmp/nix-build-source.drv-3'
error: 1 dependencies of derivation '/nix/store/jzl8ypirdrii6aik2pss84df7sds7vqr-git-lfs-3.2.0.drv' failed to build
``

It appears that the hash
`sha256-3gVUPfZs5GViEA3D7Zm5NdxhuEz9DhwPLoQqHFdGCrI=` was calculated
based on `fetchDotGit=false`, and was in cachix, so nobody noticed
that the hash doesn't match the fetcher expression.

This is yet another case of an ongoing problem with nixpkgs: we have
no way of noticing when a fetcher expression has the wrong hash if
that incorrect hash happens to be the hash of something that is in
cachix.  Apparently nobody (else) is checking that nixpkgs works with
`--option substituters ""`.
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Jul 4, 2022
@ghost
Copy link
Author

ghost commented Jul 4, 2022

I'm starting to think that nix-build needs a flag --check-fodos which, when building an FOD (Fixed Output Derivation), will ignore any FODOs (FOD Outputs) already in the store unless the deriver of the FODO is the FOD being built. I.e. it is not enough that the FOD's output-hash matches the output-hash of something in the store -- the deriver of the thing in the store must be, exactly, the derivation being built.

And OfBorg and Hydra should run with this flag enabled at all times.

Another way of looking at this is to not consider FODs to be ca-derivations. The hash of the FODO is still verified after building the FOD, but then it is discarded.

This will mean some small amount of re-fetching, but we'll catch situations like the above immediately.

The downside is that changes to pkgs/build-support/fetch* would cause a mass-refetch of everything, which is unacceptable. So there probably needs to be an escape hatch.

Copy link
Member

@vcunat vcunat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirm this fixes the issue.

nix-build -QA git-lfs.src --check

@vcunat vcunat self-assigned this Jul 27, 2022
@vcunat vcunat merged commit 2f00df3 into NixOS:master Jul 27, 2022
@ghost ghost deleted the pr/git-lfs/fix-fetcher branch July 27, 2022 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant