Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkgs/stdenv/linux: add mips64el bootstrap-files #180223

Merged
merged 1 commit into from Jul 16, 2022
Merged

pkgs/stdenv/linux: add mips64el bootstrap-files #180223

merged 1 commit into from Jul 16, 2022

Conversation

ghost
Copy link

@ghost ghost commented Jul 5, 2022

Description of changes

This PR adds Hydra-generated bootstrap tarballs for mips64el-linux. I'll be following the script established in #151399, which I previously used in #168199.

Files came from this Hydra build, which used nixpkgs revision ef3fe25 to instantiate:

/nix/store/a2bvv663wjnyhq8m7v84aspsd3sgf9h6-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64.drv

and then built:

/nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64

I downloaded these files from Hydra with the following commands:

STOREPATH=aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox

I then prefetched them into /nix/store with:

$ nix store prefetch-file --executable file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox' to '/nix/store/ai30ss23914syz6j8m95arkwffbbx44k-busybox' (hash 'sha256-sTE58ofjqAqX3Xtq1g9wDxzIe6Vo//GHbicfqJoivDI=').

$ nix store prefetch-file  file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz' to '/nix/store/nr6zjrbwbxcxs6brf042zsyqllwbwj9v-bootstrap-tools.tar.xz' (hash 'sha256-tTgjeXpd2YgnfP4JvRuO0bXd2j8GqzBcd57JI3wH9x0=').

And started the bootstrap with the following command (the --arg localSystem is needed because #161159 has not merged):

nix build -f . -L hello --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabi64'

As @lovesegfault requested in #151399, here are the the sha256sums of all the on-server components for extra verification:

$ sha256sum /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/*
b53823797a5dd988277cfe09bd1b8ed1b5ddda3f06ab305c779ec9237c07f71d  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
598e05abb69b2c1a0db46585cd2131212077c0937ce2a665daf3811f059ae767  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
Things done
  • Built on platform(s)
    • mips64el-linux
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: stdenv Standard environment label Jul 5, 2022
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Jul 5, 2022
This PR adds Hydra-generated bootstrap tarballs for mips64el-linux.  I'll be following the script established in #151399, which I previously used in #168199.

Files came from [this](https://hydra.nixos.org/build/182757245) Hydra build, which used nixpkgs revision ef3fe25 to instantiate:

```
/nix/store/a2bvv663wjnyhq8m7v84aspsd3sgf9h6-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64.drv
```

and then built:

```
/nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
```

I downloaded these files from Hydra with the following commands:

```
STOREPATH=aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```

I then prefetched them into `/nix/store` with:

```
$ nix store prefetch-file --executable file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox' to '/nix/store/ai30ss23914syz6j8m95arkwffbbx44k-busybox' (hash 'sha256-sTE58ofjqAqX3Xtq1g9wDxzIe6Vo//GHbicfqJoivDI=').

$ nix store prefetch-file  file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz' to '/nix/store/nr6zjrbwbxcxs6brf042zsyqllwbwj9v-bootstrap-tools.tar.xz' (hash 'sha256-tTgjeXpd2YgnfP4JvRuO0bXd2j8GqzBcd57JI3wH9x0=').
```

And started the bootstrap with the following command (the `--arg localSystem` is needed because #161159 has not merged):

```
nix build -f . -L hello --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabi64'
```

As @lovesegfault requested in #151399, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
$ sha256sum /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/*
b53823797a5dd988277cfe09bd1b8ed1b5ddda3f06ab305c779ec9237c07f71d  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
598e05abb69b2c1a0db46585cd2131212077c0937ce2a665daf3811f059ae767  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
```
@ghost ghost mentioned this pull request Jul 5, 2022
13 tasks
@ghost
Copy link
Author

ghost commented Jul 8, 2022

I have verified that, on mips64el-linux-gnuabi64, ef3fe25 plus this PR will successfully bootstrap stdenv and build hello.

I'm currently doing nix build -f. -L nix under the same conditions to make sure nix can be built out of nixpkgs. As soon as that build completes I will undraftify this and declare victory.

@ghost ghost marked this pull request as ready for review July 11, 2022 06:45
@ghost
Copy link
Author

ghost commented Jul 11, 2022

Good to go. A few small fixes are needed for non-cross builds of the latest nix on mips, but they have nothing to do with the bootstrap-files.

@lovesegfault, please let us know when you have uploaded these files to tarballs.nixos.org.

@ghost
Copy link
Author

ghost commented Jul 15, 2022

A few small fixes are needed for non-cross builds of the latest nix on mips

Specifically: mips64el can build its own nix binary with this PR plus #181160, #181654, and #161159 using the following command:

nix-build - <<\EOF
with import <nixpkgs> { }; nix.override {

 # aws-sdk-cpp broke on mips sometime in the past three months
 withAWS = false;

 # nlohmann_json tests broke on mips sometime in the past three months
 nlohmann_json = nlohmann_json.overrideAttrs (_: { doCheck=false; });

 # to avoid having to build all of Rust
 enableDocumentation = false;

}
EOF

@lovesegfault
Copy link
Member

Hi Adam,

I'll try to get to this today, thanks for being patient :)

@lovesegfault
Copy link
Member

They should be up :)

$ aws s3 cp --recursive --acl public-read /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/ s3://nixpkgs-tarballs/stdenv-linux/mips64el/ef3fe254f3c59455386bc92fe84164f9679b92b1/
upload: ../../../../nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox to s3://nixpkgs-tarballs/stdenv-linux/mips64el/ef3fe254f3c59455386bc92fe84164f9679b92b1/busybox
upload: ../../../../nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz to s3://nixpkgs-tarballs/stdenv-linux/mips64el/ef3fe254f3c59455386bc92fe84164f9679b92b1/bootstrap-tools.tar.xz

$ aws s3 ls --recursive s3://nixpkgs-tarballs/stdenv-linux/mips64el/
2022-07-16 01:57:13   21346844 stdenv-linux/mips64el/ef3fe254f3c59455386bc92fe84164f9679b92b1/bootstrap-tools.tar.xz
2022-07-16 01:57:13     256648 stdenv-linux/mips64el/ef3fe254f3c59455386bc92fe84164f9679b92b1/busybox

$ aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv-linux/mips64el/ ./                                            
download: s3://nixpkgs-tarballs/stdenv-linux/mips64el/ef3fe254f3c59455386bc92fe84164f9679b92b1/busybox to ef3fe254f3c59455386bc92fe84164f9679b92b1/busybox
download: s3://nixpkgs-tarballs/stdenv-linux/mips64el/ef3fe254f3c59455386bc92fe84164f9679b92b1/bootstrap-tools.tar.xz to ef3fe254f3c59455386bc92fe84164f9679b92b1/bootstrap-tools.tar.xz

$ sha256sum ef3fe254f3c59455386bc92fe84164f9679b92b1/*
b53823797a5dd988277cfe09bd1b8ed1b5ddda3f06ab305c779ec9237c07f71d  ef3fe254f3c59455386bc92fe84164f9679b92b1/bootstrap-tools.tar.xz
598e05abb69b2c1a0db46585cd2131212077c0937ce2a665daf3811f059ae767  ef3fe254f3c59455386bc92fe84164f9679b92b1/busybox

@ghost
Copy link
Author

ghost commented Jul 16, 2022

Thanks @lovesegfault, I was able to verify with the commands below.

This PR is ready to merge. Nixpkgs is now capable of bootstrapping itself on a fifth architecture.

git fetch https://github.com/NixOS/nixpkgs/ pull/180223/head
git checkout FETCH_HEAD

git fetch https://github.com/NixOS/nixpkgs/ pull/181160/head && git cherry-pick FETCH_HEAD
git fetch https://github.com/NixOS/nixpkgs/ pull/181654/head && git cherry-pick FETCH_HEAD
git fetch https://github.com/NixOS/nixpkgs/ pull/161159/head && git cherry-pick FETCH_HEAD

nix build -L $(nix-instantiate - <<\EOF
with import <nixpkgs> { }; nix.override {

 # aws-sdk-cpp broke on mips sometime in the past three months
 withAWS = false;

 # nlohmann_json tests broke on mips sometime in the past three months
 nlohmann_json = nlohmann_json.overrideAttrs (_: { doCheck=false; });

 # to avoid having to build all of Rust
 enableDocumentation = false;

}
EOF
)

@lovesegfault
Copy link
Member

Wonderful! Thanks!

@lovesegfault lovesegfault merged commit 5fed5fe into NixOS:master Jul 16, 2022
@ghost ghost deleted the pr/mips64el/bootstrap-file-hashes branch July 17, 2022 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: stdenv Standard environment 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant