Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bzlmod lock file breaks cross-platform builds #2452

Closed
cameron-martin opened this issue Feb 1, 2024 · 3 comments · Fixed by #2453
Closed

Bzlmod lock file breaks cross-platform builds #2452

cameron-martin opened this issue Feb 1, 2024 · 3 comments · Fixed by #2453
Labels
bzlmod

Comments

@cameron-martin
Copy link
Contributor

cameron-martin commented Feb 1, 2024 •
edited
Loading

If you build on one platform, it causes the bazel lock file to contain host-specific repositories, e.g.

          "rust_host_tools": {
            "bzlFile": "@@rules_rust~0.38.0//rust:repositories.bzl",
            "ruleClassName": "rust_toolchain_tools_repository",
            "attributes": {
              "name": "rules_rust~0.38.0~rust~rust_host_tools",
              "exec_triple": "x86_64-unknown-linux-gnu",
              "target_triple": "x86_64-unknown-linux-gnu",
              "allocator_library": "@rules_rust//ffi/cc/allocator_library",
              "dev_components": false,
              "edition": "2021",
              "rustfmt_version": "nightly/2023-12-28",
              "sha256s": {},
              "urls": [
                "https://static.rust-lang.org/dist/{}.tar.xz"
              ],
              "version": "1.75.0"
            }
          },

Building on a different platform than the one that generated the lock file causes the repository to be instantiated with these attributes, since instead of re-running the module extension it uses the cached attributes in the lock file. According to bazelbuild/bazel#19154, the correct way of solving this is to remove host introspection in the module extension, and instead instantiate repositories for all host platforms, and use platform resolution to select the one to use.
@cameron-martin
Copy link
Contributor Author

cameron-martin commented Feb 1, 2024 •
edited
Loading

I guess we can't use toolchain resolution during the loading phase, where we need the host tools. It looks like module_extension has some parameters that we can use to say a module extension is os or arch specific.

I think this needs splitting out into its own module extension, so that all the other toolchains don't become arch and os specific, and can be reused across platforms.

This has the side-effect of not being able to use the specified toolchain (edition, version, etc) as the host tools. However, I guess its fine to use a fixed version for these.

cameron-martin added a commit to cameron-martin/rules_rust that referenced this issue Feb 1, 2024
@cameron-martin
Make bazel lock file cross-platform
1fd0898 
Since the host tools are os and arch specific, previously bazel would cache the resolution of these in the lock file, causing the repo for the wrong OS or arch to be used when moving between machines.

Bazel has the `os_dependent` and `arch_dependent` attributes in the module extension to "key" this resolution by OS and arch. However, the `rust` module extension handles fetching of all the other toolchains as well as the host tools, and we don't really want to key these too. Therefore the host tools are moved to their own module extension. This means we can no longer match the host toolchain's version, edition, etc with the toolchain registered via `rust.toolchain` by default, and instead default to a fixed version. This can still be overriden separately in the root module. I think this is okay, because the host tools are only used for bootstrapping and I don't think there's much need to have them match.

This is tested by now checking in the MODULE.bazel.lock file of the bzlmod example, and the CI runners that test on multiple platforms will pass.

Resolves bazelbuild#2452
@cameron-martin cameron-martin mentioned this issue Feb 1, 2024
Merged
cameron-martin added a commit to cameron-martin/rules_rust that referenced this issue Feb 1, 2024
@cameron-martin
Make bazel lock file cross-platform
504e521 
Since the host tools are os and arch specific, previously bazel would cache the resolution of these in the lock file, causing the repo for the wrong OS or arch to be used when moving between machines.

Bazel has the `os_dependent` and `arch_dependent` attributes in the module extension to "key" this resolution by OS and arch. However, the `rust` module extension handles fetching of all the other toolchains as well as the host tools, and we don't really want to key these too. Therefore the host tools are moved to their own module extension. This means we can no longer match the host toolchain's version, edition, etc with the toolchain registered via `rust.toolchain` by default, and instead default to a fixed version. This can still be overriden separately in the root module. I think this is okay, because the host tools are only used for bootstrapping and I don't think there's much need to have them match.

This is tested by now checking in the MODULE.bazel.lock file of the bzlmod example, and the CI runners that test on multiple platforms will pass.

Resolves bazelbuild#2452
@cameron-martin cameron-martin mentioned this issue Feb 1, 2024
Closed
@kersson kersson mentioned this issue Mar 17, 2024
Closed
@ericmcbride
Copy link
Contributor

Im running into this as well

@kersson
Copy link
Contributor

kersson commented Apr 3, 2024

For visibility, while #2453 was meant to address this issue, I believe #2575 also addresses it (see @cameron-martin's comment), so I think this can be considered closed as of 0.41.0?

github-merge-queue bot pushed a commit that referenced this issue Apr 5, 2024
@cameron-martin
Make bazel lock file cross-platform (#2453)
fd71efb 
Since the host tools are os and arch specific, previously bazel would
cache the resolution of these in the lock file, causing the repo for the
wrong OS or arch to be used when moving between machines.

Since bazel 7.1.0, module extensions can be marked as reproducible to
exclude these from the lock file. However, the `rust` module extension
handles fetching of all the other toolchains as well as the host tools,
and we don't really want to exclude those too. Therefore the host tools
are moved to their own module extension. This means we can no longer
match the host toolchain's version, edition, etc with the toolchain
registered via `rust.toolchain` by default, and instead default to a
fixed version. This can still be overridden separately in the root
module. I think this is okay, because the host tools are only used for
bootstrapping and I don't think there's much need to have them match.

This is tested by now checking in the MODULE.bazel.lock file of the
bzlmod example, and running the bzlmod examples on multiple platforms
with `--lockfile_mode` set to `error`.

Resolves #2452
Resolves #2549
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bzlmod
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make bazel lock file cross-platform cameron-martin/rules_rust
4 participants
@cameron-martin @kersson @ericmcbride @scentini