Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-building rust 1.83.0 on & for NetBSD failure #133629

Open
he32 opened this issue Nov 29, 2024 · 12 comments
Open

Cross-building rust 1.83.0 on & for NetBSD failure #133629

he32 opened this issue Nov 29, 2024 · 12 comments
Labels
A-cross Area: Cross compilation C-bug Category: This is a bug. O-netbsd Operating system: NetBSD T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@he32
Copy link
Contributor

he32 commented Nov 29, 2024

I am doing my best to try to keep the various NetBSD targets for rust working.
As part of that process, I am cross-building binaries from x86_64 to most of the
other targets, and carry out testing of the results, for some of the targets this involves
self-hosting the rust compiler.

I have two new observations for rust 1.83.0 (1.82.0 works fine):

  1. A new seemingly-non-fatal error message which is difficult to get a grasp on has started appearing in the build log:
   1.288127806s  INFO prepare_target{force=false package_id=sysroot v0.0.0 (/usr
/pkgsrc/wip/rust183/work/rustc-1.83.0-src/library/sysroot) target="sysroot"}: ca
rgo::core::compiler::fingerprint: fingerprint error for sysroot v0.0.0 (/usr/pkg
src/wip/rust183/work/rustc-1.83.0-src/library/sysroot)/Build/TargetInner { name_inferred: true, ..: lib_target("sysroot", ["lib"], "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/library/sysroot/src/lib.rs", Edition2021) }
   1.288196443s  INFO prepare_target{force=false package_id=sysroot v0.0.0 (/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/library/sysroot) target="sysroot"}: cargo::core::compiler::fingerprint:     err: failed to read `/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-40f70deaa15a4468/lib-sysroot`

Caused by:
    No such file or directory (os error 2)

There are many such instances. Testing for the existence of the quoted file name afterwards does not reproduce the problem(!)
Looking for the file afterwards shows that it exists:

$ find work -type f -name lib-sysroot
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1-std/armv7-unknown-netbsd-eabihf/release/.fingerprint/sysroot-76159fffe79d6c08/lib-sysroot
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-904060a630664f36/lib-sysroot
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-40f70deaa15a4468/lib-sysroot
$ 
$ cat /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-40f70deaa15a4468/lib-sysroot; echo
07b22e914d1cad45
$ 
  1. While natively building the rust compiler on x86_64 works fine, and the resulting compiler also works, I get a new (to me) problem while cross- building the rust compiler, in this first instance for our armv7hf target:
using sysroot /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1
running: "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) (created at src/core/builder.rs:1704:33, executed at src/core/builder.rs:1710:26)

Command "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) did not execute successfully.
Expected success, got exit status: 1
Created at: src/core/builder.rs:1704:33
Executed at: src/core/builder.rs:1710:26

STDOUT ----


STDERR ----
/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc: Shared object "librustc_driver-299f82a8112084d1.so" not found

The asked-for file exists in the build tree, ref.:

$ find work -name librustc_driver-299f82a8112084d1.so -type f
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/lib/librustc_driver-299f82a8112084d1.so
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-sysroot/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_driver-299f82a8112084d1.so
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps/librustc_driver-299f82a8112084d1.so
$ 

but apparently, this particular rustc instance isn't being pointed in the right direction.
It is also somewhat unclear to me whether this second problem is related to the first issue listed above.

My initial build attempt was done with -j 32 (I have the resources for it), but repeating it with -j 1 to root out whether this is a bug related to parallelism of the build reveals that the answer to that question is "no".

I expected to see this happen: Cross-building the rust compiler ought to succeed.

Instead, this happened:

The build errored out with

    Finished `release` profile [optimized] target(s) in 2m 45s
using sysroot /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1
running: "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) (created at src/core/builder.rs:1704:33, executed at src/core/builder.rs:1710:26)

Command "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) did not execute successfully.
Expected success, got exit status: 1
Created at: src/core/builder.rs:1704:33
Executed at: src/core/builder.rs:1710:26

STDOUT ----


STDERR ----
/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc: Shared object "librustc_driver-299f82a8112084d1.so" not found

Traceback (most recent call last):
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/./x.py", line 50, in <module>
    bootstrap.main()
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/src/bootstrap/bootstrap.py", line 1227, in main
    bootstrap(args)
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/src/bootstrap/bootstrap.py", line 1203, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/src/bootstrap/bootstrap.py", line 202, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/bootstrap/debug/bootstrap -v dist -j 1
*** Error code 1

Stop.
make[1]: stopped in /usr/pkgsrc/wip/rust183
*** Error code 1

Meta

This build is obviously using the previous version, rust 1.82.0:

rustc --version --verbose:

$ work/rust-1.82.0-x86_64-unknown-netbsd/rustc/bin/rustc --version --verbose
rustc 1.82.0 (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: x86_64-unknown-netbsd
release: 1.82.0
LLVM version: 19.1.1
$ 
Backtrace

Backtraces and details from the build provided above.
If wanted, I can provide a copy of the entire build log, hints for how to do so appreciated.

@he32 he32 added the C-bug Category: This is a bug. label Nov 29, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Nov 29, 2024
@jieyouxu jieyouxu added A-cross Area: Cross compilation T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) O-netbsd Operating system: NetBSD and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Nov 30, 2024
@he32
Copy link
Contributor Author

he32 commented Nov 30, 2024

Update: The first issue mentioned above is not the same as the second; the first is visible also in a "native build".

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this issue Nov 30, 2024
Pkgsrc changes compared to rust182:
 * Remove patches related to rust-lang/rust#130110,
   which is now integrated upstream.
 * Remove patch to vendor/cc-1.0.79, now integrated in the current
   vendored cc crate.
 * Checksum updates.

TODO:
 * Cross-compilation fails ref.
   rust-lang/rust#133629

Upstream changes:

Version 1.83.0 (2024-11-28)
==========================

Language
--------
- [Stabilize `&mut`, `*mut`, `&Cell`, and `*const Cell` in const.]
  (rust-lang/rust#129195)
- [Allow creating references to statics in `const` initializers.]
  (rust-lang/rust#129759)
- [Implement raw lifetimes and labels (`'r#ident`).]
  (rust-lang/rust#126452)
- [Define behavior when atomic and non-atomic reads race.]
  (rust-lang/rust#128778)
- [Non-exhaustive structs may now be empty.]
  (rust-lang/rust#128934)
- [Disallow implicit coercions from places of type `!`]
  (rust-lang/rust#129392)
- [`const extern` functions can now be defined for other calling conventions.]
  (rust-lang/rust#129753)
- [Stabilize `expr_2021` macro fragment specifier in all editions.]
  (rust-lang/rust#129972)
- [The `non_local_definitions` lint now fires on less code and
  warns by default.]
  (rust-lang/rust#127117)

Compiler
--------
- [Deprecate unsound `-Csoft-float` flag.]
  (rust-lang/rust#129897)
- Add many new tier 3 targets:
    - [`aarch64_unknown_nto_qnx700`]
      (rust-lang/rust#127897)
    - [`arm64e-apple-tvos`]
      (rust-lang/rust#130614)
    - [`armv7-rtems-eabihf`]
      (rust-lang/rust#127021)
    - [`loongarch64-unknown-linux-ohos`]
      (rust-lang/rust#130750)
    - [`riscv32-wrs-vxworks` and `riscv64-wrs-vxworks`]
      (rust-lang/rust#130549)
    - [`riscv32{e|em|emc}-unknown-none-elf`]
      (rust-lang/rust#130555)
    - [`x86_64-unknown-hurd-gnu`]
      (rust-lang/rust#128345)
    - [`x86_64-unknown-trusty`]
      (rust-lang/rust#130453)

Refer to Rust's [platform support page][platform-support-doc]
for more information on Rust's tiered platform support.

Libraries
---------
- [Implement `PartialEq` for `ExitCode`.]
  (rust-lang/rust#127633)
- [Document that `catch_unwind` can deal with foreign exceptions
  without UB, although the exact behavior is unspecified.]
  (rust-lang/rust#128321)
- [Implement `Default` for `HashMap`/`HashSet` iterators that don't
  already have it.]
  (rust-lang/rust#128711)
- [Bump Unicode to version 16.0.0.]
  (rust-lang/rust#130183)
- [Change documentation of `ptr::add`/`sub` to not claim equivalence
  with `offset`.]
  (rust-lang/rust#130229).

Stabilized APIs
---------------

- [`BufRead::skip_until`]
  (https://doc.rust-lang.org/stable/std/io/trait.BufRead.html#method.skip_until)
- [`ControlFlow::break_value`]
  (https://doc.rust-lang.org/stable/core/ops/enum.ControlFlow.html#method.break_value)
- [`ControlFlow::continue_value`]
  (https://doc.rust-lang.org/stable/core/ops/enum.ControlFlow.html#method.continue_value)
- [`ControlFlow::map_break`]
  (https://doc.rust-lang.org/stable/core/ops/enum.ControlFlow.html#method.map_break)
- [`ControlFlow::map_continue`]
  (https://doc.rust-lang.org/stable/core/ops/enum.ControlFlow.html#method.map_continue)
- [`DebugList::finish_non_exhaustive`]
  (https://doc.rust-lang.org/stable/core/fmt/struct.DebugList.html#method.finish_non_exhaustive)
- [`DebugMap::finish_non_exhaustive`]
  (https://doc.rust-lang.org/stable/core/fmt/struct.DebugMap.html#method.finish_non_exhaustive)
- [`DebugSet::finish_non_exhaustive`]
  (https://doc.rust-lang.org/stable/core/fmt/struct.DebugSet.html#method.finish_non_exhaustive)
- [`DebugTuple::finish_non_exhaustive`]
  (https://doc.rust-lang.org/stable/core/fmt/struct.DebugTuple.html#method.finish_non_exhaustive)
- [`ErrorKind::ArgumentListTooLong`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.ArgumentListTooLong)
- [`ErrorKind::Deadlock`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.Deadlock)
- [`ErrorKind::DirectoryNotEmpty`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.DirectoryNotEmpty)
- [`ErrorKind::ExecutableFileBusy`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.ExecutableFileBusy)
- [`ErrorKind::FileTooLarge`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.FileTooLarge)
- [`ErrorKind::HostUnreachable`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.HostUnreachable)
- [`ErrorKind::IsADirectory`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.IsADirectory)
- [`ErrorKind::NetworkDown`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.NetworkDown)
- [`ErrorKind::NetworkUnreachable`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.NetworkUnreachable)
- [`ErrorKind::NotADirectory`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.NotADirectory)
- [`ErrorKind::NotSeekable`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.NotSeekable)
- [`ErrorKind::ReadOnlyFilesystem`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.ReadOnlyFilesystem)
- [`ErrorKind::ResourceBusy`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.ResourceBusy)
- [`ErrorKind::StaleNetworkFileHandle`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.StaleNetworkFileHandle)
- [`ErrorKind::StorageFull`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.StorageFull)
- [`ErrorKind::TooManyLinks`]
  (https://doc.rust-lang.org/stable/std/io/enum.ErrorKind.html#variant.TooManyLinks)
- [`Option::get_or_insert_default`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.get_or_insert_default)
- [`Waker::data`]
  (https://doc.rust-lang.org/stable/core/task/struct.Waker.html#method.data)
- [`Waker::new`]
  (https://doc.rust-lang.org/stable/core/task/struct.Waker.html#method.new)
- [`Waker::vtable`]
  (https://doc.rust-lang.org/stable/core/task/struct.Waker.html#method.vtable)
- [`char::MIN`]
  (https://doc.rust-lang.org/stable/core/primitive.char.html#associatedconstant.MIN)
- [`hash_map::Entry::insert_entry`]
  (https://doc.rust-lang.org/stable/std/collections/hash_map/enum.Entry.html#method.insert_entry)
- [`hash_map::VacantEntry::insert_entry`]
  (https://doc.rust-lang.org/stable/std/collections/hash_map/struct.VacantEntry.html#method.insert_entry)

These APIs are now stable in const contexts:

- [`Cell::into_inner`]
  (https://doc.rust-lang.org/stable/core/cell/struct.Cell.html#method.into_inner)
- [`Duration::as_secs_f32`]
  (https://doc.rust-lang.org/stable/core/time/struct.Duration.html#method.as_secs_f32)
- [`Duration::as_secs_f64`]
  (https://doc.rust-lang.org/stable/core/time/struct.Duration.html#method.as_secs_f64)
- [`Duration::div_duration_f32`]
  (https://doc.rust-lang.org/stable/core/time/struct.Duration.html#method.div_duration_f32)
- [`Duration::div_duration_f64`]
  (https://doc.rust-lang.org/stable/core/time/struct.Duration.html#method.div_duration_f64)
- [`MaybeUninit::as_mut_ptr`]
  (https://doc.rust-lang.org/stable/core/mem/union.MaybeUninit.html#method.as_mut_ptr)
- [`NonNull::as_mut`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.as_mut)
- [`NonNull::copy_from`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.copy_from)
- [`NonNull::copy_from_nonoverlapping`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.copy_from_nonoverlapping)
- [`NonNull::copy_to`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.copy_to)
- [`NonNull::copy_to_nonoverlapping`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.copy_to_nonoverlapping)
- [`NonNull::slice_from_raw_parts`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.slice_from_raw_parts)
- [`NonNull::write`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.write)
- [`NonNull::write_bytes`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.write_bytes)
- [`NonNull::write_unaligned`]
  (https://doc.rust-lang.org/stable/core/ptr/struct.NonNull.html#method.write_unaligned)
- [`OnceCell::into_inner`]
  (https://doc.rust-lang.org/stable/core/cell/struct.OnceCell.html#method.into_inner)
- [`Option::as_mut`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.as_mut)
- [`Option::expect`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.expect)
- [`Option::replace`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.replace)
- [`Option::take`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.take)
- [`Option::unwrap`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.unwrap)
- [`Option::unwrap_unchecked`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.unwrap_unchecked)
- [`Option::<&_>::copied`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.copied)
- [`Option::<&mut _>::copied`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.copied-1)
- [`Option::<Option<_>>::flatten`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.flatten)
- [`Option::<Result<_, _>>::transpose`]
  (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.transpose)
- [`RefCell::into_inner`]
  (https://doc.rust-lang.org/stable/core/cell/struct.RefCell.html#method.into_inner)
- [`Result::as_mut`]
  (https://doc.rust-lang.org/stable/core/result/enum.Result.html#method.as_mut)
- [`Result::<&_, _>::copied`]
  (https://doc.rust-lang.org/stable/core/result/enum.Result.html#method.copied)
- [`Result::<&mut _, _>::copied`]
  (https://doc.rust-lang.org/stable/core/result/enum.Result.html#method.copied-1)
- [`Result::<Option<_>, _>::transpose`]
  (https://doc.rust-lang.org/stable/core/result/enum.Result.html#method.transpose)
- [`UnsafeCell::get_mut`]
  (https://doc.rust-lang.org/stable/core/cell/struct.UnsafeCell.html#method.get_mut)
- [`UnsafeCell::into_inner`]
  (https://doc.rust-lang.org/stable/core/cell/struct.UnsafeCell.html#method.into_inner)
- [`array::from_mut`]
  (https://doc.rust-lang.org/stable/core/array/fn.from_mut.html)
- [`char::encode_utf8`]
  (https://doc.rust-lang.org/stable/core/primitive.char.html#method.encode_utf8)
- [`{float}::classify`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.classify)
- [`{float}::is_finite`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.is_finite)
- [`{float}::is_infinite`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.is_infinite)
- [`{float}::is_nan`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.is_nan)
- [`{float}::is_normal`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.is_normal)
- [`{float}::is_sign_negative`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.is_sign_negative)
- [`{float}::is_sign_positive`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.is_sign_positive)
- [`{float}::is_subnormal`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.is_subnormal)
- [`{float}::from_bits`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.from_bits)
- [`{float}::from_be_bytes`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.from_be_bytes)
- [`{float}::from_le_bytes`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.from_le_bytes)
- [`{float}::from_ne_bytes`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.from_ne_bytes)
- [`{float}::to_bits`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.to_bits)
- [`{float}::to_be_bytes`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.to_be_bytes)
- [`{float}::to_le_bytes`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.to_le_bytes)
- [`{float}::to_ne_bytes`]
  (https://doc.rust-lang.org/stable/core/primitive.f64.html#method.to_ne_bytes)
- [`mem::replace`]
  (https://doc.rust-lang.org/stable/core/mem/fn.replace.html)
- [`ptr::replace`]
  (https://doc.rust-lang.org/stable/core/ptr/fn.replace.html)
- [`ptr::slice_from_raw_parts_mut`]
  (https://doc.rust-lang.org/stable/core/ptr/fn.slice_from_raw_parts_mut.html)
- [`ptr::write`]
  (https://doc.rust-lang.org/stable/core/ptr/fn.write.html)
- [`ptr::write_unaligned`]
  (https://doc.rust-lang.org/stable/core/ptr/fn.write_unaligned.html)
- [`<*const _>::copy_to`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.copy_to)
- [`<*const _>::copy_to_nonoverlapping`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.copy_to_nonoverlapping)
- [`<*mut _>::copy_from`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.copy_from)
- [`<*mut _>::copy_from_nonoverlapping`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.copy_from_nonoverlapping)
- [`<*mut _>::copy_to`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.copy_to-1)
- [`<*mut _>::copy_to_nonoverlapping`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.copy_to_nonoverlapping-1)
- [`<*mut _>::write`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.write)
- [`<*mut _>::write_bytes`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.write_bytes)
- [`<*mut _>::write_unaligned`]
  (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.write_unaligned)
- [`slice::from_mut`]
  (https://doc.rust-lang.org/stable/core/slice/fn.from_mut.html)
- [`slice::from_raw_parts_mut`]
  (https://doc.rust-lang.org/stable/core/slice/fn.from_raw_parts_mut.html)
- [`<[_]>::first_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.first_mut)
- [`<[_]>::last_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.last_mut)
- [`<[_]>::first_chunk_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.first_chunk_mut)
- [`<[_]>::last_chunk_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.last_chunk_mut)
- [`<[_]>::split_at_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_at_mut)
- [`<[_]>::split_at_mut_checked`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_at_mut_checked)
- [`<[_]>::split_at_mut_unchecked`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_at_mut_unchecked)
- [`<[_]>::split_first_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_first_mut)
- [`<[_]>::split_last_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_last_mut)
- [`<[_]>::split_first_chunk_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_first_chunk_mut)
- [`<[_]>::split_last_chunk_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.slice.html#method.split_last_chunk_mut)
- [`str::as_bytes_mut`]
  (https://doc.rust-lang.org/stable/core/primitive.str.html#method.as_bytes_mut)
- [`str::as_mut_ptr`]
  (https://doc.rust-lang.org/stable/core/primitive.str.html#method.as_mut_ptr)
- [`str::from_utf8_unchecked_mut`]
  (https://doc.rust-lang.org/stable/core/str/fn.from_utf8_unchecked_mut.html)

Cargo
-----
- [Introduced a new `CARGO_MANIFEST_PATH` environment variable,
  similar to `CARGO_MANIFEST_DIR` but pointing directly to the manifest
  file.]
  (rust-lang/cargo#14404)
- [Added `package.autolib` to the manifest, allowing `[lib]`
  auto-discovery to be disabled.]
  (rust-lang/cargo#14591)
- [Declare support level for each crate in Cargo's Charter / crate docs.]
  (rust-lang/cargo#14600)
- [Declare new Intentional Artifacts as 'small' changes.]
  (rust-lang/cargo#14599)

Rustdoc
-------

- [The sidebar / hamburger menu table of contents now includes the
  `# headers` from the main item's doc comment]
  (rust-lang/rust#120736). This is similar
  to a third-party feature provided by the rustdoc-search-enhancements
  browser extension.

Compatibility Notes
-------------------
- [Warn against function pointers using unsupported ABI strings.]
  (rust-lang/rust#128784)
- [Check well-formedness of the source type's signature in fn
  pointer casts.]
  (rust-lang/rust#129021) This partly closes
  a soundness hole that comes when casting a function item to
  function pointer
- [Use equality instead of subtyping when resolving type dependent paths.]
  (rust-lang/rust#129073)
- Linking on macOS now correctly includes Rust's default deployment
  target. Due to a linker bug, you might have to pass
  `MACOSX_DEPLOYMENT_TARGET` or fix your `#[link]` attributes to
  point to the correct frameworks. See
  <rust-lang/rust#129369>.
- [Rust will now correctly raise an error for `repr(Rust)` written
  on non-`struct`/`enum`/`union` items, since it previous did not
  have any effect.]
  (rust-lang/rust#129422)
- The future incompatibility lint `deprecated_cfg_attr_crate_type_name`
  [has been made into a hard error]
  (rust-lang/rust#129670). It was used to
  deny usage of `#![crate_type]` and `#![crate_name]` attributes in
  `#![cfg_attr]`, which required a hack in the compiler to be able
  to change the used crate type and crate name after cfg expansion.
  Users can use `--crate-type` instead of `#![cfg_attr(..., crate_type
  = "...")]` and `--crate-name` instead of `#![cfg_attr(...,
  crate_name = "...")]` when running `rustc`/`cargo rustc` on the
  command line.  Use of those two attributes outside of `#![cfg_attr]`
  continue to be fully supported.

- Until now, paths into the sysroot were always prefixed with
  `/rustc/$hash` in diagnostics, codegen, backtrace, e.g.
    ```
    thread 'main' panicked at 'hello world', map-panic.rs:2:50
    stack backtrace:
       0: std::panicking::begin_panic
                 at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:616:12
       1: map_panic::main::{{closure}}
                 at ./map-panic.rs:2:50
       2: core::option::Option<T>::map
                 at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/option.rs:929:29
       3: map_panic::main
                 at ./map-panic.rs:2:30
       4: core::ops::function::FnOnce::call_once
                 at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/ops/function.rs:248:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    ```
  [RFC 3127 said]
  (https://rust-lang.github.io/rfcs/3127-trim-paths.html#changing-handling-of-sysroot-path-in-rustc)
  > We want to change this behaviour such that, when `rust-src`
    source files can be discovered, the virtual path is discarded
    and therefore the local path will be embedded, unless there is
    a `--remap-path-prefix` that causes this local path to be remapped
    in the usual way.

    [#129687](rust-lang/rust#129687)
    implements this behaviour, when `rust-src` is present at compile
    time, `rustc` replaces `/rustc/$hash` with a real path into
    the local `rust-src` component with best effort.  To sanitize
    this, users must explicitly supply `--remap-path-prefix=<path
    to rust-src>=foo` or not have the `rust-src` component installed.
- The allow-by-default `missing_docs` lint used to disable itself
  when invoked through `rustc --test`/`cargo test`, resulting in
  `#[expect(missing_docs)]` emitting false positives due to the
  expectation being wrongly unfulfilled. This behavior [has now been
  removed]
  (rust-lang/rust#130025), which allows
  `#[expect(missing_docs)]` to be fulfilled in all scenarios, but
  will also report new `missing_docs` diagnostics for publicly
  reachable `#[cfg(test)]` items, [integration test]
  (https://doc.rust-lang.org/cargo/reference/cargo-targets.html#integration-tests)
  crate-level documentation, and publicly reachable items in
  integration tests.
- [The `armv8r-none-eabihf` target now uses the Armv8-R required
  set of floating-point features.]
  (rust-lang/rust#130295)
- [Fix a soundness bug where rustc wouldn't detect unconstrained
  higher-ranked lifetimes in a `dyn Trait`'s associated types that
  occur due to supertraits.]
  (rust-lang/rust#130367)
- [Update the minimum external LLVM version to 18.]
  (rust-lang/rust#130487)
- [Remove `aarch64-fuchsia` and `x86_64-fuchsia` target aliases in
  favor of `aarch64-unknown-fuchsia` and `x86_64-unknown-fuchsia`
  respectively.]
  (rust-lang/rust#130657)
- [The ABI-level exception class of a Rust panic is now encoded
  with native-endian bytes, so it is legible in hex dumps.]
  (rust-lang/rust#130897)
- [Visual Studio 2013 is no longer supported for MSVC targets.]
  (rust-lang/rust#131070)
- [The sysroot no longer contains the `std` dynamic library in its
  top-level `lib/` dir.]
  (rust-lang/rust#131188)
@he32
Copy link
Contributor Author

he32 commented Jan 4, 2025

Well, it's about time to post some more information about this issue. This build is done within the pkgsrc framework, and as such, the C and C++ compilers are "wrapped", and certain arguments are being suppressed. Among them is "use of non-absolute paths in RPATH arguments", and, yes, this means that arguments like -Wl,-rpath,$ORIGIN/../lib is being suppressed. It is evident that 1.82 did not critically depend on that feature, but 1.83 does. I have tried looking at the diff between 1.82 and 1.83 to see if a setting of LD_LIBRARY_PATH has been removed, but have come up empty (and the diff is HUGE...).

I have also verified that if I run patchelf on the stage1/bin/rustc and add the above path, and restart the build, the cross-build completes. However, I have not found where I might insert such a workaround in the rust build system. I have found the workarounds for the "NixOS" OS/variant in bootstrap.py, but that is evidently to patch a bootstrap compiler, and not the stage1 built compiler, so extending that to set rpath on NetBSD too probably will not achieve much in terms of mending this particular problem. So ... can anyone else please point me in the right direction of where I might tweak RPATH on the stage1 rustc before it is being used?

@jieyouxu
Copy link
Member

jieyouxu commented Jan 4, 2025

@he32 maybe you are looking for

// Use relative paths to the libraries. Binaries can be moved
// as long as they maintain the relative relationship to the
// crates they depend on.
let rpaths = get_rpaths_relative_to_output(config);

@jieyouxu
Copy link
Member

jieyouxu commented Jan 4, 2025

Could you elaborate what you mean by

This build is done within the pkgsrc framework, and as such, the C and C++ compilers are "wrapped", and certain arguments are being suppressed.

Among them is "use of non-absolute paths in RPATH arguments", and, yes, this means that arguments like -Wl,-rpath,$ORIGIN/../lib is being suppressed.

Like, randomly dropping arguments seem like it's prone to produce random failures, no? Or why does non-absolute paths in RPATH arguments get dropped?

@jyn514
Copy link
Member

jyn514 commented Jan 4, 2025

A new seemingly-non-fatal error message which is difficult to get a grasp on has started appearing in the build log:

this is cargo telling you why it is rebuilding instead of using an existing cache. it is not indicative of an error. you presumably have CARGO_LOG set somewhere in your environment; you can silence this message by unsetting that env variable.

@he32
Copy link
Contributor Author

he32 commented Jan 4, 2025

Could you elaborate what you mean by

This build is done within the pkgsrc framework, and as such, the C and C++ compilers are "wrapped", and certain arguments are being suppressed.
Among them is "use of non-absolute paths in RPATH arguments", and, yes, this means that arguments like -Wl,-rpath,$ORIGIN/../lib is being suppressed.

Like, randomly dropping arguments seem like it's prone to produce random failures, no? Or why does non-absolute paths in RPATH arguments get dropped?

I'll try, but please understand that I'm sitting in the squeeze between trying to maintain our rust port inside pkgsrc and the pkgsrc development and pkgsrc infrastructure maintenance in general. I asked on [email protected] about this issue, and the start of that thread is at https://mail-index.netbsd.org/tech-pkg/2025/01/01/msg030311.html

As I understand it, the main issue is that pkgsrc tries to ensure that cross-package shared library dependencies are maintained when doing an upgrade, so pkgsrc needs to track which shared libraries a given package uses and is therefore depending on. So it's not just "randomly dropping arguments". The question has come up in the thread "why can't one just use LD_LIBRARY_PATH?", which is a reasonable question.

However, that aside, as I said above, I have found a workaround, but need to find the proper place to implant this in the rust build. In particular, I'm looking for where e.g. this phase of the build finishes (when cross-building from amd64 to armv7):

Building stage1 library artifacts (x86_64-unknown-netbsd -> armv7-unknown-netbsd-eabihf)

and before this step is started:

Building stage1 library artifacts (x86_64-unknown-netbsd)

because that is apparently the first time the "stage1" rust compiler is used. Optimally, the path of the about-to-be-used stage1 rustc compiler should be reasonably easily available at that point. I have found rustc-1.83.0-src/src/bootstrap/src/core/build_steps/compile.rs which looks like a candidate, but have yet to decipher where the appropriate place would be.

@he32
Copy link
Contributor Author

he32 commented Jan 4, 2025

A new seemingly-non-fatal error message which is difficult to get a grasp on has started appearing in the build log:

this is cargo telling you why it is rebuilding instead of using an existing cache. it is not indicative of an error. you presumably have CARGO_LOG set somewhere in your environment; you can silence this message by unsetting that env variable.

OK. But as far as I can tell, this is "new noise" which wasn't there in 1.82.0, and looks remarkably like an error message. But I'm reasonably OK with just ignoring this part of this issue and rather move on with the actual build blocker part of this issue.

@thesamesam
Copy link

FWIW, we're seeing the same issue in our cross-compiles, as are folks from Void. We at least aren't doing any sort of flag mangling and I don't think they are either.

@onur-ozkan
Copy link
Member

I will look into the problem this week.

@jieyouxu
Copy link
Member

jieyouxu commented Jan 5, 2025

@he32 in the meantime, you can see the order of Steps being run via --dry-run without having to do the actual full build.

@kleisauke
Copy link
Contributor

A new seemingly-non-fatal error message which is difficult to get a grasp on has started appearing in the build log:

this is cargo telling you why it is rebuilding instead of using an existing cache. it is not indicative of an error. you presumably have CARGO_LOG set somewhere in your environment; you can silence this message by unsetting that env variable.

OK. But as far as I can tell, this is "new noise" which wasn't there in 1.82.0, and looks remarkably like an error message.

This was introduced in commit 874d55c. To hide these fingerprint logs, you can remove the -v option from x.py invocations.

@onur-ozkan
Copy link
Member

because that is apparently the first time the "stage1" rust compiler is used. Optimally, the path of the about-to-be-used stage1 rustc compiler should be reasonably easily available at that point. I have found rustc-1.83.0-src/src/bootstrap/src/core/build_steps/compile.rs which looks like a candidate, but have yet to decipher where the appropriate place would be.

You are looking for

let rustc = out_dir.join(exe("rustc-main", host));
let bindir = sysroot.join("bin");
t!(fs::create_dir_all(bindir));
let compiler = builder.rustc(target_compiler);
builder.copy_link(&rustc, &compiler);

part of the build where bootstrap copies release/rustc-main into stage1/bin, meaning, you can do whatever is needed before stage1 compiler is being used/touched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cross Area: Cross compilation C-bug Category: This is a bug. O-netbsd Operating system: NetBSD T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

No branches or pull requests

7 participants