rust-enzyme bootstrapping error

[agarret7]

I'm seeing this error when trying to build the master branch.

MRE

cargo clean && RUST_BACKTRACE=full ENZYME_OPT=1 cargo +enzyme build &> out.txt

Meta

lsb_release -a:

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.4 LTS
Release:	22.04
Codename:	jammy

rustc --version --verbose:

rustc 1.82.0-nightly (c1a6199e9 2024-07-24)
binary: rustc
commit-hash: c1a6199e9d92bb785c17a6d7ffd8b8b552f79c10
commit-date: 2024-07-24
host: x86_64-unknown-linux-gnu
release: 1.82.0-nightly
LLVM version: 18.1.7

Error output

  thread 'main' panicked at compiler/rustc_llvm/build.rs:51:59:
  REAL_LIBRARY_PATH_VAR
  stack backtrace:
     0:     0x63b38aa2e0aa - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h384fb4c2f89cf998
     1:     0x63b38aa5977f - core::fmt::write::h93b330e7c1ab4f05
     2:     0x63b38aa1e175 - std::io::Write::write_fmt::hc450c2a975ecb6f3
     3:     0x63b38aa2de85 - std::sys_common::backtrace::print::h8e52153d3a6a5289
     4:     0x63b38aa10f4a - std::panicking::default_hook::{{closure}}::hd11d0c5b522d3c9b
     5:     0x63b38aa10c88 - std::panicking::default_hook::h0881133c5cc32d3c
     6:     0x63b38aa11368 - std::panicking::rust_panic_with_hook::hdfb83b5aefcef115
     7:     0x63b38aa2e62a - std::panicking::begin_panic_handler::{{closure}}::h2bcb770be87a9078
     8:     0x63b38aa2e2d6 - std::sys_common::backtrace::__rust_end_short_backtrace::h2d0aa5c03fed18ee
     9:     0x63b38aa11090 - rust_begin_unwind
    10:     0x63b38a9a07c5 - core::panicking::panic_fmt::h9b61d05f69f05277
    11:     0x63b38a9a0f43 - core::option::expect_failed::h10200cf5aaa1fc01
    12:     0x63b38a9a7eb8 - core::option::Option<T>::expect::h777788e728444cd1
    13:     0x63b38a9a8282 - build_script_build::restore_library_path::hfa7838251d7ef495
    14:     0x63b38a9a8e5a - build_script_build::main::ha5dddaf0e70a9334
    15:     0x63b38a9a2e13 - core::ops::function::FnOnce::call_once::h5a197f0c3568305c
    16:     0x63b38a9a3466 - std::sys_common::backtrace::__rust_begin_short_backtrace::habdc557eee7aef8a
    17:     0x63b38a9a7419 - std::rt::lang_start::{{closure}}::h506956a6d11f349a
    18:     0x63b38aa10f84 - std::panicking::try::hdef3195f632fce1a
    19:     0x63b38aa12c8b - std::rt::lang_start_internal::h6c40787615c32a94
    20:     0x63b38a9a73f7 - std::rt::lang_start::h221fcd8ada351c8b
    21:     0x63b38a9aca85 - main
    22:     0x7d7e3a829d90 - __libc_start_call_main
                                 at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    23:     0x7d7e3a829e40 - __libc_start_main_impl
                                 at ./csu/../csu/libc-start.c:392:3
    24:     0x63b38a9a0f85 - _start
    25:                0x0 - <unknown>

Full Backtrace

out.txt

[ZuseZ4]

Does building with a slightly older rustc, e.g. 1.78 work?

[agarret7]

rustc --version --verbose

rustc 1.78.0 (9b00956e5 2024-04-29)
binary: rustc
commit-hash: 9b00956e56009bab2aa15d7bff10916599e3d6d6
commit-date: 2024-04-29
host: x86_64-unknown-linux-gnu
release: 1.78.0
LLVM version: 18.1.2

cargo clean && RUST_BACKTRACE=full ENZYME_OPT=1 cargo +enzyme build &> out.txt

error: failed to load manifest for workspace member `/home/austin/src/rust/library/std`
referenced by workspace at `/home/austin/src/rust/Cargo.toml`

Caused by:
  failed to parse manifest at `/home/austin/src/rust/library/std/Cargo.toml`

Caused by:
  the cargo feature `public-dependency` requires a nightly version of Cargo, but this is the `stable` channel
  See https://doc.rust-lang.org/book/appendix-07-nightly-rust.html for more information about Rust release channels.
  See https://doc.rust-lang.org/cargo/reference/unstable.html#public-dependency for more information about using this feature.

[ZuseZ4]

Hmm, can you please give a bit more context? What do you try to compile?
Also, your MRE and the rustc version are unrelated, cargo +enzyme should give you a different rustc than the one you get from calling rustc directly. You can probably verify that using which rustc and cargo +enzyme -vv and comparing the paths. Also, based on the title it looks like you might try to do bootstrapping?
I.e. compile rust-enzyme using our rust-enzyme compiler? I wouldn't recommend that, just use the official rust compiler to compile this fork, the official compiler would be more reliable.

Edit: Also, in case that you are trying to compile rustc, these are the build instructions:
https://github.com/EnzymeAD/rust?tab=readme-ov-file#the-rust-programming-language--autodiff
Cargo isn't able to build the rust compiler directly, therefore cargo build is not expected to work, independently of the autodiff work in this fork.

[agarret7]

Ah yeah I was trying to bootstrap, which wasn't even my objective. Just reflecting my ignorance -- I originally thought you ran the examples from this repo and was trying to reduce that error.

But seeing as Cargo isn't able to build the rust compiler it seems my error is in error and I can close this?

[ZuseZ4]

yep, let's close this since even when using this enzyme fork for bootstraping you'd need to use x.py build ... instead of cargo build.
Btw., I did add a bit of logic to catch the non-lto bug you ran into here, since quite a few people got bitten by it:
#144