can't find target file on nightlies >= 2017-12-24

[SergioBenitez]

Beginning with rustc 1.24.0-nightly (c284f8807 2017-12-24), xargo fails to build a project with a custom target with the following error message:

$ rustup default nightly-2017-12-25
info: using existing install for 'nightly-2017-12-25-x86_64-apple-darwin'
info: default toolchain set to 'nightly-2017-12-25-x86_64-apple-darwin'

  nightly-2017-12-25-x86_64-apple-darwin unchanged - rustc 1.24.0-nightly (c284f8807 2017-12-24)

$ xargo build --target=aarch64-none-elf -vv
+ "rustc" "--print" "sysroot"
+ "rustc" "--print" "target-list"
+ "cargo" "build" "--release" "--manifest-path" "/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.hHc88THUWAuQ/Cargo.toml" "--target" "aarch64-none-elf" "-v" "-p" "core"
   Compiling core v0.0.0 (file://$home/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore)
     Running `rustc --crate-name core $home/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore/lib.rs --crate-type lib --emit=dep-info,link -C opt-level=3 -C panic=abort -C debuginfo=2 -C metadata=791964f4b0694a43 -C extra-filename=-791964f4b0694a43 --out-dir /var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.hHc88THUWAuQ/target/aarch64-none-elf/release/deps --target aarch64-none-elf -L dependency=/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.hHc88THUWAuQ/target/aarch64-none-elf/release/deps -L dependency=/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.hHc88THUWAuQ/target/release/deps --sysroot $home/.xargo -Z force-unstable-if-unmarked`
error: Error loading target specification: Could not find specification for target "aarch64-none-elf"
  |
  = help: Use `--print target-list` for a list of built-in targets

error: Could not compile `core`.

The same under older nightlies works just fine:

$ rustup default nightly-2017-12-24
info: using existing install for 'nightly-2017-12-24-x86_64-apple-darwin'
info: default toolchain set to 'nightly-2017-12-24-x86_64-apple-darwin'

  nightly-2017-12-24-x86_64-apple-darwin unchanged - rustc 1.24.0-nightly (169929308 2017-12-23)

$ xargo build --target=aarch64-none-elf -vv
+ "rustc" "--print" "sysroot"
+ "rustc" "--print" "target-list"
+ "cargo" "build" "--release" "--manifest-path" "/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.caIWJHjqBsOB/Cargo.toml" "--target" "aarch64-none-elf" "-v" "-p" "core"
   Compiling core v0.0.0 (file://$home/.rustup/toolchains/nightly-2017-12-24-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore)
     Running `rustc --crate-name core $home/.rustup/toolchains/nightly-2017-12-24-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore/lib.rs --crate-type lib --emit=dep-info,link -C opt-level=3 -C panic=abort -C debuginfo=2 -C metadata=c366a2fc929ae3de -C extra-filename=-c366a2fc929ae3de --out-dir /var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.caIWJHjqBsOB/target/aarch64-none-elf/release/deps --target aarch64-none-elf -L dependency=/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.caIWJHjqBsOB/target/aarch64-none-elf/release/deps -L dependency=/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.caIWJHjqBsOB/target/release/deps --sysroot $home/.xargo -Z force-unstable-if-unmarked`
    Finished release [optimized + debuginfo] target(s) in 25.38 secs
+ "cargo" "build" "--release" "--manifest-path" "/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.XOsFBtvV9YKQ/Cargo.toml" "--target" "aarch64-none-elf" "-v" "-p" "compiler_builtins"
   Compiling compiler_builtins v0.1.0 (file://$home/.rustup/toolchains/nightly-2017-12-24-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcompiler_builtins)
     Running `rustc --crate-name build_script_build $home/.rustup/toolchains/nightly-2017-12-24-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcompiler_builtins/build.rs --crate-type bin --emit=dep-info,link -C opt-level=3 -C debuginfo=2 --cfg 'feature="compiler-builtins"' --cfg 'feature="default"' -C metadata=6c835e3701061413 -C extra-filename=-6c835e3701061413 --out-dir /var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.XOsFBtvV9YKQ/target/release/build/compiler_builtins-6c835e3701061413 -L dependency=/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.XOsFBtvV9YKQ/target/release/deps`
     Running `/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.XOsFBtvV9YKQ/target/release/build/compiler_builtins-6c835e3701061413/build-script-build`
     Running `rustc --crate-name compiler_builtins $home/.rustup/toolchains/nightly-2017-12-24-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcompiler_builtins/src/lib.rs --crate-type lib --emit=dep-info,link -C opt-level=3 -C panic=abort -C debuginfo=2 --cfg 'feature="compiler-builtins"' --cfg 'feature="default"' -C metadata=47183b80144344d9 -C extra-filename=-47183b80144344d9 --out-dir /var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.XOsFBtvV9YKQ/target/aarch64-none-elf/release/deps --target aarch64-none-elf -L dependency=/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.XOsFBtvV9YKQ/target/aarch64-none-elf/release/deps -L dependency=/var/folders/jg/0xyyp7ys2752_121dvfp4nrm0000gn/T/xargo.XOsFBtvV9YKQ/target/release/deps --sysroot $home/.xargo -Z force-unstable-if-unmarked`
    Finished release [optimized + debuginfo] target(s) in 3.34 secs
+ "cargo" "build" "--target=aarch64-none-elf" "-vv"
   Compiling rlibc v1.0.0
     Running `rustc --crate-name rlibc $home/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/rlibc-1.0.0/src/lib.rs --crate-type lib --emit=dep-info,link -C panic=abort -C debuginfo=2 -C metadata=0c8f9a9e32ead1fa -C extra-filename=-0c8f9a9e32ead1fa --out-dir /$project/demo/target/aarch64-none-elf/debug/deps --target aarch64-none-elf -L dependency=/$project/demo/target/aarch64-none-elf/debug/deps -L dependency=/$project/demo/target/debug/deps --cap-lints warn --sysroot $home/.xargo`
   Compiling demo v0.1.0 (file:///$project/demo)
     Running `rustc --crate-name demo src/lib.rs --crate-type staticlib --emit=dep-info,link -C panic=abort -C lto -C debuginfo=2 -C metadata=f64bb00341a09168 -C extra-filename=-f64bb00341a09168 --out-dir /$project/demo/target/aarch64-none-elf/debug/deps --target aarch64-none-elf -L dependency=/$project/demo/target/aarch64-none-elf/debug/deps -L dependency=/$project/demo/target/debug/deps --extern rlibc=/$project/demo/target/aarch64-none-elf/debug/deps/librlibc-0c8f9a9e32ead1fa.rlib --sysroot $home/.xargo`
    Finished dev [unoptimized + debuginfo] target(s) in 0.88 secs

I've tried xargo clean, rm -rf ~/.xargo, wiping and reinstalling the latest nightly releases, and wiping and reinstalling xargo, to no avail.

From the looks of the logs, this looks like it might be either a bug in cargo/rustc or a change in the way that cargo/rustc finds custom target files.

[alexcrichton]

This is showing up in the compiler-builtins project as well and is almost for sure caused by rust-lang/rust#46630 which primarily pulled in rust-lang/cargo#4788, the real culprit.

In rust-lang/cargo#4788 the behavior of Cargo was tweaked to alter the cwd of the rustc process to always be at the workspace root (fixes a few bugs in Cargo and locks down the behavior for procedural macros). My guess is that when building libcore the cwd is changing to the rust-src root, not the project's cwd which has the custom target spec.

@japaric would it be possible for xargo to perhaps to rely on and update RUST_TARGET_PATH instead of relying on cwd?

[SergioBenitez]

I'm not sure if this is fully resolved. While everything seems to be working for dependencies declared in Xargo.toml, dependencies declared in Cargo.toml suffer the same fate:

$ xargo build --target=aarch64-none-elf -vv
+ "rustc" "--print" "sysroot"
+ "rustc" "--print" "target-list"
+ RUSTFLAGS="--sysroot /$home/.xargo"
+ "cargo" "build" "--target=aarch64-none-elf" "-vv"
   Compiling rlibc v1.0.0
     Running `rustc --crate-name rlibc /$home/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/rlibc-1.0.0/src/lib.rs --crate-type lib --emit=dep-info,link -C panic=abort -C debuginfo=2 -C metadata=430eb7b99c5e6c03 -C extra-filename=-430eb7b99c5e6c03 --out-dir /$project/target/aarch64-none-elf/debug/deps --target aarch64-none-elf -L dependency=/$project/target/aarch64-none-elf/debug/deps -L dependency=/$project/target/debug/deps --cap-lints warn --sysroot /$home/.xargo`
error: Error loading target specification: Could not find specification for target "aarch64-none-elf"
  |
  = help: Use `--print target-list` for a list of built-in targets

error: Could not compile `rlibc`.

[japaric]

While everything seems to be working for dependencies declared in Xargo.toml, dependencies declared in Cargo.toml suffer the same fate

@SergioBenitez What you are seeing, I would say, is (unintentional) breakage caused by rust-lang/cargo#4788 and is outside Xargo's jurisdiction. The issue appears even if you are not using Xargo:

$ cargo new --lib foo

$ cd foo

$ echo '#![no_std]' > src/lib.rs

$ cargo add heapless

$ rustc -Z unstable-options --target x86_64-unknown-linux-gnu --print target-spec-json > foo.json

$ cargo build --target foo
   Compiling untagged-option v0.1.1
error: Error loading target specification: Could not find specification for target "foo"

Due to rust-lang/cargo#4788 now all users of custom targets will have to set RUST_TARGET_PATH to the directory that contains the specification file if they have any dependency declared in Cargo.toml.

I would suggest opening a Cargo issue regarding the breakage. Xargo is correctly doing the only job it has: building and managing the sysroot -- as you said Xargo.toml works.