cargo build and run use inconsistent debug flags for proc-macro crates

[ilammy]

Problem

cargo build passes debuginfo flags to rustc when compiling proc-macro crates in dev profile, while cargo run does not.

Different flags cause unnecessary recompilation, since crate compilations have different metadata hashes. For example, if a workspace has a custom derive macros defined and a lot a binaries, it is not possible to run cargo build to compile all the binaries first, then run them as needed using cargo run without recompiling everything.

I'd expect cargo build and cargo run to use the same rustc flags for crates when compiling with the same profile.

Steps

Use the attached cargo-example.zip

1. cargo build --verbose
2. cargo run --bin foo --verbose

Look for -C debuginfo=2

Possible Solution(s)

As a workaround, I disable debuginfo for build dependencies explicitly:

[profile.dev.build-override]
debug = false

With this, proc-macro crates are consistently built without debuginfo in dev profile and do not cause recompilation.

Notes

Reproduces since nightly-2023-02-09

Appears to be related to

• Turn off debuginfo for build dependencies v2 #11252
• Inconsistent handling of debug = 0 vs debug = false in profiles  #12163
• Procedural macros are rebuilding all dependencies rust-playground#928
• Excessive rebuilding when nothing changed quickwit-oss/quickwit#3346

Version

cargo 1.71.1 (7f1d04c00 2023-07-29)
release: 1.71.1
commit-hash: 7f1d04c0053083b98fa50b69b6f56e339b0556a8
commit-date: 2023-07-29
host: x86_64-apple-darwin
libgit2: 1.6.4 (sys:0.17.1 vendored)
libcurl: 7.79.1 (sys:0.4.61+curl-8.0.1 system ssl:(SecureTransport) LibreSSL/3.3.6)
ssl: OpenSSL 1.1.1t  7 Feb 2023
os: Mac OS 12.5.0 [64-bit]

[weihanglo]

What if you run cargo build --bin goo --verbose? What's the output there, still got discrepancy?

[ilammy]

What if you run cargo build --bin goo --verbose?

It behaves as cargo run --bin foo – compiles proc-macro crate without debuginfo. No discrepancy there.

[weihanglo]

Thanks for the reply! Just had a look at the repro. There are things contributing to this “expected” behavior.

• Turn off debuginfo for build dependencies v2 #11252 try to speed up the compile time by skipping debuginfo generation for build dependencies, as it normally runs only on host machine but not included in your final binary. Procedural macros is also considered as a kind of "build dependencies".
• Cargo has its own rules to unify features as much as possible to cut the compile time. If you build every workspace members in one command, it'll unify features between shared packages.
• As the repro is a Cargo workspace,
  • When with --bin, Cargo knows what you're looking for so it only need to compile a single package. By specifying --bin foo, it knows only one package foo being built, so debuginfo is turned off.
  • If without --bin Cargo will compile the default set of members in that workspace. Thus, it compiles bar as a build dependency of far with debuginfo off on, as well as a top-level package with debuginfo enabled, due to feature unification.

See this section from Cargo book for more :)

[ilammy]

• When with --bin, Cargo knows what you're looking for so it only need to compile a single package. By specifying --bin foo, it knows only one package foo being built, so debuginfo is turned off.

• If without --bin Cargo will compile the default set of members in that workspace. Thus, it compiles bar as a build dependency of far with debuginfo off. as well as a top-level package with debuginfo enabled.

That's not how it works in my env though...

---

cargo build --verbose --bin foo

• Both foo and its dependency bar are built.
  • foo with debuginfo
  • bar without debuginfo

   Compiling bar v0.1.0 (/Users/ilammy/Documents/dev/cargo-example/bar)
     Running `rustc --crate-name bar --edition=2021 bar/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=254 --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C split-debuginfo=unpacked -C metadata=a1ca32715342d3dd -C extra-filename=-a1ca32715342d3dd --out-dir /Users/ilammy/Documents/dev/cargo-example/target/debug/deps -C incremental=/Users/ilammy/Documents/dev/cargo-example/target/debug/incremental -L dependency=/Users/ilammy/Documents/dev/cargo-example/target/debug/deps --extern proc_macro`
   Compiling foo v0.1.0 (/Users/ilammy/Documents/dev/cargo-example/foo)
     Running `rustc --crate-name foo --edition=2021 foo/src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=254 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=6ebaf294658eab47 -C extra-filename=-6ebaf294658eab47 --out-dir /Users/ilammy/Documents/dev/cargo-example/target/debug/deps -C incremental=/Users/ilammy/Documents/dev/cargo-example/target/debug/incremental -L dependency=/Users/ilammy/Documents/dev/cargo-example/target/debug/deps --extern bar=/Users/ilammy/Documents/dev/cargo-example/target/debug/deps/libbar-a1ca32715342d3dd.dylib`
    Finished dev [unoptimized + debuginfo] target(s) in 0.37s

---

cargo build --verbose

• Both foo and its dependency bar are built.
  • Both crates are built with debuginfo

   Compiling bar v0.1.0 (/Users/ilammy/Documents/dev/cargo-example/bar)
     Running `rustc --crate-name bar --edition=2021 bar/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=254 --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=8e10e85c8bad2546 -C extra-filename=-8e10e85c8bad2546 --out-dir /Users/ilammy/Documents/dev/cargo-example/target/debug/deps -C incremental=/Users/ilammy/Documents/dev/cargo-example/target/debug/incremental -L dependency=/Users/ilammy/Documents/dev/cargo-example/target/debug/deps --extern proc_macro`
   Compiling foo v0.1.0 (/Users/ilammy/Documents/dev/cargo-example/foo)
     Running `rustc --crate-name foo --edition=2021 foo/src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=254 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 -C metadata=caf8a09f1e2501cc -C extra-filename=-caf8a09f1e2501cc --out-dir /Users/ilammy/Documents/dev/cargo-example/target/debug/deps -C incremental=/Users/ilammy/Documents/dev/cargo-example/target/debug/incremental -L dependency=/Users/ilammy/Documents/dev/cargo-example/target/debug/deps --extern bar=/Users/ilammy/Documents/dev/cargo-example/target/debug/deps/libbar-8e10e85c8bad2546.dylib`
    Finished dev [unoptimized + debuginfo] target(s) in 0.88s

---

But you're right that it's less about cargo run vs. cargo build, and looks to be more about building specific crate vs. the workspace (--bin foo and -p foo have the same behavior).

I'd expect bar to always be compiled without debuginfo by default, since it's a proc-macro crate and thus it's always a build-time dependency. At least right now, while proc-macro crates are allowed to export only macros.

---

Another data point here: this affects only crates within the workspace. For example, if I replace dependency on bar with

[dependencies]
serde = { version = "1", features = ["derive"] }

then serde_derive – a transitive dependency – is always compiled without debuginfo, whether I build all crates in the workspace, or specifically foo which depends on serde.

[weihanglo]

Sorry that's my mistake. I've updated my previous comment. In summary, it is

• build dependencies disables debug info by default.
• If a build dependency is shared with normal dependency, or it is exactly a workplace member being built, feature unification happens. Therefore the build dependency will compile with debug info.

Your observation is absolutely correct and expected.

• serde_derive is not a workspace member or a normal dependency, so debug info is disabled.
• bar is a member so running cargo build --verbose will unify features of bar.

[dpc]

Thank you very much for this report. Just want to add it might create problems in software that attempts to "build dependencies only" as a separate step to use it for caching purposes. I was able to pin-point it to debug = false, but if it wasn't for this report I'd have a hell of a time to figure it out.

[ehuss]

I'm not sure there is anything to do here, since it sounds like this is behaving as intended.

[dpc]

While the original report is about differences between run and build, there's something else wrong that is not run vs build related, and I reported it in #12603 . The behavior changed between releases of Rust. The workaround from this report helps.

[weihanglo]

Closing as it is expected behavior.

And thanks, worth noting the workaround :)

As a workaround, I disable debuginfo for build dependencies explicitly:

[profile.dev.build-override]
debug = false

With this, proc-macro crates are consistently built without debuginfo in dev profile and do not cause recompilation.