-
Notifications
You must be signed in to change notification settings - Fork 103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't build on Windows 10 with x86_64-pc-windows-gnu
; GHA not actually cross-compiling either?
#1193
Comments
Ayyyy, got it to work. TL;DR: Compiling on Windows == still broken for me. Cross-compiling from Linux works if 1) target/host overrides are set correctly and 2) you make sure Cross-compiling from WSL wasn't working because somehow in all my faffing about I overrode the default host triple in addition to the default target. Oof. So, it compiled then, but the resulting // Build and copy cargo wrapper
let mut build_cargo_wrapper_cmd = Command::new("rustc");
build_cargo_wrapper_cmd
.arg("--target")
.arg(target_triple)
.arg("scripts/cargo.rs")
.arg("-o")
.arg(target_dir.join("cargo"))
.arg("-g");
spawn_and_wait(build_cargo_wrapper_cmd); That would also explain why CI doesn't appear to be producing a viable cross-compiled artifact! |
Dammit, just kidding. That was my system |
I have never actually tried to cross-compile cg_clif itself for windows. CI tests compiling cg_clif on windows using the mingw toolchain and it tests compiling programs using cg_clif for windows using the mingw toolchain.
Maybe removing https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/build_sysroot/Cargo.toml#L23 helps? |
Yeah, I figured that out yesterday, I just ran out of time trying weird combinations of compiling different bits on Windows and WSL to post about it. I was like, OOOOOH got it, the host/target are operating on a different layer of indirection than I assumed. It's not that the codegen is cross-compiled to run on windows, it's that the codegen does cross-compilation (running on Linux, producing cranelift debug builds that run on windows). That's awesome, but I don't feel like doubling up on dependency sources/build-artifacts between WSL and Windows... |
Ahhhh, the |
While I'm at it, is there a reason that the |
I can't remember. Maybe a leftover from trying to set up windows ci? |
oooooh, could that be a side-effect of |
It's also very probable that the windows CI build suffers from the same issue. The action only tests whether it compiles; I tried running thread 'rustc' panicked at 'can't resolve symbol _ZN9mini_core8A_STATIC17h1e90a401893c503aE', C:\Users\pormand\.cargo\git\checkouts\wasmtime-41807828cb3a7a7e\9c550fc\cranelift\jit\src\backend.rs:288:21
stack backtrace:
0: 0x6f2d2d2f - std::backtrace_rs::backtrace::dbghelp::trace::h15703146ef9749a7
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\..\..\backtrace\src\backtrace/dbghelp.rs:98:5
1: 0x6f2d2d2f - std::backtrace_rs::backtrace::trace_unsynchronized::hf8d56c17f1e136e9
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\..\..\backtrace\src\backtrace/mod.rs:66:5
2: 0x6f2d2d2f - std::sys_common::backtrace::_print_fmt::hab3c52876e76d449
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:67:5
3: 0x6f2d2d2f - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hab9c1cc6e7fc385b
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:46:22
4: 0x6f33e39a - core::fmt::write::hae96daedf1d6be29
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\core\src\fmt/mod.rs:1117:17
5: 0x6f2c6262 - std::io::Write::write_fmt::hafe5401ad6ee5626
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\io/mod.rs:1667:15
6: 0x6f2d64e0 - std::sys_common::backtrace::_print::hbe08c5b6a843015d
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:49:5
7: 0x6f2d64e0 - std::sys_common::backtrace::print::h674aa25aba0c71a3
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:36:9
8: 0x6f2d64e0 - std::panicking::default_hook::{{closure}}::h5c12d9d9abbca0f9
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:210:50
9: 0x6f2d6094 - std::panicking::default_hook::hbc4ccd03703b88a3
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:227:9
10: 0x7ff6672d28fe - cg_clif::DEFAULT_HOOK::{{closure}}::{{closure}}::h4825952fd8172fa7
11: 0x6f2d6dc1 - std::panicking::rust_panic_with_hook::h1ecdc969fd899f2f
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:628:17
12: 0x6f2d6899 - std::panicking::begin_panic_handler::{{closure}}::h36573f2e57c65019
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:521:13
13: 0x6f2d373f - std::sys_common::backtrace::__rust_end_short_backtrace::hbff34e368d9c5643
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys_common/backtrace.rs:141:18
14: 0x6f2d67f9 - rust_begin_unwind
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:517:5
15: 0x6f2d67ac - std::panicking::begin_panic_fmt::h4b4cd9d4d947fbb4
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src/panicking.rs:460:5
16: 0x7ffba855d144 - cranelift_jit::backend::JITModule::get_address::ha71330b8095d55ca
17: 0x7ffba85639fa - cranelift_jit::compiled_blob::CompiledBlob::perform_relocations::h209dee35f48b61c5
18: 0x7ffba855de36 - cranelift_jit::backend::JITModule::finalize_definitions::hca5b226777528b55
19: 0x7ffba849e66f - rustc_codegen_cranelift::driver::jit::run_jit::h9a1e46bdfa469fae
20: 0x7ffba8503f4d - <rustc_codegen_cranelift::CraneliftCodegenBackend as rustc_codegen_ssa::traits::backend::CodegenBackend>::codegen_crate::h495a51801e90fb82
21: 0xadd650 - rustc_interface::passes::QueryContext::enter::hd0572fc530ac941d
22: 0xacdc5b - rustc_interface::queries::Queries::ongoing_codegen::h2f921abc637d3d75
23: 0x9df8ed - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter::h1cb4948d439e9483
24: 0x99a85c - rustc_span::with_source_map::h6cc506dfc70b42ef
25: 0x9e1120 - rustc_interface::interface::create_compiler_and_run::hf8038fe8f954301a
26: 0x9a7b3d - std::sys_common::backtrace::__rust_begin_short_backtrace::hb1ec0507bfe8704c
27: 0x99397e - core::ops::function::FnOnce::call_once{{vtable.shim}}::h5d71226d451acc81
28: 0x6f2e4557 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h525f0061b701c388
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\library\alloc\src/boxed.rs:1636:9
29: 0x6f2e4557 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h5e31eae84f71f0a7
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\library\alloc\src/boxed.rs:1636:9
30: 0x6f2e4557 - std::sys::windows::thread::Thread::new::thread_start::h1ed0a385a606a823
at /rustc/5ad7389bdd1abe7d2c6f73a233af1a7a69e96285\/library\std\src\sys\windows/thread.rs:57:17
31: 0x7ffc41e57034 - <unknown>
32: 0x7ffc43782651 - <unknown>
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/bjorn3/rustc_codegen_cranelift/issues/new
note: rustc 1.56.0-nightly (5ad7389bd 2021-08-06) running on x86_64-pc-windows-gnu
note: compiler flags: -Z unstable-options -C debuginfo=2 -C llvm-args=mode=jit -C prefer-dynamic
query stack during panic:
end of query stack Sounds suspiciously like https://github.com/bjorn3/rustc_codegen_cranelift/issues/1137? |
I see what is going on. On Unix systems the directory containing the current executabe is not in |
Looks like the jit tests don't work. Could you try if disabling those tests will make the rest of the tests pass?
Todo:
|
Confirmed. Renaming from |
It looks like disabling jit tests and tests that invoke |
|
Circling back to |
I disabled debuginfo on windows in a branch: https://github.com/bjorn3/rustc_codegen_cranelift/tree/windows_fixes This still needs the jit tests disabled and the recursive cargo problem to be fixed. |
Disabling the jit tests should probably be as easy as setting |
Pushed a commit to disable jit tests on windows to that branch. I am not quite sure how to proceed with the |
Hmm, I've been using the "rename the binary" strat for regular use (stuck the .exe in a folder in my Maybe rename for the tests? If you don't want to rename on all platforms, you could set a Maybe a global rename (test, dev, and release builds) wouldn't be such a bad idea. I can't imagine someone wanting to give up their regular cargo all the time for the cranelift one, or having to always type the full path for one of them (or set a custom alias or something). Better, imo, to opinionatedly stabilize on something so everyone's not doing something subtlely different. Ideally, I think, it wouldn't be a full cargo replacement wrapper. I don't know how practical this would be, but maybe it could work as a cargo command (like |
In the (hopefully) near future it will become possible to select cg_clif using an option in [profile.dev]
codegen-backend = "cranelift" The cargo wrapper will be kept for local development of cg_clif, but it won't be shipped with rustc. |
Renamed the cargo wrapper to cargo-clif. Opened https://github.com/bjorn3/rustc_codegen_cranelift/pull/1195 for the fixes so far. @EndilWayfare can you please test if it fixes all problems you have? |
Sure! Replying inline on the pull request. |
Hmm, noticing some interesting things with
Only one error, but all tests fail on Fixed the failing test, all tests still fail on |
On a lark, specifying the gnu toolchain doesn't help. It's worse 🤣
|
Don't know how to determine if it can't find a clif .dll, a gnu toolchain .dll, or some other .dll entirely... |
This comment has been minimized.
This comment has been minimized.
What project are those failing tests on? They are probably a problem with cg_clif independent of the windows support. |
I think you need to build cg_clif with the gnu toolchain too then. cg_clif needs to be built with the exact same compiler as which it will be loaded into. |
I'm 98% sure that version of |
Actually wait, doesn't this project force a pinned nightly toolchain? Because I only have that one installed in gnu flavor. |
Those failing tests are from one of my own projects. I was checking if cg_clif could help with test compile times. |
I believe this should work fine now given recent improvements. There is still a case where it can crash on windows though which will be fixed by bytecodealliance/wasmtime#4747. |
https://github.com/bjorn3/rustc_codegen_cranelift/pull/1284 has added actual native Windows testing to the CI. @EndilWayfare does it work for you now? |
Closing. If you still have problems please open a new issue. |
Following the instructions in the README, everything seemed to be working fine until
./y.rs build
.Ok, fair enough. Compatibility layer time!
Hmm, dang it. I can't seem to find helpful references to
-lcfgmgr32
online. It sounds like mingw can't find the library it needs forcfgmgr32.dll
. I tried reinstalling my mingw header package withpacman
to no avail. I found out, as I understand it, that thex86_64-pc-windows-gnu
toolchain uses a bundled mingw in arust-mingw
component, so my existing installation shouldn't matter anyway. What if I copylibcfgmgr32.a
from my local mingw into the "lib" folder of the cargo source for thewinapi
it's referencing (that doesn't sound like the best idea...)? Huh, that... fixes it?But now there's a new problem:
and other variations on that theme until
I found this (https://github.com/bjorn3/rustc_codegen_cranelift/issues/977#issuecomment-830650448), which seems to be related, except I'm not using MSVC. (How do I disable DWARF debuginfo?) Based on that comment/thread, I tried to comment out the bit in
y.rs
that blows up on MSVC, just for laughs. No luck;not yet implemented: Unimplemented TLS model in x64 backend: Coff
Yep, that's what the July report said, and it seems to still be the case. From what I can gather, theADDR32
bit of it has something to do with jumps into relocatable code being too large to fit in a 32-bit pointer. I can confirm that my installedgnu
toolchain compiles other crates successfully (I have nothing nearly as large as a sysroot, though).I played around with a bunch of
RUSTFLAGS
combination (both torustc
and passthrough togcc
codegen/linking) based on tangential information (very little info onIMAGE_REL_AMD64_ADDR32
specifically) to no avail. It's possible that this is an effective direction, and I just don't know enough aboutgcc
to make it work. It seems odd to me though, because there's no mention (that I could find) of special compiler args being necessary.I tried a few scattershot previous commits, before and after the transition to a Rust-based build system. Same problem.
I tried cross-compiling with Ubuntu on WSL2. No luck:
What are you talking about? It says
info: downloading component 'rustc'
and theninfo: installing component 'rustc'
RIGHT THERE. I have no idea what's going on, as I typically don't mess with nightly/betas.I gave up on trying to compile from source and checked out the GHA artifacts. Looks like the uploading artifacts from the windows CI build is disabled? https://github.com/bjorn3/rustc_codegen_cranelift/blob/d498e6d6971575714353f11aa4d3e63a9d2030b2/.github/workflows/main.yml#L155-L162
Oh well,
cross-x86_64-mingw
sounds promising.Dang it.
Well, that definitely doesn't sound right.
./y.rs build --sysroot none
did compile, although it (understandably) does nothing and hangs forever when invoked. However:Unless I'm doing something wrong (quite possible), it seems that the cross-compile CI target isn't measuring what it's supposed to be measuring (successfully producing a Windows-runnable artifact from Linux).
TWIR sent me here. I've been following this on-and-off for a while, and it's really awesome to see how close it is to basically-feature-complete. I wanted to get in on it (faster debug builds? Yes please!), but I've tried as hard as I can on my own.
The text was updated successfully, but these errors were encountered: