Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Local bootstrap fails until I manually invoke git submodule update --init #90481

Closed
pnkfelix opened this issue Nov 1, 2021 · 1 comment · Fixed by #91252
Closed

Local bootstrap fails until I manually invoke git submodule update --init #90481

pnkfelix opened this issue Nov 1, 2021 · 1 comment · Fixed by #91252
Assignees
@pnkfelix
Labels
C-bug Category: This is a bug. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@pnkfelix
Copy link
Member

pnkfelix commented Nov 1, 2021

I have found recently for some of my checkouts that I have to manually do git submodule update --init to work around an LLVM build failure.

I was under the impression that our build system was supposed to always take care of maintaining the git submodules, at least if you weren't doing things like trying to make your own local changes to those submodules. (I have no idea what our guarantees are, if any, in the latter scenario.)

But from what I can tell, the error in the transcript below, which I have now seen twice, is resolved by a manual invocation of git submodule update --init.

I don't expect other people to be able to do much about this until I manage to reproduce what is causing it. Luckily, this time I had the foresight to make a tarball of my git repo state when I encountered the problem this second time around.

Transcript of the failing build. 
 % /usr/bin/time -lp ../x.py build
Updating only changed submodules
Submodules updated in 0.04 seconds
extracting /Users/pnkfelix/Dev/Rust/rust.git/objdir/build/cache/2021-10-22/rust-std-beta-x86_64-apple-darwin.tar.xz
extracting /Users/pnkfelix/Dev/Rust/rust.git/objdir/build/cache/2021-10-22/rustc-beta-x86_64-apple-darwin.tar.xz
extracting /Users/pnkfelix/Dev/Rust/rust.git/objdir/build/cache/2021-10-22/cargo-beta-x86_64-apple-darwin.tar.xz
extracting /Users/pnkfelix/Dev/Rust/rust.git/objdir/build/cache/2021-10-23/rustfmt-nightly-x86_64-apple-darwin.tar.xz
   Compiling proc-macro2 v1.0.30
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.80
   Compiling autocfg v1.0.0
   Compiling version_check v0.9.3
   Compiling memchr v2.4.1
   Compiling cfg-if v1.0.0
   Compiling libc v0.2.103
   Compiling serde_derive v1.0.125
   Compiling lazy_static v1.4.0
   Compiling log v0.4.14
   Compiling serde v1.0.125
   Compiling ryu v1.0.5
   Compiling regex-syntax v0.6.25
   Compiling regex-automata v0.1.10
   Compiling same-file v1.0.6
   Compiling serde_json v1.0.59
   Compiling fnv v1.0.7
   Compiling cc v1.0.69
   Compiling unicode-width v0.1.8
   Compiling bootstrap v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/src/bootstrap)
   Compiling itoa v0.4.6
   Compiling once_cell v1.7.2
   Compiling opener v0.5.0
   Compiling build_helper v0.1.0 (/Users/pnkfelix/Dev/Rust/rust.git/src/build_helper)
   Compiling thread_local v1.0.1
   Compiling proc-macro-error-attr v1.0.4
   Compiling proc-macro-error v1.0.4
   Compiling num-traits v0.2.12
   Compiling crossbeam-utils v0.8.3
   Compiling getopts v0.2.21
   Compiling walkdir v2.3.1
   Compiling cmake v0.1.44
   Compiling filetime v0.2.14
   Compiling num_cpus v1.13.0
   Compiling time v0.1.43
   Compiling quote v1.0.7
   Compiling aho-corasick v0.7.18
   Compiling bstr v0.2.13
   Compiling regex v1.5.4
   Compiling globset v0.4.5
   Compiling ignore v0.4.17
   Compiling merge_derive v0.1.0
   Compiling merge v0.1.0
   Compiling toml v0.5.7
    Finished dev [unoptimized] target(s) in 41.86s
Building stage0 std artifacts (x86_64-apple-darwin -> x86_64-apple-darwin)
   Compiling cc v1.0.69
   Compiling core v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/core)
   Compiling libc v0.2.103
   Compiling memchr v2.4.1
   Compiling std v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/std)
   Compiling compiler_builtins v0.1.49
   Compiling unwind v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/unwind)
   Compiling rustc-std-workspace-core v1.99.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/rustc-std-workspace-core)
   Compiling alloc v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/alloc)
   Compiling cfg-if v0.1.10
   Compiling adler v0.2.3
   Compiling rustc-demangle v0.1.21
   Compiling rustc-std-workspace-alloc v1.99.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/rustc-std-workspace-alloc)
   Compiling panic_abort v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/panic_abort)
   Compiling panic_unwind v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/panic_unwind)
   Compiling gimli v0.25.0
   Compiling std_detect v0.1.5 (/Users/pnkfelix/Dev/Rust/rust.git/library/stdarch/crates/std_detect)
   Compiling object v0.26.2
   Compiling miniz_oxide v0.4.0
   Compiling hashbrown v0.11.0
   Compiling addr2line v0.16.0
   Compiling rustc-std-workspace-std v1.99.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/rustc-std-workspace-std)
   Compiling proc_macro v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/proc_macro)
   Compiling unicode-width v0.1.8
   Compiling getopts v0.2.21
   Compiling test v0.0.0 (/Users/pnkfelix/Dev/Rust/rust.git/library/test)
    Finished release [optimized] target(s) in 42.02s
Copying stage0 std from stage0 (x86_64-apple-darwin -> x86_64-apple-darwin / x86_64-apple-darwin)
Building LLVM for x86_64-apple-darwin
running: "cmake" "/Users/pnkfelix/Dev/Rust/rust.git/src/llvm-project/llvm" "-G" "Ninja" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_ENABLE_PLUGINS=OFF" "-DLLVM_TARGETS_TO_BUILD=AArch64;ARM;BPF;Hexagon;MSP430;Mips;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR;M68k" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_INCLUDE_BENCHMARKS=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_BINDINGS=OFF" "-DLLVM_ENABLE_Z3_SOLVER=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=12" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-apple-darwin" "-DLLVM_INSTALL_UTILS=ON" "-DLLVM_ENABLE_ZLIB=ON" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_VERSION_SUFFIX=-rust-dev" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER=cc" "-DCMAKE_CXX_COMPILER=c++" "-DCMAKE_ASM_COMPILER=cc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64 -arch x86_64 -stdlib=libc++" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64 -arch x86_64 -stdlib=libc++" "-DCMAKE_INSTALL_PREFIX=/Users/pnkfelix/Dev/Rust/rust.git/objdir/build/x86_64-apple-darwin/llvm" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64 -arch x86_64" "-DCMAKE_BUILD_TYPE=Release"
-- Found libtool - /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool
-- Native target architecture is X86
-- Threads enabled.
-- Doxygen disabled.
-- Go bindings disabled.
-- Ninja version: 1.10.2
-- Found ld64 - /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
-- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) 
-- OCaml bindings disabled.
-- LLVM host triple: x86_64-apple-darwin19.6.0
-- LLVM default target triple: x86_64-apple-darwin
-- Building with -fPIC
-- LLVMHello ignored -- Loadable modules not supported on this platform.
-- Targeting AArch64
-- Targeting ARM
-- Targeting BPF
-- Targeting Hexagon
-- Targeting MSP430
-- Targeting Mips
-- Targeting NVPTX
-- Targeting PowerPC
-- Targeting RISCV
-- Targeting Sparc
-- Targeting SystemZ
-- Targeting WebAssembly
-- Targeting X86
-- Targeting AVR
-- Targeting M68k
CMake Error at lib/Target/CMakeLists.txt:32 (add_subdirectory):
  add_subdirectory given source "M68k" which is not an existing directory.


CMake Error at cmake/modules/LLVM-Build.cmake:42 (get_property):
  get_property could not find TARGET M68k.  Perhaps it has not yet been
  created.
Call Stack (most recent call first):
  lib/CMakeLists.txt:61 (LLVMBuildGenerateCFragment)


CMake Error at cmake/modules/LLVM-Build.cmake:43 (get_property):
  get_property could not find TARGET M68k.  Perhaps it has not yet been
  created.
Call Stack (most recent call first):
  lib/CMakeLists.txt:61 (LLVMBuildGenerateCFragment)


-- BugpointPasses ignored -- Loadable modules not supported on this platform.
CMake Error at cmake/modules/LLVM-Config.cmake:138 (message):
  Target M68k is not in the set of libraries.
Call Stack (most recent call first):
  cmake/modules/LLVM-Config.cmake:256 (llvm_expand_pseudo_components)
  cmake/modules/LLVM-Config.cmake:102 (llvm_map_components_to_libnames)
  cmake/modules/LLVM-Config.cmake:95 (explicit_llvm_config)
  cmake/modules/AddLLVM.cmake:908 (llvm_config)
  cmake/modules/AddLLVM.cmake:1209 (add_llvm_executable)
  tools/llvm-gsymutil/CMakeLists.txt:11 (add_llvm_tool)


-- Configuring incomplete, errors occurred!
See also "/Users/pnkfelix/Dev/Rust/rust.git/objdir/build/x86_64-apple-darwin/llvm/build/CMakeFiles/CMakeOutput.log".
See also "/Users/pnkfelix/Dev/Rust/rust.git/objdir/build/x86_64-apple-darwin/llvm/build/CMakeFiles/CMakeError.log".
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1

build script failed, must exit now', /Users/pnkfelix/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/cmake-0.1.44/src/lib.rs:885:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
	finished in 5.004 seconds
Build completed unsuccessfully in 0:01:42

</details>
@pnkfelix pnkfelix added C-bug Category: This is a bug. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Nov 1, 2021
@pnkfelix pnkfelix self-assigned this Nov 1, 2021
@pnkfelix pnkfelix added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 1, 2021
@Mark-Simulacrum
Copy link
Member

Cc @jyn514

My guess is we're not properly detecting we need to update the submodule - somewhere in https://github.com/rust-lang/rust/blob/master/src/bootstrap/lib.rs#L483

My initial guess is the build happening on objdir/build/ rather than build/ is causing something to go off, but not sure.

@jyn514 jyn514 mentioned this issue Nov 26, 2021
Merged
@bors bors closed this as completed in 404235e Nov 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pnkfelix pnkfelix

Labels
C-bug Category: This is a bug. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix bug where submodules wouldn't be updated when running x.py from a subdirectory jyn514/rust
2 participants
@pnkfelix @Mark-Simulacrum