Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory usage regression since 5/1 #50389

Closed
ishitatsuyuki opened this issue May 2, 2018 · 8 comments
Closed

Memory usage regression since 5/1 #50389

ishitatsuyuki opened this issue May 2, 2018 · 8 comments
Labels
I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. P-high High priority regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Milestone
1.27

Comments

@ishitatsuyuki
Copy link
Contributor

ishitatsuyuki commented May 2, 2018 •
edited
Loading

It's back, caused by presumably #50197.

cc @nikomatsakis

   Compiling syntax_ext v0.0.0 (file:///home/user/rust/src/libsyntax_ext)
error: could not exec the linker `cc`
  |
  = note: Cannot allocate memory (os error 12)
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.rustc0-81414edc9fa898f39b569bc75d0a157e.rs.rcgu.o" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.rustc1-81414edc9fa898f39b569bc75d0a157e.rs.rcgu.o" "-o" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc-c6263d330d2b9f31.so" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.crate.metadata.rcgu.o" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.crate.allocator.rcgu.o" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/backtrace-sys-efe11c48d393cb3a/out/.libs" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/miniz-sys-ed3f79652d61a4a4/out" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "test-902559d4a0cc83d5" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "term-36880752958d2c6f" "-Wl,-Bstatic" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libflate2-be463fa6b96a16ec.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libminiz_sys-faafc4fdd43b8a1b.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libbacktrace-447434a17b628693.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libbacktrace_sys-63d1b058c70a5ba5.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/librustc_demangle-ffb675b42151a6f8.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libbyteorder-f652c706c62d3d96.rlib" "-Wl,--no-whole-archive" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-Wl,-Bdynamic" "-l" "proc_macro-c7d45a460cb3a738" "-Wl,-Bstatic" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libjobserver-5189829dca651532.rlib" "-Wl,--no-whole-archive" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-Wl,-Bdynamic" "-l" "rustc_const_math-e212b3283c198d45" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "syntax-e97e6d4ccc983706" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_errors-dafdecc781917b18" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "syntax_pos-ccbedf94f6bd555f" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_target-aaec33fa4a23ec51" "-Wl,-Bstatic" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/liblazy_static-d0629a75f2783580.rlib" "-Wl,--no-whole-archive" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-Wl,-Bdynamic" "-l" "graphviz-06bb7008d45cdff0" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "fmt_macros-7fa0522df5af949a" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "arena-16294da79648b864" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_data_structures-22e458fc7e0d421f" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_cratesio_shim-7bdcd669c7579602" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "serialize-b63c42e100577192" "-Wl,--start-group" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "std-b63203ee1146648a" "-Wl,--end-group" "-Wl,-Bstatic" "/tmp/rustc.BU4zySi8u8L1/libcompiler_builtins-bbf524080a1524b2.rlib" "-Wl,-Bdynamic" "-l" "util" "-l" "util" "-l" "dl" "-l" "rt" "-l" "pthread" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util" "-l" "util" "-shared" "-Wl,-rpath,$ORIGIN/../lib"

error: aborting due to previous error

error: Could not compile `rustc`.

Caused by:
  process didn't exit successfully: `/home/user/rust/build/bootstrap/debug/rustc --crate-name rustc librustc/lib.rs --error-format json --crate-type dylib --emit=dep-info,link -C prefer-dynamic -C opt-level=2 -C metadata=c6263d330d2b9f31 -C extra-filename=-c6263d330d2b9f31 --out-dir /home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps -L dependency=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps --extern rustc_target=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_target-aaec33fa4a23ec51.so --extern proc_macro=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libproc_macro-c7d45a460cb3a738.so --extern byteorder=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libbyteorder-f652c706c62d3d96.rlib --extern lazy_static=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/liblazy_static-d0629a75f2783580.rlib --extern rustc_apfloat=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_apfloat-262722bdf7d51c3a.rlib --extern bitflags=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libbitflags-fd8a318e919a8b90.rlib --extern jobserver=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libjobserver-5189829dca651532.rlib --extern graphviz=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libgraphviz-06bb7008d45cdff0.so --extern rustc_const_math=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_const_math-e212b3283c198d45.so --extern rustc_data_structures=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_data_structures-22e458fc7e0d421f.so --extern fmt_macros=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libfmt_macros-7fa0522df5af949a.so --extern log=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/liblog-19a330a6ee3947d3.rlib --extern rustc_errors=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_errors-dafdecc781917b18.so --extern syntax_pos=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax_pos-ccbedf94f6bd555f.so --extern arena=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libarena-16294da79648b864.so --extern tempdir=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libtempdir-83e93e20d067e6f5.rlib --extern syntax=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax-e97e6d4ccc983706.so --extern backtrace=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libbacktrace-447434a17b628693.rlib --extern flate2=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libflate2-be463fa6b96a16ec.rlib --extern serialize=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libserialize-b63c42e100577192.so --extern serialize=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libserialize-b63c42e100577192.rlib -L native=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/backtrace-sys-efe11c48d393cb3a/out/.libs -L native=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/miniz-sys-ed3f79652d61a4a4/out` (exit code: 101)
@ishitatsuyuki ishitatsuyuki added regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. labels May 2, 2018
@ishitatsuyuki
Copy link
Contributor Author

Although I'm not absolutely confident as I haven't performed a bisect, and perf data is not updated yet as well.

@nikomatsakis
Copy link
Contributor

@ishitatsuyuki

It's back

What exactly are you doing to reproduce this? Is that a reference to an earlier issue? (I vaguely remember an issue about this maybe...?)

@nikomatsakis nikomatsakis added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label May 2, 2018
@ishitatsuyuki
Copy link
Contributor Author

@nikomatsakis It's a reference to #50073. Basically, I mean that the OOM error happened again on that environment:

On the CircleCI Docker remote machine which has 2 cores and 8G memory, I'm no longer able to compile rustc. This error consistently occurs in stage1.

@nikomatsakis
Copy link
Contributor

@ishitatsuyuki thanks! perf is still not updated, it seems. #50197 could certainly be the cause, that could increase the size of the interning tables.

@nikomatsakis
Copy link
Contributor

triage: P-high -- we're investigating. First thing is to get the perf server up and going again, probably. ( cc @Mark-Simulacrum )

@nikomatsakis nikomatsakis added the P-high High priority label May 3, 2018
@pietroalbini pietroalbini added this to the 1.27 milestone May 7, 2018
@alexcrichton
Copy link
Member

Scanning the current max-rss statistics of perf.rust-lang.org doesn't show much that's suspicious (no recent spikes up)

@ishitatsuyuki
Copy link
Contributor Author

What I currently know is that compiling stage1 is taking much more memory at peak, compared to stage0. It's like 5G vs 3G.

@ishitatsuyuki
Copy link
Contributor Author

As of today it seems to compile successfully; so this is no longer critical. I will reopen if this happens again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. P-high High priority regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.27
Development

No branches or pull requests
4 participants
@alexcrichton @nikomatsakis @pietroalbini @ishitatsuyuki