Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nixFlakes's nix-build segfaults in boehm-gc #3931

Closed
kalbasit opened this issue Aug 13, 2020 · 13 comments
Closed

nixFlakes's nix-build segfaults in boehm-gc #3931

kalbasit opened this issue Aug 13, 2020 · 13 comments
Labels
bug

Comments

@kalbasit
Copy link
Member

kalbasit commented Aug 13, 2020 •
edited
Loading

Describe the bug

I'm using Nix version 2.4pre20200721_ff314f1, and I'm trying to build my host (does not use Flakes). It crashes randomly with over 90% reproducibility. I can only reproduce with nix-build running as root.

nix-env --version output

nix (Nix) 2.4pre20200721_ff314f1

Steps to reproduce

Additional context

The following is only a part of the trace. The entire trace is available here.

gdb backtrace 
(gdb) run
Starting program: /nix/store/q3lp9d5s9c0jlkw3x84w029zmgnlqx29-nix-2.4pre20200721_ff314f1/bin/nix-build /nix/store/8nssa19jsrw2x1h9fryqwsqac07005ii-nixpkgs-release-20-03-4364ff933ebec0ef856912b182f4f9272aa7f98f/nixos -A system -k -I nixos-config=/yl/code/repositories/github.com/kalbasit/dotshabka/hosts/achilles/configuration.nix --show-trace
(gdb) bt

Thread 1 "nix-build" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ad1d40 in GC_push_all_eager () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
(gdb) bt
#0  0x00007ffff7ad1d40 in GC_push_all_eager () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#1  0x00007ffff7adfd90 in GC_push_all_stacks () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#2  0x00007ffff7adb7b7 in GC_mark_some () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#3  0x00007ffff7adb9d5 in GC_stopped_mark () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#4  0x00007ffff7adcff4 in GC_try_to_collect_inner () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#5  0x00007ffff7add368 in GC_collect_or_expand () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#6  0x00007ffff7add728 in GC_allocobj () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#7  0x00007ffff7addabf in GC_generic_malloc_inner () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#8  0x00007ffff7ae285e in GC_generic_malloc_many () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#9  0x00007ffff7ae2a42 in GC_malloc_kind () from /nix/store/krbh984z3lpvj614gfp2ss59b8ss2nq1-boehm-gc-8.0.4/lib/libgc.so.1
#10 0x00007ffff7deafde in nix::EvalState::allocEnv(unsigned long) () from /nix/store/q3lp9d5s9c0jlkw3x84w029zmgnlqx29-nix-2.4pre20200721_ff314f1/lib/libnixexpr.so

(gdb) p $_siginfo._sifields._sigfault.si_addr
$1 = (void *) 0x13693000

Memory mapping with info proc mappings can be seen here.
@kalbasit kalbasit added the bug label Aug 13, 2020
@Mic92
Copy link
Member

Mic92 commented Aug 13, 2020

I was able to reproduce this error on my machine with the given instruction.

@Mic92
Copy link
Member

Mic92 commented Aug 13, 2020

So the segfault address 0x13693000 matches the end of the heap:

            0x64d000         0x13693000 0x13046000        0x0 [heap]

@Mic92
Copy link
Member

Mic92 commented Aug 13, 2020 •
edited
Loading

Here is the output of gef, which also includes registers, assembly etc: https://dl.thalheim.io/d9A29KQJSmlMY6KF4tfPOg/2020-08-14_00-10.png

@Mic92
Copy link
Member

Mic92 commented Aug 13, 2020

Here is the line in Nix where it crashes:

nix/src/libexpr/eval-inline.hh

Line 92 in e11bbfb

p = GC_MALLOC(n);

@Mic92
Copy link
Member

Mic92 commented Aug 13, 2020

From the assembly instruction it seems like it zero initialize the memory but goes beyond the heap.

@Mic92
Copy link
Member

Mic92 commented Aug 13, 2020

Right before it crash I see a lot of invocations of nix-gitignore, when using --trace-function-calls:

function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:64:11 at 1597361401171364058
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:64:11 at 1597361401171371004
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:64:11 at 1597361401171378133
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:12 at 1597361401171385596
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:12 at 1597361401171392476
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:12 at 1597361401171399790
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:66:12 at 1597361401171406906
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:66:12 at 1597361401171413709
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:66:12 at 1597361401171420748
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:66:12 at 1597361401171469035
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:12 at 1597361401171476167
function-trace entered undefined position at 1597361401171483695
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:24 at 1597361401171489752
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:24 at 1597361401171497122
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:68 at 1597361401171503024
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:72:30 at 1597361401171509344
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:72:30 at 1597361401171515022
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:72:30 at 1597361401171520851
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:72:30 at 1597361401171528515
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:65:68 at 1597361401171534550
function-trace exited undefined position at 1597361401171540339
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:64:11 at 1597361401171544023
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:73:22 at 1597361401171549808
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:78:16 at 1597361401171556743
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:78:16 at 1597361401171562522
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:78:16 at 1597361401171568300
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:78:16 at 1597361401171574155
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:39 at 1597361401171581184
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:82:22 at 1597361401171589236
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:20 at 1597361401171595580
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:86:24 at 1597361401171601290
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:26:33 at 1597361401171607172
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:26:26 at 1597361401171617637
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:27:32 at 1597361401171623858
function-trace exited undefined position at 1597361401171629669
function-trace entered undefined position at 1597361401171633129
function-trace entered undefined position at 1597361401171636508
function-trace exited undefined position at 1597361401171640146
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:27:32 at 1597361401171643698
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:26:26 at 1597361401171649591
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:26:26 at 1597361401171655297
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:26:26 at 1597361401171660932
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:26:33 at 1597361401171666832
function-trace entered undefined position at 1597361401171672596
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:9 at 1597361401171676163
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:9 at 1597361401171682691
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:9 at 1597361401171688806
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:9 at 1597361401171696871
function-trace exited undefined position at 1597361401171703080
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:86:24 at 1597361401171706316
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:20 at 1597361401171712299
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:82:22 at 1597361401171718422
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:82:22 at 1597361401171724711
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:82:22 at 1597361401171730500
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:39 at 1597361401171736976
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:75:16 at 1597361401171744062
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:75:16 at 1597361401171750632
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:75:16 at 1597361401171756449
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:71:20 at 1597361401171762466
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:71:20 at 1597361401171768071
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:71:20 at 1597361401171773880
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:58 at 1597361401171780012
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:58 at 1597361401171785775
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:89:58 at 1597361401171791392
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:64:11 at 1597361401171797391
function-trace exited /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:64:11 at 1597361401171803058
function-trace entered /nix/store/bcacfn520v42pkrnm1c2y20vdls48x87-nixpkgs-release-unstable-8e2b14aceb1d40c7e8b84c03a7c78955359872bb/pkgs/build-support/nix-gitignore/default.nix:64:11 at 1597361401171808908

Thread 1 "nix-build" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ae6d40 in GC_push_all_eager () from /nix/store/sglx821fc8xwq2ibcx5zww5dcyv03iqq-boehm-gc-8.0.4/lib/libgc.so.1
(gdb)

@offlinehacker
Copy link
Contributor

I got the same segfault today on nix nix (Nix) 3.0pre20200829_f156513

@roberth roberth mentioned this issue Nov 17, 2020
Closed
@jonringer
Copy link
Contributor

For others who land on this page.

Until there's a proper fix, you may try adding GC_DONT_GC=1 to the command.

GC_DONT_GC=1 nix build ...

this seems to allow IFD to complete successfully for me

@stale
Copy link

stale bot commented Jun 2, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the stale label Jun 2, 2021
@kalbasit
Copy link
Member Author

kalbasit commented Jun 3, 2021

Stale yes, but still needs to be fixed.

@stale stale bot removed the stale label Jun 3, 2021
@stale
Copy link

stale bot commented Jan 8, 2022

I marked this as stale due to inactivity. → More info

@stale stale bot added the stale label Jan 8, 2022
@Mic92
Copy link
Member

Mic92 commented Jan 9, 2022

I think this has been addressed but I cannot point to a specific commit.

@stale stale bot removed the stale label Jan 9, 2022
@edolstra
Copy link
Member

Yeah, this may have been fixed by 5740924 so I'll close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@kalbasit @Mic92 @offlinehacker @edolstra @jonringer