Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segmentation fault (core dumped) for Ruby <3.1.0 using jemalloc (--with-jemalloc) #242

Closed
tisba opened this issue Feb 19, 2022 · 5 comments
Closed

Segmentation fault (core dumped) for Ruby <3.1.0 using jemalloc (--with-jemalloc) #242

tisba opened this issue Feb 19, 2022 · 5 comments
Labels
jemalloc The issue is related to the use of jemalloc

Comments

@tisba
Copy link
Collaborator

tisba commented Feb 19, 2022 •
edited
Loading

Extracted from #193 (comment) (thanks @jarthod 🙇).

Using Ruby build with --with-jemalloc seem to cause a segmentation fault with Ruby < 3.1.0, tested with latest mini_racer (0.6.3) and libv8-node (16.10.0.0). Using LD_PRELOAD seems to work fine for Ruby 3.0.3 and 3.1.0.

See #193 (comment) for testing Ruby 3.1.0 and 3.0.3 with --jemalloc on x86_64-linux (@jarthod: Do you have some more information you can provide on the crash? output? logs?).

The issue can be triggered by:

MiniRacer::Platform.set_flags!(:single_threaded)

I'll follow up (and update) with more tests on different Ruby versions (2.6.9 and 2.7.5) and architectures (x86 and aarch).

Update: Works with (build with --with-jemalloc, tested with https://github.com/moritzheiber/ruby-jemalloc-docker/pkgs/container/ruby-jemalloc):

  • 3.1.0
  • 3.2.1
@jarthod
Copy link

jarthod commented Feb 20, 2022

@jarthod: Do you have some more information you can provide on the crash? output? logs?

I don't have more than this in the console output of this simple test:

> ruby -r mini_racer -e "puts MiniRacer::VERSION; MiniRacer::Platform.set_flags! :single_threaded"
0.6.2
free(): invalid pointer
Aborted (core dumped)

I tried running some gdbdump (like in #193) but It didn't work for me:

> gdbdump $HOME/.rbenv/versions/3.0.3/bin/ruby core
/home/bigbourin/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/gdbdump-0.9.4/lib/gdbdump/gdb.rb:12:in `initialize': wrong number of arguments (given 1, expected 0; required keyword: pid_or_core) (ArgumentError)
	from /home/bigbourin/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/gdbdump-0.9.4/lib/gdbdump/cli.rb:56:in `new'
	from /home/bigbourin/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/gdbdump-0.9.4/lib/gdbdump/cli.rb:56:in `run'
	from /home/bigbourin/.rbenv/versions/3.0.3/lib/ruby/gems/3.0.0/gems/gdbdump-0.9.4/exe/gdbdump:4:in `<top (required)>'
	from /home/bigbourin/.rbenv/versions/3.0.3/bin/gdbdump:25:in `load'
	from /home/bigbourin/.rbenv/versions/3.0.3/bin/gdbdump:25:in `<main>'

I believe there's some incompatibility with ruby 3 here.

I ran the other tests you talked about though (2.7.5 and 2.6.9) on x86:

> RUBY_CONFIGURE_OPTS=--with-jemalloc CONFIGURE_OPTS="--disable-install-rdoc" rbenv install 2.7.5
> rbenv shell 2.7.5
> gem install mini_racer
Fetching libv8-node-16.10.0.0-x86_64-linux-musl.gem
Successfully installed libv8-node-16.10.0.0-x86_64-linux-musl
Building native extensions. This could take a while...
ERROR:  Error installing mini_racer:
	ERROR: Failed to build gem native extension.

    current directory: /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/mini_racer-0.6.2/ext/mini_racer_extension
/home/bigbourin/.rbenv/versions/2.7.5/bin/ruby -I /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/2.7.0 -r ./siteconf20220220-80964-whzcm6.rb extconf.rb
checking for -lpthread... yes
creating Makefile

current directory: /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/mini_racer-0.6.2/ext/mini_racer_extension
make "DESTDIR=" clean

current directory: /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/mini_racer-0.6.2/ext/mini_racer_extension
make "DESTDIR="
compiling mini_racer_extension.cc
In file included from /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/libv8-node-16.10.0.0-x86_64-linux-musl/vendor/v8/include/v8.h:30,
                 from mini_racer_extension.cc:6:
/home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/libv8-node-16.10.0.0-x86_64-linux-musl/vendor/v8/include/v8-internal.h: In function ‘VALUE convert_v8_to_ruby(v8::Isolate*, v8::Local<v8::Context>, v8::Local<v8::Value>)’:
/home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/libv8-node-16.10.0.0-x86_64-linux-musl/vendor/v8/include/v8-internal.h:364:48: warning: ‘stackCounter.StackCounter::isolate’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  364 |     return *reinterpret_cast<void* const*>(addr);
      |                                                ^
mini_racer_extension.cc:573:18: note: ‘stackCounter.StackCounter::isolate’ was declared here
  573 |     StackCounter stackCounter(isolate);
      |                  ^~~~~~~~~~~~
linking shared-object mini_racer_extension.so
/usr/bin/ld: cannot find /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/libv8-node-16.10.0.0-x86_64-linux-musl/vendor/v8/x86_64-linux/libv8/obj/libv8_monolith.a: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [Makefile:262: mini_racer_extension.so] Error 1

make failed, exit code 2

Gem files will remain installed in /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/gems/mini_racer-0.6.2 for inspection.
Results logged to /home/bigbourin/.rbenv/versions/2.7.5/lib/ruby/gems/2.7.0/extensions/x86_64-linux/2.7.0/mini_racer-0.6.2/gem_make.out

→ I could not compile the gem here, I didn't dive deeper.

> RUBY_CONFIGURE_OPTS=--with-jemalloc CONFIGURE_OPTS="--disable-install-rdoc" rbenv install 2.6.9
> rbenv shell 2.6.9
> gem install mini_racer
Fetching mini_racer-0.6.2.gem
Fetching libv8-node-16.10.0.0-x86_64-linux.gem
Successfully installed libv8-node-16.10.0.0-x86_64-linux
Building native extensions. This could take a while...
Successfully installed mini_racer-0.6.2
2 gems installed
> ruby -r mini_racer -e "puts MiniRacer::VERSION; puts RbConfig::CONFIG['MAINLIBS']; MiniRacer::Platform.set_flags! :single_threaded"
0.6.2
-lz -lpthread -lrt -lrt -ljemalloc -ldl -lcrypt -lm 
free(): invalid pointer
Aborted (core dumped)

So the problem is present here too.

I tried some gdbdump there too but without success (I can't get the core dump file to be created)

> gem install gdbdump
Fetching gdbdump-0.9.4.gem
Successfully installed gdbdump-0.9.4
1 gem installed
> ulimit -c
unlimited
> ruby -r mini_racer -e "puts MiniRacer::VERSION; puts RbConfig::CONFIG['MAINLIBS']; MiniRacer::Platform.set_flags! :single_threaded"
0.6.2
-lz -lpthread -lrt -lrt -ljemalloc -ldl -lcrypt -lm 
free(): invalid pointer
Aborted (core dumped)
> gdbdump /home/bigbourin/.rbenv/versions/2.6.9/bin/ruby core
/home/bigbourin/.rbenv/versions/2.6.9/lib/ruby/gems/2.6.0/gems/gdbdump-0.9.4/lib/gdbdump/gdb.rb:18:in `initialize': core core is not readable (RuntimeError)
	from /home/bigbourin/.rbenv/versions/2.6.9/lib/ruby/gems/2.6.0/gems/gdbdump-0.9.4/lib/gdbdump/cli.rb:56:in `new'
	from /home/bigbourin/.rbenv/versions/2.6.9/lib/ruby/gems/2.6.0/gems/gdbdump-0.9.4/lib/gdbdump/cli.rb:56:in `run'
	from /home/bigbourin/.rbenv/versions/2.6.9/lib/ruby/gems/2.6.0/gems/gdbdump-0.9.4/exe/gdbdump:4:in `<top (required)>'
	from /home/bigbourin/.rbenv/versions/2.6.9/bin/gdbdump:23:in `load'
	from /home/bigbourin/.rbenv/versions/2.6.9/bin/gdbdump:23:in `<main>'
> ls core
ls: cannot access 'core': No such file or directory

I also tried running ruby in GDB directly and managed to get this if it's of any help:

> gdb --args /home/bigbourin/.rbenv/versions/2.6.9/bin/ruby -r mini_racer -e "puts MiniRacer::VERSION; puts RbConfig::CONFIG['MAINLIBS']; MiniRacer::Platform.set_flags! :single_threaded"
GNU gdb (Ubuntu 11.1-0ubuntu2) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/bigbourin/.rbenv/versions/2.6.9/bin/ruby...
(gdb) r
Starting program: /home/bigbourin/.rbenv/versions/2.6.9/bin/ruby -r mini_racer -e puts\ MiniRacer::VERSION\;\ puts\ RbConfig::CONFIG\[\'MAINLIBS\'\]\;\ MiniRacer::Platform.set_flags\!\ :single_threaded
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0.6.2
-lz -lpthread -lrt -lrt -ljemalloc -ldl -lcrypt -lm 
free(): invalid pointer

Program received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737341136768) at pthread_kill.c:44
44	pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737341136768) at pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737341136768) at pthread_kill.c:80
#2  __GI___pthread_kill (threadid=140737341136768, signo=signo@entry=6) at pthread_kill.c:91
#3  0x00007ffff7a38476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7a1e7b7 in __GI_abort () at abort.c:79
#5  0x00007ffff7a7f5e6 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7bd113d "%s\n")
    at ../sysdeps/posix/libc_fatal.c:155
#6  0x00007ffff7a96adc in malloc_printerr (str=str@entry=0x7ffff7bced86 "free(): invalid pointer") at malloc.c:5543
#7  0x00007ffff7a98414 in _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:4326
#8  0x00007ffff7a9ace5 in __GI___libc_free (mem=<optimized out>) at malloc.c:3278
#9  0x00007ffff0ce5919 in ?? ()
   from /home/bigbourin/.rbenv/versions/2.6.9/lib/ruby/gems/2.6.0/extensions/x86_64-linux/2.6.0/mini_racer-0.6.2/mini_racer_extension.so
#10 0x00007ffff0bfc87f in rb_platform_set_flag_as_str (_klass=<optimized out>, flag_as_str=93824996571400)
    at /home/bigbourin/.rbenv/versions/2.6.9/include/ruby-2.6.0/ruby/ruby.h:345
#11 0x00007ffff7e8ccc6 in vm_call_cfunc_with_frame (cc=<optimized out>, cc=<optimized out>, ci=0x5555559d87e0, calling=<optimized out>, 
    reg_cfp=0x7ffff7394ef8, ec=0x55555556d188) at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/vm_insnhelper.c:1908
#12 vm_call_cfunc (ec=0x55555556d188, reg_cfp=0x7ffff7394ef8, calling=<optimized out>, ci=0x5555559d87e0, cc=<optimized out>)
    at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/vm_insnhelper.c:1924
#13 0x00007ffff7e97784 in vm_exec_core (ec=0x1731f, initial=6) at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/insns.def:765
#14 0x00007ffff7e9d5a9 in rb_vm_exec (ec=0x55555556d188, mjit_enable_p=1) at vm.c:1885
#15 0x00007ffff7eab481 in invoke_block (captured=0x7ffff7394f80, captured=0x7ffff7394f80, opt_pc=<optimized out>, type=<optimized out>, 
    cref=0x0, self=93824997167280, iseq=0x5555559d7498, ec=0x55555556d188) at vm.c:1021
#16 invoke_iseq_block_from_c (me=0x0, is_lambda=<optimized out>, cref=0x0, passed_block_handler=0, argv=0x7fffffffdc28, argc=1, 
    self=93824997167280, captured=0x7ffff7394f80, ec=0x55555556d188) at vm.c:1092
#17 invoke_block_from_c_bh (force_blockarg=<optimized out>, is_lambda=<optimized out>, cref=<optimized out>, 
    passed_block_handler=<optimized out>, argv=<optimized out>, argc=<optimized out>, block_handler=<optimized out>, ec=<optimized out>)
    at vm.c:1110
#18 vm_yield (argv=0x7fffffffdc28, argc=1, ec=0x55555556d188) at vm.c:1155
#19 rb_yield_0 (argv=0x7fffffffdc28, argc=1) at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/vm_eval.c:978
#20 rb_yield_1 (val=<optimized out>) at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/vm_eval.c:984
#21 rb_yield (val=<optimized out>) at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/vm_eval.c:994
#22 0x00007ffff7c7236c in rb_ary_each (ary=93824996571080) at array.c:2087
#23 0x00007ffff7e8ccc6 in vm_call_cfunc_with_frame (cc=<optimized out>, cc=<optimized out>, ci=0x5555559d95a0, calling=<optimized out>, 
    reg_cfp=0x7ffff7394f68, ec=0x55555556d188) at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/vm_insnhelper.c:1908
#24 vm_call_cfunc (ec=0x55555556d188, reg_cfp=0x7ffff7394f68, calling=<optimized out>, ci=0x5555559d95a0, cc=<optimized out>)
    at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/vm_insnhelper.c:1924
#25 0x00007ffff7e97878 in vm_exec_core (ec=0x1731f, initial=6) at /tmp/ruby-build.20220220151418.81097.7nH8Rg/ruby-2.6.9/insns.def:750
#26 0x00007ffff7e9d5a9 in rb_vm_exec (ec=0x55555556d188, mjit_enable_p=1) at vm.c:1885
#27 0x00007ffff7d0141b in ruby_exec_internal (n=0x555555977d68) at eval.c:262
#28 0x00007ffff7d03a01 in ruby_exec_node (n=<optimized out>, n@entry=0x555555977d68) at eval.c:326
#29 0x00007ffff7d06842 in ruby_run_node (n=0x555555977d68) at eval.c:318
#30 0x000055555555517f in main (argc=<optimized out>, argv=<optimized out>) at ./main.c:42
(gdb)

@oLeVanNghia
Copy link

oLeVanNghia commented Jul 19, 2022 •
edited
Loading

Ruby: 3.0.2
OS: Centos 7

how to fix this issue?

$ ruby -r mini_racer -e "puts MiniRacer::VERSION; MiniRacer::Platform.set_flags! :single_threaded"
/home/bunbu/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/mini_racer-0.6.2/lib/mini_racer.rb:11:in `load': /home/bunbu/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/extensions/x86_64-linux/3.0.0/mini_racer-0.6.2/mini_racer_extension.so: undefined symbol: _ZTTNSt7__cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEEE (LoadError)
	from /home/bunbu/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/mini_racer-0.6.2/lib/mini_racer.rb:11:in `<top (required)>'
	from <internal:/home/bunbu/.rbenv/versions/3.0.2/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:160:in `require'
	from <internal:/home/bunbu/.rbenv/versions/3.0.2/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
	from <internal:/home/bunbu/.rbenv/versions/3.0.2/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:149:in `require'
<internal:/home/bunbu/.rbenv/versions/3.0.2/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- mini_racer (LoadError)
	from <internal:/home/bunbu/.rbenv/versions/3.0.2/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'`

@tisba
Copy link
Collaborator Author

tisba commented Jul 22, 2022

I don't see how your issue is related to this one, @oLeVanNghia. This is about using jemalloc resulting in a segmentation fault.

@paranoicsan
Copy link

Encountered the same issue

free(): invalid pointer
Aborted (core dumped)

Prerequisites:

mini_racer 0.6.3
ruby 2.7.6p219 with jemalloc
Ubuntu 20.04 LTS
libjemalloc2:amd64 5.2.1-1ubuntu1

Installing libjemalloc1 3.6.0-11 from 18.04 bionic repository and setting up LD_PRELOAD to it works like a temporary workaround.

@tisba
Copy link
Collaborator Author

tisba commented May 26, 2023

I just compiled Ruby 3.0.6 with jemalloc on macOS Venture 13.4 via rvm:

$ brew install jemalloc
$ rvm install 3.0.6 --with-jemalloc --with-cppflags="-I$(brew --prefix)/include" -E "LDFLAGS=-L$(brew --prefix)/lib"

…and it works 🥳

$ ruby -r mini_racer -e "puts MiniRacer::VERSION; puts RbConfig::CONFIG['MAINLIBS']; MiniRacer::Platform.set_flags! :single_threaded"
0.6.4
-lpthread -ljemalloc -lgmp -ldl -lobjc

@tisba tisba closed this as completed May 26, 2023
@tisba tisba mentioned this issue Nov 29, 2023
Closed
@tisba tisba added the jemalloc The issue is related to the use of jemalloc label May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
jemalloc The issue is related to the use of jemalloc
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tisba @jarthod @oLeVanNghia @paranoicsan