LibGit2 segfault on exit

[simonbyrne]

I seem to be getting segfaults when Julia exits, and there are LibGit2 finalizers to be run (probably due to #19660).

e.g. from within current Julia git directory, this reliably causes a segfault on my machine:

├ juliadev -e 'r = LibGit2.GitRepo("."); LibGit2.get(LibGit2.GitCommit, r, LibGit2.GitHash("48e923fb1ed7c27350441bf36696abf66a015716"));'

signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
git_mwindow_put_pack at /Users/simon/src/julia/usr/lib/libgit2.0.25.0.dylib (unknown line)
pack_backend__free at /Users/simon/src/julia/usr/lib/libgit2.0.25.0.dylib (unknown line)
git_odb_free at /Users/simon/src/julia/usr/lib/libgit2.0.25.0.dylib (unknown line)
git_repository__cleanup at /Users/simon/src/julia/usr/lib/libgit2.0.25.0.dylib (unknown line)
git_repository_free at /Users/simon/src/julia/usr/lib/libgit2.0.25.0.dylib (unknown line)
close at ./libgit2/types.jl:491
jlcall_close_23653 at /Users/simon/src/julia/usr/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Users/simon/src/julia/src/gf.c:2214
jl_apply at /Users/simon/src/julia/src/./julia.h:1413 [inlined]
run_finalizer at /Users/simon/src/julia/src/gc.c:110
jl_gc_run_finalizers_in_list at /Users/simon/src/julia/src/gc.c:193 [inlined]
run_finalizers at /Users/simon/src/julia/src/gc.c:221
jl_atexit_hook at /Users/simon/src/julia/src/init.c:223
main at /Users/simon/bin/juliadev (unknown line)
Allocations: 1007473 (Pool: 1006322; Big: 1151); GC: 0
Segmentation fault: 11

[simonbyrne]

cc: @tkelman

[tkelman]

yes, just bisected to #19660 from running Pkg.test("PkgBenchmark")

[simonbyrne]

It's kind of weird. It goes away if I manually close the objects before exiting. But not if I just close just the GitCommit object.

[simonbyrne]

And I still see the problem if I revert to libgit2 v0.24.5

[simonbyrne]

@tkelman do you only see the segfault at exit as well?

[tkelman]

So far, yeah

[simonbyrne]

Yeah, it seems to be something funny happening at exit. e.g.

julia> function foo()
       r = LibGit2.GitRepo(".")
       c = LibGit2.get(LibGit2.GitCommit, r, LibGit2.GitHash("48e923fb1ed7c27350441bf36696abf66a015716"))
       return true
       end
foo (generic function with 1 method)

julia> foo()
true

julia> gc()

works fine. But if I leave off the gc() call, I get the segfault at exit.

[simonbyrne]

It goes away if I disable this atexit hook, so it seems that the problem is the finalizers are being called after git_libgit2_shutdown.

[simonbyrne]

Yup, that's it:

julia> r = LibGit2.GitRepo(".");

julia> c = LibGit2.get(LibGit2.GitCommit, r, LibGit2.GitHash("48e923fb1ed7c27350441bf36696abf66a015716"));

julia> ccall((:git_libgit2_shutdown, :libgit2), Cint, ())
0

julia> close(r)

signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
git_mwindow_put_pack at /Users/simon/src/julia/usr/lib/libgit2.0.24.5.dylib (unknown line)
pack_backend__free at /Users/simon/src/julia/usr/lib/libgit2.0.24.5.dylib (unknown line)
git_odb_free at /Users/simon/src/julia/usr/lib/libgit2.0.24.5.dylib (unknown line)
git_repository__cleanup at /Users/simon/src/julia/usr/lib/libgit2.0.24.5.dylib (unknown line)
git_repository_free at /Users/simon/src/julia/usr/lib/libgit2.0.24.5.dylib (unknown line)
close at ./libgit2/types.jl:466
jlcall_close_24615 at /Users/simon/src/julia/usr/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Users/simon/src/julia/src/gf.c:2214
do_call at /Users/simon/src/julia/src/interpreter.c:75
eval at /Users/simon/src/julia/src/interpreter.c:215
jl_interpret_toplevel_expr at /Users/simon/src/julia/src/interpreter.c:34
jl_toplevel_eval_flex at /Users/simon/src/julia/src/toplevel.c:640
jl_toplevel_eval_in at /Users/simon/src/julia/src/builtins.c:614
eval at ./boot.jl:238
jlcall_eval_18808 at /Users/simon/src/julia/usr/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Users/simon/src/julia/src/gf.c:2214
eval_user_input at ./REPL.jl:66
unknown function (ip: 0x3119c9696)
jl_apply_generic at /Users/simon/src/julia/src/gf.c:2214
macro expansion at ./REPL.jl:97 [inlined]
#1 at ./event.jl:71
unknown function (ip: 0x3119c093f)
jl_apply_generic at /Users/simon/src/julia/src/gf.c:2214
jl_apply at /Users/simon/src/julia/src/./julia.h:1413 [inlined]
start_task at /Users/simon/src/julia/src/task.c:261
Allocations: 3162223 (Pool: 3160897; Big: 1326); GC: 3
Segmentation fault: 11

[simonbyrne]

@yuyichao You mentioned in this discussion that it might be possible to add post-finalize atexit functions? How feasible would this be?

[vtjnash]

Does that function also cleanup all of the libgit2 objects? I think you could just set a flag before calling shutdown, and return early from any finalizers afterwards

[simonbyrne]

No it doesn't, all objects need to be cleaned up before calling git_libgit2_shutdown.

[yuyichao]

add post-finalize atexit functions?

That can be hard to specify what you can do and what you can't in it so I'd rather not unless it's really necessary.

I think you should just refcount the global context in the libgit2 object constructor and finalizers.

[simonbyrne]

Should be fixed by #20124.

[tkelman]

Confirming that Pkg.test("PkgBenchmark") works fine now.