Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug on julia 1.0 ubuntu #28998

Closed
jmichel7 opened this issue Sep 1, 2018 · 4 comments
Closed

Bug on julia 1.0 ubuntu #28998

jmichel7 opened this issue Sep 1, 2018 · 4 comments
Labels
bug Indicates an unexpected problem or unintended behavior missing data Base.missing and related functionality

Comments

@jmichel7
Copy link

jmichel7 commented Sep 1, 2018

I found a bug in my environment

julia> versioninfo()
Julia Version 1.0.0
Commit 5d4eaca0c9 (2018-08-08 20:58 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

To reproduce it, make a file Bug.jl with the following contents:

module Bug
using Combinatorics
v=[missing, 2, missing, 6, missing, missing, missing, missing, 
missing, missing, missing, missing, missing, 6]
end

Then start the REPL and type using Bug. Then typing any command like 1+1 or just exiting the REPL
with ^D shows the bug which crashs julia. Here is the bug message:

signal (11): Erreur de segmentation
in expression starting at no file:0
jl_gc_pool_alloc at /buildworker/worker/package_linux64/build/src/gc.c:963
jl_gc_alloc_ at /buildworker/worker/package_linux64/build/src/julia_internal.h:274 [inlined]
jl_gc_alloc at /buildworker/worker/package_linux64/build/src/gc.c:2668
_new_array_ at /buildworker/worker/package_linux64/build/src/array.c:99 [inlined]
_new_array at /buildworker/worker/package_linux64/build/src/array.c:156 [inlined]
jl_alloc_array_1d at /buildworker/worker/package_linux64/build/src/array.c:416
Type at ./boot.jl:394 [inlined]
Type at ./boot.jl:403 [inlined]
fill at ./array.jl:418 [inlined]
fill at ./array.jl:416 [inlined]
domsort_ssa! at ./compiler/ssair/slot2ssa.jl:434
construct_ssa! at ./compiler/ssair/slot2ssa.jl:869
just_construct_ssa at ./compiler/ssair/driver.jl:109
run_passes at ./compiler/ssair/driver.jl:114
optimize at ./compiler/optimize.jl:162
typeinf at ./compiler/typeinfer.jl:35
abstract_call_method_with_const_args at ./compiler/abstractinterpretation.jl:191
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:91
abstract_call at ./compiler/abstractinterpretation.jl:779
abstract_eval_call at ./compiler/abstractinterpretation.jl:808
abstract_eval at ./compiler/abstractinterpretation.jl:893
typeinf_local at ./compiler/abstractinterpretation.jl:1117
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1173
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:492
abstract_call_method at ./compiler/abstractinterpretation.jl:331
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:79
abstract_call at ./compiler/abstractinterpretation.jl:779
abstract_eval_call at ./compiler/abstractinterpretation.jl:808
abstract_eval at ./compiler/abstractinterpretation.jl:893
typeinf_local at ./compiler/abstractinterpretation.jl:1117
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1173
typeinf at ./compiler/typeinfer.jl:15
typeinf_edge at ./compiler/typeinfer.jl:492
abstract_call_method at ./compiler/abstractinterpretation.jl:331
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:79
abstract_call at ./compiler/abstractinterpretation.jl:779
abstract_eval_call at ./compiler/abstractinterpretation.jl:808
abstract_eval at ./compiler/abstractinterpretation.jl:893
typeinf_local at ./compiler/abstractinterpretation.jl:1103
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1173
typeinf at ./compiler/typeinfer.jl:15
typeinf_ext at ./compiler/typeinfer.jl:567
typeinf_ext at ./compiler/typeinfer.jl:604
jfptr_typeinf_ext_1.clone_1 at /home/jmichel/julia-1.0.0/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1536 [inlined]
jl_apply_with_saved_exception_state at /buildworker/worker/package_linux64/build/src/rtutils.c:257
jl_type_infer at /buildworker/worker/package_linux64/build/src/gf.c:275
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1784 [inlined]
jl_fptr_trampoline at /buildworker/worker/package_linux64/build/src/gf.c:1828
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
refresh_multi_line at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:1622
jfptr_refresh_multi_line_11548.clone_1 at /home/jmichel/julia-1.0.0/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
#refresh_multi_line#17 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:461
jfptr_#refresh_multi_line#17_10187.clone_1 at /home/jmichel/julia-1.0.0/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
refresh_multi_line at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:459
jfptr_refresh_multi_line_11541.clone_1 at /home/jmichel/julia-1.0.0/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
#refresh_multi_line#13 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:367
jfptr_#refresh_multi_line#13_10135.clone_1 at /home/jmichel/julia-1.0.0/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
refresh_multi_line at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:367
jfptr_refresh_multi_line_11540.clone_1 at /home/jmichel/julia-1.0.0/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
refresh_multi_line at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:201
refresh_line at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:1189
commit_line at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:1904
#109 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:1987
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1536 [inlined]
jl_f__apply at /buildworker/worker/package_linux64/build/src/builtins.c:556
jl_f__apply_latest at /buildworker/worker/package_linux64/build/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:686 [inlined]
invokelatest at ./essentials.jl:685 [inlined]
#27 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:1319
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
prompt! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:2353
run_interface at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/LineEdit.jl:2256
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
run_frontend at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:1029
run_repl at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:191
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
#720 at ./logging.jl:311
jfptr_#720_5697.clone_1 at /home/jmichel/julia-1.0.0/lib/julia/sys.so (unknown line)
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1536 [inlined]
jl_f__apply at /buildworker/worker/package_linux64/build/src/builtins.c:556
jl_f__apply_latest at /buildworker/worker/package_linux64/build/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:686 [inlined]
invokelatest at ./essentials.jl:685 [inlined]
macro expansion at ./logging.jl:308 [inlined]
run_main_repl at ./client.jl:330
exec_options at ./client.jl:242
_start at ./client.jl:421
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
jl_apply at /buildworker/worker/package_linux64/build/ui/../src/julia.h:1536 [inlined]

Finally here are my installed packages:

(v1.0) pkg> st --manifest
    Status `~/.julia/environments/v1.0/Manifest.toml`
  [6e4b80f9] BenchmarkTools v0.4.0
  [861a8166] Combinatorics v0.7.0
  [682c06a0] JSON v0.19.0
  [c03570c3] Memoize v0.3.0
  [bac558e1] OrderedCollections v0.1.0
  [f27b6e38] Polynomials v0.5.0
  [27ebfcd6] Primes v0.4.0
  [295af30f] Revise v0.7.5
  [2a0f44e3] Base64 
  [ade2ca70] Dates 
  [8ba89e20] Distributed 
  [7b1f6079] FileWatching 
  [b77e0a4c] InteractiveUtils 
  [76f85450] LibGit2 
  [8f399da3] Libdl 
  [37e2e46d] LinearAlgebra 
  [56ddb016] Logging 
  [d6f4376e] Markdown 
  [a63ad114] Mmap 
  [44cfe95a] Pkg 
  [de0858da] Printf 
  [3fa0cd96] REPL 
  [9a3f8284] Random 
  [ea8e919c] SHA 
  [9e88b42a] Serialization 
  [1a1011a3] SharedArrays 
  [6462fe0b] Sockets 
  [2f01184e] SparseArrays 
  [10745b16] Statistics 
  [8dfed614] Test 
  [cf7118a7] UUIDs 
  [4ec0a83e] Unicode
@jmichel7
Copy link
Author

jmichel7 commented Sep 1, 2018

A comment from LeoK987 on discourse:

I tried with this with the 1.1.0 nightly build, and there is no problem if I just do using Bug.

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0-DEV.127 (2018-08-27)
 _/ |\__'_|_|_|\__'_|  |  Commit 3ab56f19a8 (5 days old master)
|__/                   |

julia> push!(LOAD_PATH, ".")
4-element Array{String,1}:
 "@"      
 "@v#.#"  
 "@stdlib"
 "."      

julia> using Bug

julia> exit()

But, if I first include the file before using Bug, then Julia coredumps:

@j2b2
Copy link

j2b2 commented Sep 1, 2018

Amazing !

julia> using Bug
[ Info: Recompiling stale cache file /home/betrema/.julia/compiled/v0.7/Bug.ji for Bug [top-level]

julia> 1 + 1
signal (11): Segmentation fault, etc.

and at the second pass:

julia> using Bug

julia> Bug.v
14-element Array{Union{Missing, Int64},1}:
  missing
 2       
  missing
 6       
  missing
  missing
  missing
  missing
  missing
  missing
  missing
  missing
  missing
  missing

everything looks ok ... except the last 6 in the array is replaced by missing !

@j2b2
Copy link

j2b2 commented Sep 2, 2018

With the following module (no use Combinatorics, slight clarification):

module Bug
a = [missing, 1, missing, 2]
b = fill(missing, 9)
v = [a; b; 3]
end

I get a new kind of Segmentation fault:

julia> push!(LOAD_PATH, ".");

julia> using Bug

julia> Bug.v[1]
missing

julia> Bug.v[2]
1

so far, so good, until:

julia> Bug.v[8]
missing

julia> Bug.v[9]

signal (11): Segmentation fault
in expression starting at no file:0
jl_f_tuple at /buildworker/worker/package_linux64/build/src/builtins.c:703
eval_user_input at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:89
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:117 [inlined]
#28 at ./task.jl:262
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2182
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1538 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:268
unknown function (ip: 0xffffffffffffffff)
Allocations: 231101 (Pool: 231009; Big: 92); GC: 0
Segmentation fault (core dumped)

And of course everything works fine for:

module Bug
a = [missing, 1, missing, 2]
b = fill(missing, 8)
v = [a; b; 3]
end

or:

b = fill(missing, 10)

Baffling !

@Keno Keno added the bug Indicates an unexpected problem or unintended behavior label Sep 2, 2018
@Keno
Copy link
Member

Keno commented Sep 2, 2018

Looks like what's happening here is that jl_deserialize_value_array doesn't allocate enough space for the selector bits causing an out of bounds write with all the usual memory corrupting fun. Working on a fix. Thanks for the easy reproducer.

Keno added a commit that referenced this issue Sep 3, 2018
The array was allocated based on the serialized `elsize` of the array,
however, unions get an extra selector array after the regular storage
which was not allocated (because we didn't know it was gonna be a union
array at the time when we allocated it). According to
a48eeef we cannot look at the element
type to allocate the array, so we need to serialize a bit to indicate
that we will have a union array.

Fixes #28998
@nalimilan nalimilan added the missing data Base.missing and related functionality label Sep 3, 2018
Keno added a commit that referenced this issue Sep 3, 2018
The array was allocated based on the serialized `elsize` of the array,
however, unions get an extra selector array after the regular storage
which was not allocated (because we didn't know it was gonna be a union
array at the time when we allocated it). According to
a48eeef we cannot look at the element
type to allocate the array, so we need to serialize a bit to indicate
that we will have a union array.

Fixes #28998
Keno added a commit that referenced this issue Sep 4, 2018
The array was allocated based on the serialized `elsize` of the array,
however, unions get an extra selector array after the regular storage
which was not allocated (because we didn't know it was gonna be a union
array at the time when we allocated it). According to
a48eeef we cannot look at the element
type to allocate the array, so we need to serialize a bit to indicate
that we will have a union array.

Fixes #28998
KristofferC pushed a commit that referenced this issue Sep 4, 2018
The array was allocated based on the serialized `elsize` of the array,
however, unions get an extra selector array after the regular storage
which was not allocated (because we didn't know it was gonna be a union
array at the time when we allocated it). According to
a48eeef we cannot look at the element
type to allocate the array, so we need to serialize a bit to indicate
that we will have a union array.

Fixes #28998

(cherry picked from commit e7d7259)
KristofferC pushed a commit that referenced this issue Sep 8, 2018
The array was allocated based on the serialized `elsize` of the array,
however, unions get an extra selector array after the regular storage
which was not allocated (because we didn't know it was gonna be a union
array at the time when we allocated it). According to
a48eeef we cannot look at the element
type to allocate the array, so we need to serialize a bit to indicate
that we will have a union array.

Fixes #28998

(cherry picked from commit e7d7259)
KristofferC pushed a commit that referenced this issue Sep 8, 2018
The array was allocated based on the serialized `elsize` of the array,
however, unions get an extra selector array after the regular storage
which was not allocated (because we didn't know it was gonna be a union
array at the time when we allocated it). According to
a48eeef we cannot look at the element
type to allocate the array, so we need to serialize a bit to indicate
that we will have a union array.

Fixes #28998

(cherry picked from commit e7d7259)
KristofferC pushed a commit that referenced this issue Feb 11, 2019
The array was allocated based on the serialized `elsize` of the array,
however, unions get an extra selector array after the regular storage
which was not allocated (because we didn't know it was gonna be a union
array at the time when we allocated it). According to
a48eeef we cannot look at the element
type to allocate the array, so we need to serialize a bit to indicate
that we will have a union array.

Fixes #28998

(cherry picked from commit e7d7259)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior missing data Base.missing and related functionality
Projects
None yet
Development

No branches or pull requests

4 participants