Sometimes GAP is started multiple times from Julia

[fingolfin]

Sometimes GAP is started more than one time in Julia. That is, I've seen instances were a package depending on GAP was loaded, and I saw the GAP banner printed twice. Unfortunately, I don't have a way to reproduce it so far; but I saw it happen on somebody else's computer today, so I wanted to at least make a note of it.

My suspicion is that this could be caused by different GAP versions being loaded by different packages, each getting its "own" GAP. But I am not sure about that. I am somewhat worried about negative interactions, though.

[ThomasBreuer]

One instance of this problem is the following:
When I load a Julia package that requires GAP.jl, and when this packages has to be recompiled by Julia for some reason, one GAP banner appears in the compilation, and a second banner appears when the dependency on GAP.jl gets evaluated.
Can this phenomenon be avoided at all?

[ThomasBreuer]

@fingolfin Could the situation described in my comment above be an explanation of what you had observed?

[fingolfin]

@ThomasBreuer sorry, I missed your comment earlier.

Indeed, I think you are right. I just came upon this again, too, and it matches what you say: I loaded GAP.jl via using GAP, and then another Julia package which uses GAP.jl as well, at which point a precompilation was triggered, and GAP got started a second time.

julia> using GAP
 ┌───────┐   GAP 4.11.0-1-ge33ffb1 of today
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-apple-darwin18.7.0-julia64-kv7
 Configuration:  gmp 6.1.2, Julia GC, Julia 1.3.1, readline, KernelDebug
 Loading the library and packages ...
 Packages:   GAPDoc 1.6.3, PrimGrp 3.3.2, SmallGrp 1.4.1, TransGrp 2.0.4
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'

julia> using GAPGroups
[ Info: Precompiling GAPGroups [21fe2a1e-8ea0-11e9-0a1a-b318e3c68493]
 ┌───────┐   GAP 4.11.0-1-ge33ffb1 of today
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-apple-darwin18.7.0-julia64-kv7
 Configuration:  gmp 6.1.2, Julia GC, Julia 1.3.1, readline, KernelDebug
 Loading the library and packages ...
 Packages:   GAPDoc 1.6.3, PrimGrp 3.3.2, SmallGrp 1.4.1, TransGrp 2.0.4
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'

After starting Julia again, I couldn't reproduce this anymore, presumably because the precompilation result was stored and hence no precompilation was necessary.

This should now allow us to debug the problem.

[fingolfin]

Note that the banner is called when we call the GAP kernel function GAP_Initialize. But the call chain for that is: __init__ → run_it → initialize → GAP_Initialize

So I am lead to the conclusion that the __init__ function of the GAP module is called twice. But only if precompilation of a package using GAP is involved. I verified this by the good old trick of inserting some print statements:

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.3.1 (2019-12-30)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using GAP
[ Info: Precompiling GAP [c863536a-3901-11e9-33e7-d5cd0df7b904]
GAP.jl __init__
 ┌───────┐   GAP 4.11.0-1-ge33ffb1-dirty of today
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-apple-darwin18.7.0-julia64-kv7
 Configuration:  gmp 6.1.2, Julia GC, Julia 1.3.1, readline, KernelDebug
 Loading the library and packages ...
 Packages:   GAPDoc 1.6.3, PrimGrp 3.3.2, SmallGrp 1.4.1, TransGrp 2.0.4
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
GAP_register_GapObj: set gapobj_type = 0x1185ae4d0

julia> using GAPGroups
[ Info: Precompiling GAPGroups [21fe2a1e-8ea0-11e9-0a1a-b318e3c68493]
GAP.jl __init__
 ┌───────┐   GAP 4.11.0-1-ge33ffb1-dirty of today
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-apple-darwin18.7.0-julia64-kv7
 Configuration:  gmp 6.1.2, Julia GC, Julia 1.3.1, readline, KernelDebug
 Loading the library and packages ...
 Packages:   GAPDoc 1.6.3, PrimGrp 3.3.2, SmallGrp 1.4.1, TransGrp 2.0.4
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
GAP_register_GapObj: set gapobj_type = 0x1116cac40

julia>

This also shows that the GapObj type passed to GAP_register_GapObj really is not identical in both cases, which has deeply worrying implications overall

Repeating this immediately then avoids the precompilation and GAP is started just once:

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.3.1 (2019-12-30)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using GAP
GAP.jl __init__
 ┌───────┐   GAP 4.11.0-1-ge33ffb1-dirty of today
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-apple-darwin18.7.0-julia64-kv7
 Configuration:  gmp 6.1.2, Julia GC, Julia 1.3.1, readline, KernelDebug
 Loading the library and packages ...
 Packages:   GAPDoc 1.6.3, PrimGrp 3.3.2, SmallGrp 1.4.1, TransGrp 2.0.4
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
GAP_register_GapObj: set gapobj_type = 0x114f24e10

julia> using GAPGroups

julia>

[fingolfin]

Note that in both settings, we have GAP === GAPGroups.GAP which is somewhat of a relieve.

The Julia documentation is somewhat ambiguous on how and when __init__ is called. To quote:

In particular, if you define a function init() in a module, then Julia will call init() immediately after the module is loaded (e.g., by import, using, or require) at runtime for the first time (i.e., init is only called once, and only after all statements in the module have been executed).

One way to read it is that GAP's __init__ should ever be called at most once, as in the second output. Another is that it might called once "per instance" of the module (so once for GAP and GAPGroups.GAP each) as in the first output above. I can't think of an interpretation that fits both, though... ?

Regarding the wording of the documentation, I suspect that when GAPGroups is precompiled, a world counter gets increment, and/or the packages used by GAPGroups are marked as "stale"; and when they are accessed, Julia treats them as "new" and reloads the module(s) in question.

This suggests that we can fix the issue by remembering that we executed runit already, and not doing that again. We already have some checks for that in place, for when GAP loads Julia (instead of Julia loading GAP). I'll try my luck.

[fingolfin]

OK, problem (re)solved: I realized that the second banner is actually printed by a separate Julia process (I printed Libc.getpid() to verify this). So the only issue here is that the second banner is confusing, and merging PR #342 will alleviate this.

[ThomasBreuer]

@fingolfin So should I update Pull Request #342 and merge it?

[fingolfin]

Yes please