use shallow clone and get(ENV, "GAP_INSTALL_PACKAGES", "no")

[kalmarek]

when building GAP, just that we don't end up downloading the full git tree and 2.6GB of pkgs

~/.julia/packages/GAP/W20PN/gap/pkg on  stable-4.11 ⌚ 17:58:29
$ du -h --max-depth=1 ./ | sort -h -r
2.6G    ./
661M    ./NormalizInterface-1.1.0
465M    ./semigroups-3.2.1
247M    ./simpcomp
111M    ./CaratInterface
90M     ./transgrp
80M     ./SmallGrp-1.4.1
74M     ./sglppow-2.1
73M     ./irredsol-1.4
53M     ./tomlib-1.2.9
51M     ./YangBaxter-0.9.0
49M     ./hap-1.24
49M     ./ctbllib
48M     ./rcwa-4.6.4
47M     ./ferret-1.0.2
41M     ./primgrp-3.4.0
40M     ./PolymakeInterface-2019.09.02
40M     ./difsets-2.3.1
29M     ./kbmag-1.5.9
26M     ./digraphs-1.0.3
24M     ./smallsemi-0.6.12
19M     ./unitlib-4.0.0
18M     ./sonata-2.9.1
17M     ./FactInt-1.6.3
14M     ./anupq-3.2.1
13M     ./guava-3.15
13M     ./fining
12M     ./profiling-2.2.1
12M     ./corelg-1.53

also #285 would be greatly appreciated

[fingolfin]

There seem to be two requests mixed in one issue here, let's see:

1. "Use shallow clone":
   We used to use a shallow clone, but that makes it really awkward to track GAP fixes as they come in. The plan is to drop the clone and instead download a specific GAP release -- but this is waiting for GAP 4.11 to be released. Which was meant to happen in September... I've been pressing for this, but there is a problem with personal here. So I've been co-organizing a GAP workshop in March which is meant (among other things) to revise how GAP releases are made, so that delays with them won't happen again (well, at least not due to pure infrastructure reasons, like "there is just one person who has the knowledge and the SSH keys to do this, and they don't have time, because the release process takes several days")

2. "Use get(ENV, "GAP_INSTALL_PACKAGES", "no")": so I guess this is asking to change the default for whether all packages are installed or none? (UPDATE: just to clarify, that doesn't even work, you'd need get(ENV, "GAP_INSTALL_PACKAGES", "minimal"). END UPDATE). I guess this is rather subjective. If you view GAP packages as optional add-ons, it makes sense. If you view packages as integral parts of GAP, not so much. It feels a bit like asking to install Polymake with several of its "applications" (is that how they are called?) removed. The truth is somewhere in the middle (there are a bunch of packages I think every use install should have, and a bunch that are far more specialized).

Anyway, I could be convinced to change the default, all I want to say is that it's not that clear cut that this is the "right" choice, at least not with a good way to install missing packages later on... I think the optimal solution will be to install not all packages, but more than the minimal number, and at the very least, install the PackageManager package, so that users have a decent chance of installing missing packages later on.

[kalmarek]

I guess for development you should keep track of your own version of GAP, shouldn't you? I know that this is only for UX until BinaryBuilder is done, but since there is no serious activity on that front (that I can see), maybe a quick and dirty one here is better than waiting for the BB? I'm just suggesting, since now adding GAP takes >20 minutes on my computer, which is not ideal from user perspective.

for the second suggestion it'd be best to create a (possibly) lean list of packages that need to be there leaving out the non-essentials. Eg. SmallGrp is definitely essential, and it don't increase the compile time. On the other hand ar NormalizInterface and semigroups that important to justify taking >80% compile time? Do we need PolymakeInterface in julia version of GAP? These will be first to be put on the block ;-)

I didn't even know that there is PackageManager pkg in GAP! but it seems to be a perfect fit for the task: GAP.install_package("NormalizInterface"), or even GAP.install_all_packages() would be great!

[kalmarek]

Adding path /home/kalmar/.julia/dev/GAP/gap/.libs to DL_LOAD_PATH
 ┌───────┐   GAP 4.11dev-1190-gceb7d49 of today
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-pc-linux-gnu-julia64-kv7
 Configuration:  gmp 6.1.2, Julia GC, Julia 1.3.0, readline
 Loading the library and packages ...
 Packages:   GAPDoc 1.6.3, PrimGrp 3.4.0, SmallGrp 1.4.1, TransGrp 2.0.4
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'

are these: GAPDoc 1.6.3, PrimGrp 3.4.0, SmallGrp 1.4.1, TransGrp 2.0.4 the only (non-julia related) packages that are strictly necessary?

[fingolfin]

Yes, those are the only, and they get installed when GAP_INSTALL_PACKAGES=minimal is set.

[kalmarek]

If we can cheaply get GAPs PackageManager working, I'd vote for making minimal the default...

[fingolfin]

To make PackageManager work, we of course need that, and also the curlInterface package.