Uunf_data seems too big for ppc64 and aarch64

[kit-ty-kate]

uunf.13.0.0 seems to reliably fail on powerpc64 with the following message:

#=== ERROR while compiling uunf.13.0.0 ========================================#
# context              2.1.0~beta4 | linux/ppc64 | ocaml-base-compiler.4.12.0 | file:///src
# path                 ~/.opam/4.12/.opam-switch/build/uunf.13.0.0
# command              ~/.opam/opam-init/hooks/sandbox.sh build ocaml pkg/pkg.ml build --pinned false --with-uutf true --with-cmdliner true
# exit-code            1
# env-file             ~/.opam/log/uunf-20-26642c.env
# output-file          ~/.opam/log/uunf-20-26642c.out
### output ###
[...]
# ocamlfind ocamlc -c -g -bin-annot -safe-string -I src -I test -o src/uunf.cmi src/uunf.mli
# ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -o src/uunf_data.cmx src/uunf_data.ml
# + ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -o src/uunf_data.cmx src/uunf_data.ml
# ocamlopt.opt got signal and exited
# Command exited with code 2.
# pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
#      '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
#      'README.md' 'src/uunf.a' 'src/uunf.cmxs' 'src/uunf.cmxa' 'src/uunf.cma'
#      'src/uunf.cmx' 'src/uunf.cmi' 'src/uunf.mli' 'src/uunf_data.cmx'
#      'src/uunf_tmapbyte.cmx' 'src/uunf_tmapbool.cmx' 'src/uunf_tmap.cmx'
#      'src/uunf_string.a' 'src/uunf_string.cmxs' 'src/uunf_string.cmxa'
#      'src/uunf_string.cma' 'src/uunf_string.cmx' 'src/uunf_string.cmi'
#      'src/uunf_string.mli' 'test/unftrip.native' 'doc/index.mld' 'DEVEL.md'
#      'test/examples.ml']: exited with 10

The issue looks similar to ocaml/ocaml#9839 and since it is the only platform that has this problem, my guess would be that this has to do with the memory pagesize. On linux/ppc64, by default, the page size is 64k big instead of the usual 4k.

[dbuenzli]

I ported the patches @pqwy made for uucp for interning strings over to uunf. It will be in the next release (14.0.0), we will see if that improves things.

[kit-ty-kate]

Could this be reopened? I've tried the master branch and got the same error:

# ocamlfind ocamlopt -linkpkg -package xmlm -package uucd -I src -I support src/uunf_fmt.cmx src/uunf_tmap.cmx src/uunf_tmapbool.cmx src/uunf_tmapbyte.cmx support/gen.cmx support/gen_norm.cmx support/gen_props.cmx -o support/gen_props.native
# Loading Unicode character database.
# Note: reported sizes do not take sharing into account.
# nfc boundary property as character boolean trie map, asserting data.
#  boolean trie map (default true)  size: 7.8 Ko
#  boolean trie map (default false) size: 207.9 Ko
#  Using map with default true.
#
# nfd boundary property as character boolean trie map, asserting data.
#  boolean trie map (default true)  size: 10.1 Ko
#  boolean trie map (default false) size: 206.1 Ko
#  Using map with default true.
#
# nfkc boundary property as character boolean trie map, asserting data.
#  boolean trie map (default true)  size: 8.8 Ko
#  boolean trie map (default false) size: 207.9 Ko
#  Using map with default true.
#
# nfkd boundary property as character boolean trie map, asserting data.
#  boolean trie map (default true)  size: 10.9 Ko
#  boolean trie map (default false) size: 206.0 Ko
#  Using map with default true.
#
# ccc property as character byte trie map, asserting data.
#  trie map size: 21.2 Ko
#
# decomposition mapping as trie map, asserting data.
#  trie map size: 203.0 Ko
#
# composition to primary composites as trie map, asserting data.
#  trie map size: 43.4 Ko
#  max num. of possible composition for a base char: 19
#
# ocamlfind ocamlc -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_fmt.cmo src/uunf_fmt.ml
# ocamlfind ocamlc -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_tmap.cmo src/uunf_tmap.ml
# ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_fmt.cmx src/uunf_fmt.ml
# ocamlfind ocamlc -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_tmapbool.cmo src/uunf_tmapbool.ml
# ocamlfind ocamlc -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_tmapbyte.cmo src/uunf_tmapbyte.ml
# ocamlfind ocamldep -modules src/uunf_data.ml > src/uunf_data.ml.depends
# ocamlfind ocamlc -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_data.cmo src/uunf_data.ml
# ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_tmap.cmx src/uunf_tmap.ml
# ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_tmapbool.cmx src/uunf_tmapbool.ml
# ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_tmapbyte.cmx src/uunf_tmapbyte.ml
# ocamlfind ocamldep -modules src/uunf.ml > src/uunf.ml.depends
# ocamlfind ocamldep -modules src/uunf.mli > src/uunf.mli.depends
# ocamlfind ocamlc -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf.cmi src/uunf.mli
# ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_data.cmx src/uunf_data.ml
# + ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -I support -o src/uunf_data.cmx src/uunf_data.ml
# ocamlopt.opt got signal and exited
# Command exited with code 2.
# pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
#      '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
#      'README.md' 'src/uunf.a' 'src/uunf.cmxs' 'src/uunf.cmxa' 'src/uunf.cma'
#      'src/uunf.cmx' 'src/uunf.cmi' 'src/uunf.mli' 'src/uunf_data.cmx'
#      'src/uunf_tmapbyte.cmx' 'src/uunf_tmapbool.cmx' 'src/uunf_tmap.cmx'
#      'src/uunf_fmt.cmx' 'doc/index.mld' 'DEVEL.md' 'test/examples.ml']: exited with 10

[dbuenzli]

So if you want to have a closer look from the dev repo do:

opam pin add --dev uucd.14.0.0 
git clone https://erratique.ch/repos/uunf.git 
cd uunf
curl -L https://www.unicode.org/Public/14.0.0/ucdxml/ucd.all.grouped.zip > /tmp/ucd.zip
unzip /tmp/ucd.zip
mv ucd.all.grouped.xml support/ucd.xml
ocaml ./pkg/build_support.ml

Now checkout the file src/uunf_data.ml.

[dbuenzli]

xref ocaml/ocaml#9839

[dbuenzli]

So this also happens on aarch64.

[mro]

same here

#=== ERROR while compiling uunf.14.0.0 ========================================#
# context     2.0.7 | macos/arm64 | ocaml-base-compiler.4.10.2 | https://opam.ocaml.org#88716e72
# path        ~/.opam/4.10.2/.opam-switch/build/uunf.14.0.0
# command     ~/.opam/opam-init/hooks/sandbox.sh build ocaml pkg/pkg.ml build --dev-pkg false --with-uutf true --with-cmdliner true

do you recommend using a more recent switch?

Edit:

setting ulimit did it, however had to be numerical:

$ ulimit -s unlimited
bash: ulimit: stack size: cannot modify limit: Operation not permitted
$ ulimit -s 16384
$ opam install uunf
…
∗ installed uunf.14.0.0

[dbuenzli]

A message has been added in the opam file when in fails in and I hope ocaml/ocaml#11238 will be enough to make that go away with 5.0.0.