1.0+beta18 "Too many open files" error

[jfeser]

Building core_kernel with jbuilder 1.0+beta18 fails on macos with the following error:

Error: exception Sys_error("/var/folders/_t/3zbr3_hx0sn0tlz6j7rdwdf00000gn/T/jbuilder9d9539.output: Too many open files")

Increasing the file descriptor count to 1024 solves the problem. It seems that jbuilder opens more files now, so the default of 256 is too low.

core_kernel-9805-e7966f.out.txt

[dra27]

Can you test with #578 (opam pin add jbuilder https://github.com/dra27/jbuilder.git#limit-fds)?

[ghost]

@dra27 I'm not sure this will make a big difference. get_std_output sets tail to false, we could try to fill this field properly. Setting to true will cause the process module to close the descriptor as soon as the command has been started.

Another thing I've been thinking about is that currently we might create too much interleaving in the scheduler. This shouldn't be too hard to fix, basically when a command finishes, we need to invoke the continuation before checking computations waiting for a free job slot.

[ghost]

@dra27 actually I missed the Process.File ..., you fix seems good indeed

[jfeser]

#578 fixes the problem on my side.