-
Notifications
You must be signed in to change notification settings - Fork 413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: use Long_val
for sendfile() parameters to fix file copying in docker
#10333
Conversation
Long_val
for sendfile() parameters to fix file copying in docker
854b501
to
2acf653
Compare
Thanks, I'll try to test this. |
See ocaml#10333 Signed-off-by: Etienne Millon <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked that it fixes the issue with the test in #10334
@dra27 does this look ok or are there any considerations for using |
9333e0f
to
a7f3665
Compare
See ocaml#10333 Signed-off-by: Etienne Millon <[email protected]>
See #10333 Signed-off-by: Etienne Millon <[email protected]>
Signed-off-by: Haoxiang Fei <[email protected]>
Co-authored-by: Etienne Millon <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]>
Signed-off-by: Etienne Millon <[email protected]>
Signed-off-by: Etienne Millon <[email protected]>
This will fix files over 4GB on 64-bit systems, but it will still overflow on 32-bit systems. It's a slightly confusingly-named macro, but The only way to fix this for 32-bit systems is to use an I'm not sure how much that level of 32-bit support really matters for Dune, though? |
Yes, there's no need to bother 32 bit support for dune itself. |
See ocaml#10333 Signed-off-by: Etienne Millon <[email protected]>
…docker (ocaml#10333) * fix: sendfile() in docker Signed-off-by: Haoxiang Fei <[email protected]> * use ssize_t for return value Co-authored-by: Etienne Millon <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> * update test Signed-off-by: Etienne Millon <[email protected]> * Add changelog Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]>
See ocaml#10333 Signed-off-by: Etienne Millon <[email protected]>
…docker (ocaml#10333) * fix: sendfile() in docker Signed-off-by: Haoxiang Fei <[email protected]> * use ssize_t for return value Co-authored-by: Etienne Millon <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> * update test Signed-off-by: Etienne Millon <[email protected]> * Add changelog Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]>
See ocaml#10333 Signed-off-by: Etienne Millon <[email protected]>
…docker (ocaml#10333) * fix: sendfile() in docker Signed-off-by: Haoxiang Fei <[email protected]> * use ssize_t for return value Co-authored-by: Etienne Millon <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> * update test Signed-off-by: Etienne Millon <[email protected]> * Add changelog Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]>
See ocaml#10333 Signed-off-by: Etienne Millon <[email protected]>
…docker (ocaml#10333) * fix: sendfile() in docker Signed-off-by: Haoxiang Fei <[email protected]> * use ssize_t for return value Co-authored-by: Etienne Millon <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> * update test Signed-off-by: Etienne Millon <[email protected]> * Add changelog Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]>
* test: demonstrate overflow with sendfile (#10334) See #10333 Signed-off-by: Etienne Millon <[email protected]> * fix: use `Long_val` for sendfile() parameters to fix file copying in docker (#10333) * fix: sendfile() in docker Signed-off-by: Haoxiang Fei <[email protected]> * use ssize_t for return value Co-authored-by: Etienne Millon <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> * update test Signed-off-by: Etienne Millon <[email protected]> * Add changelog Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]> Co-authored-by: Etienne Millon <[email protected]> * fix formatting (#10356) Signed-off-by: Etienne Millon <[email protected]> * move changelog entry to the right place (#10375) Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Etienne Millon <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> Signed-off-by: Haoxiang Fei <[email protected]> Co-authored-by: Haoxiang Fei <[email protected]>
CHANGES: ### Fixed - Fix overflow in sendfile stubs (copy of large files could fail or end with truncated files) (ocaml/dune#10333, @tonyfettes) - Fix crash when a rule with a directory target is disabled with `enabled_if` (ocaml/dune#10382, fixes ocaml/dune#10310, @gridbugs) - melange: remove all restrictions around virtual libraries in Melange. They may be used as otherwise in libraries and executables. (ocaml/dune#10412, @anmonteiro) - spawn: fix compatibility with RHEL7 (ocaml/dune#10428, @emillon)
I was using dune in docker, and every it copies some large file in a data_only_dirs it reports that
sendfile: No such file or directory
. I tried digging into the source and I think converting the ocaml integer to 32-bit version will make it impossible to copy any file that is larger than 4 GB. I have tested in a docker, but I cannot really confirm whether it really works on real linux machine (I'm using macOS).