Test the Unix binding

[aantron]

NOTE: this will be much easier to work on after mirage/alcotest#177 and #701.

This is a big issue for lots of PRs. Some parts of it could be quite easy, but on the whole I'd say it is of medium difficulty.

Some care should be taken to run the tests properly sandboxed, i.e. files created in temporary directories, etc. For that, the test framework in test/test.ml will probably have to be extended with new helper functions. I'm not sure how to sandbox DNS and some of the other APIs. We should probably discuss that.

Besides correct behavior, we need really thorough tests for various error conditions. The typical system call has man pages on multiple systems, listing the various ways the system call can fail. We want to trigger as many of those as possible. See section Help below for links to man pages.

The work

To make this manageable, I would break the work into chunks, based on the current order of functions in the various .mli files of the Unix binding. These aren't listed necessarily in the order they should be tested in. We generally want simple and obvious things tested first, and more complex things that depend on them tested later.

There are a few tests already written, but I included all the functions in the category list, so make sure to check before choosing a category. I'll try to keep the check boxes up to date.

If you'd like to work on one of these items, please leave a comment, to avoid duplication of effort :)

• Bigstrings. This is pure OCaml, and should be easy. (Test unix bytes #619, by @cedlemo)
• Sleeping and timeouts, probably best tested with the help of Unix.gettimeofday. (Test lwt unix sleeping and timeout functions #630, by @cedlemo)
• Lwt_throttle. I guess this is in the Unix binding because it is aware of real time.
• Lwt_timeout.
• Conversion form existing fds. A basic test can just be a round-trip of a Unix fd, but really thorough testing here would have to exercise the optional arguments, and so this might depend on testing read or write first.
• Blocking mode.
• Lwt_unix.close, Lwt_unix.abort, and Lwt_unix.state.
• Low-level subprocesses.
• Preemptive threading (Lwt_preemptive).
• Subprocesses (Lwt_process).
• Standard I/O streams. This could be as easy as applying Lwt_unix.unix_file_descr to them, and checking for equality with Unix.stdout, etc.
• Lwt_unix.openfile. This is tricky, because there are many options to test, some of them are platform-specific, etc.
• Basic I/O.
• Vector I/O. This was already done when vector I/O was added in writev – zero-copy gather output #291, but if you see more things to test, please test them :)
• I/O status.
• Seeking; and for large files.
• fsync.
• fstat; and for large files.
• File names.
• Permissions.
• File descriptors.
• Directories.
• Pipes.
• Symlinks.
• lockf.
• Users. (Add tests for user id and group id #673, by @anuragsoni)
• Signals.
• Sockets. There are a lot of APIs in this category, and it's probably the most important I/O overall in modern programming.
• Bigstring I/O. (Test lwt bytes second part #621, by @cedlemo)
• Memory mapping. (Test lwt bytes third part #628, by @cedlemo)
• Multicast. I believe we already have decent tests here, from Lwt_unix: add support for joining/leaving multicast groups #143.
• DNS.
• Terminals.
• CPUs.
• Low-level interface. Among other things, this is used by external libraries to "plug into" Lwt.

I've left Lwt_io for a separate issue.

Help

Cheating off Unix

Some of the easiest ways to test some bindings would be to do something with Lwt_unix (or other Lwt module), and check the result using stdlib's Unix.

man pages

Here are some man page references:

• Linux
• Mac It seems Apple has archived their man pages and made them difficult to discover, so you have to do a web search for each system call.
• BSD
• For Windows, it's probably best to look at what Win32 API Lwt actually calls in src/unix/windows_c/*, and then look the API up in MSDN. A search is probably the easiest way.

Skipping tests

Many Lwt APIs aren't implemented on Windows. Tests for them can be skipped like this:

lwt/test/unix/test_lwt_process.ml

Line 27 in 4f0dabc

test "lazy_undefined" ~only_if:(fun () -> not Sys.win32)

Coverage analysis

Coverage analysis is enabled for the OCaml code of the Unix binding – run make coverage, then open _coverage/index.html. This can help to see what still needs testing, though some of the code is visited unintentionally because other code depends on it. We don't have coverage analysis for C code at the moment. That could be its own PR, but maybe it's not necessary.

Of course, high coverage doesn't imply thorough testing. Only low coverage implies insufficient testing.

cc @cedlemo

[cedlemo]

@aantron I will work on Bigstrings.

[cedlemo]

Now I will work on "sleeping and timeout".

[cedlemo]

I continue with Lwt_throttle.

[anuragsoni]

@aantron i've added some tests for unix Users (#673 )