std::io::Write::write_vectored() does not properly limit the number of iovecs.

[rustyconover]

The callers of std::io::Write::write_vectored() should not have to go through the effort of determining what the maximum number of iovec structures can be passed to the underlying system call. As demonstrated by this example:

use std::fs::File;
use std::io::prelude::*;
use std::io::{BufWriter, IoSlice};

fn main() {
    let mut file = BufWriter::new(File::create("./example-output.txt").unwrap());
    let mut write_array: Vec<IoSlice> = Vec::new();
    for _ in 1..1024 * 5 {
        write_array.push(IoSlice::new(b"1"));
        write_array.push(IoSlice::new(b"\n"));
    }
    file.write_vectored(&write_array).unwrap();
    file.flush().unwrap();
}

The output is:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 22, kind: InvalidInput, message: "Invalid argument" }', src/libcore/result.rs:1165:5
stack backtrace:
   0:        0x10621f175 - backtrace::backtrace::libunwind::trace::hb16ec6045891ce5a
                               at /Users/runner/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1:        0x10621f175 - backtrace::backtrace::trace_unsynchronized::hcacbd0efdffd74c6
                               at /Users/runner/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2:        0x10621f175 - std::sys_common::backtrace::_print_fmt::h39e22de9d6757d12
                               at src/libstd/sys_common/backtrace.rs:77
   3:        0x10621f175 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h415ddd0ba88caaaf
                               at src/libstd/sys_common/backtrace.rs:61
   4:        0x106236cb0 - core::fmt::write::h3335552e2df81c1d
                               at src/libcore/fmt/mod.rs:1028
   5:        0x10621db6b - std::io::Write::write_fmt::he6837371b9a45188
                               at src/libstd/io/mod.rs:1412
   6:        0x106220d43 - std::sys_common::backtrace::_print::h89459d14ba97f5fa
                               at src/libstd/sys_common/backtrace.rs:65
   7:        0x106220d43 - std::sys_common::backtrace::print::ha4c6688e811b8829
                               at src/libstd/sys_common/backtrace.rs:50
   8:        0x106220d43 - std::panicking::default_hook::{{closure}}::h708e66cfeb0483ba
                               at src/libstd/panicking.rs:188
   9:        0x106220a4a - std::panicking::default_hook::h39ea8ddf674c04ec
                               at src/libstd/panicking.rs:205
  10:        0x10622134b - std::panicking::rust_panic_with_hook::h9db77b22c2255a16
                               at src/libstd/panicking.rs:464
  11:        0x106220ed9 - std::panicking::continue_panic_fmt::h2dfa3a5b90265361
                               at src/libstd/panicking.rs:373
  12:        0x106220e29 - rust_begin_unwind
                               at src/libstd/panicking.rs:302
  13:        0x1062397cc - std::panicking::begin_panic::h8518f1142ed7061c
  14:        0x1062398a9 - std::panicking::begin_panic::h8518f1142ed7061c
  15:        0x1062181c1 - core::result::Result<T,E>::unwrap::h79b719002edcbf6d
                               at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libcore/result.rs:933
  16:        0x106214a2a - vector_bug::main::h3df611f13f1e9a88
                               at src/main.rs:12
  17:        0x106211c92 - std::rt::lang_start::{{closure}}::h4e4da6e08497b9a6
                               at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/rt.rs:61
  18:        0x106220e08 - std::rt::lang_start_internal::{{closure}}::hccd7db6d8a0ebab5
                               at src/libstd/rt.rs:48
  19:        0x106220e08 - std::panicking::try::do_call::hd5a3af8d00c06681
                               at src/libstd/panicking.rs:287
  20:        0x10622281f - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:78
  21:        0x1062216de - std::panicking::try::h7a0bd4c078131d2f
                               at src/libstd/panicking.rs:265
  22:        0x1062216de - std::panic::catch_unwind::h75c3fbe62776ab10
                               at src/libstd/panic.rs:396
  23:        0x1062216de - std::rt::lang_start_internal::haa52aabac43378ff
                               at src/libstd/rt.rs:47
  24:        0x106211c72 - std::rt::lang_start::h34001aec5ec8c720
                               at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/rt.rs:61
  25:        0x106214ac2 - vector_bug::main::h3df611f13f1e9a88

The underlying system call of writev() has limit on the number of iovec items that can be passed specifically the man page for writev() has

POSIX.1-2001 allows an implementation to place a limit on the number of items that can be passed in iov. An implementation can advertise its limit by defining IOV_MAX in <limits.h> or at run time via the return value from sysconf(_SC_IOV_MAX). On Linux, the limit advertised by these mechanisms is 1024, which is the true kernel limit. However, the glibc wrapper functions do some extra work if they detect that the underlying kernel system call failed because this limit was exceeded. In the case of readv() the wrapper function allocates a temporary buffer large enough for all of the items specified by iov, passes that buffer in a call to read(2), copies data from the buffer to the locations specified by the iov_base fields of the elements of iov, and then frees the buffer. The wrapper function for writev() performs the analogous task using a temporary buffer and a call to write(2).

When running on Mac OS X it appears the libc implementation is not doing the extra work of determining the limit of the number of items that can be used in one call of writev(). On Mac OS X this limit is defined by UIO_MAXIOV.

Looking at the unix implementation of write_vectored it seems that line 113 is the problem.

rust/src/libstd/sys/unix/fd.rs

Lines 108 to 117 in 3982d35

	pub fn write_vectored(&self, bufs: &[IoSlice<'_>]) -> io::Result<usize> {
	let ret = cvt(unsafe {
	libc::writev(
	self.fd,
	bufs.as_ptr() as *const libc::iovec,
	cmp::min(bufs.len(), c_int::max_value() as usize) as c_int,
	)
	})?;
	Ok(ret as usize)
	}

The code that assumes the upper limit of the number of iovecs being passed can equal c_int::max_value() is too large. It should either be limited to UID_MAXIOV on Max OS X or the result of sysconf(_SC_IOV_MAX) on Linux.

Changing this logic will help the logic work such that any number of IoSlices can be passed to io::std::Write::write_vectored().