net: skip excessive read calls

[dvyukov]

Currently we always call syscall.Read before blocking on epoll in net.Conn.Read. This is
not necessary for connection-oriented sockets. According to man 7 epoll:

"For stream-oriented files (e.g., pipe, FIFO, stream socket), the condition that
the read/write I/O space is exhausted can also be detected by checking the amount of
data read from / written to the target file descriptor. For example, if you call read(2)
by asking to read a certain amount of data and read(2) returns a lower number of bytes,
you can be sure of having exhausted the read I/O space for the file descriptor. The same
is true when writing using write(2). (Avoid this latter technique if you cannot
guarantee that the monitored file descriptor always refers to a stream-oriented
file.)"

So if a previous Read has exhausted IO space, the chances are it's still exhausted, and
we can skip syscall.Read and wait on epoll. If new data has arrived since last Read,
waitRead will just instantly return, so we don't lose anything.

The change is actually quite trivial:
https://golang.org/cl/154040043/diff/40001/src/net/fd_unix.go
But the last time I tested it, Reads were hanging episodically with this change.
It can be due to a bug in the CL.
Or due to: https://golang.org/issue/6336
Or due to a kernel bug.
The change needs to be retested.
Most likely we also need to disable the optimization on level-triggered solaris. Afact,
kqueue should be fine, but that needs to be tested as well.

[ALTree]

Closing in favour of #15735 (net: add mechanism to wait for readability on a TCPConn), which has a milestone and discussion and people in it.