[auto_discovery] process templates larger than the page buffer size

[truthbk]

We must be able to process templates as large as necessary - customer encountered we were just processing 64K configurations at most when reading from the pipe - consistent with the default page buffer on linux. Let's read the entire buffer... Also, this is necessary to avoid blocking the writing side indefinitely.

[theduderog]

Is there a possible race condition here where adPipe.available() might return 0 after the read but before the writer is unblocked? Would it be safer to introduce a separator to look for?

[olivielpeau]

hmm good point, what do you think @truthbk?

[truthbk]

I'd have to look at the internals, I'm not sure. I haven't encountered issues while testing, but this is a valid point.

[olivielpeau]

LGTM! Feel free to merge once you've removed the unused variable ;)

[olivielpeau]

you can remove this var

[truthbk]

After taking a look at the internals, I believe @theduderog was indeed correct and we will need to use a separator to guarantee we know when no more data is expected in the iteration. Will have to fix that here, and in the dd-agent side.

O_NONBLOCK disabled, n > PIPE_BUF
The write is nonatomic: the data given to write(2) may be interleaved with write(2)s by other process; the write(2) blocks until n bytes have been written.
O_NONBLOCK enabled, n > PIPE_BUF
If the pipe is full, then write(2) fails, with errno set to EAGAIN. Otherwise, from 1 to n bytes may be written (i.e., a "partial write" may occur; the caller should check the return value from write(2) to see how many bytes were actually written), and these bytes may be interleaved with writes by other processes.

We have a single writer, so it'd be less problematic, that said, better safe than sorry - adding a separator/terminator.