Flaky Test: TestMaxBytesLimit intermittently fails

[taylor-swanson]

Flaky Test

• Test Name: TestMaxBytesLimit
• Link: libbeat/reader/readfile/line_test.go
• Branch: main
• Notes: This test may have been broken by a recent change introduced in [Filebeat] keep track of bytes read when max_bytes exceeded #31824. The test seems to fail roughly 30% of the time, but this can highly vary. It's best to run the test with the -count flag, otherwise the test will likely pass, and Go will cache the results.

cd libbeat/reader/readfile
go test -test.run TestMaxBytesLimit -count 100 .

Error

--- FAIL: TestMaxBytesLimit (0.00s)
    line_test.go:402: the bytes read are not equal to the bytes input, expected: 81273 got: 76439

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[cmacknz]

We'll try to look at this today. If not we can temporarily revert the change that triggered it.

[cmacknz]

I started investigating this and I think there are multiple problems involved here. Part of the reason this test is flaky is that the test generates input randomly, and it doesn't always generate it in a way that always hits the "max bytes exceeded" path.

I am also not convinced that this code path works entirely correctly even without this change.

If the flakyness here is getting in the way before I can create a PR to fix it, just revert the change and I'll restore it afterwards. I didn't get to completely fixing this today.

[liuwenping]

sorry, I'll fix this problem today.

[cmacknz]

Fixed by #31882