fix: Update line-splitting logic in batched CSV reader

[nameexhaustion]

Updates to use the new CountLines instead of the previous accept_line based logic, which had some incorrect edge cases.

[codspeed-hq]

CodSpeed Performance Report

Merging #19508 will degrade performances by 61.98%

_{Comparing nameexhaustion:batched-csv (f10394c) with main (0f64785)}

Summary

❌ 10 regressions
✅ 31 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	nameexhaustion:batched-csv	Change
❌	test_pdsh_q1	16.4 ms	24.8 ms	-33.71%
❌	test_pdsh_q10	6 ms	8.4 ms	-28.31%
❌	test_pdsh_q14	2.1 ms	3.3 ms	-37.33%
❌	test_pdsh_q15	2.4 ms	3.2 ms	-25.64%
❌	test_pdsh_q22	6.1 ms	16 ms	-61.98%
❌	test_pdsh_q3	5.7 ms	7.8 ms	-27.28%
❌	test_pdsh_q4	4.3 ms	6.2 ms	-30.11%
❌	test_pdsh_q5	4.5 ms	6.3 ms	-28.23%
❌	test_pdsh_q6	1.9 ms	3.6 ms	-48.23%
❌	test_pdsh_q8	5 ms	7.2 ms	-31.24%

[nameexhaustion]

I'm not sure how this was passing before, but there was sort instability for this query

[nameexhaustion]

We use up to a 16MB chunk size now

[nameexhaustion]

I've changed the (initial) chunk size to the default value copied from read_impl from the in-memory CSV reader, and ignoring the user-provided batch_size option.

[ritchie46]

Yes, good one. I benchmarked that one. I think that one is a good cache friendly default.

[codecov]

Codecov Report

Attention: Patch coverage is 92.68293% with 3 lines in your changes missing coverage. Please review.

Project coverage is 79.92%. Comparing base (0f64785) to head (f10394c).
Report is 42 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/polars-io/src/csv/read/read_impl/batched.rs	92.68%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #19508      +/-   ##
==========================================
- Coverage   79.92%   79.92%   -0.01%     
==========================================
  Files        1536     1536              
  Lines      211686   211697      +11     
  Branches     2445     2445              
==========================================
+ Hits       169192   169198       +6     
- Misses      41939    41944       +5     
  Partials      555      555              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nameexhaustion]

I think the Codspeed report shows that we are executing more instructions - but from local testing the wall times show a slight improvement -

Before - python .env/data/x.py 2> /dev/null > /dev/null 6.03s user 0.63s system 600% cpu 1.109 total
After - python .env/data/x.py 2> /dev/null > /dev/null 6.01s user 0.59s system 603% cpu 1.092 total

Note that the reason this affects the timings of test_pdsh_* is because the data preparation phase is using scan_csv().sink_parquet()

[ritchie46]

Nice one @nameexhaustion.