transport: reduce memory usage for small reads in server handler

[lqs]

This pull request optimizes memory usage in the HandleStreams method. Previously, a 16KB buffer (http2MaxFrameLen) was allocated for each request and held until the call completed, which caused unnecessary memory waste when handling long-running requests with many concurrent clients, as memory usage remains high throughout the lifetime of the requests.

Changes

• In serverHandlerTransport.HandleStreams, small reads (n < http2MaxFrameLen/4) are now copied into smaller buffers, allowing the larger buffers to be returned to the pool.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.07%. Comparing base (724f450) to head (3099ffd).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7972      +/-   ##
==========================================
+ Coverage   82.05%   82.07%   +0.02%     
==========================================
  Files         381      381              
  Lines       38539    38546       +7     
==========================================
+ Hits        31622    31636      +14     
+ Misses       5602     5591      -11     
- Partials     1315     1319       +4     

Files with missing lines	Coverage Δ
internal/transport/handler_server.go	86.77% <100.00%> (+0.30%)	⬆️

... and 31 files with indirect coverage changes

[arjan-bal]

Hi, @lqs, thanks for sending this PR. Do you have benchmarks showing the effect of this optimization?

We are incurring an extra copy when the bytes read are <= 4KB. We need to decide if the time spent is worth the memory savings.

[lqs]

Hi, @arjan-bal, thanks for the feedback.

Most requests in normal cases are only a few dozen bytes, so the extra copy has almost no impact. Even in the worst case, where each frame is exactly 4KB, the copy would add about 200ns on modern hardware.

If this is a concern, we could reduce the threshold, for example, only copying when the data is smaller than 256 bytes.

[arjan-bal]

gRPC needs to work well with workloads of all types, large and small messages. If optimizing for specific workloads causes negative impact in others, we need to wight the benefits carefully. @dfawley would like to get your thoughts here.

[lqs]

If we want to reduce the impact on other workloads, we could lower the copy threshold further (e.g., 32 bytes) or start with smaller buffers and grow them as needed. Would these approaches work better?

[arjan-bal]

Hi @lqs, sorry for the delay, Doug is on vacation and should be back next week. I would like to get his input on this change.