Increase HTTP/2 stream pool max size to 100

[JamesNK]

Addresses #13750

A benchmark with 100 concurrent requests on a connection uses all 100 streams. The previous lower value resulted in streams not being reused because the pool was at capacity. They are garbage collected and new streams have to be created.

Before:

After:

[jkotalik]

I'm not against this change but we should think about it. This means we will now have significantly more idle memory per connection. For example, let's say 100 requests were sent from the client to the server (100 streams) and then the connection was left open afterwards. We would now have 100 stream objects pooled but not being used for the connection.

I agree for the benchmark it makes sense to have 100, but for normal use cases I'm not convinced yet. How much memory is idled between 40 and 100 streams?

[JamesNK]

What do we think of supporting expiring pooled streams?

When a stream is pooled we set the drain timeout (just to reuse an existing property), and every heartbeat we look at the oldest streams in the pool and remove all that exceeds the timeout.

For example, if a stream sits in the pool unused for longer than 5 seconds then it gets removed.

[halter73]

How much memory is idled between 40 and 100 streams?

We should collect a profile and get a real number for this, but I think it's safe to say the memory usage for streams on an idle connection that previously had at least 100 concurrent streams open (the worst case) will be at most 2.5x worse what it was before.

What do we think of supporting expiring pooled streams?

I think that's a great idea. @jkotalik also suggested "Pruning pool over time for long running connections" when he first added the 40 stream pool (#18601) though it wasn't really discussed much then IIRC. I think it can be done in a separate PR though as it's something we should do whether or not this PR is merged.

Will add feature to expiry old pooled streams in followup PR

[jkotalik]

Sounds good!