ArrayPool degradation above 70% memory utilization

[jkotas]

From https://gitter.im/dotnet/corefx?at=5d043a468e050f62aa1ccc18 :

using the arraypool is great in theory but since it responds to gc pressure once you start getting collections you lose most benefit from it.

[jkotas]

In worst case, the current array pool policies may send you on a death spiral above 70% memory utilization: The array pool will start flushing itself after every Gen2 GC, that produces a lot of garbage that needs to be collected, that triggers more GCs, that causes more trimming, …

[stephentoub]

@jeffschwMSFT, why did you move this issue to corefx? The ArrayPool implementation is in coreclr.

[jeffschwMSFT]

@stephtoub my mistake... moving it back :)

[stephentoub]

😄 (single repo for the win)

[JeremyKuhne]

I'm not sure what we can do here in a "global" sense. If the GC is under significant pressure the ArrayPool rooting a bunch of otherwise unrooted objects can kill performance or lead to OOM conditions as well. Adding the collection under pressure logic had a significant positive impact.

You can already create your own private pool, which doesn't collect (i.e. ArrayPool<T>.Create()), is that enough to address the concern here?

We're already doing some logic to favor larger arrays. We could try to add more logic to tweak our aggressiveness, but I worry about unintended consequences. Not saying that we shouldn't do anything, I'm just voicing my concerns. I'd love to see more detailed scenarios.

@Maoni0 do you have any suggestions on how to play nicer with the GC?

[jkotas]

create your own private pool, which doesn't collect (i.e. ArrayPool.Create()), is that enough to address the concern here?

That does not help. Close to none of the APIs that use ArrayPool as implementation detail take a custom array pool as an input.

[VSadov]

Perhaps when trimming due to pressure, the pool size should be temporarily reduced?

Like:

• add another nonconstant limit that is typically the same as MaxBuffersPerArraySizePerCore and use that for how much can actually be retained.
• every time you need to panic-trim in Gen2 reduce that limit by half before reallocating the buckets.
• increase the limit by 1 up to MaxBuffersPerArraySizePerCore every time you survive Gen2 without a need to trim.

Just a thought...

[joperezr]

@jkotas I suppose this is not new for 3.0 or a regression right? I just ask because I was about to triage this as Future but wanted to double check.

[jkotas]

Correct. This is not a regression.

[dotnet-policy-service]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.

[dotnet-policy-service]

This issue will now be closed since it had been marked no-recent-activity but received no further activity in the past 14 days. It is still possible to reopen or comment on the issue, but please note that the issue will be locked if it remains inactive for another 30 days.