Remove Object Pool

[benjchristensen]

PR #1944 rebased onto 1.x.

This removes the use of object pooling as per discussion in #1908 to achieve correctness. Performance stats will follow in comments.

[benjchristensen]

Here is the performance hit on one of the scenarios that benefits the most from object pooling:

1.x

Benchmark                                    (size)   Mode   Samples        Score  Score error    Units
r.o.OperatorMergePerf.merge1SyncStreamOfN         1  thrpt         5  4493516.238   855611.781    ops/s
r.o.OperatorMergePerf.merge1SyncStreamOfN      1000  thrpt         5    39848.200     3011.983    ops/s
r.o.OperatorMergePerf.merge1SyncStreamOfN   1000000  thrpt         5       38.017        2.865    ops/s


No Object Pool

Benchmark                                    (size)   Mode   Samples        Score  Score error    Units
r.o.OperatorMergePerf.merge1SyncStreamOfN         1  thrpt         5  4809388.856   164553.287    ops/s
r.o.OperatorMergePerf.merge1SyncStreamOfN      1000  thrpt         5    41017.885     1385.965    ops/s
r.o.OperatorMergePerf.merge1SyncStreamOfN   1000000  thrpt         5       39.981        2.307    ops/s

The following Flight Recorder comparisons show the memory and GC impact. This is what concerns me.

With Pooling

Without

With Pooling

Without

With Pooling

Without

[benjchristensen]

This test is particularly bad: OperatorMergePerf.mergeNAsyncStreamsOfN

With Pooling

Without

With Pooling

Without

[benjchristensen]

That last set of Flight Recorder tests suggests we can't do this.

[akarnokd]

I see a few options:

• longer lasting merge sessions,
• do backpressure outside the merge, and have the plain no-backpressure merge,
• use unpadded and smaller queues in merge.

[benjchristensen]

I don't see how any of those are options. Can you please elaborate?

It seems to me the option is either make pooling work or use linked lists.

[akarnokd]

• Longer lasting merge sessions: if the merge sources are always the same, create one of it and funnel multiple datastreams through it, each stream having its on custom end marker instead of onCompleted so the merge is not torn down. For example, a single incoming request is always split to 3 concurrent sources which are then merged together and the response is formed after all 3 finished. Instead of merging their plain values, merge a the (request-id, value) then split the resulting streams back to each requester.
• Change the emission rate in some way. For example, a cache might do its own backpressure directly so no downstream op has to deal with it and queue again.
• SpscLinkedQueue and SpscArrayQueue use significant padding which adds to the instantiation and in-flight overhead due allocation. Just stripping these of their padding may help. In addition, if the number emitted objects to merge is low compared to the cost of setting up the operators, you could reduce or completely remove the queueing from the merge (a new merge op with queue parameter, or a new merge op without queues and without backpressure).