Minimum number of connections

[Michal-Cho]

Jetty version
9.4.44.v20210927

Java version
java version "11.0.3" 2019-04-16 LTS
Question
We are using jetty client in our project.
From our customers we have a requirement to keep at least 2 connections up per destination all the time (to not waste time for connection a new connection establishing during load peak).
Is there a way to achieve this with jetty? Something like configurable minimum number of connections that jetty client will always try to keep open (even if they are idle). In case a server closes a connection - a reconnection should happen.

Or maybe you could point me to any documentation around this?

Thank you

[sbordet]

Keeping a minimum number of connections open is a feature that is currently not present.
It won't be implemented in Jetty 9.4.x as it is in End of Community Support, see #7958.
It may be implemented on later versions.

However, the use case you describe would not have any problem, as during peak load no time is wasted creating new connections: the 2 that you have will be busy and will never timeout.
And even if they get closed by the application or for errors, during peak load they will be recreated by the next request, which is faster than having a competing mechanism that tries to keep the number of connections at 2.

If you are so idle that all connections timed out, typically you don't care if your request took 1-2 more network roundtrips to complete a request/response, as we are typically talking few hundreds of milliseconds (or less) only.

If you have a more compelling use case I'd like to hear, but seems unnecessary to me.

[joakime]

Seems like you would need cooperation of all of the network equipment (source machine, destination machine, intermediaries) to not close the connection as well (for a variety of idle timeout related features).

That seems like "a big ask" of the network to me.

[Michal-Cho]

hi @sbordet
thank you for a quick answer.

Our use case is related to the spec 5G 29.500 section [5.2.6].
Some bullets from it :

• An HTTP/2 endpoint shall support establishing multiple HTTP/2 connections (at least two) - Already working BUT only if the first connections is fully exhausted (see the next bullet)
• When the HTTP/2 Stream IDs on a given HTTP/2connection is exhausted, an HTTP/2 endpoint, shall establish another
  HTTP/2connection towards that peer HTTP/2 endpoints. - I believe already working OK
• The 3GPP NF shall take care to avoid simultaneous stream ID exhaustion on all the available HTTP/2 connections
  towards each peer - this can happen as if we have only one connection open

If I understand the spec correctly we can not end up in scenario when we want to send a request using client and all connections have already used all valid StreamID and there is a need to create a new connection.

So my idea was to have a second connection ready in stand by and when the first one reaches ~90% of valid StreamsID the second connection takes over, a third one is being created to be a new standby and the first one is left just to handle all pending responses and will timeout as idle after some time.

I will be grateful for any hits/ suggestions
BR

[sbordet]

There is a much simpler solution.
We support AbstractConnectionPool.setMaxUsageCount(int). It counts how many times a connection is borrowed from the pool, and it's only borrowed to send a request.

So you can set it to your 90% of valid stream ids (or less as that number is quite big), and when the usage reaches the max the connection will be closed.
Note that this will require some atomicity (which we do in our implementation), since you don't want to allow borrowing a connection on one thread that is being closed by another thread for over-usage.
Any other "take over" done at the application level should be carefully locked, so I suggest you use the existing feature instead.

[Michal-Cho]

Thank you for a very fast replay.
But I'm not sure if this solves the problem of simultaneous exhaustion.
"when the usage reaches the max the connection will be closed" -> that means that if we need to send another request there is a waiting time for re-establishing the connection. I know that it should not take a lot of time but still.
I guess I have to better understand what "The 3GPP NF shall take care to avoid simultaneous stream ID exhaustion on all the available HTTP/2 connections towards each peer." really means.
If it is about zero waiting time for an established connection that can be used for sending a request then AbstractConnectionPool.setMaxUsageCount(int) does not really help as it will just speed up recreation of the connection right?
Or does it work like it triggers creation of new connection in the pool but the existing when while still be available to be pick up until the second one is ready?

[Michal-Cho]

one more comment - I've found something called

preCreateConnections​(int connectionCount) Optionally pre-create up to connectionCount connections so they are immediately ready for use.

Do you think this is something that I could use?
I've found some discussion here
#6512
I will try to investigate that

[sbordet]

There is no simultaneous exhaustion. Jetty's connection pool work by default by using the first available connection.

When the first is exhausted you may have the second already available, or amortize the creation of the connection over, possibly, 1 billion requests.

You may call preCreateConnections(int) when you reach a certain usage level (that you have to maintain in the application), but there is no guarantee that the connections created in this way may be used before they get idle timed out.

[Michal-Cho]

Thank you @sbordet for the suggestion.

In the end I've implemented some workaround - we have switched to RoundRobin pool.
I've created a ConnectionPool.Factory implemenation that returns my own pool extending RoundRobinConnectionPool.
The factory use preCreateConnections(int) in newConnectionPool(HttpDestination destination).

On top of that I've added an extra thread to the actual pool that runs periodically and checks if getConnectionCount() returns value lower than number of connections that we want to keep. If it is the case it again uses preCreateConnections(int) to create missing ones.

[sbordet]

Great you could make it work.

FTR, I still think it's a bit overkill and that the 3GPP specification is too vague about the requirement (and you worry too much after reading the spec), but fine.

Feel free to close the issue if it's resolved for you.