Make max-queued-requests and max-retry-delay configurable

[baurmatt]

New feature: Puppet agents can retry requests on a configurable delay if Puppet Server is busy

When a group of Puppet agents start their Puppet runs together, they can form a “thundering herd” capable of exceeding Puppet Server’s available resources. This results in a growing backlog of requests from Puppet agents waiting for a JRuby instance to become free before their request can be processed. If this backlog exceeds the size of the Server’s Jetty thread pool, other requests (such as status checks) start timing out. (For more information about JRubies and Server performance, see Applying metrics to improve performance.)

In previous versions of Puppet Server, administrators had to manually remediate this situation by separating groups of agent requests, for instance through rolling restarts. In Server 5.1.0, administrators can optionally have Server return a 503 response containing a Retry-After header to requests when the JRuby backlog exceeds a certain limit, causing agents to pause before retrying the request.

Both the backlog limit and Retry-After period are configurable, as the max-queued-requests and max-retry-delay settings respectively under the jruby-puppet configuration in puppetserver.conf. Both settings’ default values do not change Puppet Server’s behavior compared to Server 5.0.0, so to take advantage of this feature in Puppet Server 5.1.0, you must specify your own values for max-queued-requests and max-retry-delay. For details, see the puppetserver.conf documentation. Also, Puppet agents must run Puppet 5.3.1 or newer to respect such headers.

https://puppet.com/docs/puppet/5.3/release_notes.html#new-feature-puppet-agents-can-retry-requests-on-a-configurable-delay-if-puppet-server-is-busy

Both need to be set in the jruby-puppet section:
max-retry-delay: https://github.com/puppetlabs/puppetserver/blob/6842080c7c1cc5a11560e5263ff519569dd29c24/src/clj/puppetlabs/services/request_handler/request_handler_service.clj#L20
max-queued-requests: https://github.com/puppetlabs/puppetserver/blob/6842080c7c1cc5a11560e5263ff519569dd29c24/src/clj/puppetlabs/services/request_handler/request_handler_service.clj#L19

I will provide a PR.