Skip to content
This repository has been archived by the owner on Apr 1, 2024. It is now read-only.

ISSUE-13306: PIP-120: Enable client memory limit by default #3428

Closed
sijie opened this issue Dec 14, 2021 · 0 comments
Closed

ISSUE-13306: PIP-120: Enable client memory limit by default #3428

sijie opened this issue Dec 14, 2021 · 0 comments
Labels

Comments

@sijie
Copy link
Member

sijie commented Dec 14, 2021

Original Issue: apache#13306


Motivation

In Pulsar 2.8, we have introduced a setting to control the amount of memory
used by a client instance.

interface ClientBuilder {
    ClientBuilder memoryLimit(long memoryLimit, SizeUnit unit);
}

By default, in 2.8 and 2.9 this setting is set to 0, meaning no limit is being
enforced.

I think it's a good time for 2.10 to enable this setting by default and,
correspondingly, to disable by default the producer queue size limit.

This will simplify a lot the configuration that a producer application will
have to come up with, when publishing with many topic/partitions or when messages
are bigger than expected.

Proposed changes

In 2.10 release, for the ClientBuilder, change
  * memoryLimit: 0 -> 64 MB

For the ProducerBuilder, changes
  * maxPendingMessages: 1000 -> 0

64MB is picked because it's a small enough memory size that will guarantee
a very high producer throughput, irrespective of the individual messages size.

@sijie sijie added the PIP label Dec 14, 2021
@sijie sijie closed this as completed Jan 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant