You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am working on an application that produces messages to a single topic that has 100 partitions. The workload is high, I think we would have better performance if there were a few more sockets.
Would it be possible somehow to have 10 sockets? I thought about launching 10 supervisors with respective connection names, and then in Elsa.produce select the connection according to the partition. Each connection would be responsible for 10 partitions (disjoint). That way we guarantee order. So, while all supervisors would spawn 100 processes, effectively 90% of them would be idle (this seems a bit weird, but it is a practical consequence).
Makes sense? Do you have a different recommendation?
The text was updated successfully, but these errors were encountered:
I am working on an application that produces messages to a single topic that has 100 partitions. The workload is high, I think we would have better performance if there were a few more sockets.
Would it be possible somehow to have 10 sockets? I thought about launching 10 supervisors with respective connection names, and then in
Elsa.produce
select the connection according to the partition. Each connection would be responsible for 10 partitions (disjoint). That way we guarantee order. So, while all supervisors would spawn 100 processes, effectively 90% of them would be idle (this seems a bit weird, but it is a practical consequence).Makes sense? Do you have a different recommendation?
The text was updated successfully, but these errors were encountered: