Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][SDK] Optimize the implementation of node connection management #11595

Closed
2 tasks done
Tracked by #11564
gosonzhang opened this issue Dec 11, 2024 · 0 comments · Fixed by #11596
Closed
2 tasks done
Tracked by #11564

[Improve][SDK] Optimize the implementation of node connection management #11595

gosonzhang opened this issue Dec 11, 2024 · 0 comments · Fixed by #11596

Comments

@gosonzhang
Copy link
Contributor

Description

Currently, the node management of DataProxy SDK adopts regular node updates and adds node load status mode for management[1].

From the SDK perspective, it can only see the number of nodes in the cluster and the data reporting status of the nodes currently connected to it. Therefore, the connection management is optimized here: randomize the selected nodes and increase the regular forced node selection; monitor the connected nodes to see if too many requests are blocked and whether the sending buffer size is exceeded.

In addition, considering that the initialization may fail when constructing Sender and ClientMgr objects, in order to avoid thread residue, the thread startup and other operations involving possible failures are placed in the start() function.

  1. https://github.com/apache/inlong/blob/master/inlong-sdk/dataproxy-sdk/src/main/java/org/apache/inlong/sdk/dataproxy/network/ClientMgr.java

InLong Component

InLong SDK

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant