Add option to forward rpcs from Pggate to the local Tserver. #7794
Assignees
Comments
|
This change is useful for large clusters like our TPCC cluster with 60 nodes. Instead of this if we have all the Postgres processes talk to the local Tserver and the Tserver forwarding the request to the proper Tserver we will have 60 * 200 * 1 = 12k connections. |
psudheer21
pushed a commit
to psudheer21/yugabyte-db
that referenced
this issue
May 19, 2021
Each postgres process connects to all the Tservers in the cluster which leads to a quadratic growth in the number of connections in the cluster. To reduce the number of connections, we can forward all the rpcs to the local Tserver which can then forward the rpc to the appropriate Tserver. This feature involves 2 changes: Forward all the rpcs from pggate to the local tserver. Ensure the local tserver sends the rpcs to the appropriate tserver. First change involves creating a RemoteTabletServer for the local tserver and using that for all the PGSQL read/write rpcs. This needs the postgres process to be passed the UUID of the tserver process so that it can identify the HostPort of the local tserver from the list of all tservers. The second change involves creating the ForwardRpc class which forwards the read/write request protobufs to the appropriate tserver by using the TabletInvoker. This class is invoked by the TabletServiceImpl on realizing that the request is not meant for itself. This feature is controlled by the GFLAG: ysql_forward_rpcs_to_local_tserver which allows us to completely enable/disable the forwarding of rpcs to the local tserver. Test Plan: Java tests for pggate Reviewers: amitanand, bogdan, mihnea, timur, dmitry Subscribers: yql, ybase
psudheer21
pushed a commit
that referenced
this issue
May 21, 2021
Summary: Each postgres process connects to all the Tservers in the cluster which leads to a quadratic growth in the number of connections in the cluster. To reduce the number of connections, we can forward all the rpcs to the local Tserver which can then forward the rpc to the appropriate Tserver. This feature involves 2 changes: 1. Forward all the rpcs from pggate to the local tserver. 2. Ensure the local tserver sends the rpcs to the appropriate tserver. First change involves creating a RemoteTabletServer for the local tserver and using that for all the PGSQL read/write rpcs. This needs the postgres process to be passed the UUID of the tserver process so that it can identify the HostPort of the local tserver from the list of all tservers. The second change involves creating the ForwardRpc class which forwards the read/write request protobufs to the appropriate tserver by using the TabletInvoker. This class is invoked by the TabletServiceImpl on realizing that the request is not meant for itself. This feature is controlled by the GFLAG: ysql_forward_rpcs_to_local_tserver which allows us to completely enable/disable the forwarding of rpcs to the local tserver. Test Plan: All java tests for pggate. Reviewers: amitanand, rskannan, rsami, timur, dmitry, mihnea Reviewed By: mihnea Subscribers: dmitry, zyu, bogdan, yql Differential Revision: https://phabricator.dev.yugabyte.com/D10274
YintongMa
pushed a commit
to YintongMa/yugabyte-db
that referenced
this issue
May 26, 2021
Summary: Each postgres process connects to all the Tservers in the cluster which leads to a quadratic growth in the number of connections in the cluster. To reduce the number of connections, we can forward all the rpcs to the local Tserver which can then forward the rpc to the appropriate Tserver. This feature involves 2 changes: 1. Forward all the rpcs from pggate to the local tserver. 2. Ensure the local tserver sends the rpcs to the appropriate tserver. First change involves creating a RemoteTabletServer for the local tserver and using that for all the PGSQL read/write rpcs. This needs the postgres process to be passed the UUID of the tserver process so that it can identify the HostPort of the local tserver from the list of all tservers. The second change involves creating the ForwardRpc class which forwards the read/write request protobufs to the appropriate tserver by using the TabletInvoker. This class is invoked by the TabletServiceImpl on realizing that the request is not meant for itself. This feature is controlled by the GFLAG: ysql_forward_rpcs_to_local_tserver which allows us to completely enable/disable the forwarding of rpcs to the local tserver. Test Plan: All java tests for pggate. Reviewers: amitanand, rskannan, rsami, timur, dmitry, mihnea Reviewed By: mihnea Subscribers: dmitry, zyu, bogdan, yql Differential Revision: https://phabricator.dev.yugabyte.com/D10274
d-uspenskiy
added a commit
that referenced
this issue
Jul 19, 2022
Summary: YSQL uses `PgClientService` to communicate with remote t-server(s)/master(s). Having `TabletServerForwardService` is not necessary. The diff reverts 6077bc8 / D10274 Test Plan: Jankins Reviewers: mihnea, timur, sergei, alex Reviewed By: sergei, alex Subscribers: yql, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D18289
|
Local proxy to communicate with remote t-servers/masters was introduced in context of #9936 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Each postgres process connects to all the Tservers in the cluster which
leads to a quadratic growth in the number of connections in the cluster.
To reduce the number of connections, we can forward all the rpcs to the
local Tserver which can then forward the rpc to the appropriate Tserver.
The text was updated successfully, but these errors were encountered: