Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add delay in option for a MOVED response #1189

Closed
ohadisraeli opened this issue Sep 9, 2020 · 4 comments
Closed

Add delay in option for a MOVED response #1189

ohadisraeli opened this issue Sep 9, 2020 · 4 comments

Comments

@ohadisraeli
Copy link
Contributor

When a MOVED response is received from the server currently the client supports only the maxRedirections option.
Adding a delay option can prevent a ping pong effect until the cluster has the new sharding state stabilized.

I think that if we can have the MOVED response to be pushed to a delayed queue like the way some of the other errors are handled as seen below:

    if (errv[0] === 'MOVED' || errv[0] === 'ASK') {
        handlers[errv[0] === 'MOVED' ? 'moved' : 'ask'](errv[1], errv[2]);
    }
    else if (errv[0] === 'TRYAGAIN') {
        this.delayQueue.push('tryagain', handlers.tryagain, {
            timeout: this.options.retryDelayOnTryAgain
        });
    }
@luin
Copy link
Collaborator

luin commented Sep 9, 2020

Hi, I think cases are rare in which the delay option is useful because most commonly, the correct slot target will be returned together with the MOVED error so simply redirect the command to that target would be efficient. This process follows the official spec: https://redis.io/topics/cluster-spec#redirection-and-resharding.

However, I can see how the delay option helps for the cases you mentioned as it "can prevent a ping pong effect" but I'd like to understand your specific case and what would you use as the value of the option.

@ohadisraeli
Copy link
Contributor Author

In our case, we are experiencing under load that we get a MOVED response but the other node has not been yet updated therefore it again redirects to the original node and therefore the ping pong effect. I was thinking of the ability to support a delay parameter. Maybe a constant small number such as 50 might do the trick (currently, we see that the message is sent in about 2ms)

@ohadisraeli
Copy link
Contributor Author

Maybe the change that I have applied in my environment will help to get a better understanding, the original code is commented:

if (errv[0] === 'MOVED' || errv[0] === 'ASK') {
  // handlers[errv[0] === "MOVED" ? "moved" : "ask"](errv[1], errv[2]);
  this.delayQueue.push("moved",handlers.moved.bind(null, errv[1], errv[2]),
    {
      timeout: this.options.retryDelayOnMoved(this.options.maxRedirections + 1 - ttl.value)
    }
  );

@luin
Copy link
Collaborator

luin commented Mar 14, 2021

Closed via #1254

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants