[bitnami/redis] sentinel cannot elect new master

[KaKeimei]

Which chart:
redis-14.3.1

Describe the bug
Sentinel can not elect new master when the master pod is deleted

To Reproduce
Steps to reproduce the behavior:

1. Use sentinel with 3 replicas
2. when all replica ready, delete the master pod(kubectl delete pod redis-test-node-0).
3. The rest of the pod cannot failover

Expected behavior
The remaining two pod should elect a new master.

Version of Helm and Kubernetes:

• Output of helm version:

version.BuildInfo{Version:"v3.5.2", GitCommit:"167aac70832d3a384f65f9745335e9fb40169dc2", GitTreeState:"dirty", GoVersion:"go1.15.7"}

• Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:28:09Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.9-eks-d1db3c", GitCommit:"d1db3c46e55f95d6a7d3e5578689371318f95ff9", GitTreeState:"clean", BuildDate:"2020-10-20T22:18:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Additional context
logs for node-2 sentinel after deleted master node(node-0)

1:X 27 May 2021 11:08:12.199 # +odown master mymaster 10.0.101.172 6379 #quorum 2/2
1:X 27 May 2021 11:08:12.199 # +new-epoch 1
1:X 27 May 2021 11:08:12.199 # +try-failover master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:08:12.202 # +vote-for-leader 98b74b06e569ff7da63faa603ee1710cd6571935 1
1:X 27 May 2021 11:08:12.209 # 17127a5c54d86eedede4aeebec76970aab9d4ad8 voted for 98b74b06e569ff7da63faa603ee1710cd6571935 1
1:X 27 May 2021 11:08:12.293 # +elected-leader master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:08:12.293 # +failover-state-select-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:08:12.346 # -failover-abort-no-good-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:08:12.417 # Next failover delay: I will not start a failover before Thu May 27 11:08:48 2021
1:X 27 May 2021 11:08:43.122 # +reset-master master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:08:44.680 * +sentinel sentinel 17127a5c54d86eedede4aeebec76970aab9d4ad8 10.0.101.129 26379 @ mymaster 10.0.101.172 6379
1:X 27 May 2021 11:09:43.144 # +sdown master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:09:48.219 # +new-epoch 2
1:X 27 May 2021 11:09:48.222 # +vote-for-leader 17127a5c54d86eedede4aeebec76970aab9d4ad8 2
1:X 27 May 2021 11:09:48.381 # +odown master mymaster 10.0.101.172 6379 #quorum 2/2
1:X 27 May 2021 11:09:48.381 # Next failover delay: I will not start a failover before Thu May 27 11:10:24 2021
1:X 27 May 2021 11:10:24.281 # +new-epoch 3
1:X 27 May 2021 11:10:24.285 # +vote-for-leader 17127a5c54d86eedede4aeebec76970aab9d4ad8 3
1:X 27 May 2021 11:10:24.297 # Next failover delay: I will not start a failover before Thu May 27 11:11:00 2021
1:X 27 May 2021 11:11:00.555 # +new-epoch 4
1:X 27 May 2021 11:11:00.558 # +vote-for-leader 17127a5c54d86eedede4aeebec76970aab9d4ad8 4
1:X 27 May 2021 11:11:00.575 # Next failover delay: I will not start a failover before Thu May 27 11:11:36 2021
1:X 27 May 2021 11:11:36.868 # +new-epoch 5
1:X 27 May 2021 11:11:36.868 # +try-failover master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:11:36.872 # +vote-for-leader 98b74b06e569ff7da63faa603ee1710cd6571935 5
1:X 27 May 2021 11:11:36.877 # 17127a5c54d86eedede4aeebec76970aab9d4ad8 voted for 98b74b06e569ff7da63faa603ee1710cd6571935 5
1:X 27 May 2021 11:11:36.939 # +elected-leader master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:11:36.939 # +failover-state-select-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:11:37.015 # -failover-abort-no-good-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:11:37.087 # Next failover delay: I will not start a failover before Thu May 27 11:12:13 2021
1:X 27 May 2021 11:11:43.156 # +reset-master master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:11:44.275 * +sentinel sentinel 17127a5c54d86eedede4aeebec76970aab9d4ad8 10.0.101.129 26379 @ mymaster 10.0.101.172 6379
1:X 27 May 2021 11:12:43.209 # +sdown master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:12:48.258 # +new-epoch 6
1:X 27 May 2021 11:12:48.263 # +vote-for-leader 17127a5c54d86eedede4aeebec76970aab9d4ad8 6
1:X 27 May 2021 11:12:48.460 # +odown master mymaster 10.0.101.172 6379 #quorum 2/2
1:X 27 May 2021 11:12:48.460 # Next failover delay: I will not start a failover before Thu May 27 11:13:25 2021
1:X 27 May 2021 11:13:25.068 # +new-epoch 7
1:X 27 May 2021 11:13:25.068 # +try-failover master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:13:25.074 # +vote-for-leader 98b74b06e569ff7da63faa603ee1710cd6571935 7
1:X 27 May 2021 11:13:25.081 # 17127a5c54d86eedede4aeebec76970aab9d4ad8 voted for 98b74b06e569ff7da63faa603ee1710cd6571935 7
1:X 27 May 2021 11:13:25.137 # +elected-leader master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:13:25.137 # +failover-state-select-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:13:25.203 # -failover-abort-no-good-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:13:25.287 # Next failover delay: I will not start a failover before Thu May 27 11:14:01 2021
1:X 27 May 2021 11:14:01.213 # +new-epoch 8
1:X 27 May 2021 11:14:01.218 # +vote-for-leader 17127a5c54d86eedede4aeebec76970aab9d4ad8 8
1:X 27 May 2021 11:14:01.219 # Next failover delay: I will not start a failover before Thu May 27 11:14:37 2021
1:X 27 May 2021 11:14:37.514 # +new-epoch 9
1:X 27 May 2021 11:14:37.518 # +vote-for-leader 17127a5c54d86eedede4aeebec76970aab9d4ad8 9
1:X 27 May 2021 11:14:37.568 # Next failover delay: I will not start a failover before Thu May 27 11:15:14 2021
1:X 27 May 2021 11:15:13.750 # +new-epoch 10
1:X 27 May 2021 11:15:13.753 # +vote-for-leader 17127a5c54d86eedede4aeebec76970aab9d4ad8 10
1:X 27 May 2021 11:15:13.780 # Next failover delay: I will not start a failover before Thu May 27 11:15:49 2021
1:X 27 May 2021 11:15:49.860 # +new-epoch 11
1:X 27 May 2021 11:15:49.861 # +try-failover master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:15:49.867 # +vote-for-leader 98b74b06e569ff7da63faa603ee1710cd6571935 11
1:X 27 May 2021 11:15:49.872 # 17127a5c54d86eedede4aeebec76970aab9d4ad8 voted for 98b74b06e569ff7da63faa603ee1710cd6571935 11
1:X 27 May 2021 11:15:49.939 # +elected-leader master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:15:49.939 # +failover-state-select-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:15:50.029 # -failover-abort-no-good-slave master mymaster 10.0.101.172 6379
1:X 27 May 2021 11:15:50.092 # Next failover delay: I will not start a failover before Thu May 27 11:16:25 2021

and it keep looping the failover but no correct master elected

[KaKeimei]

here is my yaml

auth:
  ## @param auth.enabled Enable password authentication
  ##
  enabled: false
  ## @param auth.sentinel Enable password authentication on sentinels too
  ##
  sentinel: false

replica:
  replicaCount: 3
  resources:
    requests:
      memory: 500Mi
      cpu: 100m
    limits:
      memory: 5000Mi
      cpu: 2000m
  persistence:
    size: 3Gi
  livenessProbe:
    initialDelaySeconds: 60
    periodSeconds: 10
  readinessProbe:
    initialDelaySeconds: 10
    periodSeconds: 10

sentinel:
  enabled: true

[alemorcuq]

This looks similar to #6165. We have an internal task to work on it. Thanks for reporting it.

[KaKeimei]

Thanks for your reply, looking forward to your fix

[github-actions]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.