Retry GCS Resumable Upload on Error 410 (#45963) #51419
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A resumable upload session can fail on with a 410 error and should be retried in that case. I added retrying twice using resetting of the given `InputStream` as the retry mechanism since the same approach is used by the AWS S3 SDK already as well and relied upon by the S3 repository implementation. Related GCS documentation: https://cloud.google.com/storage/docs/json_api/v1/status-codes#410_Gone
original-brownbear
added
:Distributed Coordination/Snapshot/Restore
|
Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore) |
|
@ywelsch any reason not to backport this? It seems pretty risk free (fix went in without conflicts and code is pretty much exactly the same as in master for the GCS plugin in 6.8) and it might also alleviate some of our own pains on Cloud if we ever release another 6.8? |
|
I think we can backport this. |
|
Thanks Jason! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A resumable upload session can fail on with a 410 error and should
be retried in that case. I added retrying twice using resetting of
the given
InputStreamas the retry mechanism since the sameapproach is used by the AWS S3 SDK already as well and relied upon
by the S3 repository implementation.
backport of #45963
Closes #51402