-
Notifications
You must be signed in to change notification settings - Fork 657
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
api: Upon failure abort the multipart upload. #737
Conversation
LGTM though I think this PR is not so much useful in practice. I think most of the stale uploads are due to a premature exit during multipart upload. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This is similar in behavior to how aws-sdk upload manager does. Lets say if one of the parts failed to upload we will never be able to clear it unless user explicitly does it on his own, most users are not aware. We should disallow that to happen and just abort the entire request similarly how aws-sdk does this. |
I know, but in practice if a part fails to upload due to a down server or internet connection (I don't see other possible errors here), I think that most likely the next call which aborts upload would fail (but I know not always). I will approve this PR. Can you rebase this @harshavardhana ? |
Not necessarily with AWS a part might fail just because of some slow network whereas an abort might complete. it can also happen that input stream is somewhat corrupted as well due to a local filesystem error. So there are valid cases.. not just network alone. |
Since we do not have resuming capabilities we should not leave stale parts behind upon any error during putObject(). This PR pro-actively aborts the multipart upload created during the putObject() for large files.
Any reviews ? |
@donatello any reviews? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Since we do not have resuming capabilities we
should not leave stale parts behind upon any
error during putObject(). This PR pro-actively
aborts the multipart upload created during
the putObject() for large files.