Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: Upon failure abort the multipart upload. #737

Merged
merged 1 commit into from
Jul 5, 2017

Conversation

harshavardhana
Copy link
Member

Since we do not have resuming capabilities we
should not leave stale parts behind upon any
error during putObject(). This PR pro-actively
aborts the multipart upload created during
the putObject() for large files.

@vadmeste
Copy link
Member

vadmeste commented Jul 2, 2017

LGTM though I think this PR is not so much useful in practice. I think most of the stale uploads are due to a premature exit during multipart upload.

vadmeste
vadmeste previously approved these changes Jul 2, 2017
Copy link
Member

@vadmeste vadmeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@harshavardhana
Copy link
Member Author

LGTM though I think this PR is not so much useful in practice. I think most of the stale uploads are due to a premature exit during multipart upload.

This is similar in behavior to how aws-sdk upload manager does. Lets say if one of the parts failed to upload we will never be able to clear it unless user explicitly does it on his own, most users are not aware. We should disallow that to happen and just abort the entire request similarly how aws-sdk does this.

@vadmeste
Copy link
Member

vadmeste commented Jul 2, 2017

This is similar in behavior to how aws-sdk upload manager does. Lets say if one of the parts failed to upload we will never be able to clear it unless user explicitly does it on his own, most users are not aware. We should disallow that to happen and just abort the entire request similarly how aws-sdk does this.

I know, but in practice if a part fails to upload due to a down server or internet connection (I don't see other possible errors here), I think that most likely the next call which aborts upload would fail (but I know not always).

I will approve this PR.

Can you rebase this @harshavardhana ?

@harshavardhana
Copy link
Member Author

I know, but in practice if a part fails to upload due to a down server or internet connection (I don't see other possible errors here), I think that most likely the next call which aborts upload would fail (but I know not always).

Not necessarily with AWS a part might fail just because of some slow network whereas an abort might complete. it can also happen that input stream is somewhat corrupted as well due to a local filesystem error. So there are valid cases.. not just network alone.

Since we do not have resuming capabilities we
should not leave stale parts behind upon any
error during putObject(). This PR pro-actively
aborts the multipart upload created during
the putObject() for large files.
@harshavardhana
Copy link
Member Author

Any reviews ?

@harshavardhana
Copy link
Member Author

@donatello any reviews?

Copy link
Member

@vadmeste vadmeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@deekoder deekoder merged commit edf5aaa into minio:master Jul 5, 2017
@harshavardhana harshavardhana deleted the abort branch July 5, 2017 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants