Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Failed/PartiallyFailed Orphaned backup will not be removed by ObjectStorageSync #4483

Closed
kaovilai opened this issue Jan 4, 2022 · 6 comments · Fixed by #4729
Closed
Assignees

Comments

@kaovilai
Copy link
Member

kaovilai commented Jan 4, 2022

Describe the problem/challenge you have
Failed/PartiallyFailed backups hang on Kubernetes although removed from bucket

Describe the solution you'd like
Backups that are failed or partially failed should be considered for deletion as well.

BackupPhasePartiallyFailed BackupPhase = "PartiallyFailed"

BackupPhaseFailed BackupPhase = "Failed"

Anything else you would like to add:

Line of code that may need amending to implement this feature:

if backup.Status.Phase != velerov1api.BackupPhaseCompleted || backupStoreBackups.Has(backup.Name) {

Alternatively: Object Storage Sync docs should mention that only completed backups are deleted.

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

  • 👍 for "The project would be better with this feature added"
  • 👎 for "This feature will not enhance the project in a meaningful way"

cc: @mperetzred @dymurray

@reasonerjt
Copy link
Contributor

For aFailed backup , it usually does not exist in the bucket, so the backup sync controller should probably not removes it as part of syncing the data.

IMO we should update the doc to clarify it.

@kaovilai
Copy link
Member Author

Sounds good to me

@kaovilai kaovilai changed the title Delete Failed/PartiallyFailed Orphaned backup [Docs] Failed/PartiallyFailed Orphaned backup will not be removed by ObjectStorageSync Jan 13, 2022
@eleanor-millman
Copy link
Contributor

@a-mccarthy Do you have enough to do this work, or do you need engineering input? Thanks!

@sseago
Copy link
Collaborator

sseago commented Mar 8, 2022

I think part of the concern here gets at the root of what the goal of cleaning up expired backups. Obviously the most important reason is storage space in the BSL. But a secondary purpose may be cleaning up the list of backup resources in kubernetes. Currently failed backups and backups with invalid BSLs are not cleaned up, but this could lead to months worth of failed and/or inaccessible backups still clogging the list of returned backups when listing in-cluster backups. We need a way for these to be automatically cleaned up, either by default, or via some feature or configuration flag to have them cleared out when they expire.

@eleanor-millman
Copy link
Contributor

@sseago to be clear, are you suggested additional feature work? If so, probably best if you open a separate issue, since this docs issue will be closed once we get the docs updated. But if I misunderstood and you are suggesting different docs updates, then please let me know!

@sseago
Copy link
Collaborator

sseago commented Mar 14, 2022

@eleanor-millman Ahh -- I think I was mixing up the doc issue (this one) with the related dev issue that we discussed on the call last week. No, I think on the docs side, documenting current behavior is all that this issue needs. The other issue was where we were discussing possible feature work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment