This repository has been archived by the owner on Oct 11, 2024. It is now read-only.
Fetch previous kopia snapshots rooted in the same directory during backup #1404
Closed
Assignees
Comments
9 tasks
This was referenced Oct 31, 2022
aviator-app bot
pushed a commit
that referenced
this issue
Nov 1, 2022
## Description Add tags to each kopia snapshot that include all service/category pairs in the snapshot and all resource owners in the snapshot. This allows future snapshots to lookup existing snapshots by those tags so they can be fed into the snapshot function. Feeding previous snapshots into the snapshot function enables kopia to detect previously uploaded files and skip uploading the data again ## Type of change <!--- Please check the type of change your PR introduces: ---> - [x] 🌻 Feature - [ ] 🐛 Bugfix - [ ] 🗺️ Documentation - [ ] 🤖 Test - [ ] 💻 CI/Deployment - [ ] 🐹 Trivial/Minor ## Issue(s) * #1404 ## Test Plan <!-- How will this be tested prior to merging.--> - [ ] 💪 Manual - [x] ⚡ Unit test - [ ] 💚 E2E
This was referenced Nov 2, 2022
aviator-app bot
pushed a commit
that referenced
this issue
Dec 1, 2022
## Description Helper functions that allow finding the most recent complete (and maybe incomplete) snapshots for a given set of tags. The returned snapshots can later be used to do more efficient kopia snapshots by allowing kopia to determine it's already uploaded a file and skip uploading it again. The number of most recent snapshots to return can be tuned (right now it returns 1 most recent), but may cause more memory usage during backups. Kopia currently has some oddities when getting snapshot manifests via tags: * tag values cannot be empty or the comparison returns always true, selecting all snapshot manifests * kopia does not currently tag snapshot manifests made during checkpoints. A patch to upstream kopia is needed to fix this. All other manifests that match the tags will be returned though Added code is not currently connected to any backup logic. Viewing by commit may help as there was a tad of lift'n'shift code movement for the sake of organization ## Type of change <!--- Please check the type of change your PR introduces: ---> - [x] 🌻 Feature - [ ] 🐛 Bugfix - [ ] 🗺️ Documentation - [ ] 🤖 Test - [ ] 💻 CI/Deployment - [ ] 🐹 Trivial/Minor ## Issue(s) * #1404 ## Test Plan <!-- How will this be tested prior to merging.--> - [ ] 💪 Manual - [x] ⚡ Unit test - [ ] 💚 E2E
aviator-app bot
pushed a commit
that referenced
this issue
Dec 2, 2022
…les (#1430) ## Description Tell kopia about previous snapshots during the backup so it can make use of them to skip uploading some data. Currently only selects the most recent completed snapshot and most recent incomplete snapshot that contains the information being backed up. This can be tuned later if it is not working good enough. However, increasing the number of previous snapshots passed in may increase memory usage during backup ## Type of change - [x] 🌻 Feature - [ ] 🐛 Bugfix - [ ] 🗺️ Documentation - [ ] 🤖 Test - [ ] 💻 CI/Deployment - [ ] 🐹 Trivial/Minor ## Issue(s) * closes #1404 merge after: * #1427 ## Test Plan - [ ] 💪 Manual - [x] ⚡ Unit test - [ ] 💚 E2E
29 tasks
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
No description provided.
The text was updated successfully, but these errors were encountered: