This repository has been archived by the owner on Oct 11, 2024. It is now read-only.
add NoMerge flag for collections to control merging in kopia
#1914
Assignees
Labels
bug
Something isn't working
Comments
29 tasks
This was referenced Dec 22, 2022
aviator-app bot
pushed a commit
that referenced
this issue
Dec 22, 2022
## Description Adds a new func to the data.Collection iface: DoNotMergeItems() tells kopia to skip the retention of items from prior snapshots when generating the new snapshot for this collection. A primary use case for this flag is when a delta token expires, preventing an incremental lookup and forcing gc to re-discover all items in the container. ## Does this PR need a docs update or release note? - [x] ⛔ No ## Type of change - [x] 🌻 Feature ## Issue(s) * #1914 ## Test Plan - [x] ⚡ Unit test
12 tasks
aviator-app bot
pushed a commit
that referenced
this issue
Dec 22, 2022
## Description Flag doNotMergeItems for exchange collections where the delta token was marked as reset (ie, expired or otherwise unusable). Do this instead of making a (delete, new) collection tuple for that path. ## Does this PR need a docs update or release note? - [x] ⛔ No ## Type of change - [x] 🌻 Feature ## Issue(s) * #1914 ## Test Plan - [x] 💚 E2E
aviator-app bot
pushed a commit
that referenced
this issue
Dec 23, 2022
## Description The DoNotMergeItems flag denotes situations where we want to propagate changes to the hierarchy but do not want to source items from the base for a specific directory. As of now, the only time we expect to encounter this situation is when a delta token expires in M365 and we need to pull all the items for the container again. By setting DoNotMergeItems, a collection can propagate things like rename to its subtree while avoiding zombie items that would have appeared if there was a deletion in the container and the container was enumerated again ## Does this PR need a docs update or release note? - [ ] ✅ Yes, it's included - [ ] 🕐 Yes, but in a later PR - [x] ⛔ No ## Type of change - [ ] 🌻 Feature - [x] 🐛 Bugfix - [ ] 🗺️ Documentation - [ ] 🤖 Test - [ ] 💻 CI/Deployment - [ ] 🐹 Trivial/Minor ## Issue(s) * #1914 ## Test Plan - [ ] 💪 Manual - [x] ⚡ Unit test - [ ] 💚 E2E
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Under some conditions, we do not want to merge items in a collection with the base snapshot but we do want to preserve changes (or not change) the subtree rooted at the given collection. One such instance of this is when a delta token expires in M365. When a token expires, Corso will fallback to enumerating all items reachable with that delta endpoint.
However, because it's doing a full enumeration and GraphConnector has no information about what is in the base snapshot it cannot produce entries in the collection for deleted items
Making a collection marked as deleted (to stop sourcing items from the base) and then making a collection marked as new with the full enumeration of items is also not an option because it will delete the subtree rooted at the folder being enumerated
Adding a new flag instead of making new state(s) for the collection is more flexible because there are times where we'd want to skip merging base snapshot items for a specific container but still propagate changes to the subtree. An example would be moving a container in M365 but having an expired delta token. In that instance, the subfolders should be moved but no items for the root of the subtree should be sourced from the base. Because of this, at least two additional states would be required if we used states only, a
NoMergeMovedandNoMergeNotMovedstateThe text was updated successfully, but these errors were encountered: