Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Selectors: promote ResourceOwners to Selector #1617

Closed
ryanfkeepers opened this issue Nov 28, 2022 · 1 comment
Closed

Selectors: promote ResourceOwners to Selector #1617

ryanfkeepers opened this issue Nov 28, 2022 · 1 comment
Assignees
Labels

Comments

@ryanfkeepers
Copy link
Contributor

ryanfkeepers commented Nov 28, 2022

Issue

Corso design has deviated in path from original Selector design, and we're beginning to see idealogical collisions between the two. For example, in #1505 we want to transition backups to be split between resource owners (ie: moving from 1:many backup:owner towards 1:1).

Current selectors are designed for granular queries such as, "get all U1's mail, also U1 and U2's contacts". This places resource owner identification at a per-scope level, rather than a per-selector (ie: per-backup) level, causing each selector to produce a complicated venn-diagram of owner-data pairings. As Corso design has matured we've moved away from supporting granular and compound queries such as these. Therefore the current design is not only complicated to reason about, it's also more complication than Corso wants to support.

Proposal

Resource Owner identification is to be removed from the scope-level declarations. Instead, each Selector will recognize a single, canonical list of resource owners. IE:

// this current design
s := selectors.NewExchangeBackup()
s.Include(
  s.Mail(users, folders, mailIDs),
  s.Contacts(users, folders, contactIDs)
)

// will become
s := selectors.NewExchangeBackup(users)
s.Include(
  s.Mail(folders, mailIDs),
  s.Contacts(folders, contactIDs),
)

// The new selector struct would look like this:
type Selector struct {
	Service service
        resourceOwners filters.Filter
	Excludes []scope
	Filters []scope
	Includes []scope
}

All scopes in the selector would implicitly apply to each resource owner. Eg: if the resource owners contains [Alice, Bob], and the scopes include MailFolder(Inbox), then the selector can be interpreted as "select all mail in the inbox for alice and bob".

Usage Changes

Backup data aggregation currently employs the following pattern:

1. generate a selector
2. pass the selector to a Backup Operation
3. pass the selector to GraphConnector
4. for each scope in the selector's Included scopes
5. for each resourceOwner in the scope (or, as fallback, the tenant's owner list)
6. collect all data for backup as a single set
7. run the backup

In the proposed change the resource owner iteration would migrate above the operation creation. This would generate N backup operations, one for each resource owner.

1. generate a selector for a given set of resourceOwners (or, as fallback, the tenant's owner list)
2. for each owner in the selector.
3. pass the (owner, selector) pair to a Backup Operation
4. Pass the selector to GraphConnector
5. for each scope in the selector's Included scopes
6. collect all data for backup
7. run the backup

Additional Needs

To minimize cognitive load, and conflicts between intent and available data, it is not in our best interest to pass a (owner, selector) pair into operations or graph connector. Doing so provides downstream packages the option of abiding either the intended user, or the original user set in the selector. We can simplify the situation by identifying a singular resource owner within a selector instance, and passing only the selector into processes.

type Selector struct {
	Service service
        DiscreteOwner string
        resourceOwners filters.Filter
	Excludes []scope
	Filters []scope
	Includes []scope
}

Thus, the precondition for using selectors is that the caller first splits the selector along its owners with a func like below. This should produce a set of N selectors, one for each resource owner. Alternatively, the caller can set the DiscreteOwner value directly. If the selector uses a wildcard value for its user list such as selectors.Any(), the ownerList parameter is expected to provide a fallback list of distinct resource owners to iterate over instead.

func (s Selector) SplitOnOwners(ownerList) []Selector {...}
@ryanfkeepers ryanfkeepers self-assigned this Nov 28, 2022
aviator-app bot pushed a commit that referenced this issue Dec 21, 2022
## Description

selector creation now includes a parameter for
a slice of resource owners (users or sites).  This
is step one in migrating resource owner lists out
of scopes and into the selector.  next step is to
have the selector utilize the primary list instead
of the per-scope list.

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1617

## Test Plan

- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Jan 3, 2023
## Description

Switches the CLI from calling `DiscreteScopes` to `SplitByResourceOwner`
on the selector itself.  This func will take the original selector and produce
a slice of selectors, each one with a DiscreteOwner (the single user involved
in usage of that selector) and all include/filter scopes in that selector re-rooted
to that discrete owner.

Does not yet solve the per-category tuple, since we are still pivoting on the
scopes inside the selector.  That comes as a later change.

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1617

## Test Plan

- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Jan 4, 2023
## Description

Migrates code away from pulling the resource
owner from each scope, and instead usees the
selector as the canon identifier of the resource
owner.

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1617

## Test Plan

- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Jan 5, 2023
## Description

DataCollections validation step was still using the full resourceOwner list in the selector to validate every backup, rather than checking only the DiscreteOwner.

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1617

## Test Plan

- [x] ⚡ Unit test
aviator-app bot pushed a commit that referenced this issue Jan 5, 2023
## Description

Checks for resource owner matches in the top
of the reduce func using the selector owners,
instead of waiting until the path match check.

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1617

## Test Plan

- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Jan 5, 2023
## Description
    
Now that resource owners are identified via
the selector itself, rather than each scope, we
can remove the resource owner data from
scope production and data.

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1617

## Test Plan

- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Jan 9, 2023
## Description

The selectors printable code wasn't being used in any valid way, so that's out.  Backup List printing now refers to the DiscreteOwner in the embedded selector as the ResourceOwner reference.  Renamed the "Selectors"
column in the List print table to "Resource Owner" to better match what should be contained in that field.

## Does this PR need a docs update or release note?

- [x] ✅ Yes, it's included

## Type of change

- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1617

## Test Plan

- [x] 💪 Manual
- [x] ⚡ Unit test
aviator-app bot pushed a commit that referenced this issue Jan 10, 2023
## Description

DiscreteScopes is a vestigial func from when scopes contained the list of resource owners to track.  That behavior is no longer in use.

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1617

## Test Plan

- [x] ⚡ Unit test
ryanfkeepers added a commit that referenced this issue Jan 17, 2023
## Description

The selectors printable code wasn't being used in any valid way, so that's out.  Backup List printing now refers to the DiscreteOwner in the embedded selector as the ResourceOwner reference.  Renamed the "Selectors"
column in the List print table to "Resource Owner" to better match what should be contained in that field.

## Does this PR need a docs update or release note?

- [x] ✅ Yes, it's included

## Type of change

- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1617

## Test Plan

- [x] 💪 Manual
- [x] ⚡ Unit test
@ryanfkeepers
Copy link
Contributor Author

Migration is complete.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant