Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add odsp driver support to upload container type summary #9671

Merged
merged 8 commits into from
Mar 31, 2022

Conversation

jatgarg
Copy link
Contributor

@jatgarg jatgarg commented Mar 30, 2022

Fixes: #8079
This adds support for odsp driver to upload container type summary which includes the ".protocol" summary too along with app summary.
Currently there is no support at server to handle the full summary, so we get the SummaryNack in case we do upload the full summary in odsp driver.
But this is behind feature gate "SummarizeProtocolTree" in loader layer, so I think we don't need to add another feature gate for driver as if loader is sending the protocol tree then in that case odsp driver does have to upload everything, so I think another feature gate might not make sense. Let me know your thoughts @anthony-murphy .

@jatgarg jatgarg requested review from a team as code owners March 30, 2022 05:59
@github-actions github-actions bot added area: driver Driver related issues area: odsp-driver labels Mar 30, 2022
@jatgarg jatgarg self-assigned this Mar 30, 2022
@jatgarg jatgarg added this to the March 2022 milestone Mar 30, 2022
@github-actions github-actions bot added the base: main PRs targeted against main branch label Mar 30, 2022
@msfluid-bot
Copy link
Collaborator

msfluid-bot commented Mar 30, 2022

@fluid-example/bundle-size-tests: +115 Bytes
Metric NameBaseline SizeCompare SizeSize Diff
aqueduct.js 382.38 KB 382.38 KB No change
containerRuntime.js 189.49 KB 189.49 KB No change
loader.js 159.99 KB 159.99 KB No change
map.js 204.59 KB 204.59 KB No change
matrix.js 292.09 KB 292.09 KB No change
odspDriver.js 183.09 KB 183.21 KB +115 Bytes
odspPrefetchSnapshot.js 76.73 KB 76.73 KB No change
sharedString.js 311.57 KB 311.57 KB No change
Total Size 1.79 MB 1.79 MB +115 Bytes

Baseline commit: 5681a58

Generated by 🚫 dangerJS against 533325d

@GaryWilber
Copy link
Contributor

Regarding Currently there is no support at server to handle the full summary, so we get the SummaryNack in case we do upload the full summary in odsp driver.
I will need to make a change on my side to support this. Is it possible to have the client include a new flag in the summary op to indicate that it uploaded the container tree? There is an includesProtocolTree flag set today - maybe another flag can be added, such as isContainerTree

@anthony-murphy
Copy link
Contributor

anthony-murphy commented Mar 31, 2022

Regarding Currently there is no support at server to handle the full summary, so we get the SummaryNack in case we do upload the full summary in odsp driver. I will need to make a change on my side to support this. Is it possible to have the client include a new flag in the summary op to indicate that it uploaded the container tree? There is an includesProtocolTree flag set today - maybe another flag can be added, such as isContainerTree

supporting a big matrix of unshipped features is not great. i'm fine if we just switch the server behavior. just let use know when you assume includesProtocolTree means container upload and we'll make the switch too. maybe do a server side flag, and we'll just coordinate rollout as none of this is in prod.

@GaryWilber
Copy link
Contributor

supporting a big matrix of unshipped features is not great. i'm fine if we just switch the server behavior. just let use know when you assume includesProtocolTree means container upload and we'll make the switch too. maybe do a server side flag, and we'll just coordinate rollout as none of this is in prod.

The issue I'm facing is that the server needs to know what kind of tree the client uploaded.

There are a total of 3 cases:

  1. Client uploads a channel tree without a .protocol tree in it
  2. Client uploads a channel tree with a .protocol tree
  3. Client uploads a container tree with a .protocol tree

The server has to run different logic depending on each case:

  1. Server commits the summary by POSTing a container, which is a server generated .protocol tree + the client created .app tree
  2. Server fetches the client created .protocol tree, verifies the .protocol tree is valid, and POSTs that channel tree as a container.
  3. Server should not do anything (no calls to ODSP) because the client already did the commit.

The server submits a summaryAck after processing the summary in each case.

The server knows case 1 and 2 are different because of the includesProtocolTree flag.

However the server cannot determine the difference between case 2 and 3 because the flag is set for both. This means the server will have to figure out if the client uploaded a channel or container tree by calling ODSP. So "case 3" will actually involve a call to ODSP now, which is working against the purpose of what we want - I don't want Push to have to call ODSP when the client already committed the summary.

@jatgarg jatgarg requested a review from a team as a code owner March 31, 2022 18:36
@github-actions github-actions bot added the area: tests Tests to add, test infrastructure improvements, etc label Mar 31, 2022
@jatgarg
Copy link
Contributor Author

jatgarg commented Mar 31, 2022

supporting a big matrix of unshipped features is not great. i'm fine if we just switch the server behavior. just let use know when you assume includesProtocolTree means container upload and we'll make the switch too. maybe do a server side flag, and we'll just coordinate rollout as none of this is in prod.

The issue I'm facing is that the server needs to know what kind of tree the client uploaded.

There are a total of 3 cases:

  1. Client uploads a channel tree without a .protocol tree in it
  2. Client uploads a channel tree with a .protocol tree
  3. Client uploads a container tree with a .protocol tree

The server has to run different logic depending on each case:

  1. Server commits the summary by POSTing a container, which is a server generated .protocol tree + the client created .app tree
  2. Server fetches the client created .protocol tree, verifies the .protocol tree is valid, and POSTs that channel tree as a container.
  3. Server should not do anything (no calls to ODSP) because the client already did the commit.

The server submits a summaryAck after processing the summary in each case.

The server knows case 1 and 2 are different because of the includesProtocolTree flag.

However the server cannot determine the difference between case 2 and 3 because the flag is set for both. This means the server will have to figure out if the client uploaded a channel or container tree by calling ODSP. So "case 3" will actually involve a call to ODSP now, which is working against the purpose of what we want - I don't want Push to have to call ODSP when the client already committed the summary.

Yes, I think that is what Tony is saying. For now, even for 3rd you can do same as you do in 2 because it will not be worse than the current state. Then once we are confident, w can enable the flag permanently and then you can stop checking the type and eliminate 2nd state.

@jatgarg jatgarg merged commit f514de2 into microsoft:main Mar 31, 2022
@jatgarg jatgarg deleted the protocol branch March 31, 2022 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: driver Driver related issues area: odsp-driver area: tests Tests to add, test infrastructure improvements, etc base: main PRs targeted against main branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ODSP support container upload in addition to channel upload when unploading protocol tree
5 participants