Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make handle parsing/decoding functions idempotent #21102

Merged
merged 2 commits into from
May 20, 2024

Conversation

markfields
Copy link
Member

@markfields markfields commented May 15, 2024

Description

Fixes AB#8016. We need to port this back to RC3 and RC4 once merged to main.

For anyone implementing their own DDS, when they pick up an FF release with #19242 in it, they will likely be in a position where handles are going through parseHandles or serializer.decode twice.

We assumed this was idempotent but found that it is not, with two different codepaths needing to be fixed.

Reviewer Guidance

This is a superset of #19716. In that discussion you'll see reference made to some DDS Fuzz test failures for particular seeds. Abram and I took the JSON from those failures and tested those scripts against main and they were fine, so proceeding.

@github-actions github-actions bot added area: dds Issues related to distributed data structures base: main PRs targeted against main branch labels May 15, 2024
@msfluid-bot
Copy link
Collaborator

msfluid-bot commented May 16, 2024

@fluid-example/bundle-size-tests: -139 Bytes
Metric NameBaseline SizeCompare SizeSize Diff
aqueduct.js 453.98 KB 453.96 KB -17 Bytes
azureClient.js 551.22 KB 551.21 KB -16 Bytes
connectionState.js 680 Bytes 680 Bytes No change
containerRuntime.js 257.82 KB 257.82 KB No change
fluidFramework.js 359.58 KB 359.56 KB -21 Bytes
loader.js 132.91 KB 132.91 KB No change
map.js 41.45 KB 41.43 KB -16 Bytes
matrix.js 143.67 KB 143.66 KB -16 Bytes
odspClient.js 519.75 KB 519.73 KB -16 Bytes
odspDriver.js 97.29 KB 97.29 KB No change
odspPrefetchSnapshot.js 42.15 KB 42.15 KB No change
sharedString.js 160.19 KB 160.18 KB -16 Bytes
sharedTree.js 359.58 KB 359.56 KB -21 Bytes
Total Size 3.2 MB 3.2 MB -139 Bytes

Baseline commit: 1018e0f

Generated by 🚫 dangerJS against 5681b24

Copy link
Contributor

@alexvy86 alexvy86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change looks ok to me but I'm not an expert in the area, skipping approval so someone with more context reviews as well. Just a small nit below.

Co-authored-by: Alex Villarreal <[email protected]>
@Abe27342
Copy link
Contributor

Abe27342 commented May 20, 2024

FYI--'reviewer guidance' section needs update.

Mark and I looked into these failures just now and verified that the same failure JSON (i.e. sequence of steps) doesn't reproduce the assert today. Since the original PR, there have been a few fixes to SharedString/merge-tree which are plausibly what fixed the issue. Notably, we fixed an issue where objects in SharedString ops would be reference-equal to property bags in the string. It would make sense removing the round-trip in that PR would cause bad behavior given the existence of this bug at the time.

@markfields markfields merged commit 48254c5 into microsoft:main May 20, 2024
30 checks passed
@markfields markfields deleted the dds/parse-handles-idempotent branch May 20, 2024 18:39
markfields added a commit to markfields/FluidFramework that referenced this pull request May 20, 2024
Fixes microsoft#8016. We need to port this back to RC3 and RC4 once merged to
main.

For anyone implementing their own DDS, when they pick up an FF release
with microsoft#19242 in it, they will likely be in a position where handles are
going through `parseHandles` or `serializer.decode` twice.

We assumed this was idempotent but found that it is not, with two
different codepaths needing to be fixed.

Note that `parseHandles` may return the same object unmodified now,
wherease before it would always clone (via `JSON.stringify` roundtrip)

---------

Co-authored-by: Alex Villarreal <[email protected]>
markfields added a commit to markfields/FluidFramework that referenced this pull request May 20, 2024
Fixes microsoft#8016. We need to port this back to RC3 and RC4 once merged to
main.

For anyone implementing their own DDS, when they pick up an FF release
with microsoft#19242 in it, they will likely be in a position where handles are
going through `parseHandles` or `serializer.decode` twice.

We assumed this was idempotent but found that it is not, with two
different codepaths needing to be fixed.

Note that `parseHandles` may return the same object unmodified now,
wherease before it would always clone (via `JSON.stringify` roundtrip)

---------

Co-authored-by: Alex Villarreal <[email protected]>
markfields added a commit that referenced this pull request May 20, 2024
#21169)

Port #21102 to `release/client/2.0.0-rc.4.0`

For anyone implementing their own DDS, when upgrading from
`2.0.0-rc.2.0` to `2.0.0-rc.3.0` or later, they will likely be in a
position where handles are going through `parseHandles` or
`serializer.decode` twice. Without this fix, those functions are not
idempotent and will hit a fatal error being called twice.

Note that `parseHandles` may return the same object unmodified now,
whereas before it would always clone (via `JSON.stringify` roundtrip)

Co-authored-by: Alex Villarreal <[email protected]>
markfields added a commit that referenced this pull request May 21, 2024
#21170)

Port #21102 to `release/client/2.0.0-rc.3.0`

For anyone implementing their own DDS, when upgrading from
`2.0.0-rc.2.0` to `2.0.0-rc.3.0` or later, they will likely be in a
position where handles are going through `parseHandles` or
`serializer.decode` twice. Without this fix, those functions are not
idempotent and will hit a fatal error being called twice.

Note that `parseHandles` may return the same object unmodified now,
whereas before it would always clone (via `JSON.stringify` roundtrip)

Co-authored-by: Alex Villarreal <[email protected]>
kekachmar pushed a commit to kekachmar/FluidFramework that referenced this pull request May 21, 2024
Fixes microsoft#8016. We need to port this back to RC3 and RC4 once merged to
main.

For anyone implementing their own DDS, when they pick up an FF release
with microsoft#19242 in it, they will likely be in a position where handles are
going through `parseHandles` or `serializer.decode` twice.

We assumed this was idempotent but found that it is not, with two
different codepaths needing to be fixed.

Note that `parseHandles` may return the same object unmodified now,
wherease before it would always clone (via `JSON.stringify` roundtrip)

---------

Co-authored-by: Alex Villarreal <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: dds Issues related to distributed data structures base: main PRs targeted against main branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants