Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore multiple completions for awakeables #637

Merged
merged 1 commit into from
Jul 21, 2023

Conversation

slinkydeveloper
Copy link
Contributor

Fix #636

Copy link
Contributor

@tillrohrmann tillrohrmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating this PR @slinkydeveloper. Can you quickly add some more context to this PR or the originating issue which panic you saw and where it originated? I guess it was not in the store_completion method. This would help me validating the change.

@slinkydeveloper
Copy link
Contributor Author

The panic comes from Codec::write_completion. The originating issue is that we end up invoking completeCallback twice (due to the sdk bug restatedev/sdk-typescript#122), so on the second store we invoke write_completion and panic on the debug assert checking whether the entry is already invoked.

Logically, it seems to me more appropriate that we do this check in the state machine, that is in store_completion, as in terms of "entries" and protocol themselves, that debug assert is correct so ignoring the completion in the codec doesn't seem the right place.

@slinkydeveloper
Copy link
Contributor Author

I also tested this change with restatedev/e2e#171 and it doesn't panic anymore.

Copy link
Contributor

@tillrohrmann tillrohrmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. +1 for merging.

src/worker/src/partition/effects/interpreter.rs Outdated Show resolved Hide resolved
@tillrohrmann
Copy link
Contributor

Logically, it seems to me more appropriate that we do this check in the state machine, that is in store_completion, as in terms of "entries" and protocol themselves, that debug assert is correct so ignoring the completion in the codec doesn't seem the right place.

I was confused as well because I couldn't find a check whether the entry is completed or not in the store_completion method but also didn't look into the Codec::write_completion implementation. I guess the debug_assert says that the Codec does not allow the duplicate completion of an entry (even though technically it could allow it). Also because the Codec changes the header flags for some entries.

While debugging an e2e test we've found out that the runtime can panic in `Codec::write_completion` in case the user tries to complete an awakeable twice. This commit avoids this panic, but rather logs that the user is trying to complete an awakeable twice.
@slinkydeveloper slinkydeveloper merged commit 8bf53b0 into restatedev:main Jul 21, 2023
@slinkydeveloper slinkydeveloper deleted the issues/636 branch July 21, 2023 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Be tolerant with completing an awakeable twice
2 participants