Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(datastore): Remove unnecessary synchronized causing subscription slowness #2718

Merged
merged 5 commits into from
Feb 16, 2024

Conversation

tylerjroach
Copy link
Member

  • PR title and description conform to Pull Request guidelines.

Issue #, if available:

Description of changes:

When DataStore is started, Create, Update, Delete subscriptions are made for each model. If I have 10 models, this results in 30 subscriptions established. Subscriptions are established over a single websocket connection by sending a websocket event per subscription. We wait until all subscriptions have been established (ws response message for each subscription) before we proceed with syncing/merging items.

An errant synchronized is causing subscription events to being be processed 1 at a time, and not in parallel.

If I have 10 models (30 subscriptions) and each ws event takes the server 200ms to acknowledge, establishing subscriptions would take the current build ~ 6 seconds. Allowing ws events in parallel will complete this effort in ~ 200ms.

Buggy State:
Send Model1 Create -> Model1 Create Response -> Send Model1 Delete -> Model1 Delete Response -> Send Model1 Update -> Model1 Update Response -> Send Model2 Create -> Model2 Create Response -> ...

Fixed State:
Send Model1 Create, Send Model1 Delete, Send Model1 Update, Send Model2 Create .... -> Wait for all acks

How did you test these changes?
(Please add a line here how the changes were tested)

Documentation update required?

  • No
  • Yes (Please include a PR link for the documentation update)

General Checklist

  • Added Unit Tests
  • Added Integration Tests
  • Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@tylerjroach tylerjroach requested a review from a team as a code owner February 16, 2024 14:38
@codecov-commenter
Copy link

codecov-commenter commented Feb 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (c839191) 42.86% compared to head (b06f4bd) 42.87%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2718   +/-   ##
=======================================
  Coverage   42.86%   42.87%           
=======================================
  Files         905      905           
  Lines       29098    29096    -2     
  Branches     4142     4142           
=======================================
+ Hits        12473    12475    +2     
+ Misses      15268    15265    -3     
+ Partials     1357     1356    -1     

tjleing
tjleing previously approved these changes Feb 16, 2024
… could result in starting sync when it shouldn't
Copy link
Contributor

@ankpshah ankpshah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great find ! Thanks for fixing this.

@tjleing tjleing merged commit 1804551 into main Feb 16, 2024
3 checks passed
@tjleing tjleing deleted the tjroach/ds-subscribe-optimization branch February 16, 2024 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants