Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DTP-1034] Emit Live Objects lifecycle events #1958

Open
wants to merge 4 commits into
base: DTP-1147/fix-flaky-liveobjects-tests
Choose a base branch
from

Conversation

VeskeR
Copy link
Contributor

@VeskeR VeskeR commented Jan 23, 2025

Functionally complete but not tests yet as some of them depend on DTP-1147. Worth revieweing this PR API side while I complete tests setup.

Resolves DTP-1034

Summary by CodeRabbit

  • New Features

    • Enhanced event handling for LiveObjects and LiveObject classes.
    • Added lifecycle event management with new subscription methods.
    • Introduced more granular event control with on, off, and offAll methods.
    • New asynchronous utility functions for improved testing capabilities.
  • Improvements

    • Updated event naming conventions to be more consistent.
    • Separated internal and public event emitters.
    • Added new events like sync_start and sync_end.
    • Unique object ID generation now includes timestamps for better differentiation.

@VeskeR VeskeR requested a review from mschristensen January 23, 2025 09:25
Copy link

coderabbitai bot commented Jan 23, 2025

Warning

Rate limit exceeded

@VeskeR has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 0 minutes and 7 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 8c2a8de and 3c40056.

📒 Files selected for processing (2)
  • src/plugins/liveobjects/liveobject.ts (8 hunks)
  • src/plugins/liveobjects/liveobjects.ts (9 hunks)

Walkthrough

The pull request introduces significant improvements to the event handling mechanism in the LiveObject and LiveObjects classes. The changes focus on enhancing event management by introducing new enumerations for subscription and lifecycle events, updating event naming conventions, and adding methods for more granular event control. The modifications include separating internal and public event emitters, introducing new event types like updated and deleted, and providing methods to subscribe, unsubscribe, and manage event callbacks.

Changes

File Change Summary
src/plugins/liveobjects/liveobject.ts - Added LiveObjectSubscriptionEvent enum
- Added LiveObjectLifecycleEvent enum
- Introduced lifecycle event handling methods
- Replaced _eventEmitter with separate _subscriptions and _lifecycleEvents
src/plugins/liveobjects/liveobjects.ts - Added LiveObjectsEvent enum
- Renamed event names (e.g., SyncCompleted to sync_end)
- Added _eventEmitterPublic
- Introduced on, off, and offAll methods for event management
src/plugins/liveobjects/objectid.ts - Modified fromInitialValue method to remove intermediate variable for hash encoding
test/common/modules/live_objects_helper.js - Added static ACTIONS property
- Introduced fixtureRootKeys method
- Updated fakeMapObjectId and fakeCounterObjectId methods to include timestamps
test/realtime/live_objects.test.js - Added utility functions: waitForMapKeyUpdate, waitForCounterUpdate, waitForStateOperation, waitFixtureChannelIsReady

Assessment against linked issues

Objective Addressed Explanation
Emit Live Objects lifecycle events [DTP-1034]
Fix flaky tests that depend on STATE message received [DTP-1147] New utility functions improve test reliability by handling asynchronous updates.

Possibly related PRs

Suggested reviewers

  • mschristensen

Poem

🐰 Hop, hop, events now flow with grace,
Lifecycle whispers in each embrace,
Subscriptions dance, callbacks take flight,
LiveObjects shine, oh so bright!
Code evolves, a rabbit's delight! 🎉


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot temporarily deployed to staging/pull/1958/bundle-report January 23, 2025 09:25 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/typedoc January 23, 2025 09:25 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/features January 23, 2025 09:25 Inactive
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
src/plugins/liveobjects/liveobject.ts (2)

32-36: Consider adding event payload type.

The lifecycle event callback and response interfaces are well-defined but could be enhanced to support future event types that might need payload data.

Consider this enhancement:

-export type LiveObjectLifecycleEventCallback = () => void;
+export type LiveObjectLifecycleEventCallback<T = void> = (data?: T) => void;

 export interface OnLiveObjectLifecycleEventResponse {
   off(): void;
 }

166-166: Consider adding a deleted timestamp to the event payload.

The deleted event could benefit from including the _tombstonedAt timestamp in its payload for better tracking of when the deletion occurred.

-    this._lifecycleEvents.emit(LiveObjectLifecycleEvent.deleted);
+    this._lifecycleEvents.emit(LiveObjectLifecycleEvent.deleted, { deletedAt: this._tombstonedAt });
src/plugins/liveobjects/liveobjects.ts (2)

167-170: Consider adding type guard for event parameter.

Similar to the LiveObject class, there's a defensive check for nil arguments. Consider adding a type guard for the event parameter.

-    if (this._client.Utils.isNil(event) && this._client.Utils.isNil(callback)) {
+    if (!Object.values(LiveObjectsEvent).includes(event) || this._client.Utils.isNil(callback)) {
       return;
     }

314-315: Consider adding sync metadata to event payload.

The sync events could provide valuable metadata about the sync operation to help with debugging and monitoring.

-    this._eventEmitterInternal.emit(LiveObjectsEvent.sync_start);
-    this._eventEmitterPublic.emit(LiveObjectsEvent.sync_start);
+    const syncStartMetadata = { syncId: this._currentSyncId, cursor: this._currentSyncCursor };
+    this._eventEmitterInternal.emit(LiveObjectsEvent.sync_start, syncStartMetadata);
+    this._eventEmitterPublic.emit(LiveObjectsEvent.sync_start, syncStartMetadata);
-    this._eventEmitterInternal.emit(LiveObjectsEvent.sync_end);
-    this._eventEmitterPublic.emit(LiveObjectsEvent.sync_end);
+    const syncEndMetadata = { 
+      syncId: this._currentSyncId, 
+      bufferedOperationsCount: this._bufferedStateOperations.length 
+    };
+    this._eventEmitterInternal.emit(LiveObjectsEvent.sync_end, syncEndMetadata);
+    this._eventEmitterPublic.emit(LiveObjectsEvent.sync_end, syncEndMetadata);

Also applies to: 329-330

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5d78643 and eedbe01.

📒 Files selected for processing (2)
  • src/plugins/liveobjects/liveobject.ts (8 hunks)
  • src/plugins/liveobjects/liveobjects.ts (6 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (6)
  • GitHub Check: test-browser (webkit)
  • GitHub Check: test-node (20.x)
  • GitHub Check: test-browser (firefox)
  • GitHub Check: test-node (18.x)
  • GitHub Check: test-node (16.x)
  • GitHub Check: test-browser (chromium)
🔇 Additional comments (5)
src/plugins/liveobjects/liveobject.ts (3)

6-7: LGTM! Consistent event naming convention.

The event names follow the lowercase snake_case convention, which is consistent across both LiveObjectSubscriptionEvent and LiveObjectLifecycleEvent enums.

Also applies to: 28-30


43-44: LGTM! Good separation of event emitters.

The separation of subscription and lifecycle events into distinct event emitters improves the code organization and maintainability.

Also applies to: 69-70


123-126: LGTM! Defensive programming.

Good defensive check to prevent accidental removal of all event listeners when both arguments are nil.

src/plugins/liveobjects/liveobjects.ts (2)

14-16: LGTM! Clear sync event naming.

The sync events follow a consistent naming pattern and clearly indicate the start and end of sync operations.


32-34: LGTM! Good documentation of event emitter separation.

The comment referencing RTC10 provides clear context for why there are separate internal and public event emitters.

Copy link
Contributor

@mschristensen mschristensen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, left a suggestion to discuss

Comment on lines 15 to 16
sync_start = 'sync_start',
sync_end = 'sync_end',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these events should reflect the current "state" (in the state-machine sense) of the liveobjects instance, similar to how we do for channels and connections

So I would suggest initialized, syncing, and synced, analogous to initialized, connecting and connected for connection state or initialized, attaching and attached for channels.

Copy link
Contributor Author

@VeskeR VeskeR Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, I think for initialized I will need to add another flag (or more likely an explicit state) to the LiveObjects class to distinguish between sync is in progress (we received an ATTACHED with HAS_STATE flag) and when LiveObjects instance was just created. Currently there is only _syncInProgress flag that only covers syncing and synced.

Also, would you expect to always get a syncing event before getting a synced event? This is important because with the current implementation, we can be in a synced state and then reattach to a channel that doesn't have HAS_STATE flag - this essentially removes the state immediately and emits the synced event again, without emitting syncing event before that.
see here:

onAttached(hasState?: boolean): void {
this._client.Logger.logAction(
this._client.logger,
this._client.Logger.LOG_MINOR,
'LiveObjects.onAttached()',
`channel=${this._channel.name}, hasState=${hasState}`,
);
if (hasState) {
this._startNewSync();
} else {
// no HAS_STATE flag received on attach, can end SYNC sequence immediately
// and treat it as no state on a channel
this._liveObjectsPool.reset();
this._syncLiveObjectsDataPool.reset();
this._endSync();
}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this._endSync(); call will just emit synced event again in this case, but we never entered the syncing state so never emitted that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we do for channels/connections? Do we enforce that you always get e.g. connecting before connected?

Copy link
Contributor Author

@VeskeR VeskeR Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small note from the standup: HAS_STATE flag on channel attachment is always true if state_subscribe mode is set, so this guarantees we can have syncing -> synced order, given HAS_STATE flag is set (state_subscribe mode is set for a channel)

What do we do for channels/connections? Do we enforce that you always get e.g. connecting before connected?

I couldn't find an explicit mention of connecting -> connected order, but I think it is guaranteed implicitly based on the next:

The RealtimeChannel will (potentially) emit a separate update event when it gets an ATTACHED message while being in an attached state:

if (!resumed || this.channelOptions.updateOnAttached) {
this.emit('update', change);
}

And more generally, if there is no special logic for an event, then notifyEvent method won't emit an event if current state of the channel matches the one it's attempting to notify:
if (state === this.state) {
return;
}

Spec point matches the behavior described above: https://sdk.ably.com/builds/ably/specification/main/features/#RTL12

The Connection (more precisely ConnectionManager) has pretty much the same behavior, where there is sometimes special logic to emit update event when connection configuration changes without an explicit reconnection:

this.onConnectionDetailsUpdate(connectionDetails, transport);
Platform.Config.nextTick(() => {
transport.on(
'connected',
(connectedErr: ErrorInfo, _connectionId: string, connectionDetails: Record<string, any>) => {
this.onConnectionDetailsUpdate(connectionDetails, transport);
this.emit('update', new ConnectionStateChange(connectedState, connectedState, null, connectedErr));
},
);
});
/* If previously not connected, notify the state change (including any
* error). */
if (existingState.state === this.states.connected.state) {
if (error) {
this.errorReason = this.realtime.connection.errorReason = error;
this.emit('update', new ConnectionStateChange(connectedState, connectedState, null, error));
}
, and more generally it doesn't emit a state event if it is currently in the same state as attempting to notify:
/* do nothing if we're already in the indicated state */
if (state == this.state.state) return;

And relevant spec point: https://sdk.ably.com/builds/ably/specification/main/features/#RTN24

based on this we can do the following:

  • have initialized, syncing, synced and update events/states.
  • we won't (generally) emit a state event if we're currently in it (can't receive two syncing events in a row).
  • if we're currently in synced and:
    • receive a STATE_SYNC message that can be immediately applied (sync cursor is empty), we will emit an update event and stay in synced. otherwise we enter syncing and wait for sync sequence to complete as usual.
    • we reattach to a channel with no HAS_STATE flag - we emit update event and stay in synced
  • also, I suggest we still allow to subscribe to events even if state_subscribe mode wasn't set (meaning we won't receive a HAS_STATE flag on attachment) to improve DX (say you attach to a channel without the mode, get liveobjects instance and subscribe to lifecycle events now, and only at some time in the future you reattach with correct modes. if we don't allow subscribing unless state_subscribe mode is set this no longer becomes possible). To maintain a consistent syncing -> synced order, when a channel is attached while in the initialized state 1. it will transition to syncing. 2. on the next event loop, it will emit synced since it is immediately synchronized due to the absence of a HAS_STATE flag.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated implementation in:
98d66e2 - adds state to LiveObjects just like channel/connection
cf6f5fd - emits state change events

as discussed on the standup yesterday, I removed the update event as it will have different meaning here than in channel/connection (update here would simply mean that underlying data has changed, and we have live object subscriptions for that).

Copy link
Contributor Author

@VeskeR VeskeR Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also should point out that it is not technically possible to receive a initialized event as it is set on channel/connection object construction at which point we can't event subscribe to events yet.
and no other state can transition back to initialized, so it's impossible to get it later too.

thus we don't emit initialized in liveobjects either, we just start from this state

@VeskeR VeskeR force-pushed the DTP-948/error-wrong-state-mode-attach branch from 5d78643 to a75d500 Compare January 30, 2025 02:49
@VeskeR VeskeR force-pushed the DTP-1034/liveobjects-lifecycle-events branch from eedbe01 to 1f062f0 Compare January 30, 2025 02:50
@github-actions github-actions bot temporarily deployed to staging/pull/1958/features January 30, 2025 02:51 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/typedoc January 30, 2025 02:51 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/bundle-report January 30, 2025 02:51 Inactive
@VeskeR VeskeR requested a review from mschristensen January 30, 2025 09:34
@VeskeR VeskeR force-pushed the DTP-948/error-wrong-state-mode-attach branch from a75d500 to 53d07ff Compare January 31, 2025 02:02
@VeskeR VeskeR force-pushed the DTP-1034/liveobjects-lifecycle-events branch from 1f062f0 to 6824c38 Compare January 31, 2025 02:04
@github-actions github-actions bot temporarily deployed to staging/pull/1958/bundle-report January 31, 2025 02:04 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/features January 31, 2025 02:04 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/typedoc January 31, 2025 02:05 Inactive
Base automatically changed from DTP-948/error-wrong-state-mode-attach to integration/liveobjects January 31, 2025 02:06
@VeskeR VeskeR force-pushed the DTP-1034/liveobjects-lifecycle-events branch from 6824c38 to 8c2a8de Compare January 31, 2025 03:37
@VeskeR VeskeR changed the base branch from integration/liveobjects to DTP-1147/fix-flaky-liveobjects-tests January 31, 2025 03:37
@github-actions github-actions bot temporarily deployed to staging/pull/1958/bundle-report January 31, 2025 03:37 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/features January 31, 2025 03:37 Inactive
@github-actions github-actions bot temporarily deployed to staging/pull/1958/typedoc January 31, 2025 03:38 Inactive
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between eedbe01 and 8c2a8de.

📒 Files selected for processing (5)
  • src/plugins/liveobjects/liveobject.ts (8 hunks)
  • src/plugins/liveobjects/liveobjects.ts (6 hunks)
  • src/plugins/liveobjects/objectid.ts (0 hunks)
  • test/common/modules/live_objects_helper.js (2 hunks)
  • test/realtime/live_objects.test.js (78 hunks)
💤 Files with no reviewable changes (1)
  • src/plugins/liveobjects/objectid.ts
👮 Files not reviewed due to content moderation or server errors (1)
  • test/realtime/live_objects.test.js
⏰ Context from checks skipped due to timeout of 90000ms (6)
  • GitHub Check: test-browser (webkit)
  • GitHub Check: test-node (20.x)
  • GitHub Check: test-browser (firefox)
  • GitHub Check: test-node (18.x)
  • GitHub Check: test-browser (chromium)
  • GitHub Check: test-node (16.x)
🔇 Additional comments (12)
src/plugins/liveobjects/liveobject.ts (6)

Line range hint 6-36: LGTM! Clear separation of subscription and lifecycle events.

The enums and types are well-structured with a clear separation between subscription and lifecycle events. The naming convention follows a consistent lowercase pattern.


43-44: LGTM! Good separation of event emitters.

The split into separate event emitters for subscriptions and lifecycle events improves separation of concerns and maintainability.


69-70: LGTM! Consistent event emitter initialization.

Both event emitters are correctly initialized with the client logger.


109-133: LGTM! Well-implemented lifecycle event methods.

The lifecycle event methods are well-implemented with:

  • Consistent pattern matching subscription methods
  • Proper null checks to prevent accidental removal of all callbacks
  • Clear and concise method signatures

Line range hint 144-154: LGTM! Clean update to use new event enum.

The method correctly uses the new subscription event enum while maintaining the existing noop check logic.


166-166: LGTM! Proper lifecycle event emission.

The deleted event is correctly emitted after setting the tombstone state.

src/plugins/liveobjects/liveobjects.ts (3)

32-34: LGTM! Good separation of internal and public event emitters.

The split into separate event emitters for internal and public events improves encapsulation and aligns with RTC10 spec.


48-49: LGTM! Consistent event emitter initialization.

Both event emitters are correctly initialized with the client logger.


154-179: LGTM! Well-implemented event methods.

The event methods are well-implemented with:

  • Consistent pattern across methods
  • Proper null checks to prevent accidental removal of all callbacks
  • Clear and concise method signatures
test/common/modules/live_objects_helper.js (3)

28-29: LGTM! Exposing ACTIONS as static property improves test maintainability.

Making ACTIONS accessible as a static property allows for better test assertions and reduces duplication.


30-32: LGTM! Centralizing fixture root keys improves maintainability.

The static fixtureRootKeys method provides a single source of truth for fixture keys, making it easier to maintain and update test data.


301-301: LGTM! Adding timestamp to object IDs ensures uniqueness.

Appending Date.now() to object IDs helps prevent collisions in tests, especially when running tests in rapid succession.

Also applies to: 305-305

src/plugins/liveobjects/liveobjects.ts Outdated Show resolved Hide resolved
src/plugins/liveobjects/liveobjects.ts Outdated Show resolved Hide resolved
src/plugins/liveobjects/liveobjects.ts Outdated Show resolved Hide resolved
@VeskeR VeskeR force-pushed the DTP-1034/liveobjects-lifecycle-events branch from 8c2a8de to 3c40056 Compare January 31, 2025 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants