Add checkpoints during replay

[Yuyz0112]

We need to improve the 'play at any time' feature which supports rrweb-player's DND interactions.

Currently, we do this by introducing sync mode which will rebuild snapshot before time offset synchronously and then play under the timer. But this may cause a performance issue when we try to play at a large time offset.

For example, if we have 100 snapshots and we play in the third snapshot's time offset, we need to rebuild 3 snapshots synchronously which is fine. When we are trying to play at the 99th snapshot's time offset, we need to rebuild 99 snapshots synchronously which will block everything.

So the solution is to build checkpoints when we create a replayer. We may cache the result of rebuilding every 50 snapshots, so the maximize synchronously snapshots rebuilding number is 50.

[DvdGiessen]

When this is being refactored, it might be valuable to separate three distinct concepts here?

Right now we have the "storage of events", which are then converted in "playback" actions, which are then executed by timing logic. However, these three are tightly integrated, which makes it a bit hard to use one of the specific elements but not the others.

In the current code, replay/timer.ts is both concerned with storing a list of actions, and timing their playback, while replay/index.ts mostly handles generating actions which modify a output iframe but is also dependant on certain timer functions.

For example for the case of live streaming it'd be nice if I was able to reuse the event storage and playback action generation, but supply my own timing logic. Or, another example, if I wanted to output to multiple iframes, I'd like to still use the event store but have multiple replay functions use it. I'm also interested in allowing the use of a more abstracted clock for the timing logic so that other time-bound elements may tie in to it.

Splitting the concepts up might look like this:

• Event store: Just stores the events. It may do some preprocessing, but it isn't directly concerned with actually replaying anything. It provides a method to add events, and one to retrieve events between two timestamps, and probably an emitter to notify other objects its contents have been updated.
• Replay renderer: Only handles actual replaying of events, doesn't store them or determine when they should be replayed. Is given an output element and an event store, and provides a single method render(time: number) which renders the replay in its output element for the given timestamp.
• Timer loop: A requestAnimationFrame loop which calls the render(time: number) method of the replay renderer to actually make it play back. This would then also be the only part to be connected to the playback controls in the interface.

The event store would then be the place where these checkpoints (which would basically be full snapshots) are (pre)computed, so that it can return a small number of events for the replay rendererer to render to go to a certain point in time.

The replay renderer should remember the last timestamp it rendered, and only request a delta from that timestamp instead of having to render from the very first full snapshot. It might also do some higher level caching by storing a clones of rendered snapshots so it won't have to reconstruct them.

I know this is only laterally related to this issue, but it might be useful to think about before effort is put into building the checkpoint logic so that if this is something we'd like and want to consider, one can somewhat account for it while tackling this issue.

[remisharrock]

Could you try the CODECAST player scrolling https://codecast.wp.imt.fr/codecast/ maybe you can find a way to have fast scrolling to any point in time in the source code https://github.com/France-ioi/codecast ?

[Yuyz0112]

@remisharrock Thanks, I'll take a look.

[remisharrock]

I found the name of the library that enables the fast seeking through the recording: Redux-Saga, as you can see in this file : https://github.com/France-ioi/codecast/blob/master/frontend/player/sagas.js
It says that

// An instant has shape {t, eventIndex, state},
// where state is an Immutable Map of shape

My guess is that the shape could be a VirtualDOM ?
Also the code has a lot of synchronization with the audio being played simultaneously.

By the way, you can try the recorder here https://codecast.france-ioi.org/v6/recorder by clicking the "guest" button , authorizing the use of your microphone, clicking the record button (red "start recording" and by typing some code in the editor (also selecting text, scrolling, compiling if it's C code and executing step by step etc...)

Interesting here: when you click the pause button you can go back a little bit in the recording and start from this point.

Another thing interesting here: when you click the stop button, the audio is being compressed to mp3 client-side (using a javascript mp3 encoding library) ;

Finally, when you click the save button, the JSON containing the events and the mp3 file is being sent to a server and you can play back the recording using the generated URL (playback link).

Can you try it and tell me what you think ?

Also try to seek during playback and also drag and drop the time cursor, you will see immediate changes, which I think is pretty cool ;)

Hope that you can find a good solution from this project, in order to have this kind of reactivity when you seek through the recording during playback !

[remisharrock]

Also it looks like another dev has a slightly more updated CODECAST project (forked) and is working on it right now here: https://github.com/pkrll/codecast

[remisharrock]

Any news on checkpointing or an approach to fast forward in any point in time very fast?

[buremba]

This would be a great feature IMO. We plan to use rrweb for product demos and definitely take advantage of it.