Prediction --> Editable Instance is taking too long

[lisadiez]

Hi!

We have a project with 23,011 predicted frames and are starting to correct the predictions. We noticed the conversion from predicted instance to editable instance is taking a long time, as well as ID changes.

delay.mov

I'm not sure you can appreciate it in the screen recording, but it's taking around 3-5s after I double-click.

Is this because we have too many predicted frames?

[talmo]

Some preliminary analysis @roomrys and I did:

For a project with 1 video, 8 Tracks, no Suggestions, ~270k LabeledFrames, all with <=8 PredictedInstances in each, we see a similar latency spike when creating a user Instance from a PredictedInstance.

On my PC, scrolling through the frames calls on_data_update which runs in about ~1 ms. Note: Scrubbing by dragging on the seekbar does NOT set the update topics to trigger self.plotFrame() -- this is handled via the frame_idx context signal rather than in on_data_update.

Ok now, creating an Instance from a PredictedInstance is quite slow. This calls the AddInstance command, which sets the UpdateTopic.frame, UpdateTopic.project_instances, UpdateTopic.suggestions topics for updating.

Doing some profiling, reveals that this part of on_data_update takes about 89 ms:

sleap/sleap/gui/app.py

Lines 1310 to 1318 in 31cb5a8

	if _has_topic(
	[
	UpdateTopic.frame,
	UpdateTopic.skeleton,
	UpdateTopic.project_instances,
	UpdateTopic.tracks,
	]
	):
	self.plotFrame()

I'm guessing this is because we have to draw the yellow "reminder" boxes around the other instances? We can test this by profiling the lower level QtVideoPlayer.plot() method to see if just rendering frames with user instances in these conditions is slow in general.

Meanwhile, here's the big culprit -- this part of on_data_update takes 264 ms:

sleap/sleap/gui/app.py

Lines 1320 to 1328 in 31cb5a8

	if _has_topic(
	[
	UpdateTopic.frame,
	UpdateTopic.project_instances,
	UpdateTopic.tracks,
	UpdateTopic.suggestions,
	]
	):
	self._update_seekbar_marks()

The rest takes <1ms and the total on_data_update takes 356 ms.

Summarizing:

Total: 356 ms
plotFrame: 89 ms (25%)
_update_seekbar_marks: 264 ms (74%)

So these are the places we need to optimize. We'll probably get a big enough boost by just fixing how we render the seekbar.

We'll work on this and update this thread when we have a potential solution.

[catubc]

Thank you both, this is very helpful. It wasn't completely clear why this scales with # of video frames!?

In any case, we will decimate the dataset to every 4th or 5th frame for now while we wait for a speedup.