Track:None

[Nirjal444]

Hello,
I am new to sleap and I have a deadline to finish this project. I am confused in multiple steps:

1. I ran the training(topdown_test) first, got predictions and relabel those, ran training again(200 epochs) relabel those and again ran training/inference(1000 epochs). However, it is still not detecting one of my mice in most of the predicted frames. (How many times usually a user have to run training and relabel it). Is this process called human-in-the-loop training??
   

2. Lets assume, I am satisfied with the predicted frames. Now, I am on track proofreading and I see that all my instance when selected says track:none.
   

I fixed a few frames by holding ctrl and assigning the track. Is it normal that a user has to assign track for all the labeled and predicted frames? Also, the lines will pass through the successive frames. Is that something we are expecting or is it some error?

3. Can I export data for behavior analysis after I fix the track for all the frames, or is there any other steps involved? I am not sure, when I am actually done.
4. Do we get the tracking data in hdf5 file all the time or it can be made as csv file as well?

Thank you so much in advance and I am looking forward to hearing from you.

~Nirjal
.

[roomrys]

Hi @Nirjal444,

Let's see if we can help clarify things:

1. Yes, the process of correcting predictions and re-running training is what we refer to as human-in-the-loop training. The number of times you need to iterate depends from case to case and is largely subjective to your model's hyperparameters (if the hyperparameters are set incorrectly, then the model may never yield good predictions). Usually, the process starts with labeling 5-10 frames and then correcting 10-15 frames per round before training and predicting again. The first round with only 5-10 frames will have rough predictions, but predictions should be noticeably better by the third round. Again, the performance largely depends on the project and the hyperparameters.

2. SLEAP will assign tracks if you specify a tracking method in the Inference Pipeline (Predict > Run Inference); otherwise, the track will be listed as None.
   
   Fig 1: The user will need to specify a tracking method in the Inference Pipeline for SLEAP to assign tracks.
   ~
   The lines passing through your frames are likely due to a non-zero trail length (View > Trail Length) which trails where instances have been tracked for <Trail Length> number of frames. To get rid of these trails, you can set the Trail Length to 0.

3. When your model is trained to a point where the predictions look good, you will need to predict on all frames (with a tracking method specified). Then you will need to proofread to correct tracks. After the tracks are corrected, then you should be ready to export.

4. SLEAP exports analysis files to an hdf5 since we have multiple tables to stack into a single file. But, we have a notebook to help users navigate the analysis hdf5 (from there it is possible to create a csv with the specific info you need - you will need to add a few lines of code to do so). Some third-party libraries that will be useful are (pandas.DataFrame.to_csv)

Let us know if you need any more clarification or help on anything!

Thanks,
Liezl

[Nirjal444]

Thank you so much for the clarification. I will follow up if I have any more questions.

[Nirjal444]

I have multiple tracker when I try to fix the tracking. Is there a reason that track gets added? I only have 2 mice but 10 tracks. How do I remove these tracks so that it will track correct everytime?

As per the explanation, after I run inference with prediction on all videos, do I have to fix track for all (approx 261000) frames?

I am planning to run the analysis on SimBA, so is the slp file from run inference enough or do I need to export the data from file menu.?

Looking forward to hearing from you.

Thank you for your help.

.

[roomrys]

Hi @Nirjal444,

Sorry for the delay!

The extra tracks belong to the predicted instances (which should be the same as the user-created instances - we have created an issue here to fix this). You can delete tracks by navigating to Tracks > Delete Tracks and selecting the name of the track you want to delete. Unfortunately, the track names do not need to be unique (also something that we will need to implement), but will have the same ordering as the track key in the upper left hand corner.

You should be able to Tracks > Propagate Track Labels to switch the tracks of subsequent frames by changing them in the current frame only.

SimBA expects the SLP file, so no need to export to an analysis file.

Hope this helps! And thanks for bringing up the new ideas for improvement.

Thanks,
Liezl

[Alan-Marcus]

4. Do we get the tracking data in hdf5 file all the time or it can be made as csv file as well?
4. SLEAP exports analysis files to an hdf5 since we have multiple tables to stack into a single file. But, we have a notebook to help users navigate the analysis hdf5 (from there it is possible to create a csv with the specific info you need - you will need to add a few lines of code to do so). Some third-party libraries that will be useful are (pandas.DataFrame.to_csv)

Hi,
I just wanted to add a notebook link that might be helpful for anyone wanting their data as a csv. This is based on code that Talmo provided a while ago. This notebook will output the xy coordinates of all nodes of every track for all frames.
(Where the notebook references "scale analysis file" this is just your h5 file from SLEAP. You can upload straight to the notebook temporary storage and skip the Google Drive step if desired).

SLEAP Scale

Cheers,
Alan