Follow-up on Issue 83: Tracking drifting towards the end fo the samples

[KomputerMaster64]

This is in continuation of the Issue #83

the bottom two images show a significant jump which is not very suitable for tracking. You can increase the camera frame rate if possible. Otherwise, you can also try with higher iter

FoundationPose/run_demo.py

Line 21 in fce37c8

parser.add_argument('--track_refine_iter', type=int, default=2)

Hi @wenbowen123

I am sorry for the delayed response.
I ran the experiments with values for track_refine_iter variable ranging from 2 to 50, however it was observed that the tracking invariably drifted towards the end samples (last 15 to 20 percent of the video).
I would be delighted to share the files.

Regards,
Prabhav

__
Edit: I am sharing the failure case for the case with track_refine_iter variable set to 50.

Remarks: For the shown case, the tracking failure happends towards the end of the samples. The tracking is optimal till image 1714129813249.png i.e. for the first 410 samples (Total of 619 image samples).

Additional Remarks: This is an improvement it to the first case where track_refine_iter variable was set to 2, and the tracking was optimal only upto the 157 image samples.

I have uploaded the new FoundationPose_demo_05_eri-5_tri-50.zip file here (this is corresponding to the track_refine_iter variable set to 50) in the Google Drive Folder.

Question/Note: The total size of the debugging directories for the 49 experiments is 7 GB. It would be great if you could confirm if it is required to share all of the experiments' files.

__
The last frame with the correct tracking (top), and the frame after which the trakcing was not optimal (bottom).

[wenbowen123]

I checked your logs. The object is moving too fast at 1714129813249. Your camera frequency is not fast enough.

[KomputerMaster64]

Thank you for the response. I will check for better camera hardware. Do you recommend performing actions at a slower pace?
I have been using the Depth Camera D435i - Intel RealSense.
I will be testing FoundationPose with the Depth Camera D455 - Intel RealSense.

[wenbowen123]

did you check how much fps is the camera images? The upper bound of realsense is 30hz, but sometimes it could be lower than that. In the video, you are ejecting the cube to perform free-form sliding, that moment the motion is really big.