Align the depth and rgb frames from the .bag file (D415 camera).

[arake001]

I checked that the frames have different numbers of aligned frames when using align color to depth or vice versa. How can I align the entire bag file such that RGB and depth frames are aligned and there is no drop of frames?

Also, the RGB and depth frames should be synced. How can I achieve this?

[MartyG-RealSense]

Hi @arake001 A RealSense SDK bag file cannot store aligned frames, only individual streams. You need to load the bag file so that its recorded streams can be used as the camera data for a program instead of a live camera and then perform alignment of the bag's streams in real-time.

It is unlikely that you will be able to record 100% of the frames for both streams and in most cases there will be dropped frames to some extent. The best that you can do is attempt to minimize the number of dropped frames.

If you are writing your own program script then increasing the frame queue size value to introduce latency into the frame pipeline can help to reduce frame drops.

https://github.com/IntelRealSense/librealsense/wiki/Frame-Buffering-Management-in-RealSense-SDK-2.0#latency-vs.-performance

Sync between the depth and RGB sensors of the camera should automatically kick in when the FPS speed of both streams is the same. A method that helps to achieve this is to have auto-exposure enabled and an RGB option called Auto-Exposure Priority disabled so that the SDK tries to force the two streams to have the same FPS instead of permitting FPS to vary.

#8726 (comment) has a list of discussions about depth-RGB sensor sync on the same camera.

[MartyG-RealSense]

Hi @arake001 Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.