Stream IMU through callback (push) and cameras through wait_for_frames() (pull)

[rafaelspring]

Required Info
Camera Model	D435i, L515
SDK Version	2.34.0
Language	C++

I would like to have 3 streams: Depth, Color and IMU. I'd like to receive Depth and Color through the standard pull-based mechanism (pipeline.wait_for_frames()) and I would like to receive IMU data through a push-mechanism (callback).

If I use two pipelines (one for depth/color and one for IMU) that approach works on the L515, but not on the D435i (I get an exception when I call config.resolve(imu_pipe);. But even on the L515 starting and stopping the streams takes much longer when the IMU is involved than without (stopping the IMU takes about 15 seconds on my i7).

Is there a way to stream IMU through a callback without using a second pipeline?
What's a way that works efficiently and works on all RS cameras with a motion module?

[MartyG-RealSense]

Hi @rafaelspring I don't have an L515 yet to run tests with. I wonder if the link below may provide some useful guidance as a starting-point for discussion though:

#2647

[ev-mp]

@rafaelspring , please post the pipelines configuration code for review.

[rafaelspring]

@ev-mp I'm working on a minimal example reproducing the issue. Will share this shortly.

[rafaelspring]

@ev-mp
I am still trying to untangle things here. Filed three new tickets so far related to this issue: #6424 #6425 #6426
I would still be interested in best practices for using IMU through push and color/depth through pull.

[rafaelspring]

@ev-mp

I would still be interested in best practices for using IMU through push and color/depth through pull.

Actually the reason for using a pull-based mechanism through pipe.wait_for_frames() for the color/depth is because of assumed higher efficiency vs. using a callback to provide the frames (push). The assumption is that wait_for_frames() would actually wait for some kind of interrupt and only then "pull" the frames through the driver.

If, however, pipe.wait_for_frames() is implemented under the hood as just a wrapper around another push-based mechanism then the driver would handle each frame anyway and there would be no efficiency gain. In that case we'd just move everything to the callback / push mechanism and no longer worry about pipe.wait_for_frames() (one remaining question would be how depth/color sync is handled when frames are delivered through a callback).

Please let me know.