[D455] The extrinsic parameters between the left camera and the IMU for D455

[zc08]

Required Info
Camera Model	D455
Firmware Version	05.12.06.00
Operating System & Version	Ubuntu 18.04.3
Kernel Version (Linux Only)	5.4.0-48-generic
Platform	PC
SDK Version	2.36.0
Language	C/C++/python
Segment	Robot

Issue Description

I'm trying to run ORB-SLAM3 with Stereo-Inertial input using Intel Realsense D455. Specifically, I intend to use the .bag file saved by the realsense-viewer as input.
I was able to extract stereo image pairs as well as camera intrinsics/baseline, accelerometers and gyroscope readings from the .bag file. However. I'm confused about the extrinsics between the IMU and the left camera (i.e. Infrared_1).
The messages that I found in the realsense-viewer-saved .bag file that could be related to this are:

topic: /device_0/sensor_2/Accel_0/tf/0, t.secs:0, t.nsecs:1
translation:
x: -0.030220000073313713
y: 0.007400000002235174
z: 0.016019999980926514
rotation:
x: 0.0
y: 0.0
z: 0.0
w: -1.0
topic: /device_0/sensor_2/Gyro_0/tf/0, t.secs:0, t.nsecs:1
translation:
x: -0.030220000073313713
y: 0.007400000002235174
z: 0.016019999980926514
rotation:
x: 0.0
y: 0.0
z: 0.0
w: -1.0

However it's still not clear to me, how the transformation matrix that transforms a point from the left camera coordinate frame to the IMU coordinate frame, could be formed. Really need help on how this should be done, or how to do this with realsense ROS wrappers or SDK.

Thanks in advance!

PS. I found that an image describing the coordinate systems for T265 seems very intuitive, are there similar pictures for D455?

[MartyG-RealSense]

Hi @zc08 The closest example to your image that I could find is the image below that contains a D435i, found at https://lifelong-robotic-vision.github.io/dataset/scene.html

As far as I know, the IMU coordinate system for the D455 is the same as the D435i.

If you are seeking information on aligning IMU data to depth: I do not have such information specifically for D455, though IMU to depth alignment information is provided in the link below:

https://github.com/IntelRealSense/librealsense/blob/master/doc/d435i.md#sensor-origin-and-coordinate-system

Also, here is a quote from the documentation about the D435i IMU.

---

The D435i depth camera generates and transmits the gyro and accelerometer samples independently, as the inertial sensors exhibit different FPS rates (200/400Hz for gyro, 63/250Hz for accelerometer). Each IMU data packet is timestamped using the depth sensor hardware clock to allow temporal synchronization between gyro, accel and depth frames.

[zc08]

Thank you very much!
I want to get the transformation matrix that transforms a point in the left camera frame (Infrared_1) to the IMU frame.
According to the figure, there is only a translation between several coordinate frames.
The remaining question is: The message I get from the realsense-viewer-saved .bag file (below), it is the transformation from which coordinate frame to which one? E.g. Does it transform a point in the left camera frame (Infrared_1) to the IMU frame, or does it transform a point in the depth frame to the IMU frame, or inverse?

topic: /device_0/sensor_2/Accel_0/tf/0, t.secs:0, t.nsecs:1
translation:
x: -0.030220000073313713
y: 0.007400000002235174
z: 0.016019999980926514

Thank you!

[MartyG-RealSense]

Your remaining question is somewhat outside of my knowledge unfortunately. The documentation for the D435i states though that "The depth<->IMU sensor extrinsic (rigid body transformation) is precalculated based on mechanical drawings and cannot be modified".

[MartyG-RealSense]

Hi @zc08 Do you still require assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.