How to query the external parameters from camera to imu

[canglangzhige]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Hello, I run ./realsense-viewer, and I can find stereo baseline/ right intrinsics/left intrinsics in Calibration data and so on. As shown in the following figure:

But there is no coordinate system transformation of the camera relative to the imu. How can I query the factory parameters of the camera's coordinate system transformation relative to the imu?

Required Info
Camera Model	{ D400 }
Firmware Version	05.13.00.50
Operating System & Version	Linux (Ubuntu 20)
Kernel Version (Linux Only)	(e.g. 5.13.0-41)
Platform	PC
SDK Version	{ legacy / 2.0 }

Issue Description

<Describe your issue / question / feature request / etc..>

[MartyG-RealSense]

Hi @canglangzhige You can launch the RealSense SDK's 'rs-enumerate-devices' tool in calibration information mode with the command rs-enumerate-devices -c to list the extrinsics of the camera that include the translation between different sensors (including gyro and depth, accel and depth, etc).

The information shown below was retrieved from my D455 camera model. You should query your own camera to find its unique extrinsic values.

[canglangzhige]

Hi @canglangzhige You can launch the RealSense SDK's 'rs-enumerate-devices' tool in calibration information mode with the command rs-enumerate-devices -c to list the extrinsics of the camera that include the translation between different sensors (including gyro and depth, accel and depth, etc).

The information shown below was retrieved from my D455 camera model. You should query your own camera to find its unique extrinsic values.

Thanks for your answer！
I combined gyro and accel with linear_interpolation in the file rs_camera.launch:

I have launched the calibration information with the command rs-enumerate-devices -c.
Extrinsic from "Infrared 2" To "Accel" is the same with Extrinsic from "Infrared 2" To "Gyro" :

Can I assume that both parameters are Extrinsic from "Infrared 2" To "IMU" ?
In addition, what are the steps for automatic calibration of realsense d455?
The parameters I calibrated by the Kalibr method are very different from the factory parameters. How much will the automatic calibration differ from the factory parameters?
If I don't calibrate and use the factory parameters directly, will it have a big impact on my results?

[canglangzhige]

Hi @canglangzhige You can launch the RealSense SDK's 'rs-enumerate-devices' tool in calibration information mode with the command rs-enumerate-devices -c to list the extrinsics of the camera that include the translation between different sensors (including gyro and depth, accel and depth, etc).

The information shown below was retrieved from my D455 camera model. You should query your own camera to find its unique extrinsic values.

Thanks for your answer！
I combined gyro and accel with linear_interpolation in the file rs_camera.launch:

I have launched the calibration information with the command rs-enumerate-devices -c.
Extrinsic from "Infrared 2" To "Accel" is the same with Extrinsic from "Infrared 2" To "Gyro" :

Can I assume that both parameters are Extrinsic from "Infrared 2" To "IMU" ?
In addition, what are the steps for automatic calibration of realsense d455?
The parameters I calibrated by the Kalibr method are very different from the factory parameters. How much will the automatic calibration differ from the factory parameters?
If I don't calibrate and use the factory parameters directly, will it have a big impact on my results?

[MartyG-RealSense]

Accel and Gyro both come from a single IMU component inside the camera, so it makes sense that they would both have the same physical position in relation to other sensors.

Any time that the transform between two sensors is being described, it is an Extrinsic instead of an Intrinsic.

The D455 camera model has a compensation mechanism intended to mitigate the effect of thermal propagation in optics. When active (default = On) it will track and adjust Depth and RGB calibration parameters automatically.

There is no way to tell in advance how automatic calibration would change the values in comparison to the original factory-default calibration values.

As you are already familiar with using the camera calibration data interface in the RealSense Viewer, it may be worth using the guide at #10182 (comment) to reset the camera to its factory defaults using this interface and write it to the camera hardware to see if it works as well for you as performing a re-calibration.

[canglangzhige]

Accel and Gyro both come from a single IMU component inside the camera, so it makes sense that they would both have the same physical position in relation to other sensors.

Any time that the transform between two sensors is being described, it is an Extrinsic instead of an Intrinsic.

The D455 camera model has a compensation mechanism intended to mitigate the effect of thermal propagation in optics. When active (default = On) it will track and adjust Depth and RGB calibration parameters automatically.

There is no way to tell in advance how automatic calibration would change the values in comparison to the original factory-default calibration values.

As you are already familiar with using the camera calibration data interface in the RealSense Viewer, it may be worth using the guide at #10182 (comment) to reset the camera to its factory defaults using this interface and write it to the camera hardware to see if it works as well for you as performing a re-calibration.

As you said " When active (default = On) it will track and adjust Depth and RGB calibration parameters automatically." Can Infrared 1 and Infrared 2 be automatically calibrated?
Four other questions:

1. How do I turn off infrared when using ROS？
2. Why is the parameter Coeffs of Infrared 1/2 all 0, is the Infrared 1 or Infrared 2 I get undistorted? :
   
   Why is this parameter not all 0 for color images?
   
3. Topic "Infrared" refers to what video stream？
4. Is there any documentation detailing these issues?

[MartyG-RealSense]

As far as I am aware, the Infrared and Infrared 2 streams are not automatically calibrated. Although depth frames are generated from frames produced by the left and right infrared sensors, it is important to bear in mind that they are using raw IR frames in the camera hardware at the point of capture and not the Infrared and Infrared 2 streams. This is why depth can be streamed even if the Infrared and Infrared 2 streams are not enabled.

1. In ROS, if you want to disable Infra and Infra2 in the RealSense ROS wrapper then you can do so by including enable_infra1:=false enable_infra2:=false in your roslaunch instruction. However, infra and infra2 are already disabled by default in the wrapper anyway and so there is no need to disable them in the launch.

2. rs2_intrinsics coeffs[] all 0 by default #1430 (comment) provides the reason for why all coefficients are often zero. Since that advice was given, there have been exceptions to the rule though, such as the Color coefficients on the D455 model being non-zero. Distortion models are still applied even if all coefficients are zero. However, the Y16 infrared format is unrectified and does not have a distortion model applied.

3. 'Infrared' refers to the left infrared sensor, whilst 'Infrared 2' is the right infrared sensor.

4. If you mean documentation for intrinsics and extrinsics then the Projection documentation page is the main reference source.

https://dev.intelrealsense.com/docs/projection-in-intel-realsense-sdk-20

[canglangzhige]

1. But the documentation says that the depth is generated by the left and right cameras, and the IR frames just increase its accuracy.
   
2. If I only use the left and right cameras (using the d455 as a stereo camera), will the IR frames affect the accuracy of the left and right cameras because I see a lot of laser dots on the left and right images?
3. How to turn off IR frames when using ROS？
4. Are you saying that "enable_infra" and "enable_infra 1" in the parameter are the same?
5. How to get color left and right images? I changed the parameter below to "true" but it didn't work.
   
   If I use the following set of parameters, the topics only have "infra1" and "infra2", and no "infra":
   
   If I set "infra_rgb" to "true", then only colored "infra":
   
   If I set "enable_infra" to "false" and "infra_rgb" to "true", then there is no talk about "infra":
   
   What is the relationship between these parameters, and how can I get a color image of the left and right cameras?

[MartyG-RealSense]

1. The infrared projector is not involved in constructing depth frames. It is a separate component from the left and right IR sensors that projects a semi-random pattern of dots onto the scene that the camera is observing. The camera can use these dots that are projected onto surfaces in the scene as a 'texture source' to analyze that area for depth information. Turning the projector off can decrease the depth image quality but it does not prevent depth frames from continuing to be generated.

2. Because the infrared dots aid depth analysis, the more dots that are in the scene the better. For example, when two cameras are used and their fields of view overlap then there can be improved depth quality because the two dot patterns are overlapping too and multiplying the density of analyzable dots.

3. As mentioned above, in the RealSense ROS wrapper Infra and Infra2 are disabled by default. You can also disable the projector and its IR Emitter to remove the dot pattern, as described at Emitter Turns On Automatically realsense-ros#1379

4. For non-stereo RealSense camera models such as SR300 and L515 that only have one infrared stream, enable_infra may be a more appropriate option than infra1 / infra2.

5. On camera models that support color from the infrared sensor (D405, D415, D455), color is only available from the left infrared sensor and not the right infrared sensor.

[canglangzhige]

Can I get a color left infrared sensor and a gray right infrared sensor at the same time and how?

[MartyG-RealSense]

If left color is streamed then right monochrome infrared cannot be streamed at the same time. You can either have color from left infrared on its own or left and right infrared.

[canglangzhige]

OK. Thank you for your patience.

[canglangzhige]

Can I use ROS to subscribe and save trajectories that the d455 has run? In what way? What algorithm does d455 use to calculate its own odometry? Which sensors were used?

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "IntelRealSense/librealsense" ***@***.***&gt;; 发送时间:&nbsp;2022年6月7日(星期二) 晚上10:34 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [IntelRealSense/librealsense] How to query the external parameters from camera to imu (Issue #10568) If left color is streamed then right monochrome infrared cannot be streamed at the same time. You can either have color from left infrared on its own or left and right infrared. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***&gt;

[MartyG-RealSense]

Only the RealSense T265 Tracking Camera has built-in support for trajectory tracking and device pose.

It is possible to implement relative position tracking and trajectory plotting with an IMU-equipped D435i or D455 model using an adaptation of Intel's SLAM guide for ROS that a RealSense user created at the link below.

https://shinkansan.github.io/2019-UGRP-DPoom/SLAM.html

Alternatively, if it is possible for you to purchase a T265 then it could be mounted on a bracket with the D455 to provide trajectory whilst the D455 provides the depth data. Intel have a guide for doing so here:

https://www.intelrealsense.com/depth-and-tracking-combined-get-started/

[MartyG-RealSense]

Hi @canglangzhige Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.