L515 depth-camera intrinsic parameters confilct btw SDK and self-calibration\.

[StevenZzz07]

Required Info
Camera Model	{ L515 }
Operating System & Version	{Win 10
Platform	PC
Language	python }
Segment	VR

Issue Description

Hi there. I just came with a problem that the depth-camera intrinsic parameters brought by librealsense SDK are not equal to the calibrated results by myself. For SDK, I got intrinsic of depth-camera by SDK as follows:

And i utilize the Zhang's calibration using chess borad to obtain instrinsic parameters on IR images. The followings are part of images i used to compute paramters:

And the results are：

As we can see, the fx and fy confilcts. Also, the SDK does not give out the distortion parameters.
What's more, I also use two sets of intrinsic parameters to convert the same depth map to point cloud. There also exist a offset btw two point cloud (red for SDK paras and blue for self-calib):
https://user-images.githubusercontent.com/56421441/201652140-1f91d3b7-ae06-41bb-b00d-6d961a33cdab.mp4

So, which parameters should i trust? The one provided by librealsense or my own calibration results? Also, can you give some hints for how to eliminate the distortion on depth image?

Looking forward to ur reply. Thanks!

[MartyG-RealSense]

Hi @StevenZzz07 It is not unusual for there to be differences between RealSense calibration data and calibration information obtained from third-party tools such as Kalibr.

For example, the 5 coefficients are typically all set to zero in the official intrinsics (except for the color stream, whose coefficients can be non-zero). They are purposely set to zero for reasons explained by a RealSense team member at #1430 (comment)

I would always recommend to rely on the calibration data provided by the RealSense SDK and official RealSense tools such as rs-enumerate-devices as they are designed specifically for RealSense camera hardware.

If the hand is being observed by the L515 camera at very close range then the missing areas of red on the RealSense cloud may be accounted for by the hand being slightly below the L515's minimum depth sensing range of around 25 cm from the camera. When an observed object moves closer to the camera than the minimum distance then the depth image will begin progressively breaking up as the surface gets closer and closer to the camera.

Setting the L515's Min Distance option to '0' instead of its default of '400' with Python code at #8244 (comment) may improve your RealSense hand image.

[StevenZzz07]

Hi, @MartyG-RealSense Thanks for ur quick and detailed reply. As for your response, i still have some questions as follows:

1. As you can see, i obtain the intrinsic parameters by get_profile().as_steam_profile() function. I am curious about the inner mechanism inside this function. Does it work just by copying a default parameters which is fixed after the sensor delivery. If so, is it possible that the intrinsic parameters of my L515 change slightly due to temperature or other reasons? I just want to get an accurate intrinsic paras, is the result of this function reliable enough?
2. As for the mentioned rs-enumrate-devices, is this a calibration process or what? I have found depth-auto-calibration.py in python examples, but unfortunately it only supports D400 product instead of L515. Thus, could you recommend any offical tools to do the calibration (maybe a specific link thks!).
3. Then, the hand showed in issue is captured completely inside the valid range. The offset btw the red and blue point cloud is due to the intrinsic params used to covert depth image to 3D point cloud.
4. As you mentioned, the distortion cofficients of depth camera are all set to zeros. Doest it mean that there is no need to do undistortion on IR or depth image captured by Realsense sensor and why?

Thanks for ur patience!

[MartyG-RealSense]

Sincere apologies for the delayed response, as I just found your message. I will respond to the above questions tomorrow. Thanks!

[MartyG-RealSense]

1. get_profile() retrieves the stream profile from the frame handle (a smart pointer that wraps an actual frame object). Information about the frame handle can be found at What data reside inside this array -> depth = depth_frame.get_data() #6261 (comment)

It is possible for a camera to become mis-calibrated due to excessive temperature and need to be recalibrated, though it would likely have to be a very high temperature.

2. rs-enumerate-devices is not a calibration tool, though it can retrieve the current calibration of the camera when launched in calibration information mode with the command rs-enumerate-devices -c

The L515 camera model does not have a calibration tool for depth and RGB. It uses a different camera technology (lidar depth) to the technology used in the 400 Series cameras (stereo depth). Intel's original press release for the launch of L515 stated that it retains its depth accuracy throughout its lifespan without the need for calibration.

https://newsroom.intel.com/news/intel-realsense-lidar-camera-technology-redefines-computer-vision/

The L515's IMU has a compatible calibration tool, though this would not be relevant to this particular case.

https://github.com/IntelRealSense/librealsense/tree/master/tools/rs-imu-calibration

Intrinsics and extrinsics can be retrieved with the instructions get_intrinsics() and get_extrinsics_to(). Here is some example Python code for doing so, obtained from an L515 case at #10484

depth_intrin = profile.get_stream(rs.stream.depth).as_video_stream_profile().get_intrinsics()
color_intrin = profile.get_stream(rs.stream.color).as_video_stream_profile().get_intrinsics()

depth_to_color_extrin =  profile.get_stream(rs.stream.depth).as_video_stream_profile().get_extrinsics_to( profile.get_stream(rs.stream.color))
color_to_depth_extrin =  profile.get_stream(rs.stream.color).as_video_stream_profile().get_extrinsics_to( profile.get_stream(rs.stream.depth))

4. Undistorting a RealSense image is usually unnecessary. OpenCV has an undistort function, though a RealSense user who tried it found that it made minimal difference to the image, as described at Undistorting the T265 video feed using the onboard intrinsics, wrong values? #3880

RealSense does support undistorted infrared images with its Y16 format, which is unrectified because Y16 is used for camera calibration. This format is not supported on the L515 camera model though, unfortunately, as it is a 400 Series format.

Information about how undistortion of images for particular types of distortion model is handled in the RealSense SDK is described in its Projection documentation.

https://dev.intelrealsense.com/docs/projection-in-intel-realsense-sdk-20#distortion-models

[StevenZzz07]

@MartyG-RealSense Thanks for the detailed reply. I don't have any further questions.

[MartyG-RealSense]

You are very welcome, @StevenZzz07 - thanks very much for the update. As you do not have further questions, I will close this case. Thanks again!