rs2_deproject_pixel_to_point Y doesn't match

[FeiSong123]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	L515
Firmware Version	01.05.08.01
Operating System & Version	Ubuntu 20.04
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC
SDK Version	2.53.1
Language	C++
Segment	Robot

Issue Description

<Describe your issue / question / feature request / etc..>

I used depth to color alignment. I'm using rs2_deproject_pixel_to_point to obtains 3d value, it is found that the return value z is consistent with the viewer, but the y value is inconsistent.

rs2_deproject_pixel_to_point(Point, &color_intrin, colorPixel, depth);//2d to 3d

[MartyG-RealSense]

Hi @FeiSong123 When using rs2_deproject_pixel_to_point after having first performed depth to color alignment, there may be a small amount of inaccuracy that is caused by the alignment.

Instead of using alignment and rs2_deproject_pixel_to_point, more accurate results can be achieved by generating a pointcloud using pc.calculate and then pc.map_to (to map depth and color together instead of using alignment). The XYZ coordinates can then be obtained from the vertices with the instruction points.get_vertices().

#4315 is a good reference source for comparing the two different approaches.

My understanding is that the pointcloud in the RealSense Viewer is based on pc.calculate and pc.map_to instead of depth-color alignment with align_to. The RealSense SDK's rs-pointcloud C++ example program also demonstrates use of these instructions.

https://github.com/IntelRealSense/librealsense/tree/master/examples/pointcloud

[FeiSong123]

I want to obtain the 3d data of a pixel on the color image, how can I do if I get if from points.get_vertices().

[MartyG-RealSense]

You can use the rs2_project_color_pixel_to_depth_pixel instruction to get the XYZ of a single specific color pixel by converting it to a depth pixel. This method should be accurate as it does not use alignment and so will not be vulnerable to the inaccuracy issue that rs2_deproject_pixel_to_point has. #2982 has a C++ script example of using this instruction - scroll down to the section of the script commented recreate complete cloud from image

[MartyG-RealSense]

Hi @FeiSong123 Do you require further assistance with this case, please? Thanks!

[FeiSong123]

Unfortunately, the accuracy effect after using the link has not improved, but has worsened.

[MartyG-RealSense]

As you asked in #11584 (comment) about retrieving 3D data from points.get_vertices(), a more suitable method for you may be to retrieve x, y and z separately from the stored vertices.

Scripts for 3 different approaches for finding the 3D points are provided at #11031 (comment) and the third script - Method 3 – align depth and use point cloud - generates a pointcloud with pc.calculate, gets the vertices with points.get_vertices and then finds x, y and z from the vertices.

[FeiSong123]

I'm sorry, I didn't make it clear. I mean, after trying the methods in the link #2982 I found that the effect did not improve but worsened

[MartyG-RealSense]

You were clear and I understood. :)

As you said that using rs2.project.color_pixel_to_depth_pixel in #2982 produced worse results, and you wanted to retrieve 3D data from points.get_vertices(), I linked to a script in #11031 (comment) for doing that.

[FeiSong123]

I am using L515 to calculate the difference between the heights of two step surfaces.When using method 1 (only use rs2_deproject_pixel_to_point) to obtain 3D data, the calculation result is 0.0798267;When using method 3, the calculation result is 0.0845289.The actual value should be 0.100. The unit is meters.

[MartyG-RealSense]

If the surface of the steps is smooth, reflective and low-texture then this may negatively affect the depth measurement results. Sensing from above and at a diagonal angle can produce better results with such surfaces.

[FeiSong123]

The surfaces of the two steps are made of relatively rough wood, which I think should be enough to bother the designer.

[MartyG-RealSense]

You could try changing the L515's Visual Preset camera configuration in your C++ code to see whether it improves accuracy, as described at #9071 (comment)

The L515's default preset is Maximum Range, though the preset Short Range often has better results.

[FeiSong123]

I am processing a bag file, and when recording a bag video, it has been preset to the Short Range.

[MartyG-RealSense]

What are the lighting conditions in the location that the bag was recorded in? If there was no natural light, only artificial lights, then the preset No Ambient Light would be appropriate, whilst Low Ambient Light is suited to locations that have a small amount of natural light coming through windows.

Are you able to re-record the bag with a different preset or are you only able to use the bag that has already been recorded?

[FeiSong123]

I tried no ambient light mode and low ambient light mode, and the difference in the height of the two ladders became 0.090, which seemed slightly better.

[MartyG-RealSense]

You could seek further improvement of accuracy using the L515's RS2_OPTION_CONFIDENCE_THRESHOLD instruction, which sets the confidence threshold to use to mark a depth pixel as valid by the depth algorithm. The supported value range is 0-3 and the default value is 1.0. #6775 (comment) has C++ code for the instruction.

[FeiSong123]

RS2_OPTION_CONFIDENCE_THRESHOLD The setting of this option should only be able to set the threshold value for black dots, but not improve accuracy. After I set it to 3.0, the calculated result is approximately 0.090. This accuracy should be the ultimate accuracy of this camera, right? Due to the limitations of hardware equipment, further improvement should not be possible?

[FeiSong123]

Another reason for this inaccuracy should be that the value mapped when mapping a color to a point cloud is incorrect. I mean, when I want to obtain the 3D value of a pixel in the color image, the value returned by rs2_deproject_pixel_to_point to me is not the 3D value corresponding to the pixel point, because the value I view in the viewer does not match the value printed by the program.

[MartyG-RealSense]

There are a range of factors that can affect a camera's accuracy, such as lighting, environmental conditions, reflectiveness of a surface, the material that a surface is made from, the distance of the observed surface from the camera, etc. So accuracy can be improved if you can deal with problems caused by these factors. Complete 1:1 accuracy with the real-world measured distance is unlikely to be achieved though and will always have a result that is not exactly 1 meter.

The pointcloud in the Viewer is using the pc.calculate method of pointcloud generation, so it is not directly comparable to the result returned by rs2_deproject_pixel_to_point

[MartyG-RealSense]

Hi @FeiSong123 Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.

[5204338]

Another reason for this inaccuracy should be that the value mapped when mapping a color to a point cloud is incorrect. I mean, when I want to obtain the 3D value of a pixel in the color image, the value returned by rs2_deproject_pixel_to_point to me is not the 3D value corresponding to the pixel point, because the value I view in the viewer does not match the value printed by the program.

dear sir, you get 0.100 of y coordinate finally, i meet same problem, can i chat with you, can you help me?

[FeiSong123]

Another reason for this inaccuracy should be that the value mapped when mapping a color to a point cloud is incorrect. I mean, when I want to obtain the 3D value of a pixel in the color image, the value returned by rs2_deproject_pixel_to_point to me is not the 3D value corresponding to the pixel point, because the value I view in the viewer does not match the value printed by the program.

dear sir, you get 0.100 of y coordinate finally, i meet same problem, can i chat with you, can you help me?

#11584 (comment)

I didn't read the value as 0.1, but calculated the difference between the two step surfaces. The final result was 0.090, and the y height value of the step surface differed by 0.1.