rs2_deproject_pixel_to_point(): range of "points" value

[dennewbie]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	D435
Firmware Version	05.13.0.50
Operating System & Version	MacOS Monterey 12.4
Platform	PC
SDK Version	2.50.0
Language	C++, OpenCV

Issue Description

Hi, I'm using void rs2_deproject_pixel_to_point (float point[3], const rs2_intrinsics * intrin, const float pixel[2], float depth ). I would like to understand the range of values just for point[0] and point[1], since point[2] corresponds to float depth. It seems that these values are between -1 and 1. That's right? Or are the bounds different? Furthermore, that's correct to assume that these values correspond to the coordinates (x, y, z) of a point in a 3D Space?
Thank you in advance,
Denny

[MartyG-RealSense]

Hi @dennewbie My understanding from the very good reference at #1413 is that [3] represents the depth intrinsic, [2] is the XY coordinates of a particular pixel - such as [100,100] - and depth is the depth unit scale of the particular RealSense camera being used.

Whilst the depth unit scale can be retrieved in real-time with an SDK programming instruction, for all RealSense 400 Series models except D405 the default depth scale value is 0.001 (on D405 it is 0.01). This scale value does not change unless done so deliberately by the program code or by a user input, so it can be easier to hard-code the number value into the script instead of retrieve it from the camera with code.

Reading #1413 from beginning to end should help a lot to understand how the rs2_deproject_pixel_to_point instruction works.

[dennewbie]

Hi @MartyG-RealSense thank you for your answer. I think that everything will be clearer attaching the following snippet.

    pixel[0] = getBodyKeyPoints().at(i).getX();
    pixel[1] = getBodyKeyPoints().at(i).getY();
    float distance = getDistance_Image().at<float>(pixel[1], pixel[0]);
    rs2_deproject_pixel_to_point(point, & color_intrin, pixel, distance);
    Point3D * point3D = new Point3D(point[0], point[1], point[2];

After this, I use cv::imshow(...) to show the image and I write each point3D coordinate with another OpenCV function. I've no problem with z values (checked with meterstick), but I've seen that x values are between -3.2 and 3.2, while y values are between -1.1 and 1.1. I just would like to understand how should I have to interpret them. The room where I take the video is 3 meters tall and around 7 meters wide.

I was thinking to a function that could make a "mapping" between RealSense x, y coordinates in order to have values in terms of meters. I found useful information here: Projection in Intel RealSense SDK 2.0 when there's the paragraph Point coordinates.

Hope to read soon from you,
Denny

[MartyG-RealSense]

The origin of the depth coordinate system is the center-line of the left infrared imager. Coordinate values are plus or minus depending on whether they are to the left / right or above / below the center-line's 0,0,0 origin, as described in the documentation link below.

https://github.com/IntelRealSense/librealsense/blob/master/doc/d435i.md#sensor-origin-and-coordinate-system

Also bear in mind that the field of view (FOV) size of the camera may not be able to see the entire height and width of the room at the same time because it will not all fit into the viewpoint of the camera.

[dennewbie]

Perfect, thank you. So based on this information I could use an approach like the following one to convert each x and y value to meters (given the room height and width): How to determine coordinates on two different size rectangles. In this way I've x, y and z values expressed in meters. That's this the correct approach?

P.S.
I forget to tell you that in some lines before the previous code snippet, I get each z value in meters by multiplying for 1000.

[MartyG-RealSense]

Deprojection converts 2D XY pixel values to 3D XYZ world point values in meters, yes.

Another way to obtain real-world distance in meters is to multiply the raw 16-bit depth value uint16_t by the depth unit scale. For example, a raw depth value of 65535 multiplied by 0.001 = 65.535 meters. More information about this can be found at #2348

[dennewbie]

Perfect, thank you very much for the effort and the patience. I'll update here if I would be able to solve this problem in this way.

[dennewbie]

Just to update the issue. I was able to convert properly the 2D XY pixel values to 3D XYZ world point values in meters (deproject). After this, since X and Y values are in a particular range, as said in a previous comment, I've "mapped" them to another 2D XY coordinate system using How to determine coordinates on two different size rectangles. The camera's coordinate system has origins in the center of the camera approximately: the X-axis goes right, the Y-axis goes down and the Z-axis goes forward. Since there are no problems with my application on the Z-axis values, I don't change them. Now my X axis starts at the minimum left point seen from the camera, while the Y axis starts from the minimum bottom point seen from the camera (e.g. imagine a cube 1x1x1 meters and put the camera at the center of one cube's inner wall. The camera is oriented versus the center of the cube. The X-axis starts at the left inner wall, the Y-axis start at the floor and Z-axis is unchanged).

[dennewbie]

@MartyG-RealSense please, let me know what do you think of this approach. In this way, I can evaluate for instance the height of a person immediately, as well as the position of a person inside a room in a further application.

[MartyG-RealSense]

Your approach seems reasonable to me in terms of how you are keeping the axis directions of the other XY coordinate system consistent with the original RealSense axes. It reminds me of a project at #9749

[dennewbie]

Perfect, thank you for the feedback and the support. Yes, the two projects have something in common.