Are there every any non zero values for rs2_intrinsics::coeffs in the d415 and d435 cameras ?

[GrubbyHalo]

It seems like these two models have a value of zero for all the coeffs. What cameras or distortion models populate this array of coefficients ? I have only used the d435 and d415 cameras and they all seem to have this set to zero. This makes the rs2_deproject_pixel_to_point function quite inefficient in this particular case as this:

static inline void rs_deproject_pixel_to_point(float point[3], const struct rs2_intrinsics* intrin, const float pixel[2], float depth)
{
    float x = (pixel[0] - intrin->ppx) / intrin->fx;
    float y = (pixel[1] - intrin->ppy) / intrin->fy;

    float xo = x;
    float yo = y;
    .
    .
    .

    if (intrin->model == RS2_DISTORTION_BROWN_CONRADY)
    {
        // need to loop until convergence
        // 10 iterations determined empirically
        for (int i = 0; i < 10; i++)
        {
            float r2 = x * x + y * y;
            float icdist = (float)1 / (float)(1 + ((intrin->coeffs[4] * r2 + intrin->coeffs[1]) * r2 + intrin->coeffs[0]) * r2);
            float delta_x = 2 * intrin->coeffs[2] * x * y + intrin->coeffs[3] * (r2 + 2 * x * x);
            float delta_y = 2 * intrin->coeffs[3] * x * y + intrin->coeffs[2] * (r2 + 2 * y * y);
            x = (xo - delta_x) * icdist;
            y = (yo - delta_y) * icdist;
        }

    }
    .
    .
    .

    point[0] = depth * x;
    point[1] = depth * y;
    point[2] = depth;
}

That looping for 10 iterations on each pixel kills performance on largish images.

essentially resolves to this:

static inline void rs_deproject_pixel_to_point(float point[3], const struct rs2_intrinsics* intrin, const float pixel[2], float depth)
{
    float x = (pixel[0] - intrin->ppx) / intrin->fx;
    float y = (pixel[1] - intrin->ppy) / intrin->fy;

    point[0] = depth * x;
    point[1] = depth * y;
    point[2] = depth;
}

Unless there is something I am missing. Maybe a flag can be set within the rs2_intrinsics struct giving a hint that coefficients aren't being used and allow for a performance increase in rs_deproject_pixel_to_point

[MartyG-RealSense]

Hi @GrubbyHalo Most of the coefficients are deliberately set to zero by intentional design, as described by a RealSense team member in #1430 (comment)

An exception to this rule is the D455 camera model, which has non-zero coefficients (though very close to zero) on its color profiles.

Most of the 400 Series camera streams are rectified at the point of capture in the camera hardware via its Vision Processor D4 circuitry. An exception to this is the Y16 infrared format, which is unrectified as it is used for camera calibration.

OpenCV has undistort capabilities if you wish to investigate that possibility, as the SDK does not have an undistortion feature.

https://docs.opencv.org/4.5.2/dc/dbb/tutorial_py_calibration.html

A RealSense user at #3880 tried an OpenCV undistort of a RealSense image but found that it made little difference though.

---

If you are generating a point cloud and retrieving its xyz values then using points.get_vertices() may be more accurate than deproject_pixel_to_point, as discussed in detail at #4315

You can also obtain the xyz of a specific pixel instead of deprojecting the entire image (resulting in faster processing) by using the SDK instruction rs2_project_color_pixel_to_depth_pixel

#5603 (comment)

[GrubbyHalo]

Thanks for that @MartyG-RealSense