How can I convert the .raw pixel values from the Depth Map to meters?

[amaanda]

I want to generate a .ply from the Depth Map information to compare the results I get from using the Left and Right Infrared images. I opened the .raw Depth file as unsigned 16 bits, and I am having a hard time to convert the Depth pixel values to meters (which will be the input to my function).
So basically what I need to know is the following scale:

depth_meters = depth_raw * scale;

depth_raw contains values from 0 to 65,535. Any help would be appreciated. Thanks!

[MartyG-RealSense]

Using the depth_scale instruction may be what you need.

#2348

Below is another example of use of depth_scale, where the user converts X-Y pixels to X-Y positions in meters.

#2481 (comment)

[amaanda]

I ran the instruction in the .bag file, but when I get the .raw depth (not png image), multiplying the values by the scale provided by get_depth_scale() gives me values of around 49 to 65, which are not in meters. So I am not sure if this depth_scale works for using .raw images.

[amaanda]

What I have so far:

cv::Mat depth16(720, 1280, CV_16U, (uchar *)buffer.data());
cv::Mat depthfloat;
depthfloat = depth16.clone();
depth16.convertTo(depthfloat, cv::DataType<float>::type); // values from 0 to 65535
depthfloat = depthfloat * 0.0010000000474974513; // the scale I get from the instruction

[MartyG-RealSense]

Would this script be of any help? It came from a discussion where a person was trying to get the depth data from a .raw file.

#2200 (comment)

[amaanda]

This example seems to work with .png format and not the .raw images. I am already able to get disparity from the .png left and right images. You don't think I would lose information by using the .png instead of the .raw format for the Depth frames?

[MartyG-RealSense]

I apologize, RealSense programming is not one of my specialties, so I have to rely on what I can research. One of the Intel staff on this forum may be able to give you a better answer about conversion (I do not work for Intel).

As @ev-mp says in the link below, you may indeed get better results from a .raw.

#2231 (comment)

[amaanda]

I understand. I will wait for the staff's opinion then. Thanks a lot for your help!

[ev-mp]

Hello @amaanda ,
Can you look into the following lines:

depthfloat = depth16.clone();
depth16.convertTo(depthfloat, cv::DataType<float>::type); // values from 0 to 65535

The type of depthfloat is the same as origin (UINT16). Is it possible that since the target matrix is already defined then convertTo call wouldn't resize it?
Can you try to replace the line with explicit float matrix
depthfloat = depth16.clone() with cv::Mat depthfloat(720, 1280, CV_32F) ?

You may also convert to metric units with a single call by passing the scale factor as the third param to convertTo
depth16.convertTo(depthfloat, cv::DataType<float>::type, 0.0010000000474974513); // values from 0 to 65535

[amaanda]

Hello @ev-mp
I implemented the changes you asked. I agree with what you said about using depth.clone(), thank you for the tip.
This is the ply I am currently getting:
PS.: The function I am using to generate the pointcloud already works fine for .png images, so I am sure the problem I am having is with some kind of wrong conversion of the pixels on the .raw file.

[amaanda]

This is the code that is generating the .ply above:

std::ifstream file("_Depth_37140.raw", std::ios::binary | std::ios::ate);
std::streamsize size = file.tellg();
file.clear();
file.seekg(0, std::ios::beg);
std::vector buffer(size);

for (int i = 0; i < size-1; i+=2) {
file.get(buffer[i + 1]);
file.get(buffer[i]);
}
file.close();

cv::Mat depth16(720, 1280, CV_16U, (uchar *)buffer.data());
cv::Mat depthfloat(720, 1280, CV_32F);
depth16.convertTo(depthfloat, cv::DataType::type, 0.0010000000474974513);

stereo_camera_config camera_config;
camera_config.baseline = 0.0499061;
camera_config.fx = 0.50101;

cv::Mat disparityFile = StereoMatching::depthToDisparity(depthfloat, camera_config);

[ev-mp]

@amaanda , one thing I notice is that the origin frame data is row-major:
cv::Mat depth16(720, 1280, 1280,720,CV_16U, (uchar *)buffer.data());

[amaanda]

But 720 is the number of rows, and 1280 the number of columns, isn't it?

[amaanda]

I was able to solve my problem. I used the depth sensor scale and changed the for loop.

[ghost]

Hi @amaanda ,
Can you please share the code for getting depth value in metres from .raw file .
Thanks in advance .

[Resays]

That's the problem ... people want help, but they don't help others ...
I'm trying to get depth and xyz from raw file as well ...