How to get the corresponging image coordinates (rows and columns) of given 3D points captured by RealSense senor?

[JimXu1989]

---

Required Info
Camera Model	{ D400 }
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	Win (10)
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version	{ legacy / 2.<30>.<?> }
Language	{python }
Segment	{others }

Issue Description

<Describe your issue / question / feature request / etc..>
I have capture some 3D points by my D435i, and I have clustered the 3D points by Open3D.
Now I want to know the corresponding positon of each 3D points cluster in the color image, that will help me to know which 3D points cluster is needed, and which is noise.

Thanks a lot!!!

[MartyG-RealSense]

I hope that the discussion in the link below, which looks at Open3D, will have something useful for you.

#2769 (comment)

I recommend going back to the start of the discussion after reading the final comment in the link above.

[jb455]

Given a 3D point, you can find a corresponding 2D pixel using rs2_project_point_to_pixel. Assuming your 3D points originated from a colour-aligned depth image, you'd use the colour intrinsics to get coordinates in the colour image space.
If the function is not available in the python wrapper, it should be easy to translate, it's just arithmetic really. Note if you're using rectified depth/colour images (where distortion is zero) you can ignore #23-60 entirely.

[JimXu1989]

ote if you're using rectified depth/colour images (where distortion is zero) you can ignore #23-60 entirely.

Hi, Thanks for your reply!
I want to know how to get the parameter "rs2_intrinsics "?
Thanks a lot!

[JimXu1989]

Given a 3D point, you can find a corresponding 2D pixel using rs2_project_point_to_pixel. Assuming your 3D points originated from a colour-aligned depth image, you'd use the colour intrinsics to get coordinates in the colour image space.
If the function is not available in the python wrapper, it should be easy to translate, it's just arithmetic really. Note if you're using rectified depth/colour images (where distortion is zero) you can ignore #23-60 entirely.

Hello, I want to know, is the original point of the 3D points same with that of the color camera coordinate？

[jb455]

Not entirely sure what you mean, are you asking if you'll get back to the exact same pixel if you project from a 3d point which was originally associated with a certain pixel? I don't know for sure, but I think this wouldn't necessarily be guaranteed due to rounding errors etc. It should be close though.

[JimXu1989]

Not entirely sure what you mean, are you asking if you'll get back to the exact same pixel if you project from a 3d point which was originally associated with a certain pixel? I don't know for sure, but I think this wouldn't necessarily be guaranteed due to rounding errors etc. It should be close though.

Hello, I want to konw, which intrinsics should I use? the intrinsics of depth frame or color frame?

[jb455]

You should use the intrinsics that your point cloud is aligned to. So if your point cloud is calculated from a normal depth image you should use the depth intrinsics, and if the depth image was aligned to colour you'd use colour intrinsics.

[ev-mp]

@JimXu1989 hello, do you need further assistance on this topic ?
Please advise