The z-axis of pointcloud2 in the camera_link frame when displayed in rviz

[TongZhangTHU]

Hello,

Why is the z-axis of pointcloud2 (from /camera/depth/color/points topic) in the camera_link frame pointing upwards instead of towards the camera's front（which I think it should be) when displayed in rviz？Like this example https://user-images.githubusercontent.com/17433152/35396613-ddcb1d6c-01f5-11e8-8887-4debf178d0cc.gif

[MartyG-RealSense]

Hi @TongZhangTHU The ROS standard reference frame is for Z to be up, whilst on RealSense cameras Z is forward. The RealSense ROS wrapper converts RealSense's coordinate system to the ROS standard. The Intel RealSense team member who created the ROS1 wrapper provides more information about this at #1099 (comment)

[TongZhangTHU]

@MartyG-RealSense Thanks for your reply. But I still don't know whether the camera_link frame should be (X- right, Y- down, Z- forward) or (X- forward, Y- left, Z- up)?

If the camera_link frame is RealSense's coordinate system, why when I use camera_link frame as fixed frame in rviz, it still shows Z to be up?

[MartyG-RealSense]

The coordinate system of the RealSense SDK (librealsense) which the RealSense ROS compatibility wrapper runs on top of is not in the ROS standard format and so the wrapper translates the SDK's data into a ROS-compatible format. So you do not need to think about the camera's Z-depth being on the forward axis by default, as the ROS wrapper automatically makes the adjustments required for the data to be compatible with ROS standards.

[TongZhangTHU]

@MartyG-RealSense

The reason I ask this question is that after I calibrated the RealSense camera extrinsics relative to the robot, I visualized the point cloud in the robot frame with rviz /camera/depth/color/points topic (I have published the extrinsics TF), but found that the point cloud is in the wrong position and looks rotated in some way, I suspect that it is caused by the wrong coordinate system. I wonder if you have any suggestions for my problem?

[TongZhangTHU]

The image of the problem is as follows

[MartyG-RealSense]

My understanding is that camera_link should correspond to the center of the camera's left infrared sensor (which is the origin point of depth) and base_link correspond to the tripod screw-thread hole on the bottom of the camera where a camera can be mounted onto a robot arm's end-effector, as discussed at #2700

[TongZhangTHU]

@MartyG-RealSense Then can I visualize pointcloud2 (from /camera/depth/color/points topic) in the camera_link frame and keep Z forward (instead of up)?

[MartyG-RealSense]

If you have launched with the pointcloud filter enabled and Fixed Frame is set to camera_link in Rviz - like in the example in the official ROS1 wrapper documentation at the link below - then the pointcloud should be published to /camera/depth/color/points and displayed correctly in Rviz.

https://github.com/IntelRealSense/realsense-ros/tree/ros1-legacy#point-cloud

[TongZhangTHU]

@MartyG-RealSense I find I can transform the PointCloud from (x, y, z) to (-y,-z, x) while not changing the frame so that it can be visualized as I expected.

My implementation is to subscribe /camera/depth/color/points topic and then do the transformation for each point and then publish a transformed topic, but I found this runs slowly.

Is there a better way to achieve the above requirements? Do you have any idea how to transform it efficiently?

Thank you for your help!

[MartyG-RealSense]

#2190 makes the suggestions of changing the axes by either (1) using a static transform or (2) by changing the camera position / orientation in RViz and saving the configuration (see the link below)

https://answers.ros.org/question/339109/how-to-change-the-cameras-view-in-rviz-in-the-begin/

[MartyG-RealSense]

Hi @TongZhangTHU Do you require further assistance with this case, please? Thanks!

[TongZhangTHU]

@MartyG-RealSense I have found a way to solve the problems. Thank you for your kind help!

[MartyG-RealSense]

You are very welcome, @TongZhangTHU - I am pleased to hear that you achieved a solution. Thanks very much for the update!