Questions about the Oxford RobotCar Dataset

[LZL-CS]

Hi, thanks for your awesome.

About the Oxford RobotCar Dataset, I have some questions:
(1) The point cloud you use "lms_front" as the sensor, which is a 2D lidar data. Why don't choose the "ldmrs" sensor, which is a 3D lidar data?
(2) The poses you use the "interpolate_vo_poses" to interpolate the VO poses, which get the relative pose. Why you don't use the "interpolate_ins_poses" to get the absolute pose?

I appreciate any help you can provide!

[morbi25]

Hi @LZL-CS,

Thank you for your interest in our work! :)

1. We decided to use "lms_front" as the sensor because of two reasons: On the one hand, we followed RNW using the same sensor. On the other hand, similar to this issue of the robotcar_dataset_sdk repository, we faced some problems when we tried to project the "ldmrs" point cloud to the respective images.
2. As stated in the documentation of the RobotCar dataset in the section Visual Odometry (VO): "Local errors in the GPS+Inertial solution (due to loss or reacquisition of satellite signals) can lead to discontinuites in local maps built using this sensor as a pose source. For some applications a smooth local pose source that is not necessarily globally accurate is preferable." Since the "visual odometry solution is accurate over hundreds of metres (suitable for building local 3D pointclouds)", we decided to build and project the local point cloud using VO.

I hope these answers help you! Do not hesitate to reach out again if you have follow-up questions! :)

[LZL-CS]

Hi @LZL-CS,

Thank you for your interest in our work! :)

1. We decided to use "lms_front" as the sensor because of two reasons: On the one hand, we followed RNW using the same sensor. On the other hand, similar to this issue of the robotcar_dataset_sdk repository, we faced some problems when we tried to project the "ldmrs" point cloud to the respective images.
2. As stated in the documentation of the RobotCar dataset in the section Visual Odometry (VO): "Local errors in the GPS+Inertial solution (due to loss or reacquisition of satellite signals) can lead to discontinuites in local maps built using this sensor as a pose source. For some applications a smooth local pose source that is not necessarily globally accurate is preferable." Since the "visual odometry solution is accurate over hundreds of metres (suitable for building local 3D pointclouds)", we decided to build and project the local point cloud using VO.

I hope these answers help you! Do not hesitate to reach out again if you have follow-up questions! :)

Hi @morbi25, thanks for your prompt reply. I can understand it now for the questions I asked above.

But for the RobotCar dataset, I still have some questions:

(1) For the function interpolate_vo_poses, the result poses_mats is the relative pose to the first frame. In my opinion, the vo.csv is already the relative pose. Why need to get the relative pose again?

(2) For the function build_pointcloud, how to understand this code:
scan = np.dot(np.dot(poses[i], G_posesource_laser), np.vstack(['scan', np.ones((1, scan.shape[1]))]))

In my opinion, the scan is the lidar data from lms_front senosor, so G_posesource_laser @ scan is to transfer the lidar data from lms_front frame to the stereo frame (same as the vehicle frame, where stereo.txt in extrinsics.txt is [0,0,0,0,0,0]). I am confused about pose @ G_posesource_laser @ scan, here why need to pose @ at the end?

Maybe I misunderstood the calculation of these formulas and I am baffled by these coordinate system transformations. I hope to get your help, thanks in advance!

[morbi25]

Hi @LZL-CS,

I am happy to hear that the answers helped you!

Regarding your follow-up questions, to get an in-depth answer, I would recommend you to post those questions in the issue section of the robotcar-dataset-sdk repository as they are more related to the specific SDK implementation. :)

However, I can share some intuitions about the questions you have:

1. Dealing with multi-modal data (images for training and visual odometry, LiDAR data for evaluation) requires different types of sensors (stereo camera, laser scanner). As these sensors are not perfectly synchronized, they do not record samples at the same timestamp. As a consequence, the timestamps of the poses (no matter if they originate from VO, INS, or GPS) are generally not aligned with the timestamps of the recorded LiDAR point cloud; therefore, interpolation is needed to simulate the alignment. More specifically, to get the relative pose between two point cloud recordings, you can interpolate based on the available poses using the respective interpolation function of the SDK.
2. To deal with multi-modal data at different timestamps, it is also important to bring all the data in a common reference system, which we achieved via the SDK transformations.

I hope these intuitions are helpful for you. Please consider that we did not capture the RobotCar dataset ourselves, so we can only interpret the intentions of the RobotCar authors. Therefore, for a more detailed answer, I would strongly recommend to post these questions in the robotcar-dataset-sdk repository.

Have a nice week!

Cheers :)

[LZL-CS]

Hi @LZL-CS,

I am happy to hear that the answers helped you!

Regarding your follow-up questions, to get an in-depth answer, I would recommend you to post those questions in the issue section of the robotcar-dataset-sdk repository as they are more related to the specific SDK implementation. :)

However, I can share some intuitions about the questions you have:

1. Dealing with multi-modal data (images for training and visual odometry, LiDAR data for evaluation) requires different types of sensors (stereo camera, laser scanner). As these sensors are not perfectly synchronized, they do not record samples at the same timestamp. As a consequence, the timestamps of the poses (no matter if they originate from VO, INS, or GPS) are generally not aligned with the timestamps of the recorded LiDAR point cloud; therefore, interpolation is needed to simulate the alignment. More specifically, to get the relative pose between two point cloud recordings, you can interpolate based on the available poses using the respective interpolation function of the SDK.
2. To deal with multi-modal data at different timestamps, it is also important to bring all the data in a common reference system, which we achieved via the SDK transformations.

I hope these intuitions are helpful for you. Please consider that we did not capture the RobotCar dataset ourselves, so we can only interpret the intentions of the RobotCar authors. Therefore, for a more detailed answer, I would strongly recommend to post these questions in the robotcar-dataset-sdk repository.

Have a nice week!

Cheers :)

Hi @morbi25, I really appreciate your reply, I can understand it and I am now closing this issue. Have a nice Sunday!