[Meeting] 23/09

[Lotemn102]

Participants: Alon, Lotem

Projecting Vicon points into image pixels workflow

1. Annotate N 2d points from the RGB image.
2. For each point, find it's depth value. The depth frames are aligned to the RGB frames. Do i need to normalize the depth value? Alon's answer: I need to take the x,y values from the realsense api's function of get_point_cloud.
3. Use Kabsch algorithm. Returns 3x3 rotation matrix R, 3x1 translation vector t, a scalar for scale scale. Alon's answer: If the scale is not close to 1, there is a problem in the implementation!
4. Transform vicon points to realsense (3d) points: Given a matrix V of Nx3 vicon points,
   • Scale the points by V * scale.
   • Apply R * V.T + t
5. Project points into pixels: Given a matrix H of Nx3 rotated vicon points,

   • For each point h in H, apply K * h.T, where K is a 3x3 matrix:

     fx	0	ppx
     0	fy	ppy
     0	0	1

   fx, fy, ppx, ppy are taken from the realsense bag files.
   Alon's answer: I might need to try to use realsense's api implementation for the projection, since they have implemented optical distortion correction which might improve accuracy
6. The output of section 5 is 3x1 vector uvw. Scale each coordinate:
   u = u / w
v = v / w