Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shift principle points by .5 in undistortion #3071

Merged
merged 3 commits into from
Apr 12, 2024
Merged

shift principle points by .5 in undistortion #3071

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
907cd94
shift principle points by .5
kerrj Apr 11, 2024
f2e7611
fix ordering
kerrj Apr 11, 2024
34f203a
add fisheye offset
kerrj Apr 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/quickstart/data_conventions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ We use the OpenGL/Blender (and original NeRF) coordinate convention for cameras.

Our world space is oriented such that the up vector is +Z. The XY plane is parallel to the ground plane. In the viewer, you'll notice that red, green, and blue vectors correspond to X, Y, and Z respectively.

### Pixel coordinates

We assume coordinates correspond to the centers of pixels (e.g. generating a ray for pixel (0,0) will shoot the ray through the center of that pixel). This aligns more with graphics conventions, and is distinct from OpenCV where the corners are aligned with the pixel coordinate.

<hr>

## Dataset format
Expand Down
10 changes: 10 additions & 0 deletions nerfstudio/data/datamanagers/full_images_datamanager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,10 @@ def _undistort_image(
0,
]
)
# because OpenCV expects the pixel coord to be top-left, we need to shift the principal point by 0.5
# see https://github.com/nerfstudio-project/nerfstudio/issues/3048
K[0, 2] = K[0, 2] - 0.5
K[1, 2] = K[1, 2] - 0.5
if np.any(distortion_params):
newK, roi = cv2.getOptimalNewCameraMatrix(K, distortion_params, (image.shape[1], image.shape[0]), 0)
image = cv2.undistort(image, K, distortion_params, None, newK) # type: ignore
Expand All @@ -367,9 +371,13 @@ def _undistort_image(
mask = torch.from_numpy(mask).bool()
if len(mask.shape) == 2:
mask = mask[:, :, None]
newK[0, 2] = newK[0, 2] + 0.5
newK[1, 2] = newK[1, 2] + 0.5
K = newK

elif camera.camera_type.item() == CameraType.FISHEYE.value:
K[0, 2] = K[0, 2] - 0.5
K[1, 2] = K[1, 2] - 0.5
distortion_params = np.array(
[distortion_params[0], distortion_params[1], distortion_params[2], distortion_params[3]]
)
Expand All @@ -388,6 +396,8 @@ def _undistort_image(
mask = torch.from_numpy(mask).bool()
if len(mask.shape) == 2:
mask = mask[:, :, None]
newK[0, 2] = newK[0, 2] + 0.5
newK[1, 2] = newK[1, 2] + 0.5
K = newK
elif camera.camera_type.item() == CameraType.FISHEYE624.value:
fisheye624_params = torch.cat(
Expand Down
Loading