create color point cloud

[jooseuk]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info	create color point cloud
Camera Model	d435
Firmware Version	(Open RealSense Viewer --> Click info)
Operating System & Version	{Win (10)
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC
SDK Version	{ legacy / 2.. }
Language	python
Segment	{Robot/Smartphone/VR/AR/others }

Issue Description

<Describe your issue / question / feature request / etc..>
The following code configures the resolution for both the color and depth streams to 1280x720, and the output of the code, when you print "Depth Intrinsics: ", is as follows: "Depth Intrinsics: [ 1280x720 p[645.865 356.59] f[642.496 642.496] Brown Conrady [0 0 0 0 0] ]. You are asking if the following code, using open3d, is the correct way to obtain a color point cloud, and if there is a better method.
#color point cloud
import cv2
import numpy as np
import open3d as o3d
import os

for k in range(9):
# path
color_image_path = os.path.join(r'C:\Users\juseok\Desktop\example\data\ori',"color_"+str(k+1) + '.png')
depth_image_path = os.path.join(r'C:\Users\juseok\Desktop\example\data\ori',"depth_"+str(k+1) + '.png')

# img path
color_image = cv2.imread(color_image_path)
depth_image = cv2.imread(depth_image_path, cv2.IMREAD_UNCHANGED)

# camera intrinsic parameters
fx = 642.496 
fy = 642.496
cx = 645.865
cy = 356.59

colored_points = []

# 3d point cloud
for v in range(depth_image.shape[0]):
    for u in range(depth_image.shape[1]):
        z = depth_image[v, u] / 1000.0  
        if z > 0:  
            x = (u - cx) * z / fx
            y = (v - cy) * z / fy
            color = color_image[v, u]

            # 검은색 픽셀 확인
            if (color == np.array([0, 0, 0])).all():
                continue 

            colored_points.append([x, y, z, color[2], color[1], color[0]])  # BGR to RGB

# save as ply
with open(os.path.join(r'C:\Users\juseok\Desktop\example\data\point cloud',"output"+str(k+1) + '.ply'), 'w') as file:
    file.write('ply\n')
    file.write('format ascii 1.0\n')
    file.write(f'element vertex {len(colored_points)}\n')
    file.write('property float x\n')
    file.write('property float y\n')
    file.write('property float z\n')
    file.write('property uchar red\n')
    file.write('property uchar green\n')
    file.write('property uchar blue\n')
    file.write('end_header\n')
    for p in colored_points:
        file.write(f'{p[0]} {p[1]} {p[2]} {p[3]} {p[4]} {p[5]}\n')

# view point cloud
pcd = o3d.io.read_point_cloud(os.path.join(r'C:\Users\juseok\Desktop\example\data\point cloud',"output"+str(k+1) + '.ply'))
coordinate_frame = o3d.geometry.TriangleMesh.create_coordinate_frame(size=0.6, origin=[0, 0, 0])
o3d.visualization.draw_geometries([pcd,coordinate_frame])

[MartyG-RealSense]

Hi @jooseuk There is a Python Open3D script for real-time depth+color pointcloud generation at isl-org/Open3D#473 (comment) that you could try.

[jooseuk]

Thank you for your answer. However, I want to create a color point cloud from the color and depth images I already have. I want to align the color and depth images and create them using only the internal parameters of the depth camera. The internal parameters of the depth camera at this time are [1280x720 p[645.865 356.59] f[642.496 642.496] Brown Conrady [0 0 0 0 0]].

[jooseuk]

Hi @jooseuk There is a Python Open3D script for real-time depth+color pointcloud generation at isl-org/Open3D#473 (comment) that you could try.

Thank you for your answer. However, I want to create a color point cloud from the color and depth images I already have. I want to align the color and depth images and create them using only the internal parameters of the depth camera. The internal parameters of the depth camera at this time are [1280x720 p[645.865 356.59] f[642.496 642.496] Brown Conrady [0 0 0 0 0]].

[MartyG-RealSense]

There are not many Open3D references for using existing depth and color images rather than live ones from the camera. It sounds as though a RealSense user at #9970 (comment) was trying to do something similar in their scripting with offline images.

[MartyG-RealSense]

Hi @jooseuk Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.