Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

calibrating the camera #12305

Closed
Mamatharun opened this issue Oct 17, 2023 · 19 comments
Closed

calibrating the camera #12305

Mamatharun opened this issue Oct 17, 2023 · 19 comments

Comments

@Mamatharun
Copy link


Required Info
Camera Model D435
Firmware Version 5.15.1
Operating System & Version Linux (Ubuntu 22.04.3)
Kernel Version (Linux Only) 5.15.0-86-generic
Platform PC
SDK Version { legacy / 2.54.2 }
Language python
Segment others

Issue Description

Hello,

I need to perform the Intel real-sense D435 camera calibration to get the extrinsic parameters to get the accurate results in the computer vision applications like object detection. Could you please guide me how can I calibrate the camera?

Thank you.

@MartyG-RealSense
Copy link
Collaborator

Hi @Mamatharun The camera can be calibrated using an On-Chip Calibration tool that can be accessed from the 'More' option near the top of the options side-panel of the RealSense Viewer tool. A guide for this tool can be found at the link below.

https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras

@Mamatharun
Copy link
Author

Thank you for your response. But, I'm getting the following error when I tried on-chip calibration.

image

@MartyG-RealSense
Copy link
Collaborator

If On-Chip does not work on your particular computer then there is an alternative, just as capable calibration tool called Dynamic Calibration. The tool's user guide at the link below has instructions on page 14 onwards for installing it on Linux.

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html

@Mamatharun
Copy link
Author

Thank you very much. Can I do it programitically by using opencv-python ?

  1. Capture the checkerboard images,
  2. Identify the checkerboard corners,
  3. calibrating them to get intrinsic's and extrinsic's of both RGB and Depth sensors.

I tried in doing in this way. But, it's hard to find the checkerboard pattern in the depth images. So, if we can do it programmatically, can we do it?

@MartyG-RealSense
Copy link
Collaborator

The OpenCV chessboard calibration tutorial at the link below is a helpful reference.

https://docs.opencv.org/4.x/d4/d94/tutorial_camera_calibration.html

@Mamatharun
Copy link
Author

Thank you for the reference. I can get the Intrinsics of the RGB sensor with the opencv-method. But, not with the depth frames. As I mentioned already it is hard to detect the checkerboard corners in the depth images. So, could you please help me how can I get the intrrinsics of the Depth and extrinsic as well?

@MartyG-RealSense
Copy link
Collaborator

The librealsense SDK's pyrealsense2 Python compatibility wrapper can access the intrinsics and extrinsics with the instructions get_intrinsics and get_extrinsics_to respectively. For more information, please see #7667 (comment)

@Mamatharun
Copy link
Author

Thank you very much. I'll check that.

According to this comment " If On-Chip does not work on your particular computer then there is an alternative, just as capable calibration tool called Dynamic Calibration. The tool's user guide at the link below has instructions on page 14 onwards for installing it on Linux.

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html"

I tried installing the tool and calibrating the camera. Unfortunately, I'm getting the error mentioned in the screenshot. It is strange as I already using the USB-3.0. Could you please help me? And, also you can find the Device info another screenshot. Please help me.

image
image

@MartyG-RealSense
Copy link
Collaborator

If there is a part of your USB setup that is USB 2.1 then the camera will be detected as a USB 2.1 device. This could occur if using a USB cable of your own choice that is a USB 2 cable, or if the camera is plugged into a USB 2.1 hub.

@Mamatharun
Copy link
Author

Thank you very much for your support here. I'll try with different cable.

@MartyG-RealSense
Copy link
Collaborator

Hi @Mamatharun Do you have an update about this case that you can provide, please? Thanks!

@MartyG-RealSense
Copy link
Collaborator

Hi @Mamatharun Do you require further assistance with this case, please? Thanks!

@Tharunguvvala67
Copy link

Hello. Thank you very much for your response. I got this problem resolved yesterday. Thank you very much for your response again.

@MartyG-RealSense
Copy link
Collaborator

You are very welcome. Thanks very much for the update!

@MartyG-RealSense
Copy link
Collaborator

Case closed due to solution achieved and no further comments received.

@monajalal
Copy link

@MartyG-RealSense similar to this issue, should I also dismiss that I can't do on-chip calibration?

Screenshot from 2023-12-04 08-48-18

(base) mona@ada:~$ realsense-viewer
 04/12 08:47:34,411 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video0 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0
 04/12 08:47:34,412 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video1 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video1
 04/12 08:47:34,412 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video2 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video2
 04/12 08:47:34,412 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video3 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video3
 04/12 08:47:34,412 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video4 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video4
 04/12 08:47:34,413 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video5 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video5
 04/12 08:47:34,746 INFO [140685774360832] (synthetic-stream-gl.cpp:80) Initializing rendering, GLSL=1
 04/12 08:47:34,746 INFO [140685774360832] (synthetic-stream-gl.cpp:89)  0 GPU objects initialized
 04/12 08:47:34,746 INFO [140685774360832] (synthetic-stream-gl.cpp:132) Initializing processing, GLSL=1
 04/12 08:47:34,890 INFO [140685774360832] (synthetic-stream-gl.cpp:146)  0 GPU objects initialized
 04/12 08:47:35,015 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video0 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0
 04/12 08:47:35,016 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video1 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video1
 04/12 08:47:35,016 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video2 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video2
 04/12 08:47:35,016 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video3 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video3
 04/12 08:47:35,017 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video4 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video4
 04/12 08:47:35,017 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video5 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video5
 04/12 08:47:35,017 INFO [140685774360832] (context.cpp:336) Found 1 RealSense devices (mask 0xff)
 04/12 08:47:35,050 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video0 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0
 04/12 08:47:35,050 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video1 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video1
 04/12 08:47:35,051 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video2 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video2
 04/12 08:47:35,051 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video3 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video3
 04/12 08:47:35,051 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video4 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video4
 04/12 08:47:35,051 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video5 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video5
 04/12 08:47:35,052 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video0 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0
 04/12 08:47:35,052 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video1 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video1
 04/12 08:47:35,052 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video2 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video2
 04/12 08:47:35,052 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video3 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video3
 04/12 08:47:35,052 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video4 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video4
 04/12 08:47:35,052 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video5 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video5
 04/12 08:47:35,055 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video0 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0
 04/12 08:47:35,055 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video1 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video1
 04/12 08:47:35,055 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video2 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video2
 04/12 08:47:35,056 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video3 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video3
 04/12 08:47:35,056 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video4 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video4
 04/12 08:47:35,056 INFO [140685606807104] (backend-v4l2.cpp:657) Enumerating UVC video5 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video5
 04/12 08:47:35,094 INFO [140685774360832] (rs.cpp:2697) Framebuffer size changed to 1344 x 1003
 04/12 08:47:35,096 INFO [140685774360832] (rs.cpp:2697) Window size changed to 1344 x 1003
 04/12 08:47:35,096 INFO [140685774360832] (rs.cpp:2697) Scale Factor is now 1
 04/12 08:47:35,241 WARNING [140685606807104] (device-model.cpp:437) Exception caught trying to detect presets: could not open directory: /home/mona/Documents/librealsense2/presets/
 04/12 08:47:35,423 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video0 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0
 04/12 08:47:35,424 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video1 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video1
 04/12 08:47:35,424 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video2 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video2
 04/12 08:47:35,424 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video3 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video3
 04/12 08:47:35,424 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video4 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video4
 04/12 08:47:35,424 INFO [140685774360832] (backend-v4l2.cpp:657) Enumerating UVC video5 realpath=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.3/video4linux/video5
 04/12 08:47:35,424 INFO [140685774360832] (context.cpp:336) Found 1 RealSense devices (mask 0xfe)
 04/12 08:47:39,505 INFO [140685774360832] (sensor.cpp:1594) Request: Z16 Depth, 
Resolved to: Z16 Depth, 
 04/12 08:47:39,506 INFO [140685774360832] (backend-v4l2.cpp:2067) Video node was successfully configured to Z16  format, fd 25
 04/12 08:47:39,506 INFO [140685774360832] (backend-v4l2.cpp:2069) Trying to configure fourcc Z16 
 04/12 08:47:39,506 INFO [140685774360832] (backend-v4l2.cpp:2270) Metadata node was successfully configured to D4XX format, fd 33
 04/12 08:47:40,284 ERROR [140685445416512] (http-downloader.cpp:143) Download error from URL: https://librealsense.intel.com/Releases/rs_versions_db.json, error info: Timeout was reached
 04/12 08:47:42,719 INFO [140685774360832] (sensor.cpp:1594) Request: RGB8 Color, 
Resolved to: YUYV Color, 
 04/12 08:47:42,720 INFO [140685774360832] (backend-v4l2.cpp:2067) Video node was successfully configured to YUYV format, fd 26
 04/12 08:47:42,720 INFO [140685774360832] (backend-v4l2.cpp:2069) Trying to configure fourcc YUYV
 04/12 08:47:42,720 INFO [140685774360832] (backend-v4l2.cpp:2270) Metadata node was successfully configured to D4XX format, fd 29
 04/12 08:47:43,272 INFO [140685445416512] (metadata-parser.h:355) Frame counter reset
 04/12 08:47:43,272 INFO [140685445416512] (metadata-parser.h:355) Frame counter reset
 04/12 08:47:43,272 INFO [140685445416512] (metadata-parser.h:355) Frame counter reset
 04/12 08:47:43,337 INFO [140685445416512] (metadata-parser.h:355) Frame counter reset
 04/12 08:47:43,400 INFO [140685428631104] (metadata-parser.h:355) Frame counter reset
 04/12 08:48:09,817 INFO [140685428631104] (backend-v4l2.cpp:1302) V4L stream is closed
 04/12 08:48:09,818 INFO [140685774360832] (frame-archive.h:191) The user was holding on to 2 frames after stream 0x0x55fd59414310 stopped
 04/12 08:48:10,434 INFO [140685774360832] (sensor.cpp:1594) Request: Z16 Depth, 
Resolved to: Z16 Depth, 
 04/12 08:48:10,434 INFO [140685774360832] (backend-v4l2.cpp:2067) Video node was successfully configured to Z16  format, fd 25
 04/12 08:48:10,434 INFO [140685774360832] (backend-v4l2.cpp:2069) Trying to configure fourcc Z16 
 04/12 08:48:10,434 INFO [140685774360832] (backend-v4l2.cpp:2270) Metadata node was successfully configured to D4XX format, fd 33
 04/12 08:48:10,593 INFO [140685428631104] (backend-v4l2.cpp:1302) V4L stream is closed
 04/12 08:48:10,594 INFO [140685774360832] (frame-archive.h:191) The user was holding on to 7 frames after stream 0x0x55fd5942a130 stopped
 04/12 08:48:10,616 INFO [140685774360832] (sensor.cpp:1594) Request: Z16 Depth, 
Resolved to: Z16 Depth, 
 04/12 08:48:10,617 INFO [140685774360832] (backend-v4l2.cpp:2067) Video node was successfully configured to Z16  format, fd 25
 04/12 08:48:10,617 INFO [140685774360832] (backend-v4l2.cpp:2069) Trying to configure fourcc Z16 
 04/12 08:48:10,617 INFO [140685774360832] (backend-v4l2.cpp:2270) Metadata node was successfully configured to D4XX format, fd 33
 04/12 08:48:10,623 INFO [140685076330048] (environment.cpp:130) Found 2 unreachable streams, 284 extrinsics deleted
 04/12 08:48:20,638 INFO [140685428631104] (backend-v4l2.cpp:1302) V4L stream is closed
 04/12 08:48:20,639 INFO [140685774360832] (frame-archive.h:191) The user was holding on to 4 frames after stream 0x0x55fd59e385b0 stopped
 04/12 08:48:20,840 INFO [140685774360832] (sensor.cpp:1594) Request: Z16 Depth, 
Resolved to: Z16 Depth, 
 04/12 08:48:20,841 INFO [140685774360832] (backend-v4l2.cpp:2067) Video node was successfully configured to Z16  format, fd 25
 04/12 08:48:20,841 INFO [140685774360832] (backend-v4l2.cpp:2069) Trying to configure fourcc Z16 
 04/12 08:48:20,841 INFO [140685774360832] (backend-v4l2.cpp:2270) Metadata node was successfully configured to D4XX format, fd 33
 04/12 08:48:21,439 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,497 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,564 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,631 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,698 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,764 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,831 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,898 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:21,964 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,031 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,098 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,164 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,231 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,298 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,365 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,431 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,498 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,565 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,632 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,698 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,765 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,832 ERROR [140685445416512] (synthetic-stream.cpp:51) Exception was thrown during callback: Out of frame resources!
 04/12 08:48:22,894 INFO [140685774360832] (environment.cpp:130) Found 2 unreachable streams, 284 extrinsics deleted

@MartyG-RealSense
Copy link
Collaborator

@monajalal It looks as though you are using Windows. Therefore, does on-chip calibration work if you download and run the older 2.49.0 version of the RealSense Viewer by selecting Intel.RealSense.Viewer.exe from the 'Assets' file list at the link below?

https://github.com/IntelRealSense/librealsense/releases/tag/v2.49.0

image

@monajalal
Copy link

@MartyG-RealSense I am actually on Ubuntu 22.04
I only have this problem if I use realsense-viewer GUI
I don't have a problem with ros noetic capture and now I am confused if I mainly use ros noetic ros realsense for my captures, if I should worry about this?

@MartyG-RealSense
Copy link
Collaborator

@monajalal Thanks very much for the clarification about Ubuntu.

As you are able to generate a 3D pointcloud image in the Viewer, this suggests that your Ubuntu computer's video drivers are okay.

If you are unable to install librealsense version 2.49.0 to test on-chip calibration in version 2.49.0 of the Viewer then you could try resetting your camera's calibration to its factory-new default values using the instructions at #10182 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants