D435 - Calibration table update failed

[ManuelS-BSH]

Required Info
Camera Model	D435
Firmware Version	05.12.01.00
Operating System & Version	Ubuntu 18.04.3 LTS
Kernel Version (Linux Only)	5.0.0-37-generic
Platform	Notebook
SDK Version	2.31.0
Language	C++ / ROS
Segment	Robot

Issue Description

The Dynamic Calibrator fails to update the calibration table.
I'm using the the cable that is provided with the camera and the Android Dynamic Target Tool.

$ /usr/bin/Intel.Realsense.DynamicCalibrator -v
2.6.8.0

$ rs-enumerate-devices
Device info: 
    Name                          : 	Intel RealSense D435
    Serial Number                 : 	[SN present]
    Firmware Version              : 	05.12.01.00
    Recommended Firmware Version  : 	05.12.01.00
    Physical Port                 : 	/sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0
    Debug Op Code                 : 	15
    Advanced Mode                 : 	YES
    Product Id                    : 	0B07
    Camera Locked                 : 	YES
    Usb Type Descriptor           : 	3.2
    Product Line                  : 	D400
    Asic Serial Number            : 	[present]
    Firmware Update Id            : 	[present]

$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 5000M
    |__ Port 2: Dev 3, If 0, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 3, If 1, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 3, If 2, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 3, If 3, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 3, If 4, Class=Video, Driver=uvcvideo, 5000M

$ sudo /usr/bin/Intel.Realsense.DynamicCalibrator -cli

Serial Number: [same as above]
Device Name: Intel RealSense D435
Device PID:  0B07
FW version:  05.12.01.00

Calibration Type: Targeted
Laser Power: On
Auto Exposure: On
AE SetPoint: Auto
Resolution: 1280x720x30
Start rectification
Start scale calibration

Computed rotational correction angles in degrees: <rx, ry, rz> = <0.001, -0.001, 0.004>
Scale calibration completed, start RGB calibration

Computed rotational correction angles in degrees: <rx, ry, rz> = <-0.022, 0.029, 0.068>  translation in mm: <tx, ty, tz> = <0.000, 0.017, -0.007>
Calibration table update failed

It shows the correct images and tracks the target like it's shown in the GUI demo.
First, I've tried to calibrate using the GUI, there it always fails after the scale calibration with a warning "Failed to update the device ... possibly loose usb connection" or something alike.

I installed the SDK manually, following the instruction manual for Ubuntu 18.04 LTS without errors.

[MartyG-RealSense]

The CustomRW tool that is part of the Dynamic Calibrator has a "gold reset" mechanism to reset the camera to factory default state. If the calibration table stored in the camera is corrupted then the gold reset can correct it.

In the CustomRW inteface, a gold reset can be initiated with the following command:

Intel.Realsense.CustomRW.exe -g

[ManuelS-BSH]

$ sudo /usr/bin/Intel.Realsense.CustomRW -g
CustomRW for Intel RealSense D400, Version: 2.6.8.0

  Device PID: 0B07
  Device name: Intel RealSense D435
  Serial number: [same as above]
  Firmware version: 05.12.01.00

Calibration failed to restore to default.

It makes no difference if sudo or not.

[MartyG-RealSense]

The 'physical port' information in your original message lists the port as USB2. The Dynamic Calibrator software was written for USB 3.1, as it was created before Intel introduced USB2 support for the 400 Series.

[ManuelS-BSH]

I don't why is that, it's a USB 3.2 port and the RealSense viewer also detects it like that:

[MartyG-RealSense]

The rs-enumerate information is a bit contradictory. It lists usb2 for the physical port, but 3,2 for the 'USB type descriptor' further down the log. The RealSense Viewer is clearly detecting it as a 3.2 connection.

Physical Port /sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0
Usb Type Descriptor 3.2

@dorodnic What do you think?

[ManuelS-BSH]

The rs-enumerate information is a bit contradictory. It lists usb2 for the physical port, but 3,2 for the 'USB type descriptor' further down the log. The RealSense Viewer is clearly detecting it as a 3.2 connection.

Physical Port /sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0
Usb Type Descriptor 3.2

Even though I'm not dorodnic, I just did a bit of research and in the end came up with the assumption, that it should be an USB 3.2 Gen 1, respectively SuperSpeed USB, as lsusb -t tells us, that the Port offers a speed value of 5000M, being 5Gbit/s, which is the speed of USB 3.2 Gen 1.

USB 2 on the other hand only can offer a max speed of 480Mbit/s, so it just can't be USB 2 simply because of the data transfer speed.

[MartyG-RealSense]

Thanks so much for the extra detail that you have provided for the Intel RealSense support staff (I am not a member of that team).

[ev-mp]

@ManuelS-BSH, the path name

/sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0

is not indicative of the USB type of the camera. It represents the logical path of the device node.

To check the device's USB connection mode you can use lsusb -v -d 8086: | grep bcdUSB ,check the dmesg log or run rs-enumerate-devices -s

[ManuelS-BSH]

Here the proof for USB 3.2:

$ lsusb -v -d 8086: | grep bcdUSB
  bcdUSB               3.20
FIXME: alloc bigger buffer for device capability descriptors

$ rs-enumerate-devices -s
Device Name                   Serial Number       Firmware Version    
Intel RealSense D435          815412070790        05.12.01.00         
Device info: 
    Name                          : 	Intel RealSense D435
    Serial Number                 : 	[SN provided]
    Firmware Version              : 	05.12.01.00
    Recommended Firmware Version  : 	05.12.01.00
    Physical Port                 : 	/sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video2
    Debug Op Code                 : 	15
    Advanced Mode                 : 	YES
    Product Id                    : 	0B07
    Camera Locked                 : 	YES
    Usb Type Descriptor           : 	3.2
    Product Line                  : 	D400
    Asic Serial Number            : 	812113022307
    Firmware Update Id            : 	812113022307

And here's the log for the connection

$ dmesg
[...]
[  160.763561] usb 2-2: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[  160.784701] usb 2-2: New USB device found, idVendor=8086, idProduct=0b07, bcdDevice=50.c1
[  160.784706] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  160.784709] usb 2-2: Product: Intel(R) RealSense(TM) Depth Camera 435 
[  160.784712] usb 2-2: Manufacturer: Intel(R) RealSense(TM) Depth Camera 435 
[  160.784714] usb 2-2: SerialNumber: [provided]
[  160.788378] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[  160.791753] uvcvideo: Unable to create debugfs 2-2 directory.
[  160.792024] uvcvideo 2-2:1.0: Entity type for entity Intel(R) RealSense(TM) Depth Ca was not initialized!
[  160.792028] uvcvideo 2-2:1.0: Entity type for entity Processing 2 was not initialized!
[  160.792031] uvcvideo 2-2:1.0: Entity type for entity Camera 1 was not initialized!
[  160.792244] input: Intel(R) RealSense(TM) Depth Ca as /devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/input/input18
[  160.793115] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[  160.795221] uvcvideo: Unable to create debugfs 2-2 directory.
[  160.795522] uvcvideo 2-2:1.3: Entity type for entity Processing 7 was not initialized!
[  160.795528] uvcvideo 2-2:1.3: Entity type for entity Extension 8 was not initialized!
[  160.795534] uvcvideo 2-2:1.3: Entity type for entity Camera 6 was not initialized!
[...]

[RealSenseCustomerSupport]

---

@ManuelS-BSH Sorry for late response. Did you still have the issue? Did you have such issue on certain camera or also some other cameras also? Looking forward to your update. Thanks!

[RealSenseCustomerSupport]

---

@ManuelS-BSH Could you please update? Thanks!

[RealSenseCustomerSupport]

---

@ManuelS-BSH I can reproduce the issue with Ubuntu18.04 with kernel 5.3 also. We'll look into this. Thanks!

[RealSenseCustomerSupport]

---

@ManuelS-BSH We now found the root cause and have one engineering fix for this. Does your company have any NDA signed with Intel? If so, we could share the engineering drop to you by our NDA customer support platform. Or else I'm afraid you need wait for some days for the official release.

[RealSenseCustomerSupport]

---

@ManuelS-BSH Could you please update? Looking forward to your update. Thanks!

[martinakos]

I have the same exact problem. Is there any update on this?

[RealSenseCustomerSupport]

---

@ManuelS-BSH @martinakos Are you NDA customer? We have one engineering drop which fixes the issue but can only release to NDA customer now. The official release plan is still under checking.

[marc-meijer]

I have the same issue as well. Would a workaround be to perform a hardware reset from realsense-viewer? Or doesn't that clear the the calibration parameters?

[zhanghx04]

Hi, I have the same issue while calibrating my realsense camera. I am doing from a USB 3.2 port as well and received the same error message. I am working in a University research lab. So is the solution only available to NDA customers?

[MartyG-RealSense]

@marc-meijer A hardware reset from the RealSense Viewer has the equivalent effect to resetting the camera by unplugging it from the USB port and then plugging it back in.

It is not the same as the Gold Reset process mentioned earlier, which resets the camera to factory defaults and can also (usually) correct a corruption of the calibration table stored inside the camera. The known problem mentioned by @RealSenseCustomerSupport might be interfering with the gold reset to default factory settings now though.

If you are not an NDA customer then you may need to wait for an official release of a fix, as @RealSenseCustomerSupport says.

@zhanghx04 If you have access to the RealSense Viewer program, there is now an alternative to the Dynamic Calibrator called On-Chip Calibration. The calibration that it provides is not as advanced as that of the Dynamic Calibrator but may be useful if you are just seeking to reduce depth image noise instead of correcting a serious calibration problem with the image (to the extent where the image is extremely broken up).

Intel have published a white paper document on the subject of On-Chip Calibration.

https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras

[zhanghx04]

@MartyG-RealSense Hi, I just use the dynamic calibration tool on Windows. It works well. Then, I plugged the camera (D435) on TX1. In the realsense-viewer, I set a box at 1.4m far away from the camera, but in a 3-D view, the edge of the box doesn't look like a sharp 90-degree angle (still distoreted). How can I improve it?

[MartyG-RealSense]

@zhanghx04 In the RealSense Viewer's options side-panel, there is a drop-down Presets menu near the top of it that provides a selection of alternative camera configurations. When the Viewer is first started up, it is set to 'Custom'. If you select the 'Default' preset from the menu, that preset provides a configuration that has clean edges and less "spraying" of 3D point clouds.

The preset resets to Custom each time that the Viewer is launched, so you cannot do any harm to the camera by experimenting with the different presets.

[zhanghx04]

@MartyG-RealSense Thanks for the reply. Actually, that is what I did before. The result still looks like this (the pictures are dark...).

Also, it comes a new problem today. I calibrated the D435 with dynamic calibration tool with printed target (10mm). When I use depth quality tool, the distance measured is about 1.6m, but I set the camera at 2m (+- 2%) from a flat wall. And, the wall is curved. The distance ( on two sides of the view) measured is less than the distance measured in the center of the view. I have no clue what happened. Did I do something wrong?

After that, I used my phone (iPhone X) as a target, the distance measured is correct, but the wall still is curved.

[MartyG-RealSense]

@zhanghx04 If the point cloud is twisted and wavy, it may be a "ghost noise" phenomenon.

#4553

Regarding the distance issue, if the wall is low in texture detail or has no texture then it will be more difficult for the camera to analyze it for depth detail unless you project an IR dot pattern from the camera onto the wall or alternatively provide strong ambient light in the scene for the camera to use for depth analysis of low-texture surfaces instead of the dots.

[zhanghx04]

@MartyG-RealSense

1. for dynamic calibration, is there any way that I can change the range between target and the camera in the code somewhere? Also change the checker's size?

2. I want to make sure that the object that 1.5m away from the camera could have a good point cloud without distortion. Could the custom calibration do that?
   Thank you!

[MartyG-RealSense]

@zhanghx04

1. You can create your own custom version of the Dynamic Calibrator tool with the Calibration API. The programming guide of the API will be able to tell you whether the features that you want can be included in a custom tool.

Calibration API
https://downloadcenter.intel.com/download/28517/Intel-RealSense-D400-Series-Calibration-Tools-and-API

Calibration API programmers guide
https://www.intel.com/content/www/us/en/support/articles/000026724/emerging-technologies/intel-realsense-technology.html

In regard to configuring the target, if you will be doing calibration with a target image displayed on an Android screen then section 10.2.3 of the programming guide may be helpful.

2. The optimal depth resolution on the D435 for best accuracy is 848x480 (higher resolution = better accuracy). You can also use the "Threshold Filter" in the Post-Processing section of the Viewer's options side-panel to set a minimum and maximum range so that data outside of that min-max range gets excluded from the image. This can help to remove unwanted data.

A change of RealSense camera model may also help. The D415 has around 2x better image quality and 2x less depth error ('RMS Error') over distance than the D435 models due to the different IR imager component that it uses and has a higher optimal depth sensing resolution of 1280x720.

It has a smaller field of view than the D435 and a slower shutter, but as long as you are not observing an object larger than the box and the box is not moving faster than human walking pace (as the D435 is more suited to tracking fast motion), the D415 could be an appropriate choice for your project.

I would also recommend avoiding using the camera in a location with fluorescent lights such as ceiling strip lights if possible, as the gas inside them flickers at rates that are hard to see with the human eye and can cause image noise. If you cannot avoid being in proximity to fluorescent lights, adjusting the FPS rate of the camera to a frequency closer to the operating frequency of the lights can help. For some lights it is 30 FPS and for others it is 60 FPS.

[zhanghx04]

@MartyG-RealSense
Thank you for replying! Your advice is really useful!!

Actually, the object is a cow, set the (8 cameras) like 1.5m away from the cage (cows can't move a lot), and want to get a good point cloud of the cow with minimum distortion. Do you think D415 will be suitable for the project?

for intrinsic parameters, they can be given from other algorithms and write them in the device with api, is that right?

[MartyG-RealSense]

@zhanghx04 8 cameras is a good number for 360 degree capture. You can have as many cameras as you want though, as the more you have, the better the data quality. This is because there are less blind spots and there is redundancy in the data due to overlapping fields of view observing the same area.

So a trade-off for higher image quality from D415 may be having to use more cameras than you would with D435 to capture a subject as big as a cow, due to having to take a smaller field of view into account when overlapping the cameras.

The CONIX Research Center at Carnegie Mellon developed an ethernet-based RealSense point cloud stitching system that could use up to 20 cameras.

#3367

Error over distance from RMS error only starts to become particularly noticable beyond the 3 meter point though, so you may be able to use the D435 without a serious impact on measurements compared to the D415 at 1.5 m range. This is because error increases linearly over distance, so near to the camera, the D415 and D435 have similar error and only start to diverge noticably once past the 1m point. The error curve of the D415 and D435 is charted in the link below.

https://forums.intel.com/s/question/0D70P0000068rTmSAI

Ethernet support for RealSense has improved considerably since that project was created, and Intel released a white-paper document on the subject recently:

#6332

If your end goal is to get a solid 3D model of the cow instead of a point cloud, a commercial 3D scanning solution that supports multiple RealSense cameras is RecFusion Pro.

https://www.recfusion.net/index.php/en/features

Regarding incorporating our own intrinsics calculations into calibration, this should be possible. Please visit the discussion in the link below and read downward through the comments from the point that I have linked to.

#2704 (comment)

[zhanghx04]

Hi @MartyG-RealSense
Just done the semester. Thanks for the detailed information. This information I maybe use in the future!

Just to make sure (correct my thought, if it is wrong):

For a single camera, for a long-range, the 3d view in realsense-viewer, the flat plate always is wavy. (more far, more wavy). If I want to get a perfect flat-plate point cloud, I should do some algorithms.
In order to get a good point cloud, I should use the rgb image and depth image but not the point cloud exported from the realsense-viewer. Am I right? ( I'm a novice on this CV field )

my project is to get the 3D model and get the volume. I don't think I can use the RecFusion Pro. I should implement by some sources and I can learn some from it.

[MartyG-RealSense]

@zhanghx04 The problem of wavy point clouds is primarily related to distance of the observed object from the camera, with waviness increasing as the distance increases. I consider the link below to be the best reference on the subject:

#1375

[RealSenseSupport]

@martinakos @ManuelS-BSH The new version of dynamic calibration v2.11 fixes the issue and now released on AWS. Please try to install and test. Thanks!

[RealSenseSupport]

@martinakos @ManuelS-BSH Did you get chance to try the new dynamic calibration v2.11? Looking forward to your update. Thanks!

[RealSenseSupport]

@martinakos @ManuelS-BSH Considering the bug fixed, closed it accordingly. Please feel free to create another new issue if you have other questions or issues. Thanks!