RealSense error calling rs2_run_on_chip_calibration_cpp(device:0x559eecacdfa0): hwmon command 0x80( 8 3 0 0 ) failed (response -7= HW not ready)

[96daja11]

---

Required Info
Camera Model	D435i
Firmware Version	05.12.06.00
Operating System & Version	Linux Ubuntu 18.04
Kernel Version (Linux Only)	5.4.0-42-generic
Platform	PC
SDK Version	2.36.0
Language	C++
Segment	Robotics

Hi,

I am running tests on multiple cameras (around 10) to test whether all the D435i has similar behavior coming from the factory. One thing I am comparing is the performance with factory settings and after a on chip self calibration. I am also trying out some different kinds of visual presets. The issue is that sometimes the
The code I use for configuring the cameras can be seen below. The issue is that sometimes the cameras are giving the following error when running the run_on_chip_calibration command.

#include <librealsense2/rs.hpp>
#include <librealsense2/rs_advanced_mode.hpp>
#include <fstream>
#include <iostream>

void calibrateCamera(rs2::pipeline& pipe, const rs2::config& stream_cfg,float& health){
  bool previous_stream = false;
  try{
    rs2::pipeline_profile active = pipe.get_active_profile();
    pipe.stop();
    previous_stream = true;
  }
  catch (const rs2::error & e){
    std::cerr << "No active profile" << std::endl;
  } 
  
  rs2::config cfg;
  cfg.enable_stream(RS2_STREAM_DEPTH, 256, 144, RS2_FORMAT_Z16, 90);
  rs2::device dev = pipe.start(cfg).get_device();
  rs2::auto_calibrated_device cal = dev.as<rs2::auto_calibrated_device>();
  std::string calibration_file_name = "/home/daniel/catkin_ws/src/pcl_test/calibration_params.json";
  std::ifstream t(calibration_file_name);
  std::string calibration_json((std::istreambuf_iterator<char>(t)), std::istreambuf_iterator<char>());
  try{
    rs2::calibration_table res = cal.run_on_chip_calibration(calibration_json, &health, [&](const float progress){ /* On Progress */ });
    std::cout << "Health: " << health << std::endl;
    cal.set_calibration_table(res);
    //if(std::abs(health) < 0.25){
      cal.write_calibration();
    //}
  }
  catch (const rs2::error & e){
    std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
  } 
  
  pipe.stop();
  if(previous_stream){
    pipe.start(stream_cfg);
  }
}

void configureCamera(rs2::pipeline& pipe, const std::string& config_file_name = "/home/daniel/catkin_ws/src/pcl_test/HighResHighAccuracyPreset.json"){
  rs2::device dev = pipe.get_active_profile().get_device();
  if (dev.is<rs400::advanced_mode>()){
    auto advanced_mode_dev = dev.as<rs400::advanced_mode>();
    // Check if advanced-mode is enabled
    if (!advanced_mode_dev.is_enabled()){
      // Enable advanced-mode
      advanced_mode_dev.toggle_advanced_mode(true);          
    }
    std::ifstream t(config_file_name);
    std::string config((std::istreambuf_iterator<char>(t)), std::istreambuf_iterator<char>());
    advanced_mode_dev.load_json(config);
    std::cout << "Configuration done" << std::endl;
  }
  else{
    std::cerr << "Current device doesn't support advanced-mode!\n";
  }
}


int main(int argc, char * argv[]) try
{  
  bool calibrate = true;

  rs2::pipeline pipe;
  rs2::config cfg;
  cfg.enable_stream(RS2_STREAM_DEPTH, 256, 144, RS2_FORMAT_Z16, 90);
  rs2::device dev = pipe.start(cfg).get_device();
  if(calibrate){
    float health;
    calibrateCamera(pipe, cfg, health);
  }
  else{
    rs2::auto_calibrated_device cal = dev.as<rs2::auto_calibrated_device>();
    cal.reset_to_factory_calibration();
    std::cout << "Factory calibration set " << std::endl;
  }
  bool default_preset = false;
  if(default_preset){
    configureCamera(pipe,"/home/daniel/catkin_ws/src/pcl_test/DefaultPreset_D435.json");
  }
  else{
    configureCamera(pipe);
  }
}
catch (const rs2::error & e)
{
  std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
  return EXIT_FAILURE;
}
catch (const std::exception& e)
{
  std::cerr << e.what() << std::endl;
  return EXIT_FAILURE;
}

Sometimes yields

RealSense error calling rs2_run_on_chip_calibration_cpp(device:0x559eecacdfa0):
    hwmon command 0x80( 8 3 0 0 ) failed (response -7= HW not ready)

Do you know why this error occurs? Usually it works by just trying again a couple of times. From what I have noticed while working with multiple cameras is that this error seem to happen with cameras that has been used for some time, and never happens with cameras that are brand new from the supplier. But I am not sure if that has anything to do with the actual problem. Any ideas about this? Thankful for help!

[MartyG-RealSense]

Hi @96daja11 I have not personally seen that error before. If it works after a couple of attempts, it may be exactly what the error describes - that the camera is not ready at the time that the on-chip calibration call is made.

You may be able to validate your theory about whether used cameras experience the problem more than factory-new ones by performing a gold reset on the older camera to reset it to factory defaults. A gold reset can also correct corruptions in the 'calibration table' stored in the camera.

The gold reset procedure is performed by a program called CustomRW that is part of the Dynamic Calibrator software for the 400 Series cameras. You can initiate a gold reset by inputting the following text command:

Intel.Realsense.CustomRW -g

To install the Dynamic Calibrator tool on Linux, please refer to page 14 onward of the user guide for the tool.

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html

[MartyG-RealSense]

Hi @96daja11 Do you still require assistance with this case, please? Thanks!

[96daja11]

Hi! I have not really come to any more conclusion about why the problem occurs. I would like to know if you got any more inputs about this. Otherwise I do not need any more help for now. Thanks for the help!

[MartyG-RealSense]

Thanks so much for the update! No, I have not come across any other cases of this particular error from other RealSense users since our last discussion.

[Chandler]

I am experiencing this exact error when running run_on_chip_calibration() on a d415

librealsense version: 2.38.1-0~realsense0.3378
firmware version: 05.12.07.100

hwmon command 0x80( 8 3 0 1 ) failed (response -7= HW not ready)

[Chandler]

This is kind of incredible but I narrowed it down to two code samples, one that works and one that doesn't.

import pyrealsense2 as rs

def test(calib_dev):
    new_calib, health = calib_dev.run_on_chip_calibration(json_content='', timeout_ms=5000)
    print("health factor = ", health)

pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 256, 144, rs.format.z16, 90)
conf = pipeline.start(config)
calib_dev = rs.auto_calibrated_device(conf.get_device())

test(calib_dev)

This does not work

def test(calib_dev):
    new_calib, health = calib_dev.run_on_chip_calibration(json_content='', timeout_ms=5000)
    print("health factor = ", health)

def setup():
    pipeline = rs.pipeline()
    config = rs.config()
    config.enable_stream(rs.stream.depth, 256, 144, rs.format.z16, 90)
    conf = pipeline.start(config)
    calib_dev = rs.auto_calibrated_device(conf.get_device())
    return calib_dev 

test(setup())

Something to do with setting up the pipeline inside a method and returning the device is causing problems, which to me indicates that something very strange is happening with global variables and scope.

[yassineAlouini]

@Chandler Hello there, I hope you have solved this issue.

Is doing the "gold reset" enough or did you had to do something else in addition?

Any help is welcome, thanks in advance!

[joshuafishman]

I had this issue trying to do on-chip calibration through realsense_viewer on a D435. It seems to have gone away with several restarts of the application.

[MartyG-RealSense]

Thanks very much @joshuafishman for sharing your own experience!

[yassineAlouini]

That's my conclusion as well: after many restarts, the auto-calibration works.

[bzhao]

I have a single unit of d435i, My task is calibration of it. Today I also met this problem with it, and then I decide to update Linux kernel drivers( by command ./scripts/patch-realsense-ubuntu-lts.sh), I I did reboot the PC, I started saw the problem is still there. After this I try press "Check for update" , later I did try to do the "on chip calibration", the problem has gone and I got “Health-Check" value as "0" (good).

In addition I must I am remotely operating the PC in the lab at home, the font in my screen are small and fuzzy.
the remote desktop sw cannot get me to copy it over it by selecting them.
the error I record by seeing it is: "invalid value in rs2_run_on_chip_calibration_cpp ... hwmon ...response ...
I have record it" So cannot say my problem is the exact same as the one met by OP

[xianzhi896]

thank u very much, I met the same problem and use your plan. Press "Check for update" really works. All my camera is fine

[subox]

Is there a solution for this issue?

My devices are: D435i
All in version: 5.14.0.0

[MartyG-RealSense]

Hi @subox A couple of the RealSense users commenting above found that calibration eventually worked if they restarted their application at least several times.

If you have created your own script for controlling on-chip calibration then you could also try incorporating a hardware_reset() instruction into your script to reset the camera when your program launches. Below are links to examples of code for doing so.

C++
#2161 (comment)

Python
#3329 (comment)

Also, have you tested calibrating wit the RealSense Viewer tool by going to the More option near the top of its options side-panel and selecting On-CHip Calibration from its drop-down menu, please?

[subox]

@MartyG-RealSense Thank you for the response but restarting a few times sometimes works (most of the time it doesn't).

I'd like to check the self-calibration health status every time I run the application to be sure my values are good and they don't need dynamic calibration.

The problem is that when I execute the official code example from https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras#appendix-b-details-on-self-calibration-flow-and-c-api, the errors occurs 90% of times of the next run the application.
The only solution, which I found is to physically replug the USB cable before running run_on_chip_calibration.
After repluging, the run_on_chip_calibration works just once and then again the same issue appear.

Here's the code I use:

std::string json = "{\
  \"average_step_count\": 20,\
  \"step_count\": 20,\
  \"accuracy\": 2,\
  \"scan_parameter\": 0,\
  \"data_sampling\": 0\
  }";

rs2::pipeline pipe;
rs2::config cfg;
cfg.enable_stream(RS2_STREAM_DEPTH, 256, 144, RS2_FORMAT_Z16, 90);
rs2::device dev = pipe.start(cfg).get_device();
rs2::auto_calibrated_device cal = dev.as<rs2::auto_calibrated_device>();
float health;
rs2::calibration_table res = cal.run_on_chip_calibration(json, &health, [&](const float progress) { /* On Progress */ });
pipe.stop();

The behavior is the same when I use a different config, for example:
cfg.enable_stream(RS2_STREAM_DEPTH, 640, 480, RS2_FORMAT_Z16, 60);

In the realsense-viewer, the autocalibration works but the purpose is to use the sdk.

Do you know the solution to run self-calibration using the available API?

[MartyG-RealSense]

@subox The most reliable librealsense SDK version for On-Chip calibration is 2.49.0. Newer versions (2.50, 2.51, 2.53) may experience initiation failures when starting On-Chip calibration.

[subox]

Thank you @MartyG-RealSense . When I compiled 2.49.0 version, I don't see any errors running run_on_chip_calibration function.

@MartyG-RealSense Can we reopen this issue, as it looks like all the recent releases for the last 1.5 years has a bug in self-calibration SDK?

[MartyG-RealSense]

I will re-open the issue and consult with my Intel RealSense colleagues about it.

[avizipi]

I Also have this issue. I tried with a camera with firmware 05.11.01.100 with these sdks: 2.49.0, 2.50.0, without a success.
However when I changed to a different camera camera with firmware 05.13.00.50 the calibration was successful with these sdks: 2.49.0, 2.50.0.

So maybe the problem is in the firmware?

[MartyG-RealSense]

Hi @avizipi Firmware 5.11.1.100 is very old and dates back to February 2019. On-Chip calibration was not introduced until later after this date and requires 5.12.2.100 or newer according to Intel's calibration user guide.

https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras

[MartyG-RealSense]

Hi @subox and @avizipi Do either of you require further assistance with this case, please? Thanks!

[subox]

Hi @MartyG-RealSense ,

right now, the workaround is to use 1.5 years old build, which is ok, for now.
What worries me is that using the old 2.49.0 version has a lot of compilation warnings, which is not good and I don't see it in the newer versions.
As well as using the outdated version means I don't have all bug fixes, which were fixed during this time. We have a big project, with many D435i devices and stability is crucial for us.

Do you know when can we expect the fix for the self-calibration for releases newer than 2.49.0?

[MartyG-RealSense]

@subox There is not a fix being worked on at the time of writing this as it was an issue that had a small number of cases reported and RealSense users found that it could be overcome in some cases by resetting. I will raise the issue with my Intel RealSense colleagues to see if there is anything that can be done.

In #7087 (comment) you state that an unplug-replug of the camera before running calibration is a solution in the newer SDK versions. Could you try adding a hardware_reset() instruction to the start of your script before calibration is activated, as this should have a similar effect to an unplug-replug of the camera but the reset will be performed automatically each time the script is run. A simple example of implementing hardware_reset() in C++ is shown below.

rs2::context ctx;
rs2::device dev = ctx.query_devices().front(); // Reset the first device
dev.hardware_reset();
rs2::device_hub hub(ctx);
dev = hub.wait_for_device();

[MartyG-RealSense]

I have had further discussions with my Intel RealSense colleagues about this case.

They stated that the disappearance of the Intrinsic and Extrinsic calibration options from the On-Chip interface in the RealSense Viewer from 2.50.0 onwards is intentional and not a bug. The interface design was simplified for the user by removing those options and just having a 'Calibrate' button. The design change was not referenced in the 2.50.0 SDK's release notes however, and so the missing options appeared to be a bug when it actually is not.

They also stated that when trying out different versions between 2.49.0 and 2.53.1, the recommended firmware for each specific SDK version should be installed in the camera, as using firmware 5.14.0.0 with SDKs 2.49.0, 2.50.0 and 2.51.1 could cause problems. The recommended firmware for each SDK is:

2.53.1: 5.14.0.0
2.51.1: 5.13.0.50
2.50.0: 5.13.0.50
2.49.0: 5.12.15.50

[avizipi]

For us the case is worst because we want to work in production recommanded configuration. (https://dev.intelrealsense.com/docs/recommended-production-camera-configurations)

So we will work in 2.38.1 and 5.12.7...
Hardware reset didn't do the trick for us, and in our configuration we can't just plug /unplug the camera several times.

Also to follow the camera calibration every time our system is booting we thought of running the calibration and follow the health score. So it is part of the boot sequence and we can not use the Real sense Viewer for it.

[MartyG-RealSense]

If the camera is in a location where it is not going to receive physical shocks such as hard knocks, a drop on the ground or severe vibration then you will not likely need to perform a calibration check every time that the program is run as the camera should maintain good calibration. Even in a critical production environment where the camera has the potential to receive knocks there should not be the need to check calibration more often than once a week. So if the camera is in a protected position then you may be able to leave out the calibration check and use the latest SDK.

[avizipi]

In our case the cameras always suffer from physical shocks, drops to the ground or severe vibration.
so having a better work around will help us very much!

[MartyG-RealSense]

It may be worth trying to change the scan speed for the json speed setting of rs2_run_on_chip_calibration to see whether it improves stability and reduces the likelihood of the error occurring.
https://intelrealsense.github.io/librealsense/doxygen/classrs2_1_1auto__calibrated__device.html#a75f06fc23740c6c4a7b08f68d48c75d1

To quote the relevant information from the self calibration guide:

---

The rs2_run_on_chip_calibration command is a blocking call that has speed as an argument:

0 = Very fast (0.66 seconds), for small depth degradation
1= Fast (1.33 seconds), medium depth degradation.
2= Medium (2.84 seconds), medium depth degradation.
3= Slow (2.84 seconds), for significant depth degradation.

There is a tradeoff between scan speed and degree of degradation. For slight degradation, “Very Fast” or “Fast” is acceptable, but for severe degradation, “Slow” may be necessary. In most cases, the recommended speed is “Medium”, which takes ~2.8 seconds to complete when running at 90 fps and scales in speed with frame rate.

[MartyG-RealSense]

Hi @avizipi Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.

[eilat-inbal]

@MartyG-RealSense is this issue resolved in the new SDK 2.55.1? or are there any specific params that I can use that make it work?

[MartyG-RealSense]

Hi @eilat-inbal This rs2_run_on_chip_calibration issue occurs on a small number of PCs and may not necessarily be a problem in the SDK.

If problems are encountered with the On-Chip calibration tool then as an alternative to performing a calibration I recommend instead resetting the camera to its factory-new calibration in the RealSense Viewer tool using instructions at #10182 (comment)

[eilat-inbal]

thanks @MartyG-RealSense but unfortunately i am experiencing this issue. with sdk 2.45 it was not an issue. now upgraded to 2.55.1 and i am not able to run the calibration at all from the c++ and python SDK!
from the realsense viewer however, it is working. can you please share with me any commands that are run upon startup of the viewer, or prior to the on chip calibration, that may help?
or maybe any config options?

[MartyG-RealSense]

Where problems with On-Chip calibration are experienced with modern SDK versions on a particular computer, typically it will work up until 2.49.0 and then have issues in 2.50.0 and newer. Significant changes were made to the calibration tool in 2.50.0, as listed at #9336

When you upgraded to 2.55.1, did you also update the camera's firmware driver to the 5.16.0.1 version recommended for 2.55.1 please?

The SDK has an official on-chip example program for Python at the link below that you could test.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/depth_ucal_example.py

There are also examples of programming the calibration interface for Python and C++ in the official guide to the calibration tool that you could compare to your own code to see whether anything in the procedure has been missed.

https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras#appendix-c-on-chip-calibration-python-api

https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras#appendix-d-on-chip-calibration-c-api