**Feature Request**: block arming after sensor calibration, require system reboot (or graceful re-init) prior to flight

[Antiheavy]

I've seen it recommended many times that the system should get a full power-cycle reboot after sensor re-calibration. In out own experience, the step change or dynamic motion after sensor re-calibration (especially Compass Cal) often causes EKF or sensor warnings. We've had flights where the EKF got unhappy after takeoff due to a re-calibration during preflight, ultimately causing failsafe behavior and sometimes vehicle crashes.

We've experimented a lot with soft reset of the EKF and soft reset of the whole firmware, but that often cause various other errors too - certain external sensor may remain powered during a soft reset or state machine things aren't resetting and that may mess with things. The most reliable method of cleanly re-initializing the EKF after calibration is a full system power cycle.

Proposal: after completion of a sensor calibration :

1. send the message "sensor calibration updated, reboot required before arming",
2. If arming is attempted, block the arming and send the same message again.
   A system flag may be required for this.

FYI @dagar @priseborough

[LorenzMeier]

That makes total sense.

[priseborough]

I support this proposal. The step change in sensor offsets combined with the motion associated with calibration can be outside estimator tuning limits and and number of states/variables external to the estimator that would need to be handled correctly make an enforced reboot the logical choice.

[dagar]

I would propose trying to directly address these issues rather than a full power cycle.

1. The state machine shouldn't allow arming with any of these errors. If this isn't already the case in master after commander state machine fix preflight and prearm error #9193 it needs to be fixed.

2. The system should be aware of calibration status either as a new calibration_status message or possibly as an entirely new arming state.

3. The EKF should reinitialize after it sees the transition out of ARMING_STATE_CALIBRATION.

4. If the EKF has any initialization requirements we should monitor, enforce, and report them.

   • no rotation? "Vehicle must be still while EKF is initializing"
   • @priseborough anything else?
   • I don't know that it matters anymore, but we'd also have everything in place to ignore that for ARMING_STATE_IN_AIR_RESTORE

1 & 2 could be done first and used to implement @Antiheavy's proposal immediately while 3 & 4 can come later.

[Antiheavy]

no rotation? "Vehicle must be still while EKF is initializing"

the system could check if rotation and acceleration is low before re-initializing. If motion is detected a warning can be set: "Hold vehicle still to re-initialize sensors after calibration". I also think there should be some positive feedback after a calibration that the re-initialization was successful. This could be a text message, but might be better as a "all good" state indicator displayed in QGC. e.g. #8697

[Antiheavy]

and number of states/variables external to the estimator that would need to be handled correctly make an enforced reboot the logical choice.

@priseborough makes an important point here as well.

[dagar]

and number of states/variables external to the estimator that would need to be handled correctly make an enforced reboot the logical choice.

@priseborough makes an important point here as well.

He's not wrong, I just think it's a sign of a deeper problem if we can't find a way to do this cleanly/safely.

[Antiheavy]

Related issue in the QGC repo from another user: mavlink/qgroundcontrol#6768 (comment)

[dagar]

Perhaps we should finally do something about this. I propose we add a CALIBRATION state to the arming state transition table.

EKF2 already monitors the arming state to know when it's safe to save parameters for things like the learned mag bias. On transition out of CALIBRATION we can have EKF2 reinitialize itself, including asking the user to hold the vehicle still if necessary.

Thoughts?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Antiheavy]

This issue is still valid. Multiple solutions have been proposed here for consideration if/when someone is able to work on it.

[Antiheavy]

still valid. more graceful re-initialization after re-calibration is needed for commercial and consumer grade products.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Antiheavy]

Still valid issue.

[dagar]

A different solution - #15655

[dagar]

I think we can consider this one effectively resolved for a few reasons.

1. The estimator(s) are now aware of sensor calibration changes (accel/gyro/mag) synchronized with the actual change applied.
2. We actually reset the estimated mag bias on change.
3. Sensor calibrations (accel/gyro/mag) no longer involve zeroing out the old calibration first. The calibration procedures are always performed directly on raw data, and on success the calibration is updated from old -> new atomically with a synchronized estimated bias reset.

If there's still concern I'm happy to entertain additional mechanisms for further enforced user behaviour (requiring the user to stop spinning before completing mag cal?), but I'd like to see data first.

[Antiheavy]

hooray!