Failure detector / Failsafe (Parachute) triggering #10179

bresch · 2018-08-07T09:40:14Z

Replaces #10091

Warning
This has never been tested in flight and could potentially kill the vehicle in air.
~~This PR enables flight termination by disabling the corresponding circuit breaker (reverts ab40008)~~

Description
This PR enables failsafe parachute triggering capabilities for an autopilot with or without IO processor by the use of disarmed and failsafe values.

Any kind of failure detection can be made using the FailureDetector class, which is instantiated in Commander. At the moment, a simple attitude check is made and sets its internal flags to true if roll or pitch exceed some defined parameters (FD_FAIL_R/P). On each axis, an hysteresis can be configured using the parameters FD_FAIL_R/P_TTRI.
Commander then sets the force_failsafe flag (actuator_armed message) that goes to the FMU driver and to the IO chip (if available).

If the autopilot has an IO chip, one can set a parachute (or other failsafe device) pin to an IO pin and IO will also then be able to trigger it if FMU dies (through a timeout check - 0.5s).

The objective is also to do different check and to have different behaviors if the vehicle is a fixed-wing, a multicopter, a VTOL or a rover. FailureDetector will then be a base class for FWFailureDetector, MCFailureDetector, etc.

How to use it

Set the "OFF" value of the failsafe system (parachute) using a PWM_MAIN/AUX_DISx parameter
Set the "ON" value of the failsafe system using a PWM_MAIN/AUX_FAILx parameter
Adjust FD_FAIL_R/P_TTRI hysteresis if needed
Add a NullMixer (Z:) where the failsafe system has to be connected (already on output 7 of the quad_x mixer)

Mainly useful for

Parachute triggering
Foldable landing gear (unfold to protect the gimbal and camera)
Gimbal (look up to protect the camera)
Activate drone finder device
...

TODO

The only way to reset the failsafe state should be to unpower/repower the autopilot, at the moment, the force_failsafe flag is reset when FMU reboots (remove https://github.com/PX4/Firmware/blob/81f0b64ad32ffaf446ad9e3f937986711f353ac2/src/drivers/px4io/px4io.cpp#L1393 ?)
Low-pass the attitude data in the Failure Detector
Run Failure Detector before arming and check its state in the preflight checks. Deny arming if a flag is set
Replace quadchute logic for VTOL (use VTOLFailureDector class) Do that in separate PR
Test a lot in simulation and test on real drone with disabled logic (CBRK_FLIGHTTERM = 121212). The flag force_failsafe is set even if the logic is disabled but doesn't trigger the parachute.

FYI @LorenzMeier @bkueng @RomanBapst @dagar

@dagar This now prevents issues where a mixer could output some values to the parachute channel (a NullMixer has to be set to that specific output and it outputs NANs which are translated to disarmed or failsafe values only).

Flight termination logic diagram:

dagar · 2018-08-07T14:28:30Z

src/lib/circuit_breaker/circuit_breaker_params.c

@@ -113,7 +113,7 @@ PARAM_DEFINE_INT32(CBRK_AIRSPD_CHK, 0);
 * @category Developer
 * @group Circuit Breaker
 */
-PARAM_DEFINE_INT32(CBRK_FLIGHTTERM, 121212);


Just for testing?

Nope. the termination should always be available by default. At the moment, if someone sets "Terminate" in the drop-down menu of the "Safety" tab of QGC, flight termination will not be triggered. Flight termination was disabled by default "for now", 4 years ago, because "it wasn't safe" but the objective of this PR is also to review and test the flight termination logic.

That sounds fine as long as we're sure of the implications system wide and document it clearly. Especially within the px4io firmware.

This circuit breaker is now again enabled by default.

Let me try to wrap my head around this:

Setting this parameter to 121212 will disable the flight termination action.

Ok, I understand that flight termination is enabled given the param is 0 by default, so far so good.

WARNING: ENABLING THIS CIRCUIT BREAKER IS AT OWN RISK

Hm, what risk? So enabling the circuit breaker means no termination is possible which is risky, ok got it. Although having the termination enabled seems risky in its own way.

Ok, I think I get it and it's correct.

dagar · 2018-08-07T14:29:55Z

This is looking better.

src/modules/commander/commander.cpp

bkueng

Generally looks good. I remarked a few things, but didn't look at the mixer changes in-depth.

src/modules/commander/failure_detector/FailureDetector.cpp

src/modules/commander/failure_detector/FailureDetector.hpp

src/modules/commander/commander.cpp

src/modules/commander/failure_detector/FailureDetector.cpp

src/modules/logger/logger.cpp

src/modules/commander/failure_detector/FailureDetector.cpp

src/modules/commander/Commander.hpp

src/modules/commander/commander.cpp

bresch · 2018-08-21T13:09:19Z

Now IO sets failsafe PWM values if FMU is not responding but was initialized and armed. This action can be disabled by the flight termination circuit breaker.

bresch · 2018-08-28T17:00:03Z

I've rebased my local branch cleanly on master (now that the safe version has been merged). I'll do some bench tests before pushing again.

bresch · 2018-08-29T11:49:43Z

@dagar I added a check in Preflight Checks so it will reject arming if a failure is already detected on ground. Did I do it the correct way?

dagar · 2018-08-29T13:12:51Z

Do you only want it to block arming with a message, or potentially complain periodically preflight?

bresch · 2018-08-29T13:23:54Z

Block arming only.

dagar · 2018-08-29T13:28:18Z

Then I would do it in the state machine helper prearm check.

https://github.com/PX4/Firmware/blob/master/src/modules/commander/state_machine_helper.cpp#L920-L1009

What's the intended use case? Blocking rearming after a real failure or preventing a tipped vehicle from arming? Does the failure ever clear?

bresch · 2018-08-29T15:31:55Z

The intended use case is to prevent arming a vehicle if it would directly enter the flight termination state after arming. The failure flags are always updated and can freely be set and cleared by the Failure Detector. The termination (no return) is only active when the vehicle is armed.

dagar · 2018-08-29T16:32:52Z

Makes sense, prearm_check in state machine helper sounds appropriate.

bresch · 2019-01-08T08:33:40Z

Rebased on master and tested on a bench setup with a Pixhawk4.

Works as expected:

with SYS_USE_IO=1: switch to failsafe if FMU dies or if attitude failure has been detected
with SYS_USE_IO=0: switch to failsafe if attitude failure has been detected
rejects arming if attitude failure is detected during the preflight checks
tested with several angles, disabled if angle is 0
tested with different hysteresis values for roll and pitch

@PX4/testflights Can you fly this PR with a quad with your standard parameters (don't enable flight termination!) to confirm that there is no regression? Thanks!

…fe is set

…AIL parameters description

…uit breaker. The circuit breaker is used inside IO to decide if it has to go into failsafe if FMU dies and also controls is the FailureDetector sets the force_failsafe flag. The other sources of flight termination (rc loss, geo fence, ...) are not disabled by the circuit breaker

…on_triggered

…; the user does not need to check the bitmask

…nge its type to bool since circuit_breaker_enabled returns a boolean

vehicle_status as const type. Also use FAILURE_NONE enum to check for a failure

…ighttermination".

This is required because otherwise, IO would output disarmed values if FMU goes into a disarmed state.

… in air

bresch · 2019-06-05T12:05:57Z

Rebased on master, ready for merging

bkueng

Let's merge this then

dagar reviewed Aug 7, 2018

View reviewed changes

dagar added the Admin: Enhancement (improvement) 💡 label Aug 7, 2018

dagar reviewed Aug 7, 2018

View reviewed changes

src/modules/commander/commander.cpp Outdated Show resolved Hide resolved

bkueng requested changes Aug 8, 2018

View reviewed changes

bresch commented Aug 8, 2018

View reviewed changes

src/modules/commander/Commander.hpp Outdated Show resolved Hide resolved

dagar reviewed Aug 8, 2018

View reviewed changes

src/modules/commander/commander.cpp Outdated Show resolved Hide resolved

bresch mentioned this pull request Aug 9, 2018

FailureDetector (safe version) #10202

Merged

bresch force-pushed the dev-failure-detector-failsafe branch 6 times, most recently from bdb7391 to 93e0735 Compare August 21, 2018 13:06

bresch force-pushed the dev-failure-detector-failsafe branch 2 times, most recently from 23962bb to 8f266ec Compare August 28, 2018 14:25

bresch force-pushed the dev-failure-detector-failsafe branch from 8f266ec to ed38641 Compare August 29, 2018 11:46

bresch force-pushed the dev-failure-detector-failsafe branch from ed38641 to e29f7de Compare September 19, 2018 09:25

bresch force-pushed the dev-failure-detector-failsafe branch from e29f7de to 4229d2d Compare December 20, 2018 13:55

bresch changed the title ~~[WIP] Failure detector / Failsafe (Parachute) triggering~~ Failure detector / Failsafe (Parachute) triggering Jan 8, 2019

bresch force-pushed the dev-failure-detector-failsafe branch from 87b1333 to 30968e4 Compare January 8, 2019 08:48

bresch added 24 commits June 5, 2019 13:50

Failure Detector - Play TONE_PARACHUTE_RELEASE_TUNE when force failsa…

6582bda

…fe is set

PWM - Remove unused MIXERADDSIMPLE

cf342a7

quad x mixer - Add null mixer for parachute triggering

c152a7a

Failure Detector - Add Failure Detector check to preflight checks

bf809a8

Failure detector - Disable flight termination by default. Modify FD_F…

0488d7b

…AIL parameters description

FailureDetector - Add roll and pitch failures hysteresis

4bbb2de

MC Lnd detector - Remove double include

a07670e

FailureDetector - Increase min value of FD_FAIL_P/R to 60 degrees

1170f8c

Failure detector - Add flight termination comments, make format

8f3574f

quad_x_main - rename parachute output to failsafe output

a41bb72

flight termination - rename in_flight_termination to flight_terminati…

c91f7de

…on_triggered

FailureDetector - Add is_failure() function to simplify the interface…

f1c4151

…; the user does not need to check the bitmask

px4io - rename safety_param_val to circuit_breaker_io_enabled and cha…

246ab89

…nge its type to bool since circuit_breaker_enabled returns a boolean

Failure detector - change snake_case function names to camelCase

1405a0e

Failure detector - in failure detector preflight check, get

42a8a8e

vehicle_status as const type. Also use FAILURE_NONE enum to check for a failure

Flight termination - Rename "flightterm" variables and defines to "fl…

87a1686

…ighttermination".

Flight termination IO - reword flight termination flag description

182568f

protocol.h - comment style update

cf2cd0b

IO mixer - cleanup FMU timeout check (comments and indentation)

7b3cf8c

Failsafe - cosmetic changes

c99ba1c

IO failsafe - apply failsafe values to outputs when in failsafe.

7eecd70

This is required because otherwise, IO would output disarmed values if FMU goes into a disarmed state.

IO driver - Recover flight termination state from IO after FMU reboot…

ff85321

… in air

FailureDetector - update hysteresis to comply with new interface

c850c75

bresch force-pushed the dev-failure-detector-failsafe branch from 9075825 to c850c75 Compare June 5, 2019 12:03

bkueng approved these changes Jun 5, 2019

View reviewed changes

bkueng merged commit ac44584 into PX4:master Jun 5, 2019

bresch deleted the dev-failure-detector-failsafe branch June 5, 2019 14:53

weekly-digest bot mentioned this pull request Jun 9, 2019

Weekly Digest (2 June, 2019 - 9 June, 2019) #12219

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failure detector / Failsafe (Parachute) triggering #10179

Failure detector / Failsafe (Parachute) triggering #10179

bresch commented Aug 7, 2018 •

edited by AuterionWrikeBot

Loading

dagar Aug 7, 2018

bresch Aug 8, 2018

dagar Aug 8, 2018

bresch Jan 8, 2019

julianoes Feb 13, 2019

dagar commented Aug 7, 2018

bkueng left a comment

bresch commented Aug 21, 2018

bresch commented Aug 28, 2018

bresch commented Aug 29, 2018

dagar commented Aug 29, 2018

bresch commented Aug 29, 2018

dagar commented Aug 29, 2018

bresch commented Aug 29, 2018

dagar commented Aug 29, 2018

bresch commented Jan 8, 2019

bresch commented Jun 5, 2019

bkueng left a comment

Failure detector / Failsafe (Parachute) triggering #10179

Failure detector / Failsafe (Parachute) triggering #10179

Conversation

bresch commented Aug 7, 2018 • edited by AuterionWrikeBot Loading

dagar Aug 7, 2018

Choose a reason for hiding this comment

bresch Aug 8, 2018

Choose a reason for hiding this comment

dagar Aug 8, 2018

Choose a reason for hiding this comment

bresch Jan 8, 2019

Choose a reason for hiding this comment

julianoes Feb 13, 2019

Choose a reason for hiding this comment

dagar commented Aug 7, 2018

bkueng left a comment

Choose a reason for hiding this comment

bresch commented Aug 21, 2018

bresch commented Aug 28, 2018

bresch commented Aug 29, 2018

dagar commented Aug 29, 2018

bresch commented Aug 29, 2018

dagar commented Aug 29, 2018

bresch commented Aug 29, 2018

dagar commented Aug 29, 2018

bresch commented Jan 8, 2019

bresch commented Jun 5, 2019

bkueng left a comment

Choose a reason for hiding this comment

bresch commented Aug 7, 2018 •

edited by AuterionWrikeBot

Loading