Fix watchdog termination

[erlingrj]

This PR address this issue. It also moves the watchdogs to the environment struct and should be merged with this PR in lingua franca.

This PR does a couple of things:

• Solves race-condition with the watchdog threads on termination.
• Creates a single, non-terminating thread per watchdog instead of terminating at each timeout
• Registers the watchdog trigger at a logical tag equal to the expiration.
• Brings watchdog support to the Zephyr target, for this to work we must know, compile time, of the number of watchdogs so we can pre-allocate a stack for the watchdog threads.

[erlingrj]

Zephyr fails because additional "user threads" must be specified. We have previously used compile defs for this. We might consider adding a LF_NUM_WATCHDOGS. A problem is however that the zephyr implementation does not work well with dynamically creating and destroying threads. Since I did add a condition variable we could probably make it work without terminating the watchdog thread at each timeout

[edwardalee]

I think the idea with watchdogs was that on embedded platforms they would be implemented using a timer rather than thread. This will require an addition to the platform.h abstraction, I think, where the default implementation uses a thread. But with Zephyr, it seems like maybe there is a way to implement it with a timer?

Zephyr fails because additional "user threads" must be specified. We have previously used compile defs for this. We might consider adding a LF_NUM_WATCHDOGS. A problem is however that the zephyr implementation does not work well with dynamically creating and destroying threads. Since I did add a condition variable we could probably make it work without terminating the watchdog thread at each timeout

[erlingrj]

I remember we discussed an algorithm to implement watchdogs without threads, with an ISR. For Zephyr, I think using a thread is OK.

[edwardalee]

These are very useful changes. This should fix the long-standing bug but also properly puts the watchdog timer in the environment. However, I'm not sure, but I think there might be a flaw that could cause watchdogs to be randomly cancelled. Should be an easy fix, though. See detailed comments.

[erlingrj]

Thanks Edward. You are right about the spurious wakeups of lf_cond_timedwait. I also just remembered the issue with lf_cond_timedwait needing an absolute wakeup-time wrt CLOCK_REALTIME, however lf_time_physical is wrt CLOCK_MONOTONIC. It is related to our discussion on #289. It means that before going to sleep on a cond_var we need to check the offset between CLOCK_MONOTONIC and CLOCK_REALTIME and adjust for it.

We would make it easier for ourselves if we didn't try to solving the clock synchronization problem within the LF runtime and instead just used CLOCK_REALTIME and require that the underlying system is configured correctly

[erlingrj]

In order to properly implement the watchdog termination I need to first get #346 merged.

[erlingrj]

Thanks Edward

[edwardalee]

Looks good! I've made some code clarity suggestions. Also, I'm not sure about the active field. I inserted a question.

[erlingrj]

@edwardalee thanks for the suggestions improving the readability of the main loop of the watchdog thread. I replied to a comment on the active field somewhere. The active field is only written to by the watchdog thread and it indicates whether it has stopped/timed-out or whether it is actively waiting for a time-out. If it is the former, then a call to lf_watchdog_start must also signal the cond-var to wake it up from the inactive state.

If this makes sense, lets get this merged

[edwardalee]

Merged this and queued the corresponding merge in lingua-franca.