What should be the user-facing API

[erlingrj]

As discussed in #159, the Zephyr implementation oflf_thread_create uses statically allocated memory for stack and thread control blocks. It allocates space for a number of threads equal to the number of workers. This does not work if the users are gonna use reactor-c as a threading library and spawn threads and so on. There are also other functions, which are not prefixed with an underscore which I think should not be used by the user. E.g.

• lf_notify_of_event
• lf_sleep_until_locked
• lf_initialize_clock

I think this question also has to be addressed in supporting bodiless reactions. My opinion is to expose as little as possible and not try to be a threading library, but rather have the platform API as an internal API for runtime code only.

[lhstrh]

Link to the thread: #159 (comment)

[edwardalee]

The legitimate uses of the thread API that I have encountered have to do with I/O. E.g., create a thread that waits for keyboard input and then schedule a physical action when getting that input. Generally, you need a thread in LF whenever you need to call a blocking function. Is there some other way to provide such functionality?

[erlingrj]

With Zephyr I think we have three options:

1. Move to the POSIX API that Zephyr also can support. I deliberately did not use it because I think it adds more complexity to the application and more dynamic memory allocation. With the kthreads you have more control over their stacks and thread control blocks etc. With POSIX threads, I believe we can spawn an arbitrary amount of threads and memory allocation and freeing happening under-the-hood.
2. Still use kthread-API with static memory allocation for the runtime threads (NUMBER_OF_WORKERS) but add our own memory management layer for the threads spawned beyond these.
3. Not expose lf_thread_create, and then also probably not mutex and convar API either, to the users.

I am not yet convinced that we should expose a thread API, isnt it likely that the users are familiar with the API of the underlying OS? Then they can create pthread instead of lf-threads when e.g. working in Linux

[lhstrh]

I am not yet convinced that we should expose a thread API, isnt it likely that the users are familiar with the API of the underlying OS? Then they can create pthread instead of lf-threads when e.g. working in Linux

If we already provide a platform, then why should hide it from users? More use of our platform APIs will lead to more portable code... That seems to be a clear advantage to me. Is there a cost to this that I'm not appreciating?

[edwardalee]

I think one of the key advantages of having a minimal threading API exposed, in addition to portability, is the ability to make it clear what is NOT supported. For Zephyr, for example, I could see there being two distinct Zephyr platforms available, say, Zephyr-KThreads and Zephyr-Posix. The former could implement unimplementable functions by erroring out.

[erlingrj]

I will have to think a bit about this. I think it makes sense with 2 Zephyr platforms to be compatible with Applications using both pthreads and kthreads. I think the optimal design of the kthreads-platform is to statically allocate the stacks for the worker-threads, which will be the first NUM_WORKERS calls to lf_thread_create, and then do dynamic memory allocation for the remaining, the pointer to this memory and its size, must be stored such that it can be freed when the thread is freed. (Or joined? I will have to read up on this a bit)

[lhstrh]

To me, number of workers is not the same as numbers of threads. I think these things need to be kept separate...

[edwardalee]

To support federated execution, more threads than workers are needed (currently, although this may change).

[erlingrj]

I think it makes sense to statically allocate the stack for NUM_WORKERS threads (or NUM_WORKERS+1(?) in the case of federated execution). Even though a worker is not a thread each worker consumes exactly one thread and the total number of workers is known compile-time. This is information which should be used to reduce dynamic memory allocation