Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Dynamic scheduler thread scaling based on workload
Prior to this commit, the runtime would start up a specific number of scheduler threads (by default the same as the number of physical cores) on initialization and these scheduler threads would run actors and send block/unblock messages and steal actors from each other regardless of how many actors and what the workload of the program actually was. This usually resulted in wasted cpu cycles and cache thrashing if there wasn't enough work to keep all scheduler threads busy. This commit changes things so that the runtime still starts up the threads on initialization, but now the threads can suspend execution if there isn't enough work to do to minimize the work stealing overhead. The rough outline of how this works is: * We now have three variables related to number of schedulers; `maximum_scheduler_count` (the normal `--ponythreads` option), `active_scheduler_count`, and `minimum_scheduler_count` (a new `--ponyminthreads` option) * On startup, we create all possible scheduler threads (up to `maximum_scheduler_count`) * We can never have more than `maximum_scheduler_count` threads active at a time * We can never have less than `minimum_scheduler_count` threads active at a time * Scheduler threads can suspend themselves (i.e. effectively pretend as if they don't exist) * A scheduler thread can only suspend itself if its actor queue is empty and it has no actors in it's mute map and it would normally send a block message * Only one scheduler thread can suspend or resume at a time (the largest one running or the smallest one suspended respectively) * We can never skip a scheduler thread and suspend or wake up a scheduler thread out of order (i.e. thread 6 is active, but thread 5 gets suspended or thread 5 is suspended but thread 6 gets resumed) * If there isn't enough work and a scheduler thread would normally block and it's the largest active scheduler thread, it suspends itself instead * If there isn't enough work and a scheduler thread would normally block and it's not the largest active scheduler thread, it does normal scheduler block message sending * If there's a lot of work to do and at least one actor is muted in a scheduler thread, that thread tries to resume a suspended scheduler thread (if there are any) every time it is about to run an actor. NOTE: This could result in a pathological case where only one thread has a muted actor but there is only one overloaded actor. In this case the extra scheduler threads will keep being woken up and then go back to sleep over and over again. * The overhead to check if this scheduler thread is a candidate to be suspended (`&scheduler[current_active_scheduler_count - 1] == current scheduler address`) is a load and single branch check * The overhead to check if this scheduler thread is a candidate to be suspended (because `&scheduler[current_active_scheduler_count - 1] == current scheduler address`) but cannot actually be suspended because we're at `maximum_scheduler_count ` is one branch (this is in addition to the overhead from the previous bullet) * The overhead to check if there are any scheduler threads to resume is a load and single branch check The implementation of the scheduler suspend/resume is different depending on the platform. For Windows, it relies on Event Objects and `WaitForSingleObject` to suspend threads and `SetEvent` to wake suspended threads. For Posix environments, by default it relies on signals (specifically, SIGUSR2) as they are quicker than other mechanisms (pthread condition variables) (according to stackoverflow at: https://stackoverflow.com/a/4676069 and https://stackoverflow.com/a/23945651). It uses `sigwait` to suspend threads and `pthread_kill` to wake suspended threads. The signal allotted for this is `SIGUSR2` and so `SIGUSR2` has been disabled for use in the `signals` package with an error indicating that it is used by the runtime. An alternative implementation using pthread condition variables is also available via a `use=scheduler_scaling_pthreads` argument to make. This implementation relies on pthread condition variables and frees `SIGUSR2` so it is available for use in the `signals` package. It uses `pthread_cond_wait` to suspend threads and `pthread_cond_signal` to wake suspended threads. The old behavior of having all scheduler threads active all the time can be achieved by passing `--ponyminthreads=9999999` as an argument to a program (because minimum scheduler threads is capped to never exceed total scheduler threads). This commit also adds DTRACE probes for thread suspend and thread resume. This commit also switches from using `signal` to `sigaction` for the epoll/kqueue asio signals logic because `sigaction` is more robust and reliable across platforms (https://stackoverflow.com/a/232711).
- Loading branch information
