frontend: avoid being starved by CPU-intensive tasks

[BugenZhao]

More severely, constant-folding this expression causes the frontend unable to establish new connections for further requests, since tokio tends to move the IO scheduler to the same worker thread with this session, and an infinite loop starves the IO scheduler so that the main thread listening to the connection cannot be woken again! 😨

Although we should definitely fix this issue, it's also possible that the optimizer becomes more complicated and CPU-intensive. This might be relieved by using a separate runtime for frontend sessions (like #4756), but the stuck is inevitable when worker threads are exhausted (note that CPU-intensive tasks cannot be canceled). Or, we need spawn_blocking or block_in_place in every possible blocking code.

cc @liurenjie1024 @wangrunji0408

Originally posted by @BugenZhao in #8949 (comment)

[liurenjie1024]

I think we should have different runtime for different tasks, e.g. IO bound runtime and CPU bound runtime.

[BugenZhao]

I believe the core problem is that there's no way to cancel a user-mode coroutine that is busy executing synchronous code unless it's associated with a dedicated thread. But this is the opposite of our using coroutine to achieve more efficient I/O and concurrency, especially for the frontend.

[wangrunji0408]

unless it's associated with a dedicated thread.

Even if the task is running on a dedicated thread, it is not safe to kill the thread via system call, which can lead to resource leaks and undefined behavior. As evidence, there is no such API in the std library. The only way to exit a thread safely is to notify the thread and wait for its self-exit. This is the same as coroutines.

[github-actions]

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.