-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make File Operations async. (ArchivalState, etc) #75
Comments
Addresses #75. Makes file operations in archival_state use non-blocking tokio APIs instead of the blocking APIs in `std`. Wraps mmap writes and reads in tokio::task::spawn_blocking(), to make it async-friendly.
Addresses #75. Changes store_utxo_ms_recovery_data() and read_utxo_ms_recovery_data() to use non-blocking I/O via tokio APIs.
The only remaining blocking I/O that I'm aware of is in twenty-first storage. To fix that, we need to add a tokio dep in twenty-first, or move storage out of twenty-first somehow. It's also possible there is blocking I/O in triton_vm or tasm-lib, but I'm hoping not.... ;-) Another consideration there is that compute-heavy functions should also be made into tokio tasks to avoid concurrency problems. I haven't investigated if we are doing that or not, eg when exec'ing VM code. |
closed by #124 |
Related to #74.
Presently many, if not all, file operations use synchronous APIs, called from async functions. For example async fn get_latest_block() eventually calls get_block_from_record which opens a file and reads it in via (unsafe) mmap calls.
These sync operations will block the tokio executor which can result in poor performance/concurrency. So in general, it should be more performant to convert the file ops to async counterparts provided by tokio, or at least wrap them with spawn_blocking.
note: I'm unsure about making the mmap calls async; an area for research.
The text was updated successfully, but these errors were encountered: