Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-process reads for RocksDB among multiple Faust apps #330

Closed
wbarnha opened this issue Jul 26, 2022 · 0 comments · Fixed by #331
Closed

Multi-process reads for RocksDB among multiple Faust apps #330

wbarnha opened this issue Jul 26, 2022 · 0 comments · Fixed by #331

Comments

@wbarnha
Copy link
Member

wbarnha commented Jul 26, 2022

I'd like to be able to setup multiple workers/Faust Apps that have the same name to read the same Faust Table entries. I may need to make some changes to python-rocksdb in order to achieve this.

I understand some of this functionality was added in #98, the note from Ask Solem in robinhood/faust#299 (comment) is no longer true.

Taken from https://github.com/aayushKumarJarvis/rocks-wiki/blob/master/RocksDB-FAQ.md:

Q: Can I write to RocksDB using multiple processes?

A: No. However, it can be opened in read-only mode from multiple processes.

Q: Does RocksDB support multi-process read access?

A: RocksDB support multi-process read only process without writing the database. This can be done by opening the database with DB::OpenForReadOnly() call

Could we have a Faust app configured to only read from a RocksDB file so that multiple processes could share one RocksDB file and only one process writes to the RocksDB file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant