-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MRG: support check --upgrade
to upgrade old versions of RocksDB/RevIndex
#581
Conversation
I like the idea of having |
check --upgrade
to upgrade old versions of RocksDB/RevIndex
…water into fix_ro_rocksdb
Ready for review & merge! |
implemented! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lgtm!
This PR deals with the ramifications of a format change for RocksDB/RevIndex databases that probably occurred around v0.9.7, in which the 'storage' column was added to RevIndex databases. Prior to that version, RevIndex databases always stored sketches externally; see #390, #416, and sourmash-bio/sourmash#3250.
The addition of the 'storage' column breaks when trying to open old RocksDB/RevIndex databases with more recent versions of the plugin; the error message is:
Invalid argument: Column family not found: storage
.This PR:
manysearch
andfastmultigather
so that a slightly better error message is reported;index
with an option--upgrade
that will open when searching against a RocksDB, they will automatically upgrade the internal RocksDB format from older versions. This is as simple as opening them in read/write mode instead of read-only mode, which permits sourmash to upgrade the RevIndex;An alternative solution would be to always open the database in rw mode so that it just gets upgraded automatically, but it gives me the heeby jeebies to open a database in rw mode unnecessarily...
Fixes #580
TODO