Initial import of KV-snapshot benchmark #1
Conversation
tzdybal
commented
Feb 5, 2021
tzdybal
commented
- General benchmark structure
- KV-store interface
- RocksDB store implementation
- Initial approach to plotting with Gnuplot
* General benchmark structure * KV-store interface * RocksDB store implementation * Initial approach to plotting with Gnuplot
|
https://github.com/cockroachdb/pebble is another very strong RocksDB-compat KV to try out. My prediction is it will be the best |
Both KV stores doesn't support querying past state yet. For badger we need to store timestamp on commit, and use it for retrieval. RocksDB should open a new DB handle from checkpointed version directory. Badger insert performance is probably caused by adding KV-pairs one by one, on single goroutine. Batch or WriteBatch probably will improve performance.
|
how about merging this PR? |
@pkieltyka - did you benchmark pebble with RocksDB or baddger using a batched updates? |
As expected, without any form of batching it's very slow (still waiting for results).
|
Can you link to the results in a comment here? The plots still seem empty. |
I still wait for them to finish generating ;) |
| @@ -0,0 +1,20 @@ | |||
| package main | |||
|
|
|||
| type KV interface { | |||
There was a problem hiding this comment.
I think this should be the tm-db interface. That we can test what is already in use and possibly how we can modify the interface to better suit the needs here.
There was a problem hiding this comment.
I created new interface for maximum flexibility (and no dependencies). For example I need snapshots, but don't need "Sync" version of functions.
There was a problem hiding this comment.
makes sense. I would hope the sdk would remove tm-db so this along with my plans.
There was a problem hiding this comment.
Let's firstly have a PoC and validate the approach with benchmarks.
As IAVL use "number of nodes" not "number of bytes" for cache
