Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

kvdb-rocksdb overlay cache is not effective #4508

Closed
NikVolf opened this issue Dec 28, 2019 · 2 comments
Closed

kvdb-rocksdb overlay cache is not effective #4508

NikVolf opened this issue Dec 28, 2019 · 2 comments
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task.

Comments

@NikVolf
Copy link
Contributor

NikVolf commented Dec 28, 2019

Seems that cache in kvdb itself (memorizing raw key-value pairs on write), despite showing effectiveness in synthetic benchmarks, offers no performance advantage for sync speed, sometimes even negative (anybody can try run polkadot it with parity-common replaced from this branch https://github.com/paritytech/parity-common/tree/no-overlay). Not counting increased memory footprint.

This is probably due to:

  • There are a higher-level cache-s with good hit ratio.
  • RocksDb has its own raw key-value cache, and quite configurable.

Given that, and also the fact that this cache prevents other possible optimisations (such as zero(little)-copy database transactions), we might want to remove it.

There are probably considerations other than sync speed, which would be great to hear about.

@NikVolf NikVolf added the I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. label Dec 28, 2019
@NikVolf NikVolf changed the title kvdb overlay cache is not effective kvdb-rocksdb overlay cache is not effective Dec 28, 2019
@NikVolf
Copy link
Contributor Author

NikVolf commented Jan 5, 2020

In fact, it is not used at all probably

So I might draft a version of kvdb with no overlay cache + significantly reduced copy of transaction data (now any key and any value must be put into dedicated Vec<u8>, and this totally can be avoided)

@ordian
Copy link
Member

ordian commented Aug 26, 2020

This resolved in paritytech/parity-common#313 I believe.

@ordian ordian closed this as completed Aug 26, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task.
Projects
None yet
Development

No branches or pull requests

2 participants