Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rocksdb performance improvements #9080

Merged
merged 6 commits into from
Feb 6, 2025

Conversation

gfukushima
Copy link
Contributor

@gfukushima gfukushima commented Feb 3, 2025

PR Description

This PR is a alean up after #9059 which was used mostly for tests/deployment. It adds a couple params that are bringing the db to much healthier state. Iterators are performing much better now looking at the timestamps of logs (order of ms against minutes using the current config).
Example:

"@timestamp":"2025-02-03T22:52:44,526","level":"DEBUG","thread":"CombinedStorageChannel-0","class":"CombinedKvStoreDao","message":"Fetching earliest beacon block from db","throwable":""}
{"@timestamp":"2025-02-03T22:52:44,585","level":"DEBUG","thread":"CombinedStorageChannel-0","class":"CombinedKvStoreDao","message":"Earliest block slot in column: Optional[SignedBeaconBlockDeneb{message=BeaconBlockDeneb{slot=2503200, proposer_index=254597, parent_root=0x09071872d788a160ce5c4f7f01214157a719c2c1ccb80a0114300bc62c691568, state_root=0x44cb56c1b485427921866e3907665bca04ff5c1579af5c48bc742ef925ec4752, body=BeaconBlockBodyDeneb{randao_reveal=SszByteVector{0x97c9b4986e

Increasing the cache allowed to rocksdb hasn't shown any negative impacts on the memory consumption.
Log rotation also helps to keep the amount of log files we keep to a minimum reusing already created ones.

Testing node running these configs usage:

| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |
| Column Family                       | Keys            | Total Size  | SST Files Size  | Blob Files Size  |
| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |
| null                                | 14              | 138 MiB     | 138 MiB         | 0 B              |
| HOT_BLOCKS_BY_ROOT                  | 56              | 6 MiB       | 6 MiB           | 0 B              |
| HOT_STATES_BY_ROOT                  | 1               | 138 MiB     | 138 MiB         | 0 B              |
| HOT_BLOCK_CHECKPOINT_EPOCHS_BY_ROOT | 56              | 20 KiB      | 20 KiB          | 0 B              |
| SLOTS_BY_FINALIZED_ROOT             | 969506          | 38 MiB      | 38 MiB          | 0 B              |
| FINALIZED_BLOCKS_BY_SLOT            | 974629          | 35 GiB      | 35 GiB          | 0 B              |
| BLOB_SIDECAR_BY_SLOT_AND_BLOCK_ROOT_AND_BLOB_INDEX | 298522          | 10 GiB      | 10 GiB          | 0 B              |
| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |
| ESTIMATED TOTAL                     | 2242784         | 45 GiB      | 45 GiB          | 0 B              |
| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |
Displaying column usage from RocksDB folder /db
| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |
| Column Family                       | Keys            | Total Size  | SST Files Size  | Blob Files Size  |
| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |
| null                                | 10              | 256 MiB     | 256 MiB         | 0 B              |
| HOT_BLOCKS_BY_ROOT                  | 63              | 6 MiB       | 6 MiB           | 0 B              |
| STATE_ROOT_TO_SLOT_AND_BLOCK_ROOT   | 6               | 10 KiB      | 10 KiB          | 0 B              |
| HOT_STATES_BY_ROOT                  | 1               | 256 MiB     | 256 MiB         | 0 B              |
| HOT_BLOCK_CHECKPOINT_EPOCHS_BY_ROOT | 63              | 44 KiB      | 44 KiB          | 0 B              |
| SLOTS_BY_FINALIZED_ROOT             | 974595          | 47 MiB      | 47 MiB          | 0 B              |
| FINALIZED_BLOCKS_BY_SLOT            | 973159          | 128 GiB     | 128 GiB         | 0 B              |
| BLOB_SIDECAR_BY_SLOT_AND_BLOCK_ROOT_AND_BLOB_INDEX | 298362          | 77 GiB      | 77 GiB          | 0 B              |
| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |
| ESTIMATED TOTAL                     | 2246259         | 206 GiB     | 206 GiB         | 0 B              |
| ----------------------------------- | --------------- | ----------- | --------------- | ---------------- |

Memory consumption seems similar to what we currently see on canary-02.

Screenshot 2025-02-05 at 11 47 38 am

Fixed Issue(s)

Fixes #8939 and #9051

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

@gfukushima gfukushima closed this Feb 3, 2025
@gfukushima gfukushima reopened this Feb 4, 2025
@gfukushima gfukushima marked this pull request as draft February 4, 2025 10:11
@gfukushima gfukushima mentioned this pull request Feb 6, 2025
2 tasks
@gfukushima gfukushima marked this pull request as ready for review February 6, 2025 01:20
@gfukushima gfukushima merged commit 0fdf7be into Consensys:master Feb 6, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

JVM crash when restarting teku (24.10.3) running rocksDB
2 participants