Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce global RPC class storage #499

Merged
merged 42 commits into from
Jun 28, 2024
Merged

Introduce global RPC class storage #499

merged 42 commits into from
Jun 28, 2024

Conversation

FabijanC
Copy link
Contributor

@FabijanC FabijanC commented Jun 18, 2024

Usage related changes

Benchmarking

  • Running ./scripts/benchmark/command_stat_test.py (500 mints):
  • comparing Devnet on main and Devnet on this PR's branch
  • Devnet binaries compiled with cargo build --release
  • sample size: 10
  • alternative hypothesis: two-sided

With --state-archive-capacity full

Time (s):   TtestResult(statistic=-2.102220212750287, pvalue=0.04987296605083779, df=18.0)
  - Mean (main vs this branch): 6.6 s vs 7.0 s
Memory (MB): TtestResult(statistic=853.6179541610356, pvalue=6.351731250129005e-43, df=18.0)
  - Mean (main vs this branch): 997 MB vs 34 MB

With no feature flags

Time (s):   TtestResult(statistic=-1.6918289270796756, pvalue=0.10791760227433438, df=18.0)
  - Mean (main vs this branch): 6.4 s vs 6.7 s
Memory (MB): TtestResult(statistic=7.890611697084889, pvalue=2.979935622929167e-07, df=18.0)
  - Mean (main vs this branch): 20 MB vs 14 MB

Conclusion

  • Time slightly worse in regular mode
  • Memory consumption reduced in regular and full-state-archive-capacity mode, significantly in the latter.

Development related changes

  • Add property to StarknetState
    • rpc_contract_classes: Arc<RwLock<CommittedClassStorage>>
    • For every class, we record when it was added.
  • Main reading logic is in crates/starknet-devnet-core/src/starknet/get_class_impls.rs
  • Add dependency: parking_lot
    • Using its RwLock because it doesn't have to be handled like std's or tokio's
  • Expand testing of state update and trace
    • blocks_on_demand_declarations fails on main
  • StateUpdate of blocks in block-on-demand mode was wrong:
    • Bug: always applying empty StateDiff in create_block.
    • Fixed by accumulating state diffs in a new property called pending_state_diff by relying on the newly added StateDiff::extend.
  • commit_with_diff renamed to commit_diff, no longer accepts the state_diff parameter.
  • Extract global contract class creation to default_global_contract_cache.
  • StateDiff::generate accepts newly declared classes instead of the whole class storage.
    • This means that the responsibility of extracting these classes has been moved to the caller.
  • assert_cairo1_classes_equal now working with references - removes unnecessary cloning.

Checklist:

  • Checked out the contribution guidelines
  • Applied formatting - ./scripts/format.sh
  • No linter errors - ./scripts/clippy_check.sh
  • No unused dependencies - ./scripts/check_unused_deps.sh
  • No spelling errors - ./scripts/check_spelling.sh
  • Performed code self-review
  • Rebased to the latest commit of the target branch (or merged it into my branch)
  • Updated the docs if needed - ./website/README.md
  • Linked the issues resolvable by this PR - linking info
  • Updated the tests if needed; all passing - execution info

@FabijanC FabijanC marked this pull request as ready for review June 20, 2024 10:57
@FabijanC FabijanC requested review from marioiordanov and mikiw June 20, 2024 12:41
@FabijanC FabijanC changed the title Common rpc class storage Introduce global RPC class storage Jun 20, 2024
Copy link
Contributor

@mikiw mikiw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would link it with #272

@FabijanC FabijanC merged commit f6ea9e6 into main Jun 28, 2024
@FabijanC FabijanC deleted the common-rpc-class-storage branch June 28, 2024 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory exhaustion
3 participants