STR-180 Client state manager + related refactors

[delbonis]

Description

The central goal of this PR is to create the client state manager described in the ticket. Like my other recent PR, this also intends to also include some refactors trying to simplify CSM operation as described in PR #469 .

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature/Enhancement (non-breaking change which adds functionality or enhances an existing one)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactor
• New or updated tests
• Dependency Update

Notes to Reviewers

Checklist

• I have performed a self-review of my code.
• I have commented my code where necessary.
• I have updated the documentation if needed.
• My changes do not introduce new warnings.
• I have added tests that prove my changes are effective or that my feature works.
• New and existing tests pass with my changes.

Related Issues

[codecov]

Codecov Report

Attention: Patch coverage is 29.58580% with 476 lines in your changes missing coverage. Please review.

Project coverage is 54.65%. Comparing base (a0d413d) to head (f8baa16).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/storage/src/managers/client_state.rs	0.00%	95 Missing ⚠️
bin/strata-client/src/rpc_server.rs	0.00%	74 Missing ⚠️
crates/state/src/client_state.rs	43.93%	74 Missing ⚠️
...rates/consensus-logic/src/csm/client_transition.rs	45.91%	53 Missing ⚠️
crates/consensus-logic/src/csm/worker.rs	0.00%	53 Missing ⚠️
crates/storage/src/managers/sync_event.rs	0.00%	46 Missing ⚠️
crates/storage/src/lib.rs	0.00%	25 Missing ⚠️
crates/consensus-logic/src/fork_choice_manager.rs	0.00%	13 Missing ⚠️
bin/strata-client/src/helpers.rs	0.00%	11 Missing ⚠️
crates/state/src/operation.rs	41.66%	7 Missing ⚠️
... and 7 more

@@            Coverage Diff             @@
##             main     #636      +/-   ##
==========================================
- Coverage   54.84%   54.65%   -0.19%     
==========================================
  Files         313      314       +1     
  Lines       33526    33578      +52     
==========================================
- Hits        18386    18353      -33     
- Misses      15140    15225      +85     

Files with missing lines	Coverage Δ
crates/db/src/traits.rs	0.00% <ø> (ø)
crates/rocksdb-store/src/client_state/db.rs	100.00% <100.00%> (+2.33%)	⬆️
crates/rocksdb-store/src/lib.rs	0.00% <ø> (ø)
crates/rocksdb-store/src/sync_event/db.rs	96.83% <100.00%> (ø)
crates/storage/src/managers/l1.rs	25.28% <100.00%> (+25.28%)	⬆️
crates/test-utils/src/bitcoin.rs	89.47% <100.00%> (ø)
crates/test-utils/src/l2.rs	99.09% <100.00%> (ø)
crates/test-utils/src/lib.rs	87.50% <ø> (ø)
crates/evmexec/src/fork_choice_state.rs	0.00% <0.00%> (ø)
crates/rocksdb-store/src/l1/db.rs	97.65% <90.90%> (+0.36%)	⬆️
... and 15 more

... and 12 files with indirect coverage changes

[github-actions]

Commit: c115954

SP1 Performance Test Results

program	cycles	success
BTC_BLOCKSPACE	30,370,673	✅
EL_BLOCK	97,975	✅
CL_BLOCK	91,380	✅
L1_BATCH	30,420,921	✅
L2_BATCH	5,473	✅
CHECKPOINT	28,005	✅

[delbonis]

Alright this is pretty mature as-is but it somewhat makes me want to include sync events in here too so that we can really clean things up around this.

[delbonis]

Ah I just realized unit tests are broken but that's fine, this is feature complete now.

[bewakes]

Pretty good refactor. Some comments and questions though.

One question is that will this change where we are now storing whole Clientstate to db instead of incremental ClientStateWrites at every SyncEvent incur huge writes to database every sync event? Especially given that we have array of unaccepted l1 blocks inside the ClientState.
I think the storage cost might appear huge when a node has not yet synced up to the latest L2 state, but has read a lot of L1 blocks in which case the unaccepted_l1_blocks can grow very large in size. This actually happened while syncing a devnet fullnode.

This might probably get mitigated in future when we have smarter and faster syncing of L2Blocks. But until then, it seems like there's a good possibility of huge db writes.

[storopoli]

ACK 41ccb23
I agree with Bibek's recent comments

[delbonis]

Force merging to keep the ball rolling.