Skip to content
This repository has been archived by the owner on Aug 28, 2024. It is now read-only.

Commit

Permalink
feat(api): Retry read_value (matter-labs#2352)
Browse files Browse the repository at this point in the history
## What ❔

Retries calling `read_value` when it fails with statement_timeout.

## Why ❔

Eliminate sporadic failures.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
  • Loading branch information
perekopskiy authored and irnb committed Jul 12, 2024
1 parent 1b1a738 commit 6a66a9f
Show file tree
Hide file tree
Showing 4 changed files with 36 additions and 4 deletions.
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions core/lib/state/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ tracing.workspace = true
itertools.workspace = true
chrono.workspace = true
once_cell.workspace = true
backon.workspace = true

[dev-dependencies]
assert_matches.workspace = true
Expand Down
25 changes: 21 additions & 4 deletions core/lib/state/src/postgres/mod.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
use std::{
mem,
sync::{Arc, RwLock},
time::Duration,
};

use anyhow::Context as _;
use backon::{BlockingRetryable, ConstantBuilder};
use tokio::{
runtime::Handle,
sync::{
Expand Down Expand Up @@ -489,11 +491,26 @@ impl ReadStorage for PostgresStorage<'_> {
values_cache.and_then(|cache| cache.get(self.l2_block_number, hashed_key));

let value = cached_value.unwrap_or_else(|| {
const RETRY_INTERVAL: Duration = Duration::from_millis(500);
const MAX_TRIES: usize = 20;

let mut dal = self.connection.storage_web3_dal();
let value = self
.rt_handle
.block_on(dal.get_historical_value_unchecked(hashed_key, self.l2_block_number))
.expect("Failed executing `read_value`");
let value = (|| {
self.rt_handle
.block_on(dal.get_historical_value_unchecked(hashed_key, self.l2_block_number))
})
.retry(
&ConstantBuilder::default()
.with_delay(RETRY_INTERVAL)
.with_max_times(MAX_TRIES),
)
.when(|e| {
e.inner()
.as_database_error()
.is_some_and(|e| e.message() == "canceling statement due to statement timeout")
})
.call()
.expect("Failed executing `read_value`");
if let Some(cache) = self.values_cache() {
cache.insert(self.l2_block_number, hashed_key, value);
}
Expand Down
13 changes: 13 additions & 0 deletions prover/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 6a66a9f

Please sign in to comment.