Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: forward-backward message processor #3775

Merged
merged 25 commits into from
May 24, 2024
Merged

Conversation

daniel-savu
Copy link
Contributor

@daniel-savu daniel-savu commented May 14, 2024

Description

  • adds logic for iterating message nonces in the processor task in forward-backward fashion
  • removes all manual calls that were updating the next nonce to query. This is done automatically in the iterator struct now
  • stores the highest known message nonce in the db, which is used to initialize the iterator

Drive-by changes

  • Converts the concrete HyperlaneDb type stored in the processor to a trait obj, to enable mocking db responses

Related issues

Backward compatibility

Yes - if there is no db entry for the new prefix, the processor starts from nonce zero, so no migration is required

Testing

Unit testing + e2e

Copy link

changeset-bot bot commented May 14, 2024

⚠️ No Changeset found

Latest commit: 69a0859

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

codecov bot commented May 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.96%. Comparing base (f2dc925) to head (69a0859).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3775   +/-   ##
=======================================
  Coverage   63.96%   63.96%           
=======================================
  Files         124      124           
  Lines        1543     1543           
  Branches      167      167           
=======================================
  Hits          987      987           
  Misses        530      530           
  Partials       26       26           
Components Coverage Δ
core 84.61% <ø> (ø)
hooks 67.50% <ø> (ø)
isms 39.52% <ø> (ø)
token 65.41% <ø> (ø)
middlewares 77.05% <ø> (ø)

@daniel-savu daniel-savu marked this pull request as ready for review May 16, 2024 15:54
@daniel-savu daniel-savu requested a review from tkporter as a code owner May 16, 2024 15:54
Copy link
Collaborator

@tkporter tkporter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! some suggestions and the only thing I think certainly needs changing is the metric contention

rust/agents/relayer/src/msg/processor.rs Outdated Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Outdated Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Outdated Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Show resolved Hide resolved
rust/hyperlane-base/src/db/rocks/hyperlane_db.rs Outdated Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Outdated Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Outdated Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Show resolved Hide resolved
rust/agents/relayer/src/msg/processor.rs Show resolved Hide resolved
@daniel-savu daniel-savu enabled auto-merge (squash) May 24, 2024 17:55
@daniel-savu daniel-savu merged commit 214f503 into main May 24, 2024
38 checks passed
@daniel-savu daniel-savu deleted the dan/fwd-bckd-processors branch May 24, 2024 18:19
yorhodes pushed a commit that referenced this pull request May 26, 2024
### Description

- adds logic for iterating message nonces in the processor task in
forward-backward fashion
- removes all manual calls that were updating the next nonce to query.
This is done automatically in the iterator struct now
- stores the highest known message nonce in the db, which is used to
initialize the iterator

### Drive-by changes

- Converts the concrete HyperlaneDb type stored in the processor to a
trait obj, to enable mocking db responses

### Related issues

- Fixes #3796
- Fixes #3816

### Backward compatibility

Yes - if there is no db entry for the new prefix, the processor starts
from nonce zero, so no migration is required

### Testing

Unit testing + e2e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Debug seemingly missed events in message indexing Forward-Backward Message Processor
2 participants