Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Correctly instrument spawned tokio tasks #5623

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

keefertaylor
Copy link

Description

This change contains a small fix to log output

Instrumenting a JoinHandle will not correctly instrument the task. Instead, we should instrument the task itself, which will correctly display metrics.

Drive-by changes

N/A

Related issues

N/A

Backward compatibility

N/A

Testing

Here's logging output for the MerkleTreeSyncer. Log format is set to compact in both. You'll note that in the before example, the instrumentation span is not logged to the console, while in the latter it does.

Before this change:

  2025-03-01T04:48:32.076177Z  INFO hyperlane_base::contract_sync::cursors::sequence_aware::backward: return: Ok(Some(21921811..=21923810))
    at hyperlane-base/src/contract_sync/cursors/sequence_aware/backward.rs:100
    in hyperlane_base::contract_sync::cursors::sequence_aware::backward::get_next_range with self: BackwardSequenceAwareSyncCursor { chunk_size: 1999, last_indexed_snapshot: LastIndexedSnapshot { sequence: Some(59947), at_block: 21923816 }, current_indexing_snapshot: Some(TargetSnapshot { sequence: 59946, at_block: 21923810 }), index_mode: Block, domain: HyperlaneDomain(ethereum (1)) }
    in hyperlane_base::contract_sync::fetch_logs_with_cursor with cursor: ForwardBackwardSequenceAwareSyncCursor { forward: ForwardSequenceAwareSyncCursor { chunk_size: 1999, last_indexed_snapshot: LastIndexedSnapshot { sequence: Some(61117), at_block: 21949711 }, current_indexing_snapshot: TargetSnapshot { sequence: 61118, at_block: 21949712 }, target_snapshot: None, index_mode: Block, domain: HyperlaneDomain(ethereum (1)) }, backward: BackwardSequenceAwareSyncCursor { chunk_size: 1999, last_indexed_snapshot: LastIndexedSnapshot { sequence: Some(59947), at_block: 21923816 }, current_indexing_snapshot: Some(TargetSnapshot { sequence: 59946, at_block: 21923810 }), index_mode: Block, domain: HyperlaneDomain(ethereum (1)) }, last_direction: Backward }, domain: "ethereum"
    in hyperlane_base::contract_sync::ContractSync with label: "merkle_tree_hook", domain: "ethereum"

After:

  2025-03-01T04:47:15.793466Z  INFO hyperlane_base::contract_sync::cursors::sequence_aware::backward: return: Ok(Some(21941709..=21943708))
    at hyperlane-base/src/contract_sync/cursors/sequence_aware/backward.rs:100
    in hyperlane_base::contract_sync::cursors::sequence_aware::backward::get_next_range with self: BackwardSequenceAwareSyncCursor { chunk_size: 1999, last_indexed_snapshot: LastIndexedSnapshot { sequence: Some(60846), at_block: 21943780 }, current_indexing_snapshot: Some(TargetSnapshot { sequence: 60845, at_block: 21943708 }), index_mode: Block, domain: HyperlaneDomain(ethereum (1)) }
    in hyperlane_base::contract_sync::fetch_logs_with_cursor with cursor: ForwardBackwardSequenceAwareSyncCursor { forward: ForwardSequenceAwareSyncCursor { chunk_size: 1999, last_indexed_snapshot: LastIndexedSnapshot { sequence: Some(61117), at_block: 21949705 }, current_indexing_snapshot: TargetSnapshot { sequence: 61118, at_block: 21949706 }, target_snapshot: None, index_mode: Block, domain: HyperlaneDomain(ethereum (1)) }, backward: BackwardSequenceAwareSyncCursor { chunk_size: 1999, last_indexed_snapshot: LastIndexedSnapshot { sequence: Some(60846), at_block: 21943780 }, current_indexing_snapshot: Some(TargetSnapshot { sequence: 60845, at_block: 21943708 }), index_mode: Block, domain: HyperlaneDomain(ethereum (1)) }, last_direction: Backward }, domain: "ethereum"
    in hyperlane_base::contract_sync::ContractSync with label: "merkle_tree_hook", domain: "ethereum"
    in validator::validator::MerkleTreeHookSyncer

Copy link

changeset-bot bot commented Mar 6, 2025

⚠️ No Changeset found

Latest commit: d49f5b5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@keefertaylor keefertaylor changed the title apply instrumentation Correctly instrument spawned tokio tasks Mar 6, 2025
@keefertaylor keefertaylor changed the title Correctly instrument spawned tokio tasks [WIP] Correctly instrument spawned tokio tasks Mar 6, 2025
async move {
server.run();
}
.instrument(info_span!("Scraper server")),
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just as FYI: I changed this from "Relayer" to "Scraper" (since we're in the scraper).

I think that's the correct instrumentation for this code, but lmk if you'd like it reverted.

@keefertaylor keefertaylor changed the title [WIP] Correctly instrument spawned tokio tasks Correctly instrument spawned tokio tasks Mar 6, 2025
@keefertaylor keefertaylor changed the title Correctly instrument spawned tokio tasks Fix: Correctly instrument spawned tokio tasks Mar 6, 2025
@keefertaylor keefertaylor changed the title Fix: Correctly instrument spawned tokio tasks fix: Correctly instrument spawned tokio tasks Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Review
Development

Successfully merging this pull request may close these issues.

1 participant