Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use gcs for replay-verify and Make Replay-verify Resumable #10350

Merged
merged 2 commits into from
Oct 15, 2023
Merged

Use gcs for replay-verify and Make Replay-verify Resumable #10350

merged 2 commits into from
Oct 15, 2023

Conversation

areshand
Copy link
Contributor

@areshand areshand commented Oct 2, 2023 •
edited
Loading

Description

  1. use gcs instead of s3 for replay and verify
  2. add retry at partition for failed command to improve robustness

Test Plan

https://github.com/aptos-labs/aptos-core/actions/runs/6471317308

@areshand areshand requested review from a team as code owners October 2, 2023 20:56
@areshand areshand force-pushed the use_gcs_for_testnet branch from 6962419 to 24a6a88 Compare October 2, 2023 21:00
@areshand areshand force-pushed the use_gcs_for_testnet branch from 24a6a88 to 3c9e388 Compare October 2, 2023 22:57
@areshand areshand requested a review from msmouse October 2, 2023 23:23
@areshand areshand force-pushed the use_gcs_for_testnet branch 12 times, most recently from d25ff85 to 0e3076d Compare October 4, 2023 17:06
@areshand areshand requested a review from lightmark as a code owner October 4, 2023 17:06
@areshand areshand force-pushed the use_gcs_for_testnet branch 5 times, most recently from 39b9d62 to f2bddfa Compare October 5, 2023 23:21
@areshand areshand changed the title Use gcs for reading replay-verify Use gcs for replay-verify and Make Replay-verify Resumable Oct 6, 2023
@areshand areshand force-pushed the use_gcs_for_testnet branch 2 times, most recently from 23985df to 0793631 Compare October 10, 2023 17:54
msmouse
msmouse reviewed Oct 10, 2023
View reviewed changes
testsuite/replay_verify.py Outdated Show resolved Hide resolved
@areshand areshand force-pushed the use_gcs_for_testnet branch 2 times, most recently from 5f212ee to 44e4766 Compare October 11, 2023 22:59
msmouse
msmouse approved these changes Oct 12, 2023
View reviewed changes
storage/backup/backup-cli/src/coordinators/replay_verify.rs Outdated Show resolved Hide resolved
testsuite/replay_verify.py Outdated Show resolved Hide resolved
testsuite/replay_verify.py Outdated Show resolved Hide resolved
@areshand areshand force-pushed the use_gcs_for_testnet branch 9 times, most recently from e80d544 to df11a29 Compare October 13, 2023 21:30
@areshand areshand enabled auto-merge (rebase) October 13, 2023 22:00
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@areshand areshand force-pushed the use_gcs_for_testnet branch from df11a29 to 482f1c5 Compare October 14, 2023 02:13
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions
Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21

two traffics test: inner traffic : committed: 7855 txn/s, latency: 4992 ms, (p50: 4800 ms, p90: 6000 ms, p99: 11400 ms), latency samples: 3393360
two traffics test : committed: 100 txn/s, latency: 2264 ms, (p50: 2100 ms, p90: 2800 ms, p99: 4000 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.219, avg: 0.198", "QsPosToProposal: max: 0.194, avg: 0.155", "ConsensusProposalToOrdered: max: 0.635, avg: 0.594", "ConsensusOrderedToCommit: max: 0.553, avg: 0.520", "ConsensusProposalToCommit: max: 1.172, avg: 1.114"]
Max round gap was 1 [limit 4] at version 1660870. Max no progress secs was 4.782835 [limit 10] at version 1660870.
Test Ok

@github-actions
Copy link
Contributor

✅ Forge suite framework_upgrade success on aptos-node-v1.5.1 ==> 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21

Compatibility test results for aptos-node-v1.5.1 ==> 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21 (PR)
Upgrade the nodes to version: 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 4238 txn/s, latency: 4968 ms, (p50: 5000 ms, p90: 7800 ms, p99: 8700 ms), latency samples: 237380
5. check swarm health
Compatibility test for aptos-node-v1.5.1 ==> 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21 passed
Test Ok

@github-actions

This comment has been minimized.

Sign in to view
@github-actions
Copy link
Contributor

✅ Forge suite compat success on aptos-node-v1.6.2 ==> 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21

Compatibility test results for aptos-node-v1.6.2 ==> 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21 (PR)
1. Check liveness of validators at old version: aptos-node-v1.6.2
compatibility::simple-validator-upgrade::liveness-check : committed: 4663 txn/s, latency: 6604 ms, (p50: 6800 ms, p90: 9900 ms, p99: 10400 ms), latency samples: 181860
2. Upgrading first Validator to new version: 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1733 txn/s, latency: 16242 ms, (p50: 18900 ms, p90: 22100 ms, p99: 23600 ms), latency samples: 86680
3. Upgrading rest of first batch to new version: 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1288 txn/s, latency: 17030 ms, (p50: 19300 ms, p90: 22600 ms, p99: 46600 ms), latency samples: 91500
4. upgrading second batch to new version: 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3715 txn/s, latency: 8637 ms, (p50: 9900 ms, p90: 11900 ms, p99: 12300 ms), latency samples: 144920
5. check swarm health
Compatibility test for aptos-node-v1.6.2 ==> 482f1c5d8124e7062b4fc35d093f3d5b69cf1e21 passed
Test Ok

@areshand areshand merged commit 7c49f09 into main Oct 15, 2023
@areshand areshand deleted the use_gcs_for_testnet branch October 15, 2023 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@perryjrandall perryjrandall perryjrandall approved these changes

@msmouse msmouse msmouse approved these changes

@sionescu sionescu sionescu approved these changes

@lightmark lightmark Awaiting requested review from lightmark lightmark is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@areshand @perryjrandall @sionescu @msmouse