Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: enhance account key update logic to batch transactions instead of loading all with sleep #105

Merged
merged 20 commits into from
Mar 6, 2024

Conversation

jeromy-cannon
Copy link
Contributor

@jeromy-cannon jeromy-cannon commented Feb 28, 2024

Description

This pull request changes the following:

  • enhance account key update logic to batch transactions instead of loading all with sleep
  • removed unused reference to @jest/test-sequencer
  • added logic to properly close WebSocketServer connections
  • removed multiple sleep commands
  • updated for check for Terminating Netty string
  • removed extra creation of NodeClient to simplify and increase performance
  • restructured mirror node call to better leverage listr2 subtasks for user experience
  • restructured update of account keys to better leverage listr2 subtasks
  • fixed bug was not reporting skips and rejections correctly in the account key updates loop

Related Issues

@jeromy-cannon jeromy-cannon self-assigned this Feb 28, 2024
Copy link
Contributor

github-actions bot commented Feb 28, 2024

Unit Test Results

 1 files  ±0  13 suites  ±0   20s ⏱️ ±0s
83 tests ±0  83 ✅ +2  0 💤 ±0  0 ❌  - 2 
90 runs  +1  90 ✅ +3  0 💤 ±0  0 ❌  - 2 

Results for commit 50965e0. ± Comparison against base commit 3d91617.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Feb 28, 2024

E2E Test Results

  1 files  ±  0    6 suites  ±0   13m 18s ⏱️ + 11m 49s
736 tests +  1  736 ✅ +  6  0 💤 ±0  0 ❌  - 5 
736 runs  +714  736 ✅ +719  0 💤 ±0  0 ❌  - 5 

Results for commit 50965e0. ± Comparison against base commit 3d91617.

This pull request removes 702 and adds 703 tests. Note that renamed tests count towards both.
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 10 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 10 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 100 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 100 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 1000 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 1000 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 11 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 11 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 12 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 12 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 13 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 13 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 14 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 14 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 15 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 15 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 16 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 16 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 17 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] [beforeAll - ERROR]: only genesis account should have genesis key for all special accounts special account 17 should not have genesis key
…
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] checkNetworkNodeProxyUp should succeed ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] checkNetworkNodeProxyUp should succeed
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 10 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 10 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 100 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 100 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 1000 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 1000 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 11 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 11 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 12 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 12 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 13 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 13 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 14 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 14 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 15 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 15 should not have genesis key
NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 16 should not have genesis key ‑ NodeCommand node start should succeed [release v0.42.5, keyFormat: pfx] only genesis account should have genesis key for all special accounts special account 16 should not have genesis key
…

♻️ This comment has been updated with latest results.

Copy link

codacy-production bot commented Feb 28, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
Report missing for 3d916171 87.82%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (3d91617) Report Missing Report Missing Report Missing
Head commit (50965e0) 2220 1444 65.05%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#105) 156 137 87.82%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

You may notice some variations in coverage metrics with the latest Coverage engine update. For more details, visit the documentation

Footnotes

  1. Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

Copy link

codecov bot commented Feb 28, 2024

Codecov Report

Attention: Patch coverage is 88.70968% with 21 lines in your changes are missing coverage. Please review.

Project coverage is 64.65%. Comparing base (8fd976b) to head (50965e0).
Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #105      +/-   ##
==========================================
+ Coverage   63.73%   64.65%   +0.91%     
==========================================
  Files          27       27              
  Lines        2570     2634      +64     
  Branches      422      432      +10     
==========================================
+ Hits         1638     1703      +65     
+ Misses        932      931       -1     
Files Coverage Δ
src/commands/account.mjs 67.83% <100.00%> (-2.50%) ⬇️
src/core/constants.mjs 92.06% <100.00%> (ø)
src/core/k8.mjs 84.32% <100.00%> (+5.32%) ⬆️
src/core/account_manager.mjs 85.65% <89.77%> (+0.23%) ⬆️
src/commands/node.mjs 55.50% <81.81%> (+1.76%) ⬆️

... and 1 file with indirect coverage changes

Impacted file tree graph

@jeromy-cannon jeromy-cannon force-pushed the sleep-performance-issue branch from 6232bd8 to b493733 Compare February 29, 2024 18:37
@jeromy-cannon jeromy-cannon marked this pull request as ready for review February 29, 2024 18:39
@jeromy-cannon jeromy-cannon requested review from a team and leninmehedy as code owners February 29, 2024 18:39
@jeromy-cannon jeromy-cannon reopened this Feb 29, 2024
src/commands/node.mjs Outdated Show resolved Hide resolved
src/commands/node.mjs Outdated Show resolved Hide resolved
src/commands/node.mjs Outdated Show resolved Hide resolved
src/core/account_manager.mjs Outdated Show resolved Hide resolved
@leninmehedy leninmehedy self-requested a review March 4, 2024 03:37
leninmehedy
leninmehedy previously approved these changes Mar 4, 2024
Copy link
Member

@leninmehedy leninmehedy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address my comments and merge when you are ready.

src/commands/node.mjs Outdated Show resolved Hide resolved
src/core/k8.mjs Show resolved Hide resolved
src/commands/node.mjs Outdated Show resolved Hide resolved
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
Signed-off-by: Jeromy Cannon <[email protected]>
@jeromy-cannon jeromy-cannon force-pushed the sleep-performance-issue branch from f73da30 to 0ac9da6 Compare March 6, 2024 11:18
Signed-off-by: Jeromy Cannon <[email protected]>
@jeromy-cannon jeromy-cannon marked this pull request as ready for review March 6, 2024 11:23
@leninmehedy leninmehedy self-requested a review March 6, 2024 12:26
@jeromy-cannon jeromy-cannon merged commit e4bd1ef into main Mar 6, 2024
17 checks passed
@jeromy-cannon jeromy-cannon deleted the sleep-performance-issue branch March 6, 2024 13:26
swirlds-automation added a commit that referenced this pull request Mar 8, 2024
## [0.22.0](v0.21.2...v0.22.0) (2024-03-08)

### Features

* **cli:** allow user to set jvm options for Hedera nodes ([#84](#84)) ([22ff35d](22ff35d))
* download helm binary based on os and platform architecture ([#128](#128)) ([e64e9a2](e64e9a2))

### Bug Fixes

* do checksum check after downloading platform zip file ([#125](#125)) ([3d91617](3d91617))
* updated incorrect log messages ([#132](#132)) ([d890974](d890974))

### Performance Improvements

* enhance account key update logic to batch transactions instead of loading all with sleep ([#105](#105)) ([e4bd1ef](e4bd1ef))
@swirlds-automation
Copy link
Contributor

🎉 This PR is included in version 0.22.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants