Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: synchronize keys between validator client and external signer #6672

Merged
merged 17 commits into from
Apr 19, 2024
Merged

feat: synchronize keys between validator client and external signer #6672

merged 17 commits into from
Apr 19, 2024

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Apr 15, 2024

Motivation

Description

Adds new service that is responsible for keeping the keys managed by the connected external signer and the validator client in sync by adding newly discovered keys and removing no longer present keys on external signer from the validator store.

Only enabled if --externalSigner.url and --externalSigner.fetch is set.

The interval can be configured via --externalSigner.fetchInterval flag and is set to once per epoch by default.

(see referenced issue for more details)

Closes #6624

@codecov Codecov
Copy link

codecov bot commented Apr 15, 2024 •
edited
Loading

Codecov Report

Merging #6672 (fa78e38) into unstable (eb84b49) will increase coverage by 0.07%.
Report is 1 commits behind head on unstable.
The diff coverage is 94.84%.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #6672      +/-   ##
============================================
+ Coverage     61.80%   61.87%   +0.07%     
============================================
  Files           556      557       +1     
  Lines         59103    59200      +97     
  Branches       1898     1915      +17     
============================================
+ Hits          36526    36628     +102     
+ Misses        22534    22529       -5     
  Partials         43       43

@nflaig nflaig changed the title feat: synchronize keys between validator client and remote signer feat: synchronize keys between validator client and external signer Apr 15, 2024
@nflaig nflaig force-pushed the nflaig/external-signer-sync branch from e3b5466 to d63adce Compare April 15, 2024 22:53
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 15, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: a114b51 Previous: f5148b2 Ratio
forkChoice updateHead vc 600000 bc 64 eq 300000 16.092 ms/op 54.969 ms/op 0.29
phase0 processBlock - 250000 vs - 7PWei normalcase 2.2576 ms/op 6.9290 ms/op 0.33
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 42.726 us/op 148.47 us/op 0.29
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.8492 ms/op 12.393 ms/op 0.31
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.7169 ms/op 19.883 ms/op 0.29
Full benchmark results
Benchmark suite Current: a114b51 Previous: f5148b2 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 685.97 us/op 566.38 us/op 1.21
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 84.028 us/op 105.04 us/op 0.80
BLS verify - blst-native 1.2810 ms/op 1.3367 ms/op 0.96
BLS verifyMultipleSignatures 3 - blst-native 2.6963 ms/op 2.8140 ms/op 0.96
BLS verifyMultipleSignatures 8 - blst-native 5.9250 ms/op 6.1794 ms/op 0.96
BLS verifyMultipleSignatures 32 - blst-native 21.675 ms/op 22.211 ms/op 0.98
BLS verifyMultipleSignatures 64 - blst-native 42.634 ms/op 44.382 ms/op 0.96
BLS verifyMultipleSignatures 128 - blst-native 84.387 ms/op 86.316 ms/op 0.98
BLS deserializing 10000 signatures 870.50 ms/op 906.25 ms/op 0.96
BLS deserializing 100000 signatures 9.2393 s/op 9.2687 s/op 1.00
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3215 ms/op 1.3740 ms/op 0.96
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.6564 ms/op 1.5319 ms/op 1.08
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.6451 ms/op 2.4246 ms/op 1.09
BLS verifyMultipleSignatures - same message - 64 - blst-native 4.2170 ms/op 3.5064 ms/op 1.20
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.7552 ms/op 5.8519 ms/op 1.15
BLS aggregatePubkeys 32 - blst-native 26.055 us/op 25.941 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 104.55 us/op 101.38 us/op 1.03
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 65.934 ms/op 63.597 ms/op 1.04
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 66.568 ms/op 80.511 ms/op 0.83
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 36.616 ms/op 49.355 ms/op 0.74
getSlashingsAndExits - default max 146.74 us/op 221.03 us/op 0.66
getSlashingsAndExits - 2k 446.27 us/op 472.31 us/op 0.94
proposeBlockBody type=full, size=empty 5.5040 ms/op 6.8600 ms/op 0.80
isKnown best case - 1 super set check 356.00 ns/op 466.00 ns/op 0.76
isKnown normal case - 2 super set checks 305.00 ns/op 524.00 ns/op 0.58
isKnown worse case - 16 super set checks 300.00 ns/op 439.00 ns/op 0.68
InMemoryCheckpointStateCache - add get delete 5.9320 us/op 7.0400 us/op 0.84
validate api signedAggregateAndProof - struct 3.0035 ms/op 2.9167 ms/op 1.03
validate gossip signedAggregateAndProof - struct 2.7405 ms/op 2.8513 ms/op 0.96
validate gossip attestation - vc 640000 1.3411 ms/op 1.4036 ms/op 0.96
batch validate gossip attestation - vc 640000 - chunk 32 163.10 us/op 176.99 us/op 0.92
batch validate gossip attestation - vc 640000 - chunk 64 145.06 us/op 159.14 us/op 0.91
batch validate gossip attestation - vc 640000 - chunk 128 134.57 us/op 154.55 us/op 0.87
batch validate gossip attestation - vc 640000 - chunk 256 130.28 us/op 150.83 us/op 0.86
pickEth1Vote - no votes 1.2496 ms/op 1.5095 ms/op 0.83
pickEth1Vote - max votes 12.539 ms/op 11.698 ms/op 1.07
pickEth1Vote - Eth1Data hashTreeRoot value x2048 16.089 ms/op 21.620 ms/op 0.74
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.608 ms/op 33.432 ms/op 0.65
pickEth1Vote - Eth1Data fastSerialize value x2048 588.84 us/op 828.77 us/op 0.71
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.2052 ms/op 6.4259 ms/op 0.65
bytes32 toHexString 513.00 ns/op 819.00 ns/op 0.63
bytes32 Buffer.toString(hex) 301.00 ns/op 315.00 ns/op 0.96
bytes32 Buffer.toString(hex) from Uint8Array 425.00 ns/op 614.00 ns/op 0.69
bytes32 Buffer.toString(hex) + 0x 304.00 ns/op 310.00 ns/op 0.98
Object access 1 prop 0.16800 ns/op 0.24400 ns/op 0.69
Map access 1 prop 0.14900 ns/op 0.16400 ns/op 0.91
Object get x1000 7.7660 ns/op 8.1800 ns/op 0.95
Map get x1000 0.78500 ns/op 0.95900 ns/op 0.82
Object set x1000 54.057 ns/op 84.287 ns/op 0.64
Map set x1000 44.423 ns/op 61.462 ns/op 0.72
Return object 10000 times 0.24510 ns/op 0.28510 ns/op 0.86
Throw Error 10000 times 3.8498 us/op 4.2165 us/op 0.91
fastMsgIdFn sha256 / 200 bytes 3.3810 us/op 3.7510 us/op 0.90
fastMsgIdFn h32 xxhash / 200 bytes 336.00 ns/op 413.00 ns/op 0.81
fastMsgIdFn h64 xxhash / 200 bytes 375.00 ns/op 435.00 ns/op 0.86
fastMsgIdFn sha256 / 1000 bytes 11.131 us/op 12.280 us/op 0.91
fastMsgIdFn h32 xxhash / 1000 bytes 421.00 ns/op 531.00 ns/op 0.79
fastMsgIdFn h64 xxhash / 1000 bytes 424.00 ns/op 547.00 ns/op 0.78
fastMsgIdFn sha256 / 10000 bytes 100.35 us/op 114.83 us/op 0.87
fastMsgIdFn h32 xxhash / 10000 bytes 1.8970 us/op 2.1830 us/op 0.87
fastMsgIdFn h64 xxhash / 10000 bytes 1.3090 us/op 1.5450 us/op 0.85
send data - 1000 256B messages 17.114 ms/op 27.359 ms/op 0.63
send data - 1000 512B messages 23.396 ms/op 32.180 ms/op 0.73
send data - 1000 1024B messages 39.781 ms/op 48.050 ms/op 0.83
send data - 1000 1200B messages 38.592 ms/op 45.378 ms/op 0.85
send data - 1000 2048B messages 49.847 ms/op 62.919 ms/op 0.79
send data - 1000 4096B messages 42.182 ms/op 56.302 ms/op 0.75
send data - 1000 16384B messages 113.97 ms/op 150.72 ms/op 0.76
send data - 1000 65536B messages 434.95 ms/op 573.16 ms/op 0.76
enrSubnets - fastDeserialize 64 bits 1.2300 us/op 1.7930 us/op 0.69
enrSubnets - ssz BitVector 64 bits 438.00 ns/op 592.00 ns/op 0.74
enrSubnets - fastDeserialize 4 bits 169.00 ns/op 252.00 ns/op 0.67
enrSubnets - ssz BitVector 4 bits 432.00 ns/op 570.00 ns/op 0.76
prioritizePeers score -10:0 att 32-0.1 sync 2-0 205.26 us/op 238.81 us/op 0.86
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 262.70 us/op 369.90 us/op 0.71
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 332.85 us/op 453.47 us/op 0.73
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 561.37 us/op 717.26 us/op 0.78
prioritizePeers score 0:0 att 64-1 sync 4-1 697.59 us/op 900.22 us/op 0.77
array of 16000 items push then shift 1.5576 us/op 1.7660 us/op 0.88
LinkedList of 16000 items push then shift 8.7680 ns/op 11.094 ns/op 0.79
array of 16000 items push then pop 57.044 ns/op 129.29 ns/op 0.44
LinkedList of 16000 items push then pop 8.7140 ns/op 9.7640 ns/op 0.89
array of 24000 items push then shift 2.3909 us/op 2.6920 us/op 0.89
LinkedList of 24000 items push then shift 8.7340 ns/op 11.155 ns/op 0.78
array of 24000 items push then pop 110.50 ns/op 188.23 ns/op 0.59
LinkedList of 24000 items push then pop 8.4970 ns/op 9.8030 ns/op 0.87
intersect bitArray bitLen 8 5.6160 ns/op 6.7700 ns/op 0.83
intersect array and set length 8 62.665 ns/op 114.83 ns/op 0.55
intersect bitArray bitLen 128 34.993 ns/op 42.991 ns/op 0.81
intersect array and set length 128 887.17 ns/op 1.2180 us/op 0.73
bitArray.getTrueBitIndexes() bitLen 128 1.3870 us/op 2.2200 us/op 0.62
bitArray.getTrueBitIndexes() bitLen 248 2.5740 us/op 3.6100 us/op 0.71
bitArray.getTrueBitIndexes() bitLen 512 4.7440 us/op 8.3180 us/op 0.57
Buffer.concat 32 items 948.00 ns/op 1.2730 us/op 0.74
Uint8Array.set 32 items 1.5820 us/op 2.3180 us/op 0.68
Set add up to 64 items then delete first 4.3029 us/op 5.5707 us/op 0.77
OrderedSet add up to 64 items then delete first 5.3714 us/op 7.6495 us/op 0.70
Set add up to 64 items then delete last 4.6095 us/op 6.0255 us/op 0.77
OrderedSet add up to 64 items then delete last 5.6841 us/op 7.7540 us/op 0.73
Set add up to 64 items then delete middle 4.5408 us/op 5.7392 us/op 0.79
OrderedSet add up to 64 items then delete middle 7.1544 us/op 9.2501 us/op 0.77
Set add up to 128 items then delete first 9.2415 us/op 12.065 us/op 0.77
OrderedSet add up to 128 items then delete first 12.267 us/op 16.048 us/op 0.76
Set add up to 128 items then delete last 9.4415 us/op 12.202 us/op 0.77
OrderedSet add up to 128 items then delete last 12.110 us/op 15.219 us/op 0.80
Set add up to 128 items then delete middle 9.7800 us/op 11.376 us/op 0.86
OrderedSet add up to 128 items then delete middle 17.965 us/op 21.905 us/op 0.82
Set add up to 256 items then delete first 19.576 us/op 23.504 us/op 0.83
OrderedSet add up to 256 items then delete first 25.598 us/op 34.963 us/op 0.73
Set add up to 256 items then delete last 19.184 us/op 22.940 us/op 0.84
OrderedSet add up to 256 items then delete last 27.284 us/op 30.891 us/op 0.88
Set add up to 256 items then delete middle 18.356 us/op 22.976 us/op 0.80
OrderedSet add up to 256 items then delete middle 45.292 us/op 55.380 us/op 0.82
transfer serialized Status (84 B) 1.6860 us/op 2.2720 us/op 0.74
copy serialized Status (84 B) 1.2860 us/op 1.6690 us/op 0.77
transfer serialized SignedVoluntaryExit (112 B) 1.8050 us/op 2.3460 us/op 0.77
copy serialized SignedVoluntaryExit (112 B) 1.3710 us/op 1.7720 us/op 0.77
transfer serialized ProposerSlashing (416 B) 2.0280 us/op 3.6030 us/op 0.56
copy serialized ProposerSlashing (416 B) 1.7910 us/op 3.5310 us/op 0.51
transfer serialized Attestation (485 B) 1.9230 us/op 3.7960 us/op 0.51
copy serialized Attestation (485 B) 1.8410 us/op 3.0380 us/op 0.61
transfer serialized AttesterSlashing (33232 B) 2.3390 us/op 3.7480 us/op 0.62
copy serialized AttesterSlashing (33232 B) 5.5870 us/op 20.864 us/op 0.27
transfer serialized Small SignedBeaconBlock (128000 B) 2.5610 us/op 5.1360 us/op 0.50
copy serialized Small SignedBeaconBlock (128000 B) 14.532 us/op 36.207 us/op 0.40
transfer serialized Avg SignedBeaconBlock (200000 B) 2.6420 us/op 5.1810 us/op 0.51
copy serialized Avg SignedBeaconBlock (200000 B) 22.755 us/op 60.548 us/op 0.38
transfer serialized BlobsSidecar (524380 B) 2.9270 us/op 6.0200 us/op 0.49
copy serialized BlobsSidecar (524380 B) 126.78 us/op 159.42 us/op 0.80
transfer serialized Big SignedBeaconBlock (1000000 B) 2.8040 us/op 5.8760 us/op 0.48
copy serialized Big SignedBeaconBlock (1000000 B) 314.70 us/op 307.28 us/op 1.02
pass gossip attestations to forkchoice per slot 3.6464 ms/op 5.0232 ms/op 0.73
forkChoice updateHead vc 100000 bc 64 eq 0 672.12 us/op 714.60 us/op 0.94
forkChoice updateHead vc 600000 bc 64 eq 0 4.0784 ms/op 6.8959 ms/op 0.59
forkChoice updateHead vc 1000000 bc 64 eq 0 7.2025 ms/op 7.9774 ms/op 0.90
forkChoice updateHead vc 600000 bc 320 eq 0 4.1416 ms/op 4.6475 ms/op 0.89
forkChoice updateHead vc 600000 bc 1200 eq 0 4.2371 ms/op 4.9942 ms/op 0.85
forkChoice updateHead vc 600000 bc 7200 eq 0 5.7486 ms/op 6.6299 ms/op 0.87
forkChoice updateHead vc 600000 bc 64 eq 1000 10.737 ms/op 12.220 ms/op 0.88
forkChoice updateHead vc 600000 bc 64 eq 10000 11.880 ms/op 14.463 ms/op 0.82
forkChoice updateHead vc 600000 bc 64 eq 300000 16.092 ms/op 54.969 ms/op 0.29
computeDeltas 500000 validators 300 proto nodes 6.6589 ms/op 7.6026 ms/op 0.88
computeDeltas 500000 validators 1200 proto nodes 6.4582 ms/op 7.3888 ms/op 0.87
computeDeltas 500000 validators 7200 proto nodes 6.3729 ms/op 6.9354 ms/op 0.92
computeDeltas 750000 validators 300 proto nodes 9.7661 ms/op 10.657 ms/op 0.92
computeDeltas 750000 validators 1200 proto nodes 9.5473 ms/op 10.698 ms/op 0.89
computeDeltas 750000 validators 7200 proto nodes 9.5503 ms/op 10.418 ms/op 0.92
computeDeltas 1400000 validators 300 proto nodes 18.236 ms/op 21.535 ms/op 0.85
computeDeltas 1400000 validators 1200 proto nodes 18.065 ms/op 20.608 ms/op 0.88
computeDeltas 1400000 validators 7200 proto nodes 18.243 ms/op 20.142 ms/op 0.91
computeDeltas 2100000 validators 300 proto nodes 27.646 ms/op 30.415 ms/op 0.91
computeDeltas 2100000 validators 1200 proto nodes 28.246 ms/op 31.930 ms/op 0.88
computeDeltas 2100000 validators 7200 proto nodes 28.203 ms/op 33.406 ms/op 0.84
altair processAttestation - 250000 vs - 7PWei normalcase 4.1760 ms/op 7.9320 ms/op 0.53
altair processAttestation - 250000 vs - 7PWei worstcase 5.0600 ms/op 10.812 ms/op 0.47
altair processAttestation - setStatus - 1/6 committees join 293.13 us/op 412.80 us/op 0.71
altair processAttestation - setStatus - 1/3 committees join 525.03 us/op 812.36 us/op 0.65
altair processAttestation - setStatus - 1/2 committees join 717.87 us/op 1.0978 ms/op 0.65
altair processAttestation - setStatus - 2/3 committees join 828.31 us/op 1.3693 ms/op 0.60
altair processAttestation - setStatus - 4/5 committees join 1.1617 ms/op 1.7185 ms/op 0.68
altair processAttestation - setStatus - 100% committees join 1.3516 ms/op 2.0279 ms/op 0.67
altair processBlock - 250000 vs - 7PWei normalcase 9.6363 ms/op 19.435 ms/op 0.50
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.883 ms/op 89.449 ms/op 0.40
altair processBlock - 250000 vs - 7PWei worstcase 44.377 ms/op 77.257 ms/op 0.57
altair processBlock - 250000 vs - 7PWei worstcase hashState 87.782 ms/op 196.50 ms/op 0.45
phase0 processBlock - 250000 vs - 7PWei normalcase 2.2576 ms/op 6.9290 ms/op 0.33
phase0 processBlock - 250000 vs - 7PWei worstcase 32.379 ms/op 58.098 ms/op 0.56
altair processEth1Data - 250000 vs - 7PWei normalcase 455.62 us/op 1.2087 ms/op 0.38
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 13.578 us/op 35.117 us/op 0.39
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 42.726 us/op 148.47 us/op 0.29
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 29.456 us/op 59.735 us/op 0.49
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 15.087 us/op 40.492 us/op 0.37
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 180.94 us/op 321.66 us/op 0.56
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.2654 ms/op 2.7972 ms/op 0.45
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.5583 ms/op 3.9335 ms/op 0.40
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.0166 ms/op 3.3347 ms/op 0.60
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.8492 ms/op 12.393 ms/op 0.31
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.3684 ms/op 6.0161 ms/op 0.39
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.7169 ms/op 19.883 ms/op 0.29
Tree 40 250000 create 353.88 ms/op 1.6332 s/op 0.22
Tree 40 250000 get(125000) 201.41 ns/op 387.26 ns/op 0.52
Tree 40 250000 set(125000) 1.0300 us/op 5.8199 us/op 0.18
Tree 40 250000 toArray() 19.404 ms/op 59.792 ms/op 0.32
Tree 40 250000 iterate all - toArray() + loop 20.910 ms/op 65.092 ms/op 0.32
Tree 40 250000 iterate all - get(i) 70.963 ms/op 163.02 ms/op 0.44
MutableVector 250000 create 17.015 ms/op 27.365 ms/op 0.62
MutableVector 250000 get(125000) 6.5160 ns/op 11.670 ns/op 0.56
MutableVector 250000 set(125000) 284.21 ns/op 1.0745 us/op 0.26
MutableVector 250000 toArray() 2.9520 ms/op 7.7227 ms/op 0.38
MutableVector 250000 iterate all - toArray() + loop 3.0695 ms/op 8.2208 ms/op 0.37
MutableVector 250000 iterate all - get(i) 1.5965 ms/op 3.0926 ms/op 0.52
Array 250000 create 2.6445 ms/op 6.8597 ms/op 0.39
Array 250000 clone - spread 1.2864 ms/op 7.3420 ms/op 0.18
Array 250000 get(125000) 1.0920 ns/op 4.0960 ns/op 0.27
Array 250000 set(125000) 4.1460 ns/op 7.0830 ns/op 0.59
Array 250000 iterate all - loop 165.37 us/op 218.46 us/op 0.76
effectiveBalanceIncrements clone Uint8Array 300000 31.678 us/op 121.64 us/op 0.26
effectiveBalanceIncrements clone MutableVector 300000 405.00 ns/op 2.6890 us/op 0.15
effectiveBalanceIncrements rw all Uint8Array 300000 205.97 us/op 310.20 us/op 0.66
effectiveBalanceIncrements rw all MutableVector 300000 102.82 ms/op 323.01 ms/op 0.32
phase0 afterProcessEpoch - 250000 vs - 7PWei 114.49 ms/op 176.60 ms/op 0.65
phase0 beforeProcessEpoch - 250000 vs - 7PWei 55.867 ms/op 134.76 ms/op 0.41
altair processEpoch - mainnet_e81889 471.99 ms/op 1.1120 s/op 0.42
mainnet_e81889 - altair beforeProcessEpoch 83.276 ms/op 227.70 ms/op 0.37
mainnet_e81889 - altair processJustificationAndFinalization 15.602 us/op 45.629 us/op 0.34
mainnet_e81889 - altair processInactivityUpdates 6.1051 ms/op 19.414 ms/op 0.31
mainnet_e81889 - altair processRewardsAndPenalties 54.814 ms/op 96.297 ms/op 0.57
mainnet_e81889 - altair processRegistryUpdates 2.3650 us/op 10.440 us/op 0.23
mainnet_e81889 - altair processSlashings 439.00 ns/op 1.8030 us/op 0.24
mainnet_e81889 - altair processEth1DataReset 512.00 ns/op 2.2380 us/op 0.23
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.3454 ms/op 2.7041 ms/op 0.50
mainnet_e81889 - altair processSlashingsReset 3.5040 us/op 11.812 us/op 0.30
mainnet_e81889 - altair processRandaoMixesReset 8.2820 us/op 15.698 us/op 0.53
mainnet_e81889 - altair processHistoricalRootsUpdate 698.00 ns/op 3.1170 us/op 0.22
mainnet_e81889 - altair processParticipationFlagUpdates 3.0970 us/op 7.2460 us/op 0.43
mainnet_e81889 - altair processSyncCommitteeUpdates 723.00 ns/op 2.7940 us/op 0.26
mainnet_e81889 - altair afterProcessEpoch 119.82 ms/op 183.83 ms/op 0.65
capella processEpoch - mainnet_e217614 2.4166 s/op 4.8386 s/op 0.50
mainnet_e217614 - capella beforeProcessEpoch 538.06 ms/op 1.0608 s/op 0.51
mainnet_e217614 - capella processJustificationAndFinalization 15.002 us/op 45.954 us/op 0.33
mainnet_e217614 - capella processInactivityUpdates 19.713 ms/op 37.480 ms/op 0.53
mainnet_e217614 - capella processRewardsAndPenalties 558.39 ms/op 984.94 ms/op 0.57
mainnet_e217614 - capella processRegistryUpdates 24.468 us/op 50.644 us/op 0.48
mainnet_e217614 - capella processSlashings 1.0340 us/op 1.7950 us/op 0.58
mainnet_e217614 - capella processEth1DataReset 467.00 ns/op 2.1750 us/op 0.21
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.3283 ms/op 22.927 ms/op 0.19
mainnet_e217614 - capella processSlashingsReset 3.3840 us/op 10.773 us/op 0.31
mainnet_e217614 - capella processRandaoMixesReset 4.7110 us/op 13.542 us/op 0.35
mainnet_e217614 - capella processHistoricalRootsUpdate 552.00 ns/op 2.9160 us/op 0.19
mainnet_e217614 - capella processParticipationFlagUpdates 1.4710 us/op 5.2580 us/op 0.28
mainnet_e217614 - capella afterProcessEpoch 302.98 ms/op 411.99 ms/op 0.74
phase0 processEpoch - mainnet_e58758 462.04 ms/op 681.65 ms/op 0.68
mainnet_e58758 - phase0 beforeProcessEpoch 152.79 ms/op 194.96 ms/op 0.78
mainnet_e58758 - phase0 processJustificationAndFinalization 15.623 us/op 29.893 us/op 0.52
mainnet_e58758 - phase0 processRewardsAndPenalties 65.962 ms/op 72.289 ms/op 0.91
mainnet_e58758 - phase0 processRegistryUpdates 9.5790 us/op 16.124 us/op 0.59
mainnet_e58758 - phase0 processSlashings 640.00 ns/op 1.0100 us/op 0.63
mainnet_e58758 - phase0 processEth1DataReset 419.00 ns/op 622.00 ns/op 0.67
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.1464 ms/op 1.2906 ms/op 0.89
mainnet_e58758 - phase0 processSlashingsReset 3.0210 us/op 3.7600 us/op 0.80
mainnet_e58758 - phase0 processRandaoMixesReset 5.5950 us/op 6.3060 us/op 0.89
mainnet_e58758 - phase0 processHistoricalRootsUpdate 665.00 ns/op 588.00 ns/op 1.13
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.1970 us/op 4.6940 us/op 1.11
mainnet_e58758 - phase0 afterProcessEpoch 102.17 ms/op 106.40 ms/op 0.96
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.4447 ms/op 1.4257 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4674 ms/op 2.1938 ms/op 0.67
altair processInactivityUpdates - 250000 normalcase 30.665 ms/op 38.465 ms/op 0.80
altair processInactivityUpdates - 250000 worstcase 26.973 ms/op 31.807 ms/op 0.85
phase0 processRegistryUpdates - 250000 normalcase 11.772 us/op 20.381 us/op 0.58
phase0 processRegistryUpdates - 250000 badcase_full_deposits 394.61 us/op 574.55 us/op 0.69
phase0 processRegistryUpdates - 250000 worstcase 0.5 140.63 ms/op 234.79 ms/op 0.60
altair processRewardsAndPenalties - 250000 normalcase 55.998 ms/op 65.764 ms/op 0.85
altair processRewardsAndPenalties - 250000 worstcase 70.621 ms/op 103.72 ms/op 0.68
phase0 getAttestationDeltas - 250000 normalcase 11.991 ms/op 12.271 ms/op 0.98
phase0 getAttestationDeltas - 250000 worstcase 9.1325 ms/op 13.195 ms/op 0.69
phase0 processSlashings - 250000 worstcase 83.260 us/op 134.19 us/op 0.62
altair processSyncCommitteeUpdates - 250000 161.84 ms/op 189.30 ms/op 0.85
BeaconState.hashTreeRoot - No change 335.00 ns/op 761.00 ns/op 0.44
BeaconState.hashTreeRoot - 1 full validator 153.71 us/op 184.91 us/op 0.83
BeaconState.hashTreeRoot - 32 full validator 1.5425 ms/op 1.7553 ms/op 0.88
BeaconState.hashTreeRoot - 512 full validator 17.484 ms/op 19.088 ms/op 0.92
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 152.99 us/op 188.49 us/op 0.81
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.5747 ms/op 2.4198 ms/op 1.06
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 28.374 ms/op 32.610 ms/op 0.87
BeaconState.hashTreeRoot - 1 balances 132.29 us/op 150.81 us/op 0.88
BeaconState.hashTreeRoot - 32 balances 1.1615 ms/op 1.3751 ms/op 0.84
BeaconState.hashTreeRoot - 512 balances 10.732 ms/op 13.476 ms/op 0.80
BeaconState.hashTreeRoot - 250000 balances 196.88 ms/op 240.96 ms/op 0.82
aggregationBits - 2048 els - zipIndexesInBitList 56.836 us/op 45.848 us/op 1.24
byteArrayEquals 32 75.625 ns/op 84.308 ns/op 0.90
Buffer.compare 32 58.028 ns/op 59.218 ns/op 0.98
byteArrayEquals 1024 2.0682 us/op 2.4922 us/op 0.83
Buffer.compare 1024 75.147 ns/op 80.018 ns/op 0.94
byteArrayEquals 16384 32.425 us/op 34.557 us/op 0.94
Buffer.compare 16384 271.65 ns/op 287.23 ns/op 0.95
byteArrayEquals 123687377 243.41 ms/op 303.31 ms/op 0.80
Buffer.compare 123687377 6.1020 ms/op 12.570 ms/op 0.49
byteArrayEquals 32 - diff last byte 73.389 ns/op 98.967 ns/op 0.74
Buffer.compare 32 - diff last byte 58.268 ns/op 75.029 ns/op 0.78
byteArrayEquals 1024 - diff last byte 2.0225 us/op 2.5268 us/op 0.80
Buffer.compare 1024 - diff last byte 73.885 ns/op 84.220 ns/op 0.88
byteArrayEquals 16384 - diff last byte 32.284 us/op 40.096 us/op 0.81
Buffer.compare 16384 - diff last byte 279.72 ns/op 294.85 ns/op 0.95
byteArrayEquals 123687377 - diff last byte 245.04 ms/op 296.95 ms/op 0.83
Buffer.compare 123687377 - diff last byte 8.2305 ms/op 12.590 ms/op 0.65
byteArrayEquals 32 - random bytes 5.6090 ns/op 7.6330 ns/op 0.73
Buffer.compare 32 - random bytes 65.318 ns/op 67.310 ns/op 0.97
byteArrayEquals 1024 - random bytes 6.2070 ns/op 7.0850 ns/op 0.88
Buffer.compare 1024 - random bytes 65.387 ns/op 71.807 ns/op 0.91
byteArrayEquals 16384 - random bytes 5.8380 ns/op 7.7170 ns/op 0.76
Buffer.compare 16384 - random bytes 63.102 ns/op 78.324 ns/op 0.81
byteArrayEquals 123687377 - random bytes 8.7000 ns/op 17.140 ns/op 0.51
Buffer.compare 123687377 - random bytes 69.750 ns/op 96.990 ns/op 0.72
regular array get 100000 times 45.192 us/op 56.159 us/op 0.80
wrappedArray get 100000 times 45.712 us/op 58.899 us/op 0.78
arrayWithProxy get 100000 times 14.824 ms/op 18.004 ms/op 0.82
ssz.Root.equals 53.628 ns/op 62.096 ns/op 0.86
byteArrayEquals 52.206 ns/op 59.580 ns/op 0.88
Buffer.compare 10.783 ns/op 13.009 ns/op 0.83
shuffle list - 16384 els 8.4468 ms/op 9.5796 ms/op 0.88
shuffle list - 250000 els 126.14 ms/op 148.97 ms/op 0.85
processSlot - 1 slots 17.841 us/op 24.986 us/op 0.71
processSlot - 32 slots 4.5943 ms/op 4.8993 ms/op 0.94
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 61.574 ms/op 75.280 ms/op 0.82
getCommitteeAssignments - req 1 vs - 250000 vc 2.7073 ms/op 3.0088 ms/op 0.90
getCommitteeAssignments - req 100 vs - 250000 vc 4.0464 ms/op 4.2934 ms/op 0.94
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3304 ms/op 4.7927 ms/op 0.90
findModifiedValidators - 10000 modified validators 342.01 ms/op 431.38 ms/op 0.79
findModifiedValidators - 1000 modified validators 199.41 ms/op 259.87 ms/op 0.77
findModifiedValidators - 100 modified validators 169.82 ms/op 210.14 ms/op 0.81
findModifiedValidators - 10 modified validators 205.57 ms/op 214.07 ms/op 0.96
findModifiedValidators - 1 modified validators 195.03 ms/op 224.56 ms/op 0.87
findModifiedValidators - no difference 198.64 ms/op 230.81 ms/op 0.86
compare ViewDUs 4.5018 s/op 7.4018 s/op 0.61
compare each validator Uint8Array 1.5207 s/op 2.7749 s/op 0.55
compare ViewDU to Uint8Array 1.2568 s/op 1.7809 s/op 0.71
migrate state 1000000 validators, 24 modified, 0 new 731.56 ms/op 1.2691 s/op 0.58
migrate state 1000000 validators, 1700 modified, 1000 new 1.0738 s/op 1.7469 s/op 0.61
migrate state 1000000 validators, 3400 modified, 2000 new 1.2510 s/op 1.4233 s/op 0.88
migrate state 1500000 validators, 24 modified, 0 new 751.78 ms/op 1.0225 s/op 0.74
migrate state 1500000 validators, 1700 modified, 1000 new 1.5821 s/op 1.3702 s/op 1.15
migrate state 1500000 validators, 3400 modified, 2000 new 1.6961 s/op 1.4702 s/op 1.15
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.2200 ns/op 4.5000 ns/op 1.38
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3510 us/op 527.42 ns/op 2.56
computeProposers - vc 250000 18.415 ms/op 9.5596 ms/op 1.93
computeEpochShuffling - vc 250000 159.46 ms/op 127.28 ms/op 1.25
getNextSyncCommittee - vc 250000 207.62 ms/op 167.16 ms/op 1.24
computeSigningRoot for AttestationData 30.686 us/op 25.338 us/op 1.21
hash AttestationData serialized data then Buffer.toString(base64) 2.4421 us/op 2.4253 us/op 1.01
toHexString serialized data 1.2909 us/op 1.2787 us/op 1.01
Buffer.toString(base64) 247.22 ns/op 250.39 ns/op 0.99

by benchmarkbot/action

nflaig
nflaig commented Apr 17, 2024
View reviewed changes
packages/cli/src/cmds/validator/options.ts
@@ -303,15 +304,16 @@ export const validatorOptions: CliCommandOptions<IValidatorCliArgs> = {
type: "boolean",
},

// Remote signer
// External signer
Copy link
Member Author

@nflaig nflaig Apr 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to make the wording a bit more consistent within the code, and also few logs. We generally use the term "external signer" to describe an external/(remote) server that handles the signing operations for a specific public key. Whereas the term "remote signer" is also used to describe a remote signer in the local validator store.

@nflaig nflaig marked this pull request as ready for review April 17, 2024 17:37
@nflaig nflaig requested a review from a team as a code owner April 17, 2024 17:37
@nflaig nflaig added this to the v1.18.0 milestone Apr 17, 2024
nflaig
nflaig commented Apr 17, 2024
View reviewed changes
packages/cli/src/cmds/validator/options.ts
@@ -303,15 +304,16 @@ export const validatorOptions: CliCommandOptions<IValidatorCliArgs> = {
type: "boolean",
},

// Remote signer
// External signer

"externalSigner.url": {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could consider supporting multiple external signers in the future, our implementation can easily be adjusted to support this but I think it's better to not include it in this PR. Opened an issue to keep track of it and for further discussion #6679.

g11tech
g11tech approved these changes Apr 19, 2024
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wemeetagain wemeetagain merged commit 05fe0ac into unstable Apr 19, 2024
21 checks passed
@wemeetagain wemeetagain deleted the nflaig/external-signer-sync branch April 19, 2024 14:12
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@g11tech g11tech g11tech approved these changes

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.18.0
Development

Successfully merging this pull request may close these issues.

Synchronize keys between validator client and external signer
3 participants
@nflaig @wemeetagain @g11tech