Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Async Backing: Statement Distribution Tests #6755

Merged
merged 266 commits into from
Mar 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
266 commits
Select commit Hold shift + click to select a range
cd722bb
start on handling incoming
rphmeier Sep 23, 2022
c702f05
split off session info into separate map
rphmeier Sep 23, 2022
6a7dd79
start in on a knowledge tracker
rphmeier Sep 24, 2022
9bfda1f
address some grumbles
rphmeier Sep 24, 2022
0f25b3a
format
rphmeier Sep 24, 2022
60bb289
missed comment
rphmeier Sep 24, 2022
03ff76d
some docs for direct
rphmeier Sep 24, 2022
c2918fc
add note on slashing
rphmeier Sep 24, 2022
bddeff1
amend
rphmeier Sep 24, 2022
6009816
simplify 'direct' code
rphmeier Sep 26, 2022
54f06c0
finish up the 'direct' logic
rphmeier Sep 26, 2022
f4effa1
add a bunch of tests for the direct-in-group logic
rphmeier Sep 27, 2022
26c86b7
rename 'direct' to 'cluster', begin a candidate_entry module
rphmeier Sep 27, 2022
9dd68f1
distill candidate_entry
rphmeier Sep 28, 2022
9727d8c
start in on a statement-store module
rphmeier Sep 28, 2022
44c2b37
some utilities for the statement store
rphmeier Sep 28, 2022
75627c5
rewrite 'send_statement_direct' using new tools
rphmeier Sep 28, 2022
336f412
filter sending logic on peers which have the relay-parent in their view.
rphmeier Sep 30, 2022
90a91a2
some more logic for handling incoming statements
rphmeier Sep 30, 2022
4dc62b9
req/res: BackedCandidatePacket -> AttestedCandidate + tweaks
rphmeier Sep 30, 2022
2ca0e60
add a `validated_in_group` bitfield to BackedCandidateInventory
rphmeier Sep 30, 2022
e495285
BackedCandidateInventory -> Manifest
rphmeier Sep 30, 2022
585e79b
start in on requester module
rphmeier Oct 2, 2022
5fddf7d
add outgoing request for attested candidate
rphmeier Oct 3, 2022
7fefe78
add a priority mechanism for requester
rphmeier Oct 3, 2022
b6b2d13
some request dispatch logic
rphmeier Oct 3, 2022
d800805
add seconded mask to tagged-request
rphmeier Oct 3, 2022
443ffab
amend manifest to hold group index
rphmeier Oct 4, 2022
e0de216
handle errors and set up scaffold for response validation
rphmeier Oct 4, 2022
c493bca
validate attested candidate responses
rphmeier Oct 4, 2022
9303836
requester -> requests
rphmeier Oct 7, 2022
41a43d6
add some utilities for manipulating requests
rphmeier Oct 7, 2022
e9bd586
begin integrating requester
rphmeier Oct 7, 2022
a237924
start grid module
rphmeier Oct 7, 2022
cfd25b9
tiny
rphmeier Oct 11, 2022
5547404
refactor grid topology to expose more info to subsystems
rphmeier Oct 12, 2022
9df8063
fix grid_topology test
rphmeier Oct 12, 2022
2e27afa
fix overseer test
rphmeier Oct 12, 2022
596f738
Merge branch 'rh-async-backing-feature' into rh-async-backing-stateme…
rphmeier Oct 13, 2022
3b6d5da
implement topology group-based view construction logic
rphmeier Oct 13, 2022
6cf0187
fmt
rphmeier Oct 15, 2022
2d628db
flesh out grid slightly more
rphmeier Oct 15, 2022
0113341
add indexed groups utility
rphmeier Oct 15, 2022
4536646
integrate Groups into per-session info
rphmeier Oct 15, 2022
950ae30
refactor statement store to borrow Groups
rphmeier Oct 15, 2022
bc6db74
implement manifest knowledge utility
rphmeier Oct 15, 2022
d3cfa76
add a test for topology setup
rphmeier Oct 19, 2022
dc2e141
don't send to group members
rphmeier Oct 19, 2022
5fd17ef
test for conflicting manifests
rphmeier Oct 19, 2022
57c027f
manifest knowledge tests
rphmeier Oct 19, 2022
f9cf743
fmt
rphmeier Oct 19, 2022
fffae31
rename field
rphmeier Oct 19, 2022
b607890
garbage collection for grid tracker
rphmeier Oct 20, 2022
6ffbd39
routines for finding correct/incorrect advertisers
rphmeier Oct 20, 2022
f5de677
add manifest import logic
rphmeier Oct 20, 2022
e969ed0
tweak naming
rphmeier Oct 22, 2022
ab152c6
more tests for manifest import
rphmeier Oct 23, 2022
5762b9f
add comment
rphmeier Oct 23, 2022
8a6e8f5
rework candidates into a view-wide tracker
rphmeier Oct 24, 2022
f395301
fmt
rphmeier Oct 24, 2022
cd5fac3
start writing boilerplate for grid sending
rphmeier Oct 25, 2022
4c0343f
fmt
rphmeier Oct 25, 2022
421dd9e
some more group boilerplate
rphmeier Oct 26, 2022
db59d33
refactor handling of topology and authority IDs
rphmeier Oct 27, 2022
978335d
fmt
rphmeier Oct 27, 2022
671d801
send statements directly to grid peers where possible
rphmeier Oct 27, 2022
958765a
send to cluster only if statement belongs to cluster
rphmeier Oct 27, 2022
0bc269f
improve handling of cluster statements
rphmeier Oct 28, 2022
c272426
handle incoming statements along the grid
rphmeier Oct 28, 2022
b72b4ac
API for introduction of candidates into the tree
rphmeier Nov 2, 2022
cb4f2ef
backing: use new prospective parachains API
rphmeier Nov 2, 2022
83e0bab
fmt prospective parachains changes
rphmeier Nov 2, 2022
4640b1a
fmt statement-dist
rphmeier Nov 2, 2022
b46177b
fix condition
rphmeier Nov 2, 2022
6b8370f
get ready for tracking importable candidates
rphmeier Nov 3, 2022
63499a2
prospective parachains: add Cow logic
rphmeier Nov 3, 2022
19cca1a
incomplete and complete hypothetical candidates
rphmeier Nov 3, 2022
7960c80
remove keep_if_unneeded
rphmeier Nov 3, 2022
29d68e0
fmt
rphmeier Nov 3, 2022
6fa7b80
implement more general HypotheticalFrontier
rphmeier Nov 3, 2022
73ce578
fmt, cleanup
rphmeier Nov 4, 2022
99dad29
add a by_parent_hash index to candidate tracker
rphmeier Nov 4, 2022
e04caf4
more framework for future code
rphmeier Nov 7, 2022
7944c31
utilities for getting all hypothetical candidates for frontier
rphmeier Nov 7, 2022
7e99ff2
track origin in statement store
rphmeier Nov 7, 2022
238f70b
fmt
rphmeier Nov 7, 2022
dbb2a3c
requests should return peer
rphmeier Nov 7, 2022
43a2add
apply post-confirmation reckoning
rphmeier Nov 7, 2022
164a09b
flesh out import/announce/circulate logic on new statements
rphmeier Nov 8, 2022
9d9fbe3
adjust
rphmeier Dec 1, 2022
af0a76b
adjust TODO comment
rphmeier Jan 18, 2023
c446d22
Merge branch 'rh-async-backing-feature' into rh-async-backing-stateme…
rphmeier Jan 18, 2023
447fa5c
fix backing tests
rphmeier Jan 18, 2023
1b4b6d4
update statement-distribution to use new indexedvec
rphmeier Jan 18, 2023
45ef1de
fmt
rphmeier Jan 18, 2023
cd3ecf8
query hypothetical candidates
rphmeier Jan 18, 2023
bcf5f35
implement `note_importable_under`
rphmeier Jan 18, 2023
8f73988
extract common utility of fragment tree updates
rphmeier Jan 18, 2023
a150fb4
add a helper function for getting statements unknown by backing
rphmeier Jan 18, 2023
e097811
Merge branch 'rh-async-backing-feature' into rh-async-backing-stateme…
rphmeier Jan 19, 2023
80a051c
import fresh statements to backing
rphmeier Jan 19, 2023
2d2e974
send announcements and acknowledgements over grid
rphmeier Jan 19, 2023
50fb10e
provide freshly importable statements
rphmeier Jan 20, 2023
4c5b601
do not issue requests on newly importable candidates
rphmeier Jan 20, 2023
aaa41cc
add TODO for later when confirming candidate
rphmeier Jan 20, 2023
51a891c
write a routine for handling backed candidate notifications
rphmeier Jan 20, 2023
dcdce46
simplify grid substantially
rphmeier Jan 21, 2023
a7ff9f9
add some test TODOs
rphmeier Jan 21, 2023
c0196d1
handle confirmed candidates & grid announcements
rphmeier Jan 24, 2023
00939fd
finish implementing manifest handling, including follow up statements
rphmeier Jan 24, 2023
3c976fa
send follow-up statements when acknowledging freshly backed
rphmeier Jan 24, 2023
f4983e7
fmt
rphmeier Jan 24, 2023
15f08cc
handle incoming acknowledgements
rphmeier Jan 24, 2023
c8d11be
a little DRYing
rphmeier Jan 24, 2023
cc1d050
wire up network messages to handlers
rphmeier Jan 24, 2023
1380aa9
fmt
rphmeier Jan 24, 2023
3d7af96
some skeleton code for peer view update handling
rphmeier Jan 24, 2023
badfe68
more peer view skeleton stuff
rphmeier Jan 24, 2023
1e2848c
Fix async backing statement distribution tests (#6621)
mrcnski Jan 25, 2023
374cd2c
Resolve some todos in async backing statement-distribution branch (#6…
mrcnski Jan 30, 2023
20b4a9e
Merge branch 'rh-async-backing-feature' into rh-async-backing-stateme…
rphmeier Jan 30, 2023
0a078bb
Merge branch 'rh-async-backing-statement-distribution' of https://git…
rphmeier Jan 30, 2023
23bc084
fix test compilation
rphmeier Jan 30, 2023
d6a2111
add a TODO for a test
rphmeier Jan 30, 2023
e037eaa
clean up a couple of TODOs
rphmeier Jan 30, 2023
bf0c063
implement sending pending cluster statements
rphmeier Jan 31, 2023
9ca8c80
refactor utility function for sending acknowledgement and statements
rphmeier Jan 31, 2023
ecb3ae6
mostly implement catching peers up via grid
rphmeier Jan 31, 2023
d297070
Merge branch 'rh-async-backing-feature' into rh-async-backing-stateme…
mrcnski Feb 1, 2023
4ac25ad
Fix clippy error
mrcnski Feb 1, 2023
bf15adc
alter grid to track all pending statements
rphmeier Feb 1, 2023
f2aa242
Merge branch 'rh-async-backing-statement-distribution' of https://git…
rphmeier Feb 1, 2023
6b29c74
fix more TODOs and format
rphmeier Feb 1, 2023
38c4779
tweak a TODO in requests
rphmeier Feb 1, 2023
254efc3
some logic for dispatching requests
rphmeier Feb 2, 2023
04fe291
fmt
rphmeier Feb 7, 2023
a693727
skeleton for response receiving
rphmeier Feb 7, 2023
3a7864f
Async backing statement distribution: cluster tests (#6678)
mrcnski Feb 7, 2023
1cd0ae0
fix up
rphmeier Feb 8, 2023
dc1ae58
fmt
rphmeier Feb 8, 2023
e745a60
update TODO
rphmeier Feb 8, 2023
89c6c78
Merge branch 'rh-async-backing-statement-distribution' of https://git…
rphmeier Feb 8, 2023
040d877
rework seconded mask in requests
rphmeier Feb 9, 2023
6269f6c
change doc
rphmeier Feb 9, 2023
43ef927
change unhandledresponse not to borrow request manager
rphmeier Feb 9, 2023
8563f32
only accept responses sufficient to back
rphmeier Feb 9, 2023
72394c4
finish implementing response handling
rphmeier Feb 9, 2023
9d9770d
extract statement filter to protocol crate
rphmeier Feb 9, 2023
784c1e7
rework requests: use statement filter in network protocol
rphmeier Feb 9, 2023
ad94034
dispatch cluster requests correctly
rphmeier Feb 9, 2023
0face3f
rework cluster statement sending
rphmeier Feb 12, 2023
b214988
implement request answering
rphmeier Feb 13, 2023
78b1de6
fmt
rphmeier Feb 13, 2023
2a7df7b
only send confirmed candidate statement messages on unified relay-parent
rphmeier Feb 13, 2023
f09a133
Fix Tests In Statement Distribution Branch
mrcnski Feb 14, 2023
0996882
Async Backing: Integrate `vstaging` of statement distribution into `l…
mrcnski Feb 15, 2023
1e3c63e
clean up some review comments
rphmeier Feb 15, 2023
f1264d9
clean up warnings
rphmeier Feb 15, 2023
bb37d26
Async backing statement distribution: grid tests (#6673)
mrcnski Feb 16, 2023
14a731b
Add more tests in `requests.rs` (#6707)
mrcnski Feb 16, 2023
89d8d42
remove outdated inventory terminology
rphmeier Feb 16, 2023
0b2e958
Async backing statement distribution: `Candidates` tests (#6658)
mrcnski Feb 16, 2023
7e339de
Merge branch 'rh-async-backing-feature' into rh-async-backing-stateme…
rphmeier Feb 17, 2023
480c619
Merge branch 'rh-async-backing-statement-distribution' of https://git…
rphmeier Feb 17, 2023
537987e
Async Backing: Fix clippy errors in statement distribution branch (#6…
mrcnski Feb 17, 2023
4c254e6
add tests module
rphmeier Feb 17, 2023
c50d50d
fix warnings in existing tests
rphmeier Feb 17, 2023
1ffcd43
create basic test harness
rphmeier Feb 17, 2023
12947b7
create a test state struct
rphmeier Feb 18, 2023
0bfe213
fmt
rphmeier Feb 18, 2023
72b5e9c
create empty cluster & grid modules for tests
rphmeier Feb 18, 2023
d62e5dc
some TODOs for cluster test suite
rphmeier Feb 18, 2023
314b68c
describe test-suite for grid logic
rphmeier Feb 18, 2023
6a897c6
describe request test suite
rphmeier Feb 18, 2023
a45e350
fix seconding-limit bug
rphmeier Feb 18, 2023
1a93b15
Remove extraneous `pub`
mrcnski Feb 19, 2023
7eca1ee
Fix some test compile warnings
mrcnski Feb 19, 2023
2603cff
Remove some unneeded `allow`s
mrcnski Feb 19, 2023
7bd6333
Merge branch 'rh-async-backing-statement-distribution' of https://git…
rphmeier Feb 20, 2023
b733ffc
adapt some new test helpers from Marcin
rphmeier Feb 20, 2023
46a09f3
add helper for activating a gossip topology
rphmeier Feb 20, 2023
a1d56c8
add utility for signing statements
rphmeier Feb 20, 2023
786cf28
helpers for connecting/disconnecting peers
rphmeier Feb 20, 2023
a51ad18
round out network utilities
rphmeier Feb 20, 2023
643ba8b
fmt
rphmeier Feb 20, 2023
ff15c82
fix bug in initializing validator-meta
rphmeier Feb 21, 2023
de62130
fix compilation
rphmeier Feb 21, 2023
daef720
implement first cluster test
rphmeier Feb 21, 2023
c4daaa7
TODOs for incoming request tests
rphmeier Feb 21, 2023
01468f6
Remove unneeded `make_committed_candidate` helper
mrcnski Feb 21, 2023
9f9ef9a
fmt
mrcnski Feb 21, 2023
a94e203
Hook up request sender
mrcnski Feb 21, 2023
bc158da
Add `valid_statement_without_prior_seconded_is_ignored` test
mrcnski Feb 21, 2023
5845a6b
Fix `valid_statement_without_prior_seconded_is_ignored` test
mrcnski Feb 21, 2023
00ee5e7
some more tests for cluster
rphmeier Feb 21, 2023
1b7d910
add a TODO about grid senders
rphmeier Feb 21, 2023
804e8b3
integrate inbound req/res into test harness
rphmeier Feb 21, 2023
7297515
polish off initial cluster test suite
rphmeier Feb 22, 2023
91c5525
keep introduce candidate request
rphmeier Feb 22, 2023
153c9cf
fix tests after introduce candidate request
rphmeier Feb 22, 2023
070fe34
fmt
rphmeier Feb 22, 2023
0053ff4
Merge branch 'rh-async-backing-feature' into rh-async-backing-stateme…
rphmeier Feb 22, 2023
c320384
Add grid protocol to module docs
mrcnski Feb 23, 2023
2078d0c
Merge branch 'rh-async-backing-statement-distribution' into mrcnski/s…
mrcnski Feb 23, 2023
1f4f2e1
Remove obsolete test
mrcnski Feb 23, 2023
5e5b46c
Fix comments
mrcnski Feb 24, 2023
b23f515
Test `backed_in_path_only: true`
mrcnski Feb 24, 2023
9ac736b
Update node/network/protocol/src/lib.rs
rphmeier Feb 26, 2023
646184a
Update node/network/protocol/src/request_response/mod.rs
rphmeier Feb 26, 2023
920c607
Mark receiver with `vstaging`
mrcnski Feb 26, 2023
0a62d8f
First draft of `ensure_seconding_limit_is_respected` test
mrcnski Feb 26, 2023
7d514ea
validate grid senders based on manifest kind
rphmeier Feb 26, 2023
8cf3d5f
fix mask_seconded/valid
rphmeier Feb 26, 2023
06beb30
fix unwanted-mask check
rphmeier Feb 26, 2023
2987860
fix build
slumber Feb 27, 2023
dbc37d1
resolve todo on leaf mode
slumber Feb 27, 2023
813a230
Unify protocol naming to vstaging
slumber Feb 27, 2023
fe86521
Fix `ensure_seconding_limit_is_respected` test
mrcnski Feb 27, 2023
0fed5e6
Start `backed_candidate_leads_to_advertisement` test
mrcnski Feb 27, 2023
12689f4
fmt, fix grid test after topology change
rphmeier Feb 27, 2023
064f737
Send Backed notification
mrcnski Feb 28, 2023
5727105
Merge branch 'rh-async-backing-statement-distribution' into mrcnski/s…
mrcnski Feb 28, 2023
e9e8ecf
Finish `backed_candidate_leads_to_advertisement` test
mrcnski Feb 28, 2023
677a0d6
Finish `peer_reported_for_duplicate_statements` test
mrcnski Feb 28, 2023
269341f
Finish `received_advertisement_before_confirmation_leads_to_request`
mrcnski Feb 28, 2023
3202071
Add `advertisements_rejected_from_incorrect_peers` test
mrcnski Feb 28, 2023
ec9f71f
Add `manifest_rejected_*` tests
mrcnski Feb 28, 2023
c047fcc
Add `manifest_rejected_when_group_does_not_match_para` test
mrcnski Feb 28, 2023
b954560
Add `local_node_sanity_checks_incoming_requests` test
mrcnski Mar 1, 2023
1cd2152
Add `local_node_respects_statement_mask` test
mrcnski Mar 1, 2023
c5e2cbe
Add tests where peer is reported for providing invalid signatures
mrcnski Mar 1, 2023
83641b5
Add `cluster_peer_allowed_to_send_incomplete_statements` test
mrcnski Mar 1, 2023
f6fb698
Add `received_advertisement_after_backing_leads_to_acknowledgement`
mrcnski Mar 2, 2023
2def528
Add `received_advertisement_after_confirmation_before_backing` test
mrcnski Mar 2, 2023
1c531b0
peer_reported_for_advertisement_conflicting_with_confirmed_candidate
mrcnski Mar 2, 2023
a1c0791
Add `peer_reported_for_not_enough_statements` test
mrcnski Mar 2, 2023
6a45c50
Add `peer_reported_for_providing_statements_meant_to_be_masked_out`
mrcnski Mar 2, 2023
88172e3
Add `additional_statements_are_shared_after_manifest_exchange`
mrcnski Mar 2, 2023
c24a05f
Add `grid_statements_imported_to_backing` test
mrcnski Mar 2, 2023
321dbaa
Add `relay_parent_entering_peer_view_leads_to_advertisement` test
mrcnski Mar 2, 2023
d0477d6
Add `advertisement_not_re_sent_when_peer_re_enters_view` test
mrcnski Mar 2, 2023
8e2ae8d
Update node/network/statement-distribution/src/vstaging/tests/grid.rs
mrcnski Mar 3, 2023
bee35f0
Resolve TODOs, update test
mrcnski Mar 3, 2023
62e70a5
Address unused code
mrcnski Mar 3, 2023
6f21cf6
Add check after every test for unhandled requests
mrcnski Mar 3, 2023
595d5d0
Refactor (`make_dummy_leaf` and `handle_sent_request`)
mrcnski Mar 3, 2023
72412ac
Refactor (`make_dummy_topology`)
mrcnski Mar 3, 2023
a16288a
Merge branch 'rh-async-backing-feature' into mrcnski/statement-distri…
mrcnski Mar 5, 2023
68e8038
Minor refactor
mrcnski Mar 5, 2023
2b23f01
Merge branch 'rh-async-backing-feature' into mrcnski/statement-distri…
mrcnski Mar 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion node/network/protocol/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -737,7 +737,7 @@ pub mod vstaging {
/// Network messages used by the statement distribution subsystem.
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
pub enum StatementDistributionMessage {
/// A notification of a signed statement in compact form.
/// A notification of a signed statement in compact form, for a given relay parent.
#[codec(index = 0)]
Statement(Hash, UncheckedSignedStatement),

Expand Down
2 changes: 1 addition & 1 deletion node/network/statement-distribution/src/vstaging/grid.rs
Original file line number Diff line number Diff line change
Expand Up @@ -695,7 +695,7 @@ fn decompose_statement_filter<'a>(
}

/// A summary of a manifest being sent by a counterparty.
#[derive(Clone)]
#[derive(Debug, Clone)]
pub struct ManifestSummary {
/// The claimed parent head data hash of the candidate.
pub claimed_parent_hash: Hash,
Expand Down
4 changes: 3 additions & 1 deletion node/network/statement-distribution/src/vstaging/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ const COST_UNEXPECTED_REQUEST: Rep = Rep::CostMajor("Unexpected attested candida
const BENEFIT_VALID_RESPONSE: Rep = Rep::BenefitMajor("Peer Answered Candidate Request");
const BENEFIT_VALID_STATEMENT: Rep = Rep::BenefitMajor("Peer provided a valid statement");
const BENEFIT_VALID_STATEMENT_FIRST: Rep =
Rep::BenefitMajorFirst("Peer was the first to provide a valid statement");
Rep::BenefitMajorFirst("Peer was the first to provide a given valid statement");

struct PerRelayParentState {
local_validator: Option<LocalValidatorState>,
Expand All @@ -144,6 +144,7 @@ struct LocalValidatorState {
grid_tracker: GridTracker,
}

#[derive(Debug)]
struct PerSessionState {
session_info: SessionInfo,
groups: Groups,
Expand Down Expand Up @@ -1006,6 +1007,7 @@ pub(crate) async fn share_local_statement<Context>(

// two kinds of targets: those in our 'cluster' (currently just those in the same group),
// and those we are propagating to through the grid.
#[derive(Debug)]
enum DirectTargetKind {
Cluster,
Grid,
Expand Down
Loading