Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Dispute spam protection #4134

Merged
merged 72 commits into from
Nov 19, 2021
Merged
Changes from 1 commit
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
726e278
Mostly notes.
eskimor Oct 12, 2021
6cd95b3
Merge branch 'master' into rk-dispute-coordinator-spam-slots
eskimor Oct 13, 2021
435242d
Better error messages.
eskimor Oct 13, 2021
39a2313
Introduce Fatal/NonFatal + drop back channel participation
eskimor Oct 13, 2021
345a4e7
Better error messages.
eskimor Oct 13, 2021
7e308df
Utility function for receiving `CandidateEvent`s.
eskimor Oct 13, 2021
f12cd2a
Ordering module typechecks.
eskimor Oct 13, 2021
d6aa872
cargo fmt
eskimor Oct 13, 2021
162e307
Prepare spam slots module.
eskimor Oct 13, 2021
3b32fb0
Implement SpamSlots mechanism.
eskimor Oct 14, 2021
c9e944e
Implement queues.
eskimor Oct 14, 2021
af7c7fb
cargo fmt
eskimor Oct 14, 2021
595c410
Participation.
eskimor Oct 15, 2021
79e9478
Participation taking shape.
eskimor Oct 18, 2021
7452dec
Finish participation.
eskimor Oct 19, 2021
7f001f7
cargo fmt
eskimor Oct 19, 2021
8dbe4fa
Cleanup.
eskimor Oct 19, 2021
fdb8565
WIP: Cleanup + Integration.
eskimor Oct 19, 2021
84aad25
Make `RollingSessionWindow` initialized by default.
eskimor Oct 20, 2021
62c34b2
Make approval voting typecheck.
eskimor Oct 20, 2021
71b5f0d
Get rid of lazy_static & fix approval voting tests
eskimor Oct 20, 2021
edb8583
Move `SessionWindowSize` to node primitives.
eskimor Oct 20, 2021
ec6b52c
Implement dispute coordinator initialization.
eskimor Oct 21, 2021
a195007
cargo fmt
eskimor Oct 21, 2021
7558594
Make queues return error instead of boolean.
eskimor Oct 21, 2021
ff36198
Initialized: WIP
eskimor Oct 21, 2021
35154b9
Introduce chain api for getting finalized block.
eskimor Oct 22, 2021
efd28a9
Fix ordering to only prune candidates on finalized events.
eskimor Oct 22, 2021
41ae78d
Pruning of old sessions in spam slots.
eskimor Oct 22, 2021
0e27f46
New import logic.
eskimor Oct 22, 2021
76a3660
Make everything typecheck.
eskimor Oct 22, 2021
aeaaaf3
Fix warnings.
eskimor Oct 22, 2021
9784485
Get rid of obsolete dispute-participation.
eskimor Oct 22, 2021
cb82b3b
Fixes.
eskimor Oct 22, 2021
ac684e7
Merge branch 'master' into rk-dispute-coordinator-spam-slots
eskimor Oct 22, 2021
463ba02
Add back accidentelly deleted Cargo.lock
eskimor Oct 22, 2021
5bfd5eb
Deliver disputes in an ordered fashion.
eskimor Oct 24, 2021
5f244bc
Add module docs for errors
eskimor Nov 3, 2021
d2d15e3
Use type synonym.
eskimor Nov 3, 2021
d8cfc71
hidden docs.
eskimor Nov 3, 2021
10a8569
Merge branch 'master' into rk-dispute-coordinator-spam-slots
eskimor Nov 3, 2021
f17f3dc
Fix overseer tests.
eskimor Nov 3, 2021
9b4fd1f
Ordering provider taking `CandidateReceipt`.
eskimor Nov 3, 2021
efbdd0f
Fix ordering to use relay_parent
eskimor Nov 4, 2021
57e7a4e
Add comment in ordering.rs.
eskimor Nov 4, 2021
59f111d
Take care of duplicate entries in queues.
eskimor Nov 4, 2021
2dd6c50
Better spam slots.
eskimor Nov 4, 2021
7502183
Review remarks + docs.
eskimor Nov 4, 2021
cadebab
Fix db tests.
eskimor Nov 5, 2021
165239c
Participation tests.
eskimor Nov 5, 2021
97e8173
Merge remote-tracking branch 'parity/master' into rk-dispute-coordina…
Lldenaurois Nov 8, 2021
b96c4a0
Also scrape votes on first leaf for good measure.
eskimor Nov 8, 2021
b4046d9
Make tests typecheck.
eskimor Nov 8, 2021
2723ee1
Merge branch 'master' into rk-dispute-coordinator-spam-slots
eskimor Nov 8, 2021
9adcda5
Merge branch 'rk-dispute-coordinator-spam-slots' of github.com:parity…
eskimor Nov 8, 2021
37ef1a0
Spelling.
eskimor Nov 8, 2021
872d575
Only participate in actual disputes, not on every import.
eskimor Nov 8, 2021
3e68bd4
Don't account backing votes to spam slots.
eskimor Nov 8, 2021
e3adf27
Fix more tests.
eskimor Nov 8, 2021
3e66471
Don't participate if we don't have keys.
eskimor Nov 8, 2021
e428aac
Fix tests, typos and warnings.
eskimor Nov 8, 2021
e6192e2
Fix merge error.
eskimor Nov 8, 2021
54e5a27
Spelling fixes.
eskimor Nov 8, 2021
e290045
Add missing docs.
eskimor Nov 8, 2021
94f93c3
Queue tests.
eskimor Nov 9, 2021
d3a401a
More tests.
eskimor Nov 9, 2021
cc2a5f3
Add metrics + don't short circuit import.
eskimor Nov 9, 2021
5190e50
Basic test for ordering provider.
eskimor Nov 10, 2021
5708e84
Import fix.
eskimor Nov 10, 2021
d168de1
Merge branch 'master' into rk-dispute-coordinator-spam-slots
eskimor Nov 16, 2021
a97ef35
Remove dead link.
eskimor Nov 19, 2021
10506a4
One more dead link.
eskimor Nov 19, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cargo fmt
eskimor committed Oct 13, 2021
commit d6aa87292ff0fca250deb6121e4ec25aa9b66587
9 changes: 4 additions & 5 deletions node/core/dispute-coordinator/src/real/error.rs
Original file line number Diff line number Diff line change
@@ -18,13 +18,13 @@ use futures::channel::oneshot;
use thiserror::Error;

use polkadot_node_subsystem::{
errors::{ChainApiError, RuntimeApiError}, SubsystemError,
errors::{ChainApiError, RuntimeApiError},
SubsystemError,
};
use polkadot_node_subsystem_util::runtime;

use crate::real::CodecError;
use crate::real::LOG_TARGET;
use super::db;
use crate::real::{CodecError, LOG_TARGET};

/// Errors for this subsystem.
#[derive(Debug, Error, derive_more::From)]
@@ -59,7 +59,6 @@ impl From<SubsystemError> for Error {
}
}


/// Fatal errors of this subsystem.
#[derive(Debug, Error)]
pub enum Fatal {
@@ -69,7 +68,7 @@ pub enum Fatal {

/// We received a legacy SubystemError::Context error which is considered fatal.
#[error("SubsystemError::Context error: {0}")]
SubsystemContext(String)
SubsystemContext(String),
}

#[derive(Debug, thiserror::Error)]
13 changes: 8 additions & 5 deletions node/core/dispute-coordinator/src/real/mod.rs
Original file line number Diff line number Diff line change
@@ -31,7 +31,7 @@ use std::{
time::{SystemTime, UNIX_EPOCH},
};

use futures::{FutureExt, TryFutureExt, channel::oneshot};
use futures::{channel::oneshot, FutureExt, TryFutureExt};
use kvdb::KeyValueDB;
use parity_scale_codec::{Decode, Encode, Error as CodecError};
use polkadot_node_primitives::{
@@ -58,9 +58,9 @@ use sc_keystore::LocalKeystore;
use crate::metrics::Metrics;
use backend::{Backend, OverlayedBackend};
use db::v1::{DbBackend, RecentDisputes};
use error::{Result, FatalResult};
use error::{FatalResult, Result};

use self::error::{NonFatal, log_error};
use self::error::{log_error, NonFatal};

mod backend;
mod db;
@@ -140,7 +140,8 @@ impl DisputeCoordinatorSubsystem {
mut ctx: Context,
mut backend: B,
clock: Box<dyn Clock>,
) -> FatalResult<()> where
) -> FatalResult<()>
where
Context: overseer::SubsystemContext<Message = DisputeCoordinatorMessage>,
Context: SubsystemContext<Message = DisputeCoordinatorMessage>,
B: Backend,
@@ -688,7 +689,9 @@ async fn handle_incoming(
metrics,
)
.await?;
pending_confirmation.send(outcome).map_err(|_| NonFatal::DisputeImportOneshotSend)?;
pending_confirmation
.send(outcome)
.map_err(|_| NonFatal::DisputeImportOneshotSend)?;
},
DisputeCoordinatorMessage::RecentDisputes(rx) => {
let recent_disputes = overlay_db.load_recent_disputes()?.unwrap_or_default();
80 changes: 44 additions & 36 deletions node/core/dispute-coordinator/src/real/ordering.rs
Original file line number Diff line number Diff line change
@@ -14,15 +14,19 @@
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.

use std::{cmp::{Ordering, Ord, PartialOrd}, collections::{HashMap, HashSet}};
use std::{
cmp::{Ord, Ordering, PartialOrd},
collections::{HashMap, HashSet},
};

use polkadot_node_subsystem::{ActiveLeavesUpdate, SubsystemSender};
use polkadot_node_subsystem_util::runtime::get_candidate_events;
use polkadot_primitives::v1::{BlockNumber, CandidateEvent, CandidateHash, CandidateReceipt, Hash, Id};
use polkadot_primitives::v1::{
BlockNumber, CandidateEvent, CandidateHash, CandidateReceipt, Hash, Id,
};

use super::error::Result;


/// Provider of `CandidateComparator` for candidates.
pub struct OrderingProvider {
/// Currently cached comparators for candidates.
@@ -54,36 +58,36 @@ pub struct CandidateComparator {
included_block_number: BlockNumber,
eskimor marked this conversation as resolved.
Show resolved Hide resolved
/// Para id, used for ordering parachains within same relay chain block.
para_id: Id,
/// The hash of the relay chain block the candidate got included in.
/// The hash of the relay chain block the candidate got included in.
included_block_hash: Hash,
eskimor marked this conversation as resolved.
Show resolved Hide resolved
}

impl PartialEq for CandidateComparator {
fn eq(&self, other: &CandidateComparator) -> bool {
Ordering::Equal == self.cmp(other)
Ordering::Equal == self.cmp(other)
}
}

impl Eq for CandidateComparator {}

impl PartialOrd for CandidateComparator {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}

impl Ord for CandidateComparator {
fn cmp(&self, other: &Self) -> Ordering {
match self.included_block_number.cmp(&other.included_block_number) {
Ordering::Equal => (),
o => return o,
}
match self.para_id.cmp(&other.para_id) {
Ordering::Equal => (),
o => return o,
}
self.included_block_hash.cmp(&other.included_block_hash)
}
fn cmp(&self, other: &Self) -> Ordering {
match self.included_block_number.cmp(&other.included_block_number) {
Ordering::Equal => (),
o => return o,
}
match self.para_id.cmp(&other.para_id) {
Ordering::Equal => (),
o => return o,
}
self.included_block_hash.cmp(&other.included_block_hash)
}
}

impl OrderingProvider {
@@ -95,24 +99,31 @@ impl OrderingProvider {
/// Retrieve a candidate comparator if available.
///
/// If not available, we can treat disputes concerning this candidate with low priority and
/// should use spam slots for such disputes.
pub async fn candidate_comparator<'a>(&'a mut self, candidate: &CandidateReceipt) -> Option<&'a CandidateComparator> {
/// should use spam slots for such disputes.
pub async fn candidate_comparator<'a>(
&'a mut self,
candidate: &CandidateReceipt,
) -> Option<&'a CandidateComparator> {
self.cached_comparators.get(&candidate.hash())
}

/// Query active leaves for any candidate `CandidateEvent::CandidateIncluded` events.
///
/// and updates current heads, so we can query candidates for all active heads.
pub async fn process_active_leaves_update<Sender: SubsystemSender>(&mut self, sender: &mut Sender, update: &ActiveLeavesUpdate) -> Result<()> {
if let Some(activated) = update.activated.as_ref() {
pub async fn process_active_leaves_update<Sender: SubsystemSender>(
&mut self,
sender: &mut Sender,
update: &ActiveLeavesUpdate,
) -> Result<()> {
if let Some(activated) = update.activated.as_ref() {
// Get included events:
let included = get_candidate_events(sender, activated.hash).await?
let included = get_candidate_events(sender, activated.hash)
.await?
.into_iter()
.filter_map(|ev| match ev {
CandidateEvent::CandidateIncluded(receipt, _ , _ , _) => Some(receipt),
CandidateEvent::CandidateIncluded(receipt, _, _, _) => Some(receipt),
_ => None,
}
);
});
for receipt in included {
let candidate_hash = receipt.hash();
let comparator = CandidateComparator {
@@ -121,7 +132,10 @@ impl OrderingProvider {
para_id: receipt.descriptor.para_id,
};
self.cached_comparators.insert(candidate_hash, comparator);
self.candidates_by_relay_chain.entry(activated.hash).or_default().insert(candidate_hash);
self.candidates_by_relay_chain
.entry(activated.hash)
.or_default()
.insert(candidate_hash);
}
}

@@ -136,33 +150,27 @@ impl OrderingProvider {
}
}



// Queue

// - On import - check for ordering for the given candidate hash
// - If present - import into priority queue based on given `CandidateComparator`.
// - If not push on best effort queue (if participation is required)


// Import


// Chain import


// Dispute distribution on startup - ordering irrelevant, just distribute what has not yet been
// distributed.
//
//
//


// Components/modules:
//

// Ordering: Get Ordering for candidates
//
//
// participation/Queues - might be owned by participation:
//
// // Will put message in queue, either priority or best effort depending on whether ordering
@@ -184,7 +192,7 @@ impl OrderingProvider {
// running_participations: HashSet<CandidateHash>,
// }
// // removes candidate from running_participations & dequeues the next participation.
//
//
// fn handle_local_statement(session, candidate_hash) ;
// - Keep track of in-flight participation tasks - watch for `IssueLocalStatement`.
// - Process queues (priority in order and then best effort in order)
9 changes: 3 additions & 6 deletions node/core/dispute-participation/src/lib.rs
Original file line number Diff line number Diff line change
@@ -199,18 +199,15 @@ async fn participate(
.await;

let available_data = match recover_available_data_rx.await? {
Ok(data) => {
data
},
Ok(data) => data,
Err(RecoveryError::Invalid) => {
// the available data was recovered but it is invalid, therefore we'll
// vote negatively for the candidate dispute
cast_invalid_vote(ctx, candidate_hash, candidate_receipt, session).await;
return Ok(())
},
Err(RecoveryError::Unavailable) => {
return Err(ParticipationError::MissingAvailableData(candidate_hash).into())
},
Err(RecoveryError::Unavailable) =>
return Err(ParticipationError::MissingAvailableData(candidate_hash).into()),
};

// we also need to fetch the validation code which we can reference by its
10 changes: 7 additions & 3 deletions node/subsystem-util/src/runtime/mod.rs
Original file line number Diff line number Diff line change
@@ -26,11 +26,15 @@ use sp_core::crypto::Public;
use sp_keystore::{CryptoStore, SyncCryptoStorePtr};

use polkadot_node_subsystem::{SubsystemContext, SubsystemSender};
use polkadot_primitives::v1::{CandidateEvent, CoreState, EncodeAs, GroupIndex, GroupRotationInfo, Hash, OccupiedCore, SessionIndex, SessionInfo, Signed, SigningContext, UncheckedSigned, ValidatorId, ValidatorIndex};
use polkadot_primitives::v1::{
CandidateEvent, CoreState, EncodeAs, GroupIndex, GroupRotationInfo, Hash, OccupiedCore,
SessionIndex, SessionInfo, Signed, SigningContext, UncheckedSigned, ValidatorId,
ValidatorIndex,
};

use crate::{
request_availability_cores, request_session_index_for_child, request_session_info,
request_validator_groups, request_candidate_events,
request_availability_cores, request_candidate_events, request_session_index_for_child,
request_session_info, request_validator_groups,
};

/// Errors that can happen on runtime fetches.