PoA: Wait for transition finality before applying #5774

rphmeier · 2017-06-06T14:45:02Z

Summary of changes:

Engine::on_new_block gets a flag indicating if the block was the first of an epoch. For PoA engines this will lead to the contracts finalizeChange function being called.
Epoch changes split into two modes: signals and instant changes. Switching to a new fixed list is an instant change, but switching to a contract requires finality so it just signals a change.
Signals lead to instant changes at the block where they are finalized.
authority_round::RollingFinality keeps track of the current finality state for Aura engines.
Warp snapshots for PoA include a proof of the change and a proof of finality if necessary.
EpochVerifier gains a check_finality_proof function which will check a finality proof under the epoch.
AuthorityRound configuration gains a immediateTransitions flag to retain the old behavior and disregard

Although the API looks complicated, it's largely just to support light clients easily -- checking on epoch_end_signal requires nothing more than the block and receipts, as generating a proof is done only on full nodes.

is_epoch_end doesn't have access to the state, which enforces that any epoch changes requiring secure access to state are only applied on finality.

Tendermint hasn't been ported yet, but it should be pretty quick

rphmeier · 2017-06-09T16:11:46Z

Core logic is done now, but tests still need patching up. I'll leave a list of things I'd like to have automated tests against before merging:

epoch_transition_for on canonical/non-canonical chains
test that transition to fixed list is instant
test that transition to contract is not instant
test that transition within contract is also not instant
test snapshot on contract chain.
test snapshot on chain that transitions to contract.

Also the snapshot warp target needs to be changed to target the most recent finalized block to prevent an unfinalized pending transitions to be missed when there is a gap.

rphmeier · 2017-06-09T16:24:39Z

I'd also really like to have "import tests" run as part of the CI:
basically, we'll have a submodule containing tests. each test will consist of a chain spec and blocks to import and we'll test (using freshly compiled parity) parity import --chain spec_file blocks_file completes without issue. We can do something similar with snapshots.

NikVolf · 2017-06-12T09:36:33Z

ethcore/src/engines/authority_round/mod.rs

+						Some(t) => t,
+						None => {
+							// this really should never happen :)
+							debug!(target: "engine", "No genesis transition found.");


why not panic here, if it can never happen

…lizeChange

keorn · 2017-06-21T10:59:12Z

json/src/spec/authority_round.rs

@@ -49,6 +49,9 @@ pub struct AuthorityRoundParams {
 	/// Block from which monotonic steps start.
 	#[serde(rename="validateStepTransition")]
 	pub validate_step_transition: Option<Uint>,
+	/// Whether transitions should be immediate.
+	#[serde(rename="immediateTransitions")]
+	pub immediate_transitions: Option<bool>,


Shouldn't that be only relevant to ValidatorSets that do transitions (unlike SimpleList)?

Transitions to SimpleList in a Multi set are instant regardless of the setting of this flag. It could be a per-validator-set flag, but it makes more sense to have it global because
a) this flag influences whether finality checking is done, which is at the engine level
b) if any single validator set doesn't finalize transitions as necessary, then warp sync and light sync won't work and it's the same effect as having none of the sets finalize transitions.

arkpar · 2017-06-27T16:17:15Z

ethcore/src/snapshot/consensus/authority.rs

-		let firsts_iter = self.unverified_firsts.iter().map(|&(_, ref hdr)| hdr);
+		// we store all last verifiers, but not all firsts.
+		// match each unverified first epoch with a last epoch verifier.
+		let mut lasts_reversed = self.last_epochs.iter().rev();


Is this efficient enough?

a reversed [T] iterator is just pointer-bumping, same as an in-order iterator. The loop itself just walks two vectors in reverse, which should be pretty fast.

arkpar · 2017-06-27T16:18:11Z

ethcore/src/snapshot/consensus/authority.rs

+		let mut lasts_reversed = self.last_epochs.iter().rev();
+		for &(ref header, ref finality_proof, hash) in self.unverified_firsts.iter().rev() {
+			let mut found = false;
+			while let Some(last) = lasts_reversed.next() {


More idiomatic version: while let Some((header, verifier)) =

arkpar · 2017-06-27T16:55:10Z

LGTM, I've looked into changes outside of authority_round mostly.

arkpar · 2017-06-28T10:21:59Z

needs merging

afck · 2019-05-14T10:38:59Z

ethcore/src/snapshot/tests/test_validator_contract.json

@@ -39,11 +43,11 @@
 		"0000000000000000000000000000000000000004": { "balance": "1", "builtin": { "name": "identity", "pricing": { "linear": { "base": 15, "word": 3 } } } },
 		"0000000000000000000000000000000000000005": {
 			"balance": "1",
-			"constructor": "6060604052604060405190810160405280737d577a597b2742b498cb5cf0c26cdcd726d39e6e73ffffffffffffffffffffffffffffffffffffffff1681526020017382a978b3f5962a5b0957d9ee9eef472ee55b42f173ffffffffffffffffffffffffffffffffffffffff16815250600290600261007e929190610096565b50341561008757fe5b5b60006001819055505b610163565b82805482825590600052602060002090810192821561010f579160200282015b8281111561010e5782518260006101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff160217905550916020019190600101906100b6565b5b50905061011c9190610120565b5090565b61016091905b8082111561015c57600081816101000a81549073ffffffffffffffffffffffffffffffffffffffff021916905550600101610126565b5090565b90565b61045d806101726000396000f30060606040526000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff168063303e98e5146100675780639300c9261461008d578063b7ab4db5146100e4578063bfc708a014610159578063fd6e1b501461018f575bfe5b341561006f57fe5b6100776101c5565b6040518082815260200191505060405180910390f35b341561009557fe5b6100e26004808035906020019082018035906020019080806020026020016040519081016040528093929190818152602001838360200280828437820191505050505050919050506101d0565b005b34156100ec57fe5b6100f46102b3565b6040518080602001828103825283818151815260200191508051906020019060200280838360008314610146575b80518252602083111561014657602082019150602081019050602083039250610122565b5050509050019250505060405180910390f35b341561016157fe5b61018d600480803573ffffffffffffffffffffffffffffffffffffffff16906020019091905050610348565b005b341561019757fe5b6101c3600480803573ffffffffffffffffffffffffffffffffffffffff1690602001909190505061034c565b005b600060015490505b90565b600081600290805190602001906101e8929190610350565b50600143034090506000546000191681600019161415156102ae578060008160001916905550600160016000828254019250508190555060015481600019167f47e91f47ccfdcb578564e1af55da55c5e5d33403372fe68e4fed3dfd385764a184604051808060200182810382528381815181526020019150805190602001906020028083836000831461029b575b80518252602083111561029b57602082019150602081019050602083039250610277565b5050509050019250505060405180910390a35b5b5050565b6102bb6103da565b600280548060200260200160405190810160405280929190818152602001828054801561033d57602002820191906000526020600020905b8160009054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200190600101908083116102f3575b505050505090505b90565b5b50565b5b50565b8280548282559060005260206000209081019282156103c9579160200282015b828111156103c85782518260006101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff16021790555091602001919060010190610370565b5b5090506103d691906103ee565b5090565b602060405190810160405280600081525090565b61042e91905b8082111561042a57600081816101000a81549073ffffffffffffffffffffffffffffffffffffffff0219169055506001016103f4565b5090565b905600a165627a7a723058205c9ed1e1da2b93682907ac47377a662b21a5f9d89c4b21be40b098bdb00254360029"
+			"constructor": "6060604052602060405190810160405280737d577a597b2742b498cb5cf0c26cdcd726d39e6e73ffffffffffffffffffffffffffffffffffffffff16815250600090600161004e92919061005c565b50341561005757fe5b610129565b8280548282559060005260206000209081019282156100d5579160200282015b828111156100d45782518260006101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff1602179055509160200191906001019061007c565b5b5090506100e291906100e6565b5090565b61012691905b8082111561012257600081816101000a81549073ffffffffffffffffffffffffffffffffffffffff0219169055506001016100ec565b5090565b90565b61056f806101386000396000f30060606040526000357c0100000000000000000000000000000000000000000000000000000000900463ffffffff16806375286211146100675780639300c92614610079578063b7ab4db5146100d0578063c476dd4014610145578063d69f13bb146101c7575bfe5b341561006f57fe5b610077610206565b005b341561008157fe5b6100ce600480803590602001908201803590602001908080602002602001604051908101604052809392919081815260200183836020028082843782019150505050505091905050610275565b005b34156100d857fe5b6100e061031f565b6040518080602001828103825283818151815260200191508051906020019060200280838360008314610132575b8051825260208311156101325760208201915060208101905060208303925061010e565b5050509050019250505060405180910390f35b341561014d57fe5b6101c5600480803573ffffffffffffffffffffffffffffffffffffffff1690602001909190803590602001909190803590602001908201803590602001908080601f016020809104026020016040519081016040528093929190818152602001838380828437820191505050505050919050506103b4565b005b34156101cf57fe5b610204600480803573ffffffffffffffffffffffffffffffffffffffff169060200190919080359060200190919050506103ba565b005b73fffffffffffffffffffffffffffffffffffffffe3373ffffffffffffffffffffffffffffffffffffffff1614151561023f5760006000fd5b600060018054905014151561027257600160009080546102609291906103bf565b5060006001816102709190610411565b505b5b565b806001908051906020019061028b92919061043d565b506001430340600019167f55252fa6eee4741b4e24a74a70e9c11fd2c2281df8d6ea13126ff845f7825c89826040518080602001828103825283818151815260200191508051906020019060200280838360008314610309575b805182526020831115610309576020820191506020810190506020830392506102e5565b5050509050019250505060405180910390a25b50565b6103276104c7565b60008054806020026020016040519081016040528092919081815260200182805480156103a957602002820191906000526020600020905b8160009054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff168152602001906001019080831161035f575b505050505090505b90565b5b505050565b5b5050565b8280548282559060005260206000209081019282156104005760005260206000209182015b828111156103ff5782548255916001019190600101906103e4565b5b50905061040d91906104db565b5090565b81548183558181151161043857818360005260206000209182019101610437919061051e565b5b505050565b8280548282559060005260206000209081019282156104b6579160200282015b828111156104b55782518260006101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff1602179055509160200191906001019061045d565b5b5090506104c391906104db565b5090565b602060405190810160405280600081525090565b61051b91905b8082111561051757600081816101000a81549073ffffffffffffffffffffffffffffffffffffffff0219169055506001016104e1565b5090565b90565b61054091905b8082111561053c576000816000905550600101610524565b5090565b905600a165627a7a7230582041ce7e5c820bc89b1a330a3233c4f3013e77433ecba368fa234adf758d87fe1d0029"


Is that really the code from https://gist.github.com/anonymous/2a43783647e0f0dfcc359bd6fd81d6d9?
https://ethervm.io/decompile says this code contains a finalizeChange function, but the gist has a finalizeSignal. Also, the signature of reportMalicious seems to be different.

rphmeier added 7 commits May 29, 2017 15:25

final engine changes

6ce6434

migration to v13

b24093b

Merge branch 'master' into wait-for-transition-finality

e4949f9

adding and removing pending transitions

a93fa05

epoch_transition_for

dcd10dd

port snapshot to new engine methods

1a95a68

final validator set interface

d74874b

rphmeier added A3-inprogress ⏳ Pull request is in progress. No review needed at this stage. M4-core ⛓ Core client code / Rust. labels Jun 6, 2017

Merge branch 'master' into wait-for-transition-finality

416926c

rphmeier mentioned this pull request Jun 6, 2017

Support PoA contract chains in the light client #5642

Closed

rphmeier added 8 commits June 7, 2017 15:42

fix compiler errors

9475b34

revert v13/epoch_depth transition

7306d27

make call on new epoch

07de21a

rolling finality checker

1dc8e5f

tests for finality checker

5ff6cb1

constructing finality proof upon pending transition

848e99f

fix warnings and finality proof checking

5f37b7c

fix compiler warnings in tests

308d7ed

test fixes

9f5e0f4

NikVolf reviewed Jun 12, 2017

View reviewed changes

rphmeier added 7 commits June 12, 2017 15:22

don't include genesis in finality checking

b0f68b3

change snapshot test chain building logic

ca43e5d

minor refactorings

0f4edb6

fetch epoch transition based on parent, fix divide-by-zero in SimpleList

e414b26

fix formatting

0ce1c98

fix ABIs and finality checking in snapshot restoration

e4346d7

encode signal number in proof

5ca71eb

rphmeier added 5 commits June 16, 2017 14:12

create more blocks at the end of tests

abc88e6

update gist to accurate contract code

08ce15a

test for epoch_transition_for

b0e5b50

fix tests with immediateTransitions parameter

365e08f

Merge branch 'master' into wait-for-transition-finality

a0c93bc

rphmeier added A0-pleasereview 🤓 Pull request needs code review. and removed A3-inprogress ⏳ Pull request is in progress. No review needed at this stage. labels Jun 16, 2017

rphmeier added 4 commits June 19, 2017 14:01

disable force flag after forcing

a814791

rename ValidatorsChanged to InitiateChange and finalizeSignal to fina…

201ee03

…lizeChange

a few more validator set tests

b30dccf

Merge branch 'master' into wait-for-transition-finality

3a329ad

keorn reviewed Jun 21, 2017

View reviewed changes

keorn approved these changes Jun 22, 2017

View reviewed changes

Merge branch 'master' into wait-for-transition-finality

6818b75

rphmeier mentioned this pull request Jun 23, 2017

Kovan validator contract #5632

Closed

arkpar reviewed Jun 27, 2017

View reviewed changes

arkpar added A8-looksgood 🦄 Pull request is reviewed well. and removed A0-pleasereview 🤓 Pull request needs code review. labels Jun 27, 2017

Merge branch 'master' into wait-for-transition-finality

e30be1c

arkpar merged commit d069b98 into master Jun 28, 2017

arkpar deleted the wait-for-transition-finality branch June 28, 2017 11:17

afck reviewed May 14, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PoA: Wait for transition finality before applying #5774

PoA: Wait for transition finality before applying #5774

rphmeier commented Jun 6, 2017 •

edited

Loading

rphmeier commented Jun 9, 2017 •

edited

Loading

rphmeier commented Jun 9, 2017

NikVolf Jun 12, 2017 •

edited

Loading

keorn Jun 21, 2017

rphmeier Jun 21, 2017

arkpar Jun 27, 2017

rphmeier Jun 27, 2017

arkpar Jun 27, 2017

arkpar commented Jun 27, 2017

arkpar commented Jun 28, 2017

afck May 14, 2019

PoA: Wait for transition finality before applying #5774

PoA: Wait for transition finality before applying #5774

Conversation

rphmeier commented Jun 6, 2017 • edited Loading

rphmeier commented Jun 9, 2017 • edited Loading

rphmeier commented Jun 9, 2017

NikVolf Jun 12, 2017 • edited Loading

Choose a reason for hiding this comment

keorn Jun 21, 2017

Choose a reason for hiding this comment

rphmeier Jun 21, 2017

Choose a reason for hiding this comment

arkpar Jun 27, 2017

Choose a reason for hiding this comment

rphmeier Jun 27, 2017

Choose a reason for hiding this comment

arkpar Jun 27, 2017

Choose a reason for hiding this comment

arkpar commented Jun 27, 2017

arkpar commented Jun 28, 2017

afck May 14, 2019

Choose a reason for hiding this comment

rphmeier commented Jun 6, 2017 •

edited

Loading

rphmeier commented Jun 9, 2017 •

edited

Loading

NikVolf Jun 12, 2017 •

edited

Loading