Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dot/parachain/backing): handle second incoming overseer message #3682

Merged
merged 144 commits into from
Feb 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
144 commits
Select commit Hold shift + click to select a range
2f3bb6f
feat/scale: add `BitVec` (#3253)
kanishkatn May 29, 2023
f001854
feat(erasure_coding): introduce erasure coding for PoV Distributor (#…
edwardmack Jun 8, 2023
d2c0163
Fix(lint): fixes a lint issue in bitvec-test (#3306)
kanishkatn Jun 9, 2023
710b03c
feat(parachain): add types (#3297)
kanishkatn Jun 15, 2023
be2aaea
feat(parachain): Add StatementDistributionMessage varingDataType (#3316)
axaysagathiya Jun 20, 2023
d08d7f1
feat/runtime: Add few parachain runtime calls (#3241)
kanishkatn Jun 21, 2023
4a103f4
feat(parachain): Create struct for Approval Distribution Message (#3326)
edwardmack Jun 21, 2023
4928d45
chore(parachain): improve comments of statement and statement distrib…
axaysagathiya Jun 23, 2023
50f2bdb
feat(parachain): Add CollationProtocol VaryingDataType (#3337)
axaysagathiya Jun 26, 2023
c88e72d
feat(parachain): Implement request and response message for /req_stat…
axaysagathiya Jul 6, 2023
4d103a3
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 6, 2023
b8ddd99
fix(lib/erasure): Move test assertion to non-error condition (#3371)
edwardmack Jul 7, 2023
c805896
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 7, 2023
c2d1083
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 7, 2023
1104a22
chore(lib/parachain): add tests to decode varying data types regardin…
axaysagathiya Jul 7, 2023
e7ab6e9
chore(lib/parachain): remove pov byte array and use pov struct (#3375)
axaysagathiya Jul 7, 2023
59ff8eb
feat(lib/parachain): Implement request and response message for /req_…
axaysagathiya Jul 10, 2023
b445405
feat(lib/parachain): added parachain service and registering protocol…
kishansagathiya Jul 11, 2023
2fb5189
Implemented extra parachain host runtime API calls (#3237)
kishansagathiya Jul 27, 2023
4702a7a
Fix(lib/parachain): use actual forkID in parachain network protocol (…
axaysagathiya Aug 1, 2023
9e12f3a
feat(lib/parachain): introduced parachain candidate validation (#3249)
kishansagathiya Aug 4, 2023
b571559
moved `lib/parachain` to `dot/parachain` (#3429)
kishansagathiya Aug 8, 2023
d57a322
feat(lib/parachain): Create and integrate Validation Protocol struct …
edwardmack Aug 18, 2023
9ed0b70
runtime/parachain: add ValidationCodeByHash and copy tests to `wazero…
kanishkatn Aug 22, 2023
0772943
feat(lib/erasure): implement Go binding over rust for erasure coding …
axaysagathiya Aug 24, 2023
38c0c49
feat(lib/erasure): construct a trie of chunks of erasure coded values…
axaysagathiya Aug 28, 2023
33e9f9a
chore(parachain): use wazero instead of wasmer
kishansagathiya Aug 30, 2023
788a160
chore(dot/parachain): rename statement (#3470)
axaysagathiya Sep 17, 2023
12fcb0f
feat(dot/parachain): Implement overseer skeleton (#3460)
edwardmack Sep 27, 2023
eaede54
candidate backing skeleton
axaysagathiya Sep 29, 2023
ac70372
add types
axaysagathiya Sep 29, 2023
5f7cf18
lint
axaysagathiya Sep 29, 2023
c25dc7e
register subsystem
axaysagathiya Oct 2, 2023
d8fc37e
correct the channel direction
axaysagathiya Oct 2, 2023
8f10724
tag issues with todo
axaysagathiya Oct 2, 2023
703c211
chore(dot/peerset): added parachain related reputation values and rea…
kishansagathiya Oct 2, 2023
36d91ce
feat(dot/parachain): Implement candidate backing skeleton (#3497)
axaysagathiya Oct 2, 2023
bab70bc
add license
axaysagathiya Oct 2, 2023
6a2c20c
WIP
axaysagathiya Oct 3, 2023
b15e612
Merge branch 'feat/parachain'
axaysagathiya Oct 3, 2023
9ecf2b6
handle statement message
axaysagathiya Oct 3, 2023
56496bc
collator protocol skeleton (#3512)
kishansagathiya Oct 5, 2023
f4cbfc7
handle statement
axaysagathiya Oct 11, 2023
b8395cd
create func to handle data received from channel
axaysagathiya Oct 11, 2023
c9055dd
feat(parachain): availability store skeleton (#3521)
edwardmack Oct 12, 2023
b08bffd
implement import statement function
axaysagathiya Oct 14, 2023
23dccd0
feat(parachain/collator): handle Declare message received by a collat…
kishansagathiya Oct 16, 2023
491942d
implement post import statement actions
axaysagathiya Oct 17, 2023
95f9572
address reviews
axaysagathiya Oct 17, 2023
47f9d62
move subsystem types to separate file and add comments
axaysagathiya Oct 19, 2023
34a70dc
link issues with todo
axaysagathiya Oct 19, 2023
4fb70f3
feat(parachain/overseer): added message forwarding in overseer (#3546)
kishansagathiya Oct 20, 2023
d4c884c
implement todos
axaysagathiya Oct 30, 2023
e9d4593
Merge branch 'feat/parachain'
axaysagathiya Oct 30, 2023
914b0b6
lint
axaysagathiya Oct 31, 2023
f4e0d22
test statement message coming from overseer (WIP)
axaysagathiya Nov 6, 2023
981b49d
reviews
axaysagathiya Nov 9, 2023
6440a1d
feat(parachain/collator): handle AdvertiseCollation message received…
kishansagathiya Nov 10, 2023
8ed796b
handle can second (WIP)
axaysagathiya Nov 15, 2023
6556d86
WIP
axaysagathiya Nov 20, 2023
bca22c9
feat(parachain/availabilitystore): implement database store/retrieve …
edwardmack Nov 20, 2023
2be0046
handle canSecond message
axaysagathiya Nov 21, 2023
5d4f841
lint
axaysagathiya Nov 21, 2023
0ea3806
reviews
axaysagathiya Nov 21, 2023
fedb910
feat(parachain/collator): handle Seconded overseer message (#3557)
kishansagathiya Nov 21, 2023
d784db5
Merge branch 'issue-3507'
axaysagathiya Nov 21, 2023
dbc18dc
implement enum using interface
axaysagathiya Nov 21, 2023
3e4be40
Merge branch 'issue-3507'
axaysagathiya Nov 22, 2023
8366778
implement enum using interface
axaysagathiya Nov 22, 2023
8b5d176
test import statement
axaysagathiya Nov 28, 2023
e687e3b
unit test for import statement func
axaysagathiya Nov 28, 2023
f3678d1
test post import statement (WIP)
axaysagathiya Nov 29, 2023
5549e9f
implement misbehaviour enum using interface
axaysagathiya Nov 29, 2023
4477a3a
test post import statement
axaysagathiya Dec 3, 2023
5ddbba7
improve kickOffValidationWork
axaysagathiya Dec 5, 2023
9dc985a
add underscore in description of test cases
axaysagathiya Dec 6, 2023
0d2d42f
implement ExecutorParams varying datatype slice
axaysagathiya Dec 9, 2023
a693b3b
correction
axaysagathiya Dec 11, 2023
ffa5cf1
unit test for validateAndMakeAvailable func
axaysagathiya Dec 13, 2023
b1c2f6c
test func to handle statement message (WIP)
axaysagathiya Dec 16, 2023
08529db
unit test for a func to handle statement message
axaysagathiya Dec 17, 2023
f69ad4e
lint
axaysagathiya Dec 17, 2023
7bfe209
Merge branch 'feat/parachain' into issue-3507
axaysagathiya Dec 17, 2023
b800f57
conflict resolve
axaysagathiya Dec 17, 2023
c99d8a2
lint
axaysagathiya Dec 17, 2023
0d8aec7
Merge branch 'issue-3507'
axaysagathiya Dec 20, 2023
ad35d33
WIP
axaysagathiya Jan 1, 2024
0929708
improve TestImportStatement
axaysagathiya Jan 1, 2024
47d3282
improve TestPostImportStatement
axaysagathiya Jan 1, 2024
293c78c
improve TestHandleStatementMessage
axaysagathiya Jan 1, 2024
89d1780
unit tests for HandleCanSecondMessage function
axaysagathiya Jan 2, 2024
1d64d56
Merge branch 'issue-3507'
axaysagathiya Jan 2, 2024
27685d5
address PR review (WIP)
axaysagathiya Jan 4, 2024
ee90c82
add license
axaysagathiya Jan 5, 2024
2cb343c
address PR review (WIP)
axaysagathiya Jan 5, 2024
09e3a14
address PR review (WIP)
axaysagathiya Jan 5, 2024
e8abb02
remove ProspectiveParachainsMode type
axaysagathiya Jan 5, 2024
a2befdb
Merge branch 'issue-3507'
axaysagathiya Jan 8, 2024
4f2c64f
rename types and fix lint
axaysagathiya Jan 8, 2024
a5cafbe
lint
axaysagathiya Jan 8, 2024
7ee9ab9
lint
axaysagathiya Jan 8, 2024
a0dcc91
feat(dot/parachain): added overseer signals (#3638)
kishansagathiya Jan 9, 2024
2f37136
handle second message coming to backing subsystem
axaysagathiya Jan 10, 2024
22fe870
clean up
axaysagathiya Jan 11, 2024
74336bd
unit test
axaysagathiya Jan 11, 2024
af2ac15
unexport types
axaysagathiya Jan 12, 2024
55ff42e
Merge branch 'feat/parachain' into issue-3507
axaysagathiya Jan 12, 2024
ff2ebcd
clean up
axaysagathiya Jan 12, 2024
ba307cb
unexport types
axaysagathiya Jan 12, 2024
ec5ef26
address review
axaysagathiya Jan 14, 2024
885c658
unexport types
axaysagathiya Jan 14, 2024
caa9aab
lint
axaysagathiya Jan 14, 2024
58e2d5c
Merge branch 'issue-3507'
axaysagathiya Jan 15, 2024
0f29fde
unexport types
axaysagathiya Jan 15, 2024
eb95341
add nil check
axaysagathiya Jan 15, 2024
cf43c6a
Merge branch 'issue-3507'
axaysagathiya Jan 15, 2024
4c26e7a
license
axaysagathiya Jan 15, 2024
e5bb1ab
Merge branch 'axay/feat/backing-handle-cansecond'
axaysagathiya Jan 15, 2024
7dad6ae
clean up
axaysagathiya Jan 15, 2024
64e60b5
chore(lib/babe/inherents): remove duplicate types (#3668)
axaysagathiya Jan 15, 2024
41658a5
some documentation and some moving around
kishansagathiya Jan 15, 2024
84168e9
reuse existing overseer messages and other moving around
kishansagathiya Jan 15, 2024
caf6514
Merge branch 'issue-3507' of github.com:ChainSafe/gossamer into issue…
kishansagathiya Jan 15, 2024
9a89327
few more changes
kishansagathiya Jan 16, 2024
f50b912
make license
kishansagathiya Jan 16, 2024
76c51da
improve overseer message types and error handling
axaysagathiya Jan 16, 2024
36dee85
Merge branch 'issue-3507' into axay/feat/backing-handle-cansecond
axaysagathiya Jan 16, 2024
a7c6d9a
Merge branch 'axay/feat/backing-handle-cansecond'
axaysagathiya Jan 16, 2024
dfba19c
lint
axaysagathiya Jan 16, 2024
139d41a
clean up
axaysagathiya Jan 16, 2024
f7bc903
feat(dot/parachain/backing): handle `statement` message coming from o…
axaysagathiya Jan 16, 2024
b24d3c8
Merge branch 'feat/parachain'
axaysagathiya Jan 16, 2024
9d32821
address review
axaysagathiya Jan 17, 2024
ff2d40a
Merge branch 'axay/feat/backing-handle-cansecond'
axaysagathiya Jan 18, 2024
c65f95c
improve error messages
axaysagathiya Jan 18, 2024
f423c54
add details to errors
axaysagathiya Jan 22, 2024
0ede340
address reviews
axaysagathiya Jan 23, 2024
b131ff9
chore(dot/parachain/overseer): fix `TestHandleBlockEvents` the flaky …
axaysagathiya Jan 23, 2024
19a437a
feat(dot/parachain/backing): handle CanSecond incoming overseer messa…
axaysagathiya Jan 23, 2024
7c94a6f
feat(parachain/availabilitystore): implement database access function…
edwardmack Jan 23, 2024
352bb29
Merge branch 'feat/parachain' into axay/feat/backing/handle-second-me…
axaysagathiya Jan 24, 2024
eaa321a
Merge branch 'feat/parachain' into axay/feat/backing/handle-second-me…
axaysagathiya Jan 25, 2024
64af610
resolve merge problems
axaysagathiya Jan 25, 2024
569bdbf
Merge branch 'feat/parachain'
axaysagathiya Feb 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions dot/parachain/backing/candidate_backing.go
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ func (cb *CandidateBacking) processMessage(msg any, chRelayParentAndCommand chan
logger.Debug(fmt.Sprintf("can't second the candidate: %s", err))
}
case SecondMessage:
cb.handleSecondMessage()
return cb.handleSecondMessage(msg.CandidateReceipt, msg.PersistedValidationData, msg.PoV, chRelayParentAndCommand)
case StatementMessage:
return cb.handleStatementMessage(msg.RelayParent, msg.SignedFullStatement, chRelayParentAndCommand)
case parachaintypes.ActiveLeavesUpdateSignal:
Expand All @@ -249,10 +249,6 @@ func (cb *CandidateBacking) handleGetBackedCandidatesMessage() {
// TODO: Implement this #3504
}

func (cb *CandidateBacking) handleSecondMessage() {
// TODO: Implement this #3506
}

// Import the statement and kick off validation work if it is a part of our assignment.
func (cb *CandidateBacking) handleStatementMessage(
relayParent common.Hash,
Expand Down
88 changes: 88 additions & 0 deletions dot/parachain/backing/second.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
// Copyright 2024 ChainSafe Systems (ON)
// SPDX-License-Identifier: LGPL-3.0-only

package backing

import (
"errors"
"fmt"

parachaintypes "github.com/ChainSafe/gossamer/dot/parachain/types"
)

var (
errWrongPVDForSecondingCandidate = errors.New(
"incorrect persisted validation data provided for seconding candidate")
errUnknownRelayParentForSecondingCandidate = errors.New(
"attempted to second a candidate with an unknown relay parent")
errParaOutsideAssignmentForSeconding = errors.New(
"subsystem requested to second for parachain beyond our assignment scope")
errAlreadySignedValidStatement = errors.New("already signed a valid statement for this candidate")
)

func (cb *CandidateBacking) handleSecondMessage(
candidateReceipt parachaintypes.CandidateReceipt,
pvd parachaintypes.PersistedValidationData,
pov parachaintypes.PoV,
chRelayParentAndCommand chan relayParentAndCommand,
) error {
hash, err := candidateReceipt.Hash()
if err != nil {
return fmt.Errorf("hashing candidate receipt: %w", err)
}
candidateHash := parachaintypes.CandidateHash{Value: hash}

pvdHash, err := pvd.Hash()
if err != nil {
return fmt.Errorf("hashing persisted validation data: %w", err)
}

if candidateReceipt.Descriptor.PersistedValidationDataHash != pvdHash {
return fmt.Errorf("%w; mismatch between persisted validation data hash in candidate descriptor: %s "+
"and calculated hash of given persisted validation data: %s",
errWrongPVDForSecondingCandidate,
candidateReceipt.Descriptor.PersistedValidationDataHash,
pvdHash,
)
}

rpState, ok := cb.perRelayParent[candidateReceipt.Descriptor.RelayParent]
if !ok {
return fmt.Errorf("%w; candidate hash: %s; relay parent: %s",
errUnknownRelayParentForSecondingCandidate,
candidateHash,
candidateReceipt.Descriptor.RelayParent,
)
}

// Sanity check that candidate is from our assignment.
if candidateReceipt.Descriptor.ParaID != uint32(rpState.assignment) {
return fmt.Errorf("%w: candidate hash: %s; candidate paraID: %d; assignment: %d",
errParaOutsideAssignmentForSeconding,
candidateHash,
candidateReceipt.Descriptor.ParaID,
rpState.assignment,
)
}

// If the message is a `CandidateBackingMessage::Second`, sign and dispatch a
// Seconded statement only if we have not signed a Valid statement for the requested candidate.
if rpState.issuedStatements[candidateHash] {
return fmt.Errorf("%w: candidate hash: %s", errAlreadySignedValidStatement, candidateHash)
}

// Kick off background validation with intent to second.
logger.Debugf("validate and second candidate: %s", candidateHash)
return rpState.validateAndMakeAvailable(
executorParamsAtRelayParent,
cb.SubSystemToOverseer,
chRelayParentAndCommand,
candidateReceipt,
rpState.relayParent,
pvd,
pov,
uint32(len(rpState.tableContext.validators)),
second,
candidateHash,
)
}
133 changes: 133 additions & 0 deletions dot/parachain/backing/second_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
// Copyright 2024 ChainSafe Systems (ON)
// SPDX-License-Identifier: LGPL-3.0-only

package backing

import (
"testing"

parachaintypes "github.com/ChainSafe/gossamer/dot/parachain/types"
"github.com/ChainSafe/gossamer/lib/common"
"github.com/stretchr/testify/require"
)

func TestHandleSecondMessage(t *testing.T) {
t.Parallel()

testCases := []struct {
description string
cb *CandidateBacking
candidateReceipt parachaintypes.CandidateReceipt
pvd parachaintypes.PersistedValidationData
err error
}{
{
description: "wrong_persisted_validation_data_for_seconding_candidate",
cb: &CandidateBacking{},
candidateReceipt: parachaintypes.CandidateReceipt{},
pvd: parachaintypes.PersistedValidationData{},
err: errWrongPVDForSecondingCandidate,
},
{
description: "unknown_relay_parent_for_seconding_candidate",
cb: &CandidateBacking{},
candidateReceipt: dummyCandidateReceipt(t),
pvd: dummyPVD(t),
err: errUnknownRelayParentForSecondingCandidate,
},
{
description: "parachain_outside_assignment_for_seconding",
cb: &CandidateBacking{
perRelayParent: map[common.Hash]*perRelayParentState{
getDummyHash(t, 6): {
assignment: 10,
},
},
},
candidateReceipt: dummyCandidateReceipt(t),
pvd: dummyPVD(t),
err: errParaOutsideAssignmentForSeconding,
},
{
description: "already_signed_valid_statement_for_candidate",
cb: &CandidateBacking{
perRelayParent: map[common.Hash]*perRelayParentState{
getDummyHash(t, 6): {
assignment: 1,
issuedStatements: map[parachaintypes.CandidateHash]bool{
dummyCandidateHash(t): true,
},
},
},
},
candidateReceipt: dummyCandidateReceipt(t),
pvd: dummyPVD(t),
err: errAlreadySignedValidStatement,
},
{
description: "kick_off_background_validation_with_intent_to_second",
cb: &CandidateBacking{
perRelayParent: map[common.Hash]*perRelayParentState{
getDummyHash(t, 6): {
assignment: 1,
awaitingValidation: map[parachaintypes.CandidateHash]bool{
dummyCandidateHash(t): true,
},
},
},
},
candidateReceipt: dummyCandidateReceipt(t),
pvd: dummyPVD(t),
err: nil,
},
}

for _, tc := range testCases {
tc := tc
t.Run(tc.description, func(t *testing.T) {
t.Parallel()

err := tc.cb.handleSecondMessage(tc.candidateReceipt, tc.pvd, parachaintypes.PoV{}, nil)
if err == nil {
require.NoError(t, err)
} else {
require.ErrorIs(t, err, tc.err)
}
})
}
}

func dummyPVD(t *testing.T) parachaintypes.PersistedValidationData {
t.Helper()

return parachaintypes.PersistedValidationData{
ParentHead: parachaintypes.HeadData{
Data: []byte{1, 2, 3},
},
RelayParentNumber: 5,
RelayParentStorageRoot: getDummyHash(t, 5),
MaxPovSize: 3,
}
}

func dummyCandidateReceipt(t *testing.T) parachaintypes.CandidateReceipt {
t.Helper()

cr := getDummyCommittedCandidateReceipt(t).ToPlain()

// blake2bhash of PVD in dummyPVD(t *testing.T) function
cr.Descriptor.PersistedValidationDataHash =
common.MustHexToHash("0x3544fbcdcb094751a5e044a30b994b2586ffc0b50e8b88c381461fe023a7242f")

return cr
}

func dummyCandidateHash(t *testing.T) parachaintypes.CandidateHash {
t.Helper()

cr := dummyCandidateReceipt(t)
hash, err := cr.Hash()
require.NoError(t, err)

return parachaintypes.CandidateHash{Value: hash}
}
2 changes: 1 addition & 1 deletion dot/parachain/backing/validated_candidate_command.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ type validatedCandidateCommand byte

const (
// We were instructed to second the candidate that has been already validated.
second = validatedCandidateCommand(iota) //nolint:unused
second = validatedCandidateCommand(iota)
// We were instructed to validate the candidate.
attest
// We were not able to `Attest` because backing validator did not send us the PoV.
Expand Down
13 changes: 13 additions & 0 deletions dot/parachain/types/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,15 @@ type PersistedValidationData struct {
MaxPovSize uint32 `scale:"4"`
}

func (pvd PersistedValidationData) Hash() (common.Hash, error) {
bytes, err := scale.Marshal(pvd)
if err != nil {
return common.Hash{}, fmt.Errorf("marshalling PersistedValidationData: %w", err)
}

return common.Blake2bHash(bytes)
}

// OccupiedCoreAssumption is an assumption being made about the state of an occupied core.
type OccupiedCoreAssumption scale.VaryingDataType

Expand Down Expand Up @@ -526,6 +535,10 @@ type CandidateHash struct {
Value common.Hash `scale:"1"`
}

func (ch CandidateHash) String() string {
return ch.Value.String()
}

// PoV represents a Proof-of-Validity block (PoV block) or a parachain block.
// It contains the necessary data for the parachain specific state transition logic.
type PoV struct {
Expand Down
Loading