Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of gossipsub v1.1 - Hardening #263

Merged
merged 94 commits into from
Apr 18, 2020
Merged

Implementation of gossipsub v1.1 - Hardening #263

merged 94 commits into from
Apr 18, 2020

Conversation

vyzo
Copy link
Collaborator

@vyzo vyzo commented Mar 2, 2020

This PR implements the Hardening extensions as defined by Gossipsub v1.1 Specification

What is left to do

gossipsub.go Show resolved Hide resolved
@raulk raulk changed the title [WIP] gossipsub hardeninng [WIP] gossipsub hardening Mar 3, 2020
@vyzo
Copy link
Collaborator Author

vyzo commented Mar 9, 2020

Summoning @yusefnapora @raulk @Stebalien @Kubuxu @whyrusleeping

This is ready for review (modulo unit tests).

pubsub.go Show resolved Hide resolved
@@ -775,6 +784,12 @@ func (p *PubSub) handleIncomingRPC(rpc *RPC) {
}
}

// ask the router to vet the peer before commiting any processing resources
if !p.rt.AcceptFrom(rpc.from) {
log.Warningf("received message from router graylisted peer %s. Dropping RPC", rpc.from)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we do something if we get a bunch of these?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could blacklist i guess, but then we'd have to count stuff.

go ps.background(gs.p.ctx)
}

func (ps *peerScore) Score(p peer.ID) float64 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, this gets called for every single message received. It seems to me like it may get rather resource intensive. Let's keep this in mind and observe CPU profiles during tests

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the Score method gets called a lot. It should be relatively lightweight to compute, but good point about looking at CPU profiles.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also should watch lock contention there.

score.go Outdated
}

// check that we have an app specific score; the weight can be anything (but expected positive)
if p.AppSpecificScore == nil {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also check AppSpecificWeight here, or maybe even remove it altogether, as this is an app-controlled function that can regulate its own score values (in contrast with the other weights that modulate pubsub behavior).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good suggestion, we can consider removing the app specific weight and just keep the score function.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to removing AppSpecificWeight, I don't see it adding anything.

gossipsub.go Outdated Show resolved Hide resolved
gossipsub.go Outdated Show resolved Hide resolved
gossipsub.go Outdated Show resolved Hide resolved
gossipsub.go Outdated Show resolved Hide resolved
score.go Show resolved Hide resolved
@vyzo
Copy link
Collaborator Author

vyzo commented Mar 16, 2020

also merged in #268 as this fixes a tracing bug.

@vyzo
Copy link
Collaborator Author

vyzo commented Mar 17, 2020

Add a WithPeerScoreDebug option to enable periodic inspection/dumping of the peer scores.

@daviddias daviddias changed the title [WIP] gossipsub hardening gossipsub 1.1 - Hardening Mar 22, 2020
@daviddias daviddias changed the title gossipsub 1.1 - Hardening gossipsub v1.1 - Hardening Mar 22, 2020
@vyzo
Copy link
Collaborator Author

vyzo commented Mar 23, 2020

Some improvements of the backoff handling:

  • we now extend the backoff when pruning because of backoff or a negative score; we also check the exact expiration.
  • backoff is cleaned up every 15 heartbeat ticks to amortize the cost of iterating over the maps.

@daviddias daviddias requested review from dirkmc and yusefnapora March 23, 2020 08:29
gossipsub.go Show resolved Hide resolved
gossipsub.go Show resolved Hide resolved
gossipsub.go Show resolved Hide resolved
gossipsub.go Show resolved Hide resolved
score.go Outdated Show resolved Hide resolved
score.go Outdated Show resolved Hide resolved
score.go Outdated Show resolved Hide resolved
score.go Outdated Show resolved Hide resolved
@vyzo vyzo changed the base branch from feat/prune-px to master March 24, 2020 13:53
@vyzo
Copy link
Collaborator Author

vyzo commented Apr 18, 2020

Rebased on master for merging.

@vyzo vyzo merged commit ea5d2e6 into master Apr 18, 2020
@vyzo vyzo mentioned this pull request Apr 18, 2020
4 tasks
@daviddias daviddias deleted the feat/hardening branch April 18, 2020 13:53
@daviddias
Copy link
Member



YEaasasssss!!! Phenomenal work everyone!!!

@bitcard

This comment has been minimized.

@Stebalien

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.