From bcf0ae10b0c311d5921a9acebfbb4b4105ec4012 Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 14:31:47 +0200 Subject: [PATCH 01/23] move gossipsub-wip into the gossipsub directory --- pubsub/{ => gossipsub}/gossipsub-wip.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename pubsub/{ => gossipsub}/gossipsub-wip.md (100%) diff --git a/pubsub/gossipsub-wip.md b/pubsub/gossipsub/gossipsub-wip.md similarity index 100% rename from pubsub/gossipsub-wip.md rename to pubsub/gossipsub/gossipsub-wip.md From 96eb7a50633af179933e0f5500ac46874ab91f6a Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 14:36:11 +0200 Subject: [PATCH 02/23] move original specification to gossipsub-v1.0-old, gossipsub-wip to gossipsub-v1.0 --- pubsub/gossipsub/{README.md => gossipsub-v1.0-old.md} | 0 pubsub/gossipsub/{gossipsub-wip.md => gossipsub-v1.0.md} | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename pubsub/gossipsub/{README.md => gossipsub-v1.0-old.md} (100%) rename pubsub/gossipsub/{gossipsub-wip.md => gossipsub-v1.0.md} (100%) diff --git a/pubsub/gossipsub/README.md b/pubsub/gossipsub/gossipsub-v1.0-old.md similarity index 100% rename from pubsub/gossipsub/README.md rename to pubsub/gossipsub/gossipsub-v1.0-old.md diff --git a/pubsub/gossipsub/gossipsub-wip.md b/pubsub/gossipsub/gossipsub-v1.0.md similarity index 100% rename from pubsub/gossipsub/gossipsub-wip.md rename to pubsub/gossipsub/gossipsub-v1.0.md From 4a4f0e06fd8b9f93ec3b9e775ecda143b8d3d939 Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 14:41:39 +0200 Subject: [PATCH 03/23] add a README that points to the specifications. --- pubsub/gossipsub/README.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 pubsub/gossipsub/README.md diff --git a/pubsub/gossipsub/README.md b/pubsub/gossipsub/README.md new file mode 100644 index 000000000..77a2eb215 --- /dev/null +++ b/pubsub/gossipsub/README.md @@ -0,0 +1,28 @@ +# gossipsub: An extensible baseline pubsub protocol + + +Gossipsub is an extensible baseline pubsub protocol, +based on randomized topic meshes and gossip. It is a general purpose +pubsub protocol with moderate amplification factors and good scaling +properties. The protocol is designed to be extensible by more +specialized routers, which may add protocol messages and gossip in +order to provide behaviour optimized for specific application +profiles. + +## Specification + +- [gossipsub-v1.0](gossipsub-v1.0.md): v1.0 of the gossipsub protocol. + This is a revised specification, to use a more normative language. + The original v1.0 specification is [here](gossippsub-v1.0-old.md). +- [gossipsub-v1.1](gossipsub-v1.1.md): v1.1 of the gossipsub protocol. + +## Research Notes + +- [episub](episub.md): a research note on a protocol building on top of gossipsub to implement epidemic broadcast trees. + +## Implementation status + +- Go: [libp2p/go-libp2p-pubsub/gossipsub.go](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) (experimental) +- JS: [ChainSafeSystems/gossipsub-js](https://github.com/ChainSafeSystems/gossipsub-js) – work in progress; check branches and PRs. +- Rust: [libp2p/rust-libp2p#898](https://github.com/libp2p/rust-libp2p/pull/898) implements the spec but is missing some features. [libp2p/rust-libp2p#767](https://github.com/libp2p/rust-libp2p/pull/767) is an alternative, partial implementation that differs slightly from the spec (see [#142](https://github.com/libp2p/specs/issues/142) for details). +- Gerbil: [vyzo/gerbil-simsub](https://github.com/vyzo/gerbil-simsub) (simulator) From 8e4aa08f7fce70efa9f9de915e38c373ed482c23 Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 14:52:48 +0200 Subject: [PATCH 04/23] add editor to gossipsub-v1.0 spec --- pubsub/gossipsub/gossipsub-v1.0.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/pubsub/gossipsub/gossipsub-v1.0.md b/pubsub/gossipsub/gossipsub-v1.0.md index 0ac8b7144..fb47152e1 100644 --- a/pubsub/gossipsub/gossipsub-v1.0.md +++ b/pubsub/gossipsub/gossipsub-v1.0.md @@ -7,6 +7,8 @@ Authors: [@vyzo] +Editor: [@yusefnapora] + Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], [@jamesray1], [@vasco-santos] @@ -96,11 +98,11 @@ The floodsub routing strategy has the following highly desirable properties: long as the overlay is sufficiently well connected. - It is highly robust; there is very little maintenance logic or state to manage. - + The problem however is that messages don't just follow the minimum latency paths; they follow all edges, thus creating a flood. The outbound degree of the network is unbounded, whereas we want it to be bounded in order to reduce -bandwidth requirements and increase decentralization and scalability. +bandwidth requirements and increase decentralization and scalability. This unbounded outbound degree creates a problem for individual densely connected nodes, as they may have a large number of connected peers and cannot @@ -499,7 +501,7 @@ for tuning the tradeoff between eager and lazy transmission of messages. The gossipsub protocol extends the [existing `RPC` message structure][pubsub-spec-rpc] with a new field, `control`. This is an instance of -`ControlMessage` which may contain one or more control messages. +`ControlMessage` which may contain one or more control messages. The four control messages are `ControlIHave` for [`IHAVE`](#ihave) messages, `ControlIWant` for [`IWANT`](#iwant) messages, `ControlGraft` for From ee4090a507ba16cbf00ca9af1222aa47afd750db Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 15:00:14 +0200 Subject: [PATCH 05/23] gossipsub-v.1.1 stub --- pubsub/gossipsub/gossipsub-v1.0.md | 2 +- pubsub/gossipsub/gossipsub-v1.1.md | 36 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 pubsub/gossipsub/gossipsub-v1.1.md diff --git a/pubsub/gossipsub/gossipsub-v1.0.md b/pubsub/gossipsub/gossipsub-v1.0.md index fb47152e1..542660ae9 100644 --- a/pubsub/gossipsub/gossipsub-v1.0.md +++ b/pubsub/gossipsub/gossipsub-v1.0.md @@ -2,7 +2,7 @@ | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|----------------|--------|-----------------| -| 3A | Recommendation | Active | r1, 2018-08-29 | +| 3A | Recommendation | Active | r2, 2020-03-12 | Authors: [@vyzo] diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md new file mode 100644 index 000000000..64c4c084f --- /dev/null +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -0,0 +1,36 @@ +# gossipsub v1.1: gossipsub extensions to improve bootstrapping and attack resistance + +| Lifecycle Stage | Maturity | Status | Latest Revision | +|-----------------|----------------|--------|-----------------| +| 1 | Draft | Active | r1, 2020-03-20 | + + +Authors: [@vyzo] + +Editor: [@yusefnapora] + +Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], [@daviddias] + +[@whyrusleeping]: https://github.com/whyrusleeping +[@yusefnapora]: https://github.com/yusefnapora +[@raulk]: https://github.com/raulk +[@vyzo]: https://github.com/vyzo +[@Stebalien]: https://github.com/Stebalien +[@daviddias]: https://github.com/daviddias + +See the [lifecycle document][lifecycle-spec] for context about maturity level +and spec status. + +[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md + +--- + +## Overview + +## Peer Exchange + +## Flood Publishing + +## Adaptive Gossip Dissemination + +## Peer Scoring From 82e4c545eff3a336466c765608a5d85d0aabf0f1 Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 20:28:05 +0200 Subject: [PATCH 06/23] overview and px --- pubsub/gossipsub/gossipsub-v1.1.md | 47 ++++++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index 64c4c084f..46da45cf4 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -2,13 +2,11 @@ | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|----------------|--------|-----------------| -| 1 | Draft | Active | r1, 2020-03-20 | +| 1A | Draft | Active | r1, 2020-03-20 | Authors: [@vyzo] -Editor: [@yusefnapora] - Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], [@daviddias] [@whyrusleeping]: https://github.com/whyrusleeping @@ -27,8 +25,51 @@ and spec status. ## Overview +This document specifies extensions to [gossipsub v1.0](gossipsub-v1.0.md) intended to improve +bootstrapping and protocol attack resistant. The extensions change the algorithms that +prescribe local peer behaviour and are fully backwards compatible with v1.0 of the protocol. +Peers that implement these extensions, advertise v1.1 of the protocol using `/meshsub/1.1.0` +as the protocol string. + ## Peer Exchange +Gossipsub relies on ambient peer discovery in order to find peers within a topic of interest. +This puts pressure to the implementation of a scalable peer discovery service that +can support the protocol. With Peer Exchange, the protocol can now bootstrap from a small +set of nodes, without relying on an external peer discovery service. + +Peer Exchange (PX) kicks in when pruning a mesh because of oversubscription. Instead of simply +telling the pruned peer to go away, the pruning peer provides a set of other peers where the +pruned peer can connect to reform its mesh. In addition, both the pruned and the pruning peer +add a backoff period from each other, within which they will not try to regraft. Both the pruning +and the pruned peer will immediate prune a `GRAFT` within the backoff period. +The recommended duration for the back period is 1 minute. + +In order to implement PX, we extend the `PRUNE` control message to include an optional set of +peers the pruned peer can connect to. This set of includes the Peer ID and a [_signed_ peer +record](https://github.com/libp2p/specs/pull/217) for each peer exchanged. +In order to facilitate transion to the usage of signed peer records within the libp2p ecosystem, +the emitting peer is allowed to omit the signed peer record if it doesn't have one. +In this case, the pruned peer will have to utilize an external service to discover addresses for +the peer, eg the DHT. + +### Protobuf Changes + +The `ControlPrune` message is extended with a `peer` field as follows. + +```protobuf +message ControlPrune { + optional string topicID = 1; + repeated PeerInfo peers = 2; // gossipsub v1.1 PX +} + +message PeerInfo { + optional bytes peerID = 1; + optional bytes signedPeerRecord = 2; +} + +``` + ## Flood Publishing ## Adaptive Gossip Dissemination From 8cb2f4088680dbf9b186c3ea9a804de815e0a135 Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 21:12:13 +0200 Subject: [PATCH 07/23] flood publishing --- pubsub/gossipsub/gossipsub-v1.1.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index 46da45cf4..b3457c853 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -53,7 +53,7 @@ the emitting peer is allowed to omit the signed peer record if it doesn't have o In this case, the pruned peer will have to utilize an external service to discover addresses for the peer, eg the DHT. -### Protobuf Changes +### Protobuf The `ControlPrune` message is extended with a `peer` field as follows. @@ -72,6 +72,19 @@ message PeerInfo { ## Flood Publishing +In gossipsub v1.0 a freshly published message is propagated through the mesh or the fanout map +if the publisher is not subscribed to the topic. In gossipsub v1.1 publishing is (optionally) +done by publishing the message to all connected peers with a score above a publish threshold +(see Peer Scoring below). + +This behaviour is prescribed to counter eclipse attacks and ensure that a newly published message +from a honest node will reach all connected honest nodes and get out to the network at large. +When flood publishing is in use there is no point in utilizing a fanout map or emitting gossip when +the peer is a pure publisher not subscribed in the topic. + +This behaviour also reduces message propagation latency as the message is injected to more points +in the network. + ## Adaptive Gossip Dissemination ## Peer Scoring From d7985731b107654a3f6493ec77a082cab488df30 Mon Sep 17 00:00:00 2001 From: vyzo Date: Thu, 12 Mar 2020 21:30:02 +0200 Subject: [PATCH 08/23] adaptive gossip dissemination --- pubsub/gossipsub/gossipsub-v1.1.md | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index b3457c853..d30cb88fe 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -39,7 +39,7 @@ can support the protocol. With Peer Exchange, the protocol can now bootstrap fro set of nodes, without relying on an external peer discovery service. Peer Exchange (PX) kicks in when pruning a mesh because of oversubscription. Instead of simply -telling the pruned peer to go away, the pruning peer provides a set of other peers where the +telling the pruned peer to go away, the pruning peer _may_ provide a set of other peers where the pruned peer can connect to reform its mesh. In addition, both the pruned and the pruning peer add a backoff period from each other, within which they will not try to regraft. Both the pruning and the pruned peer will immediate prune a `GRAFT` within the backoff period. @@ -87,4 +87,23 @@ in the network. ## Adaptive Gossip Dissemination +In gossipsub v1.0 gossip is emitted to a fixed number of peers, as specified by the `D_lazy` +parameter. In gossipsub v1.1 the disemmination of gossip is adaptive; instead of emitting gossip +to a fixed number of peers, we emit gossip to a percentage of our peers with a minimum of `D_lazy` +peers. + +The parameter controlling the emission of gossip is called the gossip _factor_. When a node wants +to emit gossip during the heartbeat, first it selects all peers with a peer score above a gossip +threshold (see Peer Scoring below). From these peers, it randomly selects gossip factor peers with +a minimum of `D_lazy`, and emits gossip to the selected peers. + +The recommended value for the gossip factor is `0.25`, which with the default of 3 rounds of gossip +per message ensures that each peer has at least 50% chance of receiving gossip about a message. +More specifically, for 3 rounds of gossip, the probability of a peer _not_ receiving gossip about +a fresh message is `(3/4)³=27/64=0.421875`. So each peer receives gossip about a fresh message with +a `0.578125` probability. + +This behaviour is prescribed to counter sybil attacks and ensures that a message from a honest +node propagates in the network with high probability. + ## Peer Scoring From 93d58e10396990d978bbbb7b467aa468de943b60 Mon Sep 17 00:00:00 2001 From: vyzo Date: Fri, 13 Mar 2020 12:56:29 +0200 Subject: [PATCH 09/23] peer scoring preliminaries --- pubsub/gossipsub/gossipsub-v1.1.md | 35 ++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index d30cb88fe..c70a752a2 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -107,3 +107,38 @@ This behaviour is prescribed to counter sybil attacks and ensures that a message node propagates in the network with high probability. ## Peer Scoring + +In gossipsub v1.1 we introduce a peer scoring component: each individual peer maintains a score +for other peers. The score is locally computed by each individual peer based on observed behaviour +and is not shared. The score is a real value, computed as weighted mix of parameters, +with pluggable application specific scoring. The score is computed across all (configured) topics +with a weighted mix, such that faulty behaviour in one topic percolates to other topics. +Furthermore, the score is retained for some period of time when a peer disconnects, so that malicious +peers cannot easily reset their score when it drops to negative and well behaving +peers don't lose their status because of a disconnection. + +The intention is to detect malicious or faulty behaviour and penalize the misbehaving peers +with a negative score. + +### Score Thresholds + +The score is plugged into various gossipsub algorithms such that peers with negative scores are +removed from the mesh. Peers with heavily negative score are further penalized or even ignored +if the score drops too low. + +More specifically, the following thresholds apply: +- `0`: the baseline threshold; peers with a score below this threshold are pruned from the mesh + during the heartbeat and ignored when looking for peers to graft. Furthermore, no PX information + is emitted towards those peers and PX is ignored from them. +- `gossipThreshold`: when a peer's score drops below this threshold, no gossip is emitted towards + that peer and gossip from that peer is ignored. This threshold should be negative, such that + some information can be propagated to/from mildly negatively scoring peers. +- `publishThreshold`: when a peer's score drops below this threshold, self published messages are + not propagated towards this peer when (flood) publishing. This threshold should be negative, and + less than or equal to the gossip threshold. +- `graylistThreshold`: when a peer's score drops below this threshold, the peer is graylisted and + its RPCs are ignored. This threshold must be negative, and less than the gossip/publish threshold. + + + +## Spam Protection From b65be4eb18c0f4cc03290fda33f894fd5be969f8 Mon Sep 17 00:00:00 2001 From: vyzo Date: Fri, 13 Mar 2020 13:38:05 +0200 Subject: [PATCH 10/23] peer scoring function --- pubsub/gossipsub/gossipsub-v1.1.md | 58 ++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index c70a752a2..fad129740 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -139,6 +139,64 @@ More specifically, the following thresholds apply: - `graylistThreshold`: when a peer's score drops below this threshold, the peer is graylisted and its RPCs are ignored. This threshold must be negative, and less than the gossip/publish threshold. +### Heartbeat Maintenance +The score is checked explicitly during heartbeat maintenance such that: +- Peers with negative score are pruned from all meshes. +- When pruning because of oversubscription, the peer keeps the best `D_score` scoring peers and + selects the remaining peers to keep at random. This protects the mesh from takeover attacks + and ensures that the best scoring peers are kept in the mesh. At the same time, we do keep some + peers as random so that the protocol is responsive to new peers joining the mesh. +- When selecting peers to graft because of undersubscription, peers with a negative score are ignored. + +### The Score Function + +The score function is a weighted mix of parameters, 4 of them per topic and 2 of them globally +applicable. +``` +Score(p) = Σtᵢ*(w₁(tᵢ)*P₁(tᵢ) + w₂(tᵢ)*P₂(tᵢ) + w₃(tᵢ)*P₃(tᵢ) + w₃b(tᵢ)*P₃b(tᵢ) + w₄(tᵢ)*P₄(tᵢ)) + w₅*P₅ + w₆*P₆ +``` +where `tᵢ` is the topic weight for each topic where per topic parameters apply. + +The parameters are defined as follows: +- P₁: time in mesh for a topic. This is the time a peer has been in the mesh, capped to a small value + and mixed with a small positive weight. This is intended to boost peers already in the mesh so that + they are not prematurely pruned because of oversubscription. +- P₂: first message deliveries for a topic. This is the number of message first delivered by the peer + in the topic, mixed with a positive weight. This is intended to reward peers who first forward a + valid message. +- P₃: mesh message delivery rate for a topic. This parameter is a threshold for the expected message + delivery rate within the mesh in the topic. If the number of deliveries is above the threshold, + then the value is 0. If the number is below the threshold, then the value of the parameter is + the square of the deficit. + This is intended to penalize peers in the mesh who are not delivering the expected + number of messages so that they can be removed from the mesh. The parameter is mixed with a negative + weight. +- P₃b: mesh message delivery failures for a topic. This is a sticky parameter that counts the number + of mesh message delivery failures. Whenever a peer is pruned with a negative score, the parameter + is augmented by the rate deficit at the time of prune. This is intended to keep history of prunes + so that a peer that was pruned because of underdelivery cannot quickly get regrafted into the + mesh. The parameter is mixed with negative weight. +- P₄: invalid messages for a topic. This is he number of invalid messages delivered in the topic. + This is intended to penalize peers who transmit invalid messages, according to application specific + validation rules. It is mixed with a negative weight. +- P₅: application specific score. This is the score component assigned to the peer by the application + itself, using application specific rules. The weight is positive, but the parameter itself has an + arbitrary real value, so that the application can signal misbehaviour with a negative score or gate + peers before an application specific handshake is completed. +- P₆: IP colocation factor. This parameter is a threshold for the number of peers using the same IP + address. If the number of peers in the same IP exceeds the threshold, then the value is the square + of the surpluss, otherwise it is 0. This is intended to make it difficult to carry out sybil attacks + by using a small number of IPs. The parameter is mixed with a negative weight. + +### Topic Parameter Calculation and Decay + +TBD + +### Guidelines for Tuning the Scoring Function + +TBD ## Spam Protection + +TBD From 064250f745dcf34ed432e002db3724583250e3e4 Mon Sep 17 00:00:00 2001 From: vyzo Date: Fri, 13 Mar 2020 18:30:51 +0200 Subject: [PATCH 11/23] Update pubsub/gossipsub/gossipsub-v1.0-old.md interest group Co-Authored-By: David Dias --- pubsub/gossipsub/gossipsub-v1.0-old.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pubsub/gossipsub/gossipsub-v1.0-old.md b/pubsub/gossipsub/gossipsub-v1.0-old.md index 1b8d928ee..c20dd6d3b 100644 --- a/pubsub/gossipsub/gossipsub-v1.0-old.md +++ b/pubsub/gossipsub/gossipsub-v1.0-old.md @@ -8,7 +8,7 @@ Authors: [@vyzo] Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], -[@jamesray1], [@vasco-santos] +[@jamesray1], [@vasco-santos], [@daviddias], [@yiannisbot] [@whyrusleeping]: https://github.com/whyrusleeping [@yusefnapora]: https://github.com/yusefnapora From c99c7bf8b0af9d855d95902d8054fde643b3047b Mon Sep 17 00:00:00 2001 From: vyzo Date: Fri, 13 Mar 2020 18:31:12 +0200 Subject: [PATCH 12/23] Update pubsub/gossipsub/gossipsub-v1.0-old.md interest group Co-Authored-By: David Dias --- pubsub/gossipsub/gossipsub-v1.0-old.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pubsub/gossipsub/gossipsub-v1.0-old.md b/pubsub/gossipsub/gossipsub-v1.0-old.md index c20dd6d3b..64018d9d5 100644 --- a/pubsub/gossipsub/gossipsub-v1.0-old.md +++ b/pubsub/gossipsub/gossipsub-v1.0-old.md @@ -17,6 +17,8 @@ Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], [@Stebalien]: https://github.com/Stebalien [@jamesray1]: https://github.com/jamesray1 [@vasco-santos]: https://github.com/vasco-santos +[@daviddias]: https://github.com/daviddias +[@yiannisbot]: https://github.com/yiannisbot See the [lifecycle document][lifecycle-spec] for context about maturity level and spec status. From c3f172d8bd7ad2c68b8f075e9c85d7b3b3bc5ac5 Mon Sep 17 00:00:00 2001 From: vyzo Date: Fri, 13 Mar 2020 18:31:29 +0200 Subject: [PATCH 13/23] Update pubsub/gossipsub/gossipsub-v1.0-old.md Co-Authored-By: David Dias --- pubsub/gossipsub/gossipsub-v1.0-old.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/pubsub/gossipsub/gossipsub-v1.0-old.md b/pubsub/gossipsub/gossipsub-v1.0-old.md index 64018d9d5..70258437e 100644 --- a/pubsub/gossipsub/gossipsub-v1.0-old.md +++ b/pubsub/gossipsub/gossipsub-v1.0-old.md @@ -60,12 +60,6 @@ profiles. -## Implementation status - -- Go: [libp2p/go-libp2p-pubsub/gossipsub.go](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) (experimental) -- JS: [ChainSafeSystems/gossipsub-js](https://github.com/ChainSafeSystems/gossipsub-js) – work in progress; check branches and PRs. -- Rust: [libp2p/rust-libp2p#898](https://github.com/libp2p/rust-libp2p/pull/898) implements the spec but is missing some features. [libp2p/rust-libp2p#767](https://github.com/libp2p/rust-libp2p/pull/767) is an alternative, partial implementation that differs slightly from the spec (see [#142](https://github.com/libp2p/specs/issues/142) for details). -- Gerbil: [vyzo/gerbil-simsub](https://github.com/vyzo/gerbil-simsub) (simulator) ## In the beginning was floodsub From 3c0d7dfd6684a9ba70f35fb59603290ebd965729 Mon Sep 17 00:00:00 2001 From: vyzo Date: Fri, 13 Mar 2020 19:12:26 +0200 Subject: [PATCH 14/23] score parameter calculation --- pubsub/gossipsub/gossipsub-v1.1.md | 163 ++++++++++++++++++++++++++++- 1 file changed, 162 insertions(+), 1 deletion(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index fad129740..d52d7a57e 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -191,7 +191,168 @@ The parameters are defined as follows: ### Topic Parameter Calculation and Decay -TBD +The topic parameters are implemented using counters maintained internally by the router +whenever an event of interest occurs. The counters _decay_ periodically so that their values are +not continuously increasing and ensure that a large positive or negative score isn't sticky for +the lifetime of the peer. + +#### P₁: Time in Mesh + +In order to compute P₁, the router records the time when the peer is GRAFTed. The time in mesh +is calculated lazily during the decay update to avoid a large number of calls to `gettimeofday`. +The parameter value is the division of the time ellapsed since the GRAFT with an application +configurable quantum. + +In pseudo-go: +```go +// topic configuration parameters +var TimeInMeshQuantum time.Duration +var TimeInMeshCap float64 + +// lazily updated time in mesh +var meshTime time.Duration + +// P₁ +p1 := float64(meshTime / TimeInMeshQuantum) +if p1 > TimeInMeshCap { + p1 = TimeInMeshCap +} +``` + +#### P₂: First Message Deliveries + +In order to compute P₂, the router maintains a counter that increments whenever a message +is first delivered in the topic by the peer. The parameter has a cap that applies at the time +of increment. + +In pseudo-go: +```go +// topic configuration parameters +var FirstMessageDeliveriesCap float64 + +// couner updated every time a first message delivery occurs +var firstMessageDeliveries float64 + +// counter update +firstMessageDeliveries += 1 +if firstMessageDeliveries > FirstMessageDeliveriesCap { + firstMessageDeliveries = FirstMessageDeliveriesCap +} + +// P₂ +p2 := firstMessageDeliveries +``` + +#### P₃ and P₃b: Mesh Message Delivery + +In order to compute P₃, the router maintains a counter that increments whenever a first +or near-first message delivery occurs in the topic by a peer in the mesh. A near-first message +delivery is a message delivery that occurs while a message has been first received and is being +validated or it has been received within a configurable window of validation of first message +delivery. The window is configurable but should be small (in the order of milliseconds) to avoid +allowing a mesh peer to build score by simply replaying back the messages received by the current +router. The parameter has a cap that applies at the time of increment. + +In order to avoid triggering the penalty too early, the parameter has an activation window. +This is a configurable value that is the time that the peer must have been in the mesh before +the parameter applies. + +In pseudo-go: +```go +// topic configuration parameters +var MeshMessageDeliveriesCap, MeshMessageDeliveriesThreshold float64 +var MeshMessageDeliveriesWindow, MeshMessageDeliveriesActivation time.Duration + +// time in mesh, lazily updated +var meshTime time.Duration + +// counter updated every time a first or near-first message delivery occurs by a mesh peer +var meshMessageDeliveries float64 + +// counter update +meshMessageDeliveries += 1 +if meshMessageDeliveries > MeshMessageDeliveriesCap { + meshMessageDeliveries = MeshMessageDeliveriesCap +} + +// calculation of P₃ +var deficit float64 +if meshTime > MeshMessageDeliveriesActivation && meshMessageDeliveries < MeshMessageDeliveriesThreshold { + deficit = MeshMessageDeliveriesThreshold - meshMessageDeliveries +} + +p3 := deficit * deficit +``` + +In order to calculate P₃b, the router maintains a counter that is updated whenever the peer is pruned +with an active deficit in message delivery. The parameter is uncapped. + +In pseudo-go: +```go +// counter updated at prune time +var meshFailurePenalty float64 + +// counter update +if meshTime > MeshMessageDeliveriesActivation && meshMessageDeliveries < MeshMessageDeliveriesThreshold { + deficit = MeshMessageDeliveriesThreshold - meshMessageDeliveries + meshFailurePenalty += deficit * deficit +} + +// P₃b +p3b := meshFailurePenalty +``` + +#### P₄: Invalid Messages + +In order to compute P₄, the router maintains a counter that increments whenever a message fails +validation. The counter is uncapped. + +In pseudo-go: +```go +// counter updated every time a message fails validation +var invalidMessageDeliveries float64 + +// counter update +invalidMessageDeliveries += 1 + +// P₄ +p4 := invalidMessageDeliveries +``` + +#### Parameter Decay + +The counters associated with P₂, P₃, P₃b, and P₄ decay periodically by multiplying with a configurable +decay factor. When the value drops below a threshold it is considered zero. + +In pseudo-go: +```go +// decay factors +var FirstMessageDeliveriesDecay, MeshMessageDeliveriesDecay, MeshFailurePenaltyDecay, InvalidMessageDeliveriesDecay float64 + +// 0-threshold +var DecayToZero float64 + +// periodic decay of counters +firstMessageDeliveries *= FirstMessageDeliveriesDecay +if firstMessageDeliveries < DecayToZero { + firstMessageDeliveries = 0 +} + +meshMessageDeliveries *= MeshMessageDeliveriesDecay +if meshMessageDeliveries < DecayToZero { + meshMessageDeliveries = 0 +} + +meshFailurePenalty *= MeshFailurePenaltyDecay +if meshFailurePenalty < DecayToZero { + meshFailurePenalty = 0 +} + +invalidMessageDeliveries *= InvalidMessageDeliveriesDecay +if invalidMessageDeliveries < DecayToZero { + invalidMessageDeliveries = 0 +} +``` ### Guidelines for Tuning the Scoring Function From 87ca0537a77acfcd3a71c9cb806d2e00c32249d3 Mon Sep 17 00:00:00 2001 From: vyzo Date: Fri, 13 Mar 2020 20:23:07 +0200 Subject: [PATCH 15/23] spam protection measures --- pubsub/gossipsub/gossipsub-v1.1.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index d52d7a57e..e8cafd486 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -358,6 +358,17 @@ if invalidMessageDeliveries < DecayToZero { TBD -## Spam Protection - -TBD +## Spam Protection Measures + +In order counter spam that elicits responses and consumes resources, some measures have been taken: +- `GRAFT` messages for unknown topics are ignored; in gossipsub v1.0 the router would always + respond with a `PRUNE`, which opens up an avenue for flooding with spam `GRAFT` messages and + consuming resources. +- `IWANT` message responses are limited in the number of retransmissions to a certain peer; + in gossipsub v1.0 the router always responds to `IWANT` messages when the message in the cache. + In gossipsub v1.1 the router responds a limited number of times to each peer so that `IWANT` spam + does not cause a signficant drain of resources. +- Invalid message spam, either directly transmitted or as a response to an `IHAVE` message is + penalized by the score function. A peer transmitting lots of spam will quickly get graylisted, + reducing the surface of spam-induced computation (eg validation). The application can take + further steps and blacklist the peer if the spam persists after the negative score decays. From a38055fb504bc5680ee6d226650ef3ff631520d9 Mon Sep 17 00:00:00 2001 From: David Dias Date: Mon, 16 Mar 2020 07:39:44 +0000 Subject: [PATCH 16/23] docs: gossipsub specs review --- pubsub/gossipsub/README.md | 11 ++--- pubsub/gossipsub/gossipsub-v1.0-old.md | 55 +++++++++++----------- pubsub/gossipsub/gossipsub-v1.0.md | 8 +--- pubsub/gossipsub/gossipsub-v1.1.md | 65 +++++++++++++++++++------- 4 files changed, 79 insertions(+), 60 deletions(-) diff --git a/pubsub/gossipsub/README.md b/pubsub/gossipsub/README.md index 77a2eb215..3c20e2613 100644 --- a/pubsub/gossipsub/README.md +++ b/pubsub/gossipsub/README.md @@ -1,6 +1,5 @@ # gossipsub: An extensible baseline pubsub protocol - Gossipsub is an extensible baseline pubsub protocol, based on randomized topic meshes and gossip. It is a general purpose pubsub protocol with moderate amplification factors and good scaling @@ -11,14 +10,10 @@ profiles. ## Specification -- [gossipsub-v1.0](gossipsub-v1.0.md): v1.0 of the gossipsub protocol. - This is a revised specification, to use a more normative language. - The original v1.0 specification is [here](gossippsub-v1.0-old.md). +- [gossipsub-v1.0](gossipsub-v1.0.md): v1.0 of the gossipsub protocol. This is a revised specification, to use a more normative language. + - The original v1.0 specification is [here](gossippsub-v1.0-old.md). - [gossipsub-v1.1](gossipsub-v1.1.md): v1.1 of the gossipsub protocol. - -## Research Notes - -- [episub](episub.md): a research note on a protocol building on top of gossipsub to implement epidemic broadcast trees. +- [(not in use) episub](episub.md): a research note on a protocol building on top of gossipsub to implement [epidemic broadcast trees](https://www.gsd.inesc-id.pt/~ler/reports/srds07.pdf). ## Implementation status diff --git a/pubsub/gossipsub/gossipsub-v1.0-old.md b/pubsub/gossipsub/gossipsub-v1.0-old.md index 70258437e..f3dbb29ed 100644 --- a/pubsub/gossipsub/gossipsub-v1.0-old.md +++ b/pubsub/gossipsub/gossipsub-v1.0-old.md @@ -1,14 +1,13 @@ -# gossipsub: An extensible baseline pubsub protocol +# gossipsub v1.0 (OLD): An extensible baseline pubsub protocol + +> `DISCLAIMER:` This is the original specification, please refer to [gossipsub-v1.0](gossipsub-v1.0.md) from now on | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|----------------|--------|-----------------| | 3A | Recommendation | Active | r1, 2018-08-29 | - Authors: [@vyzo] - -Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], -[@jamesray1], [@vasco-santos], [@daviddias], [@yiannisbot] +Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], [@jamesray1], [@vasco-santos], [@daviddias], [@yiannisbot] [@whyrusleeping]: https://github.com/whyrusleeping [@yusefnapora]: https://github.com/yusefnapora @@ -39,30 +38,27 @@ profiles. **Contents** -- [gossipsub: An extensible baseline pubsub protocol](#gossipsub-an-extensible-baseline-pubsub-protocol) - - [Implementation status](#implementation-status) - - [In the beginning was floodsub](#in-the-beginning-was-floodsub) - - [Ambient Peer Discovery](#ambient-peer-discovery) - - [Flood routing](#flood-routing) - - [Retrospective](#retrospective) - - [Controlling the flood](#controlling-the-flood) - - [randomsub: A random message router](#randomsub-a-random-message-router) - - [meshsub: An overlay mesh router](#meshsub-an-overlay-mesh-router) - - [gossipsub: The gossiping mesh router](#gossipsub-the-gossiping-mesh-router) - - [The gossipsub protocol](#the-gossipsub-protocol) - - [Control messages](#control-messages) - - [Router state](#router-state) - - [Topic membership](#topic-membership) - - [Message processing](#message-processing) - - [Heartbeat](#heartbeat) - - [Control message piggybacking](#control-message-piggybacking) - - [Protobuf](#protobuf) +- [Implementation status](#implementation-status) +- [Context - In the beginning was floodsub](#context--in-the-beginning-was-floodsub) + - [Ambient Peer Discovery](#ambient-peer-discovery) + - [Flood routing](#flood-routing) + - [Retrospective](#retrospective) +- [Proposed alternatives - Controlling the flood](#proposed-alternatives--controlling-the-flood) + - [randomsub: A random message router](#randomsub-a-random-message-router) + - [meshsub: An overlay mesh router](#meshsub-an-overlay-mesh-router) + - [gossipsub: The gossiping mesh router](#gossipsub-the-gossiping-mesh-router) +- [Protocol Architecture - Gossipsub](#protocol-architecture--gossipsub) + - [Control messages](#control-messages) + - [Router state](#router-state) + - [Topic membership](#topic-membership) + - [Message processing](#message-processing) + - [Heartbeat](#heartbeat) + - [Control message piggybacking](#control-message-piggybacking) + - [Protobuf](#protobuf) - - -## In the beginning was floodsub +## Context - In the beginning was floodsub The initial pubsub experiment in libp2p was `floodsub`. It implements pubsub in the most basic manner, with two defining aspects: @@ -125,8 +121,7 @@ the amplification factor is only bounded by the sum of degrees of all nodes in the overlay, which creates a scaling problem for densely connected overlays at large. - -## Controlling the flood +## Proposed alternatives - Controlling the flood In order to scale pubsub without excessive bandwidth waste or peer overload, we need a router that bounds the degree of each peer and @@ -205,13 +200,15 @@ with the meshsub construction and augments it using gossip propagation of metadata with the randomsub technique. -## The gossipsub protocol +## Protocol Architecture - Gossipsub We can now provide a specification of the pubsub protocol by sketching out the router implementation. The router is backwards compatible with floodsub, as it accepts floodsub peers and behaves like floodsub towards them. +If you would like to get a video presentation and visualization on Gossipsub, watch [Scalable PubSub with GossipSub - Dimitris Vyzovitis](https://www.youtube.com/watch?v=mlrf1058ENY&index=3&list=PLuhRWgmPaHtRPl3Itt_YdHYA0g0Eup8hQ) from the [IPFS London Hack Week of 2018 Q4](http://gateway.ipfs.io/ipns/blog.ipfs.io/65-london-hack-week-report/). + ### Control messages The protocol defines four control messages: diff --git a/pubsub/gossipsub/gossipsub-v1.0.md b/pubsub/gossipsub/gossipsub-v1.0.md index 542660ae9..2fb019f72 100644 --- a/pubsub/gossipsub/gossipsub-v1.0.md +++ b/pubsub/gossipsub/gossipsub-v1.0.md @@ -1,16 +1,12 @@ -# gossipsub: An extensible baseline pubsub protocol +# gossipsub v1.0: An extensible baseline pubsub protocol | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|----------------|--------|-----------------| | 3A | Recommendation | Active | r2, 2020-03-12 | - Authors: [@vyzo] - Editor: [@yusefnapora] - -Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], -[@jamesray1], [@vasco-santos] +Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], [@jamesray1], [@vasco-santos], [@daviddias], [@yiannisbot] [@whyrusleeping]: https://github.com/whyrusleeping [@yusefnapora]: https://github.com/yusefnapora diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index e8cafd486..06c33e2f6 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -1,4 +1,4 @@ -# gossipsub v1.1: gossipsub extensions to improve bootstrapping and attack resistance +# gossipsub v1.1: Security extensions to improve on attack resilience and bootstrapping | Lifecycle Stage | Maturity | Status | Latest Revision | |-----------------|----------------|--------|-----------------| @@ -23,6 +23,31 @@ and spec status. --- + + + +- [Overview](#overview) +- [Attack Vectors considered](#attack-vectors-considered) +- [Protocol extensions](#protocol-extensions) + - [Peer Exchange](#peer-exchange) + - [Protobuf](#protobuf) + - [Flood Publishing](#flood-publishing) + - [Adaptive Gossip Dissemination](#adaptive-gossip-dissemination) + - [Peer Scoring](#peer-scoring) + - [Score Thresholds](#score-thresholds) + - [Heartbeat Maintenance](#heartbeat-maintenance) + - [The Score Function](#the-score-function) + - [Topic Parameter Calculation and Decay](#topic-parameter-calculation-and-decay) + - [P₁: Time in Mesh](#p%E2%82%81-time-in-mesh) + - [P₂: First Message Deliveries](#p%E2%82%82-first-message-deliveries) + - [P₃ and P₃b: Mesh Message Delivery](#p%E2%82%83-and-p%E2%82%83b-mesh-message-delivery) + - [P₄: Invalid Messages](#p%E2%82%84-invalid-messages) + - [Parameter Decay](#parameter-decay) + - [Guidelines for Tuning the Scoring Function](#guidelines-for-tuning-the-scoring-function) + - [Spam Protection Measures](#spam-protection-measures) + + + ## Overview This document specifies extensions to [gossipsub v1.0](gossipsub-v1.0.md) intended to improve @@ -31,7 +56,13 @@ prescribe local peer behaviour and are fully backwards compatible with v1.0 of t Peers that implement these extensions, advertise v1.1 of the protocol using `/meshsub/1.1.0` as the protocol string. -## Peer Exchange +## Attack Vectors considered + +`To be written` + +## Protocol extensions + +### Peer Exchange Gossipsub relies on ambient peer discovery in order to find peers within a topic of interest. This puts pressure to the implementation of a scalable peer discovery service that @@ -53,7 +84,7 @@ the emitting peer is allowed to omit the signed peer record if it doesn't have o In this case, the pruned peer will have to utilize an external service to discover addresses for the peer, eg the DHT. -### Protobuf +#### Protobuf The `ControlPrune` message is extended with a `peer` field as follows. @@ -70,7 +101,7 @@ message PeerInfo { ``` -## Flood Publishing +### Flood Publishing In gossipsub v1.0 a freshly published message is propagated through the mesh or the fanout map if the publisher is not subscribed to the topic. In gossipsub v1.1 publishing is (optionally) @@ -85,7 +116,7 @@ the peer is a pure publisher not subscribed in the topic. This behaviour also reduces message propagation latency as the message is injected to more points in the network. -## Adaptive Gossip Dissemination +### Adaptive Gossip Dissemination In gossipsub v1.0 gossip is emitted to a fixed number of peers, as specified by the `D_lazy` parameter. In gossipsub v1.1 the disemmination of gossip is adaptive; instead of emitting gossip @@ -106,7 +137,7 @@ a `0.578125` probability. This behaviour is prescribed to counter sybil attacks and ensures that a message from a honest node propagates in the network with high probability. -## Peer Scoring +### Peer Scoring In gossipsub v1.1 we introduce a peer scoring component: each individual peer maintains a score for other peers. The score is locally computed by each individual peer based on observed behaviour @@ -120,7 +151,7 @@ peers don't lose their status because of a disconnection. The intention is to detect malicious or faulty behaviour and penalize the misbehaving peers with a negative score. -### Score Thresholds +#### Score Thresholds The score is plugged into various gossipsub algorithms such that peers with negative scores are removed from the mesh. Peers with heavily negative score are further penalized or even ignored @@ -139,7 +170,7 @@ More specifically, the following thresholds apply: - `graylistThreshold`: when a peer's score drops below this threshold, the peer is graylisted and its RPCs are ignored. This threshold must be negative, and less than the gossip/publish threshold. -### Heartbeat Maintenance +#### Heartbeat Maintenance The score is checked explicitly during heartbeat maintenance such that: - Peers with negative score are pruned from all meshes. @@ -149,7 +180,7 @@ The score is checked explicitly during heartbeat maintenance such that: peers as random so that the protocol is responsive to new peers joining the mesh. - When selecting peers to graft because of undersubscription, peers with a negative score are ignored. -### The Score Function +#### The Score Function The score function is a weighted mix of parameters, 4 of them per topic and 2 of them globally applicable. @@ -189,14 +220,14 @@ The parameters are defined as follows: of the surpluss, otherwise it is 0. This is intended to make it difficult to carry out sybil attacks by using a small number of IPs. The parameter is mixed with a negative weight. -### Topic Parameter Calculation and Decay +#### Topic Parameter Calculation and Decay The topic parameters are implemented using counters maintained internally by the router whenever an event of interest occurs. The counters _decay_ periodically so that their values are not continuously increasing and ensure that a large positive or negative score isn't sticky for the lifetime of the peer. -#### P₁: Time in Mesh +##### P₁: Time in Mesh In order to compute P₁, the router records the time when the peer is GRAFTed. The time in mesh is calculated lazily during the decay update to avoid a large number of calls to `gettimeofday`. @@ -219,7 +250,7 @@ if p1 > TimeInMeshCap { } ``` -#### P₂: First Message Deliveries +##### P₂: First Message Deliveries In order to compute P₂, the router maintains a counter that increments whenever a message is first delivered in the topic by the peer. The parameter has a cap that applies at the time @@ -243,7 +274,7 @@ if firstMessageDeliveries > FirstMessageDeliveriesCap { p2 := firstMessageDeliveries ``` -#### P₃ and P₃b: Mesh Message Delivery +##### P₃ and P₃b: Mesh Message Delivery In order to compute P₃, the router maintains a counter that increments whenever a first or near-first message delivery occurs in the topic by a peer in the mesh. A near-first message @@ -302,7 +333,7 @@ if meshTime > MeshMessageDeliveriesActivation && meshMessageDeliveries < MeshMes p3b := meshFailurePenalty ``` -#### P₄: Invalid Messages +##### P₄: Invalid Messages In order to compute P₄, the router maintains a counter that increments whenever a message fails validation. The counter is uncapped. @@ -319,7 +350,7 @@ invalidMessageDeliveries += 1 p4 := invalidMessageDeliveries ``` -#### Parameter Decay +##### Parameter Decay The counters associated with P₂, P₃, P₃b, and P₄ decay periodically by multiplying with a configurable decay factor. When the value drops below a threshold it is considered zero. @@ -354,11 +385,11 @@ if invalidMessageDeliveries < DecayToZero { } ``` -### Guidelines for Tuning the Scoring Function +#### Guidelines for Tuning the Scoring Function TBD -## Spam Protection Measures +### Spam Protection Measures In order counter spam that elicits responses and consumes resources, some measures have been taken: - `GRAFT` messages for unknown topics are ignored; in gossipsub v1.0 the router would always From e8a163f171439ee738c3b9a25b1d692b99c18634 Mon Sep 17 00:00:00 2001 From: vyzo Date: Mon, 16 Mar 2020 11:09:46 +0200 Subject: [PATCH 17/23] Update pubsub/gossipsub/gossipsub-v1.1.md Co-Authored-By: David Dias --- pubsub/gossipsub/gossipsub-v1.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index 06c33e2f6..03f80f0d9 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -62,7 +62,7 @@ as the protocol string. ## Protocol extensions -### Peer Exchange +### Peer Exchange on PRUNE Gossipsub relies on ambient peer discovery in order to find peers within a topic of interest. This puts pressure to the implementation of a scalable peer discovery service that From e7a4d3296a3535218f433576da1a64379f2a488c Mon Sep 17 00:00:00 2001 From: vyzo Date: Mon, 16 Mar 2020 12:54:53 +0200 Subject: [PATCH 18/23] some clarifications for PX --- pubsub/gossipsub/gossipsub-v1.1.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index 03f80f0d9..5316cad36 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -60,7 +60,7 @@ as the protocol string. `To be written` -## Protocol extensions +## Protocol extensions ### Peer Exchange on PRUNE @@ -71,13 +71,15 @@ set of nodes, without relying on an external peer discovery service. Peer Exchange (PX) kicks in when pruning a mesh because of oversubscription. Instead of simply telling the pruned peer to go away, the pruning peer _may_ provide a set of other peers where the -pruned peer can connect to reform its mesh. In addition, both the pruned and the pruning peer -add a backoff period from each other, within which they will not try to regraft. Both the pruning -and the pruned peer will immediate prune a `GRAFT` within the backoff period. -The recommended duration for the back period is 1 minute. +pruned peer can connect to reform its mesh (see Peer Scoring below). +In addition, both the pruned and the pruning peer add a backoff period from each other, within which +they will not try to regraft. Both the pruning and the pruned peer will immediate prune a `GRAFT` +within the backoff period. +The recommended duration for the backoff period is 1 minute, while the recommended number of peers +to exchange is equal to `D` so that the pruned peer can form a full mesh. In order to implement PX, we extend the `PRUNE` control message to include an optional set of -peers the pruned peer can connect to. This set of includes the Peer ID and a [_signed_ peer +peers the pruned peer can connect to. This set of peers includes the Peer ID and a [_signed_ peer record](https://github.com/libp2p/specs/pull/217) for each peer exchanged. In order to facilitate transion to the usage of signed peer records within the libp2p ecosystem, the emitting peer is allowed to omit the signed peer record if it doesn't have one. @@ -160,7 +162,8 @@ if the score drops too low. More specifically, the following thresholds apply: - `0`: the baseline threshold; peers with a score below this threshold are pruned from the mesh during the heartbeat and ignored when looking for peers to graft. Furthermore, no PX information - is emitted towards those peers and PX is ignored from them. + is emitted towards those peers and PX is ignored from them. In addition, when performing PX only + peers with non-negative scores are exchanged. - `gossipThreshold`: when a peer's score drops below this threshold, no gossip is emitted towards that peer and gossip from that peer is ignored. This threshold should be negative, such that some information can be propagated to/from mildly negatively scoring peers. From c681902b2d0979f80f4f2133e43bd3bee201b61a Mon Sep 17 00:00:00 2001 From: vyzo Date: Mon, 16 Mar 2020 19:12:59 +0200 Subject: [PATCH 19/23] fix typo Co-Authored-By: Vasco Santos --- pubsub/gossipsub/gossipsub-v1.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index 5316cad36..8efee54d9 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -81,7 +81,7 @@ to exchange is equal to `D` so that the pruned peer can form a full mesh. In order to implement PX, we extend the `PRUNE` control message to include an optional set of peers the pruned peer can connect to. This set of peers includes the Peer ID and a [_signed_ peer record](https://github.com/libp2p/specs/pull/217) for each peer exchanged. -In order to facilitate transion to the usage of signed peer records within the libp2p ecosystem, +In order to facilitate the transition to the usage of signed peer records within the libp2p ecosystem, the emitting peer is allowed to omit the signed peer record if it doesn't have one. In this case, the pruned peer will have to utilize an external service to discover addresses for the peer, eg the DHT. From 1ebd52bd6e85a3aab14609607e43788c200cbc49 Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 17 Mar 2020 11:18:05 +0100 Subject: [PATCH 20/23] chore: gossipsub-js status update --- pubsub/gossipsub/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pubsub/gossipsub/README.md b/pubsub/gossipsub/README.md index 3c20e2613..b92e0185f 100644 --- a/pubsub/gossipsub/README.md +++ b/pubsub/gossipsub/README.md @@ -18,6 +18,6 @@ profiles. ## Implementation status - Go: [libp2p/go-libp2p-pubsub/gossipsub.go](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) (experimental) -- JS: [ChainSafeSystems/gossipsub-js](https://github.com/ChainSafeSystems/gossipsub-js) – work in progress; check branches and PRs. +- JS: [ChainSafeSystems/gossipsub-js](https://github.com/ChainSafeSystems/gossipsub-js) (compliant with `gossipsub-v1.0`). - Rust: [libp2p/rust-libp2p#898](https://github.com/libp2p/rust-libp2p/pull/898) implements the spec but is missing some features. [libp2p/rust-libp2p#767](https://github.com/libp2p/rust-libp2p/pull/767) is an alternative, partial implementation that differs slightly from the spec (see [#142](https://github.com/libp2p/specs/issues/142) for details). - Gerbil: [vyzo/gerbil-simsub](https://github.com/vyzo/gerbil-simsub) (simulator) From 1e029d857e0bf833f62498cad7b54817a24b2990 Mon Sep 17 00:00:00 2001 From: vyzo Date: Tue, 17 Mar 2020 16:43:58 +0200 Subject: [PATCH 21/23] fix typo, toc, rename attack section --- pubsub/gossipsub/gossipsub-v1.1.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index 8efee54d9..0c84262d2 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -27,9 +27,9 @@ and spec status. - [Overview](#overview) -- [Attack Vectors considered](#attack-vectors-considered) +- [Attack Considerations](#attack-considerations) - [Protocol extensions](#protocol-extensions) - - [Peer Exchange](#peer-exchange) + - [Peer Exchange on PRUNE](#peer-exchange-on-prune) - [Protobuf](#protobuf) - [Flood Publishing](#flood-publishing) - [Adaptive Gossip Dissemination](#adaptive-gossip-dissemination) @@ -51,12 +51,12 @@ and spec status. ## Overview This document specifies extensions to [gossipsub v1.0](gossipsub-v1.0.md) intended to improve -bootstrapping and protocol attack resistant. The extensions change the algorithms that +bootstrapping and protocol attack resistance. The extensions change the algorithms that prescribe local peer behaviour and are fully backwards compatible with v1.0 of the protocol. Peers that implement these extensions, advertise v1.1 of the protocol using `/meshsub/1.1.0` as the protocol string. -## Attack Vectors considered +## Attack Considerations `To be written` From 3bd25cb4657e16d3a2151cf16dac7a28723d5d6a Mon Sep 17 00:00:00 2001 From: David Dias Date: Fri, 20 Mar 2020 07:34:05 +0000 Subject: [PATCH 22/23] tidy up --- pubsub/gossipsub/README.md | 32 ++++++++++++++++++------------ pubsub/gossipsub/gossipsub-v1.1.md | 12 +++-------- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/pubsub/gossipsub/README.md b/pubsub/gossipsub/README.md index b92e0185f..681ef263d 100644 --- a/pubsub/gossipsub/README.md +++ b/pubsub/gossipsub/README.md @@ -1,23 +1,29 @@ # gossipsub: An extensible baseline pubsub protocol -Gossipsub is an extensible baseline pubsub protocol, -based on randomized topic meshes and gossip. It is a general purpose -pubsub protocol with moderate amplification factors and good scaling -properties. The protocol is designed to be extensible by more -specialized routers, which may add protocol messages and gossip in -order to provide behaviour optimized for specific application -profiles. +![](https://gateway.ipfs.io/ipfs/QmYtToK1z3SLcACNhZSScxh2AmS6mrceGM3XdZV1T6rJR8) + +Gossipsub is an extensible baseline pubsub protocol, based on randomized topic meshes and gossip. It is a general purpose pubsub protocol with moderate amplification factors and good scaling properties. The protocol is designed to be extensible by more specialized routers, which may add protocol messages and gossip in order to provide behaviour optimized for specific application profiles. + +If you are new to Gossipsub and/or PubSub in general, we recommend you to first: +- Read the [Publish/Subscribe guide at docs.libp2p.io](https://docs.libp2p.io/concepts/publish-subscribe/) +- Watch the [Scalable PubSub with GossipSub talk by Dimitris Vyzovitis](https://www.youtube.com/watch?v=mlrf1058ENY&index=3&list=PLuhRWgmPaHtRPl3Itt_YdHYA0g0Eup8hQ) ## Specification -- [gossipsub-v1.0](gossipsub-v1.0.md): v1.0 of the gossipsub protocol. This is a revised specification, to use a more normative language. - - The original v1.0 specification is [here](gossippsub-v1.0-old.md). +- [gossipsub-v1.0](gossipsub-v1.0.md): v1.0 of the gossipsub protocol. This is a revised specification, to use a more normative language. The original v1.0 specification is [here](gossippsub-v1.0-old.md), still a good read. - [gossipsub-v1.1](gossipsub-v1.1.md): v1.1 of the gossipsub protocol. - [(not in use) episub](episub.md): a research note on a protocol building on top of gossipsub to implement [epidemic broadcast trees](https://www.gsd.inesc-id.pt/~ler/reports/srds07.pdf). ## Implementation status -- Go: [libp2p/go-libp2p-pubsub/gossipsub.go](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) (experimental) -- JS: [ChainSafeSystems/gossipsub-js](https://github.com/ChainSafeSystems/gossipsub-js) (compliant with `gossipsub-v1.0`). -- Rust: [libp2p/rust-libp2p#898](https://github.com/libp2p/rust-libp2p/pull/898) implements the spec but is missing some features. [libp2p/rust-libp2p#767](https://github.com/libp2p/rust-libp2p/pull/767) is an alternative, partial implementation that differs slightly from the spec (see [#142](https://github.com/libp2p/specs/issues/142) for details). -- Gerbil: [vyzo/gerbil-simsub](https://github.com/vyzo/gerbil-simsub) (simulator) +Legend: ✅ = complete, 🏗 = in progress, ❕ = not started yet + +| Name | v1.0 | v1.1 | +|--------------------------------------------------------------------------------------------------|:-----:|:-- --:| +| [go-libp2p-pubsub (Golang)](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) | ✅ | ✅ | +| [gossipsub-js (JavaScript)](https://github.com/ChainSafeSystems/gossipsub-js) | ✅ | ❕ | +| [rust-libp2p (Rust)](https://github.com/libp2p/rust-libp2p/tree/master/protocols/gossipsub) | 🏗 | ❕ | + +Additional tooling: + +- Simulator developed in Gerbil: [vyzo/gerbil-simsub](https://github.com/vyzo/gerbil-simsub) diff --git a/pubsub/gossipsub/gossipsub-v1.1.md b/pubsub/gossipsub/gossipsub-v1.1.md index 0c84262d2..d27d51106 100644 --- a/pubsub/gossipsub/gossipsub-v1.1.md +++ b/pubsub/gossipsub/gossipsub-v1.1.md @@ -16,8 +16,7 @@ Interest Group: [@yusefnapora], [@raulk], [@whyrusleeping], [@Stebalien], [@davi [@Stebalien]: https://github.com/Stebalien [@daviddias]: https://github.com/daviddias -See the [lifecycle document][lifecycle-spec] for context about maturity level -and spec status. +See the [lifecycle document][lifecycle-spec] for context about maturity level and spec status. [lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md @@ -56,10 +55,6 @@ prescribe local peer behaviour and are fully backwards compatible with v1.0 of t Peers that implement these extensions, advertise v1.1 of the protocol using `/meshsub/1.1.0` as the protocol string. -## Attack Considerations - -`To be written` - ## Protocol extensions ### Peer Exchange on PRUNE @@ -100,7 +95,6 @@ message PeerInfo { optional bytes peerID = 1; optional bytes signedPeerRecord = 2; } - ``` ### Flood Publishing @@ -390,7 +384,7 @@ if invalidMessageDeliveries < DecayToZero { #### Guidelines for Tuning the Scoring Function -TBD +`TBD`: We are currently developing a multiple types of simulations that will inform us on how to best recommend tunning the Scoring function. We will update this section once that work is complete ### Spam Protection Measures @@ -405,4 +399,4 @@ In order counter spam that elicits responses and consumes resources, some measur - Invalid message spam, either directly transmitted or as a response to an `IHAVE` message is penalized by the score function. A peer transmitting lots of spam will quickly get graylisted, reducing the surface of spam-induced computation (eg validation). The application can take - further steps and blacklist the peer if the spam persists after the negative score decays. + further steps and blacklist the peer if the spam persists after the negative score decays. \ No newline at end of file From 2fcfd80d92de69f0706953c2ee5bd14e314bdd79 Mon Sep 17 00:00:00 2001 From: David Dias Date: Fri, 20 Mar 2020 07:38:18 +0000 Subject: [PATCH 23/23] add jvm and py --- pubsub/gossipsub/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pubsub/gossipsub/README.md b/pubsub/gossipsub/README.md index 681ef263d..4e14936a2 100644 --- a/pubsub/gossipsub/README.md +++ b/pubsub/gossipsub/README.md @@ -23,6 +23,8 @@ Legend: ✅ = complete, 🏗 = in progress, ❕ = not started yet | [go-libp2p-pubsub (Golang)](https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go) | ✅ | ✅ | | [gossipsub-js (JavaScript)](https://github.com/ChainSafeSystems/gossipsub-js) | ✅ | ❕ | | [rust-libp2p (Rust)](https://github.com/libp2p/rust-libp2p/tree/master/protocols/gossipsub) | 🏗 | ❕ | +| [py-libp2p (Python)](https://github.com/libp2p/py-libp2p/tree/master/libp2p/pubsub) | ✅ | ❕ | +| [jvm-libp2p (Java/Kotlin)](https://github.com/libp2p/jvm-libp2p/tree/develop/src/main/kotlin/io/libp2p/pubsub) | ✅ | ❕ | Additional tooling: