Implement peer-discovery interface

[ghost]

As per https://github.com/libp2p/interface-peer-discovery

This interface would be implemented by:

• mdns
• bootstrap
• random walk (does this currently exist in go-libp2p?)
• in the future: webrtc, ...

[vyzo]

do we want this to be dht based?
It could be the solution to the generalized rendezvous problem, and be usable for pubsub, relay, etc

[Kubuxu]

Additional provider of peer-discovery is "old peers". This way we could make the network independent from the bootstrap nodes after first run.

[daviddias]

do we want this to be dht based?

DHT is Peer Routing + Content Routing. Read:

• https://github.com/libp2p/js-libp2p/tree/master/examples/peer-and-content-routing
• https://github.com/libp2p/specs/blob/master/4-architecture.md#41-peer-routing
• https://github.com/libp2p/specs/blob/master/4-architecture.md#43-distributed-record-store
• Splitting Routing into (Peer Routing, Content Routing) ipfs/specs#1

Peer Discovery is things like MulticastDNS or Bootstrap. Read:

• https://github.com/libp2p/js-libp2p/tree/master/examples/discovery-mechanisms
• https://github.com/libp2p/specs/blob/master/4-architecture.md#44-discovery

peer-discovery is "old peers"

Yeah, we can have a resurrect Peer Discovery that knows how to store to disk old peers and then brings them back on node boot. 👍 👍

[deckbsd]

Hello everybody,

If i understand correctly, does that mean that actually from a go-libp2p peer i can't try to find some other peers ? that's why i was wondering how, when i start a host i can't make that host to connect to any other nodes. Because actually in the examples, each nodes are created like in client/server basic model (the client know the ip of the "server"(an other node that it can communicate with). m i wrong ?

[Kubuxu]

If i understand correctly, does that mean that actually from a go-libp2p peer i can't try to find some other peers ?

No, finding peers works (it is sometimes difficult in case of complicated/layered NATs).

This issue is about internal go-libp2p interface allowing better code structure when it comes to peer discovery.

[agahEbrahimi]

@Kubuxu Are there any examples of the peer discovery? I've been wandering around for 2-3 days trying to find an example and only found one in JS https://github.com/libp2p/js-libp2p/tree/master/examples/discovery-mechanisms which was also mentioned above, but none in go.
Even though the website says that they are all complete in go.

Thank you

[Kubuxu]

@agahEbrahimi most of them should work by default. We are working on clearing up the bootstraping but for an example you can look here: #302

Random walk (DHT) should work by default and so should mdns AFAIK.

[daviddias]

Can someone point me to an up to date example of how to enable/disable random-walk in go-libp2p?

[daviddias]

Just did a quick "chase the PR" exercised and discovered that:

• There is an example created for Bootstrapping Examples/bootstrapping railing #302
• There is a refactor by @florianlenz to make go-libp2p-bootstrap nicer https://github.com/florianlenz/go-libp2p-bootstrap/pull/3

Still looking for random-walk

[aschmahmann]

@daviddias Would the DisableAutoBootstrap kad option help?

https://github.com/libp2p/go-libp2p-kad-dht/blob/4fd64988b55d6e93405eceffeb3037da46d1848b/opts/options.go#L155

[daviddias]

@aschmahmann I would say no but given that there is a bunch of terms that are overloaded, the best is for me to clarify my intent: What I want to achieve is to enable/disable the routine that searches the DHT for a random peer (most likely that doesn't exist) with the purpose of refreshing the routing tables. Is there a way to enable/disable this (known in the P2P literature as random-walk)?

[aschmahmann]

Why do you think that it's not enough? The comments at https://github.com/libp2p/go-libp2p-kad-dht/blob/4fd64988b55d6e93405eceffeb3037da46d1848b/dht_bootstrap.go#L48 seem to imply that it's exactly what you're asking for.

[yusefnapora]

@daviddias I think from reading through the kad bootstrap code a bit, that "bootstrap" in the context of the libp2p kad DHT does mean random-walk.

It looks like by default it does a query for a random peer id in each bucket that hasn't been refreshed within some interval, plus a periodic query for our own id.

If you set the DisableAutoBootstrap option that @aschmahmann mentioned, it looks like it won't do the bootstrapping on a periodic timer, but it would still be possible to manually trigger the bootstrap process by calling the Bootstrap method.

[daviddias]

@daviddias I think from reading through the kad bootstrap code a bit, that "bootstrap" in the context of the libp2p kad DHT does mean random-walk.

Aaaaaaah!!! Ok, so in go-libp2p bootstrap gets used for:

• Connecting to set of bootstrap nodes when the node boots up (aka railing into the network in p2p literature)
• Refreshing the Kademlia Routing Tree every now and then (aka random-walk in the p2p literature)

It looks like by default it does a query for a random peer id in each bucket that hasn't been refreshed within some interval, plus a periodic query for our own id.

How does it actually achieve this? Given that we sha2-256 the peerIds for the Kad Routing Table in our DHT implementation, one would actually have to brute force to find another PeerId that would fall within a specific k-bucket.

Anyhow, given that random-walking is walking randomly over new ways, most k-buckets will get somewhat refreshed along the way.

If you set the DisableAutoBootstrap option that @aschmahmann mentioned, it looks like it won't do the bootstrapping on a periodic timer, but it would still be possible to manually trigger the bootstrap process by calling the Bootstrap method.

Thank you! This is exactly what I needed. ❤️ @aschmahmann @yusefnapora

[aschmahmann]

How does it actually achieve this? Given that we sha2-256 the peerIds for the Kad Routing Table in our DHT implementation, one would actually have to brute force to find another PeerId that would fall within a specific k-bucket.

Sooo.... funny story about that we basically pre-brute force 1 per bucket for the first 16 buckets.

Precomputed buckets: https://github.com/libp2p/go-libp2p-kbucket/blob/8b77351e0f784a5f71749d23000897c8aee71a76/bucket_prefixmap.go#L4

Reasoning: https://github.com/libp2p/go-libp2p-kbucket/pull/38/files#r309983750

[daviddias]

🤯

Clever though!

[daviddias]

@aschmahmann @yusefnapora your tips worked like a charm testground/testground@c874df0 . Thank you! ❤️

[aarshkshah1992]

For posterity:

@aschmahmann's suggestion is bang on. Please refer to this writeup for a good explanation on how to use the "Dht random walk"/"Dht refresh" feature.