feat: Add Identify + Kademlia chat example

Cargo.toml

@@ -136,6 +136,13 @@ env_logger = "0.9.0"
 clap = { version = "4.0.13", features = ["derive"] }
 tokio = { version = "1.15", features = ["io-util", "io-std", "macros", "rt", "rt-multi-thread"] }
 
+# For kad-identify-chat example
+anyhow = "1.0.66"
+bincode = "1.3.3"
+log = "0.4.17"
+serde = "1.0.147"
+thiserror = "1.0.37"
+
 [workspace]
 members = [
     "core",
@@ -211,6 +218,11 @@ required-features = ["full"]
 name = "distributed-key-value-store"
 required-features = ["full"]
 
+[[example]]
+name = "kad-identify-chat"
+path = "examples/kad-identify-chat/main.rs"
+required-features = ["full"]
+
 # Passing arguments to the docsrs builder in order to properly document cfg's.
 # More information: https://docs.rs/about/builds#cross-compiling
 [package.metadata.docs.rs]

examples/README.md

@@ -57,3 +57,16 @@ A set of examples showcasing how to use rust-libp2p.
   While obviously showcasing how to build a basic file sharing application with the Kademlia and
   Request-Response protocol, the actual goal of this example is **to show how to integrate
   rust-libp2p into a larger application**.
+
+- [Chat with Identify and Kademlia DHT](kad-identify-chat/main.rs)
+
+  The kad-identify-chat example implements simple chat functionality using the `Identify` protocol
+  and the `Kademlia` DHT for peer discovery and routing. Broadcast messages are propagated using the
+  `Gossipsub` behaviour, direct messages are sent using the `RequestResponse` behaviour.
+
+  The primary purpose of this example is to demonstrate how these behaviours interact.
+
+  A secondary purpose of this example is to show what integration of libp2p in a complete
+  application might look like. This is similar to the [file sharing example](file-sharing.rs),
+  but where that example is purposely more barebones, this example is a bit more expansive and 
+  uses common crates like `thiserror` and `anyhow`. It also uses the tokio runtime instead of async_std.

examples/kad-identify-chat/README.md

@@ -0,0 +1,108 @@
+# kad-identify-chat
+The kad-identify-chat example implements simple chat functionality using the `Identify` protocol
+and the `Kademlia` DHT for peer discovery and routing. Broadcast messages are propagated using the
+`Gossipsub` behaviour, direct messages are sent using the `RequestResponse` behaviour.
+
+The primary purpose of this example is to demonstrate how these behaviours interact.
+
+A secondary purpose of this example is to show what integration of libp2p in a complete
+application might look like. This is similar to the [file sharing example](../file-sharing.rs),
+but where that example is purposely more barebones, this example is a bit more expansive and
+uses common crates like `thiserror` and `anyhow`. It also uses the tokio runtime instead of async_std.
+
+Finally, it shows a way to organise your business logic using custom `EventHandler` and
+`InstructionHandler` traits. This is by no means *the* way to do it, but it worked well
+in the real-world application this example was derived from.
+
+Note how all business logic is concentrated in only four functions:
+- `MyChatNetworkClient::handle_user_input()`: takes input from the cli and turns it into an
+  `Instruction` for the `Network`.
+- `MyChatNetworkBehaviour::handle_instruction()`: acts on the `Instruction` using one of its
+  composed `NetworkBehaviour`s. Most likely by sending a message through the `Swarm`.
+- `MyChatNetworkBehaviour::handle_event()`: receives events from the `Swarm` and turns them
+  into `Notification`s for `MyChatNetworkClient`.
+- `MyChatNetworkClient::handle_notification()`: receives `Notification`s and acts on them, often
+  by formatting them and displaying them to the user.
+
+# The CLI
+Once peers are running (see below), the following 'commands' can be typed into their consoles:
+- press `<enter>`: sends a broadcast to all known peers through GossipSub. (used below)
+- `dm 2 my message`: sends 'my message' to peer number 2.
+- `ls`: shows the list of `PeerId`s this peer is aware of.
+
+> A note on peer numbers. For this example, peers are started with a peer number. This is then used to deterministically generate their keypair and peer id. This is a convenience for testing and prevents us from having to copy-paste peer ids when sending a dm: when a peer was started with `--peer-no 3`, you can send a message to it 
+> using `dm 3 some message`. 
+
+# Running a network of peers
+
+Running this example shows three peers interacting:
+- Peer 1 is a so-called bootstrap peer. Every peer that joins the network should initially connect to one bootstrap peer to discover the rest of the network. After that initial discovery they no longer need the bootstrap peer. Note that the only differences between a bootstrap peer and a normal peer are: 1) that it does not necessarily connect to a specified bootstrap peer on startup and 2) that at least one bootstrap peer is required to be running at all times so they can be an entrypoint to the network for new joining peers. Note that it is fine to have multiple bootstrap peers in a network for robustness.
+- Peer 2 and 3 connect to the bootstrap peer and register themselves in the Kademlia DHT. They also discover other peers they can reach on the DHT.
+- Peer 2 will send a BROADCAST pubsub message.
+- Peer 3's console shows that the pubsub message reaches peer 3, even though they did not have a direct connection between themselves at the time. The message is relayed through pubsub protocol by peer 1, who has a connection to both peer 2 and 3.
+- Next peer 1 and 3 will automatically reply to the BROADCAST sent by peer 2 with a direct message to peer 2: 
+  - make a direct connection (dialing) to peer 2
+  - send a direct message addressed to peer 2
+- Finally, when the user types `dm 3 foo bar` in the console of peer 2, the console of peer 3 will show that it received the message. This is because even though peer 2 and 4 wiere initially not aware of each other and connected, they learn each other's peer id and listen address through the combination of the Identify protocol and the Kademlia DHT. This allows them to send a direct message using the RequestResponse behaviour.
+
+> `RUST_LOG=INFO` is a good starting log level that shows only relevant communication between peers. But if you want to see more details on how the nodes discover each other and communicate, `RUST_LOG=kad_identify_chat=DEBUG` is good. 
+
+To run this example, open three terminals. 
+In terminal 1, run peer 1:
+
+```sh
+$ RUST_LOG=INFO cargo run --features="full" --example kad-identify-chat -- --peer-no 1 
+```
+
+In terminal 2, run peer 2:
+
+```sh
+$ RUST_LOG=INFO cargo run --features="full" --example kad-identify-chat -- --peer-no 2 --bootstrap-peer-no 1 
+```
+
+In terminal 3, run peer 3:
+
+```sh
+$ RUST_LOG=INFO cargo run --features="full" --example kad-identify-chat -- --peer-no 3 --bootstrap-peer-no 1
+```
+
+Let all of them run for a few seconds to ensure that they have discovered each other, then press enter in the terminal of peer 2 to trigger the broadcast message.
+
+Expected behaviour: 
+
+Observe the broadcast message appearing in the log of peer 1 and 3. Note that peer 3 receives the message from peer 1, and not from peer 2 as expected.
+Next, observe the direct messages being sent from both peer 1 and 3. Finally, observe those direct message being received by peer 2. 
+This concludes the interaction. 
+
+## Optional next step
+An optional next step is to add a peer 4 and 5, such that: 
+- peer 5 is connected only to peer 4 (peer 4 is its bootstrap node).
+- peer 4 is only connected to peer 3 (peer 3 is its bootstrap node).
+- we know already that peer 2 and 3 were initially only connected to their bootstrap peer, peer 1.
+
+Peer 5 should also receive a broadcast message from peer 2. It should also be able to send back a dm to peer 2, even though it is not connected to it yet. 
+This will be possible because the listen address for peer 2 will be in the DHT (because of the Identify behaviour), 
+which will be synced to peer 5 as soon as it joins the network.
+
+To run it, first kill all peers from above and restart them. This ensures none of them have a direct connection
+between them, except to their bootstrap node.
+
+Next, in terminal 4, run peer 4:
+
+```sh
+$ RUST_LOG=INFO cargo run --features="full" --example kad-identify-chat -- --peer-no 4 --bootstrap-peer-no 3
+```
+
+In terminal 5, run peer 5:
+
+```sh
+$ RUST_LOG=INFO cargo run --features="full" --example kad-identify-chat -- --peer-no 5  --bootstrap-peer-no 4
+```
+
+Then we execute the same test as before: press enter in the terminal of peer 2.
+
+Expected behaviour: 
+- Peer 3 receives the broadcast from peer 1
+- Peer 4 receives it from peer 3
+- Peer 5 receives it from peer 4
+- They all send back a direct message to peer 2, so we should see a dm from peer 5 in the log of peer 2

examples/kad-identify-chat/cli.rs

@@ -0,0 +1,51 @@
+use std::net::Ipv4Addr;
+
+use clap::Parser;
+use libp2p::identity::{ed25519, Keypair};
+use libp2p::multiaddr::Protocol;
+use libp2p::Multiaddr;
+use libp2p_core::PeerId;
+
+// Peer tcp port will be determined by adding their peer_no to this value
+const LISTEN_PORT_BASE: u16 = 63000;
+
+#[derive(Parser, Debug)]
+#[clap(name = "MyChat networking example")]
+pub struct MyChatCliArgs {
+    /// Fixed value to generate deterministic peer ID.
+    #[clap(long, short)]
+    pub peer_no: u8,
+
+    /// Fixed value to generate deterministic peer ID and port
+    /// e.g. peer_no 1 will always result in
+    #[clap(long, short)]
+    pub bootstrap_peer_no: Option<u8>,
+}
+
+pub fn listen_address_with_peer_id(bootstrap_peer_no: u8) -> Multiaddr {
+    let mut addr = local_listen_address_from_peer_no(bootstrap_peer_no);
+    addr.push(Protocol::P2p(peer_no_to_peer_id(bootstrap_peer_no).into()));
+    addr
+}
+
+pub fn peer_no_to_peer_id(peer_no: u8) -> PeerId {
+    keypair_from_peer_no(peer_no)
+        .public()
+        .to_peer_id()
+}
+
+// Deterministically create a local listen address from the peer number
+pub fn local_listen_address_from_peer_no(peer_no: u8) -> Multiaddr {
+    let mut list_address = Multiaddr::from(Protocol::Ip4(Ipv4Addr::LOCALHOST));
+    list_address.push(Protocol::Tcp(LISTEN_PORT_BASE + peer_no as u16));
+    list_address
+}
+
+// Deterministically create a keypair from the peer number
+pub fn keypair_from_peer_no(peer_no: u8) -> Keypair {
+    // We can unwrap here, because `SecretKey::from_bytes()` can only fail on bad length of
+    // bytes slice. The  length is hardcoded to correct length of 32 here.
+    Keypair::Ed25519(ed25519::Keypair::from(
+        ed25519::SecretKey::from_bytes([peer_no; 32]).unwrap(),
+    ))
+}

examples/kad-identify-chat/main.rs

@@ -0,0 +1,81 @@
+//! # Chat example with Identify and Kademlia and Identify
+//!
+//! The kad-identify-chat example implements simple chat functionality using the `Identify` protocol
+//! and the `Kademlia` DHT for peer discovery and routing. Broadcast messages are propagated using the
+//! `Gossipsub` behaviour, direct messages are sent using the `RequestResponse` behaviour.
+//!
+//! The primary purpose of this example is to demonstrate how these behaviours interact.
+//! Please see the docs on [`MyChatNetworkBehaviour`] for more details.
+//! Please see `README.md` for instructions on how to run it.
+//!
+//! A secondary purpose of this example is to show what integration of libp2p in a complete
+//! application might look like. This is similar to the file sharing example,
+//! but where that example is purposely more barebones, this example is a bit more expansive and
+//! uses common crates like `thiserror` and `anyhow`. It also uses the tokio runtime instead of async_std.
+//!
+//! Finally, it shows a way to organise your business logic using custom `EventHandler` and
+//! `InstructionHandler` traits. This is by no means *the* way to do it, but it worked well
+//! in the real-world application this example was derived from.
+//!
+//! Note how all business logic is concentrated in only four functions:
+//! - `MyChatNetworkClient::handle_user_input()`: takes input from the cli and turns it into an
+//! `Instruction` for the `Network`.
+//! - `MyChatNetworkBehaviour::handle_instruction()`: acts on the `Instruction` using one of its
+//! composed `NetworkBehaviour`s. Most likely by sending a message through the `Swarm`.
+//! - `MyChatNetworkBehaviour::handle_event()`: receives events from the `Swarm` and turns them
+//! into `Notification`s for `MyChatNetworkClient`.
+//! - `MyChatNetworkClient::handle_notification()`: receives `Notification`s and acts on them, often
+//! by formatting them and displaying them to the user.
+
+use clap::Parser;
+use tokio::sync::mpsc;
+
+use mychat_network_client::MyChatNetworkClient;
+
+use crate::cli::{
+    MyChatCliArgs, keypair_from_peer_no, listen_address_with_peer_id, local_listen_address_from_peer_no,
+};
+use crate::mychat_behaviour::MyChatNetworkBehaviour;
+use crate::network::network_builder::NetworkBuilder;
+
+mod cli;
+mod mychat_network_client;
+mod network;
+mod mychat_direct_message_protocol;
+mod mychat_behaviour;
+
+#[tokio::main]
+async fn main() -> anyhow::Result<()> {
+    env_logger::init();
+
+    let cli = MyChatCliArgs::parse();
+
+    let (notification_tx, notification_rx) = mpsc::unbounded_channel();
+    let (instruction_tx, instruction_rx) = mpsc::unbounded_channel();
+
+    let keypair = keypair_from_peer_no(cli.peer_no);
+
+    let behaviour = MyChatNetworkBehaviour::new(
+        &keypair,
+        cli.bootstrap_peer_no.map(|peer_no| vec![listen_address_with_peer_id(peer_no)]),
+    )?;
+
+    let mut network_builder =
+        NetworkBuilder::new(keypair, instruction_rx, notification_tx, behaviour)?;
+
+    // We set a custom listen address, based on the peer no
+    network_builder =
+        network_builder.listen_address(local_listen_address_from_peer_no(cli.peer_no));
+
+    let network = network_builder.build()?;
+
+    let client = MyChatNetworkClient::new(*network.peer_id(), instruction_tx, notification_rx);
+
+    // Start network event loop
+    tokio::spawn(network.run());
+
+    // Start client event loop
+    tokio::spawn(client.run()).await??;
+
+    Ok(())
+}