Add more examples of getting and putting records in the IPFS DHT

[pawelradtke-sage]

Hi,

The ipfs-kad.rs only demonstrates one operation (find closest peers) on the IPFS DHT. I think it would be useful to go a bit further and demonstrate how to read/write PK or IPNS records.

I'm willing to contribute an example file similar to these in the ./examples folder, but I myself need to be shown how perform these operations first.

[mxinden]

With #2309 merged, do you want to take a shot at this one @pawelradtke-sage?

[pradt2]

I had a brief look at go-libp2p-kad-dht yesterday, and there they have some IPFS DHT-specific primitives like PK/IPNS record validators, IPNS structs, IPLD Nodes, some factory methods, utils for signing and signature verification, etc. This makes high-level interactions with the DHT very easy.

None of these primitives exist in rust-libp2p (or I haven't found them). This means that an example of putting an IPNS record into the DHT would not only need to show how to call kademlia::PutValue, but also how to build the record from scratch (and it's a bit of a lengthy process). This doesn't sit right with me. I think it would make more sense to have these primities available in our codebase, so that we can offer an easy to use, high level API similar to that of go-libp2p-kad-dht. This would make the primitives readily reusable, and would make the example a lot less cluttered.

Specifically, we would need (almost) all of go-ipns (IPNS struct, Create method for IPNS records, Validate method for IPNS records), a little bit of go-libp2p-record (Validate method for PublicKeyRecords) and some of go-ipld-prime (Node struct and methods are used to create CBOR data used for signing and validation of IPNS records).

I understand that not everyone might be interested in these primitives, but this can be easily solved by hiding them behind a feature flag.

I am happy to give this a try, but I first wanted to ask if you agree that this is the right way to go. I may need some guidance in terms of where to place this new code, and how to make other crates (optionally) dependent on it. Right now I'm thinking we could either add more top-level crates (i.e. rust-ipns, rust-ipld-prime, etc.) and make rust-libp2p-kad optionally dependent on them, or we could just put it all in rust-libp2p-kad and add an optional feature there.

Let me know what you think @mxinden

BTW fyi, @pawelradtke-sage is my work account, I forgot to switch accounts before opening this issue. Apologies for any confusion.

[mxinden]

Thus far the Rust libp2p-kad implementation is a low level layer enabling other projects, e.g. https://github.com/rs-ipfs/rust-ipfs/ to build on top of it.

Instead of including IPNS, IPLD etc. to libp2p-kad, how about improving the interfaces of libp2p-kad to enable these higher layers to easily operate on top of libp2p-kad?

In my eyes that would (1) enable a clean layer distinction and (2) does not increase the already very large maintenance surface of rust-libp2p.

[thomaseizinger]

With #3509, all the examples are separate binary crates. Thus, they can have their own dependencies. We could thus add whatever dependencies we need, for example https://github.com/ipld/libipld to demonstrate how to interact with the IPFS network.

[pradt2]

Nice, I think I should be able to do it next week, or the week after.

[thomaseizinger]

Nice, I think I should be able to do it next week, or the week after.

Great! Let me say this upfront though: Depending on how many dependencies we need, this might still have to be something that lives outside the repository. I'd suggest that you open a draft PR fairly early on so we can evaluate, whether or not such an example is something that we are able to maintain :)

Hope that makes sense!

[pradt2]

I've just opened a PR for this: #4566