(StackOverflow bounty) Can't put records to IPFS DHT

[pawelradtke-sage]

Hi,

I'm trying to build on the ipfs-kad.rs example to create a new record in the IPFS DHT. In the snippet below I'm trying to create a /pk entry because, unlike IPNS entries, it doesn't require any complex data structures.

    let mut key = "/pk/".bytes().collect::<Vec<_>>();
    key.append(&mut local_peer_id.to_bytes());
    let key = Key::from(key);
    
    let value = local_key.public().into_protobuf_encoding();
    match swarm.behaviour_mut().put_record(Record{
        key,
        value,
        publisher: None,
        expires: None
    }, Quorum::One) {
        Ok(ok) => {
            println!("Success start put record");
        }
        Err(err) => {
            eprintln!("Cannot start put record {:?}", err);
        }
    }

Note: The rest of the code is the same as in https://github.com/libp2p/rust-libp2p/blob/master/examples/ipfs-kad.rs, except for listening to additional events in the main event loop.

When this runs, I can see "Success start put record", but soon after the query fails with a QuorumFailed error (the success vector is empty).

This behaviour is specific to the IPFS DHT, as this same code runs fine in https://github.com/libp2p/rust-libp2p/blob/master/examples/ipfs-private.rs.

It'd be great if you could either point out what I'm doing wrong, or even better yet, update the examples to demonstrate how to properly put records in the IPFS DHT.

Edit There is a StackOverflow bounty for answering this question: https://stackoverflow.com/questions/69599760/put-record-into-dht

[mxinden]

Hi @pawelradtke-sage,

A couple of things worth looking into:

• Try calling Kademlia::bootstrap and wait for KademliaEvent::OutboundQueryCompleted with QueryResult::Bootstrap where BootstrapOk::num_remaining is 0, before calling Kademlia::put_record.
• Make sure the peer is well connected before calling Kademlia::put_record. I.e. emits KademliaEvent::RoutingUpdated events.
• What does the binary print with RUST_LOG=debug environment variable set?

[pawelradtke-sage]

Hi @mxinden , thanks for replying.

I've modified my code to call Kademlia::bootstrap and only call Kademlia::put_record when the bootstrap process finishes (remaining = 0).

From the logs I can see that the Kademlia peers that are "close" to the record key are being identified and connected to (incl. protocol negotiation etc.), but putting the record fails because the peers return EOF.

DEBUG Log attached. Kademlia::put_record is called at line 33203.
log.txt

[mxinden]

Are the remote nodes also based on rust-libp2p? Which DHT are you connecting to? The IPFS one or a custom one?

[pawelradtke-sage]

Hi @mxinden, I'm connecting to the IPFS DHT. I'm using the bootstrap nodes listed in this example ipfs-kad.rs. From the Identify information I see that the nodes I'm contacting use go-libp2p.

[pawelradtke-sage]

@mxinden @thomaseizinger seems that there's a bug that is breaking interop with go-libp2p-based IPFS DHT nodes - see #2309

[dgrr]

I am still having the same problem. I get the same errors even though I am waiting for the remaining to be 0. I always get Err(QuorumFailed { key: Key(b"key"), success: [], quorum: 1 }).

EDIT: If I create my own public host (using rust-libp2p) everything is fine, but if I try to use any of the existing public hosts, then it doesn't work.

[dariusc93]

To my knowledge, when using libp2p bootstrap/ipfs nodes, the only records that can be put into DHT would be IPNS and pk records. If you want to use your own records, you would need to run your own node to handle those records.

[dgrr]

Fair enough @dariusc93 thanks for the reply