Rewrite the libp2p networking #742
Conversation
tomaka
force-pushed
the
libp2p-rewrite
branch
from
167c45e to
4f6e731
Compare
tomaka
force-pushed
the
libp2p-rewrite
branch
from
99d1713 to
93bd127
Compare
gavofyork
added
A4-gotissues
and removed
A0-please_review
|
Running the v0.2 backport on the office node after a day or two. |
|
I don't think the sync issue is new. |
|
i'll try with the latest |
|
There is still a panic happening in libp2p every 3/4 hours that I need to understand, and I should write proofs for the panics in substrate's code. I'm working on both these things right now. |
|
Should be good now. |
tomaka
added
A0-please_review
|
this is with a later p2p; still happening: |
|
i'll try again |
|
@tomaka how's this looking? |
gavofyork
added
A0-please_review
|
I can continue to improve the code by splitting it into more modules and moving some parts to the libp2p codebase, or we can review the code as it is, depending on whether it's pressing or not. |
arkpar
added
A6-seemsok
and removed
A0-please_review
|
Looks ok in general. My own tests show that it is more stable. Let's merge this |
|
Why did you choose secp256k1 for p2p level keypairs when the rest of substrate uses Ed25519? Is this a libp2p requirement? |
|
@webmaster128 Libp2p supports ed25519, secp256k1 and RSA. We can use any of the three, even on the same network. |
|
@tomaka Interesting, thanks! Why is it easier to generate keys for Secp256k1 than Ed25519? In Secp256k1, not every 32 byte value is a valid key, which makes using
Correct, they use it for transaction level signing. One advantage of Secp256k1 is that you can so public HD derivation using Slip0010. But this is not a property which is needed here. |
Sorry, it was not a problem of generating the key, but it's a similar very stupid problem: the ed25519 library we were using in libp2p ( We switched to a different ed25519 library because of all the issues we were having with ring, so we could give it a try again. |
|
Thanks! Would be great to switch that. Just in case you're still unhappy with Ed25519 libs: in one rust project I use ed25519_dalek. Looks like it is incredible flexible (you can switch out the internal Sha512 hashing if you need to) but does the job: |
Rewrites the libp2p code to use the new
SwarmAPI of libp2p.While this rewrite probably contains more bugs that the previous version, the code is globally much more clean and predictable, and is much less magic compared to the previous one.
Small walk through the code:
secret.rsjust contains loading/saving the private key on the disk, extracted from the formernetwork_state.rs.node_handler.rscontains the behavior for one single node we connect to.swarm.rscontains the listeners and all the peers we're connected to, and has an API that allows opening Kademlia and Substrate substreams towards peers. It handles things like NAT traversal internally.service_task.rswraps aroundswarm.rsand handles the topology, connecting to new nodes, reserved nodes, banned nodes, max/min number of nodes, etc. and Kademlia queries.service.rswraps aroundservice_task.rsand exposes an API compatible withnetwork.rs, which I intend to rework in the future.While this is ready for review, I keep finding small bugs from time to time (mostly on the side of libp2p), so it's not totally ready to be merged.libp2p/rust-libp2p#482 seems to have fixed everything.Once merged and backported, I think we should deploy on a single bootstrap node first, and then deploy on more and more boot nodes if it works correctly.