-
Notifications
You must be signed in to change notification settings - Fork 878
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Number of peers in logs higher than maxPeers #2689
Comments
Encountered the same problem on Besu 21.10. In a 4-nodes network, one node has 3 peers, one has 6 peers and two have 4 peers. |
This log line comes from here besu/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/tasks/PersistBlockTask.java Line 226 in 5e0301a
Not sure how/why it would exceed the number of peers available. We can aim to add some more testing around this method. |
We have observed the same behaviour. When admin_peers details are examined, the discrepancy is that we have two peers that are identical, except that one has ?disc_port=0 @RP27 Have a look at the usages of this disconnect reason code in DisconnectMessage.java |
Just an update on this issue, I have been looking into the RlpxAgent code as there is a handleIncomingConnection method that is supposed to disconnect duplicate nodes from the network. The logic within the method seems to work as expected and further tests have shown that. I believe a map is being used to store the values of the nodes, which should prevent identical keys. So next, I will look into why this is occurring. |
Repo with minimal reproduce instructions https://github.com/macfarla/besu-duplicate-peers |
and I just ruled out IBFT2. I get the same thing with ethash. |
It looks from the logs like both nodes are detecting the duplicate connection and initiating disconnect. maybe there's a race condition there. |
Hi @emclab, we have had a look at that problem and we are planning to fix that issue as part of some work in the p2p area in the near future. |
Just checking on the status of this work - Is #3945 the latest approach? |
@pinges - Have you found the source of the issue? I'm seeing a similar problem on some nodes on the Palm network. I ran a node locally for a bit with trace logging on and was seeing a large number of peers kept in
During normal operations, I saw this kind of behavior occassionally, but after opening / closing my laptop and moving around I ended up with that list full of stale peers ^. One guess is that |
@mbaxter This should get fixed with the changes that I'm doing. The main map holding the peers in EthPeers will use the peerId instead of the connection as the key. There will be another map containing potential new peers that haven't exchanged the status messages yet, and another one that will hold some peers that were already usable, but are replaced because of another connection from that peer, but they are both only holding their entries for a few seconds. These two other maps are only used when there are connection attempts from both peers at the same time, so that should not really happen in a "normal" network (just happens all the time with our acceptance tests). |
@pinges - have you found the root cause of the mismatch between Edit: Just to be clear - I'm seeing |
Next steps
|
Hey @emclab we are still working on fixing the root cause here, but we think some other improvements that have been made elsewhere in peering may have reduced the incidence of this "duplicate peer" issue, to such a low level that you may not be affected by it. Are you able to try with the most recent Besu? We have just released 22.7.0-RC2 |
Reported by a user on 22.7.0 on mainnet https://discord.com/channels/905194001349627914/1006351975467728916 |
After a few days running on goerli besu 22.7.0
|
#4304 should help with this issue |
This issue should be resolved by #4304 |
Old title: 21.7.2 node return 5 peers on 4 node IBFT2.0 network
Description
Here is log from a node (non boot) which shows 5 peers on 4 node IBFT2.0 network. The system was upgraded from 21.1 to 21.7.2 a couple of days ago. All 4 nodes are running on a single physical server with 8GB RAM:
Here is debug log for boot node showing 3 peers:
Here is a log from regular node showing 3 peers:
Acceptance Criteria
Steps to Reproduce (Bug)
Expected behavior: [What you expect to happen]
3 peers on 4 node network
Actual behavior: [What actually happens]
one node has 5 peers while other 3 have 3 peers.
Frequency: [What percentage of the time does it occur?]
Versions (Add all that apply)
besu --version
] 21.7.2java -version
] openjdk 14cat /etc/*release
] ubuntu 20.04uname -a
]vmware -v
]docker version
]Additional Information
The text was updated successfully, but these errors were encountered: