IBFT2.0 peer connecting issue when multiple Ethereum clients try connect to bootnodes

[helderjnpinto]

Description

I have a network with IBFT 2.0 with the following network setting:

• 5 validators in private network
• 2 public nodes as relayers to private network with load balancer in front for public rpc and p2p connections and this nodes actuate as bootnodes.
• Exists other nodes but for only dedicaded stuff like explorer, monitor, etc...

Flags used in validators / public nodes are --genesis-file, --required-block, --bootnodes

The problem that i facing is other clients p2p with same protocol of p2p like ethereum like Geth, OpenEthereum, Parity, etc...
are trying connect via my public nodes causing problems of peer discovery.

(print of jsonrpc admin_peers response)

Steps to Reproduce (Bug)

1. Start IBFT network with my setup private validators + public nodes
2. Giving one public node to ETHEREUM node network
3. Try shutdown one validator for upgrade process and start again

This is causing an issue because when i try to upgrade validators client to upper version.
Following the process of upgrading a validator or any node when i disconnect one validator and change the image, start again all nodes stopped to mine blocks saying "waiting for peers" (network with 5 validators).

All start working when i forced adding peers via ADMIN admin_addPeer.
I picked up one of the enodes and added to all peers after this all nodes discovery each other and start mining again.

So this solution not resolved in one day after trying updating to other version, when i see the logs are several Ethereum nodes try connecting.
Resolving this issue closing all p2p port in the firewall for external ips from my network, i restarted all nodes and the network worked well, after i restart the network i open p2p to external ips again because other clients IBFT can be connect for sync purposes and relay transactions, etc.

I think when several non IBFT peers are connecting to bootnodes maybe stop p2p discovering and when validators try connect again not work.

Acceptance Criteria

As an client of IBFT 2.0 network, I want p2p discovery filter only for nodes with my category if any peer of ethereum like Geth, OpenEthereum, Parity, GoChain, etc...

Peers of IBFT 2.0 with --genesis-file, --required-block need to validate before pass to other peers this neighbors:

• Required block
• Genesis-file (Includes chain ID and forks state ...)
• Peer caps like [eth/62, eth/63, IBF/1]

Like this example of GoChain in one of my validators in private network without public ips, the bootnodes should not pass this peers.

Expected behavior:
Peers in private network not receive p2p messagens from not filtered criteria
All nodes sync with bootnode with no problem

Frequency:

lately like 99%, a year ago nothing was happening
I have version of 1.4.0 and at least one of validators is 20.10.0 i was trying to update all my nodes when i noticed this problem

Versions (Add all that apply)

• Software version: [besu --version ]
  1.4.0 and one of validators in development network is 20.10.0
• Java version: [java -version ]
  oracle openjdk 13
• OS Name & Version: [cat /etc/*release]
  Amazon Linux AMI release 2018.03
• Docker Version: [docker version]
  19.03.13-ce
• Cloud VM, type, size: [Amazon Web Services I3-large]
  t3.small for dev network

[arash009]

Hi,
From what I can understand of the description of the issue, that you are running a private network of 5 validators using IBFT2.0 and then you have 2 bootnodes that are connected to this network, but also expose RPCs to the public. These 2 bootnodes must also be running IBFT2.0 to be part of the private network, but you are receiving peer requests from other external nodes that try and peer with these 2 public-facing nodes. Is that correct?

As far as I understand, discovery is working as intended, so to limit the "noise" of additional peers that cant connect to you anyway, you're better off using static peers. Are you intending for external nodes to connect to this network?

Are you also using the Node Permissioning functionality to limit which peers your nodes communicate with?

[helderjnpinto]

@arash009

From what I can understand of the description of the issue, that you are running a private network of 5 validators using IBFT2.0 and then you have 2 bootnodes that are connected to this network, but also expose RPCs to the public. These 2 bootnodes must also be running IBFT2.0 to be part of the private network, but you are receiving peer requests from other external nodes that try and peer with these 2 public-facing nodes. Is that correct?

• Yes, clients from Ethereum network

Are you also using the Node Permissioning functionality to limit which peers your nodes communicate with?

• I do not have permissioning, i do not want a limited peer network or adding manually each peer external or internal to organization.

As far as I understand, discovery is working as intended, so to limit the "noise" of additional peers that cant connect to you anyway, you're better off using static peers. Are you intending for external nodes to connect to this network?

• Yes i want to external nodes of same client connect to my public bootnodes, so if i use static_nodes.json config for peers and disabling network peer discovery all external peers that i do not control of my network (IBFT same clients) only connect to my bootnodes for syncing right ?
  For example:
  If i have 100 external peers connect to one bootnode this connection is 100:1 will causing one bottleneck because peer discovery is disable and external nodes will not connect each other.