Out of gas txs stuck in the txqueue in PoA network #6342
Comments
|
To accept transactions with more than 4.7 million gas, just run your sealing nodes with To remove transactions from the pool, either restart your nodes with the |
|
But isn't this a potential DDoS attack vector? A malicious user can join the network and flood it with as many out-of-gas transactions as possible. |
|
A malicious user would require access to your PoA configuration (chain spec) and some decent amount of ETH to be able to DoS your network. Reopening for a 2nd opinion. |
|
Well, our configuration is open as I've provided a link to spec.json in the first post. |
|
Transactions significantly above current block gas limit are not accepted by other peers at all. If the transaction is a little bit above gas limit (up to 0.5%) it will be accepted, but evicted from the queue (after some time) when it's not mined. Also it will get a very low priority, so it's easily pushed out of the queue by other transactions in the network. |
|
Third party accounts will also have not access to the tokens on your PoA network so will be unable to submit a transaction in the first place. If you want to restrict connectivity since tokens are widely distributed, then use reserved peers. You can only get your own account stuck, does not affect other accounts at all (try it on the mainnet). |
|
@tomusdrw what is the eviction period equal to? |
|
@tomusdrw @keorn @5chdn
According to @tomusdrw comment for transaction_queue it should have already been rejected, why do we still see them? Am I understanding it correctly that all nodes are keeping the transaction queue somewhere in memory? Netstat: http://testnet.oracles.org:3000/ Here is when it gets interesting: To reproduce:
|
|
So most likely the node that initiated the transactions has all of them and keeps propagating them to the network.
It should be enough to stop the node that contains the account that initiated the transactions, wait until they are removed from the queue of the other node and restart the first node with So to be clear: the transactions are evicted from the queue, but if there is at least one node that keeps propagating them they will be imported again. That said, I'm considering removal of accepting "slightly above gas limit" transactions at all. |
|
@tomusdrw Nope, I killed the node long time ago and all pending txs were still there. So how do I figure out which node keep propagating txs? If I just run the node right now: here is what I see: Is there any idea on why truffle migrate generates pending tx. |
|
Ok, sorry I got confused. I thought that you meant that future queue is filled up with some transactions. If the transactions are in the pending queue they will be re-propagated by all those nodes, we anticipate a block gas limit increase, but if it does not happen it won't ever be included. To remove the transaction you need to replace the one that has this super high gas limit with something that can actually be mined (Parity 1.7 allows you to "cancel" transaction that is pending, and does exactly that afair). |
|
why this issue is closed? @tomusdrw yes the future tx is filled up with the same pending transactions and they never get killed. when you said - you need to replace. There are some txs that were generated by some unknown users. That's why @phahulin mentioned that someone can just come to the network, generate bunch of pending tx that will never be killed to spam the network. http://testnet.oracles.org:4000/tx/pending - we still have pending txs. Please feel free to take this private key: run parity against oracles network with these configs: try to simply |
tomusdrw
added
F3-annoyance 💩
|
@rstormsf The PR that lowers maximal gas of a single transaction is pending: #6408. The transactions are still valid, they are just awaiting for a miner willing to rise the block gas limit to accept them (alike low gas price transactions). They occupy the queue, but they are getting a very low priority so they won't block other transactions from being included and they don't really cause any threat to the network. They cannot be killed from the queue, because other nodes are repropagating them (and they are allowed to do so with any valid transactions). That said I do understand that you want to keep your queue clean though and not accept such transactions at all. If you want to get rid of them the only way currently is to increase your gas floor target and mine a slightly bigger block to process them. |
|
@tomusdrw Thank you for taking your time to understand our problem and submitting a PR. Here is what I don't understand:
Could you explain more on that? I don't believe that PoA Kovan chain is doing something like that in order to get rid of pending txs. I'd expect them to become invalid and stop propagating. FYI - I don't fully understand how does this PR solves the problem if we still need to change gas floor target? https://raw.githubusercontent.com/oraclesorg/oracles-scripts/alphadevtestnet/spec.json - OraclesPoA - here is our spec, maybe we should add some flags to it? Here is the diff between Kovan.spec.json vs OraclesPoA.spec.json: In in I suspect that maybe some of those missing flags can solve the problem. |
|
The fix doesn't accept such transactions at all, so after you upgrade your nodes to a patched Parity versions those transactions will be discarded.
|
Hello, everyone
We have a PoA network with validators via contract. Nodes are connected and generate blocks - that's all OK.
Here is spec.json
Here is node.toml from one of the mining nodes:
If miner tries to deploy some other regular contract that exceeds gas limit (here a default truffle's migration contract is being deployed), his transaction will not be rejected (these logs are obtain with
parity -l engine=trace,tx=trace,txqueue=trace,own_tx=trace,miner=trace)but will get stuck in the queue, being skipped each round
Truffle migration just hangs
If you run contract deployment again, a second pending transaction will appear
We have a separate instance of our network (for real people to test), where there are 20+ transactions like this that have been hanging in queue for days.
If you send a valid tx from that address (e.g. transfer eth to someone), it will also remove one of the pending txs from the queue:
I did this test on a mining node, but it can also be reproduced by any node connected to the network.
I think this is incorrect and invalid transaction should be removed from the queue.
The text was updated successfully, but these errors were encountered: