POA private network: Tx pending in txpool #18082

hello2mao · 2018-11-12T07:24:26Z

System information

Geth version: v1.8.17
OS & Version: Linux
Commit hash : v1.8.17-stable-8bbe7207

Expected behaviour

Tx mined

Actual behaviour

Tx pending in txpool.

Steps to reproduce the behaviour

Keep sending tx to a POA private network.
At the beginning, it works fine.
When txpool pending grows, it stuck.
Pending tx keep 401 in txpool as below:

> txpool.status
{
  pending: 401,
  queued: 0
}

four node is mining:
account nonce: 20532

> eth.getTransactionCount("0x24ddBD0C14eE23aCa0B702AD58c5b18a60B14bFE")
20532
> txpool.inspect
{
  pending: {
    0x24ddBD0C14eE23aCa0B702AD58c5b18a60B14bFE: {
      20532: "0x1b6ce26D209039190aa9c30474A4329597Ae494c: 0 wei + 100000000 gas × 0 wei",
      20533: "0x1b6ce26D209039190aa9c30474A4329597Ae494c: 0 wei + 100000000 gas × 0 wei",
      20534: "0x1b6ce26D209039190aa9c30474A4329597Ae494c: 0 wei + 100000000 gas × 0 wei",
      20535: "0x1b6ce26D209039190aa9c30474A4329597Ae494c: 0 wei + 100000000 gas × 0 wei",
      20536: "0x1b6ce26D209039190aa9c30474A4329597Ae494c: 0 wei + 100000000 gas × 0 wei",

Other info

> eth.getBlock('pending').number
18100
> eth.getBlock('pending').number
18099
> eth.getBlock('pending').number
18100
> eth.getBlock('pending').number
18099
> txpool.status
{
  pending: 401,
  queued: 0
}
> eth.blockNumber
18122
> eth.getBlock('pending')
{
  difficulty: 2,
  extraData: "0xd883010811846765746888676f312e31312e32856c696e7578000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  gasLimit: 207004807739864450,
  gasUsed: 1515024536,
  hash: null,
  logsBloom: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  miner: null,
  mixHash: "0x0000000000000000000000000000000000000000000000000000000000000000",
  nonce: null,
  number: 18123,
  parentHash: "0xdc9329b6136e948896bd6749079071b529d8d0c099c3310ec5fb7ffde34ff172",
  receiptsRoot: "0x66a54517fd8645e7a7e39baedbbb0890528889a2eecbdfd6540d6e96c0f92a81",
  sha3Uncles: "0xc72557d5c4c995c1831de7371bab158f5fcec958434614051e4ecdca538e84e3",
  size: 123528,
  stateRoot: "0x0b684bc91114e4f0b21c3b5e3729fac88a07637a61c8e9b8693bad7104580c00",
  timestamp: 1542007208,
  totalDifficulty: 0,
  transactions: ["0x6abac7a4244c89d2c10306dc09539e7cdcbac217257783e410a6f30f610112c4",  ..., "0x535051b399c5f0abf4b2fa095ae816cbd6e7adc8db9a7a51ce27f1ad2673c555"],
  transactionsRoot: "0xcd2adce29dbfff14d04b9810c283f34506ed14ff37125fd915e6eedb26373e15",
  uncles: ["0x1398ae3d032ccc525c3520e72666242eff1aa6a1cee8940bbafb53cf594f7f22"]
}

Any one can help me with that?

The text was updated successfully, but these errors were encountered:

karalabe · 2018-11-12T11:11:47Z

Geth doesn't accept transactions at 0 gas price from remote nodes.

hello2mao · 2018-11-12T11:15:32Z

Geth doesn't accept transactions at 0 gas price from remote nodes.

I use "--miner.gasprice 0" to set Geth accept transactions at 0 gas price.

hello2mao · 2018-11-12T11:39:22Z

Pending tx in txpool with Geth logs:

[root@instance-019f58455eb863abba136c2712a670ddb65c4c56e54f78265d3a7865ac57bf7b94]# grep "5c9989" 9f58455eb863abba136c2712a670ddb65c4c56e54f78265d3a7865ac57bf7b94-json.log
{"log":"TRACE[11-12|11:04:18.719] Pooled new future transaction            hash=5c9989…a00580 from=0x24ddBD0C14eE23aCa0B702AD58c5b18a60B14bFE to=0x1b6ce26D209039190aa9c30474A4329597Ae494c\n","stream":"stderr","time":"2018-11-12T11:04:18.719630316Z"}
{"log":"TRACE[11-12|11:04:18.719] Promoting queued transaction             hash=5c9989…a00580\n","stream":"stderr","time":"2018-11-12T11:04:18.719645682Z"}
{"log":"TRACE[11-12|11:04:18.719] Broadcast transaction                    hash=5c9989…a00580 recipients=2\n","stream":"stderr","time":"2018-11-12T11:04:18.719650497Z"}
{"log":"TRACE[11-12|11:04:18.720] Discarding already known transaction     hash=5c9989…a00580\n","stream":"stderr","time":"2018-11-12T11:04:18.720764546Z"}
{"log":"TRACE[11-12|11:04:18.720] Discarding already known transaction     hash=5c9989…a00580\n","stream":"stderr","time":"2018-11-12T11:04:18.721091349Z"}
{"log":"TRACE[11-12|11:12:33.499] Removed old pending transaction          hash=5c9989…a00580\n","stream":"stderr","time":"2018-11-12T11:12:33.49937846Z"}
{"log":"TRACE[11-12|11:12:48.031] Pooled new future transaction            hash=5c9989…a00580 from=0x24ddBD0C14eE23aCa0B702AD58c5b18a60B14bFE to=0x1b6ce26D209039190aa9c30474A4329597Ae494c\n","stream":"stderr","time":"2018-11-12T11:12:48.052552342Z"}
{"log":"TRACE[11-12|11:12:48.037] Promoting queued transaction             hash=5c9989…a00580\n","stream":"stderr","time":"2018-11-12T11:12:48.053786074Z"}
{"log":"TRACE[11-12|11:12:48.057] Broadcast transaction                    hash=5c9989…a00580 recipients=0\n","stream":"stderr","time":"2018-11-12T11:12:48.085888354Z"}

hello2mao · 2018-11-13T00:28:21Z

After staying in pending txpool about 1000 blocks, the tx 0x5c99895203257b1df2cc5283bc3f4d17c9dfc7147aaaab7ee06ad7bf00a00580 finally mined. -_-.

> eth.getTransactionReceipt("0x5c99895203257b1df2cc5283bc3f4d17c9dfc7147aaaab7ee06ad7bf00a00580")
{
  blockHash: "0x78972c08e869946bd9a5d54cbfa5f6ef3e50e2c4f772cefbc80be034ef12b63d",
  blockNumber: 19540,
  contractAddress: null,
  cumulativeGasUsed: 1124595963,
  from: "0x24ddbd0c14ee23aca0b702ad58c5b18a60b14bfe",
  gasUsed: 5516042,
  logs: [],
  logsBloom: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  status: "0x1",
  to: "0x1b6ce26d209039190aa9c30474a4329597ae494c",
  transactionHash: "0x5c99895203257b1df2cc5283bc3f4d17c9dfc7147aaaab7ee06ad7bf00a00580",
  transactionIndex: 205
}

Why so long?

liuzhijun23 · 2018-11-14T06:39:16Z

I met this isusure two weeks ago.This reason is that the big amount Txs would lead the event blocked.Because the bug in the event/Feed.go. Please see my issure: #18021

hadv · 2018-11-16T04:39:53Z

I have the same problem when sending ton of tnx into a node on my private POA testnet; most of the tnx stuck in the queue and didn't be shifted into tx pool's pending. I run on latest version of geth v1.8.18

> txpool.status
{
  pending: 0,
  queued: 16055
}

And after sending some more other tnx, half of the tnx in the queue are shifted to pending status

> txpool.status
{
  pending: 5973,
  queued: 7111
}

hadv · 2018-11-16T06:34:23Z

Your transactions stuck in the txpool queued. Are you sure your nonce is continuous and not has a gap?

You're might right! I send the tnx parallel by using go routine then the tnx nonce come to the node may have wrong position. Thank you! @hello2mao

hadv · 2018-11-23T03:51:18Z

I met this isusure two weeks ago.This reason is that the big amount Txs would lead the event blocked.Because the bug in the event/Feed.go. Please see my issure: #18021

And if I stop sending the txs to the node then mining can be continue normally.

cc: @hello2mao

hadv · 2018-11-23T03:55:18Z

More than 6k tnx peending but all the network stuck on block number 617

dwalintukan · 2019-05-10T09:06:18Z

Having the same issue here with stuck pending txs in the txpool. One address sending ~20 txs with incrementing nonces and it gets stuck in the txpool.

Is a node able to handle this amount of txs from a single user? Seems like it just de-prioritizes too many txs from the same address. But I dont have any other txs that need to be processed so why isnt it processing these pending ones?

noureddinel · 2019-05-13T12:13:27Z

Please, I have the same issue. Any fix for it?

shikhars371 · 2019-07-12T13:21:18Z

txpool.inspect
{
pending: {
0xaB8E4792E833b1C66d162047829FEfa76f14040f: {
2: "0xBb165CdE6A1c60a10aed6135d9295788016030B8: 1200000000000000000 wei + 21000 gas × 21000000000 wei",
3: "0xBb165CdE6A1c60a10aed6135d9295788016030B8: 1000000000000000000 wei + 21000 gas × 21000000000 wei",
4: "0xBb165CdE6A1c60a10aed6135d9295788016030B8: 2000000000000000000 wei + 21000 gas × 21000000000 wei",
5: "0xBb165CdE6A1c60a10aed6135d9295788016030B8: 1300000000000000000 wei + 21000 gas × 21000000000 wei"
}
},
queued: {}
}

please help me i am running a private chain , and submitting transaction , its stuck in pool from more than 2 hour can any one help me what should i do?

hadv · 2019-07-12T13:33:52Z

please help me i am running a private chain , and submitting transaction , its stuck in pool from more than 2 hour can any one help me what should i do?

Firstly, you can check nonce of 0xaB8E4792E833b1C66d162047829FEfa76f14040f is correct or not?

holiman · 2019-12-19T09:56:48Z

@shikhars371 never got back.
The original reported problem seems solved. Some context:

When submitting txs with high level of paralellism, it might happen that there are gaps in the nonces. Thus, when they are entered into the pool, one might "replace" or kick out another.

Alternatively, they are submitted in non-sequential order, and some make it into queue instead of pending, and then gets evicted from queue (because we have lower limits there), and then suddenly you have a gap.

rajdasqait · 2020-01-23T12:24:24Z

Hi all,

Is there any API for reading transaction pool in etherjs or web3js.

kiliw · 2021-01-18T11:19:40Z

I think I have the same problem here. I try to setup a private network with PoA and set the accepted gasPrice for the sealer to 0 via --miner.gasprice=0 but still the new created contract are stuck in the txpool.
Any suggestions?

CremaFR · 2021-01-18T11:31:07Z

I think I have the same problem here. I try to setup a private network with PoA and set the accepted gasPrice for the sealer to 0 via --miner.gasprice=0 but still the new created contract are stuck in the txpool.
Any suggestions?

Your only solution would be to send directly the transaction to the authority node. Authority can't receive 0 gas price transactions from remote nodes

kiliw · 2021-01-18T11:41:02Z

@CremaFR Thanks for your super fast answer. I got it running with sending my transaction directly to the sealer with a gas price of 0.
Still wouldn't it be possible to somehow enable the sealer to also accept transactions from an other node, with rpc enabled, within a private network ?

CremaFR · 2021-01-18T11:50:13Z

I don't think it is possible. I would recommend using Hyperledger Besu for that. And you have the benefit of IBFT 2.0 to have instant block finality compared to Clique. I know this might not be a solution for you but this what we did in my company and it works

kiliw · 2021-01-18T11:54:15Z

Alright I will check it out. The thing is we used the Azure Blockchain and there everything worked without any ether at all so I though I should be able to get a similar behavior on my private net since they also use geth as far as I understand

hello2mao changed the title ~~Tx pending in txpool~~ POA private network: Tx pending in txpool Nov 12, 2018

hello2mao mentioned this issue Nov 14, 2018

Transactions not being mined on private net #18062

Closed

hello2mao mentioned this issue Nov 14, 2018

trie/database on insert - panic: runtime error: invalid memory address or nil pointer dereference #18094

Closed

hello2mao closed this as completed Nov 16, 2018

hello2mao reopened this Nov 16, 2018

mrFranklin mentioned this issue Nov 20, 2018

Add a flag to "Allow Zero gas price for remote tx" #18130

Closed

hadv mentioned this issue Jan 29, 2019

I find a bug in event/feed.go #18021

Closed

adamschmideg added the status:triage label Dec 17, 2019

holiman closed this as completed Dec 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

POA private network: Tx pending in txpool #18082

POA private network: Tx pending in txpool #18082

hello2mao commented Nov 12, 2018 •

edited

Loading

karalabe commented Nov 12, 2018

hello2mao commented Nov 12, 2018

hello2mao commented Nov 12, 2018

hello2mao commented Nov 13, 2018

liuzhijun23 commented Nov 14, 2018 •

edited

Loading

hadv commented Nov 16, 2018 •

edited

Loading

hadv commented Nov 16, 2018 •

edited

Loading

hadv commented Nov 23, 2018

hadv commented Nov 23, 2018

dwalintukan commented May 10, 2019

noureddinel commented May 13, 2019 •

edited

Loading

shikhars371 commented Jul 12, 2019

hadv commented Jul 12, 2019

holiman commented Dec 19, 2019

rajdasqait commented Jan 23, 2020

kiliw commented Jan 18, 2021

CremaFR commented Jan 18, 2021

kiliw commented Jan 18, 2021

CremaFR commented Jan 18, 2021

kiliw commented Jan 18, 2021

POA private network: Tx pending in txpool #18082

POA private network: Tx pending in txpool #18082

Comments

hello2mao commented Nov 12, 2018 • edited Loading

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

karalabe commented Nov 12, 2018

hello2mao commented Nov 12, 2018

hello2mao commented Nov 12, 2018

hello2mao commented Nov 13, 2018

liuzhijun23 commented Nov 14, 2018 • edited Loading

hadv commented Nov 16, 2018 • edited Loading

hadv commented Nov 16, 2018 • edited Loading

hadv commented Nov 23, 2018

hadv commented Nov 23, 2018

dwalintukan commented May 10, 2019

noureddinel commented May 13, 2019 • edited Loading

shikhars371 commented Jul 12, 2019

hadv commented Jul 12, 2019

holiman commented Dec 19, 2019

rajdasqait commented Jan 23, 2020

kiliw commented Jan 18, 2021

CremaFR commented Jan 18, 2021

kiliw commented Jan 18, 2021

CremaFR commented Jan 18, 2021

kiliw commented Jan 18, 2021

hello2mao commented Nov 12, 2018 •

edited

Loading

liuzhijun23 commented Nov 14, 2018 •

edited

Loading

hadv commented Nov 16, 2018 •

edited

Loading

hadv commented Nov 16, 2018 •

edited

Loading

noureddinel commented May 13, 2019 •

edited

Loading