Erigon seems stuck after committed state message

[mauricio-seii]

Hi,

Currently, I'm trying to run an Erigon node v2022.08.22, connecting to the Ropsten network, and synchronizing from scratch.
The node starts up fine but after some time processing, it seems to get stuck right after printing this logging line:

2022-08-22T17:54:05.063215506Z [INFO] [08-22|17:54:05.063] Committed State                          gas reached=549757269943 gasTarget=549755813888

So here's my question, should I leave it be, or do I need to take any action to be able to continue to process blocks?

System information

version: v2022.08.02
network: ropsten
platform: Kubernetes

Expected behavior

Continue to sync to the network

[AskAlexSharov]

More logs plz

[mauricio-seii]

  | [INFO] [08-22\|18:26:13.204] [p2p] GoodPeers                          eth66=46
  | [INFO] [08-22\|18:24:13.203] [p2p] GoodPeers                          eth66=46
  | [INFO] [08-22\|18:22:13.204] [p2p] GoodPeers                          eth66=46
  | [INFO] [08-22\|18:20:13.204] [p2p] GoodPeers                          eth66=46
  | [INFO] [08-22\|18:18:13.204] [p2p] GoodPeers                          eth66=46
  | [INFO] [08-22\|18:16:13.203] [p2p] GoodPeers                          eth66=45
  | [INFO] [08-22\|18:14:13.204] [p2p] GoodPeers                          eth66=45
  | [INFO] [08-22\|18:12:13.204] [p2p] GoodPeers                          eth66=45
  | [INFO] [08-22\|18:10:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|18:08:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|18:06:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|18:04:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|18:02:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|18:00:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|17:58:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|17:56:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|17:54:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|17:54:05.063] Committed State                          gas reached=549757269943 gasTarget=549755813888
  | [INFO] [08-22\|17:53:54.809] [txpool] stat                            block=12350712 pending=0 baseFee=0 queued=0 alloc=5.5GB sys=8.1GB
  | [INFO] [08-22\|17:53:49.871] [p2p] GoodPeers                          eth66=26
  | [INFO] [08-22\|17:53:47.259] [6/16 Execution] Executed blocks         number=949817 blk/s=1125.7 tx/s=2156.6 Mgas/s=1213.3 gasState=0.95 batch=933.7MB alloc=7.1GB sys=8.1GB
  | [INFO] [08-22\|17:53:27.249] [6/16 Execution] Executed blocks         number=927293 blk/s=377.4 tx/s=1041.4 Mgas/s=1270.4 gasState=0.91 batch=891.9MB alloc=4.4GB sys=8.1GB
  | [INFO] [08-22\|17:53:07.248] [6/16 Execution] Executed blocks         number=919745 blk/s=277.7 tx/s=9473.0 Mgas/s=799.7 gasState=0.86 batch=814.4MB alloc=4.7GB sys=7.8GB
  | [INFO] [08-22\|17:52:54.809] [txpool] stat                            block=12350712 pending=0 baseFee=0 queued=0 alloc=6.2GB sys=7.7GB
  | [INFO] [08-22\|17:52:47.262] [6/16 Execution] Executed blocks         number=914196 blk/s=24.2 tx/s=18619.8 Mgas/s=391.1 gasState=0.83 batch=776.9MB alloc=4.9GB sys=7.7GB
  | [INFO] [08-22\|17:52:27.253] [6/16 Execution] Executed blocks         number=913711 blk/s=50.0 tx/s=22486.9 Mgas/s=473.3 gasState=0.82 batch=775.8MB alloc=4.5GB sys=7.0GB
  | [INFO] [08-22\|17:52:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|17:52:07.253] [6/16 Execution] Executed blocks         number=912711 blk/s=814.9 tx/s=3584.7 Mgas/s=950.9 gasState=0.80 batch=774.5MB alloc=4.1GB sys=6.8GB
  | [INFO] [08-22\|17:51:54.809] [txpool] stat                            block=12350712 pending=0 baseFee=0 queued=0 alloc=5.3GB sys=6.8GB
  | [INFO] [08-22\|17:51:49.871] [p2p] GoodPeers                          eth66=23
  | [INFO] [08-22\|17:51:47.248] [6/16 Execution] Executed blocks         number=896409 blk/s=625.0 tx/s=3505.9 Mgas/s=1214.5 gasState=0.76 batch=723.4MB alloc=4.1GB sys=6.8GB
  | [INFO] [08-22\|17:51:27.253] [6/16 Execution] Executed blocks         number=883913 blk/s=1039.3 tx/s=14286.1 Mgas/s=954.9 gasState=0.72 batch=654.8MB alloc=5.9GB sys=6.6GB
  | [INFO] [08-22\|17:51:07.251] [6/16 Execution] Executed blocks         number=863124 blk/s=92.4 tx/s=15772.0 Mgas/s=386.5 gasState=0.69 batch=640.6MB alloc=3.1GB sys=6.6GB
  | [INFO] [08-22\|17:50:54.808] [txpool] stat                            block=12350712 pending=0 baseFee=0 queued=0 alloc=5.0GB sys=6.6GB
  | [INFO] [08-22\|17:50:47.247] [6/16 Execution] Executed blocks         number=861276 blk/s=1843.2 tx/s=12155.1 Mgas/s=591.5 gasState=0.67 batch=637.5MB alloc=3.5GB sys=6.6GB
  | [INFO] [08-22\|17:50:27.248] [6/16 Execution] Executed blocks         number=824413 blk/s=2642.1 tx/s=4289.5 Mgas/s=746.8 gasState=0.65 batch=619.7MB alloc=3.3GB sys=6.6GB
  | [INFO] [08-22\|17:50:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|17:50:07.248] [6/16 Execution] Executed blocks         number=771572 blk/s=1571.1 tx/s=2725.0 Mgas/s=435.3 gasState=0.62 batch=589.3MB alloc=4.0GB sys=6.6GB
  | [INFO] [08-22\|17:49:54.812] [txpool] stat                            block=12350712 pending=0 baseFee=0 queued=0 alloc=3.0GB sys=6.6GB
  | [INFO] [08-22\|17:49:49.871] [p2p] GoodPeers                          eth66=20
  | [INFO] [08-22\|17:49:47.247] [6/16 Execution] Executed blocks         number=740147 blk/s=2222.2 tx/s=2877.5 Mgas/s=716.9 gasState=0.61 batch=575.2MB alloc=4.8GB sys=6.6GB
  | [INFO] [08-22\|17:49:27.257] [6/16 Execution] Executed blocks         number=695726 blk/s=2039.5 tx/s=1943.3 Mgas/s=522.7 gasState=0.58 batch=548.9MB alloc=6.1GB sys=6.5GB
  | [INFO] [08-22\|17:49:07.257] [6/16 Execution] Executed blocks         number=654938 blk/s=2058.3 tx/s=2706.1 Mgas/s=617.8 gasState=0.56 batch=530.5MB alloc=4.3GB sys=6.2GB
  | [INFO] [08-22\|17:48:54.831] [txpool] stat                            block=12350712 pending=0 baseFee=0 queued=0 alloc=5.8GB sys=6.2GB
  | [INFO] [08-22\|17:48:47.248][6/16 Execution] Executed blocks         number=613751 blk/s=752.2 tx/s=6621.3 Mgas/s=776.5 gasState=0.54 batch=510.5MB alloc=5.0GB sys=6.0GB
  | [INFO] [08-22\|17:48:27.296] [6/16 Execution] Executed blocks         number=598743 blk/s=822.1 tx/s=5531.1 Mgas/s=973.2 gasState=0.51 batch=496.0MB alloc=4.0GB sys=6.0GB
  | [INFO] [08-22\|17:48:13.204] [p2p] GoodPeers                          eth66=44
  | [INFO] [08-22\|17:48:07.248] [6/16 Execution] Executed blocks         number=582261 blk/s=693.7 tx/s=3282.5 Mgas/s=632.3 gasState=0.48 batch=479.8MB alloc=4.1GB sys=5.7GB
  | [INFO] [08-22\|17:47:54.809] [txpool] stat                            block=12350712 pending=0 baseFee=0 queued=0 alloc=2.9GB sys=5.7GB

[AskAlexSharov]

add --pprof show output of http://localhost:6060/debug/pprof/goroutine?debug=1

[AskAlexSharov]

Ropsten - network already passed TheMerge event - do you use ConsensusLayer service?

[mauricio-seii]

Ropsten - network already passed TheMerge event - do you use ConsensusLayer service?

No, we wanted this node to be synced with the network before connecting it to a CL client.

edit: we started a fresh instance, so our expectation was to have it processing all blocks until it got to the merge event, and then we'd change the config to connect to the CL client.

[AskAlexSharov]

I think this is not possible. @yperbasis , right?

[mauricio-seii]

hang on, let me try again. We expected the node to be syncing up to around the 12m mark, and once we got notice of the merge event we'd update the config and set up the integration to a CL.

What you are saying is that, if I'm starting a new Erigon node I should immediately connect it to a CL is that correct?

[yperbasis]

For networks that have already passed the Merge, it's best to drive Erigon by CL from the start. The alternative that you're describing, namely to first sync all pre-Merge blocks, relies on block gossip (at least when Erigon block snapshots end before the Merge). EIP-3675 prohibits block gossip after the Merge, rendering this strategy brittle and depending on a small fraction of peers who are left not updated with Ropsten Terminal Total Difficulty and still gossiping. Moreover, with PR #5172 such sync without a CL node will be disabled.

TL;DR: Connect your Erigon node to a CL from the start.

[mauricio-seii]

awesome thank you very much @yperbasis and @AskAlexSharov!