My nodes are unable to synchronize

[YuXiaoCoder]

System information

Geth version: 1.12.14-stable-f1c80f7e
OS & Version: Linux

Expected behaviour

Node Normal Synchronization

Actual behaviour

Nodes cannot be synchronized, showing that they cannot connect to Peer

Steps to reproduce the behaviour

geth --classic --datadir=/data --syncmode=snap --gcmode=archive --http --http.addr=0.0.0.0 --http.port=8545 --http.vhosts=* --ws --ws.addr=0.0.0.0 --ws.port=8546

Backtrace

INFO [10-08|11:22:58.356] Started P2P networking                   self=enode://268e827ac0c8f7069a18a02fb5cf4985c377b35e1c383485707de09847788bb3d3bc41fe0a6dd4f5d351cf75ca33b6410fc14fbe1ca906c99101855426a3b7ad@127.0.0.1:30303
INFO [10-08|11:22:58.356] WebSocket enabled                        url=ws://127.0.0.1:8551
INFO [10-08|11:22:58.356] HTTP server started                      endpoint=127.0.0.1:8551 auth=true  prefix= cors=localhost vhosts=localhost
INFO [10-08|11:22:58.403] Generated state snapshot                 accounts=8893 slots=0 storage=409.64KiB dangling=0 elapsed=88.751ms
INFO [10-08|11:23:08.369] Looking for peers                        peercount=0 tried=14 static=0
INFO [10-08|11:23:09.980] Block synchronisation started
WARN [10-08|11:23:09.981] Synchronisation failed, retrying         err="peer is unknown or unhealthy"
INFO [10-08|11:23:18.453] Looking for peers                        peercount=0 tried=29 static=0
INFO [10-08|11:23:22.581] New local node record                    seq=1,696,735,378,353 id=69347bd3fbdd0b43 ip=8.218.35.139 udp=30303 tcp=30303
WARN [10-08|11:23:23.966] Synchronisation failed, retrying         err="peer is unknown or unhealthy"
INFO [10-08|11:23:28.701] Looking for peers                        peercount=0 tried=33 static=0
INFO [10-08|11:23:49.036] Looking for peers                        peercount=2 tried=38 static=0
INFO [10-08|11:24:09.361] Looking for peers                        peercount=1 tried=35 static=0
INFO [10-08|11:24:19.365] Looking for peers                        peercount=1 tried=36 static=0
INFO [10-08|11:24:29.496] Looking for peers                        peercount=1 tried=32 static=0
WARN [10-08|11:24:35.977] Synchronisation failed, dropping peer    peer=1804a0841f4853e4721e726b5cc3f201e2107be22dbccd766630f2a8b3047a65 err=timeout
INFO [10-08|11:24:39.698] Looking for peers                        peercount=1 tried=33 static=0
INFO [10-08|11:24:47.091] Syncing: state download in progress      synced=6.25% state=146.98KiB accounts=586@146.98KiB slots=0@0.00B codes=0@0.00B eta=8.494s
INFO [10-08|11:24:47.751] Imported new chain segment               number=1 hash=88e96d..406cb6 blocks=1 txs=0 mgas=0.000 elapsed=8.382ms  mgasps=0.000 age=8y3mo3w  dirty=0.00B
INFO [10-08|11:24:47.751] Syncing: chain download in progress      synced=+Inf% chain=18.00B headers=1@6.00B bodies=1@6.00B receipts=1@6.00B eta=-9.578ms
INFO [10-08|11:24:47.751] Syncing: state download in progress      synced=18.75% state=428.12KiB accounts=1705@428.12KiB slots=0@0.00B codes=0@0.00B eta=5.313s
WARN [10-08|11:24:47.754] Synchronisation failed, retrying         err="sync cancelled"
WARN [10-08|11:24:47.933] Unexpected account range packet          peer=1da5b549 reqid=2,080,842,330,742,189,430
INFO [10-08|11:24:49.703] Looking for peers                        peercount=1 tried=28 static=0
INFO [10-08|11:25:00.029] Looking for peers                        peercount=1 tried=31 static=0
WARN [10-08|11:25:02.976] Synchronisation failed, retrying         err="peer is unknown or unhealthy"
INFO [10-08|11:25:10.032] Looking for peers                        peercount=1 tried=38 static=0
INFO [10-08|11:25:20.037] Looking for peers                        peercount=1 tried=35 static=0
INFO [10-08|11:25:40.042] Looking for peers                        peercount=2 tried=35 static=0
INFO [10-08|11:25:50.273] Looking for peers                        peercount=2 tried=35 static=0

When submitting logs: please submit them as text and not screenshots.

[YuXiaoCoder]

WARN [10-08|11:41:27.494] Pivot seemingly stale, moving            old=275,074 new=275,138
INFO [10-08|11:41:30.048] Syncing: chain download in progress      synced=100.00% chain=45.48MiB headers=276,480@34.68MiB bodies=275,137@8.28MiB receipts=275,137@2.51MiB eta=3.642ms
WARN [10-08|11:41:31.887] Pivot became stale, moving               old=275,138 new=275,302
ERROR[10-08|11:41:31.889] Non contiguous receipt insert            number=275,138    hash=037423..a35626 parent=84b555..73eec3 prevnumber=275,136 prevhash=825756..5f9a2f
INFO [10-08|11:41:31.889] Syncing: state download in progress      synced=25.00%  state=569.19KiB accounts=2264@569.19KiB slots=0@0.00B codes=0@0.00B eta=50m16.090s
WARN [10-08|11:41:31.891] Rewinding blockchain to block            target=274,431
INFO [10-08|11:41:34.513] Loaded most recent local header          number=274,431    hash=49d7b5..23a811 td=1,221,400,268,830,475,647 age=8y1mo3w
INFO [10-08|11:41:34.513] Loaded most recent local block           number=0          hash=d4e567..cb8fa3 td=17,179,869,184            age=54y6mo2w
INFO [10-08|11:41:34.513] Loaded most recent local snap block      number=274,431    hash=49d7b5..23a811 td=1,221,400,268,830,475,647 age=8y1mo3w
INFO [10-08|11:41:34.513] Loaded last snap-sync pivot marker       number=275,302
WARN [10-08|11:41:34.513] Rolled back chain segment                header=276480->274431 snap=275137->274431 block=0->0 reason="syncing canceled (requested)"
WARN [10-08|11:41:34.513] Synchronisation failed, dropping peer    peer=e74339fb7e29b9832bf04d2afb1d2bccc64f30a77003abb4f5bf4962516f30e7 err="retrieved hash chain is invalid: non contiguous insert: item 62 is #275136 [825756cc..], item 63 is #275138 [03742384..] (parent [84b555ef..])"

[leevlad]

I would like to report that I have also been having this issue and at this time I am completely unable to synchronize the blockchain.

I have tried adding new peers from the following resource, with no luck: https://etcnodes.org/

Additionally, I have tried re-syncing the blockchain from scratch several times over the past week, with no luck.

I'm using default Snap sync, and run geth with the following params:

/usr/local/bin/geth --classic --http --http.api "eth,net"

My geth version is:

geth version 1.12.15-stable-2171350f

Here are the logs I'm seeing on my end:

Nov 03 15:06:04 localhost geth[709]: INFO [11-03|15:06:04.430] Syncing: state healing in progress       accounts=66,857,806@4.16GiB    slots=11,072,077@746.13MiB codes=28813@85.97MiB nodes=105,558,949@10.66GiB pending=130,428
Nov 03 15:06:04 localhost geth[709]: INFO [11-03|15:06:04.606] Syncing: chain download in progress      synced=100.00% chain=20.58GiB   headers=18,646,191@5.37GiB   bodies=18,646,065@12.26GiB  receipts=18,646,065@2.95GiB   eta=2.748ms
Nov 03 15:06:11 localhost geth[709]: WARN [11-03|15:06:11.609] Pivot became stale, moving               old=18,646,066 new=18,646,130
Nov 03 15:06:11 localhost geth[709]: INFO [11-03|15:06:11.764] Syncing: state healing in progress       accounts=66,857,806@4.16GiB    slots=11,072,820@746.18MiB codes=28813@85.97MiB nodes=105,560,466@10.66GiB pending=136,433
Nov 03 15:06:11 localhost geth[709]: WARN [11-03|15:06:11.850] Unexpected trienode heal packet          peer=cc5aa975 reqid=5,955,817,026,556,111,914
Nov 03 15:06:12 localhost geth[709]: INFO [11-03|15:06:12.769] Syncing: chain download in progress      synced=100.00% chain=20.58GiB   headers=18,646,192@5.37GiB   bodies=18,646,129@12.26GiB  receipts=18,646,129@2.95GiB   eta=2.749ms
Nov 03 15:06:20 localhost geth[709]: INFO [11-03|15:06:20.775] Syncing: chain download in progress      synced=100.00% chain=20.58GiB   headers=18,646,192@5.37GiB   bodies=18,646,129@12.26GiB  receipts=18,646,129@2.95GiB   eta=2.749ms

[ziogaschr]

@YuXiaoCoder @leevlad can you confirm if you still have issues with syncing?

[leevlad]

Hello @ziogaschr my node ended up syncing after about 3 days and has been staying in sync since, though I haven't tried restarting it since then either.