[bug]: Channel Closing hangs for a very long time

[ziggie1984]

Pre-Submission Checklist

• I have searched the existing issues and believe this is a new bug.
• I am not asking a question about how to use lnd, but reporting a bug (otherwise open a discussion).

LND Version

v19.3

LND Configuration

not important

Backend Version

not important

Backend Configuration

not important

OS/Distribution

not important

Bug Details & Steps to Reproduce

Legacy Channel Closing hangs for almost 4 hours

See also logs:

lnd 0.19.3

trying to close channel xxx:0 - unesponsive, closed after 4 hours:
2025-09-23 16:01:38.909 [INF] PEER: Peer(xxx): Local close channel request is going to be delivered to the peer
2025-09-23 16:01:38.959 [INF] PEER: Peer(xxx): Delivery addr for channel close: xxx
2025-09-23 16:01:38.976 [INF] CHCL: ChannelPoint(xxx:0): initiating shutdown
...
2025-09-23 19:19:19.297 [INF] CNCT: ChannelArbitrator(xxx:0) no close event
...
2025-09-23 19:18:03.834 [INF] DISC: GossipSyncer(xxx): applying new remote update horizon: start=2025-09-23 1
8:52:22 +0200 CEST, end=2161-10-31 00:20:37 +0100 CET, backlog_size=10

2025-09-23 19:35:54.708 [INF] DISC: GossipSyncer(xxx): applying new remote update horizon: start=2042-07-14 03:04:00 +0200 CEST, end=2042-07-14 03:04:00 +0200 CEST, backlog_size=0

2025-09-23 20:08:19.954 [INF] NANN: Announcing channel(xxx:0) disabled [requested]
2025-09-23 20:08:21.069 [INF] CHCL: ChannelPoint(xxx:0): sending shutdown message
2025-09-23 20:08:21.588 [INF] DISC: GossipSyncer(xxx): starting query for 851 new chans
2025-09-23 20:08:21.588 [INF] DISC: GossipSyncer(xxx): querying for 500 new channels
2025-09-23 20:08:21.589 [INF] CHCL: ChannelPoint(xxx:0): shutdown response received, entering fee negotiation
2025-09-23 20:08:21.589 [INF] HSWC: ChannelLink(xxx:0): stopping
2025-09-23 20:08:21.605 [INF] CHCL: Ideal fee for closure of ChannelPoint(xxx:0) is: 195 sat (max_fee=585 sat)
2025-09-23 20:08:21.641 [INF] CHCL: ChannelPoint(xxx:0): computing fee compromise, ideal=195, last_sent=0, remote_offer=195
2025-09-23 20:08:21.642 [INF] CHCL: ChannelPoint(xxx:0): proposing fee of 195 sat to close chan
2025-09-23 20:08:21.647 [INF] CHCL: ChannelPoint(xxx:0) fee of 0.00000195 BTC accepted, ending negotiation

Expected Behavior

LND immediately starts the closing process

Debug Information

No response

Environment

No response

[ziggie1984]

user runs LND with: postgres-kvdb backend, I think this issue will be resolved once the user runs LND20 with the GRAPH db being available in native SQL.

[ziggie1984]

[ziggie1984]

The complete semaphore of the AuthenticatedGossiper (1000) is locked up, because PruneGraph takes forever ....

[Roasbeef]

Yeah looks like something is blocked for a while on that mutex.

I think #10128 will help with cases like this, as right now we'll hold the cache mutex the entire time we're reading a backlog from disk. This chokes throughput, as many of the other graph calls need to obtain this mutex.

I think there're many other cases where we can go with more fine grained locking with c.cacheMu.