Lost channel and UTXO after upgrading LND while channel partner offline

[ChuckinBits]

Summary

I've been running LND on Start9OS and I recently had a hardware failure but luckily I made a network backup of my Start9 and restored it on a new device. All of my lightning channels closed except for 1 which was in the Thunderhub Channels Pending tab as my channel partner was offline. While I was waiting for my channel partner to come back online or for the force close, I foolishly decided to upgrade from LND 0.19.1 to LND 0.19.3 and this is where I suspect my problem came from. Fast forward a week or so and I was able to contact my channel partner, they came back online and the channel successfully closed but my node has not recognized the closed channel, UTXO or chain history of that closing transaction. However, it does show closed in Amboss.space and confirmed in mempool.space.
Now those sats are stranded in an address that my node doesn’t see for some reason.

Background

Before upgrading LND, I could see all of my recently closed channels and the pending one in ThunderHub. After the upgrade, in Thunderhub, my on-chain balance is still there and the chain transaction history going back years is still there but no channel history in closed or pending tabs. I confirmed this with the lncli listchannels and lncli pending channels commands which showed nothing. In Amboss.space the last update received from my node was before my LND upgrade. I also verified that my Node ID and pubkey still match in Amboss and on Thunderhub. Before I got in contact with my channel partner, I tried using lncli closechannel --force and it said something like channel not recognized. Finally after the channel closed I did the LND Reset Wallet Transactions command and after completion it still showed the same amount of sats as before.

Current State

Now that the missing sats are on-chain but not recognized by my node I’m worried to do anything drastic. For the next step, my thoughts are to sweep the much more significant on-chain balance I currently have to cold storage, backup LND to a new location, uninstall LND, then restore LND from the old backup file. I have not done that yet because I don’t want to potentially screw anything else up until I at least have some input from you all about this potential bug and the best way to attempt to retrieve my lost UTXO. I’d also hate for any more plebs to have to go something similar as well.

I appreciate your thoughts. Thank you!

[bitromortac]

Have you made any progress? In order to provide help we need to get more details about your channel. Did your channel partner force close? If you see that the channel was force closed, you should be able to recover the utxo, it just seems like lnd isn't aware of the channel and therefore doesn't attempt to sweep the output (the channel still doesn't appear in listchannels or pendingchannels?). If you don't want to share that data here, you can also get in contact on slack https://lightning.engineering/slack.html and send a PM (same username).

[ChuckinBits]

Thanks for responding. Yes the channel was force closed by my partner. Here is the update of where I am at.
So I decided to withdraw most of my on-chain balance to cold storage and left over 100,000 sats in the LND wallet just in case. I then saved a NEW backup of LND to a new location, deleted LND, then restored from my OLD backup. After restoring, I could see all of my previous channels in the Thunderhub Pending tab. However my new on-chain balance is less than half of what I left on it before restoring. The big withdrawal transaction is showing in the Chain Transaction history. Perhaps its not recognizing my withdrawal change address. Not a huge deal as I assume I will be able to recover that later by restoring from the NEW LND backup I created earlier.

The lost channel in question was showing in the Channels Pending list and there it showed the correct Close Transaction but it is still not showing under the Chain Transactions history. I ran the lncli pendingchannels command and it showed information on this channel and the local and limbo balance is what I can see on chain but my node still doesn’t see it. I decided to try the Reset Wallet Transactions again with no luck. After running that command and letting everything get synced, all of the channels and information have disappeared under all of the tabs (Open, Pending, Closed). It's as if I have never opened any channels before. They are all blank. At this point I’m not sure what to do. The only thing I can think of is restoring from the old backup again so that LND can see the channel information as pending and then perhaps using chantools sweepremoteclosed command or something. Thoughts?