Enhancement: viewkey-based background refresh for libwallet

[hyc]

Just recapping the idea in this bounty https://bounties.monero.social/posts/10/implement-background-wallet-sync-via-view-keys-on-monerujo (also discussed in m2049r/xmrwallet#785)

The idea I'm proposing is to add a background-sync-cache that is separate from the main wallet cache, encrypted with its own secret key, and used to perform refreshes with the wallet viewkey while the main wallet is idle/off. The point is to allow the wallet to stay sync'd with the blockchain without leaving the private spendkey exposed or leaving the main wallet cache in decrypted state.

When the user returns to active use of the wallet and enters their passphrase to unlock/decrypt the main wallet cache, it can then suck in whatever transaction info was recorded in the background-sync-cache, thus avoiding having to wait for a refresh in realtime.

I haven't looked in detail at the wallet cache format recently but I suggest the background-sync-cache file should be written incrementally in append-only fashion, with each transaction record encrypted independently, and in a length-prefixed binary format. This way the refresher doesn't need to constantly re-read and re-encrypt the existing data before writing new records to the file.

The basic functionality to enable periodic background refreshing should be built into libwallet, with hooks to allow the frontend to turn it on and off (e.g. for a smartphone connecting/disconnecting wifi, etc.), set poll interval, etc.

[busyboredom]

Would it be possible to also remain continuously connected to the selected remote node in the background? I've found that even when fully synchronized, wallets can often take upwards of 20 seconds to connect (even when using a private node hosted on the local network). This can be a pain when you're just trying to pay your buddy back for beer and you're both just standing there awkwardly holding your phones, waiting.

edit: continuously connected as opposed to periodically refreshing, because simply periodically connecting for a quick 1 minute sync every 15 minutes means that when a user finally opens the app, there'd still be a 14/15 chance of the user needing to wait for the connection.

[selsta]

@busyboredom on what kind of hardware is your node hosted? 20 seconds sounds long for local network

[busyboredom]

@selsta it's hosted on an ubuntu machine with an SSD, 12GB RAM and a 3GHz quad-core from 2015.

[selsta]

@busyboredom Can you start your daemon with --rpc-ssl disabled and check if you still have to wait 20 seconds? Connecting takes no longer than 1-2 seconds here on my local and remote node.

[busyboredom]

No problem, I gave that a shot just now. After starting my node with that argument, I connected 4 times from cake wallet using the "reconnect" button and got these times:

1. 5 seconds
2. 17 seconds
3. 13 seconds
4. 38 seconds

Also maybe valuable datapoint is that when I simply call rpc endpoints with curl from my desktop, I see a response pretty much immediately, so I'm inclined to think the slowness is in the wallet rather than the node.

[SamsungGalaxyPlayer]

While I like the idea of optionally revealing the view key to a local process on the machine which can scan in the background while the spend key remains locked, we can even get better than this.

The ideal for almost all wallets is to be able to send the view key to a chosen remote node, which can then act as a lightweight server and scan all blocks on the server side. Then, the wallet can request for a status update every time it connects to the remote node. This is preferred in most cases because:

• It avoids the need to have a background process running locally (which is effectively impossible on iOS)
• It saves bandwidth since blocks can be scanned directly at the source (the full node)
• It works even if a phone or computer is off; the server is far more likely to be running 24/7

The main disadvantage is that local notices will be more difficult to display if funds are received if the wallet isn't connected to the remote node. Still, a persistent connection to the remote node listening for relevant transactions is more efficient than keeping an open connection to scan all blocks locally.

Ideally, both options should be made available. But in terms of prioritization, we really should allow remote nodes to function as lightweight servers imo.

[hyc]

While I like the idea of optionally revealing the view key to a local process on the machine which can scan in the background while the spend key remains locked, we can even get better than this.

I think you need to also consider the very serious cons to your suggestion:

• It changes the security and threat model for the nodes. Instead of storing no sensitive data, they become ripe targets for attacks
• It doesn't scale; adding workload to nodes will eventually overload them. Keeping workload distributed to individual wallets is far more scalable.

[SamsungGalaxyPlayer]

All of these situations place more stress on nodes. Background connections to remote nodes aren't great either :/

Again, these aren't mutually exclusive features. I want Monero to support both!

If people don't run their own nodes, then the solution described here is better from their UX perspective. It also puts a lot more stress on the nodes though if they request an effectively permanent connection to send all block data continuously instead of only when open. They each have their own different tradeoffs. As someone who runs my own node, I would much rather store my private view key there and push all the hard work to that server.

[hyc]

Continually sending blocks is no different from what p2p already does. But the change in sensitivity of data residing on the node is a major difference. IMO it's a huge mistake.

[SamsungGalaxyPlayer]

Eh, don't use the feature then :p

[hyc]

Cavalierly dismissing security concerns isn't really a good way to proceed here.

[Gingeropolous]

random idea, but would it be useful to retrieve and store in this secondary cache a bunch of fake outputs, so even if the cache is compromised, the attacker has to deal that? The users wallet will have to re-check the cache, but its not like its going through the blockchain.

[tevador]

but would it be useful to retrieve and store in this secondary cache a bunch of fake outputs, so even if the cache is compromised, the attacker has to deal that?

If I understand correctly, the cache contains the private view key, so this wouldn't work.

However, there is a proposal in the Seraphis addressing schemes to have a separate keypair for view tags. If this private key is shared with the node, it can only detect a subset of all outputs that contains all of the real outputs owned by the wallet. The amount of leaked information is low because the node still cannot recognize which outputs are real, but the number of outputs the wallet has to check is significantly reduced.

Additionally, this view tag private key has the interesting property that any random private key will match some subset of outputs, which makes it possible to use decoy keys.

[r4v3r23]

is this necessary with the upcoming view-tag PR?

also wallet scan times aren't really an issue, unless you haven't used your wallet in a few months.