Description
Greetings! I've been exploring & diving into this library into the context of reading data directly from a live rippled nodestore. Periodically a fetch operation will sporadically fail with the 'key_not_found' error. Subsequently trying to fetch the same key using a new database connection succeeds. I'm 100% sure the values exist in the DB before starting this process.
After exploring the code my primary thought as to the cause of this is a race condition, where the rippled process is flushing data to the nodestore file just as I'm trying to read the bucket containing the data. This could result in a lookup failure, though admittedly I haven't been able to pinpoint this as being the exact cause. Subsequent lookups of the same key using the same database connection fail until it is closed / reopened, leading me to believe internal in-memory cache buckets are getting out of sync (or similar). Does this sound like it could be the cause of this issue or if not any idea on what could be the problem?
If this is the issue, care would need to be taken to support multi-process access for this to be resolved. Sqlite3 employs on-disk locking solutions to facilitate this, but even there that is less than ideal. Would there be any interest in adding something similar to this library? For the time being, we can live with closing/reopening the db on failure, but it'd be nice to utilize something more robust!
Appreciate any insights.