Cursors

[jb55]

We should support cursor queries. These don't return results in one-shot, but allow you to stream results out incrementally.

This may simplify the design of:

• Filter and reducer queries #33

So let's do this first

Goals

It would be nice to make a cursor "indexable" at the cost of extra memory. This is a memory of which indices have been visited:

operations: cursor_start, cursor_next, cursor_next

index    operation
---------------------
          cursor_start
a   
b
c
d       <-cursor_next
e
f
g       <-cursor_next
h

resulting indexable cursor memory:

cursor_index = [d, g]

cursor_at(0) => d
cursor_at(1) => g

cursor_jump_to(0) => d
cursor_next       => g

Indexable cursors would allow you to perform resumable and paginated queries in realtime. This will enable infinite scroll, you will also be able to index/seek into these queries at any point, which will be useful for building performant infinite timeline renderers: “render notes 5..8 in this cursor query”

Alternate approach

Alternately, we can simply add a function that gets the current index key from the cursor position:

data[] index_key = cursor_index()

this would allow us to save these keys in memory outside of nostrdb to create our own indexable cursor externally. This removes a lot of complexity (from nostrdb at least).

Let's say we did a query: {"kinds":[1]} and scrolled down to cursor index 10 with key A. We could close the transaction and save that index key. When we start scrolling again we can restart the query starting at A

Our virtual table renderer would see that we have cursor_index_key A saved at index 10 (the 11th result from the cursor query), When we try to render 11, we might notice we only have index 10, so we could resume the cursor query at index key A. This would enable resumable cursor queries, which should be able to implement everything we need?

We should only need to save at most one index key at the end of the scroll position, because all other entries are like what we have now in notedeck: it is simply an array of the indexed result (typically the note key). This is the index's value.