[DRAFT] feat: snapshot FFI rework

[samansmink]

Opening PR to get some feedback on the FFI design here.

This is a first stab at updating the FFIs for snapshot to allow metadata reuse. I've tested this by using it in DuckDB's delta extension and are seeing a quite significant performance increase with this across high latency network, even for tiny tables!

My main question for the reviewer now is that of how to design the FFI, particularly related to breaking the existing api.

Right now we have snapshot ffi surface that can create snapshots using various params:

• specific version
• log tail (for catalog managed)
• old snapshot (for metadata reuse)

Previously, we had:

fn snapshot(
    path: KernelStringSlice,
    engine: Handle<SharedExternEngine>,
)

fn snapshot_at_version(
    path: KernelStringSlice,
    engine: Handle<SharedExternEngine>,
    version: Version,
)

In this PR i would like to propose the breaking change

fn snapshot(
    path: OptionalValue<KernelStringSlice>,
    engine: Handle<SharedExternEngine>,
    old_snapshot: OptionalValue<Handle<SharedSnapshot>>,
    version: OptionalValue<Version>,
)
fn snapshot_with_log_tail(
    path: OptionalValue<KernelStringSlice>,
    engine: Handle<SharedExternEngine>,
    old_snapshot: OptionalValue<Handle<SharedSnapshot>>,
    version: OptionalValue<Version>,
    log_tail: OptionalValue<log_path::LogPathArray>,
)

We can also consider making a non-breaking change by adding a new function, but I feel that just clutters things and having 1 is just a little nicer, even though invoking it is now a little uglier.

Would love some feedback here!

[nicklan]

cool yeah, this approach looks good for now. we'll need to also update the example c code to pass NULL for the old snapshot, but otherwise this looks great. thanks!

[nicklan]

nit:

Suggested change

	/// - `old_snapshot`: Pass an existing snapshot for optimized creation (avoids re-reading the entire log)
	/// - `old_snapshot`: Optionally pass an existing snapshot for optimized creation (avoids re-reading the entire log)