-
Notifications
You must be signed in to change notification settings - Fork 566
Description
Discussed in #3866
Originally posted by roeap October 18, 2025
While migrating our internals to using delta-kernel-rs, we saw that there might be some room for improvement in how we structure our main structs/apis. Certain feature implementations we struggle with IMO steam from the fact, that there is some conceptual misalignment.
Leaving aside the two separate snapshots for now, I believe DeltaTable and DeltaOps could or even should be consolidated into DeltaTable. The main reason being that we have gotten into a place where we practically are using sessions without acknowledging it (at least not always). In the abstract form the session is responsible for resources required to execute a query. For datafusion based operations we create df sessions ad-hoc, while in other cases LogStore is effectively the session.
DeltaOps just wraps a pub DeltaTable to create the builders. When consolidated this would reduce the nimber of exposed stats and hopefully improve discoverability for operations. We can then also more flexibly handle session management - i.e. allowing users to provide a datafusion session we can use to execute operations.
We may even one day want to expose dataframe-like APIs on the delta table.
SO in short in believe this would simplify our APIs as well as allow for easier future optimizations.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status