-
Notifications
You must be signed in to change notification settings - Fork 338
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
The spec defines metadata columns, one of which is the _pos column. This column seems instrumental for write operations positional deletes in future, as well as for having some ways for users to identify rows (combination of filepath + position). There's also a related DataFusion issue: apache/datafusion#13261.
Describe the solution you'd like
The solution should allow building a TableScan with option to return metadata columns (in this case _pos). When that happens, the library should return batches that include this extra column. The column should preserve gaps in case of filter pushdown (and in case of merge-on-read).
Willingness to contribute
I can contribute to this feature independently
liurenjie1024
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request