Conditional Deletes

[qianyongyu]

Feature Request / Improvement

Iceberg merge-on-read (MOR) supports both positional deletes and equality deletes. I'm curious what prevents the equality delete from becoming a more generic 'conditional delete'? If we don't want to go crazy with the expressiveness of the conditions, something simple like keyA = "X", keyB <"Y", keyC > "Z" will still solve a lot of tricky problems.

As a concrete example, I'm trying to find a solution for a change-data-capture problem where each mutation event has a key and a sequencer. Whenever I write a new event to the table, it would be nice to wipe out previous events with the same key but older sequencers. A conditional delete seems like the only solution to avoid reading all the previous files.

Query engine

None

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time

[pvary]

@qianyongyu: The community thinks that even equality deletes are often pushing too big of the burden to the readers. See: [DISCUSS] - Deprecate Equality Deletes-Apache Mail Archives and considering deprecating them. I think adding even more features there would be problematic.

OTOH when we start thinking about alternative solutions your input could be valuable.

Thanks,
Peter