Skip to content

NOT IN and != predicates do not prune files when min == max #14592

@joyhaldar

Description

@joyhaldar

Feature Request / Improvement

Current Behavior

InclusiveMetricsEvaluator does not prune files for NOT IN and != predicates, even when the file contains a single distinct value where min == max.

Proposed Solution

When min == max and the file has no nulls, we can safely prune:

  • For NOT IN: if the single value is in the exclusion list
  • For !=: if the single value equals the literal

Related discussion: [Slack thread link]

Query engine

None

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementPR that improves existing functionality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions