feat(optimizer): simplify filter predicate before converting row_number + filter to topn #22295
Conversation
Signed-off-by: Richard Chien <[email protected]>
This stack of pull requests is managed by Graphite. Learn more about stacking. |
stdrc
changed the title
github-actions
bot
added
type/feature
There was a problem hiding this comment.
Pull Request Overview
This PR simplifies filter predicates with arithmetic on window functions to enable TopN optimizations by converting expressions like (row_number - 1) = 0 to row_number = 1. Key changes include:
- Adding methods in OverWindowToTopNRule to simplify arithmetic expressions within filter conditions.
- Introducing FilterArithmeticRewriter that rewrites the arithmetic comparison expressions.
- Updating test cases to validate the new optimization patterns.
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/frontend/src/optimizer/rule/over_window_to_topn_rule.rs | Added logic to simplify arithmetic expressions in filter predicates and refactored expression rewriting |
| src/frontend/planner_test/tests/testdata/output/over_window_function.yaml | Updated expected logical and optimized plans for window function predicate optimization |
| src/frontend/planner_test/tests/testdata/input/over_window_function.yaml | Added new test cases validating the arithmetic simplification on window functions |
| impl OverWindowToTopNRule { | ||
| /// Simplify arithmetic expressions in filter conditions before TopN optimization | ||
| /// For example: (`row_number` - 1) = 0 -> `row_number` = 1 | ||
| fn simplify_filter_arithmetic(&self, filter: &LogicalFilter) -> Option<LogicalFilter> { |
There was a problem hiding this comment.
Maybe simplify_comparison_arithmetic?
There was a problem hiding this comment.
simplify_filter_arithmetic is called for Filter node with no brain, I guess the name works.
| LogicalProject { exprs: [t.a, t.b, t.c, $expr1] } | ||
| └─LogicalFilter { predicate: ($expr1 = 2:Int32) } | ||
| └─LogicalProject { exprs: [t.a, t.b, t.c, (row_number + 1:Int32) as $expr1] } | ||
| └─LogicalOverWindow { window_functions: [row_number() OVER(PARTITION BY t.a ORDER BY t.b ASC ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)] } |
There was a problem hiding this comment.
Potential for future optimization? Seems like rn_plus_one is always 2. I guess this can just be a project
There was a problem hiding this comment.
Indeed, but I guess if the condition is like this one, the user won't want to keep the row number in the final select list.
Signed-off-by: Richard Chien <[email protected]>
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Note
This PR is prompt-engineered by @stdrc, coded by Cursor Agent (Claude Sonnet 4), reviewed by @stdrc.
Convert filter predicate like
row_number - 1 = 0torow_number = 1so that such queries can be optimized to using TopN operator. Fix #22290.Checklist
Documentation
Release note