Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent pylibcudf serialization in cudf-polars #17449

Closed
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion python/cudf_polars/cudf_polars/dsl/ir.py
Original file line number Diff line number Diff line change
Expand Up @@ -840,6 +840,15 @@ def do_evaluate(
return DataFrame(columns)


class GroupbyOptions:
"""Serializable wrapper for polars GroupbyOptions."""

def __init__(self, polars_groupby_options: Any):
self.dynamic = polars_groupby_options.dynamic
self.rolling = polars_groupby_options.rolling
Comment on lines +847 to +848
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually these will also need translated, but for now we can dodge it because they are always None.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I couldn't find any examples in our tests to aid that, so for now I left it as above to make things simpler.

self.slice = polars_groupby_options.slice


class GroupBy(IR):
"""Perform a groupby."""

Expand Down Expand Up @@ -873,8 +882,8 @@ def __init__(
self.keys = tuple(keys)
self.agg_requests = tuple(agg_requests)
self.maintain_order = maintain_order
self.options = options
self.children = (df,)
self.options = GroupbyOptions(options)
if self.options.rolling:
raise NotImplementedError(
"rolling window/groupby"
Expand Down
Loading