Skip to content

Implement sync get() #590

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Conversation

martindurant
Copy link
Collaborator

@pfackeldey : as stands, pytest runs on my system and master dask, with

171 failed, 457 passed, 5 skipped, 22 xfailed, 2 warnings

@martindurant martindurant marked this pull request as draft May 9, 2025 16:10
@martindurant
Copy link
Collaborator Author

Update:

24 failed, 604 passed, 5 skipped, 22 xfailed, 2 warnings

@martindurant
Copy link
Collaborator Author

Update:

15 failed, 613 passed, 5 skipped, 22 xfailed, 2 warnings

@pfackeldey
Copy link
Collaborator

I can't push to your branch, but this patch:

diff --git a/src/dask_awkward/lib/core.py b/src/dask_awkward/lib/core.py
index 4139b2e..ed1714b 100644
--- a/src/dask_awkward/lib/core.py
+++ b/src/dask_awkward/lib/core.py
@@ -1815,7 +1815,7 @@ def compute_typetracer(dsk: HighLevelGraph, name: str) -> ak.Array:
     return typetracer_array(
         Delayed(
             key,
-            dsk.cull({key}),
+            dsk.cull_layers({name}),
             layer=name,
         ).compute()
     )

brings it further down to:

8 failed, 621 passed, 4 skipped, 22 xfailed, 2 warnings

@martindurant
Copy link
Collaborator Author

I changed it locally and it didn't seem to make any difference to me? I notice that the CI run "conda Tests / pytest (macos-latest, 3.11)" also shows only 8 failures, so perhaps my environment isn't quite right.

Strange that you can't push to this branch - is there some permission I can give? I do have "Maintainers are allowed to edit this pull request" set.

@pfackeldey
Copy link
Collaborator

Strange that you can't push to this branch - is there some permission I can give? I do have "Maintainers are allowed to edit this pull request" set.

I'll try a fresh setup soon for me, I'm sure I'm doing something wrong here...

@martindurant
Copy link
Collaborator Author

martindurant commented May 20, 2025

@pfackeldey

7 failed, 621 passed, 5 skipped, 22 xfailed, 2 warnings

with latest

(reports 9 failed in CI)

@martindurant
Copy link
Collaborator Author

The ramaining problem is recursion: the arguments are not being followed all the way down or classic-task-like (func, arg, arg, ...) not getting called. Probably, _get_sync and _unwind should be one recursive function, but I can't quite phrase it right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants