Question about streaming operator

[cos120]

Hello~

I am using UDF and custom data structure in python, my job is mixed in io task and compute task and I want to process data in streaming. I want to known can daft schedule io task to thread pool and compute bound to process pool? Can I annotate tasks to different resources like thread pool or process pool? Thanks!

[colin-ho]

For the io task I recommend using async daft.func, so daft will execute it on an asyncio pool. https://docs.daft.ai/en/stable/custom-code/func/#eager-evaluation