POD5 has encountered an error: 'UDF called without return type, but was not able to infer the output type.

[henauts]

---

name: Bug report
about: Create a report to help us improve
title: 'UDF called without return type - pod5 subset'
labels: 'Template, output'
assignees: 'Sarah - henautjacobs@gmail.com'

---

Issue Description

I tried to run the pod5 subset a lot of times with different parameter/variable names and kept receiving the same error. "POD5 has encountered an error: 'UDF called without return type, but was not able to infer the output type.
This used to be allowed but lead to unpredictable results. To fix this problem, either provide a return datatype or execute the UDF in an eager context (e.g. in map_columns)".

I tried to add a template, but it keeps showing the same thing.

Logs

Traceback (most recent call last):
File "/home/sarah/miniconda3/envs/pod5/bin/pod5", line 7, in
sys.exit(main())
~~~~^^
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/main.py", line 61, in main
return run_tool(parser)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/parsers.py", line 41, in run_tool
raise exc
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/parsers.py", line 38, in run_tool
return tool_func(**kwargs)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/parsers.py", line 578, in run
return subset_pod5(**kwargs)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/utils.py", line 63, in wrapper
raise exc
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/utils.py", line 60, in wrapper
ret = func(*args, **kwargs)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/pod5_subset.py", line 658, in subset_pod5
subset_pod5s_with_mapping(
~~~~~~~~~~~~~~~~~~~~~~~~~^
inputs=_inputs,
^^^^^^^^^^^^^^^
...<5 lines>...
force_overwrite=force_overwrite,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/utils.py", line 63, in wrapper
raise exc
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/utils.py", line 60, in wrapper
ret = func(*args, **kwargs)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/pod5_subset.py", line 600, in subset_pod5s_with_mapping
assert_overwrite_ok(targets, force_overwrite)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/utils.py", line 63, in wrapper
raise exc
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/utils.py", line 60, in wrapper
ret = func(*args, **kwargs)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/pod5/tools/pod5_subset.py", line 237, in assert_overwrite_ok
.collect()
~~~~~~~^^
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/polars/_utils/deprecation.py", line 97, in wrapper
return function(*args, **kwargs)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/polars/lazyframe/opt_flags.py", line 330, in wrapper
return function(*args, **kwargs)
File "/home/sarah/miniconda3/envs/pod5/lib/python3.13/site-packages/polars/lazyframe/frame.py", line 2407, in collect
return wrap_df(ldf.collect(engine, callback))
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
polars.exceptions.InvalidOperationError: UDF called without return type, but was not able to infer the output type.

This used to be allowed but lead to unpredictable results. To fix this problem, either provide a return datatype or execute the UDF in an eager context (e.g. in map_columns).

Resolved plan until failure:

---> FAILED HERE RESOLVING 'sink' <---

SELECT [col("__dest_fname").unique()]
WITH_COLUMNS:
["/media/mordor/sarah/ds/run2.split_by_channel".str.concat_horizontal([col("__dest_fname")]).strict_cast(Categorical).alias("__dest_fname")]
WITH_COLUMNS:
[col("channel").str.concat_horizontal(["_run2.pod5"]).strict_cast(Categorical).alias("__dest_fname"), col("read_id").alias("__read_id")]
DF ["read_id", "channel"]; PROJECT */2 COLUMNS

Specifications

• Pod5 Version: 0.3.28
• Python Version: 3.13.5
• Platform: Ubuntu 24.04

[HalfPhoton]

Hi @henauts,
Can you please return the result of running the following command to report the pod5 and polars versions?

pip list | grep -e "pod5" -e "polars"

I believe this is due to the most recent polars 1.33 release
which has breaking changes.

It should be resolved by re-installing polars and excluding the most recent release.

pip3 install "polars~=1.20,<1.33"

Kind regards,
Rich

[henauts]

Hey, Rich.

My running version of polars was indeed 1.33. I changed to the one you said, and it gave me a different error "POD5 has encountered an error: ''return_dtype' of function python_udf must be set", but I saw that this error was reported before in issues. Polars 1.31 version worked!

Cheers,
Sarah

[HalfPhoton]

Thanks for letting us know - we'll release a fix for this soon.