[BUG] [flytekit] Bug in list of basemodels type conversion

[pimdh]

Flyte & Flytekit version

flytekit 1.16.3

Describe the bug

from flytekit import workflow, task
from pydantic import BaseModel


class M(BaseModel):
    pass


class C(BaseModel):
    xs: list[M] = []


@task
def f(x: list[M]) -> None:
    print(x)


@task
def z() -> C:
    return C()


@workflow
def d() -> None:
    f(z().xs)


if __name__ == "__main__":
    d()

Running gives error:

TypeTransformerFailedError: Error encountered while executing 'bug.d':
  Error encountered while converting inputs of 'bug.f':
  Error converting input 'x' at position 0:
Literal value: Flyte Serialized object (Literal):
  scalar:
    binary:
      value: b'\x90'
      tag: msgpack
Expected Python type: list[__main__.M]
Exception: Field "" of type M in __root__ is not serializable

Expected behavior

I'd expect that code to work without failure and print an empty list.

It has something to do with the fact that the z().xs gives a literal scalar containing msgpack of a list of dicts. If I'd have done

def d():
    f(C())

then the literal given to f is instead a literal of a collection of msgpacks of a dict. This is handled correctly by the type engine.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[runllm]

This is a known bug in Flytekit: when passing a list of Pydantic BaseModels (e.g., list[M]) as a field in another BaseModel, Flytekit serializes an empty list as a single msgpack-encoded binary literal, not as a collection of msgpack-encoded elements. The type engine expects a collection, so deserialization fails with "Field '' of type M in root is not serializable." This issue is confirmed by test cases and discussions in the Flyte repository, and there is no official fix yet. The recommended workaround is to avoid using a list of BaseModels as a field in another BaseModel, or to use dataclasses instead, which are handled correctly by the type engine. See the discussion and test cases in test_pydantic_basemodel_in_dataclass.py and flyte/issues/5925.

Would you like a step-by-step explanation of why this happens and possible workarounds?

Sources:

• test_pydantic_basemodel_in_dataclass.py

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

[pimdh]

I don't think I agree with the LLM's analysis