batch endpoint triggers PydanticUserError due to List[config] in dynamic model creation

[k1dav]

I'm encountering an error when enabling the batch endpoint using the add_routes function with a dynamically created batch request model.

Here is a minimal reproducible example:

from fastapi import APIRouter
from langserve import add_routes
from langchain_core.runnables import Runnable
from langchain_core.prompts import ChatPromptTemplate
from langchain_anthropic import ChatAnthropic

router = APIRouter()

model = ChatAnthropic(model="claude-3-haiku-20240307")
prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")

add_routes(
    router,
    prompt | model,
    path="/joke",
    enabled_endpoints=[
        "invoke",
        "stream",
        "stream_log",
        "stream_events",
        "playground",
        "feedback",
        "public_trace_link",
        "input_schema",
        "config_schema",
        "output_schema",
        "config_hashes",
        "batch"  # causes error
    ]
)

When the batch endpoint is included, I get the following error:

pydantic.errors.PydanticUserError: `TypeAdapter[typing.Annotated[langserve.validation.jokeBatchRequest, <class 'langserve.validation.jokeBatchRequest'>, Body(PydanticUndefined)]]` is not fully defined; you should define `typing.Annotated[...]` and all referenced types, then call `.rebuild()` on the instance.

I traced it to the following code in langserve/validation.py:

batch_request_type = create_model(
    f"{namespace}BatchRequest",
    inputs=(List[input_type], ...),
    config=(
        Union[config, List[config]],
        Field(
            default_factory=dict,
            description=(
                "Subset of RunnableConfig object in LangChain. Either specify one "
                "config for all inputs or a list of configs with one per input."
            ),
        ),
    ),
    kwargs=(
        dict,
        Field(
            default_factory=dict,
            description="Keyword arguments to the runnable. Currently ignored.",
        ),
    ),
)
batch_request_type.model_rebuild()

Temporary workaround:
If I remove the List[config] from the Union in the config field and just use config, everything works fine. So it seems the dynamic create_model doesn’t correctly handle List[config] in this context.

Thanks in advance!

Requirements

langserve==0.3.1
pydantic==2.11.4
pydantic-settings==2.9.1
pydantic_core==2.33.2
fastapi==0.115.11

[DhavalThkkar]

All of a sudden this popped up

[rsradulescu]

We found similar error trying to show openapi docs and was fixed when we downgrade to pydantic==2.10.6

[officialgoat]

"LangServe>=0.3 fully supports Pydantic 2."

• Well, no it doesnt.

This issue is open for a while now, and no reaction.