LangChain4j LLM Adapter: Support StreamingChatModel

[vorburger]

I've just tired out the (very awesome!) #102 from @glaforge.

It seems like LangChain4j's StreamingChatModel is not yet supported (with the PR as is today), I've hit:

java.lang.IllegalStateException: ChatModel is not configured
    at com.google.adk.models.langchain4j.LangChain4j.generateContent (LangChain4j.java:159)
    at com.google.adk.flows.llmflows.BaseLlmFlow.lambda$callLlm$1 (BaseLlmFlow.java:189)
    at io.reactivex.rxjava3.internal.operators.flowable.FlowableDefer.subscribeActual (FlowableDefer.java:35)
...

My suggestion would be to to proceed to with getting #102 merged without streaming already, and then use this issue to add that (eventually; no rush / stress). Rome wasn't built in 1 day, so (to me) it's totally fine to build this incrementally, instead of getting #102 to be "perfect".

[glaforge]

There are different constructors for the LangChain4j class: you can pass both a streaming and non-streaming LangChain4j model. If you use the ADK Dev UI, and want to be able to switch between streaming and non-streaming modes, you'll have to use the constructor that passes a StreamingChatModel and a ChatModel.

So unless I'm missing something here, streaming is actually supported.

[glaforge]

For example if you want to use Claude, you'd have to do:

.model(new LangChain4j(claudeModel, claudeStreamingModel, "claude-3-7-sonnet-20250219"))

(Also note that some LangChain4j models require that you specify the model twice: both in the definition of the LangChain4j model, and here as a parameter in this constructor. But it's a bug for those models.)

[vorburger]

Check out https://github.com/vorburger/vorburger.ch-Notes/blob/develop/ml/adk-ollama.md! (Based on what we discussed together yesterday.)

With that, if we change the OllamaChatModel here to an OllamaStreamingChatModel, then I'm getting the exception above.

[glaforge]

Could you please show your model definitions, your agent, etc?

I'm doing the following, and I can switch back and forth between streaming and non streaming in the Dev UI:

OllamaChatModel ollamaChatModel = OllamaChatModel.builder()
    .modelName("qwen3:1.7b")
    .baseUrl("http://127.0.0.1:11434")
    .build();

OllamaStreamingChatModel ollamaStreamingChatModel = OllamaStreamingChatModel.builder()
    .modelName("qwen3:1.7b")
    .baseUrl("http://127.0.0.1:11434")
    .build();

return LlmAgent.builder()
    .name("science-app")
    .description("Science teacher agent")
    .model(new LangChain4j(ollamaChatModel, ollamaStreamingChatModel, "qwen3:1.7b"))
    .instruction("""
        You're a friendly but detailed-oriented science teacher
        who gives highly detailed and long answers
        to all questions about science asked by users.
        """)
    .build();

I tried with Qwen, but I also tried with Gemma3:4B, and it worked too.

[vorburger]

Could you please show your model definitions, your agent, etc?

see https://github.com/enola-dev/LearningADK/tree/main/LangChain4j

.model(new LangChain4j(ollamaChatModel, ollamaStreamingChatModel, "qwen3:1.7b"))

Ah! Here I used the other (non-streaming) constructor, only; that must be the reason.

So will we also need both a streaming and a non-streaming LangChain4j connector for the ADK adapter?

Then... why do we have the other constructor? 😆 Could that help to avoid this confusion for future users?

Or "it depends"? On what? Or with the ADK being all RX'd (Flowable everywhere) anyway, might it make sense to require (only) a *StreamingChatModel, to keep the code simpler? Or are there some LangChain4j integrations which are not streaming at all? (I have not had a closer look.)

Or, at the very least we shuld add more clear (JavaDoc) documentation?

[glaforge]

There are several constructors for the key 3 use cases: 1) someone wants to just use a non-streaming model, 2) someone wants to just use a streaming model, or 3) someone wants to have the possibility to switch between both.

I did 3) with the constructor taking both streaming and non streaming models, but mainly because we can switch between streaming and non-streaming in the ADK Dev UI. I don't even see the point of being able to do that in the first place though.

I think not all models in LangChain4j support streaming. They usually all start with non streaming, and then later on add streaming.

So, at this point, at least, with all those constructors, users have options. And it's normal if they get an exception when requesting streaming but it was configured with a non-streaming model, and vice versa. That's what I would expect. But maybe we can improve the error message? (like, "A streaming request was made but this model is configured for non-streaming mode", and vice versa?)