Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions data-extract-langchain4j/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ In this example, we'll convert those text conversations into Java Objects that c
image::schema.png[]

In order to achieve this extraction, we'll need a https://en.wikipedia.org/wiki/Large_language_model[Large Language Model (LLM)] and related serving framework that natively supports https://ollama.com/blog/structured-outputs[JSON structured output].
Here, we choose https://ollama.com/library/granite3.3:2b[granite3.3:2b] served through https://ollama.com/[ollama] as it seems compute friendly and under Apache V2 license.
Here, we choose https://ollama.com/library/granite4:3b-h[granite4:3b-h] served through https://ollama.com/[ollama] as it seems compute friendly and under Apache V2 license.
In order to request inference to the served model, we'll use the high-level LangChain4j APIs like https://docs.langchain4j.dev/tutorials/ai-services[AiServices].

=== Start the Large Language Model
Expand All @@ -24,21 +24,21 @@ Let's start a container to serve the LLM with Ollama, in a first shell type:

[source,shell]
----
docker run --rm -it -v cqex-data-extract-ollama:/root/.ollama -p 11434:11434 --name cqex-data-extract-ollama ollama/ollama:0.9.3
docker run --rm -it -v cqex-data-extract-ollama:/root/.ollama -p 11434:11434 --name cqex-data-extract-ollama ollama/ollama:0.14.1
----

After a moment, a log like below should be output:

[source,shell]
----
time=2026-01-06T14:21:55.578Z level=INFO source=types.go:130 msg="inference compute" id=0 library=cpu variant="" compute="" driver=0.0 name="" total="62.2 GiB" available="49.9 GiB"
time=2026-01-07T14:28:19.092Z level=INFO source=types.go:60 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="62.2 GiB" available="62.2 GiB"
----

Then, download the codellama model, in a second shell type:

[source,shell]
----
docker exec -it cqex-data-extract-ollama ollama pull granite3.3:2b
docker exec -it cqex-data-extract-ollama ollama pull granite4:3b-h
----

After a moment, log like below should be output:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ public String toString() {
String CUSTOM_POJO_EXTRACT_PROMPT = "Extract information about a customer from the transcript delimited by triple backticks: ```{{text}}```."
+ "The customerBirthday field should be formatted as {{dateFormat}}."
+ "The summary field should concisely relate the customer main ask."
+ "Source any extracted field values from what is explicitly mentioned in the transcript.";
+ "Source any extracted field values from what is explicitly mentioned in the transcript."
+ "Extracted field values should be as accurate as possible.";

/**
* The text and dateFormat parameters of this method are automatically injected as {{text}} & {{dateFormat}} in the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ quarkus.default-locale=en_US
# Adjust as per your where your Ollama instance is running
langchain4j.ollama.base-url=http://localhost:11434
# The chat model to use
langchain4j.ollama.chat-model.model-id=granite3.3:2b
langchain4j.ollama.chat-model.model-id=granite4:3b-h
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ public class OllamaTestResource implements QuarkusTestResourceLifecycleManager {

private static final Logger LOG = LoggerFactory.getLogger(OllamaTestResource.class);

private static final String OLLAMA_IMAGE = "ollama/ollama:0.9.3";
private static final String OLLAMA_IMAGE = "ollama/ollama:0.14.1";
private static final int OLLAMA_SERVER_PORT = 11434;

private static final String MODE_MOCK = "mock";
Expand Down

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"id" : "4b63856f-cac5-4bc1-b66a-d01255ac40e0",
"name" : "api_chat",
"request" : {
"url" : "/api/chat",
"method" : "POST",
"bodyPatterns" : [ {
"equalToJson" : "{\n \"model\" : \"granite4:3b-h\",\n \"messages\" : [ {\n \"role\" : \"user\",\n \"content\" : \"Extract information about a customer from the transcript delimited by triple backticks: ```Operator: Hello, how may I help you ?\\nCustomer: Hello, I'm calling because I need to declare an accident on my main vehicle.\\nOperator: Ok, can you please give me your name ?\\nCustomer: My name is Sarah London.\\nOperator: Could you please give me your birth date ?\\nCustomer: 1986, July the 10th.\\nOperator: Ok, I've got your contract and I'm happy to share with you that we'll be able to reimburse all expenses linked to this accident.\\nCustomer: Oh great, many thanks.```.The customerBirthday field should be formatted as YYYY-MM-DD.The summary field should concisely relate the customer main ask.Source any extracted field values from what is explicitly mentioned in the transcript.Extracted field values should be as accurate as possible.\"\n } ],\n \"options\" : {\n \"temperature\" : 0.0,\n \"top_k\" : 1,\n \"top_p\" : 0.1,\n \"stop\" : [ ]\n },\n \"format\" : {\n \"type\" : \"object\",\n \"properties\" : {\n \"customerSatisfied\" : {\n \"type\" : \"boolean\"\n },\n \"customerName\" : {\n \"type\" : \"string\"\n },\n \"customerBirthday\" : {\n \"type\" : \"object\",\n \"properties\" : {\n \"year\" : {\n \"type\" : \"integer\"\n },\n \"month\" : {\n \"type\" : \"integer\"\n },\n \"day\" : {\n \"type\" : \"integer\"\n }\n },\n \"required\" : [ ]\n },\n \"summary\" : {\n \"type\" : \"string\"\n }\n },\n \"required\" : [ \"customerSatisfied\", \"customerName\", \"customerBirthday\", \"summary\" ]\n},\n \"stream\" : false,\n \"tools\" : [ ]\n}",
"ignoreArrayOrder" : true,
"ignoreExtraElements" : true
} ]
},
"response" : {
"status" : 200,
"body" : "{\"model\":\"granite4:3b-h\",\"created_at\":\"2026-01-15T15:03:26.59496985Z\",\"message\":{\"role\":\"assistant\",\"content\":\"{\\n \\\"customerSatisfied\\\": true,\\n \\\"customerName\\\": \\\"Sarah London\\\",\\n \\\"customerBirthday\\\": {\\\"year\\\": 1986, \\\"month\\\": 7, \\\"day\\\": 10},\\n \\\"summary\\\": \\\"The customer, Sarah London, called to declare an accident on her main vehicle. The operator confirmed that all expenses linked to the accident would be reimbursed.\\\"\\n}\"},\"done\":true,\"done_reason\":\"stop\",\"total_duration\":10290825797,\"load_duration\":1189765918,\"prompt_eval_count\":205,\"prompt_eval_duration\":3886962278,\"eval_count\":81,\"eval_duration\":4312366938}",
"headers" : {
"Date" : "Thu, 15 Jan 2026 15:03:26 GMT",
"Content-Type" : "application/json; charset=utf-8"
}
},
"uuid" : "4b63856f-cac5-4bc1-b66a-d01255ac40e0",
"persistent" : true,
"insertionIndex" : 6
}
Loading