Skip to content

Commit 1e0401e

Browse files
authored
new process to integrate AI to existing project (#158)
## Purpose <!-- Describe the intention of the changes being proposed. What problem does it solve or functionality does it add? --> * ... ## Does this introduce a breaking change? <!-- Mark one with an "x". --> ``` [ ] Yes [ ] No ``` ## Pull Request Type What kind of change does this Pull Request introduce? <!-- Please check the one that applies to this PR using "x". --> ``` [ ] Bugfix [ ] Feature [ ] Code style update (formatting, local variables) [ ] Refactoring (no functional changes, no api changes) [ ] Documentation content changes [ ] Other... Please describe: ``` ## How to Test * Get the code ``` git clone [repo-address] cd [repo-name] git checkout [branch-name] npm install ``` * Test the code <!-- Add steps to run the tests suite and/or manually test --> ``` ``` ## What to Check Verify that the following are valid * ... ## Other Information <!-- Add any other helpful information that may be needed here. -->
1 parent a4154c0 commit 1e0401e

File tree

11 files changed

+1128
-14
lines changed

11 files changed

+1128
-14
lines changed

docs/05_lab_openai/05_openai.md

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -69,12 +69,12 @@ We utilize the `Github Copilot Chat` extension in VSCode to help us to initial t
6969
mkdir spring-petclinic-chat-service
7070

7171
curl https://start.spring.io/starter.tgz \
72-
-d dependencies=web,cloud-eureka,cloud-config-client,actuator,lombok,spring-ai-azure-openai \
73-
-d bootVersion=3.3.6 -d name=chat-service -d type=maven-project \
74-
-d jvmVersion=17 -d language=java -d packaging=jar \
75-
-d groupId=org.springframework.samples.petclinic -d artifactId=chat-service \
76-
-d description="Spring Petclinic Chat Service" \
77-
| tar -xzvf - -C spring-petclinic-chat-service
72+
-d dependencies=web,cloud-eureka,cloud-config-client,actuator,lombok,spring-ai-azure-openai \
73+
-d bootVersion=3.3.6 -d name=chat-service -d type=maven-project \
74+
-d jvmVersion=17 -d language=java -d packaging=jar \
75+
-d groupId=org.springframework.samples.petclinic -d artifactId=chat-service \
76+
-d description="Spring Petclinic Chat Service" \
77+
| tar -xzvf - -C spring-petclinic-chat-service
7878
```
7979

8080
We may open the new application in VSCode to next operations.
@@ -98,7 +98,8 @@ We utilize the `Github Copilot Chat` extension in VSCode to help us to initial t
9898
Here we use the latest test code from Spring AI as part of the prompt. Download the latest chat client sample to your local environment:
9999

100100
```bash
101-
wget https://raw.githubusercontent.com/spring-projects/spring-ai/refs/heads/main/models/spring-ai-azure-openai/src/test/java/org/springframework/ai/azure/openai/AzureOpenAiChatClientIT.java -P spring-petclinic-chat-service/src/main/resources/
101+
IT_FILE="https://raw.githubusercontent.com/spring-projects/spring-ai/refs/heads/1.0.0-M4/models/spring-ai-azure-openai/src/test/java/org/springframework/ai/azure/openai/AzureOpenAiChatClientIT.java"
102+
wget $IT_FILE -P spring-petclinic-chat-service/src/main/resources/
102103
```
103104

104105
Open the `Github Copilot Chat` window and drag the downloaded file into the chat window. And input the prompt:
@@ -221,17 +222,17 @@ We utilize the `Github Copilot Chat` extension in VSCode to help us to initial t
221222
@Data
222223
public class Owner implements Serializable {
223224

224-
private Integer id;
225+
private Integer id;
225226

226-
private String firstName;
227+
private String firstName;
227228

228-
private String lastName;
229+
private String lastName;
229230

230-
private String address;
231+
private String address;
231232

232-
private String city;
233+
private String city;
233234

234-
private String telephone;
235+
private String telephone;
235236
}
236237
```
237238

@@ -303,7 +304,7 @@ We utilize the `Github Copilot Chat` extension in VSCode to help us to initial t
303304

304305
```bash
305306
az containerapp update --name $APP_NAME --resource-group $RESOURCE_GROUP \
306-
--source ./spring-petclinic-$APP_NAME
307+
--source ./spring-petclinic-$APP_NAME
307308
```
308309

309310
- Verify the new RAG empowered AI bot about owners.
Lines changed: 336 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,336 @@
1+
---
2+
title: 'Integrate AI to existing project'
3+
layout: default
4+
nav_order: 4
5+
parent: 'Lab 5: Integrate with Azure OpenAI'
6+
---
7+
8+
# Integrate AI into your existing project
9+
10+
In this chapter, we will learn how to create AI Java applications using Azure OpenAI and Spring AI.
11+
12+
Start from a simple spring boot application, we will add AI components to the project and leverage AI to integrate with existing the petclinic solution we deployed in this lab.
13+
14+
# Step by step guide
15+
16+
1. We have simple sample saved in the tools directory, copy it under the folder `spring-petclinic-microservices`:
17+
18+
```bash
19+
cp -r ../tools/spring-petclinic-chat-service .
20+
```
21+
22+
1. Open the chat-service project in VSCode
23+
24+
```bash
25+
code spring-petclinic-chat-service
26+
```
27+
28+
1. In the VSCode IDE, make sure you have extension `Github Copilot Chat` installed, and login to Copilot with your github account.
29+
30+
Open the Github Copilot Chat Window, you can ask copliot in the "Ask Copilot" input box.
31+
32+
1. Create a file for your system prompt.
33+
34+
Create file `src/main/resources/prompts/system-message.st` with content
35+
36+
```text
37+
You are a chatbot good at telling jokes.
38+
```
39+
40+
1. Ask Copilot to create code for open AI integration.
41+
42+
Click the file `src/main/resources/AzureOpenAiChatClientIT.java`, and input the following prompt to "Ask Copilot":
43+
44+
```text
45+
Refer to the sample file named "AzureOpenAIChatClientIT.java", add a new ChatController with POST endpoint at '/chatclient' to the chat-service project:
46+
* Use ChatClient to do the chat completion with Azure OpenAI Endpoint
47+
* Use User Prompt from file "resources/prompts/system-message.st", do not use method getPath()
48+
* The input of the endpoint '/chatclient' is a string from the request
49+
* The output of the endpoint '/chatclient' is a string returned by openAI
50+
* Use a configure file "ChatConfigure.java" file to init OpenAI ChatClient with Azure OpenAI Endpoint and API Key
51+
```
52+
53+
You will see output like:
54+
55+
![lab 5 copilot generate code 1](../../images/copilot-gen-code-1.png)
56+
57+
![lab 5 copilot generate code 2](../../images/copilot-gen-code-2.png)
58+
59+
1. Follow the steps to make changes to the project.
60+
61+
In the VSCode terminal (current directory `spring-petclinic-chat-service`), run maven command to build the project:
62+
63+
```bash
64+
mvn clean package
65+
```
66+
67+
Fix the erorrs if there are any.
68+
69+
1. Verify the chat endpoint.
70+
71+
- In the VSCode terminal, set the endpoint and api-key of your open ai instance, and run the app:
72+
73+
```bash
74+
export AZURE_OPENAI_API_KEY="<OPENAI-API-KEY>"
75+
export AZURE_OPENAI_ENDPOINT="<OPENAI-ENDPOINT>"
76+
77+
mvn spring-boot:run
78+
```
79+
80+
Ignore the warnings on `c.n.discovery.InstanceInfoReplicator`.
81+
82+
- In your commandline environment, verify the '/chatclient' endpoint with simple command:
83+
84+
```bash
85+
curl -XPOST http://localhost:8080/chatclient -d 'Hi, tell a joke'
86+
```
87+
88+
You will get a joke generated by OpenAI. Congratulations!
89+
90+
1. Deploy the `chat-service` to your Container Apps Environment.
91+
92+
In your commandline environment (current directory `spring-petclinic-microservices`), run:
93+
94+
```bash
95+
APP_NAME=chat-service
96+
97+
AZURE_OPENAI_API_KEY="<AZURE-OPENAI-API-KEY>"
98+
AZURE_OPENAI_ENDPOINT="<AZURE-OPENAI-ENDPOINT>"
99+
100+
cp -f ../tools/Dockerfile ./spring-petclinic-$APP_NAME/Dockerfile
101+
az containerapp create \
102+
--name $APP_NAME \
103+
--resource-group $RESOURCE_GROUP \
104+
--environment $ACA_ENVIRONMENT \
105+
--source ./spring-petclinic-$APP_NAME \
106+
--registry-server $MYACR.azurecr.io \
107+
--registry-identity $APPS_IDENTITY_ID \
108+
--ingress external \
109+
--target-port 8080 \
110+
--min-replicas 1 \
111+
--max-replicas 1 \
112+
--env-vars AZURE_OPENAI_API_KEY="$AZURE_OPENAI_API_KEY" AZURE_OPENAI_ENDPOINT="$AZURE_OPENAI_ENDPOINT" \
113+
--bind $JAVA_EUREKA_COMP_NAME \
114+
--runtime java
115+
```
116+
117+
In your browser navigate to the [Azure portal](http://portal.azure.com) and find the container app `chat-service`. Check the details.
118+
119+
Verify chat-service in Azure Container Apps.
120+
121+
```bash
122+
CHAT_URL=$(az containerapp show \
123+
--resource-group $RESOURCE_GROUP \
124+
--name $APP_NAME \
125+
--query properties.configuration.ingress.fqdn \
126+
-o tsv)
127+
128+
curl -XPOST https://$CHAT_URL/chatclient -d 'Hi, tell a joke'
129+
```
130+
131+
1. Function Calling with SpringAI
132+
133+
In this section, we will implement a basic RAG (Retrieval-Augmented Generation) pattern using Spring AI. The Retrieval-Augmented Generation (RAG) pattern is an industry standard approach to building applications that use large language models to reason over specific or proprietary data that is not already known to the large language model. This is critical because Azure Open AI model that we integrated in the previous step don't know anything about the PetClinic application. Refer to [Spring AI FunctionCallback API](https://docs.spring.io/spring-ai/reference/1.0/api/function-callback.html) for more information.
134+
135+
{: .note }
136+
The Spring AI API is under development and the interface may change time to time. At this moment, we are using the API version `1.0.0-M4` in this sample.
137+
138+
In this sample, we will implement a FunctionCallback interface for AI to get the owners information from existing petclinic solution.
139+
140+
- Create a new `Owner.java` for owner details.
141+
142+
```java
143+
package org.springframework.samples.petclinic.chat_service;
144+
145+
import lombok.Data;
146+
147+
import java.io.Serializable;
148+
149+
@Data
150+
public class Owner implements Serializable {
151+
152+
private Integer id;
153+
154+
private String firstName;
155+
156+
private String lastName;
157+
158+
private String address;
159+
160+
private String city;
161+
162+
private String telephone;
163+
}
164+
```
165+
166+
- Create a new file `OwnerService.java` to retrieve owner info from `api-gateway`.
167+
168+
```java
169+
package org.springframework.samples.petclinic.chat_service;
170+
171+
import java.util.List;
172+
173+
import org.springframework.core.ParameterizedTypeReference;
174+
import org.springframework.http.HttpMethod;
175+
import org.springframework.stereotype.Service;
176+
import org.springframework.web.client.RestTemplate;
177+
178+
@Service
179+
public class OwnerService {
180+
181+
public List<Owner> getOwners() {
182+
183+
RestTemplate restTemplate = new RestTemplate();
184+
var responseEntity = restTemplate.exchange(
185+
"http://api-gateway/api/customer/owners",
186+
HttpMethod.GET,
187+
null,
188+
new ParameterizedTypeReference<List<Owner>>() {
189+
});
190+
191+
List<Owner> owners = responseEntity.getBody();
192+
return owners;
193+
}
194+
}
195+
```
196+
197+
- Add FunctionCallbacks to chat client.
198+
199+
First Add an attribute to ChatController"
200+
201+
```java
202+
@Autowired
203+
private OwnerService ownerService;
204+
```
205+
206+
Add Functions to chat client. The new code segment would like, see the `functions` part.
207+
208+
```java
209+
ChatResponse response = this.chatClient.prompt()
210+
.system(s -> s.text(systemText))
211+
.user(userPrompt)
212+
.functions(FunctionCallback.builder()
213+
.description("list all owners")
214+
.method("getOwners")
215+
.targetObject(ownerService)
216+
.build())
217+
.call()
218+
.chatResponse();
219+
```
220+
221+
Fix the `import` issues with the help of VSCode.
222+
223+
- Rebuild the project in the VSCode termimal window (current directory spring-petclinic-chat-service):
224+
225+
```bash
226+
mvn clean package -DskipTests
227+
```
228+
229+
- Update the chat-service in your commandline window (current directory spring-petclinic-microservices):
230+
231+
```bash
232+
az containerapp update --name $APP_NAME --resource-group $RESOURCE_GROUP \
233+
--source ./spring-petclinic-$APP_NAME
234+
```
235+
236+
Verify the AI integrated with spring petclinic output:
237+
238+
```bash
239+
curl -XPOST https://$CHAT_URL/chatclient -d 'list the names of the owners'
240+
241+
curl -XPOST https://$CHAT_URL/chatclient -d 'list the owners from Madison'
242+
```
243+
244+
You may create more owners in the api-gateway page and ask more questions on the owners. Note, here we only implemented the owner function.
245+
246+
1. Here we have finished the first function for AI component to communicate with the existing system. Future more:
247+
248+
- You may add more functions to empower the AI strenght. Just like the getOwners sample.
249+
- You may integration a chatbox in the api-gateway page. see [Chatbox in api-gateway](#optional-integrate-the-chat-client-into-api-gateway).
250+
251+
# (Optional) Integrate the chat client into api-gateway
252+
253+
1. Update service `api-gateway` to add a new chat window.
254+
255+
- Add new route entry for `chat-service`. Note this name will be used later. Open file `spring-petclinic-api-gateway/src/main/resources/application.yml` and append new entry like below:
256+
257+
```yml
258+
- id: chat-service
259+
uri: lb://chat-service
260+
predicates:
261+
- Path=/api/chat/**
262+
filters:
263+
- StripPrefix=2
264+
```
265+
266+
- Add chatbox for api gateway
267+
268+
```bash
269+
git apply -f ../tools/api-gateway-chatbox.patch
270+
```
271+
272+
- Rebuild the api-gateway project and update the container app.
273+
274+
```bash
275+
APP_NAME=api-gateway
276+
mvn clean package -DskipTests -pl spring-petclinic-$APP_NAME
277+
az containerapp update --name $APP_NAME --resource-group $RESOURCE_GROUP \
278+
--source ./spring-petclinic-$APP_NAME
279+
```
280+
281+
- Check the new chatbox in the petclinic page.
282+
283+
```bash
284+
api_gateway_FQDN=$(az containerapp show \
285+
--resource-group $RESOURCE_GROUP \
286+
--name $APP_NAME \
287+
--query properties.configuration.ingress.fqdn \
288+
-o tsv)
289+
290+
echo https://$api_gateway_FQDN
291+
```
292+
293+
Open the api-gateway url and there is a chatbox at the right bottom corner.
294+
295+
![lab 5 api-gateway new chat box](../../images/api-gateway-chatbox.png)
296+
297+
1. Verify the new RAG empowered AI bot about owners.
298+
299+
![lab 5 open-ai-rag-bot](../../images/open-ai-rag-bot.png)
300+
301+
# (Optional) Prepare the simple project
302+
303+
1. Create a project from [spring initializr](https://start.spring.io/):
304+
305+
```bash
306+
curl https://start.spring.io/starter.tgz \
307+
-d dependencies=web,cloud-eureka,actuator,lombok,spring-ai-azure-openai \
308+
-d name=chat-service -d type=maven-project \
309+
-d jvmVersion=17 -d language=java -d packaging=jar \
310+
-d groupId=org.springframework.samples.petclinic -d artifactId=chat-service \
311+
-d description="Spring Petclinic Chat Service" \
312+
| tar -xzvf - -C spring-petclinic-chat-service
313+
```
314+
315+
1. Update `application.properties` to `application.yml`
316+
317+
```yml
318+
spring:
319+
application:
320+
name: chat-service
321+
```
322+
323+
1. Version
324+
325+
Verify the version of spring-ai is `1.0.0-M4`
326+
327+
Download the template file from branch `1.0.0.4`
328+
329+
```bash
330+
IT_FILE="https://raw.githubusercontent.com/spring-projects/spring-ai/refs/heads/1.0.0-M4/models/spring-ai-azure-openai/src/test/java/org/springframework/ai/azure/openai/AzureOpenAiChatClientIT.java"
331+
wget $IT_FILE -P spring-petclinic-chat-service/src/main/resources/
332+
```
333+
334+
1. Enable Eureka Client in class `ChatServiceApplication`:
335+
336+
![lab 5 eureka client](../../images/open-ai-eureka-client.png)

0 commit comments

Comments
 (0)