@@ -102,20 +102,21 @@ Now run the server via the command-line
102
102
``` bash
103
103
python server.py
104
104
```
105
-
106
- - [ LitAPI] ( https://lightning.ai/docs/litserve/api-reference/litapi ) gives full control to build scalable compound AI systems (1 or more models).
107
- - [ LitServer] ( https://lightning.ai/docs/litserve/api-reference/litserver ) handles optimizations like batching, auto-GPU scaling, etc...
108
105
109
106
### Query the server
110
107
111
- Use the auto-generated LitServe client :
108
+ Run this command on your CLI to query the server :
112
109
113
110
``` bash
114
- python client.py
111
+ curl -X POST http://127.0.0.1:8000/predict -H " Content-Type: application/json " -d ' {"input": 4.0} '
115
112
```
116
113
114
+ ### Summary
115
+ [ LitAPI] ( https://lightning.ai/docs/litserve/api-reference/litapi ) gives full control to build scalable compound AI systems (1 or more models).
116
+ [ LitServer] ( https://lightning.ai/docs/litserve/api-reference/litserver ) handles optimizations like batching, auto-GPU scaling, etc...
117
+
117
118
<details >
118
- <summary >Write a custom client</summary >
119
+ <summary >Example of a custom Python client</summary >
119
120
120
121
``` python
121
122
import requests
@@ -144,7 +145,7 @@ Use LitServe to deploy any model or AI service: (Gen AI, classical ML, embedding
144
145
<strong >Featured examples</strong ><br >
145
146
<strong >Toy model:</strong > <a target =" _blank " href =" #define-a-server " >Hello world</a >
146
147
<strong >LLMs:</strong > <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-a-private-llama-3-8b-api " >Llama 3 (8B)</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/openai-fault-tolerant-proxy-server " >LLM Proxy server</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-ai-agent-with-tool-use " >Agent with tool use</a >
147
- <strong >RAG:</strong > <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-a-private-llama-3-1-rag-api " >RAG API</a >
148
+ <strong >RAG:</strong > <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-a-private-llama-3-1-rag-api " >RAG API (LlamaIndex) </a >
148
149
<strong >NLP:</strong > <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-any-hugging-face-model-instantly " >Hugging face</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-a-hugging-face-bert-model " >BERT</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-text-embedding-api-with-litserve " >Text embedding API</a >
149
150
<strong >Multimodal:</strong > <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-open-ai-clip-with-litserve " >OpenAI Clip</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-a-multi-modal-llm-with-minicpm " >MiniCPM</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-phi3-5-vision-api-with-litserve " >Phi-3.5 Vision Instruct</a >
150
151
<strong >Audio:</strong > <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-open-ai-s-whisper-model " >Whisper</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-an-music-generation-api-with-meta-s-audio-craft " >AudioCraft</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-an-audio-generation-api " >StableAudio</a >, <a target =" _blank " href =" https://lightning.ai/lightning-ai/studios/deploy-a-noise-cancellation-api-with-deepfilternet " >Noise cancellation (DeepFilterNet)</a >
0 commit comments