diff --git a/README.md b/README.md index 918882d0..161ea215 100644 --- a/README.md +++ b/README.md @@ -102,20 +102,21 @@ Now run the server via the command-line ```bash python server.py ``` - -- [LitAPI](https://lightning.ai/docs/litserve/api-reference/litapi) gives full control to build scalable compound AI systems (1 or more models). -- [LitServer](https://lightning.ai/docs/litserve/api-reference/litserver) handles optimizations like batching, auto-GPU scaling, etc... ### Query the server -Use the auto-generated LitServe client: +Run this command on your CLI to query the server: ```bash -python client.py +curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" -d '{"input": 4.0}' ``` +### Summary +[LitAPI](https://lightning.ai/docs/litserve/api-reference/litapi) gives full control to build scalable compound AI systems (1 or more models). +[LitServer](https://lightning.ai/docs/litserve/api-reference/litserver) handles optimizations like batching, auto-GPU scaling, etc... +
- Write a custom client + Example of a custom Python client ```python import requests @@ -144,7 +145,7 @@ Use LitServe to deploy any model or AI service: (Gen AI, classical ML, embedding Featured examples
Toy model: Hello world LLMs: Llama 3 (8B), LLM Proxy server, Agent with tool use -RAG: RAG API +RAG: RAG API (LlamaIndex) NLP: Hugging face, BERT, Text embedding API Multimodal: OpenAI Clip, MiniCPM, Phi-3.5 Vision Instruct Audio: Whisper, AudioCraft, StableAudio, Noise cancellation (DeepFilterNet)