BentoML

All

107 repositories

llm-inference-in-production
Public
Everything you need to know about LLM inference
llm llm-inference
TypeScript
•
MIT License
•8•154•3•0•Updated Jul 14, 2025Jul 14, 2025
BentoML
Public
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
python machine-learning deep-learning model-serving multimodal mlops ml-engineering ai-inference llm generative-ai
Python
•
Apache License 2.0
•855•7.9k•132•7•Updated Jul 10, 2025Jul 10, 2025
bentocloud-homepage-news
Public
2•3•0•0•Updated Jul 8, 2025Jul 8, 2025
BentoSpacy
Public
Python
•0•0•0•0•Updated Jul 7, 2025Jul 7, 2025
OpenLLM
Public
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
llama mistral fine-tuning mlops bentoml vicuna llm model-inference llmops llm-serving
Python
•
Apache License 2.0
•749•12k•3•3•Updated Jul 7, 2025Jul 7, 2025
BentoLMDeploy
Public
Self-host LLMs with LMDeploy and BentoML
Python
•2•21•1•0•Updated Jul 7, 2025Jul 7, 2025
BentoVLLM
Public
Self-host LLMs with vLLM and BentoML
Python
•
Apache License 2.0
•17•134•0•0•Updated Jul 1, 2025Jul 1, 2025
sentence-embedding-bento
Public archive
Sentence Embedding as a Service
Jupyter Notebook
•
Apache License 2.0
•3•15•0•1•Updated Jun 30, 2025Jun 30, 2025
BentoLlamaCpp
Public
BentoML + llama.cpp
Python
•0•0•0•0•Updated Jun 29, 2025Jun 29, 2025
byoc
Public
Shell
•1•1•0•2•Updated Jun 25, 2025Jun 25, 2025
helm-charts
Public
0•1•0•0•Updated Jun 24, 2025Jun 24, 2025
yatai-image-builder
Public
🐳 Build OCI images for Bentos in k8s
docker docker-image container container-image kubernetes k8s
Go
•12•19•6•12•Updated Jun 24, 2025Jun 24, 2025
comfy-pack
Public
A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.
ai-art comfyui genai comfyui-manager comfyui-workflow comfyui-nodes comfyui-api
Python
•
Apache License 2.0
•22•166•11•1•Updated Jun 23, 2025Jun 23, 2025
Pneumonia-Detection-Demo
Public
Pneumonia Detection - Healthcare Imaging Application built with BentoML and fine-tuned Vision Transformer (ViT) model
computer-vision healthcare transformer health-care-application healthcare-imaging
Python
•2•8•1•1•Updated Jun 17, 2025Jun 17, 2025
BentoGradio
Public
Python
•0•0•0•0•Updated Jun 13, 2025Jun 13, 2025
BentoTinyCudaNN
Public
Python
•1•0•0•0•Updated Jun 12, 2025Jun 12, 2025
BentoSGLang
Public
Python
•1•3•0•0•Updated Jun 12, 2025Jun 12, 2025
BentoResnetTensorFlow
Public
Python
•0•0•0•1•Updated May 30, 2025May 30, 2025
openllm-models
Public
Python
•16•49•0•0•Updated May 28, 2025May 28, 2025
yatai-common
Public
Go
•6•2•0•2•Updated May 22, 2025May 22, 2025
bentocloud-cicd-example
Public
Python
•2•0•0•0•Updated May 20, 2025May 20, 2025
BentoTriton
Public
Python
•
Apache License 2.0
•1•3•0•0•Updated May 15, 2025May 15, 2025
BentoTwilioConversationRelay
Public
Python
•2•9•1•1•Updated May 14, 2025May 14, 2025
BentoMLCLLM
Public
Python
•1•5•0•1•Updated Apr 29, 2025Apr 29, 2025
BentoDiffusion
Public
BentoDiffusion: A collection of diffusion models served with BentoML
kubernetes ai lora model-serving fine-tuning diffusion-models stable-diffusion
Python
•
Apache License 2.0
•28•371•9•1•Updated Apr 29, 2025Apr 29, 2025
BentoMLflow
Public
Python
•2•2•0•0•Updated Apr 28, 2025Apr 28, 2025
BentoXGBoost
Public
Python
•3•1•0•0•Updated Apr 28, 2025Apr 28, 2025
BentoYolo
Public
BentoML service of YOLO v8
Python
•
Apache License 2.0
•2•10•1•0•Updated Apr 23, 2025Apr 23, 2025
quickstart
Public
BentoML Quickstart Example
Python
•
Apache License 2.0
•5•8•0•0•Updated Apr 21, 2025Apr 21, 2025
BentoVoiceAgent
Public
Build Phone Calling Voice Agent fully powered by open source models.
voice-agent ai-phone-calling-guide ai-voice-assistant ai-phone
Python
•12•49•1•1•Updated Apr 18, 2025Apr 18, 2025