Repositories list

• llm-d

  Public
  Achieve state of the art inference performance with modern accelerators on Kubernetes

  Makefile

  •

  Apache License 2.0

  •194•1.9k•46•42•Updated Oct 14, 2025Oct 14, 2025

• llm-d-benchmark

  Public
  llm-d benchmark scripts and tooling

  incubating

  Jupyter Notebook

  •

  Apache License 2.0

  •32•30•48•7•Updated Oct 13, 2025Oct 13, 2025

• llm-d.github.io

  Public
  Website for llm-d: This repository builds the website seen at llm-d.ai

  documentationopensourceai+ 1docusaurus

  JavaScript

  •19•11•3•1•Updated Oct 13, 2025Oct 13, 2025

• llm-d-inference-scheduler

  Public
  Inference scheduler for llm-d

  kubernetesainetworking+ 2inferencegateway-api

  Go

  •

  Apache License 2.0

  •81•99•44•13•Updated Oct 13, 2025Oct 13, 2025

• llm-d-kv-cache-manager

  Public
  Distributed KV cache coordinator

  aiincubating

  Go

  •

  Apache License 2.0

  •49•77•21•13•Updated Oct 9, 2025Oct 9, 2025

• llm-d-inference-sim

  Public
  A light weight vLLM simulator, for mocking out replicas.

  incubating

  Go

  •

  Apache License 2.0

  •33•50•10•4•Updated Sep 30, 2025Sep 30, 2025

• llm-d-routing-sidecar

  Public
  Incubating P/D sidecar for llm-d

  incubating

  Go

  •

  Apache License 2.0

  •24•16•7•5•Updated Sep 25, 2025Sep 25, 2025

• .github

  Public
  2•0•0•0•Updated Aug 28, 2025Aug 28, 2025

• llm-d-deployer

  Public
  Helm charts for llm-d

  helmhelm-chartllm-d

  Shell

  •

  Apache License 2.0

  •54•50•34•17•Updated Jul 22, 2025Jul 22, 2025

• llm-d-model-service

  Public archive
  Simplified model deployment on llm-d

  incubating

  Go

  •

  Apache License 2.0

  •14•27•29•6•Updated Jul 2, 2025Jul 2, 2025

• llm-d-pd-utils

  Public
  Makefile

  •4•5•1•0•Updated May 21, 2025May 21, 2025