Repositories list

• server

  Public
  The Triton Inference Server provides an optimized cloud and edge inferencing solution.

  machine-learningclouddeep-learning+ 4gpuinferenceedgedatacenter

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •1.6k•9.5k•734•76•Updated Jul 30, 2025Jul 30, 2025

• vllm_backend

  Public
  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •31•280•0•9•Updated Jul 30, 2025Jul 30, 2025

• core

  Public
  The core library and APIs implementing the Triton Inference Server.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •111•145•0•17•Updated Jul 30, 2025Jul 30, 2025

• onnxruntime_backend

  Public
  The Triton backend for the ONNX Runtime.

  inferencebackendtriton-inference-server+ 1onnx-runtime

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •68•156•73•3•Updated Jul 30, 2025Jul 30, 2025

• perf_analyzer

  Public
  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •24•97•22•13•Updated Jul 29, 2025Jul 29, 2025

• common

  Public
  Common source, scripts and utilities shared across all Triton repositories.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •74•75•0•5•Updated Jul 28, 2025Jul 28, 2025

• tensorrt_backend

  Public
  The Triton backend for TensorRT.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •33•77•0•1•Updated Jul 28, 2025Jul 28, 2025

• python_backend

  Public
  Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •175•629•0•15•Updated Jul 28, 2025Jul 28, 2025

• dali_backend

  Public
  The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.

  pythondeep-learninggpu+ 5image-processingdalidata-preprocessingnvidia-dalifast-data-pipeline

  C++

  •

  MIT License

  •33•136•23•7•Updated Jul 28, 2025Jul 28, 2025

• tensorrtllm_backend

  Public
  The Triton TensorRT-LLM Backend

  Shell

  •

  Apache License 2.0

  •126•870•311•23•Updated Jul 25, 2025Jul 25, 2025

• tutorials

  Public
  This repository contains tutorials and examples for Triton Inference Server

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •124•744•8•16•Updated Jul 21, 2025Jul 21, 2025

• triton_cli

  Public
  Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inference Server.

  Python

  •5•64•3•3•Updated Jul 21, 2025Jul 21, 2025

• third_party

  Public
  Third-party source packages that are modified for use in Triton.

  C

  •

  BSD 3-Clause "New" or "Revised" License

  •61•7•0•4•Updated Jul 21, 2025Jul 21, 2025

• square_backend

  Public
  Simple Triton backend used for testing.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •5•3•0•0•Updated Jul 21, 2025Jul 21, 2025

• repeat_backend

  Public
  An example Triton backend that demonstrates sending zero, one, or multiple responses for each request.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •7•6•0•0•Updated Jul 21, 2025Jul 21, 2025

• redis_cache

  Public
  TRITONCACHE implementation of a Redis cache

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •4•14•3•0•Updated Jul 21, 2025Jul 21, 2025

• pytorch_backend

  Public
  The Triton backend for the PyTorch TorchScript models.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •55•157•0•5•Updated Jul 21, 2025Jul 21, 2025

• openvino_backend

  Public
  OpenVINO backend for Triton.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •17•32•6•2•Updated Jul 21, 2025Jul 21, 2025

• model_analyzer

  Public
  Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

  deep-learninggpuinference+ 1performance-analysis

  Python

  •

  Apache License 2.0

  •78•482•29•6•Updated Jul 21, 2025Jul 21, 2025

• local_cache

  Public
  Implementation of a local in-memory cache for Triton Inference Server's TRITONCACHE API

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •1•5•1•0•Updated Jul 21, 2025Jul 21, 2025

• identity_backend

  Public
  Example Triton backend that demonstrates most of the Triton Backend API.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •13•7•0•0•Updated Jul 21, 2025Jul 21, 2025

• developer_tools

  Public
  C++

  •9•21•0•4•Updated Jul 21, 2025Jul 21, 2025

• client

  Public
  Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •244•636•49•26•Updated Jul 21, 2025Jul 21, 2025

• checksum_repository_agent

  Public
  The Triton repository agent that verifies model checksums.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •7•11•0•0•Updated Jul 21, 2025Jul 21, 2025

• backend

  Public
  Common source, scripts and utilities for creating Triton backends.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •100•336•0•3•Updated Jul 21, 2025Jul 21, 2025

• fil_backend

  Public
  FIL backend for the Triton Inference Server

  Jupyter Notebook

  •

  Apache License 2.0

  •37•81•52•0•Updated Jul 18, 2025Jul 18, 2025

• pytriton

  Public
  PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.

  gpudeep-learninginference

  Python

  •

  Apache License 2.0

  •55•813•9•0•Updated Jul 2, 2025Jul 2, 2025

• tensorflow_backend

  Public
  The Triton backend for TensorFlow.

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •21•52•0•2•Updated Jun 18, 2025Jun 18, 2025

• model_navigator

  Public
  Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.

  deep-learninggpuinference

  Python

  •

  Apache License 2.0

  •27•210•4•0•Updated Apr 22, 2025Apr 22, 2025

• .github

  Public
  Community health files for NVIDIA Triton

  2•2•0•0•Updated Mar 27, 2025Mar 27, 2025