Skip to content
@triton-inference-server

Triton Inference Server

README.md

NVIDIA Triton Inference Server Organization

NVIDIA Triton Inference Server provides a cloud and edge inferencing solution optimized for both CPUs and GPUs.

This top level GitHub organization host repositories for officially supported backends, including TensorRT, TensorFlow, PyTorch, Python, ONNX Runtime, and OpenVino. The organization also hosts several popular Triton tools, including:

  • Model Analyzer: A tool to analyze the runtime performance of a model and provide an optimized model configuration for Triton Inference Server.

  • Model Navigator: a tool that provides the ability to automate the process of moving a model from source to optimal format and configuration for deployment on Triton Inference Server.

Getting Started

To learn about NVIDIA Triton Inference Server, refer to the Triton developer page and read our Quickstart Guide. Official Triton Docker containers are available from NVIDIA NGC.

Product Documentation

User documentation on Triton features, APIs, and architecture is located in the server documents on GitHub. A table of contents for the user documentation is located in the server README file.

Release Notes, Support Matrix, and Licenses information are available in the NVIDIA Triton Inference Server Documentation.

Examples

Specific end-to-end examples for popular models, such as ResNet, BERT, and DLRM are located in the NVIDIA Deep Learning Examples page on GitHub. Additional generic examples can be found in the server documents.

FAQ

For technical questions about Triton Inference Server, please consult the Triton FAQ Guide. Information about future support & updates for Triton can be found in the Dynamo FAQ Guide.

Feedback

Share feedback or ask questions about NVIDIA Triton Inference Server by filing a GitHub issue.

Pinned Loading

  1. server server Public

    The Triton Inference Server provides an optimized cloud and edge inferencing solution.

    Python 10.5k 1.7k

  2. core core Public

    The core library and APIs implementing the Triton Inference Server.

    C++ 170 131

  3. backend backend Public

    Common source, scripts and utilities for creating Triton backends.

    C++ 369 103

  4. client client Public

    Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.

    Python 686 252

  5. model_analyzer model_analyzer Public

    Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

    Python 507 84

  6. model_navigator model_navigator Public

    Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.

    Python 220 28

Repositories

Showing 10 of 36 repositories
  • TensorRT-LLM Public Forked from NVIDIA/TensorRT-LLM

    TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

    triton-inference-server/TensorRT-LLM’s past year of commit activity
    Python 1 2,236 0 0 Updated Mar 25, 2026
  • server Public

    The Triton Inference Server provides an optimized cloud and edge inferencing solution.

    triton-inference-server/server’s past year of commit activity
    Python 10,467 BSD-3-Clause 1,737 778 (3 issues need help) 94 Updated Mar 25, 2026
  • dali_backend Public

    The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.

    triton-inference-server/dali_backend’s past year of commit activity
    C++ 141 MIT 35 23 (9 issues need help) 7 Updated Mar 24, 2026
  • common Public

    Common source, scripts and utilities shared across all Triton repositories.

    triton-inference-server/common’s past year of commit activity
    C++ 79 BSD-3-Clause 79 0 9 Updated Mar 21, 2026
  • client Public

    Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.

    triton-inference-server/client’s past year of commit activity
    Python 686 BSD-3-Clause 252 57 34 Updated Mar 20, 2026
  • python_backend Public

    Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.

    triton-inference-server/python_backend’s past year of commit activity
    C++ 673 BSD-3-Clause 194 0 19 Updated Mar 19, 2026
  • model_analyzer Public

    Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

    triton-inference-server/model_analyzer’s past year of commit activity
    Python 507 Apache-2.0 84 0 1 Updated Mar 18, 2026
  • onnxruntime_backend Public

    The Triton backend for the ONNX Runtime.

    triton-inference-server/onnxruntime_backend’s past year of commit activity
    C++ 172 BSD-3-Clause 78 74 5 Updated Mar 18, 2026
  • openvino_backend Public

    OpenVINO backend for Triton.

    triton-inference-server/openvino_backend’s past year of commit activity
    C++ 37 BSD-3-Clause 18 6 1 Updated Mar 18, 2026
  • core Public

    The core library and APIs implementing the Triton Inference Server.

    triton-inference-server/core’s past year of commit activity
    C++ 170 BSD-3-Clause 131 0 25 Updated Mar 18, 2026
View all repositories