10.2 GA release update (#3998)

* 10.2 GA updates Signed-off-by: Yuan Yao (yuanyao) <[email protected]> * update changelog Signed-off-by: Yuan Yao (yuanyao) <[email protected]> * revert plugin README format changes Signed-off-by: Yuan Yao (yuanyao) <[email protected]> --------- Signed-off-by: Yuan Yao (yuanyao) <[email protected]>
NVIDIA · Jul 11, 2024 · 2332a71 · 2332a71
1 parent 9db1508
commit 2332a71
Show file tree

Hide file tree

Showing 99 changed files with 1,242 additions and 539 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,18 @@
 # TensorRT OSS Release Changelog
 
+## 10.2.0 GA - 2024-07-10
+
+Key Features and Updates:
+
+ - Demo changes
+   - Added [Stable Diffusion 3 demo](demo/Diffusion).
+ - Plugin changes
+   - Version 3 of the [InstanceNormalization plugin](plugin/instanceNormalizationPlugin/) (`InstanceNormalization_TRT`) has been added. This version is based on the `IPluginV3` interface and is used by the TensorRT ONNX parser when native `InstanceNormalization` is disabled.
+ - Tooling changes
+   - Pytorch Quantization development has transitioned to [TensorRT Model Optimizer](https://github.com/NVIDIA/TensorRT-Model-Optimizer). All developers are encouraged to use TensorRT Model Optimizer to benefit from the latest advancements on quantization and compression.
+ - Build containers
+   - Updated default cuda versions to `12.5.0`.
+
 ## 10.1.0 GA - 2024-06-17
 
 Key Features and Updates:

diff --git a/README.md b/README.md
@@ -26,13 +26,13 @@ You can skip the **Build** section to enjoy TensorRT with Python.
 To build the TensorRT-OSS components, you will first need the following software packages.
 
 **TensorRT GA build**
-* TensorRT v10.1.0.27
+* TensorRT v10.2.0.19
   * Available from direct download links listed below
 
 **System Packages**
 * [CUDA](https://developer.nvidia.com/cuda-toolkit)
   * Recommended versions:
-  * cuda-12.4.0 + cuDNN-8.9
+  * cuda-12.5.0 + cuDNN-8.9
   * cuda-11.8.0 + cuDNN-8.9
 * [GNU make](https://ftp.gnu.org/gnu/make/) >= v4.1
 * [cmake](https://github.com/Kitware/CMake/releases) >= v3.13
@@ -73,25 +73,25 @@ To build the TensorRT-OSS components, you will first need the following software
     If using the TensorRT OSS build container, TensorRT libraries are preinstalled under `/usr/lib/x86_64-linux-gnu` and you may skip this step.
 
     Else download and extract the TensorRT GA build from [NVIDIA Developer Zone](https://developer.nvidia.com) with the direct links below:
-      - [TensorRT 10.1.0.27 for CUDA 11.8, Linux x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-11.8.tar.gz)
-      - [TensorRT 10.1.0.27 for CUDA 12.4, Linux x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz)
-      - [TensorRT 10.1.0.27 for CUDA 11.8, Windows x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/zip/TensorRT-10.1.0.27.Windows.win10.cuda-11.8.zip)
-      - [TensorRT 10.1.0.27 for CUDA 12.4, Windows x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/zip/TensorRT-10.1.0.27.Windows.win10.cuda-12.4.zip)
+      - [TensorRT 10.2.0.19 for CUDA 11.8, Linux x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-11.8.tar.gz)
+      - [TensorRT 10.2.0.19 for CUDA 12.5, Linux x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz)
+      - [TensorRT 10.2.0.19 for CUDA 11.8, Windows x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/zip/TensorRT-10.2.0.19.Windows.win10.cuda-11.8.zip)
+      - [TensorRT 10.2.0.19 for CUDA 12.5, Windows x86_64](https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/zip/TensorRT-10.2.0.19.Windows.win10.cuda-12.5.zip)
 
 
-    **Example: Ubuntu 20.04 on x86-64 with cuda-12.4**
+    **Example: Ubuntu 20.04 on x86-64 with cuda-12.5**
 
     ```bash
     cd ~/Downloads
-    tar -xvzf TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz
-    export TRT_LIBPATH=`pwd`/TensorRT-10.1.0.27
+    tar -xvzf TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz
+    export TRT_LIBPATH=`pwd`/TensorRT-10.2.0.19
     ```
 
-    **Example: Windows on x86-64 with cuda-12.4**
+    **Example: Windows on x86-64 with cuda-12.5**
 
     ```powershell
-    Expand-Archive -Path TensorRT-10.1.0.27.Windows.win10.cuda-12.4.zip
-    $env:TRT_LIBPATH="$pwd\TensorRT-10.1.0.27\lib"
+    Expand-Archive -Path TensorRT-10.2.0.19.Windows.win10.cuda-12.5.zip
+    $env:TRT_LIBPATH="$pwd\TensorRT-10.2.0.19\lib"
     ```
 
 ## Setting Up The Build Environment
@@ -101,27 +101,27 @@ For Linux platforms, we recommend that you generate a docker container for build
 1. #### Generate the TensorRT-OSS build container.
     The TensorRT-OSS build container can be generated using the supplied Dockerfiles and build scripts. The build containers are configured for building TensorRT OSS out-of-the-box.
 
-    **Example: Ubuntu 20.04 on x86-64 with cuda-12.4 (default)**
+    **Example: Ubuntu 20.04 on x86-64 with cuda-12.5 (default)**
     ```bash
-    ./docker/build.sh --file docker/ubuntu-20.04.Dockerfile --tag tensorrt-ubuntu20.04-cuda12.4
+    ./docker/build.sh --file docker/ubuntu-20.04.Dockerfile --tag tensorrt-ubuntu20.04-cuda12.5
     ```
-    **Example: Rockylinux8 on x86-64 with cuda-12.4**
+    **Example: Rockylinux8 on x86-64 with cuda-12.5**
     ```bash
-    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda12.4
+    ./docker/build.sh --file docker/rockylinux8.Dockerfile --tag tensorrt-rockylinux8-cuda12.5
     ```
-    **Example: Ubuntu 22.04 cross-compile for Jetson (aarch64) with cuda-12.4 (JetPack SDK)**
+    **Example: Ubuntu 22.04 cross-compile for Jetson (aarch64) with cuda-12.5 (JetPack SDK)**
     ```bash
-    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda12.4
+    ./docker/build.sh --file docker/ubuntu-cross-aarch64.Dockerfile --tag tensorrt-jetpack-cuda12.5
     ```
-    **Example: Ubuntu 22.04 on aarch64 with cuda-12.4**
+    **Example: Ubuntu 22.04 on aarch64 with cuda-12.5**
     ```bash
-    ./docker/build.sh --file docker/ubuntu-22.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu22.04-cuda12.4
+    ./docker/build.sh --file docker/ubuntu-22.04-aarch64.Dockerfile --tag tensorrt-aarch64-ubuntu22.04-cuda12.5
     ```
 
 2. #### Launch the TensorRT-OSS build container.
     **Example: Ubuntu 20.04 build container**
 	```bash
-	./docker/launch.sh --tag tensorrt-ubuntu20.04-cuda12.4 --gpus all
+	./docker/launch.sh --tag tensorrt-ubuntu20.04-cuda12.5 --gpus all
 	```
 	> NOTE:
   <br> 1. Use the `--tag` corresponding to build container generated in Step 1.
@@ -132,38 +132,38 @@ For Linux platforms, we recommend that you generate a docker container for build
 ## Building TensorRT-OSS
 * Generate Makefiles and build.
 
-    **Example: Linux (x86-64) build with default cuda-12.4**
+    **Example: Linux (x86-64) build with default cuda-12.5**
 	```bash
 	cd $TRT_OSSPATH
 	mkdir -p build && cd build
 	cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
 	make -j$(nproc)
 	```
-    **Example: Linux (aarch64) build with default cuda-12.4**
+    **Example: Linux (aarch64) build with default cuda-12.5**
 	```bash
 	cd $TRT_OSSPATH
 	mkdir -p build && cd build
 	cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64-native.toolchain
 	make -j$(nproc)
 	```
-    **Example: Native build on Jetson (aarch64) with cuda-12.4**
+    **Example: Native build on Jetson (aarch64) with cuda-12.5**
 	```bash
 	cd $TRT_OSSPATH
 	mkdir -p build && cd build
-	cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64 -DCUDA_VERSION=12.4
+	cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out -DTRT_PLATFORM_ID=aarch64 -DCUDA_VERSION=12.5
   CC=/usr/bin/gcc make -j$(nproc)
 	```
   > NOTE: C compiler must be explicitly specified via CC= for native aarch64 builds of protobuf.
 
-    **Example: Ubuntu 22.04 Cross-Compile for Jetson (aarch64) with cuda-12.4 (JetPack)**
+    **Example: Ubuntu 22.04 Cross-Compile for Jetson (aarch64) with cuda-12.5 (JetPack)**
 	```bash
 	cd $TRT_OSSPATH
 	mkdir -p build && cd build
-	cmake .. -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64.toolchain -DCUDA_VERSION=12.4 -DCUDNN_LIB=/pdk_files/cudnn/usr/lib/aarch64-linux-gnu/libcudnn.so -DCUBLAS_LIB=/usr/local/cuda-12.4/targets/aarch64-linux/lib/stubs/libcublas.so -DCUBLASLT_LIB=/usr/local/cuda-12.4/targets/aarch64-linux/lib/stubs/libcublasLt.so -DTRT_LIB_DIR=/pdk_files/tensorrt/lib
+	cmake .. -DCMAKE_TOOLCHAIN_FILE=$TRT_OSSPATH/cmake/toolchains/cmake_aarch64.toolchain -DCUDA_VERSION=12.5 -DCUDNN_LIB=/pdk_files/cudnn/usr/lib/aarch64-linux-gnu/libcudnn.so -DCUBLAS_LIB=/usr/local/cuda-12.5/targets/aarch64-linux/lib/stubs/libcublas.so -DCUBLASLT_LIB=/usr/local/cuda-12.5/targets/aarch64-linux/lib/stubs/libcublasLt.so -DTRT_LIB_DIR=/pdk_files/tensorrt/lib
 	make -j$(nproc)
 	```
 
-    **Example: Native builds on Windows (x86) with cuda-12.4**
+    **Example: Native builds on Windows (x86) with cuda-12.5**
 	```powershell
 	cd $TRT_OSSPATH
 	mkdir -p build

diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-10.1.0.27
+10.2.0.19
diff --git a/demo/BERT/README.md b/demo/BERT/README.md
@@ -30,7 +30,8 @@ This subfolder of the BERT TensorFlow repository, tested and maintained by NVIDI
        * [TensorRT inference benchmark](#tensorrt-inference-benchmark)
   * [Results](#results)
     * [Inference performance: NVIDIA A100](#inference-performance-nvidia-a100-40gb)
-    * [Inference performance: NVIDIA A30](#inference-performance-nvidia-a30)
+    * [Inference performance: NVIDIA L4](#inference-performance-nvidia-l4)
+    * [Inference performance: NVIDIA L40S](#inference-performance-nvidia-l40s)
 
 
 ## Model overview
@@ -74,8 +75,8 @@ The following software version configuration has been tested:
 |Software|Version|
 |--------|-------|
 |Python|>=3.8|
-|TensorRT|10.1.0.27|
-|CUDA|12.4|
+|TensorRT|10.2.0.19|
+|CUDA|12.5|
 
 ## Setup
 

diff --git a/demo/DeBERTa/README.md b/demo/DeBERTa/README.md
@@ -75,7 +75,7 @@ Note that the performance gap between BERT's self-attention and DeBERTa's disent
 ## Environment Setup
 It is recommended to use docker for reproducing the following steps. Follow the setup steps in TensorRT OSS [README](https://github.com/NVIDIA/TensorRT#setting-up-the-build-environment) to build and launch the container and build OSS:
 
-**Example: Ubuntu 20.04 on x86-64 with cuda-12.4 (default)**
+**Example: Ubuntu 20.04 on x86-64 with cuda-12.5 (default)**
 ```bash
 # Download this TensorRT OSS repo
 git clone -b main https://github.com/nvidia/TensorRT TensorRT
@@ -84,10 +84,10 @@ git submodule update --init --recursive
 
 ## at root of TensorRT OSS
 # build container
-./docker/build.sh --file docker/ubuntu-20.04.Dockerfile --tag tensorrt-ubuntu20.04-cuda12.4
+./docker/build.sh --file docker/ubuntu-20.04.Dockerfile --tag tensorrt-ubuntu20.04-cuda12.5
 
 # launch container
-./docker/launch.sh --tag tensorrt-ubuntu20.04-cuda12.4 --gpus all
+./docker/launch.sh --tag tensorrt-ubuntu20.04-cuda12.5 --gpus all
 
 ## now inside container
 # build OSS (only required for pre-8.4.3 TensorRT versions)

diff --git a/demo/Diffusion/README.md b/demo/Diffusion/README.md
@@ -7,7 +7,7 @@ This demo application ("demoDiffusion") showcases the acceleration of Stable Dif
 ### Clone the TensorRT OSS repository
 
 ```bash
-git clone [email protected]:NVIDIA/TensorRT.git -b release/10.1 --single-branch
+git clone [email protected]:NVIDIA/TensorRT.git -b release/10.2 --single-branch
 cd TensorRT
 ```
 
@@ -48,7 +48,7 @@ onnx                1.15.0
 onnx-graphsurgeon   0.5.2
 onnxruntime         1.16.3
 polygraphy          0.49.9
-tensorrt            10.1.0.27
+tensorrt            10.2.0.19
 tokenizers          0.13.3
 torch               2.2.0
 transformers        4.33.1

diff --git a/docker/rockylinux8.Dockerfile b/docker/rockylinux8.Dockerfile
@@ -15,7 +15,7 @@
 # limitations under the License.
 #
 
-ARG CUDA_VERSION=12.4.0
+ARG CUDA_VERSION=12.5.0
 
 FROM nvidia/cuda:${CUDA_VERSION}-devel-rockylinux8
 LABEL maintainer="NVIDIA CORPORATION"
@@ -25,7 +25,7 @@ ENV NV_CUDNN_VERSION 8.9.6.50-1
 ENV NV_CUDNN_PACKAGE libcudnn8-${NV_CUDNN_VERSION}.cuda12.2
 ENV NV_CUDNN_PACKAGE_DEV libcudnn8-devel-${NV_CUDNN_VERSION}.cuda12.2
 
-ENV TRT_VERSION 10.1.0.27
+ENV TRT_VERSION 10.2.0.19
 SHELL ["/bin/bash", "-c"]
 
 RUN dnf install -y \
@@ -62,15 +62,15 @@ RUN dnf install -y python38 python38-devel &&\
 
 # Install TensorRT
 RUN if [ "${CUDA_VERSION:0:2}" = "11" ]; then \
-    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-11.8.tar.gz \
-        && tar -xf TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-11.8.tar.gz \
-        && cp -a TensorRT-10.1.0.27/lib/*.so* /usr/lib64 \
-        && pip install TensorRT-10.1.0.27/python/tensorrt-10.1.0-cp38-none-linux_x86_64.whl ;\
+    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-11.8.tar.gz \
+        && tar -xf TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-11.8.tar.gz \
+        && cp -a TensorRT-10.2.0.19/lib/*.so* /usr/lib64 \
+        && pip install TensorRT-10.2.0.19/python/tensorrt-10.2.0-cp38-none-linux_x86_64.whl ;\
 elif [ "${CUDA_VERSION:0:2}" = "12" ]; then \
-    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz \
-        && tar -xf TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz \
-        && cp -a TensorRT-10.1.0.27/lib/*.so* /usr/lib64 \
-        && pip install TensorRT-10.1.0.27/python/tensorrt-10.1.0-cp38-none-linux_x86_64.whl ;\
+    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz \
+        && tar -xf TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz \
+        && cp -a TensorRT-10.2.0.19/lib/*.so* /usr/lib64 \
+        && pip install TensorRT-10.2.0.19/python/tensorrt-10.2.0-cp38-none-linux_x86_64.whl ;\
 else \
     echo "Invalid CUDA_VERSION"; \
     exit 1; \

diff --git a/docker/rockylinux9.Dockerfile b/docker/rockylinux9.Dockerfile
@@ -15,7 +15,7 @@
 # limitations under the License.
 #
 
-ARG CUDA_VERSION=12.4.0
+ARG CUDA_VERSION=12.5.0
 
 FROM nvidia/cuda:${CUDA_VERSION}-devel-rockylinux9
 LABEL maintainer="NVIDIA CORPORATION"
@@ -25,7 +25,7 @@ ENV NV_CUDNN_VERSION 8.9.6.50-1
 ENV NV_CUDNN_PACKAGE libcudnn8-${NV_CUDNN_VERSION}.cuda12.2
 ENV NV_CUDNN_PACKAGE_DEV libcudnn8-devel-${NV_CUDNN_VERSION}.cuda12.2
 
-ENV TRT_VERSION 10.1.0.27
+ENV TRT_VERSION 10.2.0.19
 SHELL ["/bin/bash", "-c"]
 
 RUN dnf install -y \
@@ -67,15 +67,15 @@ RUN dnf -y install \
 
 # Install TensorRT
 RUN if [ "${CUDA_VERSION:0:2}" = "11" ]; then \
-    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-11.8.tar.gz \
-        && tar -xf TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-11.8.tar.gz \
-        && cp -a TensorRT-10.1.0.27/lib/*.so* /usr/lib64 \
-        && pip install TensorRT-10.1.0.27/python/tensorrt-10.1.0-cp39-none-linux_x86_64.whl ;\
+    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-11.8.tar.gz \
+        && tar -xf TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-11.8.tar.gz \
+        && cp -a TensorRT-10.2.0.19/lib/*.so* /usr/lib64 \
+        && pip install TensorRT-10.2.0.19/python/tensorrt-10.2.0-cp39-none-linux_x86_64.whl ;\
 elif [ "${CUDA_VERSION:0:2}" = "12" ]; then \
-    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz \
-        && tar -xf TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz \
-        && cp -a TensorRT-10.1.0.27/lib/*.so* /usr/lib64 \
-        && pip install TensorRT-10.1.0.27/python/tensorrt-10.1.0-cp39-none-linux_x86_64.whl ;\
+    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz \
+        && tar -xf TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz \
+        && cp -a TensorRT-10.2.0.19/lib/*.so* /usr/lib64 \
+        && pip install TensorRT-10.2.0.19/python/tensorrt-10.2.0-cp39-none-linux_x86_64.whl ;\
 else \
     echo "Invalid CUDA_VERSION"; \
     exit 1; \

diff --git a/docker/ubuntu-20.04.Dockerfile b/docker/ubuntu-20.04.Dockerfile
@@ -15,7 +15,7 @@
 # limitations under the License.
 #
 
-ARG CUDA_VERSION=12.4.0
+ARG CUDA_VERSION=12.5.0
 
 FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu20.04
 LABEL maintainer="NVIDIA CORPORATION"
@@ -28,7 +28,7 @@ ENV CUDA_VERSION_MAJOR_MINOR=12.2
 ENV NV_CUDNN_PACKAGE "libcudnn8=$NV_CUDNN_VERSION-1+cuda${CUDA_VERSION_MAJOR_MINOR}"
 ENV NV_CUDNN_PACKAGE_DEV "libcudnn8-dev=$NV_CUDNN_VERSION-1+cuda${CUDA_VERSION_MAJOR_MINOR}"
 
-ENV TRT_VERSION 10.1.0.27
+ENV TRT_VERSION 10.2.0.19
 SHELL ["/bin/bash", "-c"]
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -84,15 +84,15 @@ RUN apt-get install -y --no-install-recommends \
 
 # Install TensorRT
 RUN if [ "${CUDA_VERSION:0:2}" = "11" ]; then \
-    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-11.8.tar.gz \
-        && tar -xf TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-11.8.tar.gz \
-        && cp -a TensorRT-10.1.0.27/lib/*.so* /usr/lib/x86_64-linux-gnu \
-        && pip install TensorRT-10.1.0.27/python/tensorrt-10.1.0-cp38-none-linux_x86_64.whl ;\
+    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-11.8.tar.gz \
+        && tar -xf TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-11.8.tar.gz \
+        && cp -a TensorRT-10.2.0.19/lib/*.so* /usr/lib/x86_64-linux-gnu \
+        && pip install TensorRT-10.2.0.19/python/tensorrt-10.2.0-cp38-none-linux_x86_64.whl ;\
 elif [ "${CUDA_VERSION:0:2}" = "12" ]; then \
-    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.1.0/tars/TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz \
-        && tar -xf TensorRT-10.1.0.27.Linux.x86_64-gnu.cuda-12.4.tar.gz \
-        && cp -a TensorRT-10.1.0.27/lib/*.so* /usr/lib/x86_64-linux-gnu \
-        && pip install TensorRT-10.1.0.27/python/tensorrt-10.1.0-cp38-none-linux_x86_64.whl ;\
+    wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/10.2.0/tars/TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz \
+        && tar -xf TensorRT-10.2.0.19.Linux.x86_64-gnu.cuda-12.5.tar.gz \
+        && cp -a TensorRT-10.2.0.19/lib/*.so* /usr/lib/x86_64-linux-gnu \
+        && pip install TensorRT-10.2.0.19/python/tensorrt-10.2.0-cp38-none-linux_x86_64.whl ;\
 else \
     echo "Invalid CUDA_VERSION"; \
     exit 1; \

diff --git a/docker/ubuntu-22.04-aarch64.Dockerfile b/docker/ubuntu-22.04-aarch64.Dockerfile
@@ -15,12 +15,12 @@
 # limitations under the License.
 #
 
-ARG CUDA_VERSION=12.4.0
+ARG CUDA_VERSION=12.5.0
 
 # Multi-arch container support available in non-cudnn containers.
 FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu22.04
 
-ENV TRT_VERSION 10.1.0.27
+ENV TRT_VERSION 10.2.0.19
 SHELL ["/bin/bash", "-c"]
 
 # Setup user account
@@ -71,7 +71,7 @@ RUN apt-get install -y --no-install-recommends \
 # Install TensorRT. This will also pull in CUDNN
 RUN ver="${CUDA_VERSION%.*}" &&\
     if [ "${ver%.*}" = "12" ] ; then \
-        ver="12.4"; \
+        ver="12.5"; \
     fi &&\
     v="${TRT_VERSION}-1+cuda${ver}" &&\
     apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/sbsa/3bf863cc.pub &&\