No NanoLLM containers function on the AGX Thor

[danmartinez78]

Search before asking

• I have searched the jetson-containers issues and found no similar feature requests.

jetson-containers Component

Docker

Bug

$ jetson-containers run $(autotag nano_llm) python3 -m nano_llm.chat --api=mlc --model Efficient-Large-Model/VILA1.5-3b --max-context-len 256 --max-new-tokens 32

Error:

Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
import(pkg_name)
File "/opt/NanoLLM/nano_llm/init.py", line 2, in
from .nano_llm import NanoLLM
File "/opt/NanoLLM/nano_llm/nano_llm.py", line 15, in
from .vision import CLIPVisionModel, TIMMVisionModel, MMProjector
File "/opt/NanoLLM/nano_llm/vision/init.py", line 3, in
from .clip import CLIPVisionModel, TIMMVisionModel
File "/opt/NanoLLM/nano_llm/vision/clip.py", line 3, in
from clip_trt import CLIPVisionModel, TIMMVisionModel
File "/opt/clip_trt/clip_trt/init.py", line 2, in
from .text import CLIPTextModel
File "/opt/clip_trt/clip_trt/text.py", line 10, in
import torch2trt
File "/usr/local/lib/python3.10/dist-packages/torch2trt/init.py", line 1, in
from .torch2trt import *
File "/usr/local/lib/python3.10/dist-packages/torch2trt/torch2trt.py", line 2, in
import tensorrt as trt
File "/usr/local/lib/python3.10/dist-packages/tensorrt/init.py", line 75, in
from .tensorrt import *
ImportError: libnvdla_compiler.so: cannot open shared object file: No such file or directory

Environment

Fresh install of Jetpack 7.0

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!