[Detector Support]: Convert onnx to rknn instructions fail due to missing torch package

[kcembrey]

Describe the problem you are having

Following the instructions here fails with the error "ModuleNotFoundError: No module named 'torch'".

Does it make sense to install the correct version of torch in the frigate rknn image, or should a separate image be created for converting onnx to rknn?

Version

0.16.0-c2f8de9

Frigate config file

Not Relevant

docker-compose file or Docker CLI command

sudo docker exec frigate python3 /opt/conv2rknn.py

Relevant Frigate log output

Traceback (most recent call last):
  File "/opt/conv2rknn.py", line 5, in <module>
    from rknn.api import RKNN
  File "/usr/local/lib/python3.11/dist-packages/rknn/api/__init__.py", line 1, in <module>
    from rknn.api.rknn import RKNN
  File "/usr/local/lib/python3.11/dist-packages/rknn/api/rknn.py", line 6, in <module>
    from .rknn_platform import get_host_os_platform, get_librknn_api_require_dll_dir
  File "rknn/api/rknn_platform.py", line 9, in init rknn.api.rknn_platform
  File "rknn/api/rknn_utils.py", line 8, in init rknn.api.rknn_utils
ModuleNotFoundError: No module named 'torch'

Install method

Docker Compose

Object Detector

RKNN

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

The error "ModuleNotFoundError: No module named 'torch'" occurs because the ONNX-to-RKNN conversion process requires PyTorch (torch), but the Frigate container does not include this package. The official documentation instructs you to use the conversion process inside the Frigate container but does not specify that PyTorch is included(1)(2).

It is recommended to perform the ONNX-to-RKNN conversion on a separate machine or container where you can install all required dependencies (including torch and rknn-toolkit2). The conversion process is not intended to be run inside the Frigate runtime container, which is optimized only for inference, not model conversion(1)(2).

After converting your ONNX model to RKNN format on a suitable environment, copy the resulting .rknn file to the proper folder on your Frigate host for use by the RKNN detector(1)(2).

📚 Sources:

1. Supported Hardware
2. Supported Hardware

---

Have another question? Just tag @inkeep.

[kcembrey]

Not bad, AI. This is definitely a workaround, and I've done that successfully. However, the docs do instruct you to run the conversion script using the main frigate container. I'm just asking if pytorch should actually be installed in the frigate-rk image, or if the instructions should be updated.

[NickM-27]

torch originally was included but caused conflicts leading to normal operation not working. For now you can just add pip3 install torch --break; before the existing command to have it work within the Frigate container.

In Frigate 0.17 the conversion will happen automatically, including installing torch, at Frigate runtime

[kcembrey]

Thanks @NickM-27, that works for me. I appreciate it.

That will be really cool to have it automatic at runtime.