[Detector Support]: OpenVINO GPU: RuntimeError in embeddings_maintainer due to incompatible tensor shape for Jina v2 model #20279
Replies: 3 comments 14 replies
-
semantic_search:
enabled: True
model: "jinav2"
model_size: largeThis issue is related to the Jina V2 model configuration for semantic search. According to the documentation, the V2 model has significantly higher RAM and GPU requirements, leading to increased inference time and memory usage(1). The error indicates a tensor shape incompatibility between the model input and the provided tensor. The documentation specifically mentions that switching between V1 and V2 requires reindexing your embeddings, as the embeddings from V1 and V2 are incompatible(1). Additionally, CPU inference with the For the V2 model configuration, the documentation shows: semantic_search:
enabled: True
model: "jinav2"
model_size: largeThe documentation also notes that V2 offers only a 3% performance improvement over V1 in both text-image and text-text retrieval tasks, which is unlikely to yield noticeable real-world benefits(1). For most users, especially native English speakers, the V1 model remains the recommended choice(1). Given your Intel N97 CPU and the error you're experiencing, you might want to consider switching back to the V1 model: semantic_search:
enabled: True
model: "jinav1"
model_size: smallThe 📚 Sources: Have another question? Just tag @inkeep. |
Beta Was this translation helpful? Give feedback.
-
|
I noticed that the problems started recently after I added genai for generating snapshot descriptions. Before that, semantic search was working perfectly without any failures. |
Beta Was this translation helpful? Give feedback.
-
|
You may be running into resource issues. The JinaV2 model is much more resource intensive than JinaV1. Can you post a screenshot of your embeddings metrics page? |
Beta Was this translation helpful? Give feedback.
-
|
No. Your particular error may be something related to OpenVINO internals and the high inference time. We haven't seen any other reports of this. |
Beta Was this translation helpful? Give feedback.
-
|
is JinaV1 large not usable? |
Beta Was this translation helpful? Give feedback.
-
|
JinaV1 is English-only, but if you want the multi-language support in JinaV2, I'd highly recommend a much faster discrete GPU. Otherwise, JinaV1 |
Beta Was this translation helpful? Give feedback.
-
|
I decided to start indexing all the events and didn't get any crashes at that moment, maybe the problem isn't performance after all? |
Beta Was this translation helpful? Give feedback.
-
|
It could be related to timing and GPU load. Reindexing moves through each thumbnail and description sequentially in batches rather than firing off requests to the models randomly. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Describe the problem you are having
I am experiencing a periodic crash of the embeddings_maintainer thread in Frigate. The error occurs when the Jina v2 model attempts to process an image thumbnail to generate embeddings using the OpenVINO GPU backend.
The exact sequence of actions is unknown; the error occurs sporadically in the background process while handling finalized events.
The embeddings_maintainer thread crashes with an OpenVINO RuntimeError, halting the generation of embeddings for new events.
CPU: intel n97
MEMORY: 16GB DDR4
Version
0.16.1
Frigate config file
docker-compose file or Docker CLI command
Relevant Frigate log output
Install method
Docker Compose
Object Detector
Other
Screenshots of the Frigate UI's System metrics pages
Any other information that may be helpful
No response
Beta Was this translation helpful? Give feedback.
All reactions