Can I use my own fine-tuned pose detection model with Frigate+?

[jaykin-jmsc-aiml]

Hi,
I am interested in using a custom pose detection model that I have fine-tuned. I see that Frigate+ allows fine-tuning for object detection models, but I am not sure if pose detection models are supported for fine-tuning or use through Frigate+. Is it possible to use my own fine-tuned pose detection model with Frigate+ or the Frigate platform in general?
Thanks

[inkeep-ai-assistant[bot]]

There is no information in the available sources about using custom pose detection models or support for pose detection models in Frigate+ or the Frigate platform in general. The documentation only mentions fine-tuning Frigate+ object detection models with your own images and annotations, but does not mention support for pose detection models or uploading your own model files(1).

📚 Sources:

1. Frigate+

---

Have another question? Just tag @inkeep.

[NickM-27]

Frigate in general does not support pose detection

[jaykin-jmsc-aiml]

I’m looking to use pose detection in our art gallery to monitor customer behavior and make sure visitors aren’t interacting with or doing anything unusual with the paintings.

[NickM-27]

I see, I mean to do that would require more than just pose detection as you'd need some type of behavior analysis based on the poses and things.

[jaykin-jmsc-aiml]

Right, what I’m hoping for is that once our fine-tune pose model flags unusual activity (like someone reaching for a painting), Frigate could use its existing alert/notification system to immediately push me an alert. That way I’m aware in real time and can respond to that person. Basically, we’d plug our pose-detection output into Frigate’s event pipeline so it triggers the same alerts Frigate already sends for objects.

[NickM-27]

you could implement it as a custom real time model, but we would need work done to support it normally without manual code changes

[jaykin-jmsc-aiml]

Following up on our discussion, I've implemented the custom real-time pose detection model support. I've integrated the Ultralytics YOLO pose model into Frigate, which now allows for pose estimation.

My changes include:

• A new ultralytics_pose detector plugin.
• Updated configuration options to support the yolo-pose model type.
• Comprehensive updates to the Frigate documentation, including a new section for the Ultralytics Pose Detector and a guide on the fork and branch workflow for contributions.

I have prepared a Pull Request (PR) from my fork (jaykin-jmsc-aiml/frigate_pose) to demonstrate these changes and propose their inclusion into the main Frigate repository.

I believe this implementation aligns with the goal of leveraging pose detection output within Frigate's event pipeline for advanced alerting based on specific pose activity.