Frigate Tip: Motion Masks

[hawkeye217]

The Frigate Tips series is a collection of posts where the Frigate developers give tips on how to best configure, use, and maintain your Frigate installation. This post's tip aims to bring clarity on when to use motion masks, something we've discovered is often misunderstood by new users and users coming from other NVR software.

What is a motion mask?

Motion masks in Frigate are used to prevent areas of a scene on a camera from triggering motion detection. They are most easily configured from the Frigate Web UI. The Frigate documentation says:

You want to mask out your timestamp, the sky, rooftops, etc. Keep in mind that this mask only prevents motion from being detected and does not prevent objects from being detected if object detection was started due to motion in unmasked areas.

This is a really important point that we've found that many users misunderstand: a motion mask only prevents motion from being detected and does not prevent objects from being detected.

Motion masks don't prevent objects from being detected

Motion masks should not be used to block out areas where you aren't interested in detecting things.

Yes, you read that correctly (though there are some caveats). Motion masks should be used very sparingly. In order to understand why, it's important to understand how Frigate works.

Again, from the Frigate documentation:

Frigate uses motion detection as a first line check to see if there is anything happening in the frame worth checking with object detection.

Once motion is detected, it tries to group up nearby areas of motion together in hopes of identifying a rectangle in the image that will capture the area worth inspecting. These are the red "motion boxes" you see in the debug viewer.

For a simple example, take this scene of a street in Orlando, Florida. A car is driving along the road, and Frigate's Debug view shows red boxes to indicate that motion is being detected on the frame in the areas where the car is moving. These regions of motion are grouped together and passed onto Frigate's object detector, which identifies the object in the frame as a car. Frigate's Debug view then shows the label and colored bounding box around it.

Let's imagine a scenario with this scene where you're only interested in people that are walking on the sidewalk on the right side of the frame, and you only want those people to be shown in the Review pane in Frigate.

It's often assumed that this is where a motion mask would be the solution. A new user may be tempted to add a motion mask like this:

However, this is not the case, and this kind of a motion mask is actually detrimental to the way Frigate is designed to work.

Again, from the documentation:

With too much masking, tracking is hampered and if an object walks from an unmasked area into a fully masked area, they essentially disappear and will be picked up as a "new" object if they leave the masked area. This is important because Frigate uses the history of scores while tracking an object to determine if it is a false positive or not.

So take this hypothetical situation:

A person walks underneath the clock (which, if using the masking example above, is in a masked area) and proceeds down the sidewalk we're interested in (which is now in the unmasked area).

Frigate would detect motion just outside of the masked area and begin tracking the person as they proceed down the sidewalk.

Seems like this would work... But wait.

What if the person realized they had dropped something as they were walking underneath the clock? They would turn around and proceed back into the masked area. Perhaps they spend a couple of minutes there talking to someone else, and then continue back down the sidewalk of interest.

Because of overmasking, the person of interest would actually be tracked by Frigate as an entirely new person as they leave the masked area and proceed down the sidewalk the second time, potentially creating two separate review items. In a high-traffic area, this would certainly not be ideal.

So, what's the solution?

Use required_zones instead of motion masks

If you're only interested in keeping track of objects in a particular area of your scene, use required_zones.

From the documentation:

Zones allow you to define a specific area of the frame and apply additional filters for object types so you can determine whether or not an object is within a particular area. Presence in a zone is evaluated based on the bottom center of the bounding box for the object. It does not matter how much of the bounding box overlaps with the zone.

Using a zone for the sidewalk and then restricting alerts and/or detections to objects in that zone is the correct solution here. This can easily be done with the Frigate UI (0.14 and later).

Create a zone called "sidewalk" in Frigate's mask/zone editor and then restart Frigate:

After restarting, navigate to Settings → Camera Settings, choose the camera in the top right, and then check the boxes to restrict alerts and/or detections to the "sidewalk" zone you just created. Then save and restart Frigate once more.

With this configuration, Frigate will track objects outside of the sidewalk zone, but only the person objects appearing in the sidewalk zone will show as alerts in the UI.

When creating zones for your cameras, remember that an object's presence in a zone is evaluated based on the bottom center of the bounding box. The example in the documentation shows that more clearly.

Applicable situations for motion masks

Motion masks are used to prevent unwanted types of motion from triggering object detection. A great use for a motion mask would be to cover a camera timestamp, to cover the sky and any moving clouds, or to cover the top of a tree for moving branches and leaves.

Official Frigate documentation

Masks

Zones

Review - for configuring alerts and detections (Frigate 0.14 and later)

---

If this has been a helpful post, give it an upvote and thumbs up. If you have any questions, leave them down below for us!

-The Frigate dev team (Blake @blakeblackshear, Nick @NickM-27, Josh @hawkeye217)

[highpeakrider]

Sorry I'm only just getting my system setup, could you expand on the 'Limit detections to specific zones' element?

[highpeakrider]

Hi Josh
That's a difficult question, as a complete newbie , non developer who can follow code by example I struggled to understand how to fit it all together and where and how to enter the code. I did this on my Synology NAS but have a dedicated machine that I'm going to try and build using Debian today. On the internet I found lots of You Tube creators but much of the content relates to old versions now which can take you down dead ends.

MGTT was real fun getting it into HA, creating the docker but understanding the password creation was a stumble but I found this to create a password file. https://dmelo.eu/blog/mosquitto_passwd_gen/

I find something like the below really easy to follow, however I'm mindful this is a free opensource app and resources may be limited, and writing documentation was not my favourite thing when I was working, although I'd love to get involved if I understand the subject matter to document it.
easy example - https://mariushosting.com/how-to-install-home-assistant-on-your-synology-nas/

I was struggling with this as it was picking up the road traffic and next doors car on the drive on the right but I've now reduced the area to see if that works better.
Feel free to delete this ramble if it distracts from the tip above.

Cheers Phil

[hawkeye217]

What you've done to change the drive_zone is exactly right. Because presence in a zone is based on the bottom center of an object's bounding box, cars driving by on the road were likely entering the zone. You can watch the Debug view and enable bounding boxes and zones there to see what happens when cars drive by now.

[NickM-27]

Make sure the zone is set as required in the camera settings page

[highpeakrider]

Here?

[NickM-27]

have you limited alerts to the zone?

[Aquathing]

What about objects that are in between a motion mask?
I am using motion masks to exclude motion from my hedges, as they trigger object detection which drastically increases CPU usage (I do not have and cannot get a Coral yet).

If somebody is between the hedges (i.e only the feet are visible outside the motion mask from debug view), would that trigger object detection on the full area of the moving object or only the feet?

[highpeakrider]

Where are you based loads in the UK. https://www.mouser.co.uk/ProductDetail/Coral/G950-06809-01?qs=u16ybLDytRbcxxqFKdbhgQ%3D%3D

[Aquathing]

I'm in Italy. Yes, I found that website, but I cannot buy a Google Coral right now, I'm trying to decrease CPU usage as to consume fewer watts while maintaining security.

[NickM-27]

Once an object is detected motion masks won't affect detection as tracking information is used to know where to look

[Aquathing]

Once an object is detected motion masks won't affect detection as tracking information is used to know where to look

That's great to hear! Thanks.

[neuviemeporte]

I think the big red text warning that big motion masks are "not recommended" in the UI should be changed. I am aware of the difference between masks and zones, and I have a required zone on my backyard camera. But I also use a big motion mask to block off the sky and the brush above the lawn because it keeps moving and increases the load on my hardware. No objects that I'm interested in are going to fly into the sky and come back down. After an upgrade, the warning text gaslighted me into thinking there could be a performance penalty from using a big motion mask, but from what I understand the worst it could do is make Frigate generate more detection events? It's good to have tips, but IMO it should read something like "large motion masks could interfere with tracking objects which step into the motion mask", and potentially it should be hidden altogether if there is a required zone also configured. Otherwise, I think it's increasing the confusion rather than alleviating it.

[NickM-27]

Many users are not aware of this, and a warning is how they find out about it. Using large masks can affect performance in that object detection is fun more frequently along with worse tracking and potentially missing important things altogether

but from what I understand the worst it could do is make Frigate generate more detection events?

No, the worst it can do is cut someone's recording short leaving out details of where a person came from before and where they went after breaking in to the property. It is very much a problem and should be avoided.

The warning only shows when a massive area of the camera is covered with a mask which in most cases is wrong. In cases like you describe it does make more sense, but often it would seem like adjusting the camera angle may be better to avoid so much resolution pointed at areas that aren't interesting (as a general example, there can of course be reasons it's necessary).