FRIGATE NVR with Object Detection Part 2 - Zones, Object Masks, Motion Masks.

in this part two of my frigate nbr series we're going to talk about masks and zones mask and zones are used to help you tweak your settings and make the nvr even more useful so let's get started [Music] so let's first start talking about what masks and zones are and by the way if you haven't watched my previous part one video on how to set up frigate initially you should go through and watch that because we talked about in that video uh the frigate.yml file and how you set up a camera and get everything running you're going to want to have that running before you start playing with masks and zones or you can do it in addition to doing the masks and zones you can set it all up at once if you want to so watch that video i'll have it linked down below and hopefully up on a card here so what are masks and zones masks are used to ignore initial detection in areas of your camera's field and view so if you have a big camera area and you only want to look at certain areas in the camera view you use mask to tell frigate to completely ignore the areas that are masked out so you have motion masks those motion masks are used to prevent unwanted types of motion from triggering detection and you can tell where you want to apply those by watching your video feed with the motion boxes enabled to see what it keeps seeing in the motion areas and then object filter masks object filter masks are used to filter out false positives for a given object type these should be used to filter any areas where it is not possible for an object of that type to be and of course in both of these the bottom center of the detected object's bounding box and the bounding box is the box that's created when it sees a an object that is trying to detect that's where the the bounding box is evaluated against the mask if it's in that mast area it is assumed to be a false positive an example here is uh you may want to mark out rooftop or mascot rooftops walls the sky treetops for people because you're not going to typically have a person in the sky or on a rooftop or on a wall or in this you know areas like that so you can set up all kinds of masking to prevent false positives from happening get pretty detailed about this let's talk about zones for a minute zones allow you to define a specific area of the frame and apply additional filters for object types so you can determine whether or not an object is within that particular area zones cannot have the same name as a camera but a single zone can include multiple cameras if you have multiple cameras covering the same area by configuring zones with the same name for each camera if you want to test draw zones should be set in the config to draw the zone on the frame so you can adjust as needed the zone line actually gets a little bit thicker whenever any object enters the zone so let's talk about how we create these masks so the first thing i want to do is show you where i have masks already set up to give you an idea what i'm talking about if we look at one of my cameras here but you can see areas where you might have a potential where you don't want to have the camera looking for example i know that this area right here is going to when the wind blows there's going to be a lot of movement in this plant and also in this plant here and up here this is kind of a green belt and there's a lot of tree activity and it's hard to see in this video but the camera sees it and the camera will detect movement here these are areas that i'm not that concerned about and one of the reasons you want to set up masking potentially is to prevent the cpu from having to work harder looking at these areas where this constant motion from wind or something else and just mask those out so it doesn't have to consider that as part of its object detection so if i turn on and you can see down here we have a bunch of menu items bounding boxes motion boxes timestamp region zones and masks under masks if i turn on a mask you will see now that this area has been masked out this area has been masked out and this area has been masked out these are areas within frigate that will be completely ignored by the object detection system or algorithm and that makes your cpu work less hard or not work as hard because it doesn't care what's going on in these areas right here so how do we how do we create these so let me just go ahead and turn these off for a moment and there's a little section right here called mask and zone creator and i'm going to click on that and it's going to open up a really neat little thing here where we can actually create masks and zones and by the way the other mask here is one that i've got set up for the the time stamp if your camera provides a timestamp on its video feed mask that out because you know especially if you have seconds sticking along here that is continuously going and so for every second it's going to have the frigate nvr look for motion and try to determine if that's an object or not and that becomes a big load on your cpu especially when you have multiple cameras all of them running this time stamp stuff going on so how do you create motion masks well you can see here that i have all these masks set up and you can also see here if you click on any of these polygon shapes it's going to show you where the mask is now it doesn't actually create the mask in the configuration for you what it does in this tool is it gives you the values that you're going to put into your configuration so let's create another mask let's just say that this area right here where all these plants are is not any concern to me and there could be motion here that i don't want it to see so i'm going to mask this completely out so i have four masks already what i want to do is i want to add an additional mask and it's going to create a blank line here and all i'm going to do is i'm going to start clicking around this area and you kind of need to go in some sort of logical order when you start adding these points everywhere where you think you might need an adjustment you need to add a point and as you add these points it'll start drawing out the polygon now you can see here as i draw that all these points i have are pretty much in line so it's a very thin line but as i add these points that come across here you'll notice here that i start adding more and more to this polygon now see what happened here i don't like it doing that so i'm going to put that back over here and then maybe drag this around you can also right click on a point and remove it so i right click there and it removes the point and if you or you put them in the wrong order or you do them out of order it's going to kind of drag this around so it's best to kind of create these in order so that then you can come around here and you drag them around just kind of put them down wherever you think is the best area for that and then you can fine tune it you can see here that i'm now just dragging this across here and creating this masking zone so it will not look in this area all right so we've created this zone and now we have this additional zone or polygon set of coordinates listed in our configuration now like i said before it's not going to automatically add this to our configuration all this is a tool to help you build out these objects or build out these polygons so we have to do is you have an option here to copy this so if i click on copy now it's now copied this entire block of code that we're going to put into our configuration all right now going back to our frigate.yml file we've already copied this block of code here we're going to put that in our frigate.yml file and again i'm using visual studio code connected to my configuration directory on my home assistant instance with us sandbush here so whatever tool you use to edit your files this is a this is where you'll do it now you'll see here that i already have motion masks created for the camera and so what i'm going to do now is you can do a couple things here you already have all of these right here defined if you were adding yet another motion mask you could just take this right here and you can just copy this it's just text there's nothing special about this other than getting the formatting correct so if i come over here and i just add a new line here and i paste that in using ctrl v you'll see now that i'm i have a new mask presented to my or added into my configuration file and so i want to save that and to have a mask actually get applied you have to restart frigate so we'll go back over to supervisor now in fact before we do that let's just go ahead and let's just move around and look at different things here let's go back to that back porch and let's pull up this options and look at our mask and zone creator see how it's not there anymore these are actually pulled when you go on to configure motion masks and zones these are pulled from your configuration file because i haven't restarted frigate yet even though i just created this file or created this zone down here or mask it's not going to show up in here until i restart home assistant as long as i put the coordinates in the frigate.yml file and restart it so let's do that now let's go ahead and restart the frigate nvr so i'll click on restart and make sure you're always watching your log files and now it's restarted it started capturing so if we go back into frigate itself and look at the web ui and we go to that camera and we look at the options and the mask and zone creator you now see that we have this zone created and added in here so it will pick up whatever we have already added so that's how you do the motion masks the same thing applies to object masks you can add uh new zones to the to the different objects that you're tracking so if you're tracking in my case cats dogs and persons people you can add these same polygons to these object masks and we'll do the same thing here let's say if i click on add here there's never going to be a person up here in the tree at least it'll never show up if they are going to be a cat burglar to get up there so now i've created this zone here and i want and see where it is i want to add it to i just add it to the bottom of the object so basically it's going to filter all objects up here if i want to add to person then i need to go ahead and add it to the person so i clicked on add to person and i'm going to do the same thing again and it's added that mask to person rather than just generic so you have object zero here so i'm going to remove that one so make sure you click on whichever one you want to add it to so if you think cats shouldn't be up here then you or you'll never see a cat you can filter out that object detection area for cats and again you have to add this whole block of object filtering to your configuration or your frigate.yml file for it to show up so again anything you create in this tool is just a tool it's text that needs to be put into your frigate.yml file for it to work correctly all right so that's object filtering so now what do you want to do if you want to just be interested in specific things for specific areas those are the zones zones are created the same way as you create the objects so right now i don't have any zones so i only want to know when somebody is up here in this deck area so the same exact process right we go to mask and zone creator and then we come down here and we're going to add a zone so the first thing i just click on add and it creates a blank area for me to start working with so again make sure when you're starting these points that you think about logically what area you're going to do and start adding points to that area so i'm going to want to know if somebody walks in this entire area i'm going to throw a few extra points that allows me to kind of adjust this come across here all the way down in fact i'm going to go up here a little ways see how i flipped it let me get rid of that one and that one and then i'll just add one here come down here here over here and then a couple over here for me to just kind of be able to play with it oops so right click on this one if you can see it oh hard for me to see here nope this is what happens when you get kind of messed up on it if you mess it all up you can just start over again you can remove it and start over so i'm going to just re-add it again and put the zone back and make sure i get it in the right order so you always have the ability to start over if you need to i'm just adding a bunch of points so it gives me areas to fine-tune if i need to you know be careful that you don't get in the middle of this red spot if you add another area over here see and i did it again if you do that without moving your mouse right click and it'll just pull it right off see and i need to i need a spot over here there we go all right so now i can start dragging this around and if this is going to create me a zone i'm going to be interested in anything that occurs in this zone i'm going to give it extra special attention so there is a zone and this is a different looking thing so let me copy this now i'm going to copy this whole block of stuff first we're going to copy it and then we're going to go over to our camera configuration our frigate.yml we're going to add a whole new section for zones we don't have any zones here yet so what we have to do is we have to come down here and we're just going to paste in our zone information now you have to make sure in in yaml that this lines up correctly so you have your motion block with your mask settings and all your polygons and then your zone so we need to back that up and then zone and if you if you're not sure look here and make sure it lines up this direction so we come two spaces in for zone zero and then our coordinates for zone zero or that now you need to give these zones a meaningful name i'm going to give this a zone a meaningful name it's going to be a deck area so my camera is called back porch and then my zone is called deck area you can't name your zone the same as you name the camera that isn't going to work so you need to make sure that your deck or your zones are called uh something different than what your cameras are called now here's an interesting thing to note as well about these let me just go ahead and save that if you have two cameras that cover the same area or overlap areas and i'll show you on my back porch camera you'll notice i have this area right here let's just see if it overlaps at least out here there's a little bit of overlap over here if you create a zone with the same name under this camera as this other camera if they have the same name under different cameras they will be included as part of a bigger zone and that's something you can do if you have the overlap alright so now we have those uh set up let's go ahead and restart uh forget once more and you'll be able to see the zone that we set up okay so it's restarted and started as capturing process so we should be able to go into our web ui once again look at that camera and if you have the options down here to show the zones as we do here you can now see that we have a zone here that encompasses the deck area so anything that goes on out here is not going to be counted as part of this zone so what do you do with this stuff well there's a number of things and this video is already getting long so what we're going to do in the third video now is we're going to talk about notifications but one of the things you can do under events and i'll show you this with a different camera on my driveway camera i have a zone set up in this area right here anything that enters this zone is going to do a couple things for me the first is it's going to show me in my events that if something occurred within this zone so then i can filter on those the second thing is is i'm going to get an alert in this zone for specific objects only if they enter the zone so let's look at the events right now let's go to driveway close end so that's that zone right there anything that occurs in that zone area is going to show up as this event for that camera and for that zone now you can have all cameras and if you have an overlapping camera or two overlapping cameras with the same zone if you choose the zone it's going to show any camera that's part of that zone in this case i only have one zone for that camera or that this zone only belongs to one camera this camera here and so in my events i can filter and see anything that occurred close in to where the areas i'm concerned about which would be that zone if i want to look at anything that happened outside of that zone i would just do all and then choose the camera that i'm interested in so we have all this stuff right here including the camera events that are no zone so it gives you a good way to filter the events that are occurring within that zone because you may want to you may want to just look at that all by itself if you have some stuff going on you may not you may want to record stuff that goes on farther out away from the zone but really want to focus on those zones so zones give you the ability to do that so we did set up that zone here and under options again if you want to look at your masks in your zones you just click and toggle these on or off so that's just a quick rundown on masks and zones and also the use of bounding boxes and motion boxes for detection now one of the things you can do when you're doing this by the way is you can leave a camera up indefinitely and watch what it's doing i'll tell you what it's doing on you're displaying it five frames per second which is just fine for motion detection remember to use a substream to save on your cpu usage or cpu cycles on your device and then when you're done with everything you turn off the bounding turn off the motion but for troubleshooting these are very important to use in my next video i'm going to talk about how i take these zones and apply notifications to those so that i can see that somebody's entered that zone it'll send alerts via your home assistant helper app on your android or your ios device or any other notification system that you have set up so let me know if you have any questions down in the comments make sure you work watch part one if you haven't already if you're not a channel member i would appreciate if you would uh just sign up for the channel you got a couple different options for that it helps the channel grow and helps me make these videos and if you like the video subscribe and hit me up on discord if you have any questions and we'll see on the next [Music] so [Music] you

Channel: mostlychris

Views: 2,614

Keywords: Smart Home, Home Assistant

Id: ApHwpfXvdOI

Length: 19min 27sec (1167 seconds)

Published: Thu Sep 09 2021