Automations with Frigate AI person detection

hey what's up this is smart slav you've probably noticed that i recently made a few videos about security cameras that were a bit different than the other videos i make usually i try to automate things make something smart but these videos they were a bit different they were more like a product reviews as i was learning and experimenting with it but today i will bring that all together and we will finally make some home assistant automations with them so first i will show you my ultimate camera setup which of the cameras that i tested i picked up to use around my house and why then how i configured ai person and car detection using frigate and with that we will use this to automate things like intelligently turning on the lights triggering recording or sending alert when nobody is home and the alarm is set and something suspicious is detected around the house and we will even save it to an off-site ftp server but that's not all today for the first time i make a giveaway on my channel i already mentioned that from all the cameras i picked one as my personal favorite the camera is anker nc400 with the chroma night vision and i will go more into a detail about why i chose this one later in the video because i made a review but i forgot few things i will make this right and i have one brand new anker nc 400 here in the box that i will send to one of you more on how to apply and win later in the video so get ready and let's get started [Music] [Applause] first what's the deal with the ai person detection most cameras have a motion detection built in so why i can't just use that right well i did cover that in detail in my previous video about the reeling rlc 820a and if you didn't see that here is the link but in a nutshell if you do not have a clean protected area where any change in pixels means that there is someone in there this won't work in my case i have a large tree taking a big part of the image that is constantly moving in the wind yes the cameras normally allow to mask areas that you don't want to detect but the tree casts shadows pretty much across the entire image in different times of the day and the shadows also move but also if i mask where the tree is it will not detect a person walking in front of the tree so after a lot of experimenting i figured that if i want to have a useful person and car detection i really to need to use an object detection that can identify a person or car in my case next for the camera selection why a good quality image is important for me well if it needs to identify a person in the image it has to actually see it right and because a big part of what i want to use it for happens at night before turning the lights on or detecting suspicious people around the house and they usually come dark it will need to work well at night many cameras have an infrared light they're black and white but other than that they will do okay job for the object detection but not in my case since again i have this large tree which is close to the camera and in case you have something close to the camera and other things that are further and you shine light on it well you have this law of inverse squares and will turn the tree pretty much over exposed and white and the other things behind it will be completely black if you are a photographer you know what i'm talking about so i figured at least for my front camera infrared light is completely useless well i figured i tested that few times and really i ended up in turning the infrared light off so then anka sent me this nc400 i made review a few weeks ago if you didn't see it again here is the link and i really like that in fact i bought another couple and i use it as my main camera since then so to make it clear i bought those camera with my own money because of my own decision anker didn't pay me to say this they did send me another complimentary nc400 but i'm going to give it away to one of you by the way since the review i found that the integration with home assistant is actually easier than i showed you in the video i did configure it manually in yaml but it actually has a great on vif support and the integration is very simple so let me fix that the onvif integration is not enabled by default i guess this is the reason why i didn't find it in the last video so the first thing we need to do is to enable it open the camera from the web browser login and then go to network advanced settings and then in integration protocol enable onvif and add an username and password and select the level i choose operator once we are there you can also go to network basic settings and set the dynamic domain name and enable it it will make it easier to identify the device on your network then to edit the home assistant simply go to integrations select onvif and it will find the camera under the username and the password and we are done can't get easier than that it will create a couple of entities among them two for the video streams one for the high res stream we will use to be the interface for recording and then the lower sub stream we will use for the ai object detection so let's get it done first let's install frigate let's find a website and check the documentation so here we go this is frigate and i'm looking for the installation instructions i have almost is done with the supervisor so i'll need to add this link to the add-on store you will find a link in the description for the other installation types the instructions are there as well then back to the home assistant click on the ellipsis on the top right corner and choose repositories paste the url from the description and edit then click on the frigate nvr and install it it will take quite some time so i will fast forward once this is done we can turn on the watchdog and show it in the sidebar but we will not start it yet we need to configure it first the documentation says that this is not done in the configuration tab but you have to create a frigate.yaml file so let me go to the visual studio and walk you through that in the root of your config directory create a new file and call it frigate.yml and let's start configuring we will start with mqtt we will enter the host ip address with the address of your home assistant the user with the mqtt username and the password then let's configure the cameras we will start with the front camera so we'll call it front and we need to configure the codec we will have two inputs so for the first one we will configure the path with the uri of the rtsp stream of the anki camera you can find in documentation or copy it from here so this is the 102 substream so this is the low resolution stream and we will use it for detection and then the second path which is the high resolution path we will use to record the clips then we need to configure the width and the height of the image of the substream for the detection and the fps for detection as well this is recommended to 5 frames per second then this is not required but we will configure it to enable the clip recording when it starts and then for the detection we will need to say what to detect so we would like to track persons and cars and there is plenty of other objects available just check the documentation and finally we need to configure our detectors if you have a coral detector you will check the documentation but in my case i'm going to use the cpu and with that we are done so we can go back to the add-on and start it after a few seconds we can check the lock to see if there are any errors but if you followed my instructions it should be running fine and indeed everything is up and running so we can open the panel and we should see the camera image remember this is the low resolution substream and and here we will see the camera and the events when i click on the image i can show the options where i can turn on the bounding box which will tell me if it identifies the object so i can check whether it works correctly so let's try that let me go outside and see if it identifies me and here i'm entering the image and uh in a few seconds i should see a rectangle around me and here we go there is a rectangle 84 percent probability of a person it took about four seconds to identify me i think i can live with that next let's add a mask so it doesn't detect cars and people that are on the street so i'm going to hit the add button and then i'm going to be clicking on the image to make a selection so i'm going to start with the rough selection and then i can add additional points in the middle it will create a configuration of the mask so when i click on this copy button it will copy it to the clipboard and then i can open the frigate.yaml file and just paste it in there then we need to restart the frigate add-on to load a new configuration and we are done to trigger the automations we're gonna add a frigate integration that's gonna automatically create a bunch of sensors so i need to add this frigate repository into hacks so i'll head to the frigate github site there is installation instruction in case you want to do it manually but what i'm going to do is i'm going to just copy this url of this github site add this to my hacks say it is a integration edit then install and because it's integration i need to restart home assistant now i can go back to the integrations and add the frigate integration it will ask me for the host address i'm not sure if the default one will work but i'm going to enter ip address of my home assistant and here we go if i go to the device you can see that it creates a bunch of sensors for different object types for the car for the person so let's test that i'm going to open another instance of a home assistant and put it on the right side i'm going to open the frigate panel on the right and leave the device entities on the left and watch the front person motion now it's clear and as soon as i walk into the scene it will switch into detect it it's a binary sensor and i will use that to trigger the automation so it works quite well actually this time it looks like it detected me even earlier so the detection works that's great let's use it to automate things at the end this is the reason why i have home assistant so i'm still on the frigate integration page and i can create automation directly from here i'm going to just hit the plus and say create automation with this device when front person motion started detective motion so i'm gonna call it something turn front light on effective motion so here i already have the trigger configured i can do some conditions so i can do the sun before sunrise and after sunset or i actually like to do it differently i like to do the sun sun sensor state below horizon so the condition is when the sun is below horizon it means it's at night and would like to do is call service light turn on and then i'm going to pick the light and i'm going to pick the outside light and that's it i'm going to save that and then i'm going to create another automation to turn it off what i actually can do is i can duplicate it and i'm going to call it turn front light off detected motion and here i'm going to say when uh the front motion stopped detecting motion i can still leave the condition here or i can remove it doesn't matter and here i'm going to change the service the turn light off and i'm going to save that go back so that was pretty easy i created two automations one is to turn light on on detected motion at night and the second one to turn it off when it stops detecting motion let's test that so here we go it's 2120 the sunset was 1940 so it's one and a half hours after the sunset and my daughter is entering opening the gate and the light is coming up let's see it again it was actually pretty quick you can see as soon as she opens the gate and she puts her feet on the path the light comes on pretty much immediately so this is pretty good i think we can work with that so we have done our first automation triggered by frigate i have done a couple more so let me show you what i have around so first of all i have two cameras and forget one in the front one in the back putting that aside what else i'm doing with the front camera i covered two more scenarios both of them are about sending a notification and saving the video to a secure location now most of the security cameras do have a feature to send a notification on detected motion or to send the video file by email or save it to an ftp server and i can still configure it in the camera but i'd like to do better than that first i'd like to trigger this by this ai person detection so i don't want to trigger it by movement of the branch but only when there is a really a person on my property but i don't want us to trigger notification i only want to send it when we are not home and someone else is around the house and i don't want to do it based on a fixed time schedule that wouldn't work for us we can do better than that so i'd like to only set it when the alarm is set and we are not in the area and if that happens i will send a notification to my mobile phone saying someone is in front of a house with a picture from this front camera and also with a link to the media library where frigate saves the video clip from this event finally after a short delay i'm going to copy the video clip to an external ftp server this is quite an advanced feature we will use a linux shell command for that but don't worry i will show you how to do that it's just one line of configuration and it's a nice example of the home assistant's flexibility but before that let me show you the media library where frigate saves the clips if you remember we have configured the frigate to save clips and i can access them through here through the events where i have the list of events and if i click on that i could see the video in high quality that is going to show me uh what's going on but additionally i can go to the media browser and here there is a folder frigate where i have those clips where i can access them as well okay that's cool so we see those videos in the browser but how do i automate this how do i copy those files uh somewhere else watch me if i log in to the home assistant server and i go to the media frigate clips directory here are the files each of the files has a name which has the name of the camera and then some unique id so i can copy it from there but how do i know which of the files refers to what video watch me frigate uses mqtt for messaging so when a motion is detected frigate will fire an mqtt event with a payload let's look at the documentation the payload contains the name of the camera the label with the type of the movement in this case person and the id of the clip it's creating so what i can do is i can create a template sensor that is triggered by this mqtt event forget events and what it's going to do is it's going to update the state of the sensor and if remember the file name is composed from the name of the camera dash id and mp4 so this is what i'm doing here i'm gonna take the camera name dash id and mp4 so in other words i have created sensor that is going to contain the name of the last clip that has recorded so in this case it should be that the z830 mk at the end so let me check that the sensor is called frigate clip and this frigate clip contains as a state name of the camera and this id so we can check that and it should be exactly the same so now we know the file name so what we can do is we can create an automation that will take this file and ftp it to an offsite location and we are in home assistant it runs on linux so it's rather simple we can create a shell command to do that and a single line of text i'm calling a shell command c url and i'm uploading the file and i'm taking this media frigate clips and state of the sensor and i will ftp it to an off-site location including username and password so i have blurred it out now that we know how it's done let me show you one more automation i have created so the first one is when there is someone around the house and nobody is home an alarm is set i also have another one which is similar and it's also triggered by the detected motion but this one works from the midnight till 8 am so at night when we are home and in this case some zones of the alarm are armed and others are unarmed so we are upstairs and the ground floor is armed and i'm again sending notification to my phone and i'm sending this video over ftp so this is what you can do with cameras and frigate in-home assistant now the giveaway how to enter the anker nc400 giveaway it's rather easy all you have to do is to subscribe to my channel and comment this video with the hash giveaway in the comment and i will pick a random winner using the comment picker website on monday june 21st end of the day and i will announce it in the comment so good luck if you signed up but even if you are not going to win the amazon prime day is coming on june 21st and 22nd so watch out for the anke amazon prime day promotion if you are in the security camera shopping now is the time to do it now if you are watching this later and you have missed the amazon prime day you can use the link in the description of this video to get 25 percent off for all the anke knight chroma power over each knight and wireless cameras from the anke official website i've personally i have no profit when you use that link but uncle liked my videos and they did send this code to me for you and i i think it's a great deal before i wrap up there is one more thing one of the things you have to deal with with these cameras is how to mount them to the wall they have a plate with a hole in the middle for the cable and the plate has three or four holes so you can theoretically mount them directly on the wall but this is not ideal first this will leave the connectors exposed to the elements and this is not a good idea i mean the box includes some kind of protective sleeve but it looks really bad and i'm not sure how much waterproof this really is in long term in fact when i was testing it i left the the anka nc400 camera the original one without any protection overnight and it was raining and quite heavily and the water got into the ethernet connector and i was using the power over ethernet and the connector kind of exploded so don't do the same mistake and mount it properly please and the second thing is if you mount it vertically it limits the angle you can tilt the camera to the sides so it's better to mount it horizontally so what i have to do is i made a mount designed for the nc 400 that will nicely hold the camera and cover all the cables and connectors inside and it will mount on on the wall so if you have access to a 3d printer you can get it on the thing words the link is in the description a lot of links in the description today so that was it for today and i will catch you in the next one bye

Channel: Vaclav Chaloupka

Views: 6,036

Keywords: HomeAssistant, Frigate, Annke NC 400

Id: T0g3EnpYP-E

Length: 24min 30sec (1470 seconds)

Published: Tue Jun 15 2021