This Just Made Face Recognition So Much Better!

a few months ago i covered the ultimate nvr install which covered combining the object detection and recording capabilities of frigate with the face recognition of deep stack to give you the best of both worlds and whilst both those systems are great to get it working required a little bit more manual intervention and they felt sort of disconnected from each other enter double take a unified interface to combine both the powers of object detection and face recognition in one easy to access location with the ability to train facial recognition images in seconds right from inside of home assistant firstly let's take a look at what double take is and some of the features and benefits that it gives you double take is an open source project created by david yakuwenko designed to be a unified interface for processing and training images for facial recognition it supports several detectors integrates with home assistant as well as mqtt and the cool bit is is that you can actually train and untrain faces right from the ui making it much easier for anyone to use than ever before that was one of the challenges with deep sac was it wasn't the easiest or most user-friendly thing to train faces on but double take helps us to easily solve that issue the other thing that wasn't the easiest to do was to run facial detection automatically after a person was detected so with a frigate and deep stack setup that we used to use we used to get frigate to detect a person and then when that happened we would use a home assistant automation to get deep stack to run facial detection on that image well double take is going to take care of that a whole entire process for us with a nice clean ui to manage the whole thing it's also worth mentioning that double take works with frigate as the nvr which handles the recording of footage snapshots and it's handling the object detection looking for a person inside of a camera image when a person is detected double take will pass that image into what's called a detector for facial recognition and as mentioned earlier double take supports three different detectors deep stack comproface and facebox we're going to be using deep stack for this guide since we've already covered that install process in the past but if you want to use those other detectors then the configuration should be pretty similar also if you want to follow along with this guide you're going to need to have frigate already installed and working which we have covered in much more detail in a previous video and i'll have that linked up here and you're also going to need to have deep stack up and running too again we have covered that in its own dedicated video previously which you can also find up here too the frigate configuration has changed ever so slightly since that first video was released so make sure to check the docs for the latest config all right so now that we have frigate and deep stack up and running let's get into the install and the nice thing is that literally just a week or two ago double take officially added the install option for running as a home assistant add-on making our lives much easier if you don't use home assistant add-ons then is a straightforward process if you want to run it as its own docker container and i'll leave links to how to do that in the description down below first head over to supervisor and then click on the add-on store and hit the three dots in the top right hand corner and click repositories paste in this link from the description which will add a new repository for the double take add-on then find the newly added double-take entry and hit the install button once installed you can take the watchdog auto update and show in sidebar options if you want to and then hit the start button since all of the config is done inside the double take ui open up the double take ui and you will land at the main dashboard and you'll see that it says no files found which is expected as we haven't yet told it where frigate is located up at the top head over to the conflict app and then we need to configure our mqtt frigate and deep stack details which are pretty straightforward first enter the mqtt broker details as follows making sure to include the ip address username and password for your mqtt broker this needs to be the same mqtt broker that your frigate server is connected to and this is how it communicates with frigate to start the facial recognition next enter the url of your frigate server if you're running frigate on home assistant then this will be the same ip or host name as your home assistant server with port 5000 at the end the last thing to add is our detector the detector is what will process our facial recognition from frigate once a person is found in this case we are using deep stack for this but if you want to use another detector either comproface or facebox then you will want to slightly tweak this config here for deep stack we need to enter our deep stack url and then you can specify an api key if you are using one here i'm not so i will leave this blank on the right hand side you can then hit the save config and restart button and once you do that double take will restart with the new config and then you'll notice at the top left hand corner that it will show you the connectivity status of mqtt frigate and deep stack if they show up red then you will need to double check your settings and logs and you can access the logs in the top right hand corner here and that is really it for the double take setup in terms of actual config if you did everything correctly you can now walk in front of your camera to trigger frigate to recognize your first match you'll then need to make sure that inside of your frigate config you have person detection enabled what i will do is leave a sample frigate config down in the description down below if you want to copy some of the basics that i have here once you walk in front of your camera and trigger frigate you should get an image like this appear in your matches from the get go we can see a few things here firstly it gives us our detector name as deep stack and also returns the name as unknown because we haven't yet trained any faces the percentage that is currently zero is the confidence level which will show up again after training underneath you have the camera that it came from as well as if it came from mqtt or from frigate snapshots you may get duplicate images here because frigate will send the image of the person over mqtt first when the initial detection is made and then when it saves a more accurate snapshot it will also send that image to double take for processing too you can disable this behavior if you want now we're going to start training some faces so that we can start matching some images head over to the train tab and then using the drop down at the top select add new and then enter the name of the person you want to match now we have a couple of ways that we can actually match faces here either we can upload our own images for training or we can use some of the images from the camera itself to train or we can do both i'll show you how to upload your images first hit the upload button and then select some images that upload and you can select multiple images at one time as soon as you upload these deepstat will start training a model based on your face how cool is that and now if we head back to matches what you can now do is reprocess the images that were previously unmatched to see if it now gets them by pressing the refresh button this will then ask a deep slack to recheck the images after we have now trained our face on it if we want to use the images that frigate has detected to train our deep stack model then we can simply select the images from the camera on the matches page and then from the drop down select the name of the person and hit the train button and once again deep stack will now start learning based on these faces and this is an incredibly useful and powerful feature for being able to learn new faces by using the actual camera that will be used for detection so that is quite straightforward to use you can train your images right from double take ui using images from your actual camera or you can train images by uploading your own and you can easily see an overview of all of the faces that were detected how confident double take was that it actually matched the face and you can easily reprocess images that were just trained really easy to use and works really well now let's take a look at creating a really simple automation to notify you about matched images first before we can create an automation let's take a look at some of the sensors that are now exposed to home assistant so that we can actually use them in our automation if we look inside of devtools we can now see that we have a few sensors listed for double take firstly we have the sensor for our back guard and camera itself which contains the last match for that particular camera and we also have sensors for the people we have matched through facial recognition as well as an unknown sensors for when someone can't be matched there is also a ton of extra information inside of the sensor attribute that you power users may want to utilize inside of your notifications head over to configuration automations and create a new automation in the trigger section from the drop down we're going to select state and we're going to use those two new sensors that we just discovered i'm going to first use the unknown sensor leaving the on and off boxes blank but entering a trigger id so that i can use it later then i'm also going to add the sensor for my own name again entering a trigger id then under the actions box i'm going to use the choose action which allows us to have different actions run based on what just happened super useful for combining automations for option number one under the condition i select trigger from the drop down and select my unknown trigger id for the action i enter call service and then select the notify service i'm going to switch over to yaml mode first because we're going to use a template and then i give my notification a title and in the message i'm going to use the state of the unknown sensor which will be the camera name i then add a second option this time filling in the details for the sensor that finds a match and uses my name and then i'm going to enter a slightly different message this time when i then walk in front of the camera you will then get a simple notification that tells you someone was detected inside of the camera frame and whether or not the person was actually recognized there are far more advanced things you could do here like having it run on a schedule tell you the confidence level attach images from the camera of the face that was recognized in fact i've done an entire video on ios and android actionable notifications which covers how to do all of those things which i'll leave linked up here and i'll also leave a link to the double take documentation which has a notification example that you can also follow and that's we're going to do it for this video that was your first look at and how to install double take for detecting processing and training facial recognition images easily using the double take ui you guys have been asking quite a lot about this one for the last few months now so it was really good to finally check it out and see what it was like and i was super impressed at how well it works david has done an absolutely fantastic job with this so far in this short amount of time that it's been around so make sure to show him some support but let me know what your guys's thoughts are on this in the comments box down below i know that training faces wasn't the easiest job to do before because you had to use the command line to do it so having this built right into the ui is super useful but anyways i hopefully you guys enjoyed this video hopefully you find it useful and if you want to support the channel then you can do so by becoming a patron on patreon and your support allows me to keep on making these videos thank you to all my current patreon supporters as always your support is very much appreciated make sure to drop this video a like and get subscribed and i will see you in the next video

Channel: Everything Smart Home

Views: 19,791

Keywords: home assistant, smart home, home automation, home assistant setup, home assistant guide, home assistant face recognition, home assistant frigate, home assistant deepstack, home assistant double-take, home assistant double take, smart home face recognition, home assistant object recognition, home assistant person recognition, home assistant AI

Id: _61-hIL1AjQ

Length: 12min 27sec (747 seconds)

Published: Tue Nov 09 2021