How I CREATED My Voice Assistant Device With WAKE Word Using ESP32 And Home Assistant

hey folks and welcome back and in today's video we will be looking at this device that I created as my personal Voice Assistant with home assistant now I created this device using es32 a speaker a microphone and an LED strip so today I will show you how I created this device so let's get [Music] started to start off let's look at this model that I created so now this is a 3D printed model I will provide the STL links to this model in the description below as well as in the blog that I will create so let's explore what are the various things about this 3D model so this is a circular shaped model in which I have fixed in right now only one speaker here but the idea is to put in the second speaker here also now right now let's look at the behind section so I have fixed the speaker using hot glue and this is this is the hot glue that I have put in here for the lights I have used this LED strip so now this is a individual addressable LED strip that is there and using this strip I will light up this voice assistant such that whenever home assistant detects a v word it will turn on this LED strip in this circular fashion telling me that hey it's right now listening to your voice command so that's the idea right now so now let's look at the part where in where we will be putting the esp32 so initially my idea was to put the esp32 in this direction here but I figured out that um the wires would not fit in so that's the reason I changed the direction and now what I'll be doing is I'll be using this esp32 which is the USB C powered one and I'll be placing it in this direction so I'll be putting it this way and I have put in these stilts here so these are plastic tilts that I have mounted here using hot glue and I'm fixing them in this direction right now so what I'll be doing is I'll be connecting the wires to this here and then afterwards I'll put in a little bit of hot glue here such that it remains stable this part I will now cover it with probably some kind of a tape or something like that and then this will look really nice now I'm not an expert in soldering so the shortcut mechanism that I'm using is I'm going to use these jumper cables and then I'm going to connect them directly and I'm going to use hot glue for this so that's what I'm going to do for this right now if you see we have the speaker here right now you need some kind of an audio amplifier for this so I'll be using this max 98 357a let it focus a bit so max 98 357a and this is a 3w output amplifier supporting 4 to 8 ohms of speakers so this supports i2s audio Channel and using ESP home we will channel the audio using this audio amplifier now this is for the audio part let's look at the input part that is this microphone so I'm using this inmp 441 so this is a omnidirectional microphone it's pretty cheap I got it for like nearly €3 from AliExpress I'll provide links to all of these components into the description below as well as into my blog so now in my previous video I had shown you this setup here wherein I had shown you how you can use an esp32 with microphone here so this is the same microphone that is here and the speaker is present here as well as the audio amplifier so I programmed this esp32 using ESP home and then connected to home assistant using the es home while I was doing this experimentation I figured out that when the LED comes on the light kind of bleeds out basically uh the light gets fired in the center Direction but it does not come out as such right so that's why I created this kind of a cover so what my plan is right now is to put in this cover this way such that whatever LED that gets turned on it will now fire all the light from here only it will not bleed out inside now while doing this I figured out that I need to make some space for this section here so right now this which fits here right now I have to cut it out such that I give some space for the esp32 as well as give some space for this wires also so let me do that right now and get back to you so right now I have cut this section here so as to allow me to connect this here and give some gap for the esp32 as well as these wires that are going to come in here okay so there we go so it's now sitting flush inside so there is no Gap now for the light to bleed out and this is how it is right now so let me actually put the es32 back here and it sits perfectly so now this is also helping me to level this this USB C Port also and this is how it's going to be right now so this takes care about the LED part right and all the connections are here now where am I going to put this microphone so for this microphone I have this small slot here so I have this slot here wherein I will actually so I will have the microphone like this in this direction such that this hole allows me to actually have the microphone input so that's what I'm going to do right now but it looks like I can make a small little Dent here such that this fits in flush like this so let's do that I'm going to now make a small little incision here so right now what I've done here is I have now made a small incision here such that this microphone can now fit in flush so once this microphone is now placed here this will be actually from where it will be listening to so that's what I've done I've also made this hole a little bit bigger than how it was originally there so right now I've just melted it a little bit so that this hole is a bit bigger let's look at the connections that we need to do so before that let's look at this these things that have been done so I have covered this entire LED strip with this section right now and I have one speaker in place let's put the other speaker so I have this speaker here I'm going to put this here and there we go so this one is also now in place so now let's place this here and we are now going to do the circuit diagram that we can see here on the screen so here the circuit diagram which is there I'm going to make the connections for all of these things so let me start doing the connections now okay so with this I have now connected the speakers to this amplifier next what I'm going to do is I'm going to do the connections that are required for this I'm going to use this jumper cable and I'm going to connect this so the wiring of the speakers are now completed let's actually now do the part for the microphone so let me connect this now so now this should seal the connection to this microphone and we are now going to connect to the final piece that is we're going to connect it to the es32 so I have this esp32 here let's actually connect this now before I made the connection with the esp3 2 I need to make these common connections which is this right now that is connecting the BLC and the lrc pins with the Ws and theck pins so that's what I've done here right now I made these common connections between the microphone and the audio amplifier now I'm going to actually connect this to the es32 so now I have everything connected in this Jungle of wires here uh all the different components that we need are all connected the W LED strip is connected everything is now connected to this es 32 now what we need to do is actually flash our code so let's look at the code that we need here now this is my ESP home dashboard now if you have not set up ESP home there's a link somewhere here wherein I've shown you how to set up ESP home in home assistant so right now I'm running this separately using Docker container so I can access the UI directly so now let's actually create a new device so I'm going to click on new device here and I'm going to click on continue and let's call this as Wireless assistant now I'm going to click on next and I'm going to select esp32 so now this is the encryption key that we might need while connecting to home assistant right now I'm going to click on skip here and I'm going to go inside this Wireless assistant inside this what I'm going to do is I'm going to paste this code here so this is the code that I need right now to flash onto my esp32 and let me explain you the various aspects for this so as you can see we have these connections that are being made the i2s audio connection then we have the microphone here with the DN pin connected to gpio 33 then here I'm using a media player so what's the advantage of the media player is that I can control the volume of the output of the speakers so that's why I'm using media player right now at this moment when I'm releasing this video the media player is not supported yet with this voice assistant because there was a bug related to this and this is right now fixed and it is working on the dev Branch so by the time we have the December release of Home assistant 2023 that is 202312 version you will get this fix so I'm already using the dev version so that's why I can use this right now so considering this is there and and it you have already 202312 version of Home assistant it should not be a problem now here what I'm doing is I'm connecting the media player here using this ID from here as well as the microphone which is this ID from here okay now these are the various parts that I've just connected together right now I had to actually do the stuffs related to when it detects a vord what it should do what it should not do and that's what I have configured here so let me show you the part about the LED strip so if I go down here I have this LED strip which is connected to pin 32 here and I have one of the effects that I have put in here that is the scan effect with custom values so I have kept in certain values here such that I can do that round scan thing with the LED strip now let's look at one more configuration here so if I go back up here here on V word detected what what I'm doing is I'm turning on the LED and then afterwards when it is going into the listening mode I'm turning on the LED strip with the effects being set up here I'm specifying the color that I need to display into the LED strip and finally what I'm doing here is when the speech to text is completed I'm turning it on but setting the values to Green here so here I'm setting the LED strip to be green when the pitch to text completes finally we have certain error conditions here so I have said that if there is some kind of an error then I just turn on the LED strip in the none mode that means it will just display something in white color um along with these there are some fixes that I had to do this is meant for all the voice assistants that are creating using esp32 and you're running the assist pipeline completely local so you will need these fixes also I will provide the entire yaml into my blog which I will link into the description below so you can get all of this configuration and figure things out along with this this binary sensor is also needed to make sure that the connection with home assistant remains stable all the time so this is what is there also and finally we have this switch which is required for enabling and disabling the VW so this is what these configurations are now what we're going to do is we are actually going to now go ahead and install this so for the initial sake I will have to install it manually so I'm going to download it manually here and let it compile and create the binary and I'm going to download this on my laptop so now I'm going to select here the modern format I'm going to click on this and I'm going to save this now next what I have to do is I have to flash this Bally so for this I'm going to use this web. home.io and I'm now going to connect my es32 to my laptop using the USB cable so let me do that if I find this so I have this USB cable I'm going to now connect this here and for the first time what I need is I need to actually hold the boot Button here so there's this boot Button and I'm going to hold it while I connect this to my laptop so I have connected this to my laptop now I'm going to click on connect here I'm going to select my esp32 and then I'm going to click on install and I'm going to select the binary file from here so let me click on this and I'm going to click on install now I'm going to wait till it goes into this erasing phase and I'm going to leave my boot Button from here so now this will start installing the binary that we just downloaded from our es home so let it install this right now now our esp32 is successfully flashed with the code now what we have to do the interesting part that is connected to home assistant so let me close this here and let me go to my home assistant so this is my home assistant instance and I'm going to click on devices and services and here you should be able to see the device already so now it has not figured out this device yet so what I'm going to do I'm going to click on integration and I'm going to search for ESP Foria and here I'm going to put in the IP address of my esp32 so I have figured out this is the IP address so I'm going to click on submit now and then it's going to ask me for the encryption key so let me put in this encryption key from here I'm going to copy the API encryption key and I'm going to paste it so there we go our wireless assistant is now connected to home assistant let's go ahead and see this device now so I'm going to go to ESP home here and I'm going to click on this device and as you can see we have the esp32 configured in a home assistant right first thing what I'm going to do is I'm going to select my pipeline that is the home assistant Pipeline and the speaking detection I'm just going to keep it as default now the use wake word is on so let's try it actually so before that what I'm going to do is I'm going to flip this device so now the moment of truth let's try this hey Jarvis turn on the dashboard turned on switch there we go so it's all working all fine right now so now what I'm going to do next is I'm going to put all these components inside properly Stitch It Up put hot glue so that it fixes things properly and then show you the final output so before that I want to show you how I made this circular thing so my initial idea was something related to having an acrylic plate which would be a little bit on the darker side and then I could cut it into a round shape but now I don't know how to do that as well as as well as finding something like this similar was pretty difficult so I did a small shortcut here so what I did was you must have seen envelopes like this so these are some envelopes that you get to keep your papers I've used this type of an envelope for this and I created this round thing from that so I placed this on on on that plastic thing and then after I traced it around and I cut it into this round shape so now and I put this on top of this it looks really sexy so let me try this again hey Jarvis turn on the dashboard turn onp switch so there we go it looks really nice with this so let me now put all this components inside and place it over here so now I have connected everything inside and it is now looking like this so this is how my voice assistant is going to look like so let's go ahead and Trigger this using the vake word hey Jarvis turn on the dashboard turn on switch so there we go so now I'm going to use this as my daily driver for voice assistant to invoke home assistant using the V word so I hope you were able to understand how I created this personal voice assistant for me which works with home assistant now now in my next video I'll be showing you how you can route the output of The Voice Assistant to an external speaker using home assistant so make sure to hit that like button as well as hit that subscribe button for more such videos to come till then take care and I'll see you in my next one

Channel: Smart Home Circle

Views: 12,943

Keywords: raspberry pi, smart home, home automation, home assistant esp32, wake word detection, wake word, home assistant local control, home assistant voice assistant, home assistant local voice control, local voice assistant home assistant, home assistant voice control, home assistant voice, home assistant guide, local voice assistant, voice assistant esp32, home assistant setup, esphome voice control, home assistant wake word, home assistant assist, voice assistant, whisper, piper

Id: aEs5m-sn_wg

Length: 18min 22sec (1102 seconds)

Published: Thu Nov 23 2023