A Face for Home Assistant Voice!

this is uh the esp32 S3 box 3 development the home assistant dudes have made this project uh available uh which hopefully will work pretty well for us right off the bat this is how this is what it's going to do basically so it's going to make a voice assistant but what's unique about this particular thing is it's got a screen it should put a little face on there on the screen and let us uh see the home assistant um Voice Assistant like thinking and stuff so that's what we're going to try and do you guys ready so these are ready-made projects on ESP this is a readymade projects page I I assume that this readymade projects page is going to expand because they already have some things you can just click on here and then you can connect and install so you can make uh an empty ESP home device you can make a media player and that gives you the devices you can select which device you have so they're really trying to make it easy to do this uh you can do a Bluetooth proxy which I actually need to do I need to do some of these or you can do a voice assistant we're going to do voice assistant so select your variant I have the S3 box 3 and it comes with just like it showed on that Amazon thing it comes with a whole bunch of things it comes with a a base that you can actually put a battery in uh and then this base has other sensors on it like it's got not sure what sensors exactly it says radar this looks like a light sensor or something so we'll have to play with that and see what that does oh infrared this is an infrared sensor but then it also has a couple of other bases which are very diyable so this one has all the pins um that you could potentially use that you could just connect jumpers to this one goes like as if it were going to bolt into another device like I don't know maybe a animatronic parrot and does the same thing where it breaks out the the pins the gpio pins that you could use for other things right here and gives you some uh screws that you can you know screw this onto the face of something else and it's just got a nice flush uh finish and you can pop this thing on and off okay or it also comes with this one which is the ultimate DIY version which doesn't give you really any base at all but it just gives you headers just gives you headers there for the pins so it comes with all those things I think I'll just start with this one connect this to the computer I guess when we first Power this up it'll probably show us the demo let's see touch to return next functional button customized by user so this has buttons on the sides all right let's just see what it does uh press button to reset the device user customized steps for voice assistant say hi ESP to wake up the device whoa say command turn on the lights see you next time didn't like me this is just an example right this isn't gonna I don't know what this because it's not linked to home assist or anything yet so this these are just example Pages anyways we're going to flash it with something else and it's going to all go away all right connect there it is connect install Voice Assistant here we go wow that was really fast now we've got a home assistant logo or at least just the blank all right we're going to add to home assistant okay that's my home assistant URL this guy office finish hey he got a little face already okay so look it's got a face it's a little home assistant guy he's gray right now and he has a little smiley face this guy here you guys see this allow this this device to make home assistant service calls yes okay finish so it is on use wake word let's let's see if it's set to the default wake word hey Nabu turn on office light did you guys see that that so that's all you got to do hey Nabu turn off office light turned off its own backlight though just turned off all the lights yeah see my names are so screwy I I haven't been naming things with the intent of being able to have them turn things on and off hey Nabu turn off keyboard it worked it worked it worked okay Nabu turn on keyboard yeah turned on so if you guys can see this I wish you could so he's got a little happy face right then when I say hey Nabu turn off keyboard that's his thinking face and then he turns it off so when he's gray so right now he's gray but then when you do when you do the Wake word hey Nabu turn on keyboard he does a thinking face and then he turned them on um yeah he sticks his tongue out so now let's look at something I want to just show you this uh this is the ESP home sketch and what you can do when you make your own is you can change the images that it puts on the screen so if you wanted to do your own little images for what it does when when it's thinking or whatever uh in this ESP home sketch somewhere around here I was looking at it earlier and I saw it there it is it right here it uh you can re uh assign which files it looks at for the different things so it's calling this thing Casita and when it's doing and up here somewhere is Cita Idol and uh it's going to take this image and put it on the screen when it's idle it's going to take this image and put it on the screen when it's listening um it's going to take this image and put it on the screen when it's thinking so you could change that to whatever you want it to be to make it look different for you is't that awesome that was pretty dog on easy oh I do want to adopt this we're going to call it the ha voice box that makes sense or maybe we should just call it the voice box we're just going to call it the voice box we're adopt okay just to summarize this little guy is really easy to set up wasn't that simple like I just went through this these steps here went uh to home assistant and just uh like accepted it and then way way it goes oh that was the thing we were going to do we were going to do the custom wake word okay custom wake word time guys ready custom wake word for the first run it's recommended use OK Nabu just to run the setup sorry you're fine that's okay once you have it all set up you can create your own wake wordss well yeah all right this is what you got to do you got to have one of these development kits whatever this one works fine create your own wake word think of a wake word there is a whole list because you have to kind of train these wake words but there is a whole repository that somebody wake word repo there it is found it so these are wake words that somebody already went through the process uh think of the word three or four syllables kind of tells you open the Wake word training environment and this was this is like a um Google collab page put a word in here so you find a word that's going to work and you you have to kind of spell it phonetically in here and then this part is what takes forever is training the model training the model takes a long time so but this is how you train it so you could have your own custom word if you went through this repo here these are all ones that other people have done but there's some fun ones here like hey Frank hey honey hey Homer so you get the word uh you go this is how you go through that process to create what those guys already have in their repository right um does it sound good run all it'll make these files once the process is finished you should have two files in your downloads folder TF light and Onyx files only TF is used okay so that's the TF light is the file we need so if somebody else wants to make your wake word for you or you want to download somebody else's TF light file that's the Wake Word file congratulations you just applied machine learning to create your own wake word model yay next step is to add it to home assistant boom we can do this there it is it is an add-on so you do have to go in and and um install this open wake word add-on okay but you just go here just search open wake word you know and it'll and you have to install that first does the ha voice assist have audio reply yes it does I'm sorry you guys couldn't hear it because it's pretty quiet but yes it does samb install yes so okay there it is we got it share folder excellent new folder open wake word great all right now I got that folder uhuh okay drop your shiny new wake word model into that folder all right let's go get a a wake word here which wake word model which one do we want hey Santa sure let's do Santa download download the raw file hey Santa excellent whoop so now hey santa. tfight is in our share open wake Word file or folder sorry okay go to settings and voice assistant either create a new assistant and and select add it select add assistant or edit an existing assistant okay voice assistants let's add a new assistant and we're going to call it voice box voice yeah sure whatever we'll just leave that for now okay wake word open wake word and then what wake word do you want and hey Santa is on the list create all right we're almost there we're almost done almost done for the day smells like bacon down there we're going to go to ES P home 26 devices we're going to find our voice box here it is and then in here we will switch the pipeline to Voice Box until to use the Wake word and that should be it toggle the Wake word okay good call vob good good call we'll see that's it okay hey Santa yes I love you yeah so that was the trick turn the Wake word off and back on vcob got it Turtle plate was not far behind Okay so now this guy is there and what was it we had him do keyboard right hey Santa turn off keyboard hey Santa turn on keyboard [Laughter] so happy woo we did it that was awesome that was it we did it we did it we did it okay so exciting so exciting I was just looking at all the other settings on this thing um so so clearly in this for this device they have not taken advantage of all of the sensors and stuff that are available here which is probably possible but what we've got to do is instead in in in the ESP home sketch instead of what's in there we need to put this in there so we can modify it it's not bad 50 bucks it's not bad I I wonder if there are I mean I'd have to look at the pinout and stuff it would be great if you could hook it up to bigger speakers you know hook up to a powered speaker um would be one thing to do but everything else about it is pretty awesome I mean the fact that it's got a little face on it and like I said you can if if we take this sketch here I am not uh probably ever going to um train my own wake word I seriously doubt it I I it's great that there's a repository here full of custom wake words and that's good enough for me and I'm sure that it's going to keep on growing actually it hasn't grown much in two months come on community had some Wick words gosh instead of this just being this if we uh use this instead then we can go in and uh change the files and we can do other things like this has got a lot of stuff um but I don't think it has all of the sensors that this thing has available to it the screen is a touch screen and uh with this base there's um a radar an infrared and a temperature and humidity sensor but there's these two buttons on the side as well a boot Button and another button that you could assign stuff to but this is this is awesome so we had a good day did we have a good day everybody agree we had a good day can we I'm going to set it right here next to my Google home and my ala screen genuine question does does it accept naughty words yeah yeah any word you want want to build a voice assistant into a Disney Builder Droid yeah uh and there's going to be a contest there's going to be a um home assistant voice contest for who can make the coolest looking stuff watch this girls you ready where are we watching this little guy here you see him yeah okay tell say hey Santa turn on keyboard hey Santa turn on keyboard all right as always thanks for watching until next timeos at least if it's at least a half hour long it counts as one of the streams of Christmas that's my new rule and like I said before I make the rules so bye

Channel: DrZzs & GrZzs

Views: 17,782

Keywords: DrZzs, wLED, Dig-Quad, Dig-Uno, Dig2Go, Dig-Octa, Home_Automation, Home_Assistant, DIY, Electronics, Tasmota, Arduino, 3d printing, smarthome, hassio, esp8266, iot, nodemcu, drzzs, wHobbit, wHobbit-Hole, wemos d1, esphome, home automation, automation, tech, drzzzs, dr zzs, dr zzzs, hassos, haos, home assistant, home assistant automation, iot home automation, drzzs sonoff, tech hacks, technical hacks, esp32, iot automation, OpenHASP

Id: P4SV8jO3-r4

Length: 15min 42sec (942 seconds)

Published: Fri Dec 15 2023