LOCAL Voice and Hearing for your Home Assistant Assist!!!

new release of Home assistant 2023.5 is here but today we are not going to look at all the things that are new in this release of Home assistant instead we will focus on The Voice chapter 2. we'll cover three methods on how you can use the new voice functionality in your home assist installation and we will test it a couple of times on couple of different systems while the latest release of Home assistant does bring some changes for example we now also have Android TV remote web hooks can be categorized to be externally or internally available workday yaml has been removed and so has been the coronavirus integration no we are not going to talk about them instead we will look at the voice functionality of Home assistant and how you can control your home assistant via three different methods first option will be to use the assist button click on the mic and turn the kitchen lights off for the second method we are going to use ESP home and M5 atom Echo device this device is great and I have been testing it for a raspy yet we can use it as a voice assistant in our home assistant and the last thing that we are going to play with is the world's most private voice assistant or we are going to dial in into our home assistant but before all of that let's look what you have to do in your home assistant setup to get everything working and installed let's start by mentioning that this installation does require you to run supervised version of Home assistant or version of Home assistant Os or supervised that has supervisor installed unfortunately at this point I don't think there is a at least not an easy way on how to recreate that inside Docker so sorry for that one but let's start installing everything we need let's go to add-on Store and install text-to-speech component called Piper install while it is installing and it should be a brief installation you can go to documentation and read documentation here the documentation may help you on choosing what type of language if your language is available can you use but also what will be the quality and the voice type for example for English we have Great Britain and us so we have English English and American English you have male and female voice but you have them all in a low quality if you go for American you have low quality medium and high quality voice types my recommendation is start with low if your language is supported unfortunately for this type of installation that is using the local text-to-speech method you don't have some languages but if you are a lucky one and have nabukasa's description you will not have that limitation nabukasa out of box supports much more languages than this one for example creation which I will also be testing in this video but also some other languages that are not available here when everything has been installed go to configuration and select your language type of the voice and the quality of The Voice we're going to use Great Britain southern English female law save it and now you can start this component I would also recommend that you tick the box next Watchdog that means that if the application does crash it will automatically restart if you go in the log files you can see that the language has been downloaded in the background anytime you choose a language quality of the language or voice type male or female it will download in the background that voice pack okay so now we have Text to Speech but we also need speech to text for that once again click on add-on store type in whisper and download whisper speech to text with whisper install it and once again while it is installing let's check the documentation it gives us information about option which is language and model I would once again recommend that you start with tiny int 8. this one has been created for the Raspberry Pi 4. if you do have Raspberry Pi 3 you can test it even on Raspberry Pi 4. it's a very slow process to get from the speech to text so I do recommend that you don't use it on Raspberry Pi 3 unless you are just testing how everything works out if you are running this on the proxmox on some Intel Nook or similar with the Intel i5 processor you should be okay my test setup this test setup is actually yellow it's working but it's not that great I still haven't tested on my main setup because I'm waiting for the beta 6 or release to update my main instance but in this video I will also show you how fast it runs on a Zima board for the model we will start with the tiny int 8 which is also recommended model for the Raspberry Pi 4 and since this is running on a cm4 board we should be good to go in terms of languages these are the languages that are supported if you want to have seamless experience I really would recommend that you try the nabukasta we will also be checking that one out a little bit later down the road let's click on Save we will restart everything and now if you go into your Integrations page you will see two new devices or Integrations discovered one is Wyoming protocol whisper and the other one is Wyoming protocol Piper Wyoming protocol is also a new protocol that has been developed by the developers to enable easier integration of various voice related or voice assistance related components and those will be also available for integration with third parties so if you want to develop something that sends or uses some of the components you can for example use this Wyoming protocol let's configure it submit and let's also configure and submit this one here in terms of installation Piper and Whisper are now both installed there is also one new setting in home assistant which didn't exist previously and this one is called voice assistance here you can control and configure and play with all the voice assistants that are available in your system for example these here are internal ones we have home assistant Cloud because I am using navocaster actually I didn't use it up until this point I did have it in my system configured and paid but I wasn't using it but since there are a lot of changes and benefits of using it I will now start using home assistant cloud in my main setup to get better text to speech and also speech to text okay we have no option of Home assistant cloud and for the testing purposes I also created my assist and English let's look how easy it is to create a new settings setting can be either the type of the voice for example the quality of The Voice or the language or male or female voice type click on ADD assistant name it recording what language do you want to use let's for the testing purposes select Slovak what type of conversation agent should we use for speech to text we have home assistant cloud and for text-to-speech you can use Google Translate or once again home assistant cloud create as I said I already have set up my home assistant cloud account it is using language English conversation agent is home assistant languages English speech to text is home assistant cloud with Australian English text-to-speech is home assistant cloud with once again Australian English and Natasha is the voice type if you want to test the voice and hear how it sounds before you save everything you can click on try voice and click play hello How can I organize if you're satisfied click on update next one I have created and this is my currently default setting is this my assist English name is my assist language is English home assistant is conversation agent languages English here we are not using nabukasa instead we are using fast whisper this is local speech to text languages English and for text to speech we are using once again local Piper English is this one here and the voice is this one here hey how come is this then I have a third pre-configured option if you subscribe you will hear it creation language this time using Cloud because unfortunately creation language is not locally available try Voice and the last one is the Slovak one which we just created on this page you can also control and Google Assistant you can either tick the box to expose new entities or don't do it see how many entities have been exposed and here also select which assistant which area which whatever is available from your home assistant to any of the assistants you have in your system okay at this point we are good to go to test everything but we have to use what we have currently the only option for me to test the system with text to speech and speech to text is by using this button here if we click it we can talk with it this is currently using home assistant Cloud English language OneNote your instance needs to be running with the SSL certificate valid SSL certificate unfortunately this is not an issue with the home assistant this is limitation the security purposely made a limitation of browsers that do not allow any microphone access if the connection is not secure so if you're having issues with that check that your SSL certificate is valid if it's valid click on start listening turn the kitchen lights on turned on light turn the kitchen lights off turned off light and we are now using default first value from the system but we can also use my assist which is different type of English turn the kitchen lights on light turn the kitchen lights off end of light this experience has not been cut or edited what you see is the initial response from the system when I command those two commands if you have previously issued voice commands and you did get text-to-speech and speech to text the system should cache something and the next response should be much faster at least that's what the home assistant devs tell us so let's try it once again turn the kitchen lights on sorry I couldn't understand that oh that's great because we can now test something else we have been using my assist if we go to the settings Voice assistance my assist if you click on Three Dots and click on debug you will see all the information from your system run done turn the key Channel lights is what it synthesized from my speech and it returned the prompt sorry I couldn't understand that and all of the information but also audio file is available here let's try it once again turn the kitchen lights on and this is how the system would run if you run it with the home assistant yellow with the cm4 with only one gig of RAM let me actually check that one just to be 100 sure sorry 2 gigabits of ram okay uh this works great so let's also check some other languages let's click on assist here if we now select subscribe please which is a crucial language we can test it once again this is a nabugasa speed and this part of the video will not be edited foreign I don't believe that even Googles or Amazon devices can respond this fast but this is just the beginning of this voice game for now we are using only couple of new components and those components are Piper whisper and the Wyoming protocol but what if you don't want to sit at your laptop and want to control device new remotely or semi-remotally I did mention ESP home let's look at what you have to do to get the ESP home assist working or voice part working first part of course is for you to Source this atom Echo M5 device which was sold out the day after the year of the voice event was held hook up your device to your PC go to the link and the link will be down in the video description it will tell you that you need to run home assistant 2023.5 or later you have to have home assistant cloud or manually configured assist pipeline which we did with the Wyoming protocol Piper and Whisper the password to your 2.4 gigahertz Wi-Fi network because we will need to connect this device to our Wi-Fi network Chrome or Chrome based browser like Edge or a desktop not Android or iOS M5 stack atom Echo development kit which is this device here and of course USB cable to hook up everything to the PC where we will be flashing this device from click on connect button I do already have this device programmed but we will refresh it once again unhook it and hook it to see the port of this device for me and you will not be seeing this on the screen here this is USB port com3 connect it will give you couple of options you may not see everything here my device has been pre-flashed so I see change Wi-Fi and also add to home assistant you will probably only see in style voice assistant and logs and console click on install Voice Assistant install and the installation will start the installation can take up to 2 minutes trust me I know what I'm doing next now we can join the network if you are lucky one and the network is in range you can refresh this screen here and your network name will pop up here type in the password and click connect next step is to add this to our home assistant if your device didn't pop up automatically on the screen you can click on ADD integration tsp and set up another instance of ESP home I know my IP address you have to figure out your own you can do that on your router or on any network controller that you are using at your home select area and click finish under ESP home you should now see M5 stack atom Echo one device two entities media player and is assist in progress or not in the configuration panel you can see the assist pipeline you can use preferred which is the one that has star home assisted Cloud which is the one that is used by nabucasa recording and my assist let's try my assist this is the English version let's now test it if we press and hold the button and say the command it should turn on and off this button turn the kitchen lights on and the kitchen light is turned on turn the kitchen light off this model has been processed locally on home assistant yellow once again running on a cm4 with 2 gigs of RAM with emphasis it was processed locally if we for example change this to home assistant cloud and test it turn the kitchen light on it is much faster in responding to command turn the kitchen light off turned off light yes this was processed by nabukasa and let's test it with the creation language this was processed once again with nabukasa but in Croatian language okay so now we've seen how it works if we use assist in a home assistant by the button and microphone we see how we can use this one here home assistant now also brings us voice over IP voice our IP is used to handle both text to speech and speech to text meaning that if you have phone like this one you can pick it up use the microphone and speaker to send and hear the feedback from your home assistant yeah I know you may be wondering why well actually people are used to talking via the mobile phones or phones so speaking in the headset should be normal unfortunately my phone doesn't support speakerphone if your phone does support speakerphone you can do it via the speakerphone too the difference between all of these systems so far and this one here is that when you pick up the headset or put your home assistant on a speaker it is continuously listening for the commands in the home assisted web interface you have to push button here on the ESP home this Echo device you have to press and hold the button but this one here will constantly do what you tell it to do either via the headset or via the speakerphone itself so let's see what you need to do to install everything inside home assistant with the analog phone this one requires home assistant 2023.5 or later the analog phone with the rj11 socket the analog phone adapter and it was tested with the grand stream ht801 there are other types that could potentially work but it all depends I will be leaving a link to where I bought mine down in the video description and also I will be leaving a link to AliExpress the ones that are sold on AliExpress are used ones so be careful about them they may not come in a pristine condition but you can get them cheaply just be warned most of them if not 100 of them are used ones but you can still use them in your home assistant and there can be a big price difference between the ones that will be on the link when I bought mine and also the ones on the AliExpress links down in the video description you also need RJ 11 phone cable to connect your phone to the grain stream this one arrives usually with the island phones unfortunately Amazon has had issues with my shipment so I will have to use this phone that I bought locally here in a shop inside and once again you need either home assistant Cloud which is much faster or manually configured assist pipeline which which we did previously you need to set up the device the setup of device is very easy you plug in one cable in the RJ45 Port you plug the next cable the cable from your phone inside the headset Jack and then you plug in the power adapter in my case this micro USB adapter was included in the box where I bought the green stream module after everything is set up you need to find the IP address of your telephone one way of how you can try and get it is to pick up the headset type in three times star then zero two unfortunately for me this one didn't work but it's also on the other hand very easy to go to your router and check up the IP address of the grand stream and use that IP address the next step we need to login to the Grand stream the username and password are admin and you need to copy this string here which needs to be later on changed to the IP address of your home assistant remember this needs to match the IP address of the home assistant where you are going to use this VoIP functionality login and this is how the screen looks for me we have to go to fxport page and remember this device is slow so it may take some time for everything to load up it's normal for those all devices then scroll down all the way to this line here of hook Auto dial what this means it tells the system that each time you take the headset of hook it will automatically connect to the IP address you see here paste it and change to match the IP address of your home assistant we have star 47 this needs to be as is then we have the IP address 192.168 1 100. my system is not 100 my system is 201 then once again star 5060. click on apply and I'm not sure if it's needed but I also clicked on reboot to make sure that everything has been load up properly back in home assistant go to settings Integrations click on ADD integration and type voice for voice over IP click on submit finish and that should be it if we now pick up the phone is connected you may hear this message because everything has been set up properly but to ensure your privacy security and everything you have to enable this phone or whatever phone you have inside your home assistant click on one device and tick this box to allow calls also you need to select what pipeline you are going to use for example home assistant cloud turn the kitchen lights on turn the kitchen lights off but as you may have seen I did send command after the command so I send the command received response set another command which means that if you are using either speakerphone or headset like this you can continuously speak to your system one additional thing that you can do with such functionalities have your phone always listening for example near your bed you have your speakerphone always turned on and then you just say the command turn the kitchen lights on and it would turn on the lights turn off the kitchen lights but once again you can play with the languages this was nabukasa let's try now with my assist which should be local only text to speech and speech to text turn on the kitchen lights on lights turn off the kitchen lights or we can try with the Croatian language foreign foreign and that's it you now have control to either use the assist button M5 atom echo or the analog phone to talk to your home assistant I know this is not the perfect and a lot of you would like wake up word but wake up word is also the hardest part to do so we'll leave that for the year of the voice chapter 3 4 or whatever chapter there will be when the Wake upward is added now that we've tested everything let's see if there is a difference of using this on home assistant yellow or using it on a Zima board device that I tested couple of weeks back on Zima board I don't have nabucasa cloud or any of the assistants so I will use this one English agent will be home assistant language will be English speech to text to be whisper we'll be using English and we'll be using Piper once again English update and we can now test unfortunately since this instance has no SSL certificate I cannot use assist if I try I don't see Mike here that's why we either have to use this one here or we have to use the analog phone which I've already set up on this system turn the ledge on turn the ledge off remember this time we are only using a local control turn the ledge on turn the ledge off and this is how fast it responds if you are using zimabort or similar Intel based device let's try the analog phone and call our home assistant turn the ledge on turned on light turn the ledge off your mileage may be different this all depends on what type of system you are running your home assistant on the faster the system the faster responses you will get from your local instance but for the best experience of course you can try and use nabukasa what do you think is the best with this release of Home assistant do you think that voice inside home assistant has future will you be using any of these suggestions either using the mic inside home assistant the atom echo or the analog forms I really am looking forward to see what your opinions on this are and if you will be implementing any of this functionality in your home assistant and I also want to take this opportunity to thank all those wonderful people that are supporting me and that have become YouTube channel members thank you all for all of your support but let's not forget each and every one of you who has watched liked or subscribed to my channel if you do want to support the channel you can do so by clicking the join button down below and for only two dollars or two Euros per month becoming a YouTube channel members or go to my merchandise store once again the link is down in the video description and buy something there and I will be seeing you next time until then bye bye and have fun

Channel: BeardedTinker

Views: 21,808

Keywords: text to speech, speech to text, esphome voice assistant, esphome voice, home assistant voip, home assistant voice, home assistant voip integration, home assistant voice 2023, home assistant voice control local

Id: MOJQU5zyoIY

Length: 28min 35sec (1715 seconds)

Published: Wed May 03 2023