How to Use Your Voice as Input in Unity - Microphone and Audio Loudness Detection

how can you use your own voice as an input in your game in this video i will answer this question by showing you how to detect the microphone loudness that we will use for two examples animate the mouth of a character and a blue detection system so without further ado let's get started with the tutorial okay so as you guys might already know this youtube channel focus is on virtual reality development but for once what i will show you in this video can be applied for any type of game the only thing that you really need is a microphone so if you are new to this channel and want to learn more about vr don't forget to subscribe down below okay so the first thing that i'm going to do is create a new empty game object we can call it audio loudness detection now on this game object i'm going to create a new component called the same name which is audio london's detection now before being able to detect the loudness of a microphone we need to be able to detect the loudness of any audio clip so i'm going to create a new function that will return a float call get loudness from audio clip this function will take two parameters an integer called clip position which will represent the position in the audio where we want to check the loudness then the audio clip itself called clip now here is the trick to get the audio clip loudness so if this is our audio clip wave data we are going to take the clip position we want to look at then collect a small array of data before this position and this collected data will be used to get the information we want about the loudness in this part of the clip okay so first thing first we can go at the top of our script and add a public ink parameter called sample window we can set it to 64 for example and this will be the amount of data we will collect before the clip position now to get the data as explained we can do int start position equals clip position minus sample window then we can initialize our data array with float bracket wave data equals new float bracket sample window and then fill our array of data starting at start position with clip dot get data wave data start position perfect now we have the information we need to get the loudness of the audio clip there is obviously multiple way we could digest this data to get the loudness but for this video i'm going to take the absolute mean value on this array we can do so by creating a float cool total loudness and set it first at zero then we can browse our array from 0 to sample windows in a for loop and simply add to the mean loudness the value we are at so total loudness plus equals mat f dot abs wave data i now here it's important to note that the value of wave data range from -1 to 1 with 0 meaning there is no sound this is why it's important to get the absolute value of the wave data to always get a positive number which is the intensity of the signal now finally that we have the total loudness we can simply return it divided by the length of array so sample window in our case to get the mean value in the array okay guys the hardest part of the video is behind us with this get loudness function we we we should be able to use the loudness of any audio clip in our game and later on the microphone now just a little tips if we are taking the loudness at the beginning of the clip the start position can be a negative number which will cause an error so an easy fix is simply in this case to return a value of 0 and taran now everything should be handled correctly now to test this i'm going to create a little cube in my scene and place it in front of the main camera perfect now on this cube we can create a new component called scale from audio clip for this component we want to use our audio clip loudness detector to change the scale of the cube and so i'm going to add a public audio source called source to public vector 3 min scale and max scale finally a public reference to the audio loudness detection component we just made and in this case the audio source that we have here will play a new the clip that we want to listen for the change of scale and simply in the update function we can do float loudness equals detector dot get loudness from audio clip and then give it the position in the clip the audio source is add so source dot time samples and the clip itself so source dot clip and using this loudness we can change the scale from mean scale to mass scale with transform.localscale equals vector3.lerp mid scale max scale longness and this will learn the value from min scheme to max scale depending on how big loudness is okay okay now let's save our script and go back to unity so let us not forget also to assign the value of this component so we can first add a new audio source on this cube and then drag the audio source component that we just added to the scale from audio clip component then we can drag the loudness detector in the good parameter following scale i want one one one and for my scale we can say three three three and we could also use any audio clip that we want in the audio source but in my case i just recorded this simple audio with my microphone of me slapping my finger which will not get me any copyright infringement so we can simply drag it in the audio source and make sure that it plays on awake ear and for the one who really dig that audio you will find a link in the description to download it of course and maybe it will really soon on spotify but without further ado let's click on play to try our game and there it is our cube is scaling depending on the audio but as you can see this is not perfect for example i would prefer if the cube would scale only if the loudness reach a certain threshold and same goes for the sensibility of the clip maybe i want the loudness to influence more the scaling and just because we are here together and that i know you've already left a like under this video i'm going to show you how we can do this first we can add two parameters at the top of our script a public flowed loudness sensibility and then a public float threshold for the sensibility we can set it first at 100 and for the threshold at 0.1 you can obviously play later with these settings for the desired result but now using this parameter i can simply multiply the longness by the landeth's sensibility parameter to increase its influence over the scale and if the londoness is less than our threshold we can set it to zero now let's save our script go back to unity and click on play and there it is now you can see the difference the result is way more cleaner with the audio with these two parameters okay so this was for using the loudness of another clip now let's learn how we can use this exact thing but this time for our microphone so let's go back to our audio loudness detection component the trick here is to create a new clip for the microphone so for this we are going to create a new function called microphone to audio clip first we need to get our microphone so we can do string microphone name equals microphone dot devices zero this will return the name of the first microphone plug-in but if you have multiple microphone you can also choose any one you want in that device array or directly specify its name then for the audio clip we can create a private global variable called microphone clip here and use it in our function with microphone clip equals microphone dot start the microphone name true this parameter is to say if we want the clip to loop then 20 and this one is for the length of the audio clip the length does not really matter because we are looping but you need to give it a number big enough and finally the frequency of the clip that you can do with audio settings dot output samples rate and this is to directly take the frequency in your project settings and there you go just like this we have created a new clip based on our microphone now what we need to do is simply to call this function at the start of the game in the start function just right there the last thing we need to do is go below and create a similar function as this one that returns a float call get loudness from microphone and now the real magic happens we can simply return get loudness from audio clip with microphone.getposition the device name so the device is bracket open 0 and the microphone clip tada there it is guys everything is now ready we can save our script and go back to unity now in the same way that we did for testing the audio clip i'm going to test our new function by first duplicating our cube we can remove the audio source and the scale from audio clip component and we will add this time a scale from microphone component now inside the script i'm going to simply remove everything go back to the scale from audio clip component copy the interior of this component and pass it here there you go now the only thing we need to do is replace the gate loudness from audi clip to get loudness from microphone and this time without any parameter needed perfect now we can try to give the same parameter as before so with min scale 111 max scale 333 sensibility 100 and threshold 0.1 and now if we click on play the cube should react to my voice and there it is guys when i speak as you can see the cube scale on the right based on the microphone eh and it worked pretty well nice no there is obviously plenty of things that we could do with this thing and because it's a via youtube channel i want to show you two examples that use this in vr and here it is now you can have a look at the first example that i have here in vr as you can see i've simply used the scale from voice script that we just made to animate the mouth of this avatar and it works so great the trick was just to make here this little sphere here and to use our script scale from microphone with two different uh scale value to make it look like i'm talking okay so now the next thing that i want to show you as you can see i have here a bubble stick and what happens when i blow it work for this bubble stick it's actually a bit the same script as always simply i have here my longness sensibility and threshold and if the longness exceed the threshold i play the particle system that have the bubble otherwise i can stop them and as simple as that we are able now to use our microphone loudness detector for multiple behavior and i hope that this gives you plenty of idea for your own project and there it is guys i hope that these two examples will give you plenty of ideas for your project and i hope that you enjoy watching this video now if you want to support the channel you can leave a like and subscribe below as always you can find the source code of this project in my patreon with other exclusive content like this video on how to make a portal with the oculus quest path through thank you for watching and see you soon bye

Channel: Valem Tutorials

Views: 37,196

Keywords: voice detection unity, voice loudness detection, audioclip loudness unity, how to get audioclip loudness, use microphone as input, voice controlled game unity, music reaction unity tutorial, how to react to music unity, unity voice reaction, voice input virtual reality tutorial, how to detect blow unity, how to react to blow unity

Id: dzD0qP8viLw

Length: 13min 34sec (814 seconds)

Published: Wed Feb 16 2022