Using Text to Speech with ChatGPT!

hey what's up this is Ashley from immersive insiders and in my previous video I showed you how you can integrate open Ai and unity to create an Android application like chat GP if you haven't watched that video already then you should definitely watch that first over here and then get back to this video in this video we are going to take the project a step further by adding speech recognition and text to speech so let's get started the first thing that we want to do is to download this repository that was created by Jimmy 29 now this repository has the speech and text Unity plugin that we require so click on code and download the zip file once you have it downloaded go ahead and extract it then open up the main folder Unity project asset in here you will find plugins and speech and text so these are the two folders that we require so open up your Unity project select these two folders and drag and drop them inside the project window and wait for Unity to recompile once that's done Unity is going to show you some errors and that's basically because one of the script is trying to reference the iOS and the xcode now since we are working on Windows platform and targeting Android we do not need that script so go inside speech and text editor select build post processor and delete it with that we have installed the plugins now it's time for some coding all right so here we are in Visual Studio let's get rid of the code over here and the first thing that we want to do is to create a function that can start the voice recognition or speech recognition so for that we will have it as public void start speech recognition and in here we want to make use of the speech to text class now before that we need to import the text speech Library so you can say using text speech and now inside this function we can call the speech to text dot instance dot start recording so when you call this function it's going to start recording and then you can talk and once you finish talking it automatically detects that you have finished talking and then it generates a callback so in the start method we will register this callback so create a start method and in here we will have the speech to text Dot instance dot on result callback and assign it to some other function like on speech result so let's generate a method for this so you can right click on it and generate a method so here we go instead of having the string obj will rename it as result and what do we want to do with this result the text result that we get we want to input it into the input text field that's there so for that we need to import the library text mesh Pro and let's create a serialized field private text mesh Pro input field and we'll name the variable as input field so once we get the result we want to add that result into the text field of the input field so input field dot text is going to be equal to the result okay so now let's write the code for text to speech so for that we need a function which is again going to be public void we'll call it as start text recognition and in here we need to pass in the string message or string text whatever it's going to be so string text and we can call the text to speech dot instance dot start speak and pass in the text okay so that's about it now the third speech recognition will be called by a button from the unity editor whereas that text recognition is something that has to be called from charge apt and that's because once you record and send it once we get the result that result has to be converted into speech right so once we have the result here we want to convert this into speech so for that we will create another serialize field private and it's going to be speech controller rename the variable as speech controller then once we get the result we can call the speech controller dot start text recognition and pass the result that we get now you can save this save the pitch controller script as well and let's go back into Unity all right so here we want to create three new game objects the first one is going to be called as page to text now make sure that the way you write it is the exact same way that it's written here speech to text with S T and T in capitals next we'll create another game object which is going to be text to pitch and here again this has to be in exact same way select speech to text and the component speech to text select text to speech and add the component text to speech now I'm going to repeat this once again make sure that these two components are on two different game object with the exact same name and then we'll create another game object we'll call it as speech controller and add the speech controller component to this all right so it needs an input field so let's go ahead and add the input field and now if we check out AI or chat GPT needs the speech controller as well so let's drag and drop that here now we need to create another button which will call the start speech recognition function so for that we'll select this button we'll duplicate it and bring it slightly above we'll take this below here we'll move these to our password all right so now select the second button that you duplicated and will change the text from send to record and when this button is clicked we want to take the speech controller and we want to call the start speech recognition function now there's one last thing that we need to and that is to go inside plugin Android and open the Android manifest here you need to paste these lines of code that you can find in the description then save it and go back to Unity all right so now let's save the scene and let's build it and test it out so the app has been successfully built let's test it out I'm clicking on the record button who are you send I am an artificial intelligence designed to respond to questions this is super cool and just like that we have converted a normal chat bot into a virtual assistant now where can you use this or where can you make use of this concept well if you're making a game where NPCs are interactable or if you have a educational app or something like that you can definitely make use of these Concepts also you can find the source code Unity projects and other exclusive content like sending the speech results directly to open AI or changing the language of interaction on our patreon so do check it out and if you like this video or if you found it helpful then I'd highly appreciate it if you can leave a like And subscribe it will help us to make more videos just for you and as always I will see you in the next one

Channel: immersive insiders

Views: 2,819

Keywords: AR Foundation, AR, Augmented Reality, Unity Setup, unity ar tutorial, xr, arfoundation, ChatGPT, chatgpt, AI, android, OpenAI, opneAI, virtual assistant, Unity2021, Unity 2022, tts, stt, text to speech, speech to text, voice recognition

Id: 321h9wDT-gM

Length: 6min 30sec (390 seconds)

Published: Thu Mar 09 2023