Speech-to-Code - The Future of Programming with AI? | feat Claude 3 Haiku

can you generate a code that searches my download folder for a song named arcade uh open it and play it in the desktop okay so I want you to generate a python code that writes an HTML website that has the H1 title hello YouTube In Like A Big Font let's say like size 100 and also so generate a python code to save it as index.html and open it in the browser hey what I just showed you here is my attempt at creating an AI speech to Code system that will always generate code from your natural language input so let me just show you a bit more on how this works so we start here on the top left with our speech input right this is going to be transcribed to text using fast whisper this is built on open eyes whisper this text is going to be sent to our python Hub the first thing it's going to do is to identify the programming language we are going to use right and you're going to send that information over to the um model we are you going to use that is going to be Claud tree hu uh we're going to use the API and oh I got to say this model is amazing so I have a plan to do like a dedicated video on Claude 3 Haiku uh probably on Sunday because this model is yeah uh just look out for that video and Hau is going to generate our code send it back to the hub and the Hub is going to try to send it over to identify all the dependencies we need to run this and it's going to install it after we have installed all the dependencies we actually going to execute this code and display the output and I added this function that when we have executed this code if we get any errors or something like that is going to be added back to our python Hub so we can use this as context right and that means that we can iterate so let's say I said uh I want the font size to be 100 then I can just follow up it change the font side to 50 right so that is kind of the uh I'm going to show you some examples of that uh but yeah basically this is kind of the workflow I've set up for this and yeah I'm very happy with it so before we do some more examples let's take a look at a few key lines uh in the code okay so if we look at the execute code function here you can see we have two uh arguments here we have the file path and we have the language so what I did here is that I wanted to kind of set the different languages I want to use in this so I have selected python JavaScript uh golang and HTML so I can always add more languages here if I want to use this right so this is is kind of customable and it kind of detects if I say I want a JavaScript code uh this is going to detect that or not this one but I have a function to do that and this also mean we had to update our find and install dependencies function here we're going to take in the code as an argument and the language we are going to use so here you can see if the language is python uh yeah we going to import and kind of run pip right and stuff like that if the language is Javascript we're going to use use npm and goang we're going to go get and that is kind of how I set this up so yeah it's been working pretty good and also the clean code so this is where we kind of remove uh things we don't want and just replace it with uh whites space so we kind of have the python JavaScript JS go and yeah works pretty good so I wanted to show you one more key line in this code before we do some more examples so what I had the biggest struggle with when creating this was kind of identifying the language we are going to use uh so I kind of ended up with using this prompt here and it's kind of two-part so identify the programming language uh for the following user input and then we kind of pass our user input here that is transcribed from our speech right and I just to only respond with identified programming language right uh but I also had to add like a custom system message here so if you go up and look at that I just set it to system message tree you are an expert at identifying programming languages from a text and I follow up with always respond with just the name of the identified programming language and after I put in that it seemed to work pretty good so when I say like I want a python code to generate this and this uh it identify and just takes out python uh because we kind of need that phrase right up in here so it has to be like JavaScript or let's say python so then we identify this as python then our function is going to work so uh yeah it's been working pretty good after I updated this so that was kind of my an issue I had with this but uh I think it's working pretty okay now so yeah that is kind of the key Parts I wanted to Showcase from this uh code and as always if you want access to this I'm of course going to upload this to the community GitHub so just become a member follow the link in the description and you will get access to this full code I'm going to upload it to GitHub so you can try it out yourself uh but now let's do some more examples and see what kind of things we can do with this okay so this is kind of what I wanted to try first so this is going to be a whole sequence of operations right so the first thing we're going to do is generate a python code to open the browser on the web page Hacker News take a screenshot of the website and save it to our directory images then I want to generate a python code to just show the screenshot and then I want to generate a python code to open the screenshot in Ms paint draw a circle on the screenshot in paint and I want to do 100 circles on the screenshot and then I want to try to do 500 red circles uh on the image so let's just try this and see if we can do all this in like a sequence here so yeah let's see what happens okay so I want you to generate a python code to open the browser on the web page Hacker News take a screenshot of the website save it as hacker. PNG to the directory images okay so here you can kind of see it identify the language we are going to use as python it's going to write the code it's going to install the dependencies we need and hopefully execute the code okay so it's going to Hacker News took the screenshot that's good okay good job now just forget the previous code now generate a new python code to show the screenshot you just saved okay perfect so here we have our screenshot that is working well good job just forget the previous code now now I want you to generate a python codee that opens the screenshot in Ms paint okay so I think you misunderstood me just forget the previous code generate a new python code that opens the image in Microsoft Paint yeah perfect so now we have the co image here in paint so that's good okay good job now generate a python code that draws a red circle on the screenshot in Microsoft Paint yeah that's good we can see the red circle here perfect again good job now I want you to generate a python code that draws 100 red circles on the screenshot in Microsoft Paint wow that was a lot of circles so let's up it to 500 again good job uh now generate a python code that draws 500 red circles on the screenshot in Microsoft Paint oh wow okay so we got there in the end so that kind of covered the full screenshot we took right so pretty funny uh so yeah well executed and I would say that was a success also you can see here I guess I didn't explain this but we also have an assistant that kind of explain everything that is happening with the code so you can see Loops 500 times and generates a random Circle for each iteration so this is just something I added to I guess I didn't mention that there but it's pretty cool so yeah good job on the first example let's do one or a couple of more I guess okay so let's try to do something a bit different so now I'm going to try to generate a python code that just creates a text file name llm in our current working directory and then I want to append the following to the file and kind of save it again and then I want to try to generate an HTML website that takes the content from the text file and kind of displays it on this site so let's see if we can come up with commands that will execute this okay so start with generating a python code that will create a do text file named lm. text in our current working directory okay good job now I want you to generate a python code that will append the following to this file hu is a great llm okay good job let me just uh let's just create a python code to open the file so I can see that everything looks good okay that's good so the next step is going to be to generate uh a python code that will create an HTML website that will display the contents of the text file so generate a python code to do that okay so I think you're trying a bit too hard here so just forget the previous code generate a new code that we use uh python to just create a simple HTML website that will display the contents of the llm do text file that is in our current uh working directory so use a python code to do this okay good job now I want you to generate a python code that will open the index.html uh in the browser and yeah we we got it in that was a lot of back and forward but yeah uh I know it didn't spell Hi C right but uh we got the website we got the IND this HTML and yeah I guess that was assignment executed well took some time though okay so I guess the only thing we haven't done now is try to generate a code in every single language so we're going to try to do this sequentially so yeah I haven't tried this before but I hope it works so yeah see how this works okay so can you generate a code in JS that counts to [Music] 69 yes that worked perfect now forget that code generate a new code that counts to 69 in go okay so we got go that's good yeah looks good perfect okay good job now just forget that code and do the same in Python okay so that was a bit risky but it worked good job again well done so the last thing is to generate a code in HTML that counts to 69 okay so I don't know if we have any way to execute this uh but let me open this and see uh if it worked uh but yeah very good we changed from JavaScript to go to python to HTML uh without any issues so every single extension worked and every single extension compiled so yeah I'm very happy okay so here is the HTML by the way count to 69 and yeah I guess I guess that worked so uh good job okay so that is basically what I wanted to share today like always if you want access to the code just become a member member of the channel I will put this up on the community GitHub and you can download it from there and try it out for yourself and make more features if you want to so I have some plans I want to do but uh yeah that was basically what I had for today uh look out for the video on Sunday about the Claude 3 hu uh very happy with that model so I have some cool things to show on that thank you for tuning in have a great day and I'll see you again yeah on Sunday

Channel: All About AI

Views: 7,666

Keywords: speech to code, ai engineering, ai engineer, claude 3 haiku, ai coding, speech to text, llm, python, ai, software engineer, future of programming, future of coding, future of software development with ai

Id: gW0RmrhoSyA

Length: 13min 59sec (839 seconds)

Published: Wed Mar 20 2024