I tried to build a REACT STABLE DIFFUSION App in 15 minutes

there have been some serious enhancements when it comes to AI pad image generation largely thanks to machine learning one of those models includes stable diffusion but the gui's for the majority of those apps kind of suck so today we're going to be taking a look at a little bit of JavaScript namely react what's happening guys welcome to another episode of Code that in today's episode we are going to be building not one but two parts of an application we will be building a full-blown stable diffusion API using fast API and a whole bunch of other amazing libraries but we're also going to be building a full stack application using react to be able to render images from stable diffusion pretty cool right now this is largely in part because there is no API that is made available for the stable diffusion model which is made available inside of hugging face so we're going to build our own but as per usual we've got a couple of rules when it comes to code that first and foremost I'm not going to be allowed to use any pre-existing code outside of the react application shell namely because it takes a little while to install if I do look at any pre-existing code stack Overflow or use GitHub pilot it is a one minute time penalty second that brings us to the time constraint I'm only gonna have 15 minutes to do this but I'm allowed to pause the timer if I am testing this out I'm gonna need every single minute I can get out of this guys seriously this is going to be and last but not least what happens if I don't make that time limit well it's going to be a 50 Amazon gift card to you guys ready to do it let's get to it alrighty guys 15 minutes on the clock let's go okay so the first thing that we're gonna need to do is create our API so I've got a virtual environment set up for our python environment so I'm just going to create a file called app.pi and then we're going to need to import a bunch of dependencies we are going to need a puncher for this so I've got an auth token for hugging face so this is my hugging face or token so I'm going to be using that so um from auth underscore token import auth underscore token and then we're going to import some stuff from Fast API to be able to leverage that so let's make that a little bit bigger so from bus API import fast API and then we're also going to need a response object because that's how we're going to send back our stable diffusion image all right then from Fast API Dot middleware so we need to be able to enable cores or setup cores and this is going to last to make a API request from our JavaScript application to our API backend all right cool so then we need to import torch then from torch import AutoCast so we're going to use that to send our stuff to our GPU then we need from diffusers we're going to import the stable diffusion pipeline I think it's stable to Fusion pipeline um there's a bunch of other ones and then we need to import some stuff to encode our image so from i o import bytes IO and then import base 64. so that's going to allow us to encode our image all right let's save that and then we need to go and what do we need to do uh app equals why is this so stressful all right so that's our app we then need to add middleware and we are going to add our core's middleware so this thing over here and then uh allow I think it's allow credentials equals true we then need to allow all the different Origins to be able to hit our endpoint so this is effectively saying allow our AP or allow our app to be able to call to this API so we're going to say allow all Origins uh allow or methods on pump and then we're gonna say uh allow headers boom boom then we'll jump over to the app once our API is built all right so that's our middleware we then need to import stable diffusion so let's just quickly create a framework so app.get so this is going to be our route so we're just going to have a basic endpoint which is just forward slash and then def scoring or actually we'll call it generate oh no one of those lights died all right and then that is going to take in an object called prompt and prompt is going to be the string that we passed through to actually generate our image so we'll come back to this in a second but we actually need to load up our stable diffusion model so our model ID oh Lord this is intense company oh this is intense is for slash stable Dash diffusion Dash v1-4 beautiful and then we need to create our pipeline so pipeline equals stable diffusion pipeline Dot from pre-trained it's pretty similar to this table to Fusion API we built for the the python app but now we're going to be doing it in react because we like JavaScript okay so from pre-trained stop talking nick uh model ID and then we're going to say revision equals fp16 so this is loading the the floating Point 16 version which allows to load it into a GPU with a little bit less memory and then torch and the score D type equals torch dot let's uh word wrap portrait d-type equals float 16 and then it's used auth token equals auth token so that's from up here tip place that back down okay that's good and then we want to say pipe two I don't like the word wrap pipe two uh we're gonna send this let's create a new variable for device so if you've got Cuda installed we can send it to our GPU using the device right so that is our model loaded we then need to actually go and make some generations so with AutoCast we're going to send it to our device oh my God type parentheses we're going to send it to our device and then we are going to what are we going to do so we're going to take our prompt so this is going to be the result what are we doing um come here want to know a secret are you looking for your next dream job in data science machine learning deep learning or just data in general well you need to join jobs from net each and every week I send you a curated list of the best jobs in data these are jobs with great perks great people and great roles plus you'll get access to exclusive content like amas interviews and resume reviews so why not join plus it's completely free link is in the description below what are you waiting for all right back to the video so this is actually going to be our image we'll call it result equals pipe Dot dot dot will then get our prompt we then want to pass through guidance scale we'll set that to 8.5 so this is how strict we follow the prompt than to extract the image be run dot image dot image dot image in zero something like that yeah because it's going to be a key all right I think that should be right and then we'll test that out so then let's just run um so this should actually be image so then what we'll do is we'll just save our image so image.save we'll test this out um test image dot PNG and then we'll return just a blank response for now um uh hello out equals hello world we're gonna have to fix that later on but um all right let's start up this app so it's UV corn app I reload just checking we're still on the screen what nine minutes 19. I reckon we can make this guys I reckon we can make this we only have a little bit left to actually go and this was intense I need to catch my breath okay I think that's looking okay so it's starting up right now so this looks promising so we can go and test our API by going to this route here and opening up Swagger UI so then we can go forward slash docs pound symbol if it starts successfully so this should say um app running so we'll get some info from our server all this code is going to be available on GitHub as well by the way guys it takes a little while to start up because it's actually loading the model uh no attribute API not found in module app I think I've written the wrong command app I I've called it app uh all right let's just tweak one thing so we're going to call this rename it to API so that should the file should be called API let's just and then we can run the command that command so then it's going to run which should actually switch this around so this should be API app because basically we're running this app inside of the API file so the command is API colon app dash dash reload okay fingers crossed this looks promising so it's loaded those files [Music] [Music] foreign that is promising we go and reload this boom that looks okay so right now we're not going to be returning an image because we haven't gone and set up the response but if we go to fast API of the Swagger UI open this up and if we type in a prompt over here let me get a little zoomy thing if I hit try it out and then I type in um I don't know Elon Musk buying Twitter and firing everyone and then hit execute this should generate an image if this works so you should see stuff happening down here oh that is looking promising so it takes a little bit of time the first time it starts up guys I think we're in business looks okay attribute pipeline image has no attribute image ah no is it I think it's the images actually hold up let's just try that again I think this might be images not image execute [Music] [Music] all right so we've got everything up until here is working so the fact that it came up basically means that it's not extracting that nicely [Music] okay that worked we've got a 200 response back if I open this up take a look that's our boy Elon trying to buy Twitter and firing everyone okay so that is so we're good we are working successfully so if we go and test it out again um we can write a unicorn riding a dragon through the gates of McDonald's Iris 4K Neon cyberpunk execute boom this works this is looking promising oh that little thing moved that looked okay oh my God there is that image I love you that is that image generated okay but we're not done yet so we've got our API we're looking good we now need to finish this off so we actually return our image because right now we're only returning our response is giving us this out hello world that's not going to be great for our app because it's just returning a random string we actually need to go and return that image back this is where the base64 encoding comes back in okay we're going to kick off the timer again this is intense let's go all right cool so what we now need to do is we need to encode this image so we are going to create a new buffer equals bytes bytes IO and then we're going to go image dot save send it to our buffer and then go format I think it's format equals PNG so that is going to convert it into or it's going to save it to our buffer and then we want to go um image string equals base 64.b64 in code in code and then we are going to send buffer dot get value to that and I think that is okay and then in terms of returning our response we're going to Output response and then I think it's content equals yet that image string and then format it's a format or is it media type that's the one um image or PNG I think that is good for sports I suppose all right let's start this up again so that oh it's already running so it's reloading let's wait for it to finish reloading we are going to need all the time to get this done eight minutes eight minutes to build a react get up guys Lord have mercy this is going to be interesting okay that is reloading right now this looks promising famous last words right oh this is your dad's okay so that looks okay and so it is returning response oh we haven't tested it up let's go and test this out now so if we go and what's our new prompt a um Spartan God going for a job interview at um Google wearing armor 4K high-res Neon cyberpunk there's a theme happening here so if I execute this we should get a little broken image so that for whatever reason Swagger UI doesn't actually return the images I think there's a problem with the actual UI for Swagger UI but if we get an image back then we should be good all right we've got that response buddy so that is looking good right so it's basically saying we're returning our image.png we're turning our image we are looking good guys so if we go and take a look at the image that's been generated by our API well that's our app we'll come back to that in a sec take a look that's our spot and God I don't know what's happening with the green but anyway that is working so we're successfully returning our response back from our API so we are going to leave that and we are going to then go and build our app so this is looking great we've got an app running this looks like it's all working fine and because we've set up the cause middleware should be able to call this from just about wherever we want if we deployed this we'll be able to call it okay we are now going to jump into our app so this is just a react create react app show so we're then going to create a complete app from scratch and we're going to use something called chakra UI which just gives us a nice UI alrighty time is kicking back on let's do this okay so we are going to create an app so I'm now jumping in to draw the script so const app equals uh we're then going to create a empty application all right and then um so that is the Baseline app so we're then going to return uh let's just return um I don't know hello world and then we're just gonna put this inside of H1 tags let's just make sure this is actually we'll call it let's let's do it properly so we're going to call it stable diffusion uh what are we doing there what have we done wrong to return this should not be curly braces should be regular parentheses and then we're going to export default app so if we go and start up this app so npm uh my back blocking it npm start dot is to got this npm start pauses it takes a little bit of time to start up so let's just let this start up okay that looks okay so we've got a couple of warnings that's all right it should tell us that it's raining on like poor okay there it's actually going to open up should just say hello world at the moment in a H1 tag my heart is beating I should get like a heart rate monitor and show this inside of the video okay stable diffusion look how great our app looks okay so it looks crap right now we're gonna make it way better so um let's go Okay so we've at least got our app running so I'm just going to push that over to the side we're going to come back over here and then we are going to make this look significantly better let's get a rocket emoji rocket emoji add this boom already looking better cool got a rocket emoji all right now we're going to use something called charcoal UI so it just makes everything look better so um import so we want chakra provider c-h-a-k-r-a provider from chakra Dash UI and then we need to hunt resolve tracker UI what we are in here that should be okay is it tracker UI oh no oh it's ch at Chakra UI react um at Chuck raw Dash UI let's Dash react sure [Music] I think that should work and then we want to bring in chakra provider over here and we're gonna wrap our entire UI inside of that all right let's see oh no what is happening chakra is not defined get rid of that cool all right so that is looking much better so we've got stable diffusion all right so then we're going to import a bunch of stuff so we are going to import heading we're going to import container let's just grab those for now we'll also import some text so I've got a text file inside of the Python application I know a little bit confusing so we're just going to wrap this inside of a container Tanner this is gonna should Center everything and let's format the document beautiful so okay so that is looking all right we've got stable diffusion over here if we've blow it out it's in the center let's wrap this inside of a heading foreign Fusion app we then need an input so let's grab an input input and then button from up here and we're also going to import wrap which is just going to allow us to wrap this all up together so wrap oh this is intense and then input and then we need a button and the button is going to say generate and then we can actually get add a color theme color scheme and we're going to set that equal to Yellow now if we bring up our app all right that is looking okay for now so uh why are we not in line uh then let's add a width equals 350. DX how's that looking beautiful cool so we've got to generate we've got that's looking all right how are we doing on four minutes oh my God we do not have enough time for this so then from react we're going to need to import we need to import uh axios from axios that that's going to allow us to make API calls we then either import state so from uh no import we need use state from react and that's going to allow us to handle our application state so we need a way to make an API call so we're actually going to create a functions with cons generate is going to take in our prompt it's then it should return make an API call to uh do our API and then it's going to return our image back oh Lord okay so then we are going to say uh axios a CEO stock get and then we're going to go to the endpoint that our app is running at so it's going to be this so we're going to go there and then we actually we should put this inside of back text because we're going to actually append our prompt so it should be um question mark prompt equals and then we'll actually pass through our prompt here and that should be okay and then we are going to make this async and will await the response from there const result is going to equal that so that should be our API call fine so then we're going to we need to manage our state so we're going to hold our image inside of a state variable so we're going to say cons image update image equals use state and then our image is going to be stored inside of that so uh update image then going to say result.data and that should actually return our image back but then we need a way to trigger this so we're going to trigger this on the button click so we are going to say button on click equals e and then uh how are we calling this generate and then we actually need to grab the value from our input so right now we haven't set so we need another use State variable so let's copy that to cons prompt and then we're going to say update prompt so this is going to be our updating function and then over here so our value is going to equal the prompt and then we want to say on change so this is going to be an unchange Handler so where effectively every time we change our input we're going to be updating this prompt value so the way to do that is to go easy so on the the event trigger we're then going to go um update prompt and we're going to say e.target.value okay I think that should be all right and then over here we're then going to pass through the prompt think that should work you know we've got errors oh Lord what's happening const result what's the actual era unexpected token oh we haven't actually passed anything in through through to our generate function so hold up let me let's go over here this should take in a prompt from over here so that should be okay now all right so this is looking great blah blah um a space man running from an alien if I hit generate is this going to trigger our API where's our API this running oh that looked like it ran so you can see it's going a Spaceman running from an alien perfect so that looks like it's actually gone and grabbed the image all right what we then need to do is get that image and return it back so we're going to create an image object oh my God I had such big plans for this I had the ability to go and create a spinner how are we doing on time oh foreign okay so we clearly haven't made that time limit but we didn't have too much left to do so let's let's all right let's finish this up so um if I say I image source equals and then we are going to then assign that the value that we get back from our image over here so it's going to be um what is it now so it's data colon image forward slash PNG semicolon bass 64 comma and then the name of the Stream so we're just going to put this inside of back ticks and then we are going to add dollar sign and then pass through the image name over here and we just want to have a ternary so we're only going to return this back if we've actually got an image so um let's actually create a loading oh we'll go let's actually apply the ternary first so we'll go um image question mark so that's if we've got an image then we're going to return the image back otherwise return nothing I think that looks okay take a look so we're now returning back our alien and we can also do this um think we can add background shadow Brown shadow uh how is how do you how do we uh box Shadow that's it box Shadow equals LG and then we're gonna add a little bit of Shadow cool right so that is our image generating so we can obviously clean this up a little bit and we're also going to add a spinner because I wanted the UI to look a little bit better so that's also toggle word wrap because this is killing me okay so that is the that that's our base App working right so we we can type in stuff so I if I go um uh what's um a space dragon coding and AI model if I then go and hit generate this should trigger our API in Python you can see that's running over there and all things holding equal you can see stuff is happening over here take a look at that space Dragon coding and AI model all right but I want to clean this up a little bit and we this is kind of looking a little bit weird so we can obviously flesh out this this JavaScript application so I've got this text inside of my python app and I'm just going to paste this in so it's literally just information Tech so we know that our API is working now so then let's add in a text object we've already imported it so I'm just going to throw that here text and then I'm going to paste that in so that's looking a little bit nicer so then got a little bit of information about our UI we can also make this a link so rather than having just a um like a random bit of text we can actually add in a link so we can go link at that piece that there Source equals actually should be href equals z this you link over here paste that in and then I'm just going to say GitHub repo now if we go and take a look you can see we've got this link that we can actually go and click we can even add a little bit of padding towards the bottom of our text so let's go and do that so we can say uh margin bottom equals uh I don't know 10 pixels all right so we've now got a little bit of space so that's looking much nicer right you can add a little bit of margin below this text box as well or our input margin bottom equals MPX no why are we opening up Outlook close that bad boy um did we do that right uh let's add it to this wrap there you go all right so we've now got a little bit of space we've got our Dragon coming out the one thing that I did want to generate or show you how to do is to add like a skeleton so as we're loading we know that we're loading we're not just waiting for that image because if I go and write something else right um Twitter Developers building a new UI right so right now you don't know what's generating so we know that our API is running in the background right this all this stuff is happening here we don't the user would not know which I don't personally like as you use your experience so obviously in this particular case it's going in generating stuff but we want to indicate that we're actually generating stuff and we can do that really nicely using a skeleton from charcoal UI so I'm going to import two additional things so I'm going to import skeleton uh actually I want skeleton Circle and skeleton text skeleton cool and then we're just going to create another use State uh instance over here so we're going to use that react hook I'm going to say we're going to determine whether or not we're loading and we're going to create an update function called update loading and so when we make our call to our API we're going to set loading equal to true so I can say update loading and set that equal to true and then after we've returned our image back we're then going to send update loading equals file so I can go update uh loading and I'm going to set that equal to false and then over down here we just need to add in a update our ternary so that we can determine when we're loading versus when we're not loading and if we are loading we're just going to show the um the skeleton back so then we're going to update this let's format this document format make it a little bit cleaner okay so then we're going to say so right now we're saying if there's an image then return the image otherwise no we're going to add in one additional part to our turnery so we're going to say if loading then we are going to return back a skeleton so I'm going to say it's going to be wrapped and then inside of that we're going to say skeleton Circle and skeleton text beautiful and then so we need a colon over here so if we're loading then we're going to return the skeleton if we're not loading and we've got an image then we return the image otherwise we return null and over here we're going to be updating our loading state to true or false depending on what we or where we're at in this generation process all right let's try this out now and uh developer attempting to code something ridiculous in 15 minutes imagine it generates me how they're like hmm interesting all right so wait hold on the circles work but the text hasn't what's happened there skeleton text did I import that correctly yeah all right so that let's try generating all right well you can see that it's at least gone and generated uh or at least we had the skeleton Circle but let's try that again because I would have thought the skeleton text would have popped up um uh Old City by a lake uh high resolution ominous why isn't the skeleton text working weird so the the circle's working so you can see that Circle loading but the skeleton text should have been do we have any errors nope I don't think so all right but you can see that is definitely generating images how cool is that that is actually sick and the cool thing is that you can obviously save these images down if you wanted to so then you've got them saved and you can do whatever you want with them weird I wonder why the skeleton text is not working oh wait no skeleton text oh I think this is meant to be I might have made a bit of an error there or not necessarily an error this should be stack get rid of the oh we still need wrap we need wrap for our input and our image but if I put this inside a stack get rid of this I was testing something else out get rid of this stack let's try generating again um uh formula One driver being cheated of a title there you go we got our skeleton text all right so you can see that obviously looks much nicer now so we actually know when we're generating let's see what it generates take a look at that that's a Formula One driver being cheated on the title anyway we clearly didn't make the uh time frame in this particular case but I really wanted to show you how to do this so either way I did have made the time frame so here's the Amazon gift card first one that manages to get it congratulations there's 50 bucks in your bank account to spend whatever you on whatever you want through Amazon but in this particular case we've gone and built our stable diffusion app as per usual all the code is going to be available on GitHub so I will include the full react application as well so you can see that it is quite a reasonable amount of code but this does allow you to make image generate or generate images via stable diffusion via an API so in a nutshell we went and built a ton of stuff but thanks again for tuning in guys hopefully you enjoyed this one I'll catch you in the next one a peace thanks so much for tuning in guys hopefully you've enjoyed this episode of Code that and uh we've taken a little bit further I am trying to get back into building some more stuff with JavaScript and extending this out to really truly full stack machine learning applications in the future let me know what you thought of it hopefully you uh test it out thanks again for tuning in guys peace check out how cool this guy is it actually allows you to generate awesome nebula lights take a look boom

Channel: Nicholas Renotte

Views: 49,612

Keywords: machine learning, stable diffusion, react, javascript

Id: 3l16wCsDglU

Length: 34min 49sec (2089 seconds)

Published: Fri Nov 04 2022