My Favorite AI Tutorial Yet: Generate Images from Scribbles (Next.js, Tailwind, Convex, Replicate)

all right I have a really interesting tutorial video for you all today where we're going to be trying to build out a generative AI web application that allows the user to basically draw a picture for example if I want to draw a picture of you know a chicken okay let's just try our best to draw a chicken this is what our chicken looks like let's give it a wing to give it some eyes so you can draw a chicken and then over here you can say a happy chicken running in a field of sunflowers and I'm gonna go down here and I'm gonna click submit a couple times let's just do this like eight times one two three four five six seven eight and behind the scenes that is kicking off convex mutations and actions and it's going to invoke the replicate API behind the scenes to basically take the image that I drew take the prompt and it's going to generate AIS for all of these different prompts so as you can tell we got some chickens coming back with a field of sunflowers if uh if we want to change it up a little bit let's try something else let's do um I don't know a kid sliding down a water slide at a park okay let's just clear this let's see if we can just get this going real quick um I'll draw a little kid here and we're gonna go ahead and put some legs on him some arms some eyeballs let's see what happens here I'll do like four of these and hopefully this is uh gonna be pretty good wow okay that works pretty good so what's happening behind the scenes is it's taking this react canvas it's converting it to an image and it's also taking the prompt and we're taking both that image and the prompt we're passing it to the replicate API that is going to generate the image for us and then convex which is what we're using for our backend here is automatically going to update our UI when our data models are updated behind the scenes I think this is a really cool project in a tutorial so if you are interested in seeing how I built this be sure to stick around and I do want to say before I start building out this application this video is sponsored by convex which is a back-end as a service so basically you they give you the tools necessary to quickly build out a back-end and they do a lot of really awesome stuff behind the scenes that make your UI kind of update live as your data models change for example you can build like really live chats or interactive collaborative application using convex definitely really cool I'm actually really impressed with using this so that's what we're going to be using in this video we're going to use convex.dev so go check them out if you want to just find a new way to build out a backend without having to do all the manual database stuff yourself and also we're going to be using replicate to set up the control net models that we're going to be using to basically take a a scribble and have it output an image of whatever the prompt is so without further Ado let's just go ahead and jump into the code and try to build this out together all right let's just go and get started so I always start my tutorials with a repo so I'm going to call this convex replicate how about that and we'll just make this MIT license and we'll make this repo so we can start pushing our changes to repo so that you all can actually see those changes so now that we got our project here I'm gonna go ahead and just clone this I'll click code I'll click copy and I have a vs code terminal open I'm going to go to my workspace and I will go ahead and clone that repo all right and then I'm going to go ahead and just CD into that and replicate and we can start developing on this project so we're going to be using next 13.4.5 or 6 and we're going to be using the app router so let's just do npm create next app at latest at least I think that's command for it okay gonna ask us what version yep that's fine 13.4.6 so if you are watching this video in the future keep in mind the versions of the project and I'm just going to go ahead and do the defaults for everything include Tailwind app router aliases all that should be good all right and then let's just go ahead and run this and make sure that if we go to localhost 3000 we can see our next application running there we go so we've got the next project set up now let's actually read through the convex tutorial and try to set it up to work with our next JS application so if you go to Quick Start they already have one for next JS and they have a cool little toggle here where you can choose the Pages directory or the app router since we are using the app router here we can follow these steps we already did this step so now I'm going to go ahead and just do npm install convex I can just go ahead and do that in another terminal down here so now let's do an MPX convex Dev and that should kind of walk us through a setup a terminal guide where I'm going to say create a new project we're going to use the same name so I've already run this once on my machine but if it's your first time running this I do believe It'll ask you to like verify a a key and then you have to go to the convex site and type that in make sure it's good and once you do you'll see that your project shows up in your convex dashboard so I did have to create a convex account and be logged in before I did all that process but we can actually click into our project now and there's a nice UI that kind of gives you all the information you need about your convex backend so for example when you create new functions they'll show up here logs for all your API endpoints will show up here and I'll walk you through some of this stuff as we're kind of adding new functionality and testing stuff out okay so the next steps let's just keep on going through this guide it's really not not too big so they tell you how to like set up some example data and you can kind of import that data here by using some Json L format but we're not going to be doing that part I just want to kind of set up the convex provider and then we'll come back and reference this this query stuff in a little bit so let's go ahead and make an app context or convex client provider here The Source app and we're going to go ahead and make a convex client provider we'll copy this code like so now this is already set up in my.anv.local I'm going to go ahead and just do a exclamation mark so that typescript knows that it's defined and then in our app layout we could just copy this whole layout I'll just go to my layout page here I'm going to overwrite it okay and basically what this is doing is pulling in that provider that we just created in this other file and this is going to enable all of your children to have access to the convex hooks that we might need for when you need to do queries and mutations and stuff okay and so this one's just kind of showing you how to display the data okay I'm going to go ahead and rename some of these over here I'll say this is convex and this is going to be next okay so you have two terminals running convex is your backend right it is for creating all your backend functionality and your API routes and your database and it kind of needs to run on a separate terminal so that it can listen for when you're changing files in your convex folder and redeploy your functions to the remote service and then next of course is just like the the next app so I'm going to go ahead and commit a lot of what we already done just so that we can kind of not have so many file changes I'm going to say initial setup with next and convex and I will add all these changes and I will commit them to a repo so let's kind of play around with convex a little bit and try to understand like what is the purposes like why would we use this over using next API routes or anything like that so the first thing you can do is in your convex folder you can create new files and those files basically map to namespaces for your function so if I make a new one so for example what we're generating in this app is AI images right so when a user fills out a sketch and then tries to generate an AI image from that sketch we should probably track that information somehow so I'm just going to go ahead and say like I'll just go and say like sketches and then I will put TS after that so now you'll notice that convex is actually checking that we just added a folder and now when you start adding and changing this file it's you'll see in the dashboard it'll start adding functions here that you can actually hit like these are live functions that are updated as you save files and you can invoke them from your UI all right so let's try to make our first function so like let's let's pretend that we want to allow someone to save a sketch so I'm going to go ahead and say export const save sketch equals and then I'm going to go ahead and say mutation all right so the first time you're writing your first function in convex the mutation method might not show up for you so you might have to manually import it you can just say import mutation from dot slash generated slash server and that'll have access to that function now and this this is kind of similar to trpc where you have like queries and mutations mutations are for when you need to modify something on your back end like you need to write data to a database in a queries from when you want to fetch data from your database like you know fetch a list of to do items and show those in your UI so in our case we're going to go ahead and create a mutation and this thing takes in a callback like this and inside the Callback the first parameter so this gives you access to a database right out of the box and you can actually start writing stuff and storing stuff inside the database now the second argument is things that are going to be passed in from the front end so in our case what would a front-end user want to save maybe the prompt or something like that so we're just going to go ahead and put a prompt here like this and we're just going to go ahead and put a console right here let's make sure this is working because we kind of typed a lot of code it might not make a lot of sense until you actually see it being connected to the front end but notice that I save this file and if we go back to our convex UI our function is actually right there okay so we can actually click on the function and we can actually invoke it manually here so I'm going to say run function and see what happens and we can pass in a prompt if we wanted to I could say like hello and click run mutation and that's actually going to invoke the mutation do the logic and return you whatever response you can go look at the logs here so like I mentioned every time you save the file convicts is going to recompile and redeploy your code and that is actually accessible on a live environment right so it's deployed somewhere in this case this is our Dev environment but you can switch it to production at some point so how do we actually hit this endpoint from the front end right it's like cool we have a function that's deployed somewhere but how do we actually like call it from the UI and do something so what I'm going to do is I'm going to minimize some of this stuff and we're going to go to our page here let's clear out most of this stuff in the page and we're going to try to just do some logic where if you click on a button or fill out a form it'll send a request to that back and endpoint so I personally like to use react hook forms so I'm going to import that so we can actually start generating a form to send some data over to the back end so I'm going to go to react hook forms and then going to the getting started guide I'll copy this just go ahead and install this this is a library I kind of use for all my form requests it just makes dealing with forms a little bit easier if you're on the front end or using like a client component technically you could do like a server action but again I'm not doing I'm not going to teach you guys about Alpha things in my videos so let's just go ahead and grab this example form here and we're going to just paste it in here and then we're also going to copy some of this logic put it up here and then we're going to grab this import and put it up there okay so what react hook form does is basically it gives you a bunch of helper functions you can call on your form so that when you're feeling in information you can do like validation on it and when you submit you can have it auto validate and show errors if there's any errors in our case we're going to get rid of some of these comments here and we just want to use a prompt so I'm going to go ahead and register on this input we have an input here that technically we don't need a default value so I'm going to delete that but we're going to register the name of prompt onto this input and we're going to make that thing required so I'm going to go ahead and also put required here as this parameter I'm going to delete the second input because we don't really need two and then here I'm going to go ahead and say prompt and this is going to basically show this as an error if you forget to fill in the prompt then we finally have a an input button that submits the form here and then technically we can get the form data and we could just have a callback happening right there so I'm gonna go ahead and just move some of this logic here so we have like typescript baked in for us already okay does that make sense all we're doing is creating a form and listening for some inputs and getting that information notice also we can type the form here so if I were to go ahead and just say like um prompt is a string like that adding types to the form actually gives you the types when you submit so let's just try this out so when we submit this form we should see a console log printout in our front end and it should print out whatever we have on the form itself in our case let's refresh the page um I forgot to add use client here so on the page itself we are just going to say use client technically you could like do some you know server-side rendering here or do like react server components I'm just going to make a client page just because it's easier for right now and now we have the form showing up here all right so notice that when we type like the text is is white right so we probably want to add some styling to this I'll say class name text black so that the input actually has some black text in it now I can type in here now the thing I'm trying to show you is we want to verify does the form work so I'm going to just say like hello world click submit and notice that we get that prompt information down here I'm also going to verify that the the required field works and it looks like it does I try to submit without filling this in the validation will show up we're going to clean this up and make it look pretty in just a little bit but I want to kind of make sure that we can connect to the convicts mutation that we just created and verify this all works so when someone submits the form how do we call that function that we just created because remember we have in convex we have this function that we can actually invoke but how do we invoke it from the front end it's actually super easy and if you watch any of my other videos with like trpc it's a very similar approach where you just have to import a use mutation hook so I'm going to go ahead and say const um save sketch mutation is equal to use mutation and I'll just go ahead and Auto Import that from that generated folder and inside of this you can actually do a string and I can actually see that it auto completes or that mutation that we just created okay so now if you want to send that data over to the back end over to convex and have it process that form data you can just invoke it like this I'll just pass in some form data so let's let's verify this works I'm going to go to my app I'm gonna go ahead and type in some information I'll click submit and notice that down here there is a blue log so convex is actually going to log out as you make requests and this is the logs that are happening on your API endpoint so if we go back to our convex dashboard and go to logs you'll see down here that it prints out the form that we inputted so we are actually sending data over to the back end and it's being processed and we're going to use this in the the future all replicate to generate some AI images using this prompt and also using like a sketch pad just to make sure that this all makes sense if we go back to this function here let's say you want to do something else like let's say you wanted to return a message that says success that to the UI once this this stuff finishes processing and I'll do some stuff we can do it like this so now if we were to save this file again Comics is going to check that you just saved it's going to redeploy those changes now if I go back to my page let's just go ahead right here I'm going to convert this to an async function and I'm going to put a honest results is equal to await this now I'm going to go ahead and just console log the results here just to show that we are going to get some data back after we call this mutation so it's no different from doing like a post request to your API endpoints right so just go ahead and clear that I'll say hello world click submit notice that we get back a message that says success okay so now what we want to do is we want to actually try to store something in the database right right now we're just console logging stuff which isn't you know that useful but we can use convex to store data into the database so I can say DB dot insert and then you can actually insert like just insert into whatever table you want so in our case we'll say sketches and we're going to go ahead and insert a value so in our case we will insert the prompt here we'll just say the insert the prompt make sure we're doing a weight on this and I'm going to go ahead and put an async here so we can wait on it so some things to point out is that when you do mutations they're acid compliant so everything that you do like if you were to write to different tables and do multiple things all of them have to work or it's going to basically roll back on all the mutations that you did which is pretty useful because often you need transactions when you're like dealing with databases and this is just a nice feature that it does for you out of the box second thing I want to point out is that we haven't even generated a schema at all right this is just a document store and it's using the sketches name for a table and it's going to store whatever data that we want so now if I were to go back to our UI and type in hello world and click submit that is going to run our convex function and then it'll create a table for us so notice that there's a sketches table that I got created and our data is automatically added here which is pretty cool so we've gone full circle we have the UI showing a form we input something we submit it that invokes a convex function which is going to store the data inside a table that's Auto created for us and then as you like where to add more things to the data that comes in it's just going to expand this table and add more things so for example if I want to say like created that and say date.now or something next time I do request the endpoint you should see another column called created although there already is a creation time so I don't really need to do that but let's just show you an example but now it's there we got a created at and we have an ID here all right so the next thing is we have the ability to store data in the database now how do we actually get this data back so we can like display it on a page right so after you've generated a bunch of AI images maybe you want to keep track of a collection of these sketches so you can go back in time and view images that you've created so let's try to do that I'm going to go ahead and make a new page here I'll call it collection and I'm going to make a page.tsx in this directory and we're going to go ahead and try to copy some of the approaches that we took for another our main page here let's go ahead and paste this in and really we don't need a form here so I can kind of delete a majority of all this code and we don't need a form um for that so the difference is now we need to do a query we need to fetch data from an endpoint that's going to fetch from the database and send that back to us right so we need to First go and create a endpoint so if we go to convex and go to our sketches file we can make like a new query endpoint or function I guess you can say so I'm going to say git sketches equals query make sure you import that and then I'm going to go ahead and kind of copy some of the same logic because it's the same signature in a sense but we don't need to pass in any type of parameters in that case so we have access to I make this a constant instead so instead of doing the insert we're going to say DB query and we're going to put the name of the table which is sketches and then we can also say dot and we can start doing a bunch of other methods here right so in our case we want to just grab everything and a query is just going to grab everything by default I believe so what we could do is just say collect and I think that'll give us back all the data so I'm just going to say const sketches equal is equal to this and then I'm going to return sketches that makes sense so basically getting all the data from the sketches table returning it back from our API endpoint and now since we have a query here and since we also saved the file remember go back to convex and you'll see that we have sketches we have the ability to get the sketches and save the sketches let's just go here and run this function real quick you see here there's a run function button let's just go ahead and click this and we are going to go ahead and just try to run this function with no parameters and see what happens and actually it gave us back the results here so we got back the information that's in our database and uh that should be good so we know the endpoint works and we tested it on this console let's actually hook it up to our UI so let's go back to our page here and instead of saying use mutation we're going to say use Query and again we could just go ahead and make sure we get that intellisense working let's just go ahead and I think that's the wrong input import let's just go ahead and Auto Import that from here and there we go our query is auto completed there and it shows up so some of the trpc this does the query and this is going to give us back all the sketches that are in our backend so we can just go ahead and loop over them and we can just go ahead and do a map on this and I do believe this could potentially be null so I'm going to go ahead and just do question mark Dot and then I'm going to say sketch and then I'm going to render out a bunch of different sketches so in our case we'll say sketch dot prompt and I think we also have an ID that we can use so I'll say key is equal to sketch Dot uh I don't know what it is I think it's underscore ID okay so now let's try to go to our new page we made to also go to collections here and if this works we should see our information get displayed cool all right so there you have it we have two functions one's for saving a sketch one's for getting back all the sketches in our database and I want to point out something else that's really cool about convex is everything when you update um data in the table everything is live right so if I were to kind of open up a new window here I'm going to pull this up to another monitor you guys can't see it but I'm going to go to the main page on that form and I'll just type in like hello again on this other monitor and I'll click submit and notice that hello again pops up automatically this is pretty cool because what convex is doing if you actually go to the network Tab and go to websockets there is a sync websocket that gets connected and it's listening for events when you change your data in your database so you can have multiple people kind of changing the same records at the same time and you can do a live collaboration type of web application where stuff will just get updated as these messages are coming in so right now if I were to clear this out and go back and submit another thing from the other monitor notice that the websocket event comes in and it says that hey like something changed let me go ahead and pull this up so you can see it it says hey there's a modification here to the query that you're kind of using behind the scenes and it kind of passes you all the new data that needs to be displayed so it's pretty cool you can make really live interactive websites you can make like games with very little extra effort on your part because everything every time you do a query it just automatically updates with the latest greatest data but with what we're building um we might not see that too much but we can probably try to utilize that in an example all right before we get too much further I'm going to go ahead and just Commit This and I'll say ability to write in batch sketches okay go ahead and push that up so going back to the home page what is it that we're trying to build here we're trying to build the ability for someone to type in a prompt and then they want to be able to draw some type of sketch and when they submit that prompt and sketch together to the back end the back end is going to use replicate it's going to run it through stable diffusion or some type of machine learning model to generate the image for us that uses both the image and the prompt so the first thing I want to do is I'm going to bring in this Library called react etch canvas um it seems like it's pretty cool it's MIT license but basically if you just install this you can get a nice canvas or you can draw around on stuff it also has a bunch of additional functions you can use to clear the canvas and like export the canvas as an image and that's the key part we want to be able to export the canvas to an image so we can pass it to replicate to have it use that okay so let's just go ahead up here I'm going to MP install that and when that's done okay let's kind of follow this a little bit so if you just go ahead and bring this in here react sketch canvas and we're going to put that probably we could just keep it inside the form I guess we want to keep it uh I'll put it above the submit button okay let's just go and Auto Import that you don't need Styles actually for Styles um I'm actually going to put width of 256 in height 256 here and we're not going to put on there and the stroke color will be black because I do believe the AI needs like black and white backgrounds for it to work properly so let's go back to our app which is this one so now we can actually draw which is pretty cool right very little work I didn't do anything other than install a third-party package but we can draw and also there's ability to export our drawing into a base64 encoded image which we're going to use in a bit okay um I probably want to style this up again let's just put a little bit of space between all these things so on the form itself I'm going to give it a class name of flex Flex column and a gap of two just so that some of the stuff is like spaced out because right now it's kind of there's not much space between these and then down here our button where it says submit so let's just go ahead and give this a class name I don't know a background of blue 400. rounded and that's good enough for like a really basic button okay so the first thing we want to do is like how do we export what we draw so if I were to refresh the page and just like do a smiley face here I want to be able to export that to a base64 image how do I do that well if you look at the library here and scroll down they tell you you can pass a ref like this and then when you have that ref you can actually invoke methods on it I think down here they tell you what those methods are like you can reset the canvas you can clear the canvas you can export the image that's the thing that we want to do so we're going to go ahead up here and we're going to make a ref here I'll say const canvas ref is equal to use ref pass it and all I do believe we can pass this a um a type of data structure I think it's called like react sketch in this ref I believe could be wrong yeah that worked okay so make sure you import use ref and you import this type so now in the form submitted what we're going to do is we're going to say hit the ref get the current value and I want to get the image so I'm going to say export image and we're going to go ahead and just do a jpeg and this is potentially undefined so I think what I'm also going to do is say if there is no ref set I'm just going to return like don't allow people to submit this form if there's no image setup so we can basically call this I do believe this is a promise if you hover over it says promise here so I'm going to go ahead and just say honest image is equal to await this and we're going to console log this out the console log image image okay we don't need to invoke the back end just yet but let's just go ahead and um check this out so going back to the UI I don't think we want that anymore let's go here I'm going to type in a prompt that says like hello world and I'm gonna do a smiley face here and click submit but let's go ahead and get the console loaded up so we can see do we get that base64 image and it is not printing anything so I'm kind of wondering what do we do wrong let's just go ahead and console log here are we getting here at all we are oh okay I know the issues I didn't I didn't pass the rough here so I need to pass in the canvas ref so that it can actually bind to the right react reference I think we should be good so let's click submit and now notice here we get this base64 image and if you command click it we have a JPEG this is the image we're going to send to the AI so they can use this as a reference point for the control net in generating your your image okay so let's go ahead and um change up what we're doing on the back end so they can actually accept an image and maybe even use the image to generate an output okay so I'm going to add that back in and we're going to pass in form data like this but we'll pass in the prompt but then we also want to pass in the base64 image that we just generated okay so obviously we're getting errors because we haven't set up the backend to accept that so let's go and open sketches here and we're going to go ahead and import image as well I'll say that is also a string and technically if you wanted to store the image you can do that you could store it in the database convex has a file storage as well so we could potentially store these images in their file storage which I think I'm going to try doing later on but let's just kind of keep pushing forward and get the AI working because we have the prompt we have the image we're sending out to the back and now we can kind of use replicate to generate these images for us so let's move on to um how do you get replicate going so if you go over here I have a account set up for replicate.com and I'm using this control net scribble model they have like a bunch of different models you can kind of search through but this one is the one I'm kind of you wanting to use right so you can draw an image of a turtle and then this is the output that you'll get when you you know do that and of course you can like add in the prompts and stuff to make it more interesting but let's just go ahead and go to the API in order for this to work you do have to install replicates let's just go back to our app here I'm going to go ahead and install replicate so they do give you a replicate access token that you have to basically put into convex I'm going to go to convex and then down here in settings you can go to your project settings they have a section for environment variables so I went ahead and added it in I didn't want to share that token with you all but it's pretty simple you put the key in the value here and you're done but the way onvex is set up you're not supposed to do API requests from me from a mutation you're instead supposed to be doing that from an action so what we could potentially do is after we insert the sketch here we could actually schedule an action to run which is going to do the AI generation I'll replicate and then when the AI generation replicates done that could kind of update a database entry and that our UI will just kind of refresh so I'm going to go ahead and make another function here I'll say export const generate and then we'll say internal action and we're going to go ahead and just pass in uh we'll keep this empty for right now okay and I'll put it like a to-do Implement me and the thing that we need to pass to the action would be like a prompt obviously I'll just go say prompt and string but then we also probably need to pass in the image as well I'll just go ahead and say image and that could be image but again if mutations are supposed to be deterministic so if you're like invoking third-party libraries from a mutation that's not proper and it won't work the way you think it should with convex so how do you actually invoke this action well you can actually pull in something called a scheduler so I'll say scheduler I'll say scheduler dot run after we can just go ahead and set this to zero because we want this to immediately start executing something but we can give this the name of the function we want to invoke right so the action we want to invoke is called generate and then we're going to pass in prompt and then also image now I do think we want to await on that maybe I'm not sure and we probably should also pass in the sketch ID so like if we were to go ahead and say like sketch ID is equal to this um I shall say sketch we can go ahead and pass in the sketch uh ID okay yeah we have an ID here so I'll just say sketch ID is equal to sketch.id just so that we know like what table in the database we need to update when this thing is done because this could take up to five to ten seconds depending on how low replicate is going to be and I think this is a data type of ring potentially all right so now for the AI part actually let's Okay so let's make sure this actually works if I just print out a console log like Hello World uh GG make sure this thing works and I'm also going to print out all this information that came over just to make sure that that stuff is indeed getting set all right let's try this out so when we invoke this mutation it should at some point call the action the action should get this information and then it should be able to call replicate go back here to our little smiley face click submit and notice that we do get the logs of the sketch ID The Prompt and the image so we do know that the action is getting invoked and I don't know if I said this I'll say it again the purpose of the internal action is that it's private the only thing that can call this is like other backend functions so it is an extra level of protection you still want to make sure you have some type of authentication checks on your public methods but just know that this cannot be called from your front end now how do we generate an image what we want to do is we want to go back to replicate which I think we had here and we're going to go ahead and copy this stuff so I'm going to go ahead and paste all this in here and we also want to paste this in here okay we can put an async in front of this so we can do a weight on it okay so what this is doing is creating a replicate object with our token that we set in the environment of convex and now we're using this model notice this long like string that's the model ID and we need to pass in the image here so what we could potentially do is just pass in the image that we that got sent in and there's some other things we need to pass in like the scale we could set that as like a seven the prompt needs to be passed in the image resolution I'm going to say two or five twelve and then for the imprompt so like the negative prompt it's always good to have negative prompts when you're dealing with AI to make sure that like you don't have low quality images so I'm going to put this in here basically telling AI like make sure you have good looking hands and Anatomy sometimes AI will give like people five extra arms and stuff like that so that's kind of what we're going to do here and when we get the results back this is actually going to be an array and the first index of the array or index 0 is going to be our original input image and index one is going to be the output so what we could do is when this is all done we can invoke another mutation so I'm going to go up here I'm going to say run mutation because we're going to do a database update and what we want to do is we're going to have to go down here and like update an existing thing so I'm going to say const say export const update catch actually I'm going to say I'm going to go here and say export const update sketch result equals and then I will say internal mutation again like no one else should be able to call this this should only be called from this internal action and we're going to go ahead and give this a DB like that and we also probably want to pass in a sketch ID and then probably the image itself so here we could probably just give like the result okay so here we could say await DB dot insert or say DB Dot hatch and we're going to give the sketch ID and then we also want to pass in the uh we'll just say the result okay let's just put async here so this all works and we should probably type this so it actually knows that these are going to be strings and results will also be a string actually I guess it doesn't like this it needs to be ID screen okay so now what we could do is up here we're going to say run mutation and we're going to pass in update sketch result and we're going to pass in sketch ID and then result of one I'll put oh this actually needs to be output of one and this will be result it doesn't like that this is a string so again I'll do ID of string and this one and also typescript is not happy about this so I'll just say as during string these are both URLs to the images the result will be like a URL all right let's see if this actually works I'm gonna go ahead and just clear out my my database a little bit I'm gonna go ahead and just like delete I'm gonna clear the table and then I should be able to go here click submit and we should see some information come up so we have this prompt come up the functions are being invoked all right so we're getting an error that says set timeout is not defined so we actually need to allow this function the generate function to run with the node runtime most of the mutations and queries actually run on a faster runtime that doesn't have node support you can go read through the docs for more fine-grained details like what what I mean by that but if you want to be able to do things like set timeouts and other node related Library function calls you actually need to put a used node at the top of your file so I'm going to just put a generate here and I'm going to cut this code out and put it inside of this file and we're going to go ahead and take a lot of this code out and put it at the other put at the top over here and at the very top of this generate file I'm going to say use node okay so I can save this and now this generate function will have access to the node library and it won't crash next time now the main difference is now that we change that to another file we have to change this to generate colon generate so that it calls the right namespace go ahead and save that all right let's try this one more time go ahead and click submit and we should see okay some stuff is happening let's go to our data and let's verify if uh I'm gonna add some console logs real quick just to make sure this is actually getting invoked and then this one over here I'm gonna go ahead and just say like running replicate I want to make sure that these things are getting called so I am getting air down here I think it's something to do with this patch method I'm calling I think I actually need to pass in I need to convert the string of the ID into a natural like ID class that I can actually like process it correctly all right let's try this one more time go ahead and click submit okay so run generate is getting called running replicate is getting called hopefully this works and there's no errors if this does work we should see the data get updated with a URL to the generated image so let's go back over here we're not seeing that here let's just go and print out what this is I'll just say ID and print out the ID and I think I forgot to wait on this too maybe that's part of the issue okay that was the issue I wasn't awaiting on a promise and now I think if I go to data I should see some results here so let's just go ahead and see if we can like open this up and see there it is that's what my smiley face generated for us for some reason um so let's see if we can kind of get this thing displayed in the UI so after you submit it we should probably have convex return like the ID for the entry so over here we're getting the results back save sketch mutation is getting called up here if sketch it returns okay right now it's returning message success but we actually want to have it just return probably the sketch let's do that we'll have that return the sketch which means we'll have access to the ID which means I think what we can do up here is we can actually like use a query to get live updates from the sketch is updated so we're going to go back to convex and we'll make another endpoint we'll say const get sketch and that's going to be a query fixing the DB and we're just going to basically return in await db.query and we do want to pass in like a sketch ID here all right so now for our git sketch we're going to say return db.get and I believe we can just pass in the sketch ID here right I think I might need to actually say new ID is equal to sketches ID like that and I think that'll just grab the one individual record and send it back to us so let's go back to our UI and see if we can get like the Edge query going so let's say use Query we'll say sketches get sketch I believe this might need a sketch ID which we're going to have to dynamically change this so I think we're going to need some State here so I'll say like const sketch ID set sketch ID equals you state and we're going to go ahead and just set this as a an empty string for right now all right let's just try updating this so after we get the results back I'm just going to go ahead and say results Dot ID and that'll store the sketch ID and hopefully that'll kick off the query and get some data back for us and if I understand this correctly we should be able to display the image somewhere so I'm going to say image source is equal to that dot result like that and then I only want to display this if this is like that I'll just do this okay this is throwing an error I think it's because our backend like needs to just not throw an error if ID is not turned in if ID is not sent in so I'm going to go ahead and say if there is no sketch ID I'm just going to return null let's try this again I'll do a smiling face I'll click this I'll click submit and we'll see what happens wow creepy but it kind of works let's just style this up a little bit um so that we can have the form and the input let's have the results like outside of the form like this and I'm gonna go ahead and just say this will be a div last name is grid grid false two gap of four we're gonna wrap this whole thing there we go now the image is kind of big I might make the image a little bit smaller I'll say like width is 256. I 256. all right let's just do a refresh and see what happens so let's try a bird flying in the sky and then I want the bird to be right here and we'll do like a mountain [Music] this maybe we'll do a beach this would be like a beach with a palm tree here on the beach let's see what happens if you do this cool looks very futuristic um I'm gonna say Illustrated cartoonish and we'll try it again maybe we'll put like a sun over here in the back maybe a little ship a little sailboat let's see what happens if we do this beautiful we got our little banana sailboat there with an umbrella we got it looks like a shell and we have the uh the palm tree over here I don't have a way to clear the canvas let's just restart refresh the page let's do one more I'll say a panda eating bamboo okay and we'll have like some bamboo shoots over here okay we'll have like a panda here he's just sitting right there on the grab make his arms is it ears that's it this this is a bad picture get some eyes right there and he's eating bamboo okay let's see what happens if we do that one so I mean it tried I mean it got this stick in the right location and uh it mistaked its head for looks like some wheat and we got some bamboo over here so pretty cool and I guess the main thing I want to emphasize is it kind of you have to kind of wrap your head around how convex is going to automatically update your queries so like you don't have to have a long polling or any type of like interval to kind of keep on checking like hey is this stuff done you just kind of have to in your back and modify a row and then your front end is already doing a query on that row by an ID and it'll get that data and it'll update it which is why this image will show after some amount of time because on the back end we told convex to update a database row and put the result in that row and that causes it to basically show the image the moment it's ready so there is a little bit more I want to do on the application in terms of functionality but before we start doing that I do want to bring in a component Library to help style this app a little bit because right now it's just looking pretty basic so I'm going to be using something called Chad cn.com it's kind of like a it's not really a component Library it just has a collection of components you can install but we're going to go ahead and just follow the docs here and get this set up so we already have a next app created I'm gonna go ahead and run this go ahead and do this and we're going to go ahead go to our project here let's just go ahead and run this I do believe this is going to like kind of modify your Tailwind CSS file to include some of the defaults that this Library likes to use I'll just say yes you can go ahead and overwrite those defaults and then what this allows you to do is they have a bunch of components over here and you can just kind of add them as you decide that you need them so for example if you want a card you can go over here and you can run one command and then you have a component that's kind of copied into your components directory which allows you to have full control and flexibility over that card component because it's in your code base so you can kind of modify it and style it the way that you want to do it they also have a lot of like code examples here how like you could build this like a card you go to code and it kind of shows you um and then also if you go to examples all of this is in code so you can kind of see like how they did all this stuff here so let's start with like making like a little nav bar so I'm going to go ahead and view the code over here and try to figure out how is the nav bar created and I think it's as simple as just adding like a border bottom here I'm going to grab this and what we're going to do is we're going to go to our components let's go to app I'm just going to add one called navbar ESX this will be a simple component that would just say export function nav bar and we're going to return that code inside the nav bar we could add maybe a next link that users can go to their collections so let's just go ahead and add a next link and we're going to say href is equal to selection like that make sure I end that off and I'm gonna go and say collection here and this is a client component so let's make sure you say use client and let's also figure out why this is complaining there's no closing tag oh I put the Slash the wrong location all right so now we have a nav bar and we have the Shad cnui setup so we can go to our layout here and let's just go right above the children and we're going to put our nav bar go ahead and say nav bar like this we're going to Auto Import this okay so now if we go back to our app um we didn't give it any height so I think we do need to like at least put something in it um actually let me do a hard refresh and make sure this is working so I do see collections popping up over here and I think by default this is in light mode so I'm just going to hard code this to be dark mode for right now but there is a library you can bring in to make this easier to switch like a dark light mode toggle but we're not going to be doing that uh in this video Let's see what is going on here let's go to Tailwind oh I think because I have my components nested in the app directory we do need to put a prefix of source inside of this so it knows how to find all these things and there we have it we have the Chad CN UI working pretty good we got a link here that takes us to collection yeah so let's work a little bit more on the nav bar the inside of here I'm going to add a container I'm going to say MX Auto so that should kind of like Center this stuff before it was like at the very far Edge and it wasn't that great and then we could probably give this some some height are some padding just so there's like some extra space in the nav bar okay and let's just go ahead and Center this I'll just say text Center right now and that's good enough so if we decided that we want to make this an actual like product we might want to actually add a logo like over here we'd have like a div that says logo and then over here we have some links and then over here we have like a sign in button or something okay now if you wanted to do that what you could do is you can actually give this a Lex and I can say justify I think it's justify between okay so that'll push all those items um aside let me just hide my head so you can see that real quick okay so we got a sign in button that we're not going to use at all but it's there so that at least makes the app look a little bit nicer we could kind of style this form a little bit I do believe the component library has some built-in components for like inputs and forms let's go and check out input down here and let's see how this works so basically you just need to add in this input I'm going to copy this command I'm gonna go back to our terminal I'm going to type that in press Tab and I'm going to actually prefix this with Source slash because that is where our because that is where our code is actually set up okay so now that we've done that we have if you look over here this is the first time we've added a component from the shed C and UI so there's an input here if you look at it you have all the code that's needed for a styled input the pops props are already set up they're being passed in the children are set up they're being passed in stuff like that so if you wanted to use this in an existing page let's go to our main page and we're going to go down a little bit to where we have the input here here it is I'm going to change this to a capital I input and make sure I import this from the component that we just set up and for the class name it's already styled so we don't need to like pass a class but you do have the ability to overwrite Styles if needed now we got an input here okay um we could probably put a label on this so let's again let's go back to here looks like they have a label okay and let's go ahead and copy the label as well all right so now we should have the ability to bring in a label I'm going to look at the example here that is kind of paste in the label like this Auto Import that from the components Library and I'm going to go ahead and just say prompt okay and this will be for prompt all right so if this works correctly we should be able to click on the prompt and it was supposed to be able to highlight this so it's not highlighting this when you click on the label I think we do need to add an ID here and just give it prompt so that it'll highlight and select this I guess I should say it should focus the input when you click this cool now maybe we should add a label to the canvas as well just so it's more apparent with this white thing is that's like has nothing so I'll say like canvas you're all something below so people know like what what is this right okay that looks pretty good now maybe we could add a little bit of margin top to the label a class name margin top four or two just to kind of Base that a little bit it might be cool to revert the colors so that the background is white in the the actual color of the drawing is black I don't know if that's even possible to be honest let's see if there is a background color because if there's not um yeah I don't think there is so you know we won't we don't need to worry about that too much so the button here the button doesn't really match the theme so let's go back to here and let's see if there's a button which there should be so let's just go here I got a button let's just add it in over here and put source there we go all right and then let's find that submit submit button let's just import the button like that we're gonna go ahead and say submit save that and get rid of the class name on this here we go so now we should have a button on our app that says submit so it looks a little bit better right I mean it's better than before it's not not too awesome but it works now we can at least go to our Collections and also we should probably have another link here that says like generate um so let's go back to the nav bar I'll add another link here now again this is a space between so we do need to kind of group the links to here I think we can call this nav and we could have two links here so this one could be home or let's call it generate and then we should probably space out this a little bit so I'll say Flex gap for some space between these links um now I don't think these things are links for some reason I don't know we could probably style them to make them look like links but for right now it's probably good enough okay so we can kind of switch between our app awesome the one thing I want to do is like the power of convex again is like everything is super reactive and we have that websocket that's getting Live Events so to demonstrate that a little bit more what we could do is let the user be able to generate multiple prompts at the same time and have those kind of load in as they finish so let's do a little bit of refactoring on the actual page so I'll just go to app age and what we're going to do is when we submit this form right now we're like setting a sketch ID but what we could do is we could potentially have this be an array of sketch IDs and we could just kind of push those into an array and then display that array down below and you'll see them kind of pop up when they're finished and we can also add in a loader State as well so let's change this to an array real quick I'll change this to sketch IDs and let's actually do a query on git sketches here okay so that'll just get all the sketches um actually what do we call this it was git yeah sketches and this should be the all the sketches that are in the system we could probably change this to like make it only return a subset or we could have it so like there's a batch number so that only Returns the ones that we're creating but let's just go ahead and try to change this up a little bit so if we already did the query for the sketches here we just have to render them out so I'm going to go ahead and delete that code for like keeping track of a sketch ID you don't care about the results here and what we're going to do is we're going to say sketches query and that is going to be an array if it's defined so I'll do a map and we'll say sketch and for every sketch that we have we're going to render out an image okay so this is complaining because it's expecting it to be like an image prop but it's it's fine so now we have every image that we've ever generated in the system down below and what we could potentially do is change this UI up a little bit so I'm going to put a label here or a title that says recent sketches so I'll say like H2 recent sketches let's putting it over here for some reason um and we're going to move all this we're going to move this down below the form you'll put it here we'll save this the grid Gap stuff I think could potentially just go away for right now um little div could probably go away because it's already in a form and let's wrap this in a div that's a grid grid columns four Gap four but every image is basically in a a grid system down here and as we generate images we can see them kind of pop up over here so um I don't like how there's so much space here so let's go above the prompt and let's just go ahead and say Minh screen padding we don't want padding on this let's go over the padding maybe padding top of four would be good enough does that seem okay now I think about it maybe it'd be better if it is on the left and we have all these on the right so I'm changing my mind a little bit which is fine um we just need to put this whole thing on the right and this whole thing on the left okay so to achieve that we could give this a grid we could say grid Columns of two again gap of four we're gonna wrap this in a div and then this whole thing will be a section like so all right and then this thing should probably be in the container as well so let's go back and let's put this whole thing in a container and MX Auto it just so it brings it in a little bit that looks better and then also this so now that I'm looking at this like this form has a lot of extra space like we don't need all the space it really just needs like to be one-fourth of the page so instead of doing grids one approach is you can actually say Flex and then on the actual form itself we could give this a width of 1 4. okay and that'll kind of push it to the side and then the grid over here will be the remainder of the page yeah so this looks a little bit better maybe you guys can tell me but let's just go ahead and see what happens if we were to just clear out the database all right so we have all our sketches here I'm just going to clear them out we're going to start fresh okay so we have no sketches here I'm going to go ahead and say a running chicken and let's see if we can draw a little chicken with the beak maybe a body here some little legs cool let's see click submit and what I wanted to show you is if this thing is actually working correctly okay so we can actually submit multiples of these and this is what I'm going to show you is behind the scenes since we use the convex action and we do the scheduler these things are going to be updated live as the data comes in right we didn't have to add any extra logic in our front end to like worry about that which is really cool um we might want to sort these by the creation date descending because right now it's just going to keep I'm going to pinning them to the end so maybe we can kind of do that so for the sketches query we can probably go over here and make a derive so I can say they cons sorted sketches is equal to the sketches query we're going to default it to an empty array because I do believe this can be null in some cases and we're going to say sort and we're going to sort it by um I don't know what it's called I think it's called like created time or something so if you look here it's underscore creation time and that looks like it's a number so we should be able to say a DOT creation time minus B dot creation time and we can use that sort of sketches down here like this so let's let's make sure this actually works so if I were to submit a new one still adding it to the end so I think what we want to do is we just want to swap the sort so maybe this should be B minus a um I think we did forget to add a key here so I think we should go over here to where we're doing the map and we need to add a key to this so I'll say key etch dot underscore ID I'll say an Angry Chicken let's see cool so I mean this app is looking pretty nice we have the way to go and see all of our collections well that's pretty cool we'll come back and I'm going to add a reset to this canvas because right now it's kind of a pain that you can't reset it but on the collections page we're gonna probably do something very similar um so let's go back we're going to copy all this code let's go to our collection page and we already have this save mutation going on so we already have the sketches it's just a matter of like can we render out what we need to render out so I'm going to paste this in and really we could just go ahead and copy I think we could kind of do this I think potentially that could be null so let's make sure that's set and um yeah let's go here we got all of our recent sketches now these are going to be again in the wrong chronological order so we want to go back to this page and probably find the sort we can pull that into here like this and that should be fine let's just do this so it's a lot of duplicate code between the two pages um but maybe what we do on this page is just limit it by like 10 or 20 results or something okay so the last thing I want to add in is like the ability to clear this canvas so let's go back to the main page and let's find the canvas ref and I think we could probably just add a button that says like clear I'll just go here I'm going to say type of button here and we're going to call this a clear button and maybe we can give this an ALT or a variant so we have like ghosts we have other things let's try let's try ghost that looks pretty good okay so clear and submit clearing is supposed to call a function that's on that ref and we're going to just go ahead and say unclick we're going to go ahead and just invoke a function here and we're going to say canvasref dot current dot clear canvas okay so now if I were to draw something and click clear it gets cleared up okay so I think that might be as far as I'm going to take the functionality but I do want to talk a little bit more about convex because there's some things that I did following their philosophy of how data models should be mapped so there's a couple more things I do want to talk about with convex and maybe we can Implement those so you guys have a better understanding of like schemas and better practices for convex if you go and read their schemas page they basically say we recommend beginning your project without a schema and that's kind of what we just did in this whole video it allows for rapid prototyping and then you can add a schema once you've solidified your plan I think our application is pretty solid so I think we can come back and add a schema to our convex model so what this allows us to do is we can make a schema.ts file inside this convex folder okay and we're going to go ahead and copy some of this and we only have a single table I guess it's called sketch I think it's called sketches um this name that you use here needs to match the name that you used here right so sketches is our table name so we're going to call it sketches and the things that we care about what like what are the the pieces of data that are on that sketches record right we are storing The Prompt in the image I think the ID is already there by default but we just need to store The Prompt in the image so we're going to say prompt is a string then we have image which is also a string so the reason we're adding this is I don't know if you saw earlier like sometimes when I was doing the intellisense autocorrect for like typescript we weren't getting information about like what this stuff is that's coming back but now if you hover over this it knows what should exist on this data model so when we do get sketches from the front end okay the sketch query knows that this is an array of some type before this wasn't there like if I were to delete if I were to delete my schema let's just copy this I'm going to show you if I delete that schema notice that this says any or undefined okay which is not the best developer experience right so this is why at some point you probably want to add a schema dot PS and make sure that so your project knows like what property should exist on your data models I definitely recommend going through and read through these schemas because they do have a lot of information about like the things you can add as data types and stuff like that and how you could probably do joins potentially if you go to Advanced I think it tells you how to do joins another thing I'll point out which I did mention on is that mutations run as transactionally okay so I would read through this stuff every page here has like good information about like what these different things are and it gets a little bit more advanced um like if you don't know what a transaction is basically a way to have a collection of Rights onto your database that it's an all or nothing so every right has to pass or else it'll just roll back and not apply those changes super important when you're dealing with systems that need to deal with like updating money on like a user's balance or you have multiple parts of your system trying to modify the same record his actions become super important to kind of Ensure data integrity and you have consistency in your system so another thing I didn't really do in this tutorial is argument validation so whenever you're doing a mutation or action or query you can actually put validation in front of this so that the data that comes in across the wire is actually validated so we can actually add this in it's not too hard to do let's go to any of our existing mutations so I'm gonna go to the sketches file and let's find where we do a mutation okay so remember we allow the user to send in a prompt and an image and we just say that these are straight type of strings this doesn't actually validate the data right so if someone sends in a number or sends in a Boolean it's not going to validate anything um so what we need to do is on the mutation itself we can add in an object here and this could be a Handler like this and we could say I'm going to import this V thing because that's what we'll need but then we say args like that let me make sure that this is all set up I'm going to Define prompt and that's going to be a V Dot string and then I'm going to say image and that's also a VDOT stream okay so now if I were to try to invoke this save sketch with some bogus data it's going to validate before it even gets to the Handler which is nice let's go to our main page and see if we can do that so I'm going to say save sketch mutation and figure out where do we call this okay we call it down here so let's instead of image being an actual string let's just pass it a number and see what happens okay let's go back to let's go back to our app and let's just go ahead and type some random stuff in and click submit so we are getting some typescript errors I think it's because I made this a string that isn't optional so we have to go back and figure out how do we make stuff optional so let's go here and let's see um V optional and then you pass it this I'll say V optional because the image could potentially be undefined or null because it hasn't been created yet and I think another thing I might have messed up on is I think it's called result not image so I think I'm going to change this to result hopefully that'll fix my typescript errors looks like it's working now okay so now notice here like if we say image false and pass it a Boolean it's complaining because that's not what it's supposed to be if I say result false again that's not what it's supposed to be so it's kind of giving us that intellisense and I think we can also submit it with bad data and we do get back an error saying that hey like you're passing a Boolean but we expected a string so that's kind of like the benefit of using the argument checking and argument validation kind of similar to using Zod if you've done that or trpc where you actually like validate your arguments so definitely useful if you have a production system you probably want to validate everything that comes in and then also since we do have the argument validation going on we don't even need to type these anymore because I think it'll be smart enough to know by doing inference that these things are strings so these will also be strings it just makes your code a little bit cleaner and easier to manage all right I'm going to do a final commit and I'll just say um showing all collections on generate page all right so I think that's all I kind of want to show you with using convex and using the replicate API to generate some AI images again these images don't look the best you can play around with the prompt and try to get it more descriptive you can make better drawings to give the AI more inference as to what it should kind of draw but overall it's pretty cool it's pretty simple to make this application and I do recommend go check out convex play around with it a little bit I do find it actually a really nice developer experience and it's really easy to build out completely live updating applications with very very little work I'm actually kind of impressed and I might personally try to start playing around this a little bit more outside of this sponsor video because I do think this is a really efficient way to build out a entire application without ever having to build your own like back in from scratch also the code for this video is on this convex replicate repo you can go and check it out if you want it'll be in the description of the video alright so that is about all I want to show if you do have any other comments or suggestions about this Little Feel free to leave a comment below or you can join my Discord where you can talk to me directly or just find a place to talk with other developers I have a Discord in the description below other than that have a good day and happy coding

Channel: Web Dev Cody

Views: 6,837

Keywords: web development, programming, coding, code, learn to code, tutorial, software engineering

Id: 5cGdBDXqCJ0

Length: 75min 58sec (4558 seconds)

Published: Tue Jun 20 2023