How to code an AI powered Text Adventure Game (Next.js, Convex, OpenAI, DALL-E)

how's it going everyone so we just kicked off the hackathon today and with the kickoff I want to give you some type of longer form tutorial video that anyone who just needs some extra help understanding how convex works or understanding maybe how you can use AI to potentially submit to this hackathon I'm hoping that this video will give you some inspiration and also just kind of teach you some things about connecting with openai and just using combat so let's walk you through what I built and then you can watch through the whole tutorial and look at the code that I will have in the public git repo that you can use at any time as a reference point when you're building your own hackathon submission so here's the walkthrough we got a little project here where it's a text-based RPG game right and the dungeon master is open AI the dungeon mass is going to help guide you through this adventure and you're going to answer questions that I might ask you it'll throw enemies at you it'll give you items we're going to go ahead and start this adventure as the warrior but I have a wizard a warrior and an Archer I'm gonna go ahead and say the Warrior and I'll say start in adventure so when this game first initializes we are asking open AI to basically create a text based RPG adventure and I kind of give it some rules about like what's going to happen I'm going to have it ask us questions and we have to answer with what we want to do next on this adventure right so here is the the dialogue we're having with the AI here and it's basically asking us to start of Adventure where we're in an undead dungeon with enemies and other things and one thing I want to show you is that all of the items that the open AI decides to give you I'm actually using Dolly under the hood to generate icons for those right so you saw those things loading when Dolly finishes loading those previews I'm actually displaying them here okay and also we have this image here that's going to kind of paint a picture of what the current scene is in our adventure and then also we have the health now the cool thing about this entire thing on the right is I ask open AI to give me a Json format of my current inventory and my current health points and I take that Json format and I use it to display all the data okay so let's just go ahead and start exploring the dungeon okay go ahead and click submit and I will say that this is more of like a prototype I spent two to three hours working on this um if I had more time I would spend a ton more time making this perfect and Polished but I just wanted to kind of get something out there for you all so you can learn how to use convex learn how to use open AI you learn how to use ombeck's image storage all these images are being stored in convex's image storage and all these images like I said are being generated from Dolly so I said explore the Dungeon the dungeon master says as you've entered deeper into the dungeon your footsteps Echo ominously throughout the Cold Stone corridors you cautiously explore the twists and turns hoping to encounter your first set of enemies after a few minutes of exploration you come across a dimly lit room there are three skeletons standing guard they spot you preparing to attack roll the dice to determine the outcome of your battle remember the one is the worst outcome and six is the best outcome please enter your dice roll if you have like a physical dice you could just roll it and then you could type it in here but I added this little thing here I found a cool library that basically rolls the dice for us and based on the outcome it will just put that right into the input so I'm go ahead and submit that and let's see I rolled a six so we should probably kill all these skeletons pretty quickly okay it says with the roll of a six your battle against the skeleton goes extremely well you skillfully maneuver around their attacks making precise strikes with your broadsword in a matter of minutes all three skeletons lie defeated at your feet after the battle you search the room and find a small chest hidden behind a loose Stone upon opening it you discover a health potion you add it to your inventory and bring your total account to two health potions what would you like to do next I'll say search the skeletons okay so we found no items with that one I'll say explore the Dungeon some more all right so I guess export some more I found some other enemies or zombies I'm fighting now one thing I noticed with this project is sometimes open AI decides to just roll for you which is not there needs to be a lot of tuning right you have to tune open Ai and do a lot of prompt engineering to make sure it's not like doing dumb things for you unfortunately sometimes this happens from what I've seen where it basically rolls for you and you don't get a roll the dice yourself which kind of stinks but it is what it is we kill the Zombies we got a silver key they dealt two damage to my HP so you'll see here my actual Health actually dropped and for some reason the preview of this this battle is not even showing up that's another thing I'll have to look into after this video but I think this is a pretty cool prototype to introduce you to the category of the best Innovative AI submission I think it does a lot of cool things and it really stresses all the different things you can do with convex in terms of your actions your mutations your queries um and your image storage as well so if you guys are interested in seeing how I built this stick around this tutorial should be like two three hours and I think it has a lot of good information so let's just go ahead and start diving into the code and we will figure out how to build this together all right so like always I'm going to create a new repository where we can kind of keep all this code as I'm working on it so for the repository name I'll say convex and then we will say text Adventure okay I'll go down here just say MIT go ahead and create it now I will say if you are participating in the hackathon if you make a public repository remember do not commit your keys to the repo because everyone can get your keys and if you're integrating with like open AI or some other type of AI software your keys could be exposed so just keep that in mind make sure you ignore your EV files all right so we're going to go ahead and clone this down go ahead and copy this link and then I'm going to go ahead and say git loan and go ahead and grab that repository down and I'm going to go ahead and open up a code Editor to that text Adventure project so going to the convex docs they have some quick start guides that kind of walk you through exactly how you can spin up a new project I'm going to be using next JS but just keep in mind they have like next node python rust let's just click next and let's just follow through this guide real quick so let's just copy this command and I already have a repo set up so we're going to change this to a DOT instead of my app I'll just go ahead and say Dot and that is for creating a next JS application I'm going to go ahead and just use the defaults that it gives us and like I mentioned before the first thing I always do is I go through here and I make sure EnV file is here I'm just going to go ahead and say like just all.env files never commit those okay just in case and now we have an xjs project set up let's just go ahead and make sure that we can run it that's always the first thing I do when I set up a new project is can I access it on the port that they tell us that it should be accessible and it is so it's looking pretty good let's go back to the quick start guide because we do want to continue setting up convicts so I'm going to go ahead and say mpm install convex here go and stop that and I will run npm install convex okay looks like that's done so now we can go down here we can see MPX convex Dev right so that's kind of going to initialize your convex project and it will actually ask you is this a new project and I'm going to say yes now I did already configure my local CLI to work with convex so this is the first time running this you may have to go and like log in and click a link to like have your local setup sync to the convex login but I already did that so I'm going to say create a new project I'll say convex text adventure and at this point we are good to go if you notice that it does have a check box down here it says convex functions ready and it also populated this dot EnV local with two things here this environment file is used for the local running convex so that as you save files convex will automatically deploy them out to their service you can also command click this which will take you to your convex dashboard where you have access to the project that you set up you also have access to your prod and Dev environments in this case we're going to keep it a Dev for the most part and then you also have access to seeing what functions you have to deployed the data that's there any files schedulers logs we'll kind of go through some of this stuff as we need it um but this dashboard is very useful when you're needing to debug or just verify things are getting deployed correctly but at this point you can kind of follow through this if you want to ingest some example data into convex we are actually going to kind of skip over a lot of that but we are going to go ahead and go down to the context client provider here we do want to actually set this part up so I'm just going to go ahead and copy this naming convention I'm going to go to the app directory and I will make that file and then make sure we copy all of this code click on this copy button paste that in and now we should be able to use that provider and wrap it in our layout so they do explain that if you go down a little bit further they tell you how to use it you import it here inside your layout and then you can wrap your children using it okay so let's go to the layout let's go ahead and import that and let's copy this line and I'm going to paste it right here and if you're following this tutorial to the T they'll tell you how to make a mutation in a query how to basically use that in the front end but I'm going to cover that at a later point when that's necessary so although I may not cover authentication in this video they do have some good tutorials on the convex docs I'll walk you through how to get set up with Clerk or auth zero I have used clerk personally on another side project where I'm playing around with convex and you literally just follow these steps it works really well it's super easy to get it set up kind of just need to wrap your app with a clerk provider and a convex provider with Clerk and then you have access to doing all of these different sign in buttons using clerk sign out buttons and then more specifically since we've integrated convex with the clerk authentication you can actually use this use convex off to know if you're logged in and to get your user IDs and stuff like that for example like if you want to get your user information you just say use user straight from clerk right there um check it out but I'm not gonna be doing that in this video so I think we should be good to come kind of start playing around with this so the first thing that I personally want to do is I want to just get this integrated with open Ai and just allow us to have a UI where we can start having a conversation right I haven't showed what we're planning on building but this is kind of a SketchUp or a mock-up of the idea of what I want we have a panel here that shows your chat history with open AI you have the ability to add in some new text and submit it this is going to be a die so like you can roll a six-sided die to help like roll if you hit or if you miss an enemy Etc this text Adventure is kind of going to be like Dungeons and Dragons but obviously not as complicated it's going to be the most straightforward text Adventure game I can think of over here on the right this is something cool I'm going to try to do and this is where we're going to be taking the output of open AI we're going to feed it into an image generation Ai and hopefully have it print out nice descriptive scenery of what is going on right so if you're fighting some enemies or some skeletons hopefully it shows a battlefield with like a skeleton or two and then I do plan to ask openai to give us some Json so that we can kind of update our UI based on our HP or health points and also if we were to happen to find items along this adventure we're going to use open ai's Dolly maybe we'll use um replicate or some other stable diffusion API to generate icons and have those be displayed as we're kind of progressing through our adventure so we're going to start with this part hopefully this part's not too hard um ignoring the the dice let's go back to our app and in fact before we do dive into that let me just commit everything I'm going to go ahead and say initial commit you've watched my tutorials I like to explain things along the way as we end up needing them so I will talk more about convex and like the mutations and queries and actions as we start stumbling upon them in order to implement the functionality so don't don't be uh too worried that I'm not kind of giving you a complete overview yet all right so let's go to the main page here and we're going to go ahead and just delete as much as we can okay and let's go back and make sure that this properly updates so we've deleted as much as we can and I do want to state that you typically have to have two terminals running when you're dealing with making an application with convex right the first one is going to be you're running convex terminal this is going to listen for your convex files recompile them and deploy them and over here we're going to have our next JS application so I'm going to say npm run Dev um like I spell that right and now we have two running Services locally that need to be open so that we can actually like integrate all right so we have the app running the first thing we're going to do is we're going to create a form and an input and a button here and then probably above that we will add some type of chat history like a scroll bar that has some chat history right that should be pretty straightforward let's go ahead and try to do that so on this page let's make a div we're going to give it the background color of white we're going to make it rounded of Excel and let's just go ahead and give this like a fixed height of we'll do like 400 pixels right now we'll play around with this make sure it's good and then we'll give it a fixed width of 200 pixels okay this is how you can do this in Tailwind the fixed width with pixels and that should be a panel that should show up on the left okay so this will be like our history panel and it's going to look really rudimentary until we actually like make progress right all right so now let's work on a form let's say form and we're going to go ahead and add an input here that we have the ability to like type into a prompt and kind of talk to open Ai and I'm going to go ahead and just I'm going to name this message and uh technically we could like make this set to a value here that we don't have we probably also want um probably also one like an on change but we do want to keep track of some State up here I'll say State message and set message equals you state and we basically just want to like keep track of the message as they're typing in right so I'll say set message e.target.value um and make sure you default this to an empty string so that typescript is happy but at this point I'm going to verify this works if I go ahead and just like try to interpolate this message out and I go over here oh um so again we're using xjs App router so make sure you put use client at the top of your stuff I don't plan on using experimental server actions I don't plan on really doing much server-side rendering or react server components the beauty of convex is that like you can quickly prototype applications using client components and when you start using server actions and react server components it just becomes a little bit more friction especially with what we're trying to build we're building a really highly interactive UI and I don't think we really need that much server rendering going on okay so that's why I'm using use client here just keep that in mind so let's make sure this works I have input box if I type into it the text shows up so we know this thing's working pretty good now you notice that the input is on the far right we want to get that input underneath this panel okay so one way we could do that is we could make this be a flex we'll say Flex column and we're going to go ahead and just wrap all this inside of that okay so now we actually have these things stacked on top of each other it doesn't look that great but we are going to make it look better now just to add a little bit of Polish I know this is going to take a lot more work to make it look really nice let's just move this down a little bit I'm not going to waste too much time on polishing up the input because I do plan on bringing like a component Library a little bit but I always want to make sure that we have like something that we can click on and try to get something displayed here so we also want a button Like A Smith button let's go over here I'm gonna add a button and we will say submit and when someone submits and the clicks this button what we want to do is we want to call an on submit callback okay now what this on submit callback needs to do is we're going to take the form and of course we're going to prevent default and then we can start doing some stuff with the data that was inputted inside of the state right so we're going to take this date and we're going to do something with it so the first thing that we're going to do is we're going to find out a way to send this off to some type of action so when someone submits this what I want to do is I want to send this to a convex action okay now when you're dealing with convex you have queries that's basically for grabbing data from your convex database you have mutations which is for writing data into your database and then you have actions and actions are used for integrating with third-party services such as open AI right so in this scenario the UI is going to talk to an action the action is going to talk to open Ai and then when the action is done doing that third-party integration it can then write some data to the database which we could display in this chat panel okay so let's try doing that now we haven't made a convex action yet okay so how do you do that you go to this convex directory and you just make a new file here I'm just going to go ahead and make a file called chat.ts you can name this whatever you want um but we'll call it chat TS and we want to make an action so again in I'm going to go to the convex docs because using documentation when you're learning something is a great way to understand it and we're going to go down to this back-end functions panel we're going to talk about actions here so if you want to talk to open AI you want to use an action and this is how you can define an action Handler or an action function so let's just go ahead and paste this code into this and we're going to go ahead and just save it now notice that when I save this you'll see your convex terminal basically see that you made a change I'll do it one more time in case you don't see it if I save it it says preparing convex functions and then when it's ready you can actually go to your convex dashboard and go to functions and you will see that function show up over here okay if you click on it you can see when it's getting invoked how many errors you get you get a little snippet of the code you can actually view logs of that if you go down here so let's try this out let's go to the front end now and let's integrate with this can we actually invoke this action from the front end and maybe not calling it do something let's call it let's call it like um handle player action okay because this is basically going to take in the player's input and we have to talk to open Ai and figure out what we're going to do next okay so now notice when I save this that gets updated in the convex dashboard and we can view that new renamed action over here so going back to the front end how do we use that okay well let's go ahead and go back to the docs and a way you can basically do that is go to the calling actions from client section over here and we are going to go ahead and import this use action from react or from convex react okay so let's import that and we are going to go ahead and invoke it okay we're gonna evoke it like this now this is a custom hook right so you have to actually call this at the top of your component and this thing is going to be the function you can invoke whenever you want so we're going to call this handle player action and inside the use action hook we actually want to import our API and something that's great about convex is everything is typesafe so if I do API dot you'll see chat pop up which is like the namespace of the things related to chatting with openai and then you click Dot and notice that our function pops up pretty cool now secondly we need to be able to allow a user to pass in some type of arguments how do you do that well down here you can call the function okay but how do you pass in the actual like message how do I do that right because if you try to do this it's going to complain what you need to do is if you go back to your convex action here you can actually specify args here and so that I'm not going off a memory let's just look at here as an example you basically need to pass it in object and that object's gonna have some properties which are going to Define what comes in so I'm going to say like message I'm going to say V and I'm going to import convex values this is kind of like it kind of reminds me of Zod it's like a Zod validator but you can Define the things that need to come in so this needs to be a string okay so now if you go back to your page and go to message it's all good because now it knows that you're passing a string if you were to change this to a number on accident it'll complain it'll give you a warning and everything is good so for those who have watched my content you'll know that this is very similar to trpc and it's one reason why I really like convex I love trpc I love the T3 stack and when I started doing some work with convex it just made a lot of sense it just felt like it was a nice transition from trpc into a complete back and as a service and I really like it so I think it's really cool all right so let's go down here I'll type in some message um notice that I do need to fix the styling of that but I'm gonna click submit and I do want to point something out is that convex does the communication with websockets so you'll see that we actually sent a message to the back in convex service using websockets and convex is going to return responses using websockets as well and we'll see this in a bit where when you start changing things on your database your UI will automatically get that message and update it's like one of the coolest things I've seen but anyway we just submitted some text let's go back to our convex dashboard which I believe is this one and you'll notice that we did get an invocation over here and if I go to my logs you'll see that if I click on this drop down filter you will see that let me go ahead and just select the one we want we got two invocations I invocated this um earlier on accident but this is the one that we just did right here and notice that it was a success so what we can do at this point is we know that we can send a message to the action everything's working good so if we go to the action we want to actually integrate this with open AI so we can send it like a message and ask open AI like to do some text completion okay so so let's go to the openai node API library and I'm going to go ahead and install this this is what we're going to use to integrate with openai on convex let's just go ahead and install that and while it's installing let's just go ahead and grab some of this example code okay let's go here I'm going to paste that in and you'll notice that this defaults to process open API key okay so for right now we will just uh Delete all this okay and then some more code let's grab this completion model and what we want to do is we want to send the message that came over from the front end and we want to send that to a completion okay so now how do you actually get that message well we haven't talked about the callback for handlers but the first thing that you get is called a context and context gives you access to the various convex related things right so from here you have run action run mutation run query you have scheduler and you have storage if you want to store a file or something um but what we want to do is get a second argument here and I will say args okay and args is actually going to be typed have that message in a property okay so we want to make sure that we put async here so that we can properly do async await and then we are going to go ahead and take that message I'll say args args.message and we want to send that over to open AI let's just go ahead and return the completion here now this won't work because we haven't set up our API keys but let me show you how you do that in convex right so you can't just go and put your API Keys here in EnV local you actually need to go into your convex dashboard and you need to go down to settings and make sure you're on the dev mode right here and I'm going to go down to environment variables right so again as you're saving your convex functions those are getting deployed to a real environment so you're going to have to set up environment variables to like get access to that alright so I don't have an open AI key just yet so let's go over to open Ai and we're going to create a new secret key now just keep in mind whenever you make secret Keys make sure you don't leak them I'm gonna go ahead and click on this and we will say convex text adventure and go ahead and create that and now I have my secret key so I'm going to copy that I'm going to go ahead and paste it here and I will delete this after this tutorial so don't worry that you guys can see it I know people always get worried when they see videos that have keys in them I'm gonna delete it don't worry and that's all you need to do with open AI you just make a key and as long as you're using this opening AI package you can start doing text completions with openai chat okay so let's let's make sure this is all wired together we got the the thing set up properly we've got the environment variable we got the function so before I actually test this out in the UI let's just console log the completion here now when you console log in the back end of your convex action or your mutation or query it'll actually show up in your terminal down here so if I type in some text is today's day click submit we will see that print out the open AI response and says today is Thursday obviously that's wrong today is Monday when I'm recording this video but this is what's going to print out so we can actually use this completion for various things okay again we're making a text Adventure game so we're gonna have to make like the thing that we ask it a little bit more verbose and uh intelligent but that's basically the idea you can make a request to open AI you have the response here and then in your front end you can use that if you want to there's also something cool I want to share with you if you actually go to your convicts dashboard if you don't want to test in your UI you can actually go over here and click run function or you can actually go down here and say there's a shortcut and you can pass in the function input okay so if I wanted to go ahead and pass in a message here and say like what is today's day and if I click run action you'll notice over here we get the same log all right so this is like a kind of a shortcut if you don't want to have to like keep going to your UI and doing stuff you can actually test your convex actions and mutations directly in this UI which is really useful all right so let's see if we can complete the circle I want to go ahead and just change on the main page itself I want to just change the input just a little bit and say text black so we can actually see what we're typing same thing with over here I'm going to see text black [Music] um so now what we're trying to do is we want to display the history okay now unfortunately when we talk to open AI we didn't store the inputs or the outputs anywhere right this is just like a you do an action you get back some responses but we're not storing it in a database convex gives you the ability to kind of store things in their own database it's actually really easy to do since you have access to this context option okay so if I want to for example store some data um let's go and see what the data structure is of the open AI request that comes back which I think we should already have this looks like this is typed properly so let's say completion dot choices is going to be an array I think the first one that comes back and message should be what we want okay let me just double check so this is what we got back from openai we got an ID we got choices we got the first thing that came back message content now there's other stuff in here that you might want to use but in our case we just want to store the content that came back and maybe the input so this is going to be the response I'm just going to go ahead and save that as response and then over here I'm going to say input is equal to args.message okay these are the two things we want to store somewhere so how do you store these in some type of table well you can simply say context and then I can say run mutation and we haven't actually gone to this part yet but actions can't talk directly to a database you have to call a mutation and the mutation is what is going to talk to the database the documentation in convex explains why this is the case mutations or Atomic they're kind of basically transactional type of Rights read through this I definitely recommend reading through this but this is the one place that you can actually insert data into your database so going down here and reading this part is probably super important it says mutations run transactionally and if you don't know what a database transaction is basically it's like if you were to write five different things into the database if one of those things happens to fail due to some type of database constraints or whatever all of those rights will basically roll back it kind of acts like an All or Nothing type of thing right so like everything has to work or else it just rolls back other changes but that might be over your head you might not even need to know about that if you're a beginner so how do we take the response and the input that came back and store it somewhere well let's go back and let's go so let's so let's scroll down a little bit and it's actually super easy to make a mutation I'm going to go ahead and say export const and I will say store entry and I'm going to say equals mutation okay go ahead and Auto Import that and again just like we did with the action this thing can take in some orgs and we can go ahead and say input we'll say V Dot string we will say response VDOT string okay and then over here we'll do a Handler and we'll say async we will say context we'll say args and we're going to go ahead and just set that equal to a function here so now if you do the CTX dot you'll notice that we get back this DB object this is what you can use to write to the database or you can read from the database okay so let's just go and say DB and then I'm going to say insert and now the first argument is the table name again you don't have to define a schema with convex off the bat convex allows you to quickly prototype things and then once you're happy with what you've inserted into your database you can actually go to the convex dashboard and Export a nice schema for your database I'll show that in just a second let's just go ahead and get this data inserted I'm going to insert into an entries database and we're going to insert a input so let's say args I'll say input is orgs dot input and then we'll do a responses args.response and then make sure you await on this right this is a promise you want to wait on this and then you could potentially return something back from this mutation if you want to um but in our case we don't we don't really care all right so let's scroll up a little bit how do you actually run this mutation from this action okay so we're going to go ahead and say API and I went ahead and auto imported that up here so notice I imported API from generate API and you can say dot chat Dot door entry and I might actually rename this to insert entry maybe that makes more sense um let's save that and then we're going to go ahead and we're going to pass it the things that this thing needs right this thing needs the input and then it needs a response now I do believe this is complaining because response can potentially be null so if we wanted to we could kind of fall back on an empty empty string or something or you wanted to do more advanced error handling that's probably a good idea too let's just go ahead and add that in and save it and this does return a promise right so we should probably await on this thing all right so what this is going to do is after we've talked the open AI we get a input we get a response and we're going to store that via kicking off a mutation and we did that by calling API chat insert entry that's going to basically come down here and that's going to invoke this mutation which is going to call the database and store that information okay so let's make sure this works I'm going to go over here to create next app click submit and that should have kicked off that action and notice that if I go to insert entry we have one invocation so it looks like that did get invoked and if I go to data over here you'll see that there's a table now called entries and it has the input of hello world and we also have open ai's response that's pretty cool right we don't have to really Define much but I do want to talk about a schema so now that you have like this this prototype and you think everything's good if you go down to show schema that'll tell you all of the correct schema types you need to add to your project so that you get better type safety okay so I'm gonna go ahead and add this with schema.ts file over here I'll say right click on this I'll say schema dot TS and I'll paste that in all right so this is the thing that you use to basically give better type safety and tell your local code and typescript what tables you have and what properties those tables have and what values those properties will be okay so let me go ahead and save that now let's try to figure out how do we get these entries displayed in the UI okay now when it comes to convex you can use something called a use Query inside react to basically tell your UI to listen for changes with a particular table or a particular like entries of a table so in our case let's say const entries is equal to use Query I'll go ahead and Auto Import that from convex and we are going to call a query now we don't have one created yet right so let's go back over here and I'm going to go ahead and just make one more function I'll say export const I'll say get all entries equals query and this thing probably doesn't need any arguments so I will just give it a Handler actually make sure we import this first okay and then I will say Handler and that's going to be a function that has a context okay now how do we get data from a database well we can say const entries is equal to a weight on tax dot EB we've already done that before so this shouldn't be too new to us but we want to do a query now queries are a little bit different in convex so I definitely recommend go to the docs and let's look at queries real quick here is an example of how you can do a query that you basically say do a query and give it the table name so in our case we're going to say query and we're going to give it entries look at that nice Auto completion with typescript and then we want to get all of the things that are in this table so we can simply just say collect right but if you want to actually filter down for something that's more specific you can actually do a filter and then this thing takes a callback where you can do different types of special queries right so for example you can query to get everything that has a task list of ID inside of this argument okay now we'll probably see this later on I just want to show you like the bare minimum of how to get this going but let's just go ahead and return all the things in the database and then I'm going to say return entries all right so like the mutation in the action after we've created that we can simply just go over here and I'm gonna say api.chat dot notice that it shows up right here get all entries super easy to do and now this thing is automatically going to hook into convex and when those entries get updated this will re-fire behind the scenes with a websocket event that comes in and you'll have the latest greatest entries right here so what do I mean by that I'll show you let's just go ahead and loop over every entry here and for every entry I'm going to return I'll just say a list item and we're going to interpolate entry response and we're also going to we'll do this twice we want two list items we'll do one for the entry response and then above that we'll say entry Dot input and for right now instead of lies I'm just going to do divs because I do want to basically wrap this another div and give it a key of entry.key or sorry entry.id go ahead and close this off go ahead and save this I think I just need to add an extra div here so I can save that Now entries will potentially be undefined so what you want to do is you just want to do a question mark there so typescript is happy about that because you can't map over an undefined thing right so at this point I'm going to go ahead and just add one more thing here I'm going to say class name I'll say Flex Flex call the Gap of two text to Black and then also let's just go ahead and add some padding to this because it was looking kind of rough let's just go ahead and add padding a 2 to the entire thing here we go so here is where it all should just click together right convex again is a highly reactive service where as you start doing your mutations your queries are automatically going to fire off and you're going to see that data come back okay so I'm just going to say um how many sides to a normal dice click submit and notice that these answers automatically get back they automatically update it in our UI we didn't have to write any update logic okay that's the main takeaway we just fired off an action and we have a query that's getting data back from the database and when that data comes back our UI just Updates this is probably one of the coolest features I mean convex has a lot of cool things to it I'm not gonna lie but this is the one thing that like made me realize that you can achieve a lot more with a lot less code by just having this all work behind the scenes with automatically updating your UI so let's just do a little bit of tweaks because right now this UI is still pretty bad like for example this input should clear out after we submit so I'm going to go ahead and go to here and I will say that message is going to be equal to an empty string that should automatically clear that out enter and that'll go away another thing that might be good to do is put this in a scroll bar and increase the width of this a little bit so I'm actually going to make this be like 700 and I'm going to make this a and I'll make this an overflow y Auto okay oh I changed the wrong thing this needs to be width of like that I'll do a height of 300. about 300 300. I'm gonna do one more request I think I just don't think we have enough data there we go so there we have it we have the scroll bar that is going to basically show all the entries that we have asked open Ai and those will be stored in the convex database and just kind of reflect as we're doing work and just in case you're not really impressed yet like this thing hasn't clicked with you let's go to the convex dashboard and I'm just going to go ahead and delete everything I'm just going to clear this table out I'm gonna go back to my UI notice everything just wiped out right because again your query is listening for the database changes and as those change your query fires off your UI updates so I can actually go through here and I can just add entries directly in this table if I want to and you will see them show up in your UI so in this case I think we wanted like an input let's say hello world and then we want a response here that says GG go ahead and save this I'll go back to my UI and there it is all right we made some good progress I think I kind of covered the main basics of actions queries and mutations you're going to need all those to basically build out any type of working application that you're doing for this hackathon so I think if you understood what I just showed you there you'll be in a really good spot let's just go ahead commit what we have I'll go ahead and add everything and then I will say initial work to ask open AI a question and store the response and display it okay so if you ever wanted to kind of like go back to this Commit This is where you basically go to find that all right so we have a decent prototype where we can ask open AI questions and we'll get a response and have that displayed I want to take a step back and we want to actually make a page that allows someone to basically set up an adventure and click a button so that we can prompt open AI with some information about what it needs to do right right now it doesn't understand that like we're going to play a text based adventure game but we need to let open AI understand that the simplest thing we could do to kind of set that up is I'm going to make the home page be that you know create an adventure and I'm going to make a sub route here called like Adventure and we're going to move some of that code inside that Adventure page so I'm gonna go ahead and move that and make sure this is the right import and what this is going to allow us to do is now we can go to slash Adventure like this we can see that page so the home page needs to present a little bit more information about the adventure that we're going to set up okay so let's just go ahead and say export default function and I'll say like Main and I do want to go back to the other page real quick I might change this name to add Venture just so we kind of have it named properly and now what we need to do on this main page is we're going to add a button here so I'm going to go ahead and return a div and let's just go ahead and say button start and add enter all right so when someone clicks on this button again we're going to fire off some type of action or some type of mutation potentially you kind of kick off this whole process right you might have different Adventures for different users and you need to store those adventures with an adventure ID so let's try to get that going so when someone clicks on this button I'll just say on click we want to call something so in our case we're going to make a new action up here in convex and we're just going to go ahead and make a new one called Adventure and we want this to be a mutation so I'm going to go over here and just look for like mutation just to make sure that we're doing this right here we have one I'm just going to copy this I'm going to paste this in the adventure file like this make sure that we import everything that we need import the V from convex values and what we want to do is we want to basically create Adventure okay we want a mutation where someone can create a new adventure and what this is going to do is basically set up a new adventure that we can track all these entries so I'm going to say Adventure here and so when I say Adventure like we want to kind of track the scenery you know maybe the theme are you in a forest are you in a dungeon are you like a wizard are you a warrior so these are some of the things that I'm thinking off the top of my head that like we want to store in the adventure configuration and then this could have like a list of entries attached to it right that's like a foreign key or something right now let's not really even worry about arguments I'm gonna go ahead and just comment that out and over here but for Adventures we're going to go ahead and say class or say character class because class is kind of reserved for keyword I'm going to say warrior so we're going to hard code the character class to Warrior right here and notice that we are getting some typescript errors here because we have a schema file we haven't defined Adventure so I'm going to copy this I'll say add Ventures and we want to go ahead and say character class and that's going to be a string all right let's go back to this I'll put an S here because I call it adventures in the schema and now when you click on this button we want to invoke this mutation let's go over here and I'm going to go ahead and just call a function and as we saw before in order to invoke mutation in convex you have to import it right so I'm going to go ahead and say const create Adventure equals use mutation I'll say API dot Adventure dot create Adventure okay now we have a function that we can actually invoke when someone clicks on this button okay so let's just go ahead and say create adventure and I will put an asynch here so we can wait on it and we're going to go ahead and pass it nothing because it doesn't really need any arguments right now maybe later down the road will add some but we're going to create that adventure and when that is set up in our database we probably want to navigate to The Adventure page so how do you do that with nexjs well you can bring in the router so say const router is equal to use router and make sure you bring the router in from navigation the other router is for the pages router and we're using the app router here as you can tell we have the app directory so make sure you pull in next slash navigation use router and then we could say router.push Adventure all right let's make sure this works um it looks like we need to put a use client at the top of this all right hard to see but we got a little button over here um we are going to Center that real quick let's just go ahead and put a class name here I'll say this will be Flex I'll say justify Center I'll say item Center I'll say with a full I'll say height of V screen or actually how do you do it it's a h screen there it is so now we should have a button right in the middle okay we basically just made this did be full width full height you put this button right in the center if you click on it we should start an adventure we'll go to that Adventure page so going to convex you'll notice we have an Adventures table now and we have an adventure ID which has a warrior for the character class integration time so now that I think about it maybe it would make more sense when we start in the Venture it'll take us to that Adventure ID up here so we can kind of use that as a primary key for getting back all the entries that we've done historically on this adventure all right so easy to do we just go up here and we're gonna go ahead and make a new directory here I'll say Adventure ID and we're going to put that page right there okay if you're not familiar this is how you kind of allow you to get path parameters you just put these brackets here Adventure ID and that'll give you access to them inside of the props that are passed into your page component okay now if you do change the directory structure make sure your Imports are good you can also probably do some type of like um aliases here I haven't had that set up so we won't worry about that right now but let's go back to our page because now we want to actually get the adventure ID that came back when we created a new adventure we want to push it inside this router.push right so let's go ahead and hopefully we'll say Adventure ID which we haven't even set up yet right this thing doesn't return in adventure ID it returns null so we want to go into this function click that and we want to make sure that we get an ID here and we're going to return that ID right so if you hover this it says this is an ID of Adventures and we can return that to the front end and now the front end is going to have that and we can just simply push that like this into that path there and I do think I'm going to rename instead of a venture I'm going to say adventures with an S at the end kind of the convention that a lot of people use like if it's a a collection of things you probably want to have an S at the end here all right let's try it out let's go back to the home page I'll click Start an adventure and now we get an ID up here and that is your unique Adventure ID so if you ever were to you know pause and come back the next day we can associate all the entries to this adventure ID um but notice here in the UI it's actually loading all the entries that are in our database what we want to do is we want to kind of correlate these two tables together so let's go to our schema here and what I'm going to do is on entries I'm actually going to put another field called Adventure ID that's going to be V Dot ID and we can pass it the table that we want this thing to be associated with so we're going to say Adventures like this okay so now we basically set up a foreign key and says okay all entries must associate with an adventure okay um now at this point we don't have that data set up so what I'm actually going to do is I'm just going to clear out both of these tables I'll clear this table I will clear this table too we can start fresh all right so let's go to the creative entry mutation and at this point what we want to do is we want to fire a an action and we want to tell openai that hey we're about to play a text-based adventure game and let's just kind of get these first entries set up in our system so that we can continuously have a historic list that we can keep feeding the open AI um now we haven't fired an action from a mutation yet but it's pretty easy you do context you do scheduler and then you can say run after and right here we can actually give it a delay right so we want this to run immediately but if you ever want to like have something wrong with some type of delay like after a second or two you can do the scheduler and run actions like this I'm going to say zero there's also a um contact scheduler dot run at if you look at this you can give it a time stamp right but let's just go ahead and try to run a particular action so we don't have an action out here for kind of handling that so we are going to go ahead and probably just bring in some code okay so we have this this is handling the player action but really we want to do something more custom so I'm gonna go ahead and just pull a lot of this code out and we're gonna put it over here okay I'll move that to the top go ahead and delete some of this code delete delete this now another thing I haven't talked about is that when you're using convex there's something called an internal action right if you have an internal action this is going to be a private action that you can't invoke from the front end by default if you create a mutation or an action or a query your uis can hit them if you have an internal action then only other convex mutations and actions can evoke them right or the schedulers can evoke them let's just go and rename this one to set up Adventure entries and this thing is going to take in at Adventure ID that could be an ID of Adventures okay and we're going to figure out how to do this in a second I'm gonna comment out all that but again we want to call this thing from the scheduler so the way we do that if you go to the docs it says calling internal functions this is called an import internal here this and now we can go ahead and say internal dot Adventure dot setup Adventures entry and then finally we're going to pass the arguments here okay so like this thing takes in an adventure ID and we're going to say ID like this so again we're just saying immediately run some type of asynchronous process and this asynchronous process is going to do an open AI request to basically prompt and set up the entries right so I'm gonna go ahead and bring this code back real quick and what we're going to do is we're going to go ahead and pass it some content here and let's just go ahead and write out some text so I'm going to say you are a dungeon master who is going to run a text based Adventure RPG or me you will need to set up an adventure for me which will involve having me fight random enemy encounters reward me with loot after killing enemies give me goals in quests and finally let me know when I finish the overall Adventure during this entire time please track my health points which will start at let's just say 10 my character class which is a warrior we're going to go ahead and replace this in a bit um and my inventory which will start with a Broad Sword a bronze helmet and a health potion which heals for three HP okay so you can see we're kind of like telling the open AI like this is the scenario you're going to have to walk us through this and help us play a game now you can get more descriptive to really describe what the scenario is and what I've seen when kind of experimenting with this is that it won't really give you like diverse scenarios right the adventures are going to be kind of the same thing every time you ask it to start one so you can kind of describe the setting of The Adventure I'm going to say the scenario so let's just say the adventure should have some of the following hero must clear out a dungeon from Undead enemies the dungeon has three levels each level has one set of enemies and it means to fight the final level as a boss the final level has a chest filled with one eel sword which deals two damage okay and I'll say a broad sorted that deals basic damage of one and Deals a base base damage of two okay so it's going to take a lot of trial and error to like really perfect this but I just kind of want to let you know like this is kind of the scenario that we're setting up and then we're going to go ahead and just say go ahead and go ahead and describe the adventure for me and then ask me to and then ask me for my next steps when I am fighting enemies please ask me to roll a six sided dice with a one being the worst outcome of the scenario and A6 being the best outcome of the scenario all right so hopefully that's enough information I don't know we're gonna find out but basically what I'm doing is I want to ask open AI this question this the scenario it should give the search it should give us this response and then we're going to go ahead and store that input in output right so very similar how we did it before but I'm actually going to pull this whole thing out to an input variable so I'm going to say const input is equal to this and then down here we will say input save that and now we're going to store those response here and then I'm going to go ahead and just remove that return because I don't think you can return from amvex actions it could be wrong I need to go back and read the docs about that um but I think this would be good enough now like we mentioned when we created the adventure we set that we're a character class a warrior but like we didn't tell this that we're a warrior right we hard-coded it so I'm gonna go ahead and say like character class and we need to somehow get that the way we can do that is we can actually run a query from inside of the internal action okay now we don't have a query for getting that so I'm going to go ahead and just kind of copy this I'm gonna go down here for right now I'll say internal query because I don't know if we want other people to like get the adventure I'll say like get Adventure like this this will take in an adventure ID and this will take into Vita ID of insurers and all we want to do is we want to basically get the adventure and return it so I'm gonna go ahead and just do this I'll say DB dot it Adventures I think I'm going to do the adventure ID I'll say like args Adventure ID like this and I'll pass an arcs here all right so we have a nice internal query no one can access this from the UI disk yet we could easily just switch it to be a real query if we wanted to but what we could do down here is we can say Adventure is equal to a weight context run query the internal adventure adventure and we're going to pass in the orgs like that because remember a venture ID is passed in here so we're basically doing a query against the database we get back the adventure and we got the character class right there hopefully this is of Interest possibly and also we do need to check if there is no Adventure then we should probably throw an error so I'll say Throw new error Adventure not found and obviously this needs to be an error so new error all right so there is a lot going on here right there's a lot going on but hopefully um this should work pretty fine so let's just go ahead and when you click on this start an adventure we are going to call create a venture which does all that stuff we just talked about it also kicks off an internal action we're going to redirect the user to an Adventures page and now we can actually do a query to get all the entries that are related to that Adventure ID okay so I think the way nexjs works with like props is uh you know I don't remember so what we're going to do is we're going to just go ahead and say props also log props okay and we'll go back to our UI which is here and we want to see what do we get back we get prams of enter ID okay so this is what we want to get prams Adventure ID I'm going to go ahead and type that if it'll let me copy it there we go all right there we go we got props and we have the params of intra ID so what we can do is we can use this event ID and when we call use Query here instead of get all entries we're going to pass in adventure ID and we'll say props Rams Venture ID but obviously we didn't make that query to work with an adventure ID as an argument before so we have to go to it and we have to Define orgs and we're going to say this thing takes in an adventure ID and I'll say VDOT ID Adventure ID oh this would be Adventures Adventures alright so now we should have arcs here we can kind of change this query to instead we want to get back all the entries that match this adventure ID so we can say filter and that takes in a callback that you can kind of do like a query Builder with and I'll say Q dot equals U dot field and we're going to go ahead and say Adventure ID and we want to make sure that the field is equal to the args dot eventually ID those passed in so that should filter down the query and give us back all of the entries that are specifically for this eventual ID and there are other ways like setup indexes with convex that make it more performant but again I'm just trying to do the bare minimum to get this working I do notice some right over here so if insert an entry has a type error because we haven't yet set that up so I do think we can fix this real quick this could take in an adventure ID uh again let's just go ahead and copy this down here if I just copy this paste it up there and we're going to go ahead and say Adventure ideas orgs.inter ID so we modify this query we go back to Adventure uh actually not this one I say oh with me so I do notice there is some red here if you look down here the Run mutation it looks like since we just changed this this thing also needs to take in an adventure ID which we have uh up here somewhere I believe orgs.aventure ID yeah let's make sure we're passing the right stuff there make sure we don't have any red anywhere got some type errors here handle player actions okay so this one again this will need an adventure ID which we don't probably have handle player actions we probably want to take one in yeah so if you if you're not familiar with typescript this is a good example of why you want to typescript because it's going to help you out when you forgot to pass certain things to your different functions right so let's just go ahead and make sure we pass that there that file's happy no red this file is happy no red now this one has a little bit of red let's try to figure out why this is now if you go read through the convex docs they have a whole section about document IDs and basically there is this ID type that you can use if you wanted to kind of cast things because technically this isn't a string this is actually an ID of uh Adventures and let's make sure we do the right oh yeah let's just go ahead and import that here all right so now we're basically telling next that hey although this thing actually is a string it's an ID specifically for adventure so that we can get that type safety and uh we should be good there now let's go down here and just fix any more error areas where we're missing stuff props params Adventure ID I might pull this out to a shared variable let me go up here I'm going to say cons Adventure ID is equal to this go ahead and clean that up a little bit and we're going to reuse that right here oh and I'm importing this from the wrong thing this needs to be data model make sure you import from the right package now I don't see any red in our code base that's good hopefully this works again now we just did a bunch of code changes and I haven't tested anything out in the UI hopefully typescript is going to let us know we're doing the right stuff so let's go over here and I think we added new functionality to like create the entries so we might need to go back and just start fresh here so like let's delete this Adventures table and we should be good okay so let's go to our app like start an adventure and we did get a warning so luckily convicts will warn us if we're not awaiting on a promise thing so scheduler dot run at it needs to be awaited but we did get back those entries um let's go ahead and fix that one issue if I go over here we just want to make sure that we're awaiting somewhere this is saying the function create a venture so let's find where that is okay and we want to make sure we await on this and for right now let's delete that so let's scroll back up I think we fixed this warning let's scroll up it says that you are dungeon master so that's all the pre-made input that we made um but if you scroll down a little bit um it actually says like welcome Brave Warrior to the exciting world of Adventure okay uh if you read through this you'll notice it actually starts playing for us right roll the dice to determine your outcome rolling the dice you roll a four so I think we need to kind of tweak what's going on here because like it's literally trying to play the Avenger for us um so here we have to kind of figure out a better way to kind of word this I think when I say go ahead and describe the adventure for me I think that's like the main area where we messed up so I'm going to say given this scenario please ask the player or their initial actions and I'm going to make that the last sentence right so it's kind of clear this is the thing I want to happen next but I will say up here um I'm just going to say when I'm fighting enemies please ask me to enter I'm gonna put the win on fighting enemies please ask me to roll a six-sided dice up here do not roll the dice for me I as the player should input this and you need to describe the outcome with my input okay so hopefully I don't know we'll see if that's good enough I'm gonna go back I'm gonna click Start an adventure and hopefully that gives us some better information I will say that this prompt just needs to be a little bit wider it's it's very hard to play this game with not enough with let's go to the page oh I think we have the with hard coded here so I might just do this and that'll make it full width but we actually want to give it half and half so I think if we were to instead do like a CSS grid here maybe above that I can say the grid and we're gonna say grid calls two okay let's just go ahead and do this so that it's half and half right we have the prompt and we have the dialog here and we have over here we're gonna have like a maybe an image of what's going on in the scene and then some icons maybe even some more height to this thing looks like it's a little a little small Let's Do height of 400 maybe even 450. all right let's see if this works now so welcome Brave Warrior guys says welcome to the thrilling world of Adventure you find yourself standing at the entrance of a dark Eerie dungeon ready to embark on treacherous Journey before you begin let me introduce you to your current status here to class Warrior Health points 10 inventory broadsword okay now it's time for you to decide on your initial actions what do you wish to do okay so this is a better prompt right the AI seems like it understood a little bit better what we're trying to get so what I'm going to do is we need to kind of modify the handle handle player action here and when we ask the AI to kind of play for us so what I want to do is in the handle player action okay we're going to go up and we're going to figure out where that's defined what we want to do is we want to take the entire list of entries that we've done in our current adventure and we're going to feed that as a prefix to open AI so it knows what we're doing because right now we don't feed it the the previous history right we just kind of send it some information and uh yeah like we just sent it a message if you just send it what's in your input prompt like it's not going to understand we're playing a game it doesn't have that context so what I'm going to do is before this we're going to go and say const entries is equal to a weight and we want to grab all of the entries related to this adventure ID now again we're going to have to do a query over here because this is an action right so I don't think we have access to the database so we actually have to go up here I'll say export const get entries or Adventure equals query in fact I'm going to make this an internal query internal query like this and we're going to say this thing takes in args pass an adventure ID and also it's going to take in a Handler like that and we would want to just basically say return contextdb we want to query a particular database we'll get all the entries and remember we've done this already right we just do a filter we do q u dot equals and we'll say q.field Adventure ID and I believe over here we can just say orgs.aventure ID okay and then finally we'll say collect right you have to pass it collect so that you can get all the information back all right so now we have an internal query that we should hopefully be able to invoke down here right so I'll say context run query we're going to pass it internal we're going to pass it Adventure we're going to pass it um oh we're on chat this is chat so I'll say chat we'll say get Adventures get entries for adventure and of course the second thing we need to pass it is the adventure ID so I'll say Venture ID is orgs.aventure ID I sort of basically just doing a query against this code that we just wrote okay and now we get back all the entries here so we want to basically look at this make sure that it's typed correctly and it is and we want to get a prefix so before we just send a message here we're going to go ahead and just say const prefix is equal to entries.map and then for every entry that we have we should have an input actually let me just do this I'll do entries.ap and I'll say entry then for every entry we have I'm going to say return a string of Entry Dot input um maybe do two new lines I'll say entry dot output or response two more lines all right so now we have like an array of a bunch of strings technically I don't need to add the new lines here because what I'm going to do is I'm going to say dot join with some new lines all right so now we have a prefix you can simply just add that prefix here we're going to go ahead and take the prompt that the user wrote we'll pass that directly right there okay and I'm gonna go ahead and rename this I'll say cons user prompt is equal to rxi message just to make it a little bit more descriptive because right now not that descriptive so I don't know why but like having this return like breaks a lot of stuff technically we don't even need to send this to the front end so I'm just going to get rid of it right for some reason that makes typescript happy um and I'm okay with that so we run the mutation and then all of this should be stored so let's test this out um hopefully it works blurring the dungeon let's see if this gives us some type of response you take your first steps into the dungeon the cold stone walls closing is in around you as you venture deeper you come across a Crossroads to your left a quarter stretches in the darkness while to your right you can hear faint Whispers echoing afar shout loudly you see if the Whispers spawned okay a shout out loudly someone says who dares to stir my Slumber my sword and ready for battle okay we're about to get into a battle okay so a group of creatures um approach they merge from the darkness their decaying bodies ready to attack roll the dice let's just roll a dice I'm gonna say um I'll say we did a a four okay so I mean it says I became Victorious I did take some health points I got lost to health I searched the remains of the defeated Undead you find a small pouch containing some loot inside you discover a rust key in a tattered map so then it prints out my inventory so I think this is a good start I think we have a nice prototype and we can continuously tune this if we want to but ultimately I want to show you more about you know using AI to generate images and what we could do is the next steps is we want to basically take the entire prompt that open AI has like all the history and maybe not even the entire history we can just take like the most recent output and we want to feed that into either Dolly which is an open AI service that can generate images and we can maybe show an image that describes a scene that we're currently playing so I think that would be the next steps that I want to tackle all right so now what we want to do is I want to get the AI probably I'm going to start with Dolly and see how that works to just take the prompt and try to generate a scene so we can kind of like visualize what's going on so I'm going to go ahead and just make a new file I guess and I could just say um visualize Maybe and we could potentially just use an action for this we're probably going to need an action mutation so I'm going to go ahead and find out where we are doing a action already okay this looks like we are doing an action here so I'll go ahead and grab this and we'll put it in the visualize function okay so this needs to be a public action so I'm just going to import action here and uh what we're going to have to do is we need to basically use the previous Adventure history join it all together and then ask open AI or Dolly to basically um well yeah ask open AI to summarize it and give us a nice description of like an image of what's currently happening let me make sure I Auto Import some of this stuff we're going to need the adventure ID we're going to need to call this visualize latest entries Maybe um we'll see how this works right I'm playing around with this maybe this will work maybe this won't all right so here's where we are basically fetching all the entries and combining them together for right now I'm just going to copy and paste that into what we want um so we will just go ahead and paste it here maybe and make sure we import some of these internal functions right so we need to be able to get the adventure based on the adventure ID and then we check to make sure it's a valid Venture and then we get all the entries for that adventure and then we set up a prefix a lot of this code could probably be dried up but I'm just not going to worry about it just yet so once we have this we want to basically ask open AI we want to say hey can you summarize and give us like an image that describes this Adventure okay so I'll just go ahead and say like um previous entries like this bind maybe I'll make it more explicitly named and we want to basically ask it a question I'll say given a list of previous Adventure entries for my for a game or for a text Adventure game we are playing I'll say please create a descriptive prompt for an artist with no context of the adventure so that he can draw the most recent events in the adventure okay I'll do that and I'll say list of Adventure history actually I'm gonna say history of Adventure of Adventure here is the history of the adventure so sometimes I don't really know what I could ask open AI hopefully this will give us what we want but this should give us back some information which we can use to kind of pass that into the dolly API to like create an image or something so we get the response here and what we actually want to do is we're going to go ahead and try to pass that to the dolly API which let's go and look that up so going over to the docs of Dolly you literally just have to do this and it'll give you an image URL so we're going to go ahead and copy this in and I will say image response and it's saying create image does not exist why does that not exist at this point I don't know if this is work I don't know why this is red I'm going to assume this is the correct interface maybe they didn't properly type this so it's kind of confused but let's just go ahead and try to get the image URL and I'm going to say console log on the image URL and this will be the image response like this and so this thing is the response that open AI is giving us from this prompt right so we actually need to kind of pass that to here hopefully it's good I'm going to console log that too just so we can like visualize it and make sure that we're on the right track because this could potentially give us some bad stuff yeah so let's just try that out I'm going to go to the convex dashboard I'm going to go to functions and we're going to go to all right so this isn't even compiling because like it's having some issues here so I'm going to say as any and if this doesn't work I'm going to have to Google to figure out why this isn't working so if I go to this function I should be able to just invoke it from the convex dashboard and let's just say run function and we want to give it a adventure ID which I don't remember what ours is so I think if we go to our app here it's this one let's just go ahead and pass that string there and I'll click run action all right open AI create images not a function so at this point we got to figure out why do we now have access to create image because if you read the docs with openai it says openingi.create image um yeah I don't know people are saying that you can import open AI API from the open AI package I'm trying to do that it's not working so I don't know if maybe the docs that I'm reading are at a date something's going on but I've done this before right I've done this before on another project um so what I need to do is I'm going to go ahead and go to my other project that I've done this and I'm going to grab this fetch method which is how you can potentially generate images okay this kind of sucks I have to do this but um it is what it is right you got to improvise sometimes so let's go here I'm going to say cons response is equal to a weight Fetch and we're going to hit this open AI endpoint here now we have to make sure that we pass in the dolly API key so make sure I set that and second thing we want to do is we want to say generate as one image 512 by 512. um you can do a response format if you want a Json but I think by default you could like not put this and I'll give you back an actual like URL this could be image response and uh let's just go ahead and look at this a little bit further I get response okay hit the Json back all right so we're going to go ahead and just say image batch response and then I'll say const image response equals image bestresponse.json and I'll wait on that and then console log this and all the stuff down here I guess for some reason their docs are just wrong so I'm going to delete that if you guys know leave a comment um you know be useful to actually understand the real way to use that but I think what we need to do instead of passing prompt here we need to pass response and let's just go ahead and run this now we should be able to go back to convex the dashboard go back and put our Venture ID here let's click run and it crashes so you cannot find your API key and this is probably because I called it like open AI API key maybe there's a separate API key for Dolly I don't know we'll figure this out go ahead and try running it again okay so it looks like it's doing something and there we go we got data back we got a URL so let's look at this URL and let's see is this something that describes our previous thing kind of I mean it looks pretty good I think the previous thing we did was we killed some some monsters and now we're in like a dark dungeon okay so that's good enough for me I mean it's a good start and let's go ahead and see so this is what the format looks like that we logged out here it's gonna look like this so technically we should do image response data um the first entry URL and that's what we need so instead of just returning this from the action to the front end if you think about convex in a way that's like you know reactive to the websocket events we should actually probably just store this URL somewhere so that we can update some data in the database and then our query in the front end will just automatically update right you have to kind of think with that mindset to really help you be more productive um otherwise you're going to send back the URL and then react you have to set up your own state store that somewhere in state and then refresh it when it changes so we're gonna go ahead and make a mutation here and I can just make the mutation here right in this file um I could say const export const add entry visualization equals mutation make sure we import this and this is going to take in um args and then also of course we need a Handler and I think I'm actually doing this wrong this needs to be yeah I'm doing this wrong let's do that in Handler will be a function that has async on it the curly brace there and a curly brace there all right save that it looks like it's good and what we want to do is we want to make this actually an internal mutation not a mutation internal so that it's private and then we can call this I'll say internal dot visualize dot add entry visualization now in order for this to work we're going to have to be able to pass in an entry ID okay so I'll say V Dot ID and then we'll say entries I believe is the table like that and we're going to have to take in the context argument here and we want to store that visualization that we got so I'm going to say URL vda string and we're going to pass those things so we don't have the entry ID we might need to track that and then we also need to pass in the URL which we call the image URL right here so let's see can we get back the entry ID like the very last entry and I see that we have an entry here but that's mapping over all of them so what I might actually do is I might have this taken an entry ID as well just so we know like we're talking about the very last entry ID and we'll pass that there that'll update the database which we haven't implemented yet so I'm going to say await to contextdb dot patch I think this is the first time we've done a database update maybe we want to go ahead and update the entry ID that we're passing over so I'll say like args dot entry ID and what we want to do is we want to set a URL here so I'll say orgs.url okay so this is complaining because obviously we haven't updated the schema so let's open up the schema file and I'm going to go ahead and just say entries input response of enter ID and I'll say URL maybe I should rename this image URL so it's more descriptive and also this is going to be optional because we don't have it defined until later on so you can kind of wrap things with optional like that I believe so now I'll change this to image URL like that and let's just do a little bit of cleanup I'm like using the name URL and image URL in various places so I think this will make the code a little bit easier to follow all right so let's try this again but what we're going to do is we are going to pass in an entry ID here and we don't know what the past entry ID is so maybe I can just command click that and go to my data let's find the very last entry uh which is where I rolled I think this is the last entry this is sorted by date descending yeah so I can go ahead and grab this ID like that and we're going to go back to the other one and I'm gonna go ahead and just run this and I want to make sure that this all works and that's going to create an image and store it object contains extra field entry ID that is not in the validator I think the issue is because I'm passing too much data here um so I think I think I just need to do a venture ID and pass it the Venture ID like this okay I was passing too much data to that query and I think it was causing an issue let's try it again yeah it looks like it's making some progress okay so we got a success after seven seconds and so if we go to here you'll see that the image URL is actually stored there now and I should be able to open it and it's a picture of a hallway so that doesn't really probably um describe the current history um so what I would actually want to do is I want to print out what is the prompt that is trying to think that we need to generate an image for I'm gonna go ahead and say console log response and then hopefully if I were to run that again we'll see what it's trying to visualize and paint all right the response says explore further into the dungeon okay so I would think that maybe getting more descriptive would be useful let me also print out what this is we're just doing a little bit of debugging right now um I will say when you're dealing with AI sometimes it does take a lot of time to like tune it perfectly so you actually get good results I've spent um like hours trying to tune my icon generator on my other project you're a dungeon master let's try rewarding this completely I'm going to say summarize the following uh adventure of a text based RPG please give a two sentence visual description [Music] for an artist who can use the description [Music] paint us a picture here's a history of the adventure maybe if I wrapped it in double quotes that might help I don't know um let's try it again though I'm also going to say with the most recent events being at the end now I think actually okay this one looks a little bit better the artist is to paint a image of a brave Warrior standing at the entrance of a dark Eerie dungeon equipped with a Broad Sword wearing a bronze helmet so let's see if we were to go to this image now hopefully it's a little bit better I mean it's a soldier it's a person with the sword and stuff um so maybe two sentences is a little bit too much please give us one sentence to of a visual description for an artist who can anyway I'm not gonna spend all day like fine-tuning this um I'm just trying to show you how to build something cool hopefully but at the very least we're storing the image inside of that data structure right after the image gets generated we're going to show it so let's see if we can actually use this I think this thing is good if we go to our UI after we submit a new thing like we want to trigger this action from the other convex so I I think let's go to page and we call handle player action let's go ahead and like pop up to that one and this thing is where it gets all the entries it's going to summarize get a response but also what we could do is trigger off that internal action that we've created in the visual visualize page of the visualize file in fact I think this could be an internal action okay let's just do an internal action here and we can go back to chat and we're going to go ahead and kick that off as well so I'm going to say con await I'll say context I'll say run action and we're going to say internal dot visualize dot visualize latest entries um and that thing needs to take in an adventure ID and then also a entry ID which hopefully we have both of those we have a adventure ID we do so we can say args.aventure ID and then do we have an entry ID we have a list of all the entries so potentially we could just hit the last one in the list and use that so let's just go ahead and say like some entries uh entries dot length minus one dot ID okay so fingers crossed moment of truth I I'm always nervous when I have arrays that have this type of logic because they always crash sometimes if you don't properly initialize it but I think when we first create the adventure we do create some entries so I think this should be good let's look at the last entry in the list and then we'll say generate an image for it so let's see if this actually works I'm going to go ahead and just type in [Music] um continue exploring the dungeon or a staircase down all right so we did that and it says you continue to explain the dungeon searching for staircase leading further down as you make your way to the winning quarters you come across a locked door the rest of key you found earlier seems to fit the lock perfectly you want to use the rusty key to open the door so hopefully that use that last entry response and generated some type of image for us if we go back to data here and uh let's look at the entries it looks like it didn't put a URL so let's let's go see you use the rusty key to open the door so it looks like it ran the visualize latest entries ran but um it definitely didn't update my entry let's look at this one real quick I think I'm a little bit confused I think I updated the wrong thing we see a key um but it should have updated this one so let's go back and try to figure out why that happened so so here's where it adds the entries and I think maybe when we insert the entry here maybe that's where we should actually run this code I'm going to try to do that I'm going to cut this out I'm going to save it and down here in the mutation after we've inserted the entry we should get back an entry ID and then I'm going to run this but this one has to be a scheduler so I could say like scheduler run after you know zero we're going to run this one but now we have the entry ID directly because when you insert you get back the ID here so we could just tell it to visualize using that latest entry I think that's what we want to do and while we're at it I mean let's just go ahead and try to display that because I think that'll be pretty cool so somewhere on the page itself we have this Flex box we have the grid I'm gonna go ahead and put a div here and inside of this I'm going to do an image and we want to basically depending on if the entry has um if the last entry has an image then we can display it otherwise we can show like a loader so I'm going to do a drive State up here I'll say last entry is equal to entries um entries dot length minus one that should give us the last one so entries is potentially undefined so I'm actually going to do entries um and this so get the last entry otherwise this should be undefined at some point if it's not set and then I'm going to say if we have a last entry and the last entry has an image URL then we could potentially show I know this is getting a little bit crazy but uh hopefully this will work we're going to show the image here so we'll show the image and we'll say last entry dot image URL otherwise we're going to show a span that says loading right now it says loading because I don't think the last entry even has anything let's just try it again I'm going to say I open the door with the rusty key all right well we got a huge response back it looks like it actually roll the dice to the term the outcome of the battle to roll with a two so it's like doing the doing the thing for us for some reason all right so obviously this thing definitely needs some some some work right this thing just literally played the entire game for us without even asking us to roll anything so we'll have to tweak that AI is not that smart let's just be honest and the image I mean it looks okay except for this guy's super deformed face um but it's progress right we have it generating images using the previous prompts so let's just do a little bit tweaking I'm going to give this a a height I'll say class name height of like 300 pixels maybe we can do like a width of 100 I guess that's with full because we're using Tailwind and that'll be a little bit skewed but you know what we're not building an actual awesome looking application and then what I wanted to do next I mean I think this is pretty good obviously like I said you can spend hours trying to tweak this and make this perfect um but we're using Dolly which I personally Dolly doesn't make the best images I think something such as replicate with stable diffusion can make you better descriptive images and then also we have to tweak and kind of figure out like the summary of the last entry like what is it telling the artist to paint because it could be a response that's pretty bad all right so now let's move on to like the next part I want to try to show the inventory and also like display the health from the recent um Adventures so we can try to do a similar thing basically after we get our response back like when we insert a new entry we could potentially ask open AI to summarize uh what we currently have in our inventory and our health and output some type of Json for us and maybe we can use that Json to like further update the UI let's just try to do something similar so I'm going to go ahead and make a new file called inventory.ts and we're going to go ahead and just try to visualize using open AI again I'm going to copy everything from visualize I'm gonna put it in inventory because I'm assuming we're going to need a lot of the same stuff so I think for the inventory I'm going to make a internal action called summarize inventory and that's going to use open AI to basically take the the list of entries like we did before and we are just going to update the latest entry to have the current summary of inventory items in health let's do this again I'm going to go down here to this part and we're going to say summarize the following Adventure text RPG and return a Json string with the following format so that I can know what inventory items I have and also what my health points are here is a history of the adventure with the most recent events being at the end so let's just do like a Json format I'm going to say health and that could be like five or something or number and then we're going to say inventory and that's going to be an array of item which would be a string and that's it so hopefully this I can just say this is a typescript type please only give us Json no other output so I don't know how this is going to work probably won't work too well but we're hopefully going to get the response back and uh We're not gonna try to fetch any images just yet I'm gonna go ahead and comment comment out all that stuff and I will say that at this point the adventure is getting pretty long and doing all this stuff is actually going to start costing a little bit more money um as we use more tokens with the open AI so keep that in mind um openai does cost money so if you just keep on doing requests over and over again keep an eye on your bill and how much you're spending especially if you're doing Dolly because dolly is like two cents per image so if you accidentally write some type of recursive thing that like generates tons of images uh it won't be a pretty sight but yeah let's just try do we even need the entry ID I don't think we do so let's try to call this internal action from the convex dashboard go to functions go to inventory um let's go to summarize inventory I'm going to run this and let's just see what happens we should print out the response that we get back and it did give us some Json I'm gonna JS fiddle or something I'm going to say console log I'm gonna say object is equal to json.pars I'm going to paste in all of that how do I run this click run and this is what it looks like we got Health inventory with some items so that worked that's pretty good and he actually gave us the correct thing we have a health potion bronze helmet um bronze sword health so let's use that output and I this probably won't work every time from my experience with open AI one of these times is going to give us some bad Json and it's going to crash probably but we're going to assume happy path and then it's all going to work so what we're going to do is we're going to store some of this information inside of the current entry okay so let's go to our schema and let's try to save like what is our health at this point this could be like a number um and then also we're going to store what our inventory looks like which will be a v uh array of v string and I'm just going to make them optional for right now because like there's other entries that we don't have so this will kind of Crash without it now I think right now these are going to be required so it might crash um since our data doesn't have those defined anywhere but what we're trying to do is we want to basically update those things let's try to like convert the response I'll say stats is equal to json.pars response log that out and then we want to store that so how do we store that in the database well we have to create a mutation so let's just go down here and I'm going to repurpose this one and I'm going to call it store stats into entry okay so we have the entry ID this thing needs to take in a health and then inventory the array of each string and we're going to go ahead and do a patch and make sure that we are patching the health and inventory hopefully that's good and of course how do you run it let's just go ahead and copy this code here and we are going to pass that information here so entry ID should be set we need to get the last entry ID so I could say like const last entry is equal to entries entries dot length minus one and we're going to say the last entry entry ID and then this could be let's make sure we call the right method this will be like inventory Dot door stats and this will be health we'll say stats.health and then we'll say inventory is stats dot inventory now there are some other places where we kind of need to update right so this one expects the health to be defined now in our case we'll just set it to zero or set it to I think I said 10 is like the default and then for inventory I'm gonna give it an empty array okay so we're getting an error because again there's like data in our database that's missing the stuff that we need so I'm actually going to clear everything out anyway I'm just going to clear those things out we're just going to start fresh right we're going to go back and we're going to start a new adventure click this and we should get a response from openai soon we should get an image that is going to display that and then hopefully we have the inventory stats stored somewhere let's go here click on entries and we have health is 10 and inventory is empty now I would expect that we need to actually call this action because I don't think we're calling it yet so that I can actually read through the response because I think my inventory I should start with some stuff right I should have a helmet and a potion and some other stuff like that so let's are we actually calling this I don't think we even are okay so let's go up here and let's figure out summarize inventory do we call that anywhere we're not so going back to chat we could basically run a scheduler here I could say a weight context scheduler dot run after and then we're going to go ahead and do something similar where we can say internal dot inventory Dot summarize inventory and we are going to go ahead and just pass in the entry ID here um probably also the adventure ID if I need to go back and I need to add entry ID and in fact we have access to the entry ID so we don't need to do this last entry ID anymore we actually have it so let's just go ahead and do that and I'll say orgs dot all right moment of truth let's try to continue on our adventure and now we'll say start exploring the dungeon click submit now I will say that I didn't update the entry so let's go back and try to make sure if I were to look at that particular function that was supposed to update that let's go to inventory and then summarize inventory that printed out Health attend inventory abroad sword so we got the correct response back um and that should have been stored inside the database so I wonder if maybe we're just not passing something correct your stats in to entry might just put a log here I'll say args let's just see um or that's light a I'm going to say light a torch see what happens okay I light a torch uh looks like I'm in a battle um it's because I have a typescript error um and this is not compiling so like let's look at this here where this is failing it says uh chat TS is failing oh I think I forgot to save my file rookie mistake I forgot to save the file so I think we should be good now hopefully let's try it again I'm gonna say I rolled a one so it says my health points decreased with seven um but I managed to strike back hitting the enemy and I roll a six I don't know why it keeps rolling for me I wish it would stop and we have an error so at this point it did respond with some Json which I think was not proper Json oh it's because we're passing the wrong thing so I told this is going to be an array of strings but uh open AI is giving us back objects so let's go back here this actually just needs to be ring like this or the dungeon a more click submit all right finally we have the health here I have three Health left I guess I just decided to play the entire game for me it looks like you did um but we also have the inventory here you have a steel sword a broadsword a bronze helmet a health potion and a health potion so I guess during that whole battle at some point we found another health potion inside you find a health potion okay um what about a steel sword I found a steel sword somewhere um but at the very least the health we have right the health we can display so if we have some hearts over here on the right of this image we could probably get those things displayed so what we're going to do here is this is a a column here I'll just say Flex and we're going to try to put another thing I'll wrap this in a div and then over here I'll do another div that says like health okay so what I'm trying to do is I want to draw some parts based on the current health right so I'm gonna go ahead and say new array and we're going to say in last entry.health and then we're going to go ahead and just map over that and we don't care about the value of that we just want to like render out some components here so let's just do let's just find a heart I'm going to click on this and copy the SVG I'm going to paste that right there uh and I probably need to put an index here so I'll say key is index all right so hopefully we have three health it should be able to figure that out and draw three hearts now what I should probably also do is give this a class name of like text white so we can see the hearts I don't know why that's not showing up so let's go back we have health of three is this thing actually being oh it's because I didn't fill right so when you do this new array hack you kind of got to fill it with something I'm just going to fill it with empty strings there we go so that is our current health we have three left now we could potentially make this red it's because I'm not doing I need to do text red like 600. there we go all right so there we have it uh we're kind of getting closer this is like the initial design I was going for and this is what we have it's it's getting there um it's got some little kinks we got to work out but I think it'd be cool to try one more Adventure and just see how this health thing kind of updates so I'm gonna start an adventure and we're going to start off with 10 10 hearts explore the Dungeon I don't know I just printed a picture of a forest I don't think I'm in a forest I think I'm in a dungeon but they probably saw which path left or right and it thought of drawing out a a line but anyway we have as you can tell we have stuff displaying I kind of wanted to get into a battle go left see if it can get us into a battle all right we're in a battle I'm gonna say I rolled a six okay we got some skeletons that were about to show up but I rolled the six too quick and I was trying to actually see what happens if they were to hit me my health goes down please enter the number you rolled I'm gonna say I rolled a one what is this eight babo the skeletons Manchester Atlanta strike so I should be down to nine now if I go back I have nine health if I count these one two three four five six seven eight nine yeah so that did update so I think the very last steps that I want to do is get the inventory displayed and then I also need to polish this up I want to make it look nice and then maybe tweak open AI to just not be so bad some reason this image is taking forever to generate so for some reason it failed to visualize that for some reason not sure why but I think when it comes to the inventory I think we could also use Dolly to like generate images for the inventory and I think we could just cache all the images so we don't have to keep on recomputing them so like here we are getting the inventory from stats and we could potentially Loop through all the inventory items and if it's something that we haven't seen before then we could just use Dolly to generate a new image icon so what I'm going to do is I'm gonna make another action just grab this one I'm gonna put in this file I know this is becoming a giant mess but we're going to try to just use Dolly again and we're going to say generate inventory icon and this list needs to take in a probably a item name and then also I think that's it it just needs to take in an item name and hopefully what we could do this needs to be a string by the way is we're going to take that item name and we'll say args dot item name we're going to generate an image and then when we get the image URL back we're going to store that somewhere so I'm just going to go ahead and say like image url url and I'll say item name args. item name and we're going to make another internal thing that we can call I guess a mutation just go ahead and do this I will say your item image and that's going to take in an item name which could be a string and that will also be a image URL which will be a VDOT string make sure we call this one so store item image will be internal inventory dot that okay so basically we're going to run the mutation after we get the image URL we're going to store that make sure that we we probably want to check to see if we have an item already so I'm going to say item is equal to contextdb Dot query we want to search over items so we don't have an item schema let's just make one I'm going to define a new table called items and this will have item name an image URL beta optional VDOT string and we're going to query that we're going to say filter down and we want to get to the one that Has a Field of item name equal to args that item name and then we'll say take we'll say first and then this will tell us like if the item is this so if it does exist then we'll have to do something otherwise we don't I think if it doesn't exist then we can just create it right so we could just go ahead and like all this we'll say insert make sure it is items and we want to insert the item name args.itemname and then also item URL or image URL orgs Dot image URL okay so basically we're caching the images right so as we see bronze sword or steel sword we're just going to use [Music] um Dolly to generate the images and one thing I also want to do is I don't want to waste money generating these images over and over again so I think what we should probably do is like make sure that we're not gonna waste money doing that I'm going to go ahead and delete all this code so here we're going to say stats dot inventory dot for each because it is an array and I'll say item and then for every item that we have we could potentially just kick off these actions right so I'm going to go ahead and just say like context run you know let's do a traditional Loop I'll say four let item of thats.inventory and then I will run this I'm sorry this again this is again complicated right so I'm not really doing a good job explaining anything I'm trying to solve this live for you all but we're going to go ahead and take the item which will be the name so item name and we're going to go ahead and say run action internal inventory Dot generate inventory icon okay and we are going to go ahead and pass in the item name here and is there anything else that we're missing oh that's it so we just Loop over everything in the inventory and just go ahead and store those but what I think I'm going to do is I want to Short Circuit here I do not want to do this if we already have it so I'm going to say const item is equal to we'll have to do a query now so I'll say context run query and we'll have to make a method down here so let's go down here I'll say internal query it item by name internal query we'll do item name and then we're going to basically just do this return if the item exists or not I actually I'll just return the item make sure we wait on this I think I forgot to wait over here definitely want to do that okay so now we have a query that we can call up here I'll say internal dot inventory dot get item by name and we'll pass in item name args.itemname and I'm going to say if we have an item already we can just return we don't want to do any work if we already have the item and make sure you await it can also reduce the size of this so it'll be a little bit faster 256 and uh yeah let's just try this out I'm I'm kind of worried to see what happens so we are going to just let's just start fresh I'm going to clear everything out we have nothing in items right now and entries we have a bunch of things in Adventures we have two Avengers going on let's clear those out I'm going to start a brand new adventure click it I'll say explore the Dungeon Actually I don't even need to explore we should have some data that's been inserted already so if I go to items we have a bronze sword I would expected okay we're getting a bronze helmet um I think it's because I'm doing it in a weight here yeah so it's because I'm doing like that four or let whatever yeah so this needs to be a promise all probably um I'll do a weight promise that all and then we're gonna go ahead and just like do a map so just restructure this a little bit so that we can do these all at the same time and not have it be so slow all right so the idea was like if I were to I can use these item names and get the images for those so going to our UI here we have to display an inventory somewhere so underneath this stuff uh we could probably just display the inventory you know I'm gonna do Flex I'm gonna wrap this in a flex column there so that we can have the inventory down here um div and then we're going to go ahead and just Loop over every inventory item so I'll say like last entry dot inventory dot map item name actually this would just be item so this would be item name and then we need to return an image where the source is going to be something that we don't know yet so I think for right now let's just print out item name so we can kind of see that put those in curly braces but we see the items printing on here this looks pretty good but we want to actually display the images now we don't have a way to get that lookup table right so I think we have to go to our inventory and do we have a query that returns every item I don't think we do so I'm just going to go ahead and make one I'm going to say const get all items and that'll just be a query that's public and we're going to go ahead and just return everything um so let's go ahead and do a query for items we'll say collect and we're going to return that say that we should be good to be able to call that in the front and now so let's go to our front end and let's go to here we'll say const items is equal to use Query API we'll say inventory get all items and hopefully it's thinking that this thing needs arguments I don't know why I didn't need any arguments I pass that null or empty object Maybe that should be good enough so this will be the array that has all of the items in there and we could basically just look up like we can convert this to an image like this and uh we could basically say if this thing that has an item that matches item name then we are going to go ahead and display an image otherwise we're going to display a div that says item name and at this point we have I have so much going on I'm just so lost um items dot find the item that matches the item name okay if that's defined we're going to return an image otherwise we're going to return a div okay and then we're going to close that off close that off put a parentheses close that off close that off e will be the index get rid of this and we can actually say source and we can say items dot find item item dot item name equals item name Dot image URL uh it's getting pretty crazy so I'm going to go ahead and say like we're gonna do this we're going to pull that out and then we're going to make this code a little bit simpler so I'm going to say if the item is defined like this and the item has an image URL then we will display this image otherwise we're going to display the item name there we have it we have some items showing up that's working cool um we could also give this a grid grid uh calls three gap of four see what happens if we do that all right so we have the items and let's go ahead and say like search around for more items let's see if we can maybe have it find an item and like display it I'm gonna say I'm roll a six so at this point I should have like four health potions all right so again this is like a super super prototype but we have everything like being generated if we actually go back to items over here um they're all displayed there that's good and if we were to find a new item hopefully that will display I'm going to say explore the Dungeon items other than potions I just wanted to find a new item that's not a potion okay it looks like I found some leather armor and at some point this will update when Dolly's done and we have the leather armor being displayed I think also showing the name under the image would be very useful so I think like we did a div here and then here we could say um item name this could be a class name of Lex Flex column and probably also text Center all right so that's what we got this is the Prototype this the mock-up that I was going for and this is what we have I we'll probably spend time now after this just like polishing it up I think this is a lot of good work I'm just going to go ahead and add everything I'm going to say using Dolly to generate images of the inventory display health and show Adventure image okay go ahead and sync that up so often when I do these videos I do them like spread out over a couple of days so I just came back to this project and I loaded up an adventure and I noticed that all of the images that were previously hosted on open AIS image URLs now don't work right those have all been expired I guess open AI only gives you a certain amount of time to actually use those images so I think it'd be much better to have these images stored directly in convex now it turns out convex has the ability to store files inside of their convex storage all right so let's actually try to store them and when you store a convex image you get back a storage ID so instead of having the URL pointing the open AI we can point to our own storage URL I think all right let's try this out so it should be pretty straightforward if we go to the area where basically we create the images let's just start with inventory it might be the easiest somewhere we generate these images right so we have image URL here and if we just follow this approach download the image convert it to a blob store it in the convex and then we could store that somewhere that might make our life a little bit easier so let's just run those and let's hope this works hit the image URL get the image or the blob so instead of storing the image URL here um which would actually point to the open AI image URL let's try using this weight contact storage git URL I haven't tried this before this is the first time I've made a project using convex's storage so hopefully this works pretty fine I'm gonna go ahead and just point the storage ID here and I'm guessing this could potentially return null if you pass a bad storage ID so what I'm actually going to do is we're just going to fall back our default to um an empty string if this isn't defined alright so that should make it happy let's just try this out we might have to start a new adventure to actually try this so let's just go back to the main page I'll click Start in adventure and hopefully that should kick off and it should compute the images using dolly for our broadsword bronze helmet in health potion um okay so I think we have these things cached right I need to go and actually clear them out so let's go to data I forgot we have these and go to all the items let's just go ahead and click all these because we want Dolly to kind of recompute them I'll delete them all we're going to try this one more time I will go back and create an adventure and click that and that should hopefully recreate those images all right there we go if I look at them those are pointing to my convex rapid spider 997. notice I have a URL directly to those images and I'm actually surprised that was super easy to get set up and store images in convex so very impressed with that user experience literally like four lines of code and I have the ability to store images directly in convex awesome so we're going to do the same approach for this prompt that's being generated so I think I just have to kind of copy the same logic here like this and let's find out where we do image response so that's on the visualize page we're going to go over here and we're going to copy that in again um actually the let's do that underneath the image URL and we're going to take the image URL we're going to get the image data we'll see image data.blob we got the image we're going to store the image here and then we get a storage ID now we're gonna do the same thing we're gonna do a weight context storage it URL pass it the storage ID otherwise will default to an empty string okay so now um in the the mock-up I did I have like a dice over here I want to basically allow a user to click a button and when they click on this dice button it's going to roll a six-sided Dice and it's going to automatically put in a value here that they can submit so let's try doing that all right we got a library called react dice roll which I might actually try grabbing it's I don't know when this has been updated or if it's even used two years ago so I guess we'll roll the dice to see if this is even good enough of a library go ahead and install that and while that's installing yeah so that's not a good sign it's already failing saying that like it needs react 16 and I'm going to react 18. I'm going to try to force install it probably not the best approach but if this isn't really maintained what else can you do let's try importing it and we're going to try to display that on the UI so let's go to the adventure page and we're going to import this and then we're going to display this next to the input okay so we should have input somewhere on this page um here we go and to the left of the input I'm just going to go ahead and put that Dice and let's hope this thing shows up on our page well it does so awesome what we could do is we can change the size of this thing and it seems like when it's done rolling we get the value back so what we could potentially do is like when this thing is done rolling I'm just going to set the message as the value interpolated in a string technically you might not even need to do the string interpolation there but I think value might be a number it's like a t value I don't know what that is so maybe we should do a DOT um it looks like we can do value.2 strings so that might be a better approach okay so let's see if this works click the dice and then it should yeah put a five there and put a four there it's just a little big so we gotta first of all let's go ahead and basis a little bit on the form itself we really should probably say class name we'll say Flex gap of two or something just to space these things out um and then also we do some height of 10 maybe and then on the dice itself let's try to reduce that size now I think you can just say size and we could pass it maybe like a 10. and see if that reduces the size of it okay so it's super small I think this is in pixels so maybe 40 would be better there we go um probably more Gap I would love to have some more gap between these things maybe like two or four sorry there we go well that was easy so at this point I think the project is pretty good I want to polish at this UI because it looks awful um so let's let's just spend some time trying to add some Gap in between some things and might move this over here um because I know my head's gonna be hiding some stuff and let's just kind of format this a little bit let's add some some Gap in between these things and kind of make it look a little bit nicer because right now it's looking pretty rough um for the hearts maybe we can actually put a grid here let's find where we do the hearts and inside of this I'm gonna say class name I'll say grid calls three okay that looks good but there's like a bunch of vertical space in between these I'm not a fan of that let's just say h fit that should hopefully squeeze them close together and we can also add a little bit of gap between these things so I'll say two so here is going to be our health grid and I kind of wonder if this should maybe be like by the inventory or above the inventory um let's look at this real quick so I kind of wonder why this is not expanding to be the full width I think I need to add on this thing itself this needs to be a flex grow so that it can expand to be the full width of the thing I'm not a fan of how these things are kind of spaced out so if you look at the actual heading so the hearts I would like to like have them centered so if we look at the hearts here um I think maybe we could just put a div here and say like Lex and then justify Center I think that's how you can like kind of enter those things I'll move key up to the div all right so now they're actually centered on the page we can make them a little bit bigger too I think that looks a little bit nicer does it look pretty good all right so now just a matter of like adding gap between the inventory and like your hearts and stuff like that so let's let's figure out where this is set up on this one I think this div is this whole thing here so what we could potentially do is say gap of four and that should space some things out in between here um probably put a little space in between this too likes a gap of two or maybe even four as well there the image itself we could probably round so like let's find this image and we're going to go ahead and just put a rounded XL on that thing okay so now we got some nice rounded Corners there same thing with these images down here that we're displaying so let's add a class name to these let's say rounded Excel to make those rounded kind of peek at these are these actually like gapped these are grid Gap four so if we have more items there should be a four Gap those look pretty good now adding space between these there's just it's just too crowded let's go back and figure out where that's happening I'm sure there's a grid somewhere here sometimes it's easier just to scroll up to the very top and just do a command F and type grid okay and here it is grade calls two go ahead and put in eight spacing now the the approach I like to do is like the bigger components put a bigger Gap right give extra white space in between like larger um different things on the page I don't know how to explain it but like bigger Gap here and then the things inside of this smaller Gap so like this would probably be like a four this could be a four and then in between this potentially you can make this a two or something just so that things that are like related to each other are closer together okay so now for this let's try to make this uh input be full width and then the button itself we could probably apply a better styling to it so on the submit button here um where's the Smith we'll say last name rounded XL we'll say BG is blue of 400 or 500 and then BG blue 400 when you hover okay and then padding X could be two padding y could be one hover could be text Gray 800 okay maybe maybe not that maybe we won't touch the text and you might ask like why don't you just use like a component Library um I could and I probably should but I want to keep this as simple as possible and bring it in like Shad CN or bringing in um some of these other component libraries like there's a good one called Daisy UI like that could give us some additional dialing but sometimes it's fun just playing around with tailing yourself and like making something look custom although obviously my styling skills aren't the best let's go to input this thing could be a flex grow so we make it the full width of the container and now that I think about this Gap is a little bit too much I'm going to go back to gappa 2 here and I wonder if we even need the height here let's just delete the height yeah that wasn't even needed the button looks a little too rounded so let's just go back here I'm going to say rounded of medium okay that looks pretty good now here remember put some gap between I think this needs a gap of two or four let's figure out where that's happening it'd be here we could say gap of two that puts a little bit of space there it looks a little bit better and for the input itself we could probably add padding so let's find the input element and let's just go ahead and say padding of two I might even make this a four oh I'm doing the wrong thing I need to do the text area so let's find the text area I'm sorry what is this thing is this not a text area I feel like sometimes it gets so lost in the actual like HTML that I can't find what I'm looking for it should be this oh it's a div that's why it's not text area so on the the div this is the input this is a response we could style this up a little bit I could put in for input I could say U and then on response I can say DM or dungeon master okay so just so that we know like this is the dungeon master speaking um now on this div itself we could probably say padding a four just to give this some more padding and then I feel like the text could probably be improved a little bit we're just going to play around with this we could do a horizontal rule here and see what happens there looks a little bit better this could be a div a flex Flex column a gap of one okay and then same thing over here we can make this a div and put an HR there and let's see how this looks okay not too bad and then this whole thing we could give this gap of a four so that the um the space between the dungeon master and you could be much higher so I'll do like a gap of eight there we go I don't like how this had colons here U dungeon master one thing we could also do is look through Google web web fonts and try to find a font that would match like a a text base Adventure more just go here and see if we had anything Rubik ISO that one looks pretty good it might be very hard to read I don't know rubic burn looks pretty cool let's see if this comes up so when you're using next you have the ability to pull in like fonts directly from Google so if I say Rubik burned here we go and instead of using enter here I could say rubric burned and then here I could say font saying that the weight property is missing so let's just go ahead and say wait uh 400 number is not assignable to a string let's go ahead and make that a string uh let's see if this works so I'm gonna go over here and hopefully I'm not sure why the font is not showing up I figured it would but I'm not sure all right so since we are using Tailwind it turns out [Music] um if I comment these out the font does show up so I think simply what I'm doing here with just like trying to put it as a class name is not going to work I think we should probably like add it as a variable Maybe and we'll say like Rubik earned and then what we could potentially do is go back to the docs because I don't know what I'm doing um we'll say variables so they say font whatever so we can go back here and say font rubric burned and then in the CSS variable I'm assuming we could potentially just like style it using this and then here they do the dot variable here so let's go ahead and change this to variable and then inside a global let's just go ahead and say body and I'll say font family and that's going to be VAR of font rubric burned all right let's keep our fingers crossed does it apply The Styling and it does not now maybe actually what we should do is like we should do font family here and we could say Rubik burned as you can tell I don't really mess with fonts too much I'll say font Rubik burned and then that should hopefully give us access to like style these things this is not working let's delete that and then we are going to go ahead over to the page let's try this one more time we have the div here and we should be able to say font family I'm sorry font Rubik burned is that gonna all right there we go and applied okay so that's one we could do it um obviously this font is too super hard to read so we should probably go back and maybe find a different font and this font might work as long as like we have the panel not be a background of white let's try changing this to just not have a background at all and we'll change the text color to be white okay that's obviously that's that didn't help at all let's let's go back and try to find something else at least we figure out how to style it properly this one doesn't look too bad it kind of looks like a game chakra pitch uh let's go and try that one I guess let's say chakra pitch place that I'll just call it chakra and then we're going to go back to the Tailwind we will say chakra I think that over here we'll say font chakra yeah I guess that looks okay I think the font is really small so like we could say font extra large Maybe or I think it's a text extra large here we go and then we could pull this font up higher so like if we wanted to apply it to everything on this page I think we could just do this and then it should start getting applied to various things oh the issues I have font in mono here that's probably what the issue is those look a little bit better let's go down to the items so we'll go to the inventory down here we have item name I'll just go ahead and make this a text of extra large and see what happens I'm gonna apply that to this one too X to extra large there we go now I think this input is a little bright so we can go to the input we could just make the background maybe a little I will say like background Gray we'll do this and we'll say this is text to White and then the text here we'll say text of extra large as well keep it consistent I think the padding here padding X should be a little bit more and here we go looks good now the submit button kind of looks kind of weird I think if we kept it maybe a monochromatic it might look a little bit better so like again we could say gray all right now for some more polish so over here the first entry is always going to be like us priming open AI to know what we're doing so the player shouldn't have to see all this stuff this is like something that we tell open AI so we should kind of hide the first message and what we could potentially do is where do we get the messages we get the messages from here entries and we could potentially either just not show the first input if we're index zero that seems like an easier approach so if index 0 is greater than zero then we'll show the input there now as far as making these icons I think it'd be much nicer if the background wasn't just like a bright white but uh we could probably tweak that a little bit so if we go back to where we're generating these inventory icons let's go and see where do we pass to open AI it's right here the prompt so I'm just going to go ahead and Tack on black background and hopefully this works and then we could just delete the broadsword real quick let's go over the items let's go find the broadsword I'm just going to delete it so that when we kind of restart an adventure to start a new one all right so now it has a dark background I think giving an outline of white might look pretty good let's go to the inventory and here is the image so I might just say like border um White there we go it doesn't look too bad so I think these things were recomputed it would look pretty fresh um you can even make it like a nice gray color white might be a little bit much like like 500 there we go now some things you'll notice is that when this first loaded like it'd be nice if these things had Spinners instead of just showing like some text alright so flow byte let's just go and grab this and see if this works I don't know if I have to install flow byte to actually make this work but if I were to just go ahead and put this right here go ahead and put some class names here and we're gonna we're gonna test this out I'm gonna go ahead and just invert the order of it that is weird I don't know why it looks like that it actually looks pretty cool it's like a ring that's like spinning you know we're gonna go with this I don't think that's how it's supposed to look I think you actually are supposed to install flow byte but we're gonna we're gonna roll with that I think what we could potentially do is just change the blue and I'm gonna just make it gray so it's not so like weird looking I'll make it white actually let's just say fill white see what happens if you do that you know I've tried three different Spinners from three different libraries and they all do this so I don't know if maybe there's a bug in my my page or my tail and component this is the weirdest thing I've ever seen you know we'll come back to this later I don't know maybe I need to like change my Tailwind version or there's a bug but let's just the very least let's just Center this um so let's find out where we do this I'll say text Center hopefully that centers that stuff it does not so let's actually say Flex justify Center that didn't do it because I need to do Flex column I think and that means I need to do item Center there we go and then for the width we should at least do like 20 that's crazy um let's do the same thing for this image so when the image is loading uh we could potentially copy this whole thing out and like let's just make it into a spinner component I'll just say like functions spinner is equal to uh I'll just have that be returned okay so that should show those spinners and then also the image when the image is still loading we are going to show spinner and what we're going to do is negate this so we can see what this looks like wrap this in a div all right so I mean I think the the adventure part's pretty good I think that page is looking pretty good um let's move on to this page I think we can make this look a little bit better and maybe allow the person to select the character that they want to start with so I'll do like an H1 here I'll say text of like 4XL text to White and of course let's see if we can just do that same font I think there's font of uh uh chakra and I'll just go ahead and say welcome to the next based RPG game okay this would probably be a flex column and then the button down here let's go ahead and make this like a text we'll say BG Gray 500 however is BG gray 400 adding exit 2 padding y of one and then rounded medium okay we'll do a gap of eight here and let's see if we can just like generate three images for different characters here okay I'm gonna put like three columns once for warrior one's for Archer one's for Wizard dungeon Dungeons and Dragons Warrior character let's see what happens there um Dark Fantasy this looks more like a warrior go ahead and download that one and then I'll say archer that looks pretty cool and then wizard I don't really like any of these I might just do one more and that one looks okay all right so I went ahead and added those Warrior Wizard and Archer to this and what I'm going to do is I'm going to try to do a grid and I'll say grid calls three and I'll say gap of eight and we're gonna put those three images which we're going to say archer I'll do Warrior first like this wizard Archer and let's see if these show up awesome and then what we could do is you could select one which would basically um outline it okay so let's just go ahead and like add some functionality for that I'll say const selected character and set selected character is equal to use State and I'll just go ahead and say warrior will be the first one go ahead and import that and then we will say on all of these last name is equal to class names I think I need to import you know I'm just going to do this the old-fashioned way I'm going to say if the selected character is equal to Warrior then we will go ahead and say border border White otherwise you will say just do an empty string and then we can simply go through here this is super like I'm just in the rapid prototyping stages right now and let's see what happens so we should be able to select these oh wait I have to add an on click so as we click these things on click and we will say this will be a function that says set select the character of warrior you can go ahead and say wizard here and we can say archery here now at this point there's a lot of duplicate code we can go ahead and say characters would be like Warrior um wizard Archer and we can go ahead and interpolate a map like this and we could simply just return this image so we don't have all this like duplicate logic everywhere and we can just do this and then make sure you put a key uh and this would be I gotta interpolate this now so this would be a character .png okay so now we have the selected character when we create the adventure though we could probably pass that in here so let's just go ahead and say like character is equal to select the character and of course we have to go and update our convex function um which I think is in here we have a function called setup Adventure entries which was an internal action from creative Venture so this thing we should probably type it and say we want to take in a character which is a V Dot string like this and we're going to use that right here we'll just say args which we don't have so let's say args and we'll say dot character awesome so now we basically dynamically allowed the user to change who they're starting as and if this works correctly if I were to go back and create a new Venture here let's try Archer and click set in adventure you can see that it actually says that we are an Archer and it shows a picture of us being an Archer let's just play through one game wait why did I just roll the dice okay there's a little bug when you actually try to play the game it like rolls the dice for you let's find where I did that dice thing and I'm going to move that out of the form put it out of the form and instead I'm going to say this is a flex of gap2 and we're going to wrap all the form inside of this so now I hit enter explore the Dungeon it shouldn't roll the dice for me I don't know what this image is this is not replicate um what we're doing one thing okay so what I'm going to do is I'm actually going to put the summary of everything here I'm going to try to change this up so like the images that it generates are a little bit better and I'm going to say using the above Adventure history please describe the current scene so that I can use the description to draw a picture so I'm just going to try putting the history above and then I'm going to like just ask it to use the above Adventure history please describe the current scene that I can use the description to draw a picture um because right now like what is this this is just not useful at all I'm going to say open the door that looks a little bit better okay so the air goes colder as you descend I think sound of shuffling footsteps Echoes through the holes blah blah blah roll the dice to determine the outcome of the battle because a group of Undead enemies emerge from the darkness uh let's just go ahead and roll a one in fact let's just use the new dice that I have there we go I'll do a submit oh I found a new Rusty key that's cool and I think the adventure is going a little bit better than it did previously it doesn't seem like it's just playing for me now um try to open the chest oh I found a steel sword huh I don't know why that one's just stuck I think Dolly's confused right now um but I did find a new item and it printed out for us okay so this is I think as far as I'm gonna take this project I could obviously spend more hours like plenty more hours polishing this up and trying to make it perfect but I just want to show you how quickly you could build a really awesome like potential hackathon submission using convex using convex storage you saw how easy that was using the actions to talk to open AI I'm generating images I'm generating text so with this you know two to three hour tutorial video I'm hoping that you all with two full weeks can build something actually really awesome so all this code I will put somewhere so that if anyone wants to just have a reference point of like you want to try to submit a really cool creative AI type of project you can grab that code you can play around with it now now I will say Don't just take my code and like submit it in the hackathon because obviously I'm not going to let you win if you just take my code you have to make your own thing maybe later on I will come back to this because I think this has potential to make something really cool but I just wanted to uh share that with you all so hopefully I wish you the best of luck at this hackathon if you guys are serious about contributing I think it'd be a lot of fun to see what other people build and remember we have a Discord Channel where if you ever get stuck you have questions about convex or questions about open Nai or questions about like how to make something super interactive I'm there to help people out um obviously I'm not gonna like give you too much information so you feel like there's unfair Advantage during this hackathon but I'm here to help people if they're stuck with their coding or just stuck with particular small problems with the hackathon submission that they're working towards right that is it that is the project that I just wanted to share with you all hope you guys enjoyed it other than that have a good day and happy coding

Channel: Web Dev Cody

Views: 7,141

Keywords: web development, programming, coding, code, learn to code, tutorial, software engineering

Id: cminnoNmZWY

Length: 156min 14sec (9374 seconds)

Published: Wed Sep 06 2023