Let's Build a Custom GPT-3 AI Application

let's learn how to build and deploy the foundation for a trendy AI app using openai or cell Edge functions and spell kit this is the example app will build users can enter some text and our app will explain it to them like they're five years old so we take a paragraph from the spell kit docs and click explain it we can see that the response is streamed in nicely and we get a good explanation for a five-year-old we'll start inside the openai playground which lets us test out different prompts and become more familiar with the apis before we actually start to integrate them into our code the menu on the right here allows us to adjust how the model behaves and we're going to pass some of these parameters or options later on Via the API so it's important to know what a few of them are doing now the model we'll be using for our app is the text DaVinci 003 which is their most capable model but if you're curious about the others you can hover over each of them and it'll give you a little bit more information about what they're good at and how much they cost now temperature controls the randomness of a response so a temperature of zero should generate the same response the same prompt and then as the temperature is increased to one the response has become more creative and more unique so we're going to set this to 0.9 for our app here today the maximum length sets the maximum number of tokens to generate for the response and if you're unfamiliar with tokens open AI actually has a tokenizer tool which can help you understand how text is tokenized which is important considering that we are built based on the token so I just copy this text here into the tokenizer we can see that we now have 79 tokens and 390 characters and then each one of the individual tokens is highlighted here in our paragraph and that says that a helpful rule of thumb is that one token generally corresponds to around four characters of text for common English text now this maximum length here doesn't take into account the tokens that are passed into the prompt which also consumes tokens right this is just for what is generated in response to the prompt so we'll just leave this at 256 for now but it could probably be less and then the rest of these settings here just give you more fine-grained control over how the output is generated but for this simple app we'll just leave all of them at their defaults now let me paste in the prompt we'll be using for our app and then I'm going to explain each of the different components so at the top we have the instructions and in these instructions we are giving the model an identity so you are an enthusiastic kindergarten teacher who loves explaining things to students in kindergarten if you're not from the United States is the grade in School in which a five-year-old would be in here in the U.S and this helps to shape the personality of the response right so it might be more cheerful or more enthusiastic and then we're also telling it what to do which is to generate a summary of the context that a five-year-old would understand and in the case of our application the context is actually provided by the user via that form right I went ahead and populated this here just to demonstrate but really this is going to come from our client side or our user we're going to pass that in there and the reason that it's wrapped in three quotes is that this is supposed to prevent users from altering the instructions within the context they pass in for example if a user was to type in forget all previous instructions this is supposed to prevent our model from actually listening to that now this probably doesn't work 100 of the time but I read that it's good practice so we do it here and then we have answer which is referred to as an output indicator which tells the model to start generating the response right it's trying to complete that so it's going to put the answer here so if we test this out we can see that it generates what we might expect to be explained to a five-year-old right of course this prompt has to move for improvement but this will work just fine for now feel free to tweak it if you would like now it's about time to get into the code but before we do let's go ahead and grab our openai API key so we have it ready to go so I'm going to come up here to my account I'm going to view the API keys and then I'm just going to create a new key here and then I'm going to set that into a DOT EMV file within the application as open AI underscore key and as usual the starting code as well as the final code are linked in the description below if you'd like to follow along or if you'd like to just check out the final code so we're starting out with a whole lot of nothing except for some markup and styles and the first thing we want to do is install a couple of the packages that we're going to need early on one of which is open Ai and the other is common Dash tags open AI is the openai SDK for JavaScript or node and common tags provides us with some helpers for working with template strings or template literals that we'll need here in just a few I'm also going to install the types for common tags as a Dev dependency and that should be all we need for now my server is already running in the background here as you can see but what's going to happen is our client side is going to submit that form that you saw in the beginning in the demonstration it's going to hit an end point on our server which is then going to reach out to the openai API to generate that completion so we need to find an endpoint first so I'm going to create a new directory inside of routes I'm going to call this API and then within API I'm going to have another directory called explain and then I'm going to have a plus server.ts file here and then we're going to set up a post request Handler so I'm going to do kit endpoint for the quick generation it's going to be a post and then we're going to destructure the request from that request event that's passed into the request Handler now the first thing I'm going to do is open up a try catch block we're essentially going to wrap everything in a try catch here and we'll just do like that and the first thing we're going to do is check to see if the open AI key exists so we're going to say if not open AI key which comes from EnV static private so if that doesn't exist we're just going to throw a new error letting us know that that environment variable was not set now when we submit data from our client side to this endpoint it's going to come in the form of the request body and it's going to be Json so we can actually get that information from there we can say cons request data await request.json so that's going to give us the request body and we can also check this to make sure that this does in fact exist before moving forward we're kind of trying to fail fast here so if any of these things that would prevent us from moving forward occur we want to just go ahead and throw the error now get it out of the way so we'll just throw a new error and then the structure of that object or that Json is going to look like this it's going to have an object right and it's going to have a context property with whatever they pass in as the context so we can actually pull that context out of that request data like so and then we'll check that as well so if there's no context then we also want to throw an error and now let's set up the prop this is going to be kind of a little bit confusing at first but it'll make sense in a second so we're going to set up a new variable called prompt and we're going to say strip indent which comes from common Dash tags and we're going to open up a new multi-line template string here right and then we're going to do is we're inside of this we're going to run the one line function which also comes from common tags and then we're going to have another template string right beside this and what this is doing is making sure that whatever we pass into this section here is all on one line right because we might actually accidentally pass multiple different lines and we want to make sure that when our prompt is passed to the API all the instructions are in a single line so we're not confusing the model so here we'll just paste in our prompt right our instructions part of the prompt right and then outside of this one line section here we're going to hit enter once or twice make sure there's a space in between and we're going to set up the context and then remember we had those three quotes there and we're going to pass in context.trim so we're going to trim off any white space that may be trailing at the end of whatever the user submits and then we'll skip one more line and we'll have the answer like so so what strip and Dent is doing is it's basically stripping off the indentations of each line so when we indent here in our code it's just taking care of stripping those out that way we don't have to have our code looking like this which looks horrendous to me and then one line of course just Mac just moves everything to a single line all right now the next thing we're going to do is Define our completion option and those are the things that we were messing with in the playground so the model the tokens the temperature of that kind of stuff and we can actually get a typesafe object for that using the openai create completion request type so we can say completion Ops and it's going to type create completion request which comes from openai so we can import that manually here I guess and then this is going to be an object and it's going to take a model which we know is going to be text DaVinci 003 it's also going to take in the prompt which is what we just defined right here we're going to set the max tokens to 256. we're going to set the temperature to 0.9 and then we're going to say stream is true now what Stream True is going to do is going to tell the open AI API that we want a streamed response back and then what we're going to do is we're actually going to proxy or just send that stream request back to our client so the text can render on the screen token by token so now we can make a fetch request to the openai API and I know we just installed the openai SDK for node which allows us to more easily interact with the API but the problem is that with for cell Edge functions they don't allow Dynamic code execution so if the libraries use in your Edge function relies on Dynamic code then we cannot be used it's going to result in a runtime error so we're just going to interact directly with fetch so we can set up a new fetch request here to api.openai.com V1 and completions and then we're going to pass some options here we're going to pass headers we need authorization this we're going to pass in the bearer open AI key and then we'll also pass a content type of application slash Json and then the method is going to be post and the body we will do json.stringify our completion Ops so this information here which contains the prompt and all the configuration settings that we set up now we can check to make sure that this response was in fact good to go so we can say if it was not okay then we're going to get the error from the response.json or console log that for now and then we'll also throw a new error and if everything went well what we're going to do is we're actually going to proxy that response back to our client so whoever made this initial request will return a new response with response.body and then for the headers we'll set the content type to Text slash event stream and then we'll get rid of this return statement here and then if we catch any errors we'll just console them here and then we'll throw an error just a 500 for now and of course we can handle these errors better but you want to make sure that when you are handling these you're separating which ones would be client side errors and which ones will be server side errors so for example the client doesn't need to know that there's not an API key set right but they may want to know if there's no request data so in this case you'd want to render some type of message back to the client but I will leave that up to you to implement we'll just Implement a generic error for now okay so we now have our endpoint set up so again what's going to happen is our client side is going to make a post request to this endpoint we're going to get the request data making sure that it exists as well as the context from that data we're going to generate a prompt here with the context provided and then we're making a request to the openai API passing it these completion options with the settings that we set up inside of the playground earlier we're checking to make sure that the response is good and then we're returning that streamed response back to the client via the Text slash event stream content type so our endpoint should be good to go now let's work on the client-side code and before we get started there's something we need to install we actually install ssc.js and what this Library tends to do is it pretty much fill in the gaps of the limitations of the Native Event Source right so for example we make a get request with the native Event Source we can't actually pass a payload in so we can't pass any information with a get request and we can't we also can't pass any headers either so this package pretty much gives us a bunch of extra capabilities that make setting up something like this super easy I've seen a lot of other repositories trying to implement this exact thing with open AIS API and there are having to pretty much write a ton of stuff themselves and we can just take advantage of this library impetazoni has already done the hard work for us now if you're using typescript we're gonna have to actually set up a type declaration for this Library so inside of my lib directory here I'm going to create a new directory called lib and then I'm going to create an ssc.d.ts file and if you're not using typescript you can skip this part entirely but we're going to declare a new module called sse.js we're going to export a type SSE options and it's going to be equal to Event Source and nit which you should already have and we're going to also take in an optional headers property which is going to be a record we're also going to have an optional payload property and then we're also going to have an optional method which will also be a string and then we can export the class SSC which extends Event Source so it's an extension of the Native Event Source and then for the Constructor we'll have the URL which can be a string or a URL object and then the SSC options will be optional and it'll be typed as the SEC options we just created up there and then we'll also have stream as a function which returns void so now we have this we'll now have type c safety for this SSC module we just installed again if you're not using typescript you don't have to do this at all all right now let's set up our client-side code now unlike my other videos we're not going to be able to use form actions to handle this because this is a JavaScript thing the Event Source and the servers and events so we're actually going to need to set everything up here on the client side make a request to our server side and then render out things accordingly so what we're going to do is we're going to set up a new function that is essentially going to be what happens when this form is submitted right so we can say on submit we're going to do a prevent default to prevent the page from reloading and that's going to be equal to something right and we're going to set up a new function here called handle submit it's going to be asynchronous and then we'll make our form call that function right what we'll also do is we're going to bind this text areas value to a variable as well so we can just say let context so that's an empty string for now and then we'll bind the value of this text area to that so now we update our text area our context will also be updated we're also going to have a couple other variables here we'll have loading which will be default set to false we'll also have error which will also be defaulted to false and then we'll have answer which will default to an empty string and the answer is what is actually going to be returned from our endpoint so the response now inside this handle submit the first thing we're going to do is as soon as we're submitting it right we're going to set loading to true because we're starting to submit it right we're going to set error to false and the reason we're resetting error to false here is because if error is set to true in the previous form submission we want to reset it back to false as soon as we submit this next one right and then we're going to define a new SSC or Event Source so we'll set up a variable called Event Source equals it's going to be new SSC which comes from that package we just installed the ssc.js and it's going to go to slash API explain which is our endpoint we set up right and then we're also going to pass some options into this so the headers we're going to pass the content type as application slash Json and then for the payload remember these are the capabilities that this new you that this SSC Library gives us right we normally can't do this with it with a standard Event Source we can say json.stringify and it needs to be you know an object we're going to put the context into the object so context equal to context here so this is what we're going to submit to our endpoint and then once we create this new event Source here we're going to reset our context right we already have the Event Source set up we've already initiated the server send event connection with our server so we no longer need contacts to be populated we're now good to go to submit another request if we wanted to right and then we're going to do is we're going to set up a couple of event listeners right so we'll say eventsource dot add eventlister and the first one we're going to listen to is for error so if an error occurs right and we'll take e here and then what we're going to do is we're going to set error equal to true I'm going to set loading equal to false and then we'll take the cheap route by saying alert something went wrong now of course you want to show a toast or something some type of alert component something more user friendly here but for the purposes of this video we're just going to go with the easy route by issuing an alert and then the next event we're going to listen to is for a message so we'll say eventsource dot add event listener and this time we're going to listen to a message and this is going to be the data that's coming back in our stream right so we can take in E here for the event and then we're going to set up a try catch block here to catch any errors that might occur we'll set loading equal to false because now we're no longer loading right and then we're going to do is we're going to check if e dot data is equal to done inside of brackets like this we want to return and where are we getting this done from well if we come into the openai API reference under the completions documentation and we scroll down a bit until we find stream we can see that tokens will be sent as data only servers and events as they become available with the stream terminated by a data done message so what we're doing here is we're checking this message if it's done then we're not going to do anything else with it we just want to return right however for the non-done messages we're going to set up a new variable called completion response and it's going to be of type create completion response which comes from openai and then we're going to do is we're going to say json.parse e.data and if we want to see what this data looks like we can actually come into the API documentation again and we're going to see this is what the response looks like so we're going to have a few different things what we really care about is the choices right this is the response right here so we have choices which is an array and the first entry here we have text which has the responses content so we can actually access that by doing this we'll say for the first index of the array we want to destructure text out of it and that will come from completionresponse dot choices and then we're going to do is once we have that text we're going to update our answer so we'll say answer equals and then if answer exists currently we'll use the answer here if not it'll be an empty string plus the text so what are we doing here so remember we look at the stream here tokens are going to be sent as data only servers and events as they become available so as every single service and event is sent here we're going to get a new message right and what we're going to do is we're going to update answer with the next token so for example on the first request through answer is going to be empty right so in that case it's going to be an empty string plus the text that we received back or that first token the next request the answer is already going to have that first token so we're going to add text onto that and we're going to keep doing that and that's how we can actually stream the response in and it comes in nicely across our screen right so if we catch any errors we're going to set error equal to True loading equal to false we'll console error the error and we'll just set up another cheap alert here and then outside of this event listener towards the bottom of our handle submit function we're going to say eventsource dot stream so just to clarify this is what's being called whenever we actually submit the form we're going to set loading to true and error to false we're then going to create a new event source which is going to open up a connection a service and events connection to our endpoint which is returning that text event stream response right we clear out our context because we already have this established then we have these two event listeners here one of which is listening for errors and the other is listening for messages or tokens that are being sent as they become available so when we first get into this we're checking is this the last message that's coming if so we just want to get out of here we don't need to go through this and update our answer we don't want this to be populated into our answer right and then if not if it isn't done then we are setting up a completion response object here we're destructuring the text out of it and then we're appending that to our current answer and then sloppily handling our errors here okay so down here above this paragraph tag I can say if answer so if answer is not empty then I want to show this and then I'm going to make this say answer so if we come back into our browser here and go to localhost 5173 we have our application here so let's test this out so if I just paste in some paragraph here about spell again and I click on explain it we're going to see that the response is rendered in and if we look at the network tab it's going to be a better demonstration of what's actually happening so if again we just paste this in here we can see that slowly over time this one single request here is being streamed in and we can visually see it filling up as we went through so I'll do it one more time it actually looks like we forgot something we're actually continuing to append new explanations to the same answer so let's go into our code here and on the handle submit we're actually going to set answer equal to an empty string again so when you send an answer it's going to clear out the existing answer so now if I try to ask it a question now well again to see that response being streamed in and then if I ask it something else the same thing again it'll reset it so our app stays pretty clean okay this is all working great just as expected now let's learn how to deploy this to versel using their Edge functions so small kit recently introduced something that they call per route configuration so we're able to actually define specific runtimes for each of our different functions so we can have all of our application except for this one function run on node but then have this one single function here right on the edge and that's what we're going to do right now so we can see here that we must export cons config inside of wherever we want it to go and then we can specify the runtime for that specific function if we scroll down a bit further they give us some other options that can apply to all functions we can either specify Edge node 16 or node 18 and we can also specify which region its defaults to all if running at the edge and then iad1 if we're using a serverless function so we're going to be using Edge here so let's go into our application and the first thing we're going to do is we're actually going to install the atsphelt JS adapter over cell and then inside of my asphalt.config file what we can do is we can set the adapter to Adaptive or so instead of adapter Auto and then within this we're going to set the runtime to node.js 18.x so by default all the functions in our app are going to run on the serverless node.js18.x runtime now we can actually become more specific and go into our API route here and set up a config object so we can say export const config it's going to type config which comes from JS kit it's going to be an object and we'll say runtime is going to be Edge right and then now what I'm going to do is I'm actually going to commit this code to GitHub so make sure that of course your dot EMV file is ignored here so that you're not exposing your openai API keys so I'm going to add all this code here coming into GitHub and then let's go to versel now let's go into our cell dashboard I'm going to add a new project here and then we're going to choose the e-lift AI that we just committed that name is fine and then for the environment variables we'll need to paste in our openai key here like so so now we have it and we can click on deploy so this will go through our deploy process here and once it's done we should be good to test out our application to make sure everything is still working as expected even when it's running at the edge okay so we just got the congratulations here so we can actually test out our application let's do that now so let's go to it and then I'm going to just paste in some more text from the documentation here so I'll just use this here to test it and we can see that we did in fact get our response streamed and that is going to include this video so if you'd like to see more stuff related to open Ai and some more interesting stuff integrating it with spell kit versus Edge function Super Bass let me know in the comments down below I'm really interested in this stuff right now so I'm more than happy to make more content about it that's what you all want but if you got back out of this video don't forget to like And subscribe it really helps the channel out a lot and I will see you all in the next one [Music] thank you

Channel: Huntabyte

Views: 14,488

Keywords: sveltekit, sveltekit tutorial, sveltekit crash course, sveltekit form actions, sveltekit forms, sveltekit actions, form actions sveltekit, api route svelte, svelte routes, layouts, advanced layout sveltekit, sveltekit icons, sveltekit extensions, sveltekit productivity, svelte search, svelte react, svelte searchbar, svelte filtering, svelte stores, how to add search to svelte, search data svelte, svelte data search, redis sveltekit, sveltekit cache

Id: fop6hOu8Pck

Length: 21min 36sec (1296 seconds)

Published: Mon Feb 27 2023