Using Server-Sent Events with PHP to Stream Responses from OpenAI ChatGPT API

hi there in a previous video I created this chatbot that uses the open AI chat GPT API so it works just like chat TPT so I can ask something like can you create a bouncing ball animation in JavaScript and then it will generate the answer now there is a bit of a problem here because this doesn't work like chat GPT in the sense that it doesn't stream the response so I have to wait quite a long time as you can see but I think the only reason why I have to wait so long is because it doesn't start producing the answer right away so in today's video I am going to change this so that it actually produces the answer token by token so as you can see now we actually got the response but because this is such a long response it took a long time to make it but it would be nicer if we could see it as it's writing it so let's try and do that too now I'm actually curious if this code will work so let's copy this and see what it does now I have my project open here I will just create a new file here ball.html and I will paste this code here and let's open this ball.html and look at this this is actually a lot cooler than I expected I thought it's just going to bounce up and down but it actually bows us from corner to corner that's pretty cool okay but enough of this let's go back to our chat so how we can do this is we're going to use the streaming option in the open AI API so if we go to platform.openai.com and we go to the API reference which is up here in the menu and we go to chat then here in the request body we have all these fields here and one of them is Stream So ifset partial message details will be sent like in chat GPT tokens will be sent as data only server sent events as they become available with the stream terminated by a data done missing so now we have to use server sent events now we are using PHP so let's first talk a bit about servers and events and how they work in PHP I will delete this ball from here and I will create another folder here called ssde where we can test these server send events so I'll create a new file here called events.php so here is how server send events work so basically normally when you send a request to a PHP server you get back one response so then if you want to listen to some event you have to keep pulling the servers ending new requests one after another but that is pretty slow and there's a lot of overhead from the creation of the request so with servers and events we can create a persistent connection to the server and then the server can send multiple little packets of data during that persistent connection so let's try and do that so the first thing we have to do is we have to set a header which will be the content type which will be Text slash event stream so this will tell the browser that this is actually an event stream and we also should set a cache control header I don't remember what it's called cache cache control is it like this no cash I think it's this way and then we have to send OB and flush so that we will actually flush this request straight away to the browser so that it gets this message and after this we can have what is called an event Loop which is just a fancy word for just a loop so we can say while true so this is an infinite Loop and then in this infinite Loop we can send some message so the way we send a message with the event stream is we just Echo out data and then some data here now we can also optionally Echo out a type sorry it's not color type it's called an event and then we can specify by what kind of event so by default it will be a message event but we can actually set whatever we want here normally there's like message and ping so right now let's set this to next and we have to have a new line after this and we have to have two new lines after this so basically the data will be something like this event message new line data something two new lines and then another event message data something it will look like this and then of course we need to add some sleep here because we don't want to actually use all the resources of the server so let's put a one second sleep here and after this we actually have to call flush so that we will flush this message straight to the browser because normally PHP will buffer their requests and only after the script is done it will send the message out but with the flush we can send partial messages out and then we should add here if connection aborted then we will break from this Loop so this means that if the client closes the browser or closes the connection then we will stop this Loop so we don't actually keep sending data I guess the server will automatically do this anyway like if there's a stale connection it will kill the PHP script that's running I think I'm not sure but we want to make sure that we are not actually leaving these infinite Loops open the server so let's do that and that should be just a basic implementation of servers and events now how do we listen to these events let me create here an HTML file and let me add here some boilerplate code and SSE test and then here let's create some deal with ID results and then let's put some script here and here in the script we can say const Event Source will be new event source and here we pass in the URL to the source which will be events.php in this case and then we can say Event Source on message and this will get a function which will get the event and then inside this we will have event dot data which will be this data right here so let's just say document query selector results inner HTML plus equals event dot data and then let's put a semicolon here now I mentioned before that you can have other types of events so if you put your own event here then you can do it this way you can say Event Source add event listener and then you put your own event here and then you put the function here and then you get the event inside this function and we can actually put message here so we could do it this way too so let's in fact do that so what this should do now it should listen to events which should send this message every second and then it should be appended to the results so let's see if this works now it's a bit difficult to test servers and events when I'm running the PHP built-in web server because it can only take one connection at a time so I cannot have two event sources listening at the same time so if you're testing this on your PHP built-in server then you might have some problems where everything gets stuck because you have the Event Source doing something in the background so you can't get any more requests but let's see what will happen so if I go to slash SSD then I should get something every second but clearly I made some mistake so let's inspect the situation so let's refresh events.php not found perhaps I have to add it as slash SSE slash events and let's refresh this okay and we got sdfs sdfs and it's getting more and more all the time so in fact what we could do is we could say something like to simulate that GPT we could say message is this hello I am your assistant how can I help you today please tell me what to do and then we could do something like for each of SDR split message as letter and then we could send one letter at a time and then if we refresh this page and actually this is the problem with using the PHP built-in server it is still the connection is still open to the event stream so we are not actually able to load this but I think if I stop loading this page then it will stop the event as well and if I refresh now then we get now it's pretty slow but as you can see it's automatically typing one character at a time so let's actually go here and change this to you sleep which is micro seconds so I guess a thousand microseconds is a millisecond so if we put like 50 milliseconds and let's do this again and reload then now it's writing the message like this so now we have to do this with chat GPT and here's a problem normally when the event stream connection is closed it will be started or over again so now it's writing this over and over when it stops so we have to have a way to stop it so let me try and stop this and let's do this let's add in the end Echo event stop and a new line and I'm not sure if we have to have a data with the events but let's add something in here so now after we have finished reading the whole text then we send a stop event and then we can go to our index.html and we can add another event listener for event stop and then we can say eventsource dot close so now this should only print it one time so if I refresh this hello I am your assistant how can I help you today please tell me what to do and then it will stop there and it will not continue anymore and we can actually check in here in the network tab if we open this and we go to event stream then we see all of the messages here so we get the message type and then the data so we got all these and then finally we got a stop and then we know when to stop now you can do all kinds of cool stuff with this event stream because if you have this forever Loop here if I go back to that code you could add some kind of if statement here like if file exists test.txt then you send the message and then we could add here file exists so now what will happen is that when we create this file then the client will get the message so if we refresh this again we get nothing nothing is happening here right now but if I go to the terminal and I create a new file here touch SSE slash test Dot txt and I click enter then we should get a message here which which I'm sorry if I actually type the path correctly so it's in PHP slash SSE slash text.txt so now if I hit enter then we should get file access here and it's going to keep repeating right now but anyway you get the idea if you want to learn more about servers and events then leave me a comment down below and maybe I'll make a separate video on servers and events and what you can do maybe build like a chat application or something like that but now let's get back to chat GPT so how do we do this with attractivity API so if we go to our message.php here here we are sending the message to chat TPT so we use the openai library which is this one by the way if you want to use that and we use the chat method and the chat method takes the options and an optional stream argument now the stream can be a function which is the right function for the curl library now I'm not going to get into what that is in this video but I might make another video on the servers and events and then I will talk more about that so if you want to see that then let me know in the comments but anyway we have to pass in the function here and the function takes two arguments it takes the actual curl handle and it takes the data that comes from the open AR API at least the built-in curl takes these two arguments I think they use the same here and then here what we can do is we can Echo the data and then from this function for some reason we need to return the length of the data otherwise it will give an error don't ask me why I don't know maybe the data can vary in length and it is not known at that moment so we have to pass in how much we read of the data something like this now the data from the opener API will look something like this there will be data just as in the event stream example I did there will be data and then there will be some Json object here now I don't remember exactly what it looks like so what I will do is I'll adjust error log this response and then let's see what it looks like and I will in fact just die here and let's open up the logs here and let's go to our chat and let's send a message and it's obviously not going to work now wait if I didn't work it worked because I forgot at the stream option here true so let's try this again refresh and hello okay now it doesn't work so let's see curl exec could not call the curl right function hmm okay I think that is because I have to die here so maybe that did something unexpected but anyway we get this kind of response from there so let's copy this and paste it into a file so actually as you can see this is one error log so we actually got two pieces of data at the same time so we have to deal with that so what do we have here so in the First Data we actually have only the row so it says now the role changed and then in the next one we actually get some content so this is the first token that we received so I'm not going to actually care about this first one at all because we know it will be the assistant role so I'll care only about these ones so let's see how these are formatted so we have data and a colon and a space and then we have a Json object which has an ID object created model choices which is an array which has an object with Delta which is an object with content hello and we have an index and a finished reason okay so we care only about this content so it will be in choices 0 Delta content so what we can do here is first of all we have to decode the Json so let's do Json equals Json decode and we will get a substring of this because we have the data in the beginning and we don't want that and this is six characters long so we will say substring of data starting from 6. so that should be the Json object which is this and then we need to get choices zero Delta content so then we can say content equals Json choices 0 Delta content and now we should do the same thing as we did over here so we should see if this exists then the text will need that and actually I will not replace these new lines here I will do that on the client side and I'll call this content and otherwise the content will be Json error message if we have an error message I'm guessing it's the same way and otherwise we'll just set it to sorry but I don't know the answer to that so now we should get the content now here's the thing this connection to the open air API will be an event stream but we have to have this whole thing to be an event Stream So This message.php should have the headers here for the event stream so we are like replaying the event stream from open AI to our own application so what we have to do is we will not Echo this data directly because I want to only pass in the content to the client now we could just pass in the data and then read it on the client side but I don't want to pass all that data there I want only to pass the content of the message so what I will do do is I will say here Echo data and here I will pass in the content and then I will copy these headers from this file and I'll put them in here and I guess I can put this connection aborted inside this Loop now I'm not sure if this matters like if the client closes the connection then we are still connecting to open AI but I guess it doesn't matter but we're not going to break we have to return something I'm not quite sure what to return here return zero because this should return the length so maybe returning zero will stop listing this is optional but I will put it there anyway and then when we Echo we of course have to flush so that we get the data right away now I have to rethink this common Mark converter which converts the markdown to HTML because I can't really do this on a partial content I can't do it for every token so we have to maybe do this on the client side or something and actually we don't need any of this stuff anymore because we will only get the stream and then here we should say that it will end so we would say Echo event stop new line and Echo data stop and two new lines so that should actually be all we need in the message.php but now we have to change the client-side script so let's go to our script.js now here we have this send message function which will post to the message.php so we have to convert this into the Event Source thing I'm not quite sure how I can post to an Event Source I guess it's possible but at least we can get but we will have the chat context here which we are posting every time which I actually could save in a session or something so we don't have to send all this stuff every time but let's do this let's say const Event Source is new event source and we will pass in here the slash message dot PHP now before I delve into how to send it as post I will just add it as git so I'll just copy this in here and that should work and then we should say Event Source on message we will get the event and we will say now what will we say we will say update message because I have made this update message which replaces the message box with the new message and the reason why I did this before is because I want to add the message up here as the blinking cursor but now I want to update the message to the new message but we can't just update the message to the event.data because that is just one token so should we do something like let message equal an empty string and then we will say here message plus equals event dot data and I have used message already so I will use something else I will say response and then we will add to the response and then we will update the message with the response and sorry this is the wrong way this has to be actually the message and this will be the response so now we set an empty response and then every time we get a new token we add the token to the response and then we update the message to be that full response and then we have to know when it stops so we have to say Event Source dot add event listener stop and we will have a function here which I really don't care what is in there if the event is stopped then we will say eventsource dot close and then we will know that we have now the full message so then we can push the message to the message history at this point so actually how this context works is it's just an array with the question and the answer so this will be the question that we asked and then we will say respond odds will be the response so that should be all we have to add this message Focus as well actually we will add it just here okay so let's see now what will happen if we run this program again and we say hello it is not working let's inspect the situation even Source own message is not a function event sources response has a mime type application Json aha I forgot to remove the old headers from here and is that the reason for the on message not being a function because it cannot do it so let's refresh hello aventura's own message is not a function okay but we are actually getting a message sorry but I don't know how to answer that and event stop so I forgot to add some new lines here so I have to add here two new lines and let's go back here and refresh this let's clear all of this stuff and let's say hello and we get a message sorry but I don't know how to answer that and stop stopped but we still keep doing it um event stop not quite sure what is happening there do I need to pass in the event even though I'm not using it let's try that hello no it is not working um and in fact we are not updating the message let's add here a console log of the event and here as well stop event and let's refresh hello on message is not a function so we still get that okay um Event Source on message so should I do it this way and let's try this one more time refresh and I will empty this and say hello okay now we got actually the response and it is not sending another response okay so that works so now we have to figure out why we got sorry but I don't know the answer to that so let's error log this Json and let's error log the data well we don't have to reload it let's error log this the substring of the data and let's open the logs here and try this again hello and what do we have here invalid request error null is not of type string messages zero content undefined array key sorry I forgot to change it to get so we are not using post we are using the get method I forgot to do this because I had a small break in between and I forgot where I was at so now it should work refresh hello sorry but I don't know how to answer that so now we did get some content hello and an exclamation point and then we got an error so let's see what is the problem first of all I cannot log Json because it is an object so I have to say print r with the true for return and let's try this again oh I'm getting some answers so let's say hello okay sorry but I don't know how to answer that how can I assist you today so it actually worked but I have a problem here um we don't actually get this every time because the first one will be this so in fact we should get the Delta every time so I can do something like if we have the Delta then we get the Delta content or an empty string if we don't have the code now I should probably do some better error handling here but let's do this for now so if we get the first response which is just the change of the Roll then we just treat that as an empty string but we have to have the Delta property in the Json every time or otherwise we're going to say sorry but I don't know how to answer that unless we have the error so now there is also one more thing the data can be just done because this is how that gbt it tells that it is done so so if the data is data done and I will probably trim this if it is Data done then we will also say that the content is in now this is terrible code right here I will fix it later maybe but it should work so let's refresh this and let's say hello um we are still getting some problems data done so let's log these in here where we get the error and let's see what the data is here hello so what did we get here this one choices Delta ah okay I forgot so we got two of these things at the same time so what we actually have to do is we have to say that Deltas equals explode data we can't really do that because there might be data inside of the response just explode from a new line data and then for each of the Deltas as data then we do this and I can move this over here and now this should be one single row so we can skip the rows that do not start with data so if string position in data of data is is not zero then we continue we don't care about that otherwise we will do this we will get the Json part from the response and if we have the Delta then we return with the Delta the content will be the Delta if we have an error then we will return with the error if it is Data done then we will also say that it's empty and otherwise we'll say sorry but I don't know the answer to that and we flush it every time when we get for every row we get in the response we flush it so now if we say hello hello well that was a very short message and the reason is I used the same variable so this should be the string length of the original data so what do they what do I name these variables let's call this a Delta and let's try this again hello hi there how can I assist you today so now it is working so now I can do something like create a bouncing ball animation in JavaScript now I will not get the code highlighting it will just be in markdown format so sure here's a simple example of a bouncing ball animation in JavaScript okay so now it's actually writing the code and it seems a lot faster because we don't have to wait for the whole thing before we get some response but now we have this problem that we are not converting this into HTML and then highlighting the code so how do we fix that because before I did that in the end of the message so I converted it into HTML but now I can't really do that because it's probably going to fail on a partial markdown is there some kind of JavaScript markdown converter JavaScript markdown conversion Showdown JS let's see if this one will work Showdown is a JavaScript markdown to HTML converter based on the original works by John Gruber can be used client-side in the browser or server side great okay let's try this out so I will use this CDN and what version should I use 2.1 points so if I go to 2.1.0 and I should have some code here yes so I will put this on my index.html and I'll add it here okay so then how I use this is all I will have to do this new Showdown converter then I have the text and then converter make HTML check okay let's do that so I'll go to my script and here I will just set a const converter I'll call it mark down converter okay and then when I update the message I won't just add it here when I update the message I will first say new message equals converter which I should call Mark down converter make HTML new message so this should convert the message into HTML from markdown now let's see how this handles partial markdown so let's open this and say hello can you write a PHP function that generates the Fibonacci sequence sure here's an example well this is not working let's see when it gets to the end will it then work yes so it works after it has finished it but it is not turning it into a code block it is um how about my highlight.js I have it here highlight all now I think the problem with this is that the Highlight JS wants this code to be in a pre block so it has to be like this pre and then code I'm pretty sure so if I now call hljs highlight all and now it works but the problem might also be that I'm not converting the new lines into actual new lines because they will just be like this in the code in that message so I'll say that first new message equals new message dot replace slash m globally with the new line is that how it works let's try this out again right uh hello world in C oh what happened there why is this an H1 console log new message and new message what is happening here right hello world in Rust main sorry I don't know how to answer that um I'm getting too many logs here now I'm logging something else as well I don't have to log these events anymore right hello world in C plus plus okay so why do I not see any slash ends um what is that is that a tab or is that a yes that is a tab three spaces so I'm actually not getting the new lines so why is that let me move this log up here and let's try this again so let's refresh right hello world in C plus plus and this is interesting how it ah because it didn't add the mark down so it just started with the hash symbol so that's a problem but do I have new lights in here content return wait a minute does Json decode convert slash n in the data into actually a new line so do I have to say here string replace new line with at Slash new line is that the reason let's try this again right hello world in BHP it disappeared what happened okay now we are getting actually slash n but what happened to the message where did it go it converted it into a comment what is this madness the reason is that it didn't add the markdown again so is that like is this considered a comment in markdown format let's try something else write a bouncing ball in animation in JavaScript I don't have a copy as an AI language model I do not have the capability to produce a visual animation but I can provide you with a code that can create a bouncing ball animation in JavaScript um okay so now we got a result here but it stopped there for a while when it started to write the markdown so I'm guessing this is a problem with The Showdown markdown converter so it does not know how to handle partial markdown so let's try something else now let's say write a bouncing ball script in JavaScript and now it starts to write this and then it starts the HTML okay that looks good but now here again it's not converting the partial JavaScript so this doesn't look that great but it did actually convert this HTML into a code highlighting and now as it finished this it converted it into an actual code block so that works pretty well now I might do something like count the number of code blocks markdown code blocks so I could do something like this let code block count equal to how do you count the number of occurrences of a string in JavaScript JavaScript account occurrences in stream match length okay so we'll do that and here we have to pass in these ticks I hope the ticks work inside their regex that is very weird is this really how you do it in JavaScript or this what what is this language so this is the code block count and then if it is odd how do you check odd if code block Count modulo 2 is not zero is that how you do it then what we will do is we will say new message plus equals new line and three ticks so now it should add the ending ticks to the code block if they are not there yet if I did that correctly so let's see if that will work refresh write a JavaScript bouncing ball animation now something is not working because it is not writing every anything there was probably an error in my account no it is writing something temp is not defined what did I do temp new message okay refresh right at JavaScript code that produces a bouncing ball animation as an AI language model I cannot produce visual output okay look at this we are actually getting Live code generation from jet GPT and it is highlighted that is great so now it works just like the real chat GPT now this is going to be the end of today's video I hope you enjoyed this and learned something I will keep working on this a bit more off camera and then I will publish this on my GitHub so if you want to use this you can go there and check it out and make sure to subscribe to my channel if you want to see more videos like this in the future and also let me know what kind of videos you want to see and I will make sure to make those kind of videos so thanks for watching and I will see you in the next one

Channel: Unconventional Coding

Views: 1,472

Keywords:

Id: lYq6r6jqB1U

Length: 38min 39sec (2319 seconds)

Published: Sun Apr 23 2023