ChatGPT Clone | Stream API | Custom Code Explanation

in last week's video we showed you how we cloned the charge GPD web app for mobile in two hours in this video I want to go back and explain how we connected the charge GPT streaming API or custom code and pass the responses and achieved the streaming effect just like the original app but before we move on to the custom code let me show you why we could not use the API feature of flutter flow so this is the API that we are going to be using which is the chat completion API make sure that you're not using the Legacy API in this API you do have the model which in this case could be the GPT 3.5 Etc and we have a list of messages this is important because we want to give the context of the entire conversation to charge gbt so that it can give you a better response depending on your previous questions and its responses and one more important part if you can see here is that there is this something called No streaming and streaming section so no streaming is basically that you will get the response all at once but in streaming you will get the response in chunks so if you have used the charge GPT app you will realize that the response is being displayed to you in chunks which is the streaming effect that I'm talking about to achieve that we must make sure that we keep the stream as true so let's see if we can actually use this API in our native API feature in front for itself or not so we have the API already added here we have the API URL the headers which is the authorization header for the token key that you will get from the charge GPD account and also the content type being application Json in this case you're telling the receiving end that the data in the body of the request is in Json format this will help the receiver know how to parse and handle the incoming data so added the headers we've also added the body as you've seen in the API documentation in this case we are going to keep the stream as false first and see how the response comes out so we're going to use a response and test feature and let's add the API key here so we can test it right now let's press the test button and as you can see the response is received where you get a list of choices which has the message Etc so it is the entire response at once but now when I change the body and make the stream as true let's see how this changes the response I'm just going to make sure that I save it now call the test button and if you can see the API is successful but as a response we're just getting some null data let me tell you why this is happening so when you are making the stream as true instead of false instead of getting one single HTTP response you're getting a stream of responses we call this procedure servers and events which is a straightforward way to get real-time updates from the server to your client when we use SSC or server send events we are creating a single HTTP connection from the client to the server the connection is open for some time and the server is pushing updates down the stream till server closes the connection let me show you how the response of this API actually looks like so we're going to open the terminal and try to trigger the API call from here so I've copied the curve from here and make sure that you add the API key here and I'm going to be pasting this in the terminal now see how the response looks like so as you can see there are multiple responses coming here it does not look like the first response that we had got when the stream was false and if you can notice that each stream event is wrapped with the data key wrapper here and then you have the entire Json body which looks similar to the Json body that we saw earlier when the stream was false and if you notice the content part of each data event here if you see this is an empty string and then you have I can assist with tasks so this is to show you that the API is sending you the response breaking the answer in chunks and sending you the response in parts and this way you can keep on showing those parts in the UI and that is achieving the streaming effect that you see in chart GPT and once the response is completed it will send you a closing data event this is why it was not working in the native API feature of dataflow because it is not able to recognize this response as a Json body because first of all there are multiple responses here and each of them are wrapped with the data key wrapper here and if you see this again in slow motion then you see that each data is being received one by one rather than all at once currently in Floral flow we do not support this kind of of a streaming API natively in the platform but of course we can do this with custom code today as you saw in the last video let's go back to our custom code feature of flutter flow and see the code that we had written in our last video and try to understand what exactly is happening here but before we jump into the code here I want to show you some other parts which will be useful once we go through the code so if you notice the response of the streaming API you can see that it has an ID object created model list of choices inside it there will be something called Delta and inside it will be the content map you can create your own dart class for the content response and map this Json structure with it so that it's easier to work with this data I have already done this here you can use any online tool as well where you can just search for convert Json to Dart class and this will create the mapping classes for you in this case the main class is the content response where you are going to have the from Json and the two Json methods which means that you're going to provide the Json data and it's going to convert it into the content response class and that makes it easier for you to work with the response I've already added it to the end of this action so now let's go back to our main action which is uh the streaming API response and let's talk about the arguments in this action so we have a list of dynamic Json body which is nothing but a list of Json data and as I mentioned earlier you must have a list of messages provided to the body of the API because it needs the context of the conversation so it can provide you the right response and also we have a callback action we will get into the details of this later so now in your stream API response function before that you must add your Imports here in this case we have the dart convert import which would allow us to encode and decode our responses and we also have the HTTP library because we are doing the HTTP calls or cells in the code let's start with creating a client object of the HTTP client and now we can start preparing the request which means that we're gonna Define the URL the headers the body just like we did in the native API feature itself so in this case the URL is defined as we showed in the API documentation and then we have the headers as I described it earlier the content type and the authorization here in this case the API key is going to come from the App State but if your API key is stored somewhere else make sure to retrieve it first and then prepare the header map and then we are going to be creating the Json body I've just extracted it out to a different function just to make the code more readable but the function just looks like this where we have this map object of key value pairs which contains the model part the list of messages and if the stream is false or true and we encode this into a Json and send it to the body here and then we are going to create the HTTP request we are going to define the post method in this case it's post because we're sending a body with the call and then we Define the URL we add the headers and the request body and after that we are going to be sending the request here we are going to get back a streamed response object here streamed response is essentially a streaming HTTP response which means that you can read the data as it arrives rather than waiting for the entire thing to load once we get the response let's skip to this part of the code where we are going to listen on these events and do something once the data has been received here here if you can see we are getting back a list of integers which is nothing but a raw bytes of a part of the response body since the response is a list of integers we have to decode it and convert it into string format so this part of the code is taking this list of integers or the byte values and decoding them into a utf-8 string this is usually what you do when you want to read the textual content of the HTTP response and now we have the response chunk as a string so you can do anything we can basically extract the Json part of it parse it display it in the UI but before we talk about the data part of things let me show what I skipped here I actually added a chat message at the end of the chart history with an empty content and author as a system this registers it as the last item in the chat history list so now when I'm listening to the responses and processing the data I know that I need to update the content of only the last item on the list for example let's go to our listen stream blog first and understand what is happening there so here I am receiving the chunks of my responses we first check if this response contains the data key or not then we split the string into two parts where the first part is the data clone and the second part is the Json body so technically the split function is dividing the spring into an array or a list of two parts we are only concerned with the second item in the list which is the Json body as you know in programming the count starts from zero so here we're denoting that the bracket 1 means I need to retrieve the second item as 0 is the first item now that the Json body is ready we are now going to convert it into a Json format and eventually convert it into the dart class that we showed you earlier moving on from there we do some null checks to see if we have the choices data or not do we have the Delta part or not or do we have the content part of the response or not and then we extract the content out of it and that's it just updating the last item in the chart history and making sure to append the new data onto the existing content so let me show you an example if your content is the entire response or the entire answer is I am really good then in the first second it's going to be I and then the data changes to I am and then the data will change to I am really and then the data will change to I am really good so this is how you can see we keep on adding the new content onto the existing content as you get more response and that's how the entire progression looks like after that you have to call the Callback action so now this callback action is required because the entire API call is outside of the screen where the API is being called and we want to update the UI of the screen because the content is constantly getting updated because of the response but the UI will not get updated if you do not explicitly update this date here so when we call this custom action from the action flow editor we are calling an empty update app state which will simply call a method to update the UI State and also scroll to the end of the list because without the scroll the answer will keep on generating and the user will have to manually scroll to see the end of the answer which is not idle here and that's it that's the entire explanation of the stream API response method that we showed you earlier we also had another method which is the convert list to Json which is called before the stream API action as this is the one that is eventually converting the App State chat history list to a list of Json map and pass it to the stream API we actually got this code from code copilot so I did not even write this myself but if you're curious about the explanation then you can see that it is taking a response list which is basically nothing but the list of the chart response data type that we had created earlier and it first declares an empty list of Json then it reads through the entire list of chart response and converts them into a map object each having a value of role which is the response to an author and the content which is the responsor content and adding them into the Json list and returning the Json test technically this is the Json list which is being sent to the stream api's arguments all of these transactions are happening in action flow editor so that's it that's how you create a streaming effect in charge apt app this basically will apply to all such apis where the server is sending stream data instead of a single HTTP response till the feature is natively available in flutterflow this is how you can do it with custom code

Channel: FlutterFlow

Views: 5,705

Keywords: Flutter, FlutterFlow, No Code, No Code Tools, How To use FlutterFlow, Google Flutter, Dart, No Code App, FlutterFlow App Builder, No Code App Builder

Id: ji2XbsDT_PY

Length: 14min 17sec (857 seconds)

Published: Wed Aug 30 2023