Get consistent JSON response from OpenAI / Chat-GPT

in this video I'm going to show you how to get a valid Json response by using the jgpt API with python now make sure to watch this video till the end because at the end of this video I'm going to show you how to prevent and invalid Json respawns caused by inefficient amount of tokens provided and I'm going to show how to ensure that the data schema is always the same for those who are new to the channel my name is Thomas and you're watching Tom stack Academy let's start right away as you can see I've already made a start with the following codes so I've included the open a Library I've also imported a Json library and then I'm reading the file API text.txt which is referring to this file and that's where I have stored my API key this is where I'm providing the API key and as you can see I'm just providing f. R which is referring to this file again um if you have any questions about this uh function about reading uh data from text files then you can refer to this website where you can find all the information and as you can see I've also designed the promp provide valid Json outputs provide the top 10 largest key resorts in Europe ranking them on slope kilometers descending provide one column name and a column slope kilometers representing the total slope kilometers so this seems a good prompt um before I'm going to start I'm just going to refer to the documentation from open Ai and you can see here the Json mode and all the documentation regarding this mode so there are three important things the first one when using Json mode always instruct the model explicitly to produce Json via some message in the conversation the second one the Json in the message model returns may be partial so it is possible that in case you have set max tokens to a pretty low value that the um Json text cannot be completed and that you get incomplete Json which is then causing an error in your script so make sure that you provide enough tokens and I'm going to tell a bit more about solving this specific specific issue at the end of this video so make sure to watch till the end and then the Json mode will not guarantee the output matches any specific schema only that is valid and pares without errors so especially the third one is a bit tricky and in this video I'm going to show you how you can at least minimize the chance that you get Json back in a different schema so let's go back to vs code and let's start with querying the API chat completion equals client do ch get. completions do create and then I'm going to start by selecting the model so the model is equal to GPT 3.5 turbo and here you have to make sure at least at this moment that you're referring to 1106 or that you're referring to GPT 4 because those are the models that currently support Json response I'm as well going to set the resp response so response format is equal to type and that's Json object so as I mentioned earlier this functionality is only available in the latest models um if you excluded you can also get Json uh but this will guarantee that the in that the output will be in Json so let's continue with the messages and that's a list containing dictionaries and first I'm going to send a message from RO system where the content is equal to provide output and valid Json so and this this is what we have earlier seen in the uh documentation even if you use response format is Json object you still have to mention either from a system message or from the user that's the um the output has to be in Json so that's pretty important and let's just copy this one now the role is user and here I'm just going to provide the prompt referring to the variable prompt then let's refer to the output and let's store the output in a variable so let's say data is equal to chat completion Dot choices zero. message do content and I let print data on the screen and this is the output that I get most of the times when I query uh this specific prompt so first of all you get an object ski resorts and then you get um as we requested the name and the slope kilometers where um the name is in string format and slope kilometers are an integer but the problem here is that I don't always get ski resorts sometimes it it's just not there uh we are just starting with the name and the slope kilometers and there was not like some kind of a root object and sometimes um this is this this one is not called ski resorts but it's called something like top 10 ski resorts Etc so the response is is highly unstable and it differs a lot and in order to avoid that I'm just going to copy this contrl C and I'm going to save it in variable so let's call this one um example Json is equal to and let's make sure that we also close the list and that we also close the dictionary so now we have a valid Json object and what I'm going to do here so we have here please provide provide output and valid Json data schema should be like this plus and here I'm going to type json. dumps example Json so we are just going to return the Json that we received the first time and if you received a different output uh when executing the API call just make sure that you take um this one that you take this um schema that I have on my screen here so we're basically taking this from the API and then we are giving it back to um jgpt so it can use it as inspiration for the next uh call and this is a way to stabilize the uh the output of this model okay now let's use the output here so I'm going to say ski resorts is equal to json. loads and the Jason that I'm going to load is data for ski Resort in ski resorts and here you have to make sure and this is a bit confusing that you refer to ski resorts because that's basically the parent um the parent object that we see here so we're referring here to this object if key resource wasn't there and we started directly with um uh these variables then we could just uh leave this one out but it's not it's here for now so let's use it then I'm going to print ski resort name Key Resort slope kilom k m showing that we are using kilometers and here you have to make sure um because this variable is returned as an integer so you have to make sure that to convert it to a string with the S strr function let's run the script and as you can see we now get back the ski resorts in a predictable format I run this API call quite a few times and every time the format seems to be the same but the the secret here is that you provide the Json as input um via this um via this object here example Json and that you then inject it here I mentioned at the start of this video that it's possible that um you run out of tokens and because of that you get invalid Json response so what basically happens Let's uh say that I'm going to put Max tokens let's put it on on two for example what basically can happen is that we um provide not enough tokens and then we for example get only back this part of the string and then um we are not able to close the Json and this this makes the Json invalid and will also cause an error in our script so I've set the max tokens to two let's run the script and let's see what happens I see that immediately again an error because the Json is invalid H and I'm going to show you how to solve this issue or at least detect it when it happens and if you navigate to the documentation of CH GPT you will see that we have choices here then we have Choice zero for example and within Choice zero we have the Finish reason which is equal to stop here so let's see what the Finish reason is um once we reach the Max uh token so I'm just going to say that finish reason is this and that should then be finish reason but finish reason print finish reason let's run the script again and you see here that the Finish reason is length and let's now remove Max tokens is two and let's also see where the Finish reason is now and in this case it's stop so if we get like a healthy response the Finish reason is a stop so let's do it like this if finish reason is equal to stop then I'm going to do all of this and in any other case we're going to print error provide more tokens please let's put now Max tokens again on two Max tokens is two let's see what happens error provide more tokens please okay let's remove it okay so these are some ways to make sure that you can detect it when you didn't provide enough tokens and you don't get invalid Json and another way to make the Json response more predictable I hope this video was useful for you if it was please give it a thumbs up and don't forget to subscribe to my channel and I hope to see you in my next video

Channel: Thomas Janssen | Tom's Tech Academy

Views: 6,346

Keywords: openai, api key, api, chat gpt, openai api, openai key, openai key free, free openai api, free openai api key, free openai account, chat gpt key, chat gpt api key, chat gpt account, artificial intelligence, machine learning, ai, artificial intelligence openai, artificial intelligence openai chat gpt 3, chat gpt 3, chat gpt 4, machine learning tutorial, machine learning python, machine learning tutorial in python, machine learning tutorial for beginners, python, uipath, rpa, coding

Id: UtwDAge75Ag

Length: 11min 25sec (685 seconds)

Published: Sat Dec 09 2023