JSON Schema Validation in Python: Bring Structure Into JSON

what is going on guys welcome back in this video today we're going to learn about Json schemas which Define what Json files have to look like and allow us to validate them easily so let us get right into [Music] it all right so we're going to learn about Jason schemas in this video today and we're also going to learn how to validate Json files or Json data in general using Json schemas in python and a Jason schema is essentially just a Json file itself which defines what other Json files have to look like and you can have multiple schemas in your application for different use cases so for example you might have one schema that validates uh Json files uploaded by users and then you might have a different schema that validates post request data that comes in the form of uh Json data and then you can have another one that validates config files which are in Json format so in general Json schemas are used for API validation config files automated testing um input validation all sorts of things and as I said we're going to learn how to do this in Python today so how to take Json schemas apply them to Json data using python uh to validate any sort of Json data that comes into our application or maybe even goes out of our application so for those of you who don't know what Json is in the first place it's JavaScript object notation however it's not really related to JavaScript only it's just the name and Json files are used in general to store information in a semi-structured way uh using just text so what we can do here for example is I can create a new file here and I can call it person. Json and this Json file here can store information about one person for example this is a very simple example now uh but I can have maybe a field name and the name of this field uh the value of this field name uh could be something like John Smith and then I might have a field H and H could be uh let's use a number 25 and then I could have uh another one is student for example and is student would be set to false so you can see here that adjacent file is essentially just like a dictionary we have key value pairs we have uh the keys on the left side the values on the right side they can be booleans uh integers float strings everything and we can also have list so we can also actually say okay we have actually people and then people is just a list square brackets there you go a list of these instances here so I can take this here and I can put it into CI brackets and then I would have a list of people um but let's keep it simple for now let's say that one Json file in our example today in this video is going to just represent one person very simple like this we have name age is student now the thing is if I allow users to upload their information in form of Json uh and they just have to submit a Json file with their information they might mess this up so for example instead of saying 25 here they could use a string 25 or they could even say something like 25 or something like this or they could even misspell H or use a different uh I don't know maybe H is a stupid example here but maybe they say instead of is student they just say student so the problem is since Json is semi-structured data we don't have a rigid system we don't have a um reliable structure so what we can do is we can introduce schemas that Define what the Json file has to look like and before we do anything with the Json file we validate it to see that it actually is in the correct format so for this of course we need to Define what the format is and in our case we're going to keep it simple we're going to just say we want to have a name an age and is student field this is this is a string this is an integer this is going to be a Boolean and some of the fields are going to be required others are not going to be required and then we can decide if we want to allow for additional properties so what if all this information is here and correct and then I want to add a uh an extra field for example favorite programming language and then I say python for example is this allowed or is this not allowed uh this can also be defined in the schema so the schema itself as I said is a Json file or at least Json data so we can go ahead and create a new file here and I can call this person- schema. Json and here now I need to have a couple of certain uh or I I need to have a couple of fields here so I start again with curly brackets and then I have two special Fields one of the uh one of the fields is very important and it's the first one it's the schema field so it's the key dollar schema and this dollar schema refers to a definition of Jason schema there are different versions different versions of the schemas have different functionality in this video today we're going to focus on very very simple aspects of Json schema we're not going to go too deep into uh references or if statements or something like that we're going to keep it very simple we're just going to define properties data types and requirements nothing to uh fancy about this but of course if you use newer versions of the schema you're going to get additional features um and if you want to know more about these Json schema features let me know in the comment section down below if this video goes not necessarily viral but if it's popular amongst you guys I'm going to just um release another video video going into more detail here um but for now we're going to just use one version of the Json schema we're going to say HTTP colon and then Json schema.org draft d07 this is the version and then slash schema and a hashtag even though I don't know if that's uh required but you can also change this draft 07 to something else you can just go to the Json schema orc site and look at the different versions but for our purposes today this is going to be enough we're going to just keep this one uh and the second field it's not required but it is recommended is the ID field or to be precise the dollar ID field and this is the unique identifier of this particular schema now the thing is it's recommended to use a URI and even if you don't own a domain so even if you don't have an actual domain you can make one up you can um you know uh use a fake domain that does not exist or you can um use Local Host if you want or something like that but you should use a URI because then you can have a base Ur URI that you can use for references if you want to use some more advanced features but a common example of this would be something like um your own website in my case neural 9.com and then slash my-ers schema this is just a unique identifier uh that identifies this particular Json schema so it's recommended to have something like this even though it's not required so you can also emit this if you don't want to have it uh then we want to have a title for the schema we're going to just call this person then we want to have um and this is now the important part we want to have properties so properties are definitions of what we want to have here in our Json files and in our case we want to have name age and a student so we want to say here name is going to be one property and the property itself is again a dictionary of certain key value pairs in an our case we're going to just use one which is going to be the type and the type of name is going to be string like this actually with a lowercase s so this uh Json schema defines now that there is a key name and the value has to be string of type string um then we can say h same procedure we're going to say type and the type of the H is going to be integer now we can add an additional field here for example we can say minimum the minimum H is going to be zero because we don't want to allow for negative ages uh and then we can say is student um and we want to say type is Boolean so those are now the definitions of the properties what we can now do is we can say which of those properties are required and we can do that down here with a separate field required and required is going to uh point to a list of um strings and the strings are going to just say which properties are required so name and AG for example in this case is student would be an optional uh key value pair we don't have to um we don't have to actually Define it also with this Json schema that I have right now additional properties would be allowed so I could just add some stuff at the end and it would be valid even though the keys are not listed here if I only want to allow for keys that I listed here I want to uh specify the key value pair additional properties and I want to set it to false because false means I'm not allowing additional properties all right so this is the Json schema now you can validate this with different tools with different languages but we're going to focus on python today so we're going to go and open up a new python file main.py and we're going to have to install a package called Json schema so pip or pip 3 install Json schema and in my case it's already installed and then we're going to say um UT import Json and then from Json schema import validate from Json schema. exceptions import validation error and now what we want to do is we want to load the Json file so we want to say with open and then person dot come on person. Json uh open this in reading mode which is the default as F and then we want to say the document that we want to validate is going to be f. read and then we want to say open also the person schema Json SF and this is going to be our schema f. re or actually sorry we don't say f. read we say json. load F and we say json. load F down here as well um so now we have adjacent object schema adjacent object document and what we want to do now is we want to say try the following try something try the validation and then if the validation fails with an Val with a validation error s e we're going to just print validation failed and then we can say print maybe an F string here error message is going to be e do message um and the actual validation happens here now in the tri section and we're going to just say validate the instance which is equal to the document with the schema which is equal to the schema and then we're just going to say print validation succeeded if it worked now I closed it there you go uh so this is actually the validator I can just run this now and it will say validation succeeded now if I go to the person and I change the age to 25.4 which makes it a float now uh you're going to see that when I run this validation failed error message 25.4 is not of type integer okay this is also the case when I use quotations so if I run this again not of type integer okay okay um I can also you know omit this key so I can just do it like this it will then tell me that the H is a required property it cannot be found it will not tell me that if I just remove the student or is student you can see it succeeded but if I add something else so if I say test here for example 1 2 3 uh then we're going to see that additional properties are not allowed test was unexpected this is also the case if I have is student in here so it is not related to that um yeah and this is how you validate Json files or Json data in general using Json schemas in Python so that's it for today's video I hope you enjoyed it and I hope you learned something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye that

Channel: NeuralNine

Views: 11,215

Keywords: python, json, json schema, jsonschema, python json, python json schema, python json validation, json validation, json schema validation, python json schema validation

Id: o3ViNZpTaKE

Length: 13min 44sec (824 seconds)

Published: Sat Nov 04 2023