Pydantic V2 - Full Course - Learn the BEST Library for Data Validation and Parsing

penic is one of the most important if not the most important library for data validation and passing in Python its main advantage lies in its strict type checking while python is traditionally known as dynamically typed language pantic allows developers to Define clear and secure data models this eliminates extensive error checks and boilerplate code making the development process far more efficient and reliable in this video I want to show you how penic works and show you everything from its basic to very Advanced features you can code along with me and find the link to the code in the description okay I'm here in vs code and as you can see there is a code. IPython notebook we first have to install penic you do it just with Pip install and then penic if you already installed it make sure that you use a version with a major number of two penic to was Rewritten in Rust at least its core package its data validation package and some of the syntax was breaking so not everything might work if you use penic one so you can check it if you if you run the first line of code I can see I use Python 2 uh penic 2 3 0 and yeah just make sure you've got two as first number here so first let's check an example with data validation without penic so if you write larger code bases you normally want to make sure that a function or a method receive the data type it expects in other languages like typescript or Java you can use that kind of type annotation to make sure that the data type which is passed to a function is always correct and the compiler will check for that in Python that's not possible because it's dynamically typed but you can check it like this so for example if you want the ID to be an integer you can use the following syntax if not is instance you can check if the ID is an instance of an integer and then you can raise a type error but using that kind of type validation is pretty hard as you can see that works I create a user with an ID of one to three with a string and we we get the error expected ID to be an INT gut string so this was correct but of course I don't want to write that kind of code myself ptic can help us with that kind of task so now let's do it with pantic and in pantic you use the following workflow from pentetic you import a base model and then you define your own class and this class inherits from base model so we define a class user which inherits from base model and we now set properties like like in a python data class so you set the ID and then use type annotation to actually set the data type so we want an ID of type in and we want a name of type string and we can also set the default value with setting an equal sign here so that's a very basic base model or custom model which inherits from base model so let's run that code and now what we do is we use an ID of one to three and as you can see this works if we change that to a string then we receive an error so this is the type validation you get from pantic input should be a valid integer unable to pass string as an integer as you can see it tries to par the string to an integer that's the normal behavior we will take a look at a more at a more advanced feature which is strict mode but this also works so it tries to convert that string to an INT and if that's possible it will also work so we can see here it prints the ID as an integer penic classes come with some helper methods and properties we can use to analyze our models so for example we can use the model field set property and if we take a look at that we can see this is our our first model here which we defined here but we get a default with chain do so we can see the first model only contains an ID so the default values are not printed in model field sets if we create that class again and use another value for the name so we don't use the default then we will see it in the model field set property we also have some nice helper methods model dump model dump Json and model Json schema so let's have a look at them so the first one Co converts your data set so your model to a dictionary the second one model Dam Json converts it to a Json object so this is something you can pass through an API for example if you want that and then the model Json schema is a model with more information so you can see properties U first one is ID you can see the title and the title is ID and the type of that property is an integer you can see a lot of more information in which one you need of course depends on your specific requirement so now let's have a look at nested model so so far we only defined a model that has basic data types like a string a float or so on and we can also Nest models so for example we've got a restaurant and the restaurant has got different kinds of foods so we inherit from base model and our food has got a name a price and some ingredients for the ingredients we use other glasses from typing tool so for example we make this an optional property and we want to set it to a list of strings so the ingredients are of course a list of strings and this is the way we Define it so the default value for that is none so we don't have to set it and if we don't then we actually will only have a none value here and now we can use that food model in another model so for example we've got a restaurant which has got a name and a location and we want to have our restaurant uh list of foods which we can serve so we Define it like this with list and then in Brackets our model so we Nest our model our food model now into the restaurant model and we can set it up like this so we Define an a name and then a location like before for the user but for the foods which is the nested model we can use a dictionary so this works the same so we set it here and the name is cheese pizza we've got a price and an ingredients so this is a list of strings and for our second food we've got the name wedgie Burger which has got some specific price but we don't actually set the ingredients here so this works because we've got this ingredients list as an optional argument here so we can now try to set it up and if everything works we can see this is our model and we can also use model dump to dump a nested model so as you can see this is just a dictionary representation of our model now pentetic also provides some more advanced data types we can check for for some we have to use additional sub models so if you want email validation we can install pantic and using brackets email this is not installed by default and if we do it like this we can now import the email string class which is like I said not part of the basic penic um library and we can also use other models so here are a lot of different classes which we will use in a more advanced model like positive in con list field and HTTP URL so now we create another restaurant and we use an address here which is inherits from the base model this is nothing new it only contains strings the employee has got a name a position and an email and now we use the email string here so this validates emails like an add symbol does it contain a.com or dode at the end so this is all handled by this email string class we don't have to use some regex pattern but we can actually use a regex pattern and we can do that if we use the field class so the field class allows us to be more specific for our data type so for example here we can say our name of the restaurant has has to be of type string and these ellipses so the three dots tell us that we have this as a required um property and that the pattern of name is from A to Z and only is allowed to contain numbers so we Cann not have back slashes or dollar signs or everything else inside the restaurant name we will have a look at field in a few minutes a little bit more in depth so for now that's okay so what's new is also the con list con list is a function also of pentetic and this allows us to be more specific about the employees so now we can say instead of we want a list we can say that we want a list of employees but we want a Min length of two so that comes with con list you also can say that we want the number of seeds not to be an integer but we want to set it as positive integer so this is under the hood handled like this and now the number of seats cannot be negative for the website we use HTTP URL so if we provide an URL which is not valid like we missed the.com at the end then we will also raise an error so now we can instantiate it like this so maybe let's now comment the first employee out so we can check that con list function and now if we instantiate our class we can see we get an error and here the validation error is list should have at least two items after validating not one so I think the arrow messages are very helpful so if anything is wrong in our document model or in our restaurant model we will get the information pretty clear so now it works and we can see now we've got our restaurant instance okay now let's have a look at field validators so before we used the builtin classes to validate data like for example here that con list or positive int but we can also so validate the fields so the attributes of a class on its own so to do this we use the field validator decorator provided by pentetic and then we have to pass in the name of the property we want to validate and then we have to set up a validation function we will have to write a class method to validate that kind of property because this happens before we create the instance of our owner model so then we Define a function and the first argument is CLS so that's the class itself and the second one is the argument we want to evaluate so we want to evaluate a string and return a string here so now we can check if there is a space in our name and if that's not the case then owner must contain a space we can raise that as value error and also we can do some data manipulation here and return this data string so the name String and we can make it uppercase or title case or whatever we want so for example let's do it uppercase so this is just a string method because here we Prov provid a string so we can call the upper method of that so let's now use John do and don't provid a space then we can see that our model should not be created validation error for owner and we can see that now we can see our custom value error owner must contain a space so if we write it like this then it should work and we also should see the name now in uppercase so this is very cool because we cannot just validate our data but we can actually change it to be correct so if we provide data in lower case we can set it to upper case or like before we can set it to title case and I think this is a very nice feature of penic okay now let's check out model validators so what are model validators and what is the difference between a model validator and a field validator so the field validator is like its name states there for validating a field and a model validator is there to validate a complete data model so let's have a look at the first model validator we can see that we've got a mode before and in the second model validator we've got a mode after and first let's check a little bit of the logic here so now we check if the data we provide this is the data we pass in here is an instance of a dictionary and this is actually done because this is of the because of the inner workings of python so in Python the attributes will be stored in a so-called class dictionary and this is the reason we want to check for dictionary so we want to see if we provide a password in that dictionary or if you provide card number in that dictionary and then if that's the case we want to raise a value error password should not be included or card number should not be included otherwise we will just return the data so the data is all of this so what is data data is everything we provide here as attribute in the Constructor phase of creating the object so now let's go back to the mode before and after so before means that this function will be run before actually checking for example for the data type which has to be a string and so on and so on and after means that this function will run after the data type here is checked to make it easier to understand I think we will just run the code and then see what happens so we've got that owner and that owner has an email and a password so this is currently a valid email so now let's break that so now the Doom is missing and this should be validated um and yeah create an error and then we also provide that password as you can see and you can see the mode here is before so this will raise the error before that email will be validated so now let's try that and as you can see we've got a validation error value area password should not be included so this is the first error and if we delete that now I think everything should work or of course not work because actually we want to raise that error I'm currently not too sure why we get this error because normally type conversion should be possible this should be easily converted into a a string let's do it like this so we can only see the email error and yeah now that works fine and now let's set the name to maybe John do and as you can see we still only get the email validation error that's because of the mode so the space will only be analyzed after creating the instance of that user so we can actually see this if we analyze the signature and here you can see this is a class method so we run a class method before the instance of that object exists and here this is a normal method and we can see we we don't have CLS as first argument but we have got self so we only got self if we've got our instance so okay maybe let's just fix it here and make that to. and see that the value error or validation error works fine we can see that now we've got our validation error own a m container space okay as you can see okay I can delete this this is nothing new and now we will have a look at Fields you've already seen the class field but we can do a lot of stuff with it so the very basic usage of field is nothing new to you use the field and then you provide the default that's the exact same like this so you provide a name and set this to chondo so if you only want the default I think this is overkill but you can do a lot of more with that field class so let's just run it to see that it works and as you can see it works fine so with field we can make our classes a little bit a little bit more complex so instead of using just a default string we can use a default Factory and a default Factory is able to run a function so for example we want an ID of a user but we want to create that ID on the Fly we don't want to set a fixed value when we create that instance but we can use a function and run that function to create a new ID every time we run the Constructor so as you can see we print the IDE and every time we run that we get a new value so if you want to have a dynamic value then we can use a Lambda function to create that on the Fly okay now let's have an even deeper look at fields and Fields allow us to create aliases so if we create a new class with an attribute username we can use the Alias to make it work so we provide username and set username to Chon do so if you run that and now have a look at our model we can see that the name is actually the name we provide here as attribute but under the hood that that attribute name still gets saved so we can uh use model dump and use by Alias and set it to true and then we get this as dictionary and this is not name but username if we set it to false we can now see that it should be name instead of username so this might be very helpful in the case you've got your data model in for example an SQL database with SQL aremy and uh an API and the attribute names differ so you can use ales to exchange that kind of names very easily so now let's have a look at field constraints so uh Fields allow us to do multiple field constraints so for example we've got a username and we want to provide a string but now we can use a Min length and set it to three so this is a little bit more specific than just using specific classes here like email string which does the validation on its own but we can't actually provide a lot of custom functionality in the field we can set it to a Min length of three and a max length of 10 and also providing some raex pattern to not allow any numbers in the name we can also find integer set it here with greater to zero and uh smaller than 120 for age for data types like decimal which is for for money we can set the max digits to 10 and decimal places to two so we can use a lot of different stuff here with the field class okay now let's actually use it so let's run the code here and then create an instance of that user and as you can see the name here is wrong so this are more than 10 characters and now we run into a validation error and we can see yeah that doesn't work so now let's try to change it and set it only to John then we will get no validation error so this is very helpful if you want to provide some custom constraints in our Fields okay now let's have a look at computed Fields computed fields are very helpful if you want to calculate a value dynamically so for example if you want to check whether a person is older than 18 or younger than 18 and that is of course not a fixed date but actually changes every day so now let's set a property here which is birth year and based on that birth year we want to create a new property called age and this is a computed field and inside we can do some calculation so we create daytime now so this gives us back the current date and we can extract the year from that and now we've got the current year and now we can calculate the current year minus the birth year and now we get the value if that's larger or smaller than 18 so if we run that we can now see that if we set birth here to 2000 we can see that we have the name we've got the birth here and we also got age so this value is of course able to change even that stays the same so here's just another example and we can use a field validator and then calculate that on our own so we can actually create that and calculate that on the fly but we cannot do the calculation on the Fly here on that computed property so we have to do do it on the original Valu we have to set it for the birth year and cannot do it for age but we can combine it and then if we create a new person if 2,000 as set the bir here to 2,000 then it works if we set it to 23 we get an value eror because the person is not 18 years or older so now let's have a look at data classes data classes are built in into Python and look very similar to the classes we just defined but data classes do not provide that data validation which we get with penic and of course if you want that validation and also use data classes that would normally not work but penic provides some kind of functionality which allows us to do that kind of uh data validating so first we uh import from data class the data classes decorator so we can use that syntax here to and here we calculate a normal field so there is no data validation in it but we can also use the field class from identic and here we can actually make that type save so if we would provide a value for height which is larger than 300 then we would get a type error normally with data classes this would not be possible if you want some kind of other functionality like the Json schema for the data class you also would normally not have that but pantic provides a so-called type adapter we can provide a Class A data class for that type adapter and then run Jason schema on that so if you run that we can see okay that does not work because we actually have to Define it of course so let's now run it again and as you can see that works and we get our Json schema despite normally that's not possible if you get rid of that we can see that this is not possible if we just get an error but with a type adapter that's possible so to be honest in general I would recommend not using data classes in combination with pantic so in general I recommend using the base class and inherit from that okay now let's have a look at the last feature and that is the strict mode maybe you can remember the very first example which was user and we provided in the Constructor an ID with a string 1 2 and three and and it worked it didn't create an error and that that is because pantic tries to coers the data values and if it finds a string that can be converted to an integer everything works fine normally here we do the same functionality like in the Constructor but we can use the model validate class method and provide a dictionary like this and this the same like using the Constructor but this is just for validating a model so we can provide an ID and set it to 4 to and set the username to chono so this is the interesting part because we don't provide an integer here but we provide a string so if we run that we can see that this is now an integer and this is because of type coercion which is the standard uh used by penic but we can use the so-called strict mode we can also use it in the Constructor and if we do that then you can see we've got a validation error input should be a valid integer and if you use strict and provide a string then we will get an error and pantic will not do or try type coercion okay after having a look at pantic models we can also use penic settings and pantic settings work pretty similar based on a syntax like base models but are there for loading settings or configs or environment variables and this is very helpful to work uh with that and now we can see that we instead of using the base model we import from poentic base settings so we have to install it first and then we also inherit from settings so the syntax is actually the same we can use the field class and so on and so on so very similar but yeah a little bit different also too so I will show you what happens now and as you can see what we've got here is an attribute or key and API key and if we create that instance and we don't provide anything it will actually get the values for for the environment somewhere and where does it get the values from it tries to get it from the local environment and if there are no values then it will look per default in a end file so we can see everything worked and this is because of the do n file if we have a look at it we can see that there is an odd key and there is an a value or a key my API key and we can see that is n or key and N test and these are the values provided here so this this is how base settings work out of the box but it of course provides a lot of functionality so let's have a look at a more complex example so in this example we want to provide the environment variables by setting it in the code itself so we don't want to get it from the end file and if you run that again we can see the following as you can see now we've got the service name this is default we don't have to create or set an environment for that if we provide a default value it will just work the or key will be used here from that value so we've got test or key and if we we've got that then it will not read from the end file so we now get our API key which is test and the URL is my super URL as you can see so this overrules our end file so also very important to know and what is also new is that validation Alias so we can use Alias choices and this will evaluate from left to right so if we don't have an N1 here it will look for N2 so we can use a default value and if that's not provided we can use a fallback okay now let's have a look at another feature and this is a model config so we can use the settings config dictionary this is a class we can import here from poentic settings and we can set an N prefix so if we set this n prefix for example to production so let's say you've got a development environment and a production environment so this prefix allows us to change the environment variables based on its prefix so if we set it like this so we've got production or key production API key and production N2 now if we instantiate that we can see that the values in the settings are still service name or key and also API key is not test so if we want to change the production and development environment we can just make that on the Fly and the property names are not affected by that change in the environment variables name so this feature allows us to work with different environments pretty dynamically okay now let's have a look how to work with do end files a little bit more specific before we also worked with a end file but everything was handled like Black Magic by penic so now we can use the settings config dict class and provide an end file name here so if we provide a wrong value then it will have a fallback and the fallback are the environment variables we set here so these were set here inside that kernel so we can still use it and now we try to run that and if we don't see that file it will use the default values but if you change it to correct values it will override the provided values so now if you run that we can see that we get an error because it tries to read that values from the end file as we can see we've got service name or key and API key so something is missing here and here we've got or key my API key and that N2 and that's inside and that's not part of that uh settings config so we can see it complains one validation error and two extra inputs are not permitted so if we remove that or comment that out we can see that that our model now works because for the service name we provided a default value that behavior that complaint here if you comment that in again we can see that extra inputs are not permitted but we can also change that behavior we can do it by this extra attribute so we can set extra to allow so if we set it to allow we can now see and if we set allow and if we set it to allow we can see that our base settings now also contain this N2 value so this is not what you normally desire and I think the best solution is to use ignore here so if you've got extra variables which are not part of the attributes here in that class you can just ignore them and then you won't get an error but you also won't add some extra variables to your class so I think this is this is the best way and this should be the default for using a you should have very specific reasons okay that's it thank you for watching this introduction to pantic if you like the video feel free to subscribe to my channel and give me a like thank you very much see you bye-bye

Channel: Coding Crashcourses

Views: 4,895

Keywords: pydantic, parsing, validators, validation, pydantic v2

Id: 7aBRk_JP-qY

Length: 30min 9sec (1809 seconds)

Published: Tue Oct 31 2023