Parsing data with Pydantic - A Python short by Michael Kennedy

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
You've got json data. Maybe you got it from an API maybe you got it from a file. How do you parse it. How do you make sure it's valid and how do you convert all the data types? sounds complicated with Pydantic. It's easy and we're gonna see how in this video. Let's imagine you're a motorcycle enthusiast you love to go riding so you get a job at a company like REVER building apps where people can record and upload and share their ride's chances are a lot of this is being exchanged as json data. Maybe your job is to integrate with some partners so that they can upload this data. well how do you know it's going to be accurate. How do you know what you said your API needs is actually what you've received sure you could do that by hand, but we're going to see that with 'pydantic' a cool new python library we can make this very much automatic and at the same time, our editor will actually help us write better code because Pydantic uses Python types let's jump over here to PyCharm and fire it up so we have some data, this is some sample data that might be submitted for a user over in some kind of app like our 'Rever' app. Let's go create an app and we'll just call it receive data and we're not going to talk about loading this data just yet. We're just gonna come up here and put it into our code normally you would get this from an API As I said. So we want to parse this data now if we look at it. There's a couple of things that, maybe will stand out to you. For example there's an age 28 who doesn't want to be 28 right now that's a number except for it's actually a string here. It might look like a number but it's technically the data type of string in Python it's going to treat it super differently we've got our location. Do we need to specify the country I don't remember and down here we have rides like maybe these are recorded rides and just the index of them well that one. What was that who typed that in that was weird, so there're some problems with this data, we would need to validate all those types of things, but it also turns out it's probably okay right, we could see that. well that really just needs to be parsed as an integer and same for that other other piece there. How do we deal with this. Let's go and create a class we'll call this 'user' and in here we want to give it basically the information that represents this data, so it's going to be a name and in Python we can say this is a string like this and it's going to have a location. We have no idea what the location is going to be. This is another class we'll deal with that in a second we have a bike, which is going to be a string and we have rides, which is a list of 'int' these are all great and maybe if you don't have any rides. We'll even set it to be empty like this. Now let's work on that location real quick. We'll have a location we spell it right and it's going to have a city which is a string. It's going to have a state which is a string and it's going to have a country which is a string. Although we didn't get, that did we this is a cool way to define classes in python but it doesn't allow us to do that validation, but here we can actually be a little more specific it doesn't allow us to do any sort of parsing and validation it just when we program against it lets us work with these types right, for example: if we have a user and we say 'u rides.' you can see we get all the list stuff. That's great but we want to use Pydantic how do we do that first thing we have to do is install it we have a virtual environment active, actually let's put it in the requirements file first I'm going to put it over here into this requirements file so we can pip install - our requirements like that wait a moment and now we have Pydantic so up here what we're going to say is that this is a base model. Now this comes out of Pydantic finish parsing things and say we need to import this up at the top here and the same thing right there now when we do that we're going to be able to come down here and say well now that we have this data we have this user we're going to be able to say user and just pass in the data by keyword argument and in Python, we do that by saying '**' a dictionary that turns it into things like name=michael and so on so we'll say '** data' like that now this is not really going to work out because this says the country is required and the country is missing, but let's go ahead and give it a try and find out what's happening so we're gonna go run this look at this. We don't just get uh some kind of error saying data is missing. No we get specific errors like the locations country is missing. How cool is that? well one thing we can do is we could set this to have a default value and then it works because it's missing, but there's a default so we'll just use the default. The other thing we could do here is we could say the country is actually optional. This is a Python type thing but Pydantic reads that and says well if it's optional. It doesn't have to be there but if it is there, it has to be a string so now again it runs. Let's go ahead and print out this like so we found a user Michael location look at this city is Portland state is Oregon country, none the bike down here is a KTM and wait a minute. Check this out. Look at that 70. Remember, the 70 was busted the 70 was in quote but Pydantic said you know what yes it is a string, but it's parsable to the data type that we actually said. which what we said was a list of integers so it saw strings that i could make that an integer successfully so it it tried and succeeded same thing for the age if we would actually remember to put that on our user. Let's say age is an int and we run it again. Now our age also gets parsed over so cool so here's the story if we have invalid data or missing data, we get meaningful errors like we put this here. It'll say you know what the age is not a valid integer right if it's parsible to an integer, we're good to go but if it's not then Pydantic will give us meaningful errors so all we have to do in order to work with Pydantic is have some json data create a class and these can nest together as we see here and if we put none, that should probably be optional but we put in the types the fields and the types, including the nesting, including stuff within lists and so on and then we just load up the data by keyword argument, the easiest way is to stay ** dictionary and it parses it converts, it validates it and then we have a class that we can work with like user.rides and that we know this is a valid list of integers Fantastic. That was simple, wasn't it incredibly easy to use Pydantic to parse and validate data that we get from other locations in any json format as a Python dictionary effectively so hopefully you enjoyed this video. If you did please click on my face up there and subscribe to the channel also like this video. It really helps support our work. I, actually interviewed the creator of Pydantic over on 'talk python to me' so check out the podcast episode there and we have a whole class on 'Fast API' and 'Pydantic' at talk python training so many more things to explore if this got you interested in Pydantic. Thanks for watching, catch you next time.
Info
Channel: Michael Kennedy
Views: 24,168
Rating: undefined out of 5
Keywords: Python, pydanitc, json, pythonshorts, kennedy, talk python, education, learn python
Id: aHv7-6WIxNM
Channel Id: undefined
Length: 7min 46sec (466 seconds)
Published: Fri Jan 14 2022
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.