Parsing JSON Using Python Requests

today i'm going to be talking about javascript object notation otherwise known as json json is a data format that is commonly used in rest apis that usually sit in front of databases on the internet so when you're trying to get data from some service over the internet it's usually through a rest api and the most common format used to return data from that api is json so in python the most common way to interact with the rest api is the request library which comes with support for parsing json so we're going to take a look at that and we're also going to take a look at the standard json library to see how we can do some additional functionality with json to make our lives easier so let's get started so what i did was i searched on the internet for a public api that was free that returned json and i got this really cool list on github of public apis in here i searched for well i didn't search for but i found one related to songs called searchly and it says it supports similarities search based on song lyrics so i thought that was pretty cool so i came over here to the documentation and i took a look at this bit of code here i'm going to start a new pycharm project and pycharm is pretty awesome because it automatically creates a virtual environment which is what you need to separate your dependencies your libraries that you install or your packages that you install when you're executing python code so that way you can take that code and move it to a different machine without any issues so it created this python project folder and you can see it has this bit of code here in order to run that there's a couple different ways we could do it we could say python3 and run it like that and say hi pycharm however i like to run my python with just a dot forward slash this fails though because it says permission denied so you have to do a change mod and give it execution permissions and then you'll notice my terminal made it red now and now when i run it error why did that happen so that happened because by default the uh shell uh bash or zsh in this case is going to try to interpret the python code itself as like that as like bash shell shell scripting it's like no that's not going to work so what you have to do is you have to go into your python file come up here to the top of the file and you have to tell it user bin environment python3 to use the python 3 interpreter so now when i run it boom hi pie charm so that works as expected that's what we want however this is not the code we want so i'm going to delete that out and again i'm going to grab this code over here if it's not in my clipboard and paste it perfect so now you notice right away my editor tells me hey request is not installed before i install requests i should source this virtual environment in here oops that was created by pycharm and this is how we do that vm's bin activate there you go and you'll notice my shell tells me hey you're now inside this virtual environment so now we need to do a pip install request to install the requests library okay that installed now that that's installed i should be able to run this code over here and get no errors let's see cool so that runs without errors it also isn't doing anything so what we want to do is just like in the documentation well first i should explain there's a url for the the part of the api we're going to hit and then there's a payload that we're going to send the api and so query is going to be cyrus for miley cyrus which follows the documentation example and then here we're capturing the response we're saying request.get this url passed the parameters this payload okay and then it says the response is formatted in json response response equals response.json so this response.json takes with the api or what the url returns and loads it into a dictionary and we're going to see that we're going to print out the json response okay so now when i run this dot slash main.py boom wall of text but you can see in here you're getting some some songs about from miley cyrus so um that's not very helpful to look at and that's typically the first thing you're going to see when you deal with an api is like you get some wall of tax response and you're like that is that's not helpful so how can we you know start working with this data a little better the best way to do this in my opinion is to import json at the top and then take that response and we actually have to to reconvert this to json and i'll show you how we do that so we take the response and we say i want to print out json dumps the json response okay and i'm just going to comment this out for a second the old one so now i'm going to rerun that and now it's printing out valid json what's the difference the difference is this the key in the valid json has double quotes if i go back comment this out and comment this back in where we're just printing out the dictionary you'll see that the key has single quotes around it why does that matter why why do i care about having valid json right now well here's why there's a trick that i love which is to use something called json tool to pretty print the pi the json response so if i try that right now it's going to fail and it says expecting property name enclosed in double quotes so uh shocking since i was just like telling you about this so now again if i if i json if i do a json dumps that sort of like serializes the data to be the correct format um and you load this dictionary json response and now it's going to print the correct format and then i'm piping that output that's what the pipe here does to python dash m json tool and by doing that look what happens boom very pretty sort of hierarchical structure here of the json data it's tabbing and doing all kinds of great things so i'm scrolling up now because we need to sort of like understand what the json response is so we can parse through it so for example there's an error key so typically when you interact with an api you're like was there an error so you'd say if you know json response key equals error true if there was an error then do something with that error but in this case we know there's not an error because we could see that it successfully pulled some response results so what we do want to do though is now that we kind of have an understanding of that data structure we want to kind of pull out the songs themselves so we're going to say four songs in what do we got to do here so from this top level dict dictionary we're going to pull out the response key so we're going to say json response and the key we want is response we can use single or double quotes here doesn't matter and just to prove a point i'll do it and then we're going to want results so the next key right and then you'll notice that that key is a is a list see that that indicates that it's a list so that's why i'm doing a loop here for songs and actually um technically i'm gonna get a song at a time so i should do four song right because inside of this list we get one of these objects per iteration we're looping through where that comma is and so we're saying boom for each song you have an id and a name so now in here we could say print song id comma song whoops shoot name okay let's run that and see if it works we don't want to run it with the python-m json tool we just want to run the regular code now so boom so now we get a printout of ids and the artist name and song name which is included in the in the full name response so now just to do something a little cooler we could say if song id is greater than um let's take this one the id of 68 680 print it out let's see what we get all right here we go we got four responses and i'm really sorry for that little bit right there not sure how that got in there um that's funny we love you miley um so uh the thing i want to point out here though is that this is an integer right i'm not quoting this in any way so that's kind of what indicates to my interpreter that it's an integer and you can see it's highlighting it there in blue and then i'm saying greater than this response over here or rather this response is greater than this number over here and that's really important because python is doing something here that's kind of interesting it's uh it's using its duct typing which is to say like it looks like a duck it quacks like a duck it's a duck to say hey you're trying to use this this actual string because this is technically a string not an integer that's being returned from this api because if you look at the response i am lying to you right now i thought it had quotes on it okay i love it when i make mistakes that's what makes these videos fun but anyway it actually has it actually is an integer that's returned there because they're not quoting it in any way but i just wanted to prove a point like so let me prove that point real quick because i think it's an important lesson if i did quote oh god i can't do that sorry if i put this as a string if i typecast it to a string and i tried to compare it to this integer it's going to fail and actually you can see my interpreter is actually already telling me that so you get type error a comparison operator is not supported between instances of string and int right but i could like make that a string let's see if that works i think this will fail too oh wow that works you could actually compare with strings i didn't i wasn't sure about that because you know generally speaking you wouldn't want to compare numbers as strings you would want the integers to be compared which is what's happening here and that works out whoops just fine whoopsie as we showed before so the other thing i wanted to show is like if you know just to get freaking crazy make this a string oh actually let's do it the other way let's make it a string like this okay so now that's a string right this should fail with the type error right but now i can like re-type cast it into an integer and voila that works now this is one of the things i love about python but this is one of the things that drive other programmers absolutely insane who are used to typed languages or strongly typed languages so that's as far as i'm going to go in this video but i hope this was super helpful for you for a javascript object notation and i hope the nugget about duct typing was also helpful see you in the next video

Channel: jasonriedel

Views: 391

Rating: 5 out of 5

Keywords:

Id: FCSFtkKYdhg

Length: 11min 57sec (717 seconds)

Published: Mon Nov 09 2020