Build A Python App That Tracks Amazon Prices!

okay if you're poor you're gonna love this episode so ever since I did this YouTube thing I really wanted to get a new camera and that's the Sony a7 marquetry not that that matters but it costs so much money two thousand euros so the plan was I'm gonna build a small app that checks the price on Amazon and if that price falls down whether whatever I want basically like $1,600 then it's gonna automatically send me an email that's what we're gonna build we're gonna flex our snake skills okay our Python skills and then we can celebrate and we still don't have enough money together to buy the damn product so yeah you can use this all multiple websites whatever you prefer let's stop talking and let's get going okay let's get going so I just have a scrape or dot pie opened up here it's just an empty empty Python script okay so you can do this on your own and vs code and what we need to do is actually install two separate things so we can open up here oh my god this is from the finished project okay so what you can do is say NPM actually NPM is it's not what we want okay we're working with Python we can say pip install requests and ps4 all right I'm gonna explain what these two things do so install those two after you do that we can import let's say import requests and what this basically does is we can get a page we can access a URL so maybe Amazon and we can pull out the actual data from that website okay so all I'm gonna do is I'm gonna say URL I'm gonna set this equal to and the one I used is this from amazon.de so as you can see here I just saw a Sony a7 you can get this one to amazon.de search for Sony a7 I think it's the first or second link if you want to follow along with exactly this yeah and we can paste this back in here and our URL perfect now I'm not every website is gonna work with web scraping some block it some don't this one works so this is the one we're gonna use okay that's the first thing the second thing I want is headers so I'm gonna set this equal to a dictionary here I'm gonna say user agent alright and this is just basically gives us some information about our browser and we need to pass this in to our header so how can you get it you can search for my user agent all right and you can just copy this and paste it in so let me just paste this in here like so perfect that's the second thing I have and now we can actually make a call so I can say page equals two requests all right the library that we're using yet and I can get the URL and the second argument is gonna be requests equal not requests my bad headers equal to the headers okay that's it that's all we need perfect and this actually returns all the data from that website now what beautifulsoup does for us is we can parse it and we can pull out individual items from it so I can say from PS for import beautifulsoup okay now if you're from JavaScript you will do this reverse so it would be imported off from well in Python we are cooler and we do the opposite way okay so once we have that I can create another variable called soup I can set this equal to beautiful soup and passing the page thought content and the second argument is gonna be a string called HTML dot parser alright this is going to parse everything for us and now we can actually pull out individual pieces of information if you want to see what this did actually we can I can show you we can say print soup and there's a method on it called prettify all right let's run this we can call scraper dot python scraper dot pi ok run this take a look prove all my goodness hacker man there's so much it's not stopping it will never stop just like my mental state ok so there we go take a look we have lives here with spans and everything I don't know Amazon's crazy and we're just gonna close this up okay so what we can do is we can go back here press f12 and we can just pull out any data we want I can hit this icon here click on this take a look we have a spam with a product title how can I pull that information out I can create a div here called not a div a variable called title I can just say soup thought find all right and you have a lot of different ones that you can experiment with as you can see you can have name attributes and you can have a class in here if you want search by class and we can also do by ID that's what I'm gonna show you so I can say find IB and set this equal to product title because that's how it's named there now this actually if we print this out title this actually returns me behold div I believe yeah take a look we have a span here and we have D span with the actual information and there's a lot of empty white spaces for some reason Amazon what she's doing I'm not sure okay so to actually get the text out of it we can do dot get text here alright that's only gonna return the text but we still have that problem of hemming having a lot of empty spaces here as you can see so we can actually strip that out as well and how do you do that well my saying dot strip I just said it so there we go let's do another Python scrapper dot pie take a look and we should get back only Sony Alpha blah blah blah perfect now I don't actually need this but hey why not let's have it here what I actually want to have is the price so let's analyze the price click on the price it has an ID of price block our price so that's exactly what I'm gonna do I'm gonna say price is equal to soup dot find I'm gonna say ID equal to price block our price all right make sure you don't misspell this and we can also extract the text here and get text now the problem here is that this is actually a string alright and we cannot really compare the values I cannot compare a number to a string so we need to actually modify this in just a bit so how can we do that well first of all what I can do is I can add a converted price instead that equals you and I can just extract the first five characters are I so I can say price add these break brackets and say zero to five and all this does is it checks the first five so it's gonna be one two three four five so it only extracts this information it doesn't extract the dots ninety nine with the euros as well all right so if we console.log this out friends converted price let's take a look at what we have run this and it's gonna give us back one point nine eight nine all right otherwise it would give us back 1.98 9.99 and a euro symbol okay so that's why we did this so we only have back the first five characters but it's still a string which is not good so what we can do is we can add a float here wrap this round in the float and now it turns it into a float if you don't know what a float is well basically an integer and int is a whole number so like five and a float is like five point eight nine all right so you can add a dot to it okay so we have this back or converted price so what we actually want to do here is something like this if our converted price is smaller than a thousand eight hundred or whatever you want the price to be like that then I want to send an email all right so I'm gonna say send mail perfect so that's what I want to happen okay and we can actually leave everything like so what I'm gonna do is take everything we have up here and wrap it inside a function so I'm gonna say def define check price all right this is just a function and I'm gonna add everything that we have from here inside this function alright get rid of some empty spaces perfect so we need to define the send email so I'm gonna just go down here say def send email it's gonna be another function and we're gonna take a look at how we can actually send an email with the updated price so the first thing we need to do we're gonna do this with Gmail and for this to work we're gonna have to enable the 2-step verification there's another way we can do it the simpler way let me quickly show you some what we can do I'm gonna pull this up here is if you search for something like Google allow less secure apps or something like that let's scroll down here let's secure apps Google account ok so you can enable this one there's gonna be in the enable button and that's gonna work just fine you can use your gmail password and you can send emails just fine if you don't want to use your gmail password what you can do is you can enable these 2-step verification so just Google 2-step verification or whatever it is called so here or the second link alright just enable here 2-step verification and once you do that what you can do is you can google for Google app password alright and you can create passwords for different things I'm gonna sign into my Google account and as you can see you can create one you can hit here select app and you can create a generate a new password for your email for you your YouTube calendar whatever you want so in this case you would select mail and you would select Windows computer all right that's what I'm on you might probably have Mac or whatever I'm gonna delete this old one I had and I'm gonna generate alright so this just generates us a new password that we can use whoo okay let's go back to our scraper and see what's up ok so we actually need to import another package here we don't need to install this separately so what we can do is we can just go up here and say import smtp lib alright SMT flip what this is is basically a simple mail protocol or something like I'm not sure exactly what it's called but it enables you to send emails now let's go down here and what we need to do is actually establish a connection between our connection and Gmail's connection so to do that we can say set up a server here and set it equal to SMT be blip dot SMTP and what we need to pass here is smtp.gmail.com all right so this is Google's or Gmail's SMTP and the connection number is gonna be eight five eight seven all right that's what we need to pass in and then what we're gonna do is we're gonna say server dot hello all right if you don't know what in lo is let me pull it up for you here on the screen basically it's a command sent by an email server to identify itself when connecting to another email all right so it's kind of establishes a connection between D two again so we need to call that after we do that what we need to do is go down here and say server dot start TLS which basically encrypts our connection and then we can call server dot hello again all right I know this is getting pretty pretty jargony but hey that's what we need to do so that's what we're gonna do if you don't remember this you can just always come here and check so next up what we want to do is we actually need me to login so we can call server down login the first argument is gonna be the user as you can see and the second one is gonna be the password so my user is gonna be ad magician I know that gmail.com and the second one is gonna be the password now again it depends what you did if you did the 2-step identification then you can just generate this password which is exactly what I'm gonna do okay so I'm gonna pass this in here we have the connection and now we can actually set up our email I can add a subject set this equal to hey the price fell down and the second one is gonna be a body set that equal to check the Amazon link right and I actually can copy this link down here so our email actually gives us dealing back with the product perfect and then we can set up a message I'm gonna format this string by adding an F which means we can just basically interpolate subject here and just add subject all right kind of like JavaScript with the dollar sign and curly braces and then we can add a new line new line and say body okay which is the actual message perfect and then we can send the email we can say server dots and mail and here what we need to pass in as from as you can see from - and the actual message so from I'm gonna say add magician and gmail.com and the second one is gonna be I'm gonna send it to myself again so - another email I'm gonna say my full name here hi mouth and these third argument is gonna be the actual message which we did up here so from - and the message and finally I'm just gonna print out that hey email has been sent yeah perfect and finally we need to close up D connections I'm gonna call server da quit quit my life okay so that's what we have so now up here what I can say is hey if the converted price falls below at one point seven thousand all right so this is like a thousand seven hundred euros oh the JavaScript then we can send an email that's it that's how we need to do now in this case this is not gonna work because our price is this one is bigger so let's give it a shot boom well what we actually need to do is call this function this check price because we edit it into a function so we're not actually calling anything here as you can see so let's go down here and just call this function okay let's take a look huh please all right so we get back the price and the title because we just printed that out on the screen but nothing really happens we don't get this Hey email has been sent however take a look if our price changes in the future so I'm gonna just add a bigger sign here to make this true all right because this is bigger but you can add any price you want here well now this is gonna be true and it's gonna send me an email so let's take a look boom boom boom come on hey email has been sent whoa let's take a look here let me go to my email and prove prove it to you so I'm gonna zoom in here oh never mind all right hey so take a look boom boom boom email has been sent price has fell down and we get the Amazon link to it so everything works just fine perfect now what we can do is we can actually make this run for longer so multiple times and to do that what we can do is we can just import something this is again you don't need to install this we can just import time and let's see what we do rather than just running this once I can just add a while loop here while true so this is always gonna run like that what I can do is say check price and then I can add a time sleep and I can add the number of seconds I want this to sleep so basically it runs once and then it pauses the execution for a number of seconds so if I add 60 here then it's gonna pause it as you can see delay the execution for a given number of seconds so this would check every minute for you again however that's probably not a good idea so you would want to check this at least like maybe once a day so you would do I don't know my math is really bad so you would do I mean 60 times 60 times 60 would be probably an hour so there you go now you acquired the skill of scraping the web and also sending some simple email so I highly recommend you to try it out with different websites see what information you can get now again not every website is gonna let you do this some have some checks anti web scraping kind of checks with future CAPTCHAs or whatever so not everything is gonna work out unfortunately but yeah experiment and also don't send like a thousand requests to mess around with your server yeah have fun with it I also did one with getting all the recent movies of the IMDB so the latest movie releases and it also saves it down into a file for you a CVS file if you want to see that I might put it up on patreon just leave a comment in the description and I might upload these source files for that as well ok thanks again so much for watching I apologize for no magic tricks for today I'll do this one because just super simple and stupid but I'll catch you in the next one have a great day bye

Channel: Dev Ed

Views: 972,362

Rating: 4.9423995 out of 5

Keywords: python tutorial, web scraping, learn python, python tutorial for beginners, python tutorial for beginners with examples, learn python for beginners | programming tutorial, learn python in one video, python app, build python, build python application, python web scraping, python requests, python amazon, python price track, python project

Id: Bg9r_yLk7VY

Length: 18min 44sec (1124 seconds)

Published: Wed Jun 26 2019