Price notifier in Python with Sending an email (Web scraping project)

[Music] hello and welcome to today's python tutorial today we are going to build a price notifier that is i want to buy these headphones and now the price has dropped to 40 euros from 50. so there has been a 10 euro decrease in the price i want an email to be sent to me automatically i do want to check this page all the time i want to build a program that checks this page once a day and if the price has dropped to less than 40 send me an email so that is what we're going to make and i'm not going to use amazon or ebay or other online store what we are going to learn would be applicable to most online stores so we are going to yet use this website called books.toscrape.com this is basically like a sandbox for practicing web scraping and i'm going to buy for instance this book sapiens so this is a price what we are going to do is we are going to send a request to this url to get a response that's why we're going to use request module then we're going to use beautiful soup to use that response and go through the elements and find which html tag is responsible for displaying this so now if i click on inspect i can see here you see there is a p tag with this class price color which is responsible for displaying this information so whenever this information is this number is below 40 an email should be sent to me so this is basically how it's going to work now before getting to our code editor i'm going to talk about web scripting in python and data camp i have bought a yearly subscription here and i'm enjoying it that's why i'm recommending it so you can go to datacamp using the link affiliate link in the description below and there are a lot of free courses as well you do not need to pay for all courses there are a lot of projects a lot of career tracks you can go and learn from for example there is python programmer our programmer data analyst with python data scientists with python and a lot more okay now let's go to our code editor of choice i'm going to use pycharm we can use sublime text visual studio code or atom now the first thing we need to do in our terminal let's install pip and let's pip install requests requests so i'm going to install this one now that it's done up there let's import it so import requests now why have i imported requests because i'm going to send a request to that url which url where was that this url i'm going to get access to this page so i'm going to create a variable called i don't know re for example and this r is going to be the requests to get a url so we are sending requests to get this url and what we can we get in response let's say if i print let's have it more beautiful print re and if i run this we should see we get response 200 which means a successful status code if you get something like 404 it means the page was not found so now we have made a successful request and response he got a good response now this response is not useful on its own so i'm going to grab the content of this response so i'm going to create another variable like res and that rest is going to be re that we just got and i want to grab the contents of it now if i print res you would see what we get it's basically a long line including all this html that you just saw but in a very ugly format so we are going to use now beautiful soup to take advantage of this response turn it into an html structure and grab that p tag so let's go to our terminal and pip install beautiful soup for so for me yeah it didn't take that long because i've already installed it for you might take a bit more now that we've installed beautiful soup let's say from bs4 import beautiful soup uppercase b uppercase s okay now we're going to make a variable that gives us access to all the good things about beautiful soup let's call it soup normally people call it soup you can call it whatever you want and that soup is going to give us access to everything in beautiful soup now here we get two arguments the first one is the contents that we just got that is in rest so let's just suppose the second one is we should mention that this is going to parse an html page so we say html dot parser because this is what's basically including html codes now we are going to parse it in html now if i print the soup it's a bit ugly so let's just say pretty fine make it more beautiful so now if i run this you see this is basically the same thing as inspect here so this is our soup now so this is going to be our basically our soup everything is here now if we look for the price for example up there it should be somewhere around here um let's see where we can find it so where is the price yes here so this is the title and this is the the price with the p tag with the class of price color let's just copy this class because we need it so this is our soup now we're going to find a p tag with this class in our soup so that we get access to this number so now that we have the soup let's create another variable call it price and that price is going to go and into our soup and find we're going to find one element one price and that's the first one that's why we just use find a p tag not every p tag but the one with the class of price color i remember a class here you can see there's an underscore after it because this is a css class not a python class okay so now we are going into our soup all this code and find a p tag with this class now if i print price what do you think i get let's run this and we get this p tag with this class name but we do not need the html tags so what i can do is here like a price soup after you found it we say dot text that is get me the text of it so let's run this and you see now we only get a text and inside the text this is a string i do not need the uh pound sign at index zero we want only this one from index one two three four five so from index 1 to the end so that is a string remember that's why we use square brackets here and we mentioned from index 1 colon up onwards if i say 5 means up to 5 but if we leave it out it means to the end so now if i run this i get this number but still this is the type of this is basically a string so it is treated as text not as number so i cannot say price plus 2 it wouldn't give me 56 it would just give me an error now because i see because the type is actually a string so i have to turn all this into a float so that i can compare it with other prices like if it gets if the price is less than 40. so to compare these we should turn it into a float and why float because it has decimal numbers that is it is 54 point something so now i can say type of price what is the type of the type of price i don't even have a decent keyboard uh okay so now you can see the type of price is float okay so we can we can compare it now good now what shall we do with this price we want to compare it right you want to say if if the price is less than for example 40 because it's now 50 something less than 40. print buy it but i know the price is not less than 40 so this will not run so just for having an example let's say less than 60 for example now if i run this i get buy it great so it works the thing is i do not want to print anything here i want to have an email sent to me right so for having an email sent to me i'm going to get rid of this and use smtp so we say import sm you can see smtp library so this is basically like a protocol just like http is a protocol for sending web pages across the internet smtp library is also smtp is also used for sending emails now if the price is less than 60 we should configure the email thing so what should happen first we need to establish a connection and introduce uh our domain and names of the servers for example is it gmail or yahoo so let's just say that we have smt as an object we are going to store information here smt and this is going to be smt smtplib dot capital smtp so we're establishing connection here and what is the name of your email provider smtp.gmail.com means you're using gmail gmail means you're using gmail so and then we need to why is this happening like that yeah gmail.com and the next one is the port number imagine think of this as like a street number and think of this as a door number of your apartment so i'm going to use yahoo so i'm not going to use gmail here i would say mail.yahoo.com and this is the port number so we've established a connection using this domain name and this port now we need to say hello to the server greet the server just like imagine you go to this address number to this door and you knock on the door what happened is smt that the object that we have dot a low so we are going to say hello that hey hello basically it's like extended hello or something like that so we are going to greet our server once we have done that we are going to establish trust security and encryption make sure no one's listening so i would say smt dot start tls so it's going to start a security security measures for encryption once i've done that i have to um make sure that i have the right id that is login information for my email so again we go to this address we knock on the store we say hello we make sure no one's listening and then we introduce our id so we will say smt dot login and here we get two parameters one is your email address i'm going to use my yahoo email address here and the second one is the password to your email so for yahoo and for gmail is a bit tricky because you don't want to use the password to the whole google account that's why you can go under your google account you can find generate app password i'm going to do the same thing with yahoo now i'm going to show you how so i'm here in my yahoo account and under account security you can go down below and find a password the same concept is also in gmail and here if i generate this i already have one so i can delete this one and then i can produce not here i can produce another one and i will delete it later so let's call it scraper generate and this is going to be the password to my app that is yahoo email let's copy this and go back and insert it here now that we've established also our credentials it's time to send mail that is from who to who and what is a message so smtp.send mail it takes three arguments first one from who well it's going to be from me to who comma at the end to me again but another point email right python yahoo.com and then the last one is the message the message i'm going to use this price variable that we got inside the message that's why i'm going to use f strings so f and the first word should be subject and what is the subject let's say price notifier something like that or headphones price notifier and then two new lines so backslash and backslash n means two new lines and here is going to be the body so hey price has dropped two and curly braces price so price has dropped to price why is it in curly braces because this price is different from this price this is like a value that i get from here so the price has dropped to whatever price go buy it something like that buy it okay now once i've sent this email i should make sure that i quit and close the server so i would say smt dot quit that's it now that we've done everything okay let's check it if i uh now run this program yes let's see everything's fine yeah i guess so okay if i run this and let's go to my email nothing is here so let's see if something happens so let's refresh this and yes i got it headphones price notifier hi price has dropped to this price buy it great so we've done that but there's an issue here let's delete this the issue is i don't want to manually come here and run the program every time i want this program to run let's say once a day and once a day you should go all over this again and check if the price is lower and then send me an email that is why i'm going to use a while loop so let's just use a while a loop here and say while true and true is always true so this will going to this is going to run all over and over again so let's press a tab but i'm not going to run this because this will be sending requests like crazy to this url and send me also an email like crazy i want this to run once once a day so that's why i'm going to use import time module now that we imported time module here let's go down below and say okay why when everything is done like this it's time to sleep for how many seconds if i say 5 it means 5 seconds so let's just try this for five seconds now and let's see if i get an email every five seconds so let's see if it's correct yeah everything should be good okay i'm going to run this now and let's go to our email so let's check once again so this is one let's say in five seconds what happened let's see and another one so every five seconds this is going to send me an email let's let me stop this here this is going to be crazy so instead of five seconds i want to say 28 20 well four for 24 hours multiplied by 60 for a minute that is 60 minutes in an hour by 64 seconds now this should be basically the number of seconds in a day so that means when you've done this go to sleep for one day and then wake up again and do this so this is basically um it yeah so this is basically it so again we've imported requests and beautiful soup to send requests and get to the price we've imported smtp library uh to send emails and we've imported time to set a time to sleep otherwise this will run all over again like crazy and we don't want that so whenever you have a while loop you should make sure that you stop it at some point you break it at one point and for us that break point is this time here it's going to sleep for a day and then it will check again the price and if the price is less than 60 then it will send an email and the price in our case is of course less than 60 is 50 something but let's say if we put it to less than 40 then we can make sure that whenever the price the price gets less than 40 there will be an email sent to us okay i hope you liked the video and i would really appreciate it if you like the video i mean press the like button if you want to share it or subscribe to my channel and thank you very much for watching and listening [Music]

Channel: Pythonology

Views: 11,401

Keywords: webscraping, python, smtplib, send email python

Id: yQiLr4sB3Gs

Length: 20min 40sec (1240 seconds)

Published: Thu Mar 03 2022