Python: Real Time Stock Price Scraping and Plotting with Beautiful Soup and Matplotlib Animation

[Music] in this video in the first part you will learn how to scrap websites using the beautifulsoup library to be specific we will put real-time stock data out from Yahoo Finance website in the second part of this video we will learn how to plot these written later with the use of animation from matplotlib after the video you should be able to pull any informations that you want from the website and plot the live data in real time we performed at her son's task it's often that we want to use data fun on the Internet sometimes we will be able to assess this data in CSV format or via an API however there are times when the data you want can only be assessed in as parts of a webpage in cases like this you will want to use a technique called a web scrapping to get the data from the web page into a format that you can work with in your own analysis to make such requests in Python we can use surfer HTTP libraries like HTTP lip URL leap cress to me the most elegant and simple stuff above mentions libraries our request to so to request the response from surfer there are many to Memphis get to request the data from the server host to submit the data to the server beautiful ship is a library that make it easier to scrap information from web page by combining the beautiful ship and get requests together we can easily extract the data it's that we want first thing first we need to import to library that is the request and also the beautiful ship just mentioned so first of all we need to get the URL of the websites that contain the data that we want and then we can simply you say be cress function and enter get a URL to extract all the information from this website let me show you the result here you can see that there are lots of the information but in xtml format so in order for us to get the results or extract the data that we want so we need to use the beautiful strips to help us to do so so let's move to the website first of all we highlight the data that we want to download and then we right click and then click inspect so here we inspect the element on this page and then we go to find the body tag that contains the data that we want and notice that this diff right here with the class items is what we want so we double click it and then copy it once we copy the class name under the div tag right now we can work with these beautiful shoe so first of all we put the web content into the beautiful ship object first here we the web content and then we use beautiful shoe unshin XML and then once we aside this web content with the use of beautiful shoe right now we can use the web content or we can use the beautiful ship functions to find the data that we want and we use the fine function at least web content dot fine and then Tagg that with the class coast to a name that we just copy from the website here we use the five function but actually under the beautiful ship there are two ways for us to find the objects to find it tab objects to be specifics so you can either use the five functions to find the first and next match and one by one and then or you can use to find all functions to find all the matches no matter we use the find function notify all functions normally we need to input two parameters into this function the first parameter is the objects that you would like to locate and the second parameter is the attributes that match with these ten object so combining them it means that we would like to locate that if ten objects that with the class name specified here here after the class we have in underscores because class is a keyword reserved by Python for defining classes so just in any case if you want to pass in more than one parameters here all you have to do is to make the second parameters as a dictionary of the arguments you want to include so say for example just change it to a dictionary by adding curly braket you so let me print out the web content here to see what we have so right now you can see that we just located if objects that with the class name echoes to this one so the thater that we want to locate is hidden inside these defects right here right here so we need another fine functions to locate this pan so here we just at another fight function that locators been you so to pour out this value we just need to na doc tests at the end cool so now we can put everything inside a function in order for us to use that you know Tom price have the stopcock you and we modify a bit of the URL in order for use to input even stock code you and we just return the value so let's see II and work it out a wee that's pre no there manual you because the website is keep refreshing so sometimes we might get a null value from the website so in order for words to avoid this situation we can just assign another values to this web content say for example we just società say nine nine nine nine nine here so later on when you perform the data analysis when you come across this nah nah nah nah nah you can either use the interpolation functions to replace the value or use other methods to do so so the next things that we wanted to do is to use a for loop or a while loop to keep pulling the data from the webpage so let's say for example here we just step in range say for example 1 to 101 so we have a hundred steps and then we just assign a priceless in the column is here in order for us to for the price here and also without thee everything's inside a columns and then we will just a pan it into the data frame and then finally it's saved in a CSV file in order for us to do so we also need to import the Pender's et and also in Part D each time but they time function here is just used to create a time stem for us to understand at what time we Pro out the data so we go back to the cook and then create a stem here eight-time top eight and oh now and then The Times them we need a specific format so here we just we moved the millisecond you make the daytime function more clear so here we just pairen the data into the price and then we just at these time stand inside this column and then X extend it the price that means we just put the times them and also the price together here so finally we can just create a to frame that to store this result because this data frame is say just a column so we just transfer it in order for us to just pull a bowl of the result in each of the times them so finally just some EF talk just save it to a CSV file here we just name it as a real-time stock later Oh CSV and then we just use the a pen Oh well II and every time we just saw you know the column in order for us to Trek where we are the reason why I use a price a pen here is because I want to track more than one price so here we create an other for loop here in we say we have an HSI list there we are going to pass into it you so say for example we want to check zero zero zero one zero zero zero ooh Co Co Co free and so the zero zero zero five so less one the program you so I just stopped the programs and here you can see that about three to four seconds we just extract the pilot data from the web page or these four stocks there we just outside it right here and then the values are saved at inside this CSV file the second part of this video is to use the map part leap animation package to create animations so here you can see that we just import some the animation from the map a leap and also we just pull out there's import the style and also import the py plot in order for us to to plot the graph we also import the handles right here in order for us to get the data from the CSV file that we just saved it to animate by using the animation function so the logic here is that we keep pulling the data from the web page and then we saved in the CSV file and then we extract the data from the CSV file and then we plot the result so we repeat these pauses whoo whistle eyes the live data in real time so first of all we just assign a style that we are going to use the five ratty eight and then we create a plot and we will create a two by two figures right here so we use a subplot Dubai - and then this is the first figure and then this is the second fakers birth and therefore the second thing that we need to do is to create a anim function so we first of all we pull out the data from the CSV file it contains the data and then we just get the value from the second columns and container stop price or our by Y SS and then for XSS we just release that contain the step size that is equals to this YS you to better illustrate the results every time we just clear everything and then we plot the later you so we can do that similarly for the second earth and fourth graph so by changing this to free Oh life and we did not need the xs/s because that should be the same for all the graph and then we clear this and change these and Shanxi you and then we set a title or each of the graph you and font size change it to 12 you so tested for function and then why now we can animate the result by using the animation dot fun animation the frog's argument is the figures and then the second argument is the function that we just create and then the time in the fro to update the graph I just said it 1,000 this is the mini second demean Jassim for Effie one second we just update the graph and finally we just some give a tight layout and then show a bigger so now what we just need to do is to go back to the original program that helped us to pull the data from the internet and then we just won the program and then on the other hand we in parallel we just won this animation okra so that's it how we use beautifulsoup and map a leap animation to automate the process in pulling the information that you want from the website and plot the live data in real time of course by no means this simple program can compare with other powerful service provider such as the stock service provider commodity service provider however this should be good enough for you to tell me some function that these survey provider cannot provide for example applying new network to predict their price for the next few minutes if you have any questions or suggestions about what we covered in this video please feel free to us in the comment section below and I will do my best to answer and if you enjoyed this tutorial you can subscribe my channel simply like the video and it is a great support to share this video with anyone who you think would find them useful and thank you all for watching

Channel: eMaster Class Academy

Views: 39,620

Rating: 4.9590697 out of 5

Keywords:

Id: _Bu6povAuSU

Length: 22min 44sec (1364 seconds)

Published: Thu Apr 02 2020