Pandas Time Series Analysis Part 1: DatetimeIndex and Resample

hey friends today we are going to start new set of our tutorials called time series analysis in pandas now what is time series time series is a set of data points indexed in time order for example this stock price chart where on x-axis you have dates and y-axis has prices so this is your time series now in finance data analysis space uh time series analysis is used a lot so the people who are connected to finance domain they are going to find these tutorials extremely useful okay so the first thing i'm going to do is download the historical prices of apple stock from google finance here you can type in the ticker for apple and click on historical prices so this shows the prices of last year and click on this link here to download these prices in this csv file all right so i have this csv file with open high low close and volume information as usual i have launched my jupiter notebook from the command prompt and i created a new python notebook now i have imported pandas as pd and i am reading the csv file here and when i load the data frame the data frame looks like this now the topics that we are going to cover in this tutorial is date time index and resampling so let's start with date time index when you have this data frame your index is this integer index and your date column is actually a string so if you want to verify that you can just do df.date 0 take the first element and if you print the type of it you will find that this is a string it is not a date now read csv has an option called pass dates and you can specify the name of the column which you want to pass as a date so when you supply this what happens is this date column is now of type date so if i execute this once again you will see that the data type changed to be the timestamp okay now we are going to make this date column and index of the data frame so remember the index right now is integer index all right and we are going to change that index to be this date column so you can do that by saying index column is equal to date so i want the date column to be my index and when you do that now your index is date and you can verify by type in df dot index now you'll see that this index is of type date time index now why did we change the index to be date time index well there are many obvious benefits for example this data frame is containing data of entire past one year but let's say i want to retrieve only january 2017 uh stock prices all right so you can do that easily by supplying this partial index here here i'm saying give me data for only jan 2017 and you can see that it will return you jen 2017 now let's say you want to find an average price of apple stock in the month of january all right you can do that easily by just saying that okay i want the close price so this gives you the time series of close closing price and you can just run mean on that and that returns you the average stock price of apple uh in a month of january 2017. you can also retrieve the price on any given date for example on third january this was the price so this is pretty useful you can use your date as an index in a data frame and get the prices on that particular date not only that you can supply the range also so for example i want the prices between uh the 7th january and the 1st january let's say which is the first week of uh of your job 2017 and month january all right so when i do that uh let me supply first here and i will get the prices from the first week of january so you can see numerous benefits of having datetime index you can retrieve a specific month's data and average price in that particular month and also the prices on a given date range now these are the typical use cases for any finance person so if you're using pandas with time series as a finance person you can do all this analysis quite easily all right next thing we are going to talk about is resampling so here i have daily data okay and now let's say for my analysis purpose i want to retrieve a monthly data so i'm not no longer interested in daily stock prices but i want to know what was a monthly stock prices for apple stock you can do this by selecting a column first and then say resample to a monthly frequency and i am interested in mean of that so when you do this what happens is now it is showing you the monthly average of a close price all right so again step by step first i selected a specific column from a data frame which will give me a time series and then i'm calling resampling on a monthly frequency and then the the method of resampling here is to take average price all right so here what you're seeing is an average price in a month of july 2016. this is august september and so on you can also plot nice chart by calling plot function here and all right so let me just call plot now you need to say this in order to actually plot the chart okay so this is uh the inline the line magic okay and here now i'm seeing the stock prices on a monthly frequency you can also convert it to weekly frequency and that will show you weekly stock prices if you want to look at all the possible frequency then you can hit shift tab here which is going to open the help and then in the help you can find out the frequency part of it so let's see where is the frequency so here uh if you just open this link here in your browser then you are going to see all the frequencies so here we use weekly and monthly frequency but you can do semi-annual you can do business day so business days excluding all the holidays uh weekends etc quarterly so let's try uh quarterly frequency to plot the quarterly chart so let me close the help here supply quarterly frequency and you can see that now i'm seeing quad q1 q2 q3 and so on all right you can also change the type of chart to be the bar chart all right okay i think it is called kind and not type okay so you will see this is all quarters and you have nice bite bar chart here okay so the difference between this plot and the plot without doing any resampling is that all right let me just print the normal closing price chart and you will see the closing price chart is much more granular it is on a daily basis okay but this same daily frequency you can resample to be quarterly monthly daily whatever is your use case okay so that's all we had for this tutorial i have this jupyter notebook available in the video description below it is on github feel free to download it and play with it i highly recommend you doing coding as i teach these tutorials because trying it out on your own is the only way you can learn this thing if you just watch it as if you are watching some entertainment show or anything then this that's not gonna help you you have to make good use of this uh code that i'm providing and try these tutorials on your own okay so in the next tutorial we will continue with few more topics on time series analysis so stay tuned and thank

Channel: codebasics

Views: 134,694

Rating: 4.941545 out of 5

Keywords: pandas time series, pandas time series analysis, python pandas time series, time series python, time series analysis python, time series pandas, pandas resample, pandas datetime, pandas datetimeindex, pandas tutorial, pandas python tutorial, pandas dataframe, python pandas tutorial, time series analysis, python pandas dataframe, pandas time series analysis tutorial, time series analysis in python, python time series, time series in python, time series forecasting python

Id: r0s4slGHwzE

Length: 10min 23sec (623 seconds)

Published: Sat Jul 15 2017