Using the python CDSAPI to access the climate data store

[Music] hello and welcome to climate unboxed today i'm going to introduce you to the cds api python package which will allow us to directly connect to the climate data store and download large data sets in automated ways using python scripts now you don't have to worry if you don't have a background in python this is exceedingly simple you can just make small edits to the file and actually use the api without any prior knowledge of python whatsoever so there are four steps that we have to undertake once only the first consists of the registration on the cds site the second step then is to install the cds api package in python that we will use to download the data we then need to set up the ssh keys which is essentially to avoid the use of a password each time last but not least we need to go to the website once to tick the box to allow the permissions to download the data and then we are ready to download the data of our choice so the first step we need to consider is the registration so if you go to the web page that's indicated below you can see a button on the top right corner login stroke register you can click on that and click create new account you'll need to enter your email address your name country sector and also if you scroll up you can give information about your organization accept the terms and conditions and then click on the green button below create new account you will receive an email which you will need to click to confirm the address and then you'll be able to go to the login page enter your chosen password and login if everything's done correctly you'll see your name displayed on the top right hand side so to install the cds api you will need to open a terminal window i'm using a mac here under ubuntu you can also find a terminal window under the menu and it's as easy as this we need the pip 3 command and then we type space install and then cds api and then it will download the package and install it directly and we have a message successfully installed cds api if you want to quickly test that it's been installed correctly you can then open python 3. i'm using python 3 and pip 3 and simply type import cds api the next step is to put into place the api ssh keys now this is a special code that we put into a file that basically replaces the need to have to type in a password each time we want to access the database now there's a wiki api how-to page which is extremely helpful it has all the information on it that we will need scroll down you'll find that there's this section entitled install the cds api key now if you're not logged in you'll just have this generic text here so what you need to do is ensure that you click the red button and that you're logged in and your name is displayed here in the top right hand corner of the web browser now if that is done then in this window here you should see displayed text rather similar to this so there is a url address and then there's a key displayed below inside this box if it's not displayed you may need to click the reload button on your browser and that should ensure that the new information pops up inside this window so then what you need to do is you need to highlight that information displayed in this window and press ctrl c to copy it or you can use copy from the edit menu of the browser now i've got the information here in this window and i will copy it let's try that again okay these are fake ids then you need to open a terminal window ensure that you're in the home directory so you can just press cd for change directory or cd if you wish space dollar and then home in capital letters it's important that it's in capital letters if we type print working directory pwd we see that we're in the home directory in my case uses tompkins and then you need to open a text editor now i use emacs and the name of the file is very important it needs to begin with a dot and then we have c d s a p i r c so now we have a window launch this is my emacs window and i'm just going to paste that key into that window url and here we have the key just below so i will save that and exit the window so we have now installed the api key so if we scroll up to the top of this screen we can click on data sets now just as we saw in my previous video you can open the product types i'm going to click on reanalysis and now we're going to use era5 data on single levels there's actually two options now an earlier period from 1950 to 1978 which is in a beta release as you can see but we're going to use the later period so i'm going to scroll down and find era 5 monthly average data on single levels so i'll click here and we need to go to the download data tab and we're going to select monthly average reanalysis i'm selecting two meter temperature again as an example and we scroll down now i'm going to actually hit clear all and i'm only going to select one year as an example 2020 i've got all of the months selected and the time i've only got one option but it needs to be selected because it's monthly average data we're selecting data for the whole region and we've got net cdf selected that's the format term now if you haven't used the interface before to let download data you will need to click on this license box here to accept the terms now you only need to do that once and it's already been stored for me on the system you will remember from my earlier video that we then could hit the green button submit form in order to use the web interface to download the data but now instead we're actually going to click on this green button on the left show api request and that reveals this window below with a small piece of python code now i want to highlight this code and press ctrl c to copy it and then i'm going to go back to my terminal window and i'm going to open now a new file i'm just going to call it download.pi so this is a python code i'm using emacs again but you can use any text editor of your choice and i'm going to paste that code inside the window now as i said at the start of the video if you are not used to python if not used python before that doesn't matter because you can see that the length of this code is very short and is extremely self-explanatory so the first line of this code says import cds api that simply imports the package that we've just installed we then set up an incidence of a client but you don't have to know what that means okay and then there's this retrieve command and you will recognize immediately the information that we've just selected so there's the analysis type single levels monthly means product type monthly mean reanalysis 2 meter temperature year is 2020 and so on and we have a list of months time the format equals net cdf now the last entry on this line is the name of the file name and it has a generic name download.nc now you may want to change that to something more appropriate for example t2m underscore 2020.nc i'm going to save that and exit and now to actually test out our download we simply need to type in python3 space download dot pi which is the name of my program where i've saved that snippet of code and i press enter you can see that my job has been sent to the system there and is currently waiting in the queue so if everything has worked correctly for you up until this point you should see the same without any errors and now we simply have to wait [Music] and now you can see it's downloaded the file directly to my desktop i'll do ls minus lt r and there we see the file [Music] that's been downloaded [Music] and we can open it with nc view and there we have it one data file downloaded directly to our desktop so you see we only need to do these steps once once all of these steps are put into place you only need to get the example script from the web interface and you can download directly the data of choice to your desktop the advantage of the api is once you've got it set up you can write simple loops to download far greater data volumes than the web interface allows and you can even run your script automatically in the background at regular intervals now i'm going to show you how to do the loops in a separate video i hope this has been useful and don't forget to click subscribe if that's so and i'll see you soon again at climate [Music] unboxed

Channel: Climate Unboxed

Views: 1,833

Keywords: python, era5, climate data store, copernicus, ecmwf, netcdf, cdsapi

Id: AmF1nn7o6Hc

Length: 11min 40sec (700 seconds)

Published: Wed Mar 17 2021