Remote sensing with Python in Jupyter | burdGIS

[Music] that fans something a little different today we are going to be looking at remote sensing with Python in jupyter or Jupiter I don't know how to say it we have looked at using Python and Jupiter before and we were using that with GIS pros but today we're going to do something a bit different and have a look at Google Earth engine this video came about because a luck got in touch and he wanted to use part of the modus data set for a project that is doing on aerosols I think so if you go to USGS you can find all the Modi's data sets this one's MTD 1918 very catchy name and it's all about I'm not too sure never used it before but a lots problem was that he wanted to use this and this is a daily dataset that's been recorded since 2001 I think and so obviously there is an awful lot of data to work with I believe the lock was downloading chunks of data and trying to process them using arcmap or pro one or the other and I've thought about it for a while and this really is where Google Earth engine comes to the fore now I've not used this for a long time but it did give me a little nudge and if you want to go to Google Earth engine slightly different from Google Earth and you can sign up so that's the first thing to do now I've signed up already for this I'll just show you what the sign up page looks like it already knows that I'm logged in that's very useful now once you have signed in for Google Earth engine or signed up for it you will get a email and that will tell you how to login and cetera now I'm already signed in it does take a little while for the email to come through but it should be with you within the day and I'll just show you quickly the code editor this is the kind of stuff that we want to be working towards today and the code looks like this so this is my owner account I've got a few scripts in there that I've used previously and we've got some examples here so I'm going to go for the hillshade example if I click on that then you can see all this code let's have a look it's gonna convert degrees to radians we are going to get its rain slope aspect and we are going to get several different views I think which is kind of cool so how Google Earth engine works is you can put in JavaScript code you have access to a huge vast amount of data that is stored elsewhere and you can put in these codes and then just hit run and here's a hill chain example you can see all the layers loading up and this I believe is global fantastic now the nice thing about Google Earth engine is that all the protesting takes place elsewhere so you don't need to worry about how powerful your computer is you can write the scripts you have access to all the data and off you go so pretty special now what a lot wanted to do was instead of learning JavaScript or getting to grips with JavaScript he was more interested in using Python and so what I decided to use here was anaconda navigator now if you're not familiar with anaconda it is a really handy way for managing different environments and what I want to do here is create a virtual environment on my computer that's kind of shielded from everything else and it's just going to be used in order to use Google Earth engine Python API so the first thing I need to do is to create a new environment so I'm going to go here and for my Python version I'm gonna go with 3.7 I'm going to include our just because well it might be useful in the future why not and for the environment gnome I'm gonna call it Google and Google eating out something that's easier and there we go so once I had created anaconda is going to go off and create that environment for me our Google ie environment has now been created and you can see all the packages have been installed along with it over on the right-hand side now to start this environment you can hit the play button and then you can open the environment using the terminal with Python with ipython or with a jupiter notebook now before we dive into the jupiter notebook and do you want to set up a new kernel using i pile so that when we started uber notebook we can started using this environment specifically so if i just open up the terminal here and type in Python installed and I'm going to give it the name GE now you'll see when we open to but a notebook that this means I can select any notebook to open in this environment that I've just created yeah I'll just run that and then you can see we've got a new kernel installed that's great let's close it off and then go back to Google EE and open this up with Jupiter notebook now when to put notebook starts it should start in your browser you can see that the localhost is running there we get redirected and here's all my notebook mr. plural so if I want to start a new notebook now I can go over to new and you can see I've got all these potentials in here and here's our new one the GE e and that's what we want to start it with now we're not actually going to start this yet because there is a couple of other things that we do need to do before we can start this environment successfully so what else do we need to install well we need the Google Earth API that would be very useful the Python API so what I can do here just start up the terminal again now I'm gonna have to do some jumping around here just to show you things so hold tight I've got the window open and I'm just going to check that Conda is installed because we're using anaconda we should be fine here and typing in Conda it brings up all the commands that we can use so we're going to use Conda in order to install at the Google Earth engine Python API and you can find where that is why nipping across to the Python installation webpage the dots on this are really good now it's got loads of stuff in here about check and feed Condor install kind of just did that install condom in Econ derp we don't really need all of this mainly because we're using anaconda and it's working pretty well so installing the API this is what we really need it's a Conda Forge package that's good and we can use a condor install and here we go we've got the environment we've already got our environment up and running and this is what we want here so condor install c condor Forge Earth engine API so I'm just going to copy that and then we are going to go back to here now you can see that I've got my Google e environment working I'm just going to paste that in there and all that whatever let's go now it's collecting the package metadata and Conda is going to do its magic and install Earth engine API for us so I'm going to leave that going that has now run its course and you can see that everything is complete it did tell me that I need to update condor yeah we'll do that soon promise anyway so now that we have done that we can go back to the Python installation instructions and what do we need to do oh we need some credentials so we need to do earth engine and authenticate so let's have a look at this if we go back to our and prompt let's just try typing into the engine see if this works it does indeed excellent so let's earth engine and then two kids now what this does is it opens up a web page and it will ask you to sign in once you sign in it will provide you with a one-time code you copy that and paste it back into your command prompt and hit enter and then you'll be authenticated and you can use Google Earth engine awesome so I'll just do that I'm going to switch off the video because these are private things so the private things are complete and I've assumed right end so the audience's successfully saved the authorization token that is great news what we can do next is start with our Jupiter notebook and we can test whether or not Google Earth engine API is working but we are in anaconda and here's our Google ie environment that now has the Google Earth engine API already in it so let's open this up with Jupiter notebook this is going to bring up again and it's going to open up in our browser your browser of choice and I'm going to go for new and I'll go for a GE e let's open this up very good currently it's untitled I'm gonna change that and call it GE demo you know that and the first thing that we can do to test this is just run a quick command at saying import EE when I hit run on that it has worked you can also do a quick print they ve just so that you get some output and there's a module EE running from this particular location which means our Earth engine is installed and working happy days congratulations go make yourself a coffee now we can really start to explore what EE can do or end and this is where the fun starts first of all we are going to need to do a little import so from - display we would like to import image and this will allow us to play around with images the next thing that we need to do is run a quick method called initialize and that's from the EE side of things so we're just given that kickstart to go alright and I just run that just do a quick run it and make sure that we've got no errors now we can get on to the call stuff and if you have a look at the Google Earth engine Python documentation python api you will find this little snippet and what this is doing is we're creating a variable called BM and we are going to pull in a DM give it some coloration and we're going to display it this is pretty fun so let's enter that and hit run and there we go we have got ADM of the world which is kind of neat it's just displaying a thumbnail what would be really nice is if we could have an interactive map so back in the Google Earth engine documentation I found this section on the Python installation a collab notebook now this is where a jupiter notebook is hosted and you can play around that without having to install anything disadvantage of that is that you can't add your local files that easily because I've got the jimpat a notebook and Google Earth engine set up on my local computer I can do that which is kind of nice now one thing that they've got here in this section is the interactive map and we can use something called the folium package to create a leaflet map this sounds pretty cool I would like to do this so I'm going to copy all this code yeah that's right just copy and paste and I'm going to add in a cell below oops there we are and I'm going to paste that in excellent I don't need to authenticate because I've already done that so I'll just blank that out we can leave the initialize in and what this is going to do is it's going to make a little method in order to okay but it's just a method for displaying up engine image tiles to a folium map and folium is not something i've used before so let's give this a run and see how it works oh it does not work and that is because we do not have a module named folium what can we do about this well we can go back to our condo installation and we can have a look for folium so if we search packages and we'll do a search for folium no there's nothing there which means we should be able to use condor instead folium see condor forge and it looks like this is installing off folium package forest now I may need to restart too but in notebooks in order to get this to run but we'll try it see what happens so con has installed that you can see that that's all complete excellent now let's get back to our back to Jupiter notebook and let's try this again I would be mighty surprised if this works I think we're going to need to restart this thing but let's see what happens oh this is looking good and what we're doing we've got that diem in again so let's have a look visual parameters my map those are diem on a leaflet map and look at that we can zoom in there's go and have a look at South Africa do to do to do so you bird so hopefully you can see the benefit of using Google Earth engine into butter it does allow you to do a lot of things that you otherwise wouldn't be able to do and you've got direct line into satellite imagery so you can use Sentinel data Sentinel to Sentinel one and also you have direct line into tensorflow as well so if you're looking at making artificially intelligent models and the light you can also do that now the UK fairly recently back in November we had some flooding and so I went through some of the examples and some of the scripts that people have posted up to Google Earth engine code repositories and the like and found one that uses radar data from The Sentinel one program and I've got this code here that I've been working through what I wanted to do was use this and use a before image and an after image so you can see these images here before we had heavy rains and after we had heavy rains and I wanted to see if we could spot a flood so I'm just going to run this this is just one example of what you can do obviously without needing the processing power on your actual computer this is kind of neat so I think in terms of a lots needs this might work really well for head you can code up what you want to do and not worry about running out processing now this is an area of Doncaster called fish lake that got flooded quite heavily and the blue areas where we've got floods now I did do a bit of ground truthing on this I know for sure that this area was heavily flooded there are photographs of floods all down this road so it doesn't look like it's worked perfectly but I'm no expert with radar data but I mean if you zoom out you can certainly see which areas were impacted more than others by the floods Wow looks like it's over shots and watts but an interesting idea anyway and certainly something that could be refined might be quite useful to look at hurricane heed I might work on that for another video and on that note I think I've kept your attention enough if there's any part of this that feels over your head and that you're not too sure about do leave me a comment and let me know what you're struggling with and I might do a separate video just on that section if you've stuck all the way through to the end here well done you get a lot of gratitude from me for doing that we also had a comment of the week month Channel I suppose I've never done this before but we'll call it comment of the month and that is to Victor because Victor said it's quantum baby and Victor that means that you are possibly a fellow member of the the Church of wit attainment if you not no worries I like to comment anyway so do get in touch via email let me know where you're at and I will send you a much sought after birdie is sticker that's right you can represent stick it on your laptop you'll be the envy of everyone you meet so thanks very much for watching don't forget to Like subscribe and of course leave a comment below if you want one of those awesome stick is happy mapping

Channel: burdGIS

Views: 10,455

Rating: 4.9319148 out of 5

Keywords: burdGIS, GIS, mapping, geospatial, cartography, remote-sensing, GEE, google earth engine, python, jupyter, gee python api, ipykernel, sentinel, sentinel 1, folium, radar data, flooding, big data, remote sensing, satellite imagery

Id: OsgZSlv4t-U

Length: 19min 24sec (1164 seconds)

Published: Tue Dec 17 2019