calculating heat wave statistics and climate extremes with netCDF masking

[Music] hello and uh welcome back to climate unboxed today we're going to be talking about masking but not these kinds of masks we're going to be talking about of course masking net cdf data so let's get to it shall we so what exactly do we mean by masking well let's have a look at this example on the top left above my head over here i've got one example filled which i've labeled field one and on the other side field two so we could ask ourselves a question and the question is when does each point in field one exceed that off field two in other words when does this field exceed this one each of these fields might just be a snapshot in time or they might have multiple time steps and so what we want to do is for each individual slice in time we want to compare the value in a particular grid point in the left field to that in the right field now in order to do this we're going to introduce new logical operators in cdo and i've listed them here and each of them has a very simple abbreviation so greater is abbreviated by g less by l equals by e and then by t g e is an operator for greater than equals g t greater than l e less than equals and so on now at the bottom i've listed equals but we should be very careful i give a warning underneath we should only use the equals function when comparing integer fields not floats and that's because equals is a very dangerous logical operator for floating point numbers where we can have rounding errors on the output so our simple first command would be cdo space gt for greater than and now in this particular instance of course we're comparing two input net cdf files so now rather than just having most of our commands we've seen before only have one input file we now have two input files field one and field two and then we need an output and i'm gonna call this mask so we have mask.nc so what will mask look like i give an example here just to my left so what we're going to get is a filled output that has the same resolution so the same number of latitude points and the same number of longitude points but that field is going to be filled with just ones and zeroes so when the first field is greater than the second field here the output file mask will have a one if that's not true so in other words it's false then we simply have a zero now what about an example where we have a single field and now we want to ask the question where does this field exceed a threshold so we might want to ask when does this field exceed this number for example when does the temperature exceed 25 degrees c we might want to say when does the precipitation exceed a certain threshold we might want to do this in order to calculate number of days when we had intense rainfall we might have a crop that has a sensitive temperature threshold in the flowering season or for example when we go below a threshold we might want to calculate the number of freezing days in a month now we have to add an extra variable to our logical function lists so before we've already seen these top four functions and now we're going to add another abbreviation c for constant so what we do is we will combine these so now we have g e c so g e if you remember was greater than and equal to and now we have gec which means greater than or equal to a constant and so all of these logical operators can be combined with an extra c so in this case we want to compare this field to a threshold of 25 degrees so the command starts cdo space gec but now we need to give it an argument so after the gec we put comma and then the argument so in this case it's 25 that's of course assuming that our input file has units of degree c of course if it's in calvin we need to give a calvin argument here and then we have space and then we have the input file name and then the output file name so again for every time and location we'll get an output file which says 1 if the condition is true and 0 if the condition is false so again we end up with a file with a function of longitude and latitude and once again if we have multiple time steps then we will end up with multiple time slices so now we've actually calculated the mask we can start to do some interesting things where we can say well how many days in 2020 did each particular grid point exceed a temperature of 25 degrees and so we can do that with these masks by simply summing up each of the mass slices so for example we could say cdo year sum so if we had daily time step input data by doing year sum we would end up with a field with the all of the ones added up and each location would have the number of days in which that threshold was exceeded so we can do cdo day sum month sum year sum or if we want to add up over all time steps in the input file we might have an input file with 50 years of data we can add up all of these masks over the whole 50 years using time sum so let's see this in action shall we in this directory i have one file it's called hourly t2m.nc and it is a file which if we open with nc view it's the temperature i like using temperature for my examples and it's running from the first of january if i actually step back one step we'll see that the last step is the 30th of june at 2300 so it actually has an hourly resolution if i click on the animation we can see the pulsing of the dyno cycle each day and it's an area that's basically has the continent of africa in its center so let's quit this and let's try our first masking so in the video we had an example where we looked at which steps were larger than 25 degrees so we'll do cdo space gec now the units of this fire actually in carvings so in this case i'm going to use 2 9 8 for 298 kelvin space then we have the input file i only have one file in the local directory so i can just hit tab for completion and then i'm going to call the output mask now what's going to happen when we hit enter is again we're going to get the error due to the packing of the data so once again we actually need to convert the file to floats by using minus b and then f 32 to convert and unpack the data from shorts we can see here type equals nc short which is the pac data and unpack it to full floats so we've unpacked the data and we've converted it to a mask successfully now so if i do ls minus ltr we have a new file called mask let's have a look at that and see what's inside so if i have a look at mask using nc view now the file looks rather strange now we have blue colors and we have red colors now if we look at the scale on the left hand side for nc view we can see that this the range of the data runs from zero to one and in fact the map shows us that all of the values are either zero one if i move my cursor in the window here the number is actually displayed from the x and y point where the cursor is pointing and as i move it around you can see that it's either zero or in the red region it's exactly one so we have either zero or one so that's telling us wherever this number is one we have a temperature at that hour which exceeds 298 kelvin and i can animate this mask as well and every day over the continent of africa as the temperature warms up those hourly steps are exceeding 25 degrees now that mask can be added up for example over each of the calendar months so we could do cdo and we can say mon sum because now we're adding up the mask values for each of the hours and we called it a mask and now i'm going to say mon some hour for hourly resolution so we add this up it takes a little while because there's a number of steps four seconds it's done and now when we look at this output file and see view we now when we click on the data remember the metadata hasn't changed it's still called t2m we click on this and now we have a variable which has a range that runs from 0 to over 700 which means when we have these red colors with 700 let's change to a different color scale these red colors essentially mean we have 744 hourly steps with the temperature exceeding 25 degrees over the sahara the number seems surprisingly low but remember this is in january the first time step we had six months of data so the first time step was for january if we look at the date it says 16th of january as i step forward with the forward button the next step will be the data for february so now it has a date of 15th of february remember the time is in the middle of the month by default but you can specify the timestamp to be at the beginning or at the end of the month using a command line option with cdo as you saw in a previous video with the link in the pop out banner above my head right now as we move forward in time march now in april the sahel really starts to heat up until may and june where the sahara now is getting very hot indeed so if you want to of course we can do this at a daily resolution we can ask how many days have an average temperature over for example 25 degrees so we do cdo demean and then the input file was hourly t2m dot nc and then the output we're going to call demean.nc and now we have the day mean and we can actually apply the mask to the daily averages now we say cdo greater than equal constant 298 they mean and then we are going to say masked.nc and i forgot again to convert this to a float and then we can do cdo mansum mask and then some and now if we open up the man some file and i press t2m now we have something that looks quite similar before but rather than being the number of hourly steps it's the number of daily averages that exceed this threshold and if we animate forwards and i slow things down a little bit we have six steps in the file because the file original file ran from january until june you can apply this to a much longer series of data that runs for many years and you can do annual means you can look at how the annual means change over time so we hope you've enjoyed the way we've covered the topic of masking which is a musk have skill to acquire sorry for the joke and we look forward to seeing you on climate on box again very soon thank you [Music] [Music] let's try again okay just done don't sneeze [Music] [Applause] hello and uh welcome back to climate unboxed so today we're going to be talking about masking but not these kinds of masks we're going to be talking about masking next

Channel: Climate Unboxed

Views: 226

Keywords:

Id: G0nUE6mkINg

Length: 14min 42sec (882 seconds)

Published: Wed Oct 06 2021