Unlocking the mysteries of the most violent tornadoes and the storms that produce them

well thank you all for the making it to the last session of a day it's always a a good good uh place to to be um I'm going to talk about work I'm doing on Blue Waters that involves simulating thunderstorms at extremely high resolution ultra high resolution we're running out of adjectives almost we get down to say 30 or 20 meters um I want to quickly acknowledge uh Robert wilh helson especially he's got the first P got the ball rolling got the research going he's also been my tie to getting in some Illinois money we've got a pack in review now which is great um hoping for good news on that um and also my collaborators Bruce Lee Kathy Finley Adam Houston uh who are part of this work so briefly I'll talk about the motivation for this work look at the process that we are going through to to try to answer these questions briefly reflect on the last three or four years of Blue Waters and then mostly talk about some scientific results that we're very excited about and where the compass points after that so the problem tornadoes are a problem um roughly the depending on a lot of different things you get about a thousand observed tornadoes per year many of them occur in the United States the good news for Humanity in general is that most of these are very weak tornadoes and they don't last very long but a tiny fraction of the tornadoes that do form are the ones like Greensburg Kansas eleno Oklahoma um more Oklahoma pick a decade right where you get a big long track EF5 strength tornado that's the top wind speed um category for tornadoes and what makes this worse is we're not very good at predicting whether a storm that has formed even if it's already formed whether that storm will go on to produce a tornado a weak tornado no tornado or a long track say EF5 um we're getting much better at predicting the environments in which these occur um and and these these uh the larger Scale Models the the that are predicting three days out four days out that there's going to be an outbreak in in the midwest but we just don't know the details yet that we're getting much better at that but the details of storm scale stuff is much more difficult so basically my Approach is well if you're going to as meteorologists we're always trying to predict to protect uh Society by predicting getting people out of the way of storms before you can uh forecast how a storm is going to behave you sort of know have to know how it works and I'm all about just kind of figuring out how stuff works I just love to pick apart something and drill down into it till we get to the small pieces so running a supercell simulation very high resolution is is is kind of a neat thing um I should that much of the work that's been done to try to answer these questions has involved field programs observing real storms and this is incredibly important um much of the work that I do I'm always looking to the observationalist to to to communicate to make sure that we're what the model is showing makes sense put it in context of real meteorology and that's why I spend so much time one of the reasons I spend so much time on visualization because I truly believe that you know you really need to um if you can't make the storm in the model look like the atmosphere there's something wrong with your model so we were doing this we want to be able to sort of pick out uh storm scale and tornado scale features at very high temporal resolution so I'm saving data like every second on Blue Waters every uh second of model time um in order to capture all the really fast moving air because air in the vicinity of tornadoes is moving very fast we want to know what can make discerns these storms from other less uh harmful storms and then maybe enable comparison and you know it sounds a little off maybe to say to guide but maybe this work can help guide F future field studies future field studies to look at what part of the storm we see some of these features so to do this you need a good model uh cm1 I'm using George Bryan cm1 as has been used in some previous talks you need a good set of atmospheric conditions adjacent to a real event to initialize the model and is this is sort of related to what Jeff just said about the hardest part is getting a good control case getting a supercell to produce a long track EF5 tornado was the scariest part of this whole project it was the part that came at the end it didn't come at the beginning most of what I spent my time on was some of this other stuff so um building code I anticipated this huge data problem we were going to have down the road I started working on this almost a decade ago and so I sort of over time built this environment um to to make this stuff work pretty well and essentially I'm not going to go into a lot of detail on this today I've talked about this in previous talks but I'm using hdf5 and I'm subsetting on a per node basis saving data layering extra uh using the the core driver to layer different time levels in memory buffer them to Memory um and then you need code to interface to this data output because you don't want to have to convert this data to anything else there's just too much of it so this has taken up a lot of a lot of time briefly you know so we started when the first PX came out we started using during the friendly user period um they you know so we're just tuning the model getting it to run and it became extremely apparent very early on that it was IO that was going to kill us and we had to do something about IO um in order to achieve our research goals so that was what most of the work I've done on this project up to say a year or so ago has involved just handling IO I've not it wasn't by I never intended to become an i guy but it just sort of happened uh and in the link to visualization and then so once we had the environment you know we're coming to the end of an allocation and we still don't have our high high-end tornado so I'm trying different atmosphere conditions different atmosphere conditions and and Lou wicker at the national Center for severe nssl um he gave me the magic sounding that got this work really going and he deserves a ton of credit um so it was the condition of the 24 May 2011 elino EF5 storm we took a sounding off of that uh the re the right flank of a of of The observed storm at least as ingested by The Ruck model uh the environment around there so thus far recent work um we have simulated a storm from start to finish that produces a a tornado that's on on the virtual ground for almost 2 hours it's a long track EF5 uh We've published two papers on this work this one is just hot off the press or impress should I say in bams very happy about that um we also have a 20-m simulation I'm going to spend some time uh showing you today that is even more intriguing at least in the tornado structure so um the size of the problem what makes this a blue water size problem um just look at the the the Bold things so 30 m simulation costs about about 2 billion grid points down 20 M uh eight billion grid points we're using uh these runs were kind of small only 625 nodes these runs are getting a little bit bigger we're now getting up to say uh you know a quarter to a third of the machine and you know we can go higher than that the model scales very well and um I'm I think the io load I can handle now these numbers for Io I can make them bigger or smaller just depends on how often I save data how many variables I save whether I use compression or not and all that stuff but these are ball numbers And archiving this is not fun um so an overview of the storm before I get into some of the the graphical results so the storm turns into a supercell it's not a surprise based upon the environment lots of Cape lots of Shear lots of lowlevel uh Shear storm relative helicities are huge if you're a meteorologist you all the stuff you expect is there uh the tornado forms about an hour and a half into the simulation it lasts for about two hours and then it's the details of what happened during the tornado formation that we're really focusing on because that's really one of the big areas of research in in mesoscale meteorology is tornado Genesis what leads to the tornado forming and why don't tornadoes form sometimes and why do they in other times um we've to our surprise to some extent much of the uh pre- tornado features that we see are not in parts of the storm where we have sort of the conceptual models have have told us like in the rear flank of the storm the downdrafts in the rear flank of the storm really don't seem to be directly influencing the tornado in a major way it's the forward flank of the storm and I'll show you pictures of this shortly where a lot of the uh the the Hol the vorticity that is going to serve to spin the mesocyclone and the tornado seems to originate from um so I'll talk about these different features and show you uh pictures shortly so this is a supercell just so we're on the same page here this is a nice picture Roger Hill took it um this uh region of the storm is typically referred to as a mesocyclone it's just a big rotating updraft so this updraft huge amounts of of momentum air rising up upwards but it's also rotating by virtue of the fact that you have environmental wind shear um you can see that there's a big fat tornado on the ground here that's a good thing to have for a tornado research project and over here this is the forward flank of the storm so this this Cloud feature here is often called a tail Cloud it forms along and behind the Leading Edge of the forward flank gust front so behind this sort of where the tail the arrow is you see this this is rain or hail it's falling it's melting it's evaporating it's cooling the air hits the ground it spreads out and the boundary between the the environment and the storm uh This Cloud sort of marks that boundary to some extent but another reason I showed this is because the feature we're going to be studying really shows that air from this region of the storm is being tilted into this into the mesocyclone and this picture certainly at least suggests that um so that's why we're using it now this is what it looks like when you use visit so I'm trying really hard to capture the scale of the storm including as many hydrometers as I can to make it you know to see if it looks reasonable so including hail shows that you see all these these these sort of fall streaks that fall down um and in fact I'm since so much of the activity that I I'm I'm looking at in this is in the forward forward flank of the storm I'm wondering if perhaps I'm going to be looking at hail a lot and looking at how it falls how it melts how the temperature gradients that are caused by it in the cold pool um I don't know this is just these these visualizations give me ideas of where to go and that's one of the reasons I do this now just to compare to the previous there's the tornado you can barely see it there's that little tail Cloud you can barely see and you'll see it in minute okay when I start doing this but it's really and there's that the hail I was referring to and up here are little at least suggestions of what we call madus clouds you often see associated with supercell um so this is you know it's a challenge to to make these kinds of graphical images but that's not the end goal that's just the the first part we want to be able to to know where to look here's another another you know changing some opacity things there's the tornado you can barely see it but this should at least give you an appreciation of the scale of the problem most of where I'm going to show you from here on is just focused on this little region right here uh and I I've been working with David Bak at enar or I'm sorry I should say NCSA my MPA um so he's he has a renderer that does just a great job and his his really draws out the madus clouds up here um here's a tornado in this his rendering this is a a a shaft of heavy rain um it it doesn't show all the rain just to heavy rain and then they just done a beautiful job at seeing these madus type clouds now I'm not entirely sure this is what a real Cloud would look like because we haven't included hail and snow but it certainly shows that those patterns are there so in other words the visualization helps us to understand the storm get a first you know a first cut Glimpse at it and then it drives the research so this is a a volume rendered image that that the visit software used created or vapor excuse me thenar um this feature we're calling a extremized verticity current this is the updraft of the storm this arrow is supposed to indicate the general flow of the air from the forward flank up into the updraft and this Arrow points to this this new tornado it's just forming and you see all these little guys right here uh these are all little regions of concentrated vorticity I'll call them vortices they do spin in a Vortex relative sense and they're all kind of converging into where the tornado ends up forming so there's multiple things going on at different scales um this is just a movie showing the tornado forming we're looking at the cloud and the precipitation field this little nub here is an anticyclonic Vortex it's not a tornado but it does show a little bit of condensation you'll see the cloud streaming in from the right to the left this is that tail Cloud the tornado condensation funnel descends to the ground some rain starts to wrap in around the the mesocyclone or the wall cloud from the rear flank of the storm this this uh is a reasonable realization of a supercell thunderstorm that produces a tornado um at least when you compare it to uh field studies and such so we're happy about that um even this is a a figure from the Bam's paper that just hit the early Online release um even little tiny you know Sub Sub sub toric vortex structure is there here's a just a little Glimpse this is a uh I think Tim samz took these pictures I think um so you've got this little horizontal roll that's ascending up the outer periphery of the tornado Vortex we have those two in our simulation we're barely resolving them that's this little guy right here this is the vorticity field and this little Vortex here you have enough pressure drop in the center of that to to condense water vapor to form a cloud and there it is and and when you start capturing little stuff like that and you start seeing that and seeing it in nature it gives you a some confidence that your results are probably uh in the right ballpark now I'm going to zoom in on this in a second but this shows you this the height of a tornado uh this is just 10 kmers of the domain it goes up to 20 and the tornado actually goes up quite a bit higher the left is the vorticity field shaded by the vertical component of vorticity so we want to see all three components of vorticity that's the vorticity magnitude but we're shading it with the vertical component what means what that means is the red guys are rotating cyclonically or counterclockwise and the blue guys are rotating anticyclonically and if it's gray or something in between it means it's primarily horizontal vorticity but um that's how I've shaded this this is the downdraft in the center of a tornado this is a two cell tornado so it actually has a downdraft in the center of it and that is consistent with Theory and and chamber simulations and this is the pressure field pressure deficit field so we just kind of go up here um you know it's just to show you what the tornado looks like you can see these sort of intertwined vortices which is consistent with the two- cell model the downdraft in the center Vortex breakdown to some extent um and out and out and so forth um and you see in in the lower part of the pressure that a lobe of low pressure off the side seems to be affiliated with the flow feature that we're calling the streamwise vorticity current so this is a uh Vapor image showing uh trajectories they're unsteady trajectories so showing the motion of the air um and so basically what you see is air along this boundary this is the forward flank of the storm is is rising Rising and rotating at the same time essentially it's like a we call it a streamwise vorticity current because it's like a current of vorticity that's rotating in the same the rotational axis the vorticity vector is pointed in the same direction as the wind speed Vector a storm relative um and this is what it would look like projected in two Dimensions so this is this would be North the camera eye from here is from the Northwest looking towards I'm s looking towards the southeast um here's our tornado which is full of rain somebody's going to fix that problem who's doing uh guy North North Dakota is he in the room he's he's doing work on getting centrifuging working uh for precipitation hydrometeors and supercells that's very important in other words in real tornadoes the precipitation gets centrifuged outwards we don't have that code in our model yet um so because it doesn't exist yet so the streamwise verticity current if you project it in two Dimensions seems to come from the forward flank of the storm and they drawn up and rotate into the the the mesocyclone which is the giant Vortex that's sort of uh supplying you know but that is it the rotation of the of the supercell as opposed to the rotation of the tornado and there's definitely scales in between that that kind of show up on these images so um and let me show you a quick movie of tornado Genesis so this gray stuff here this is just vorticity magnitude and these are cyclonic vortices or this is Isis surfaces of cyclonic vorticity and the blue ones are anti-cyclonic vorticity so one thing to realize right away this is just during Genesis this is the anticyclonic tornado that's or Vortex that's already formed our tornado is going to form out of the pool of these guys here but it's definitely weighted higher towards cyclonic verticity than anti cyclonic vorticity um and much of this is horizontal but this isn't this is just showing you the vorticity magnitude so Watch What Happens um if you're looking for a simple tornado trigger in this simulation you're not going to find it what you see is a gradual coalescence of ticity at least in the these vortices and a strengthening of the streamwise verticity current that's feeding the updraft and now you have a tornado um stuff going on in the rear flank of the storm doesn't seem to have a direct effect on this particular simulation at least that's what our our our results look like so so far um during the maintenance phase of a tornado so this is when it's just been cranking along forever and ever um this the red arrow now points to the region of the stream wise vorticity current we're dropping Parcels in that region there the the green guys that don't show up so well are trapped just a bit in the forward flank these guys seem to feed the tornado almost directly so I'm interested in in how the properties of that air and then we go over towards the rear flank and this is just a rotated view of the same thing so you can just kind of see them at the same time um and yeah it's a tornado all right it's rotating like crazy um you can see that primarily the parcels that seem to feed the tornado at low levels come from the cool side in the um in the in the cold pool and this is an interesting result because it's easier to lift warm less dense air than it is to lift cold higher density air that's kind of intuitive and it's actually true if you have cold pools that are too cold you're going to have a hard time uh forming tornadoes and there's other reasons beyond the lift issue but um it's definitely a problem now I'm going to show you the depth of this tornado So This Tornado the 30 m simulation um the tornado dies after about 2 hours so in this case I'm volume rendering the the W component of the wind so this is downdraft air and this is updraft air so Blues are downdrafts and these warmer colors are updraft air and the tornado is this is is represented by vorticity magnitude in this white tube you can see there's some more of verticity outside there that's a common feature in the simulation but right you just see this thing just dies really fast um right about there it just poof the downdraft uh you know the source of the air from that downdraft is we'll have to look into that but it is raining heavily in this region but there's also probably non-hydrostatic pressure forces at play here but regardless the the tornado much like it was born it h sort of came about rather quickly it dies rather quickly it it just gets stomped on essentially and that has been observed it's not the most common form of tornado uh Decay usually you see the the Gus front sort of stick out pull the tornado out kind of stretches it out ropes out as they say that this one doesn't do that it just it holds on to for its dear life until it until it dies so for the rest of a talk I want to focus on two uh looking at an earlier phase of the 30 meter simulation this is the one we've published we haven't gone back we're just starting to go back even earlier to sort of look at how the storm structure changes over time well before tornado Genesis so in this movie I'm going to show you um again Gray is this is vorticity magnitude and it's shaded by the vertical component if there's very little vertical component it's going to be gray so this is horizontal verticity so you can imagine air rotating in this plane and you can imagine air sort of becoming rotating into the vertical plane over time and it if it turns red it's rotating the direction we like in the northern hemisphere for tornadoes if it's blue it's rotating the other direction and this is the cold pool and the camera view and I'll just let this run so keep an eye here's our vorticity kind of streaming in it it gets tilted and it becomes vertical and over time you will see a sort of increase inv vorticity in the mesocyclone up here um but I'll just let it go so let me just say a few words about volume rendering one of the I was asked by a hardware manufactor who's working on an app or or a software to do volume running why do you use it why don't you just do ISO surfaces because a lot of scientists like ISO surfaces because you can do this you can show different scales of motion or different um scales of magnitude of values and can sort of Look Into the Storm so here we go again with the vorticity taking the turn and then you're going to start to see a lot more Reds here um and you're seeing both the sort of diffuse vorticity coming from the the forward flank and then you're going to see very sharp regions in the vortices that follow and right about look keep your eyes peeled right in this region and it's a tornado at some point I mean I there it is so it's if I were to say you this is a process of apparently um what's causing the tornado is the big question but there's certainly an increase in the amount of of vorticity the streamwise vorticity current seems to be getting more intense you're seeing more Reds you're seeing more ingestion of that and this is just one view but you see um these little vortices are only piece of the story so you see these little vortices merging together into the tornado but as I've decreased the lower threshold of the vorticity so I can see more what's going on I realize that these vortices are important they're probably probably important because they directly provide vorticity to the tornado but the streamwise vorticity current is providing vorticity to the mesocyclone which is then causing the tornado and that whole process is I'm just dying to pull it apart because it's it's really interesting um and yeah this goes on so um but it's F there's just so much detail it's beautiful um and I I'm I'm there's some a whole lot of scientific questions that I think can be answered from some of these simulations over time you'll start to see so here's where the rear flank of the is you're getting some some uh low low Theta e air here some uh positively buoyant but dry air probably from high levels getting kicked out and then you start to see the the tornado becoming uded so you start to see the tornado as it's in there you can sort of see it but it's hiding behind some other vorticity as well as the rear flank sort of kicks in so we've still got work to do on this simulation at 30 m but um we decided to try a 20 met simulation in the same environment we did tweak uh the turbulence closure on this to a better one than we did in the first simulation um but other than that it's identical and we get another EF5 tornado although in this case the tornado itself looks really really good so I'm just going to show you this is just hot off the presses well not too hot but it's not too cold either um I haven't really analyzed it much yet but uh I'll start out with the cloud and the rain field as before and just to let you know right in the first frame we've already got an anticyclonic tornado um going on and so again this is the region you want to focus in on and you'll see the real tornado the EF5 forms about here it very quickly becomes an EF5 um that streamis vorticity current I haven't looked for it yet but and you do see that the the clouds here in the tail Cloud region tend to be more conve they tend to be more cellular than we saw with the other simulation where it was more laminer and that could be just resolution now we're going to look at the vorticity field during tornado Genesis for the 20 M run I will put the cloud field up here as a reference in a second cuz this goes back a little earlier so we started out with this blue guy uh going on and and actually you know re there have been people have seen anti-cyclonic tornadoes before but look at all of a sudden you see this eruption of Reds and yellows where all of a sudden you have all this cyclonic vorticity and boom there's your tornado now I look at that and I go what could cause that the only thing I can think of is the updraft just must get really strong at low levels really quickly so that it's starting to stretch foric that's near the ground that's my theory now what I'm doing here is I'm playing with the the opacity Maps I'm now going down to a much higher threshold so you can start to see the tornado circulation really well and this is really exciting so the tornado is really broken down into a a a multiple vortex and you can see the two vortices you can see actually there's several vortices in here and this is consistent with other models that have been run that are just tornado models just using the chamber approach where you just simulate the updraft as a boundary condition um but this is this is really encouraging because I I think this is probably how most tornadoes that are this uh large really turn out and you can see the the how it goes now I'm going to com kind of go back to lower thresholds playing with the opacity threshold so you can see as the tornado winds down a bit it widens the maximum vorticity values go down the circulation probably doesn't change nearly as much but um and then you can see what's going on and eventually what happens is it just R it's a huge Flux Of Rain rain comes down and downdrafts and everybody kind of gets squashed but the tornado kind of dies after that so sort of coming to the end um for future work we obviously have made a lot of done a lot of work on visualization and I do this not just because of the pretty pictures but because it is key to understanding these storms with all this data you've got hundreds of terabytes of data and you need to make sense out of it and I found that you know volume rendering especially can be very effective very dense amount of information per frame you could say we definitely want to dive more into a 20- meter simulation do more of these um incorp surface friction properly play with the surface boundary condition I also need to upgrade to the latest version of cm1 George tells me he's done some stuff with the surface that's probably important and do more environments um and just some parting thoughts um you know this is a lot a lot of a talks I've seen here are like they blow my mind this is like the first time this has been done you just can't do this kind of research on other machines or without HPC resources but it can be done and this is really exciting we have the convergence of Hardware software and uh the meteorological observation these different features we've identified and we've written in our paper um are intriguing and we need to look more deeply into them more quantitatively we want to do more work with the 20 meter run and um really ensembles at 30 MERS maybe not at 30 MERS but we do need to do more simulations because it's very dangerous to draw too many conclusions from a single simulation or in a single environment so we've certainly got plenty of work cut out for us and I'd be happy to answer any questions so I'm curious when I when I see these visualizations I see these sort of vorticity tubes it seems like they have very characteristic sizes what is it that causes those the characteristic that's a really good question that's a very good question um and I don't have a clear answer there's something called the swirl ratio when you start studying tornadoes and looking at you know when you look at tornadoes they typically start out as sort of like a thin little little thin tornado it's called a single cell tornado with it's dominated by updraft there and then over time it widens and you have to look at the ratio of the air coming in to the strength of the updraft so you just get a whole spectrum of of Vortex sizes um is there a characteristic Vortex width for a multiple vortex tornado that's a tough question I don't have the answer to that um I do think that most tornadoes probably have multiple vortex characteristics um and I don't think some of our Radars are able to capture that but um but but yeah that's a good question and I would assume that that may be sensitive to other parameterization choices as well friction and and looking at turbulence and turbulence closure Jee great stuff obviously um so these simulations were done lower yes so I I guess I'm still we talked about this before I'm still having trouble trying to reconcile the use of a free boundary and therefore the absence of absence of con radial convergence and they make tornado absolutely I guess not just that so much but youed to the possibility of a Vortex breakdown and yeah I'm trying to understand how I am too in fact and I continue to try to incorporate friction I continue to turn it on and I continue to run simulations with it and it has a extremely dramatic effect on the Storm structure and this is all and it shouldn't really surprise us too much and I'm not entirely sure that the the way that cm1 is doing friction in version 16 is necessarily as good as version 18 George has done some things with friction but the the fact of the matter is when I turn friction on even if I use a very small drag coefficient or a very small zot rough this length it it affects the surface in a in a significant way you get a tornado that is I do get an EF5 tornado or a tornado at least I have done that but it's like it starts in well deep in the cold pool it's weak it just kind of so this is It's kind of infuriating in a way because the free slip I don't want to have the free slip condition but it it's wor working and maybe just maybe I mean I've seen enough video of tornadoes where the air the inflow into a tornado you a lot of a damage that is caused is the inflow that's coming in to fill the void that the tornadoes produced the air near the ground is moving so fast so if your first grid point is say 15 MERS above the ground and you're basically saying I'm going to slow all this air down it may just break things so that's why we need to go to higher and higher vertical resolution get a this would be a a two blue water size problem to get down to meter resolution near the ground but we're going to head in that direction and again I don't know and I continue to try to to incorporate friction and look at its effects in a in a reasonable manner can ask one more question so when I see this I'm wondering you made a comment about this affecting the mees cycle but when when I see this I'm wondering what what role is there right mes is there is there really a mesy is there it doesn't a quantifiable larger scale circulation at the ground or near the ground that precedes your your tornado formation when you look at like what's the typical vorticity threshold usually to determine a mesocyclone 0.01 yeah and we're talking about like we've got two orders of magnitude larger than that when you look at them as a cyclone like you take a like a good old cross I don't have it one ready but you do see the general large scale sort of tilting of ticity BZ Cyclone you'd expect but embedded within that it's a turbulent mess so that stream vorticity current is sort of maybe it's you know it's feeding part of the mesocyclone but it's it's too narrow to be the mesocyclone so it's like a new flow feature that's somehow incorporating and it must get in twined with the the tornado at a loft I haven't looked a off too much but it's a new thing I don't know if it exists in nature honestly we haven't seen it um we haven't really drilled down um it but it's certainly an efficient mechanism to transport horizontal vorticity that's generated in the forward flank into a nice compact region of vertical vorticity and that has to have an effect I mean you're adding all this lovely rotating air in this huge this updraft which is really strong so maybe there's some feedbacks going on where the pressure drops the the the the the low level updraft gets really strong and then that gets the tornado forming I I think there's probably some feedbacks involved in this what do you perceive as the biggest computational challenge going forward um biggest computational challenge going forward is going to be handling IO I mean I it sounds so tra but the truth is the model runs pretty well at it scales well the model will scale well getting that data to disk and and then being able to analyze it I can the the system I've built can handle the 20 simulation it can probably handle a 15 to a 10 met simulation as we get to like use more of the machine if we get that opportunity um it's going to be IO it's going to be handling all that data and then because I can the part of the storm that's important is pretty small compared to the whole thing so I can just save the data the way my D my IO works is I can just save data in that region because they're in separate files toss out the rest of it I don't want to but then I could at least do analysis on the tornado so but it's really that's that's the main problem um and properly incorporating friction it we may need to get down to just ridiculously high resolution at the surface to get friction right and that's going to be challenging actually if you use the Daran profiling tool to look at IO performance is that relevant to your work or you pretty much know where the iio bottlenecks are just wish was faster um the io bottlenecks Daran I I've just started to use it a little bit it hasn't given me any information that's useful a lot of the io problems I see I think have to do with the fact that it's it's you know a shared resource and sometimes IO takes a long time when other times it doesn't and I'm not you know and I'm actually working with with NCSA on trying to sort of figure out how to speed up iow and such so we're sort of profiling doing things but the buffer the core driver of the hdf core driver should be really fast because you're at least when you're writing data to memory right you're writing data to memory you don't have to send it to IO so that should be super super fast and sometimes it's not and I don't know why so there's some there's yeah we're trying to figure out some of these things um but the truth is we've managed to I can create a petabyte of data in 12 hours on Blue Waters easily challenge accepted maybe not but you know I mean my point is we can get but then it's like what do you do with it and what if you know I've had individual luster uh file individual luster machines run out of data that's always fun one of your files it says you know uh you're out of you're out of space in this in this no you're not it's just a big luster file so well one of the actual luster things out of space so there's these all these issues of the io where you you lose a you lose a piece of your storm because some Hardware failure or some software thing so those challenges are also pretty big one last question so you sharing the dinner with the community or some I my plans are to share both the data and to share the code that I've written for instance to get the data to do and visit but I'm not there yet um I just transitioned to a new job I just you know I'm just starting to get into a more research Focus position um so I want to sort of make sure I get my steak in the ground before I give all my cool tools away but the truth is yeah I I do intend to I would love to uh actually put this on like a uh somewhere it could be curated and we could do all people could play with it and maybe do you know do research with it seriously but I get the first cut okay please please after I get you know the grants worked out then we'll give it away but seriously thank you you

Channel: Leigh Orf's Thunderstorm Research

Views: 92,483

Keywords:

Id: 8Prk_TT1CsE

Length: 34min 30sec (2070 seconds)

Published: Wed Jun 22 2016