Quantitative Risk Analysis for overall project risk

I'm really I've got a mic front here so it's fine okay good morning everybody thank you very much for the welcome I'm just gonna turn this on I hope you can see at the back I'm afraid this is all it is I'm just this big so I'll try and make myself visible it's great to have the opportunity to open this session on quantitative risk analysis like what I want to do today in my half an hour or so and leaving some time for questions is to give you something of a thought leadership slightly stretching position piece about one of the applications of quantitative risk analysis that you might not have thought about and so what the idea in my presentation is to answer these two key questions which have been troubling me ever since I started working on risk analysis which I guess was about 1984 I was about six at the time I met Peter Simon in 1985 I think he was about eight actually you're younger than me Peter aren't you so but it was a long time ago and so I was doing risk analysis in a major engineering company and we went through did the the workshops and various things and the boss said so how risky is this project and I said well we've got 10 red risks and we've got 22 yellow risks and we've got 35 green risks and so he said and how risky is the project and I went I'll get back to you because it's that doesn't answer the question to give you writ a list of risks in the risk register and so I started thinking about that that really interesting problem very early on in my work as a risk specialist and I think we've come up with an up with an idea that gives you the answer so how do we answer these questions what how risky is your project and then what can we do about it and there are two levels at which we can answer the question let's get that arrow off there so one for the project manager one of the things the project manager needs to know about is what are the risks in my project and where do I look for those we find those in the risk register and we include them we describe them in our risk reports we talk about the top 10 risks and about trends in the number of risks in red yellow and green can a high medium and low priority and so on so that's where we answer the question about individual risks but that's not the answer to the other question how risky is your project which is something that the project owner the project sponsor or the the customer or client is going to want to know and the answer is not a list of risks or even a risk map with some kind of distribution that you get a kind of sense from looking at at how risky the project is there's a concept which is not the risks in the project but the risk of the project and they're quite different so this idea of overall riskiness we've come up with a term when I say we it's not just my own idea this is some of the sort of a thinkers in the industry and we cap captured these as you'll see in a moment in some of the leading guidelines we call it overall project risk overall as distinct from individuals so what is overall project risk if we look for some definitions the APM first defined this in the second edition of the pram guide over 10 years ago and it's also described in there in the latest edition of the APM body of knowledge so an uncertain event or set of circumstances that shouldn't occur will have an effect on objectives that's individual risk PMI has something very similar both in the latest PIM bot guide the project management body of knowledge chapter 11 but also in the practice standard uncertain event or condition if it occurs as positive or negative we know all that that's individual risks but also in those documents we also define this this term called overall project risk and if you've ever read the pram guide or the APM body of knowledge or the pin box guide you should know this so in the in the pram guide we say the overall risk is the exposure of stakeholders to the consequences of variation in outcomes and we say it's arises from individual risks and other things other sources of uncertainty not just individual risks PMI says the same thing not surprisingly since some of the same people were involved in these documents the effect of uncertainty on the project as a whole now you can't get that from your list of risks it's not very easy it's more than the sum of the individual on the project is this in scope for the project manager absolutely we're responsible for identifying the risks in the project and reporting on those and for managing them effectively the individual risks but we're also accountable to the project sponsor and the and the stakeholders for the overall riskiness of the project is this project going to succeed or not and that's the overall project risk these are both our areas of interest in the world of project management not just the list of risks and what we're doing about them so how do we put together something about the list of individual risks that we get from our risk register and other sources of uncertainty what whatever they may be to get a sense of how overall risky the project is and that's what I'd like to address in this presentation so obviously we need to identify where overall project risk comes from we need to quantify it respond and report in the time I've got available I only want to just cover the two which are highlighted here because we're talking about quantitative risk analysis how do we quantify that the answer to the question how risky is my project and then how do we tell people about it and I think those are the areas that that we might find most interesting but obviously the other aspects are important as well and there's a paper on my website which will describe some of those things so let's get straight into quantification that's I know what you're interested in that's why we're here here we've got the two definitions that kind of summary definitions from a p.m. and PMI of overall project risk it's the exposure of stakeholders to variation in the outcomes of the project is that good or bad how much can they take how big is it and how big how big would they be comfortable with it's also the effect of uncertainty on the project as a whole so as a whole we're then talking about overall project objectives the ultimate end delivery date the overall budgets the delivery of functionality the satisfying of customer requirements what's the effect of uncertainty on that as a whole so we've got to find some way of synthesizing the individual risks and other sources of uncertainty to answer these questions if we wanted to make them quantitative numbers then we could sort of distill those two as BECs down to these two questions in terms of the effect of uncertainty on the project as a whole how likely is the project to succeed or fail which of course is the same thing one is the obviously the inverse of the other and then the exposure of stakeholders to fit to variation in outcome or how much variation is there when we have a target could it be better or worse and if so by how much and clearly Monte Carlo simulation gives us the answer to these questions that's why we should be interested in quantitative risk analysis not just to say how much contingency do we need although that's part of the answer to this question not just to say which are the biggest risk drivers to to the critical path or to the overall budget but that's also important but to give us a sense of overall how risky is my project and then I can design overall project risk strategies to deal with the overall risk exposure and I think this is an area that we've been missing for 10 years since we first put it in the pram guide since we first put it in the PMI practice standard in 2009 and we said hey guys don't forget our responsibility as risk specialists advising project people or as project managers or program managers is to answer the question for the stakeholder how risky is the project and is that okay and this is what I'm going to do about it and we have the methodology we have the tools and techniques to answer that question we just haven't been doing it so I hope at the beginning of today what I'd like to do is to lift our thinking and remind us that there's a bigger context question as well as the individual detailed things about putting individual risks into our project schedule or our project budget or a combined cost schedule a project plan we do that in order to say how risky is the project as a whole so how are we going to do that well let's look at an example output just something very very simple this is a cost s curve and here we here we've got a variation in the likely project budget and there's an S curve that says you know it could be that this is the the probability of it being less than two million right there up to the probability of being less than 2.8 million and the probability obviously increases as we get a bigger a bigger overall budget so you will see these sorts of curves coming on probably every slide set from now on through the rest of the day and from this kind of standard output from quantitative risk analysis there's a number of important things we could learn if our target was 2.2 million then we can say well the chances of meeting that are about 23 percent so we immediately got some quantitative information which answers one of those questions what's the probability of success if we're aiming for this the answer is 23 percent we could also from this get a sense of the variation question so if we took not the very extremes because it's very unlikely we'll be right down here and very unlikely we'll be right up there but if we took a 5th to 95th percentile as a kind of a sensible realistic range of potential variation then the 5 to 95% point gives us this variation of about half a million from actually 2 point 1 2 to 2 point 6 that's a quantitative answer to an overall risk question we can also say what else can we get from this we can get an expected value the sort of average of all of these results it's generally a little more than 50% because these things tend to be right skewed so here we've got from this one the expected value is 2.35 against a target of 2.2 and we could also read off if we wanted to target values and the corresponding probability of achieving them so if we wanted to reset our budget target to something where there was an 85% chance of achieving it we should do that at about two point four five this is standard right so far but rather than just saying you know we've got all of these individual risks and and other sources of uncertainty that we've plowed into our risk model and then we're just looking at how they affect what the risk drivers are and so on what we can say from this is answering specifically our overall project risk questions how risky is the project and we could put numbers on it so now we can report to our senior stakeholders to our project sponsor to our customer and answer this question with numbers so if we wanted the question how likely is the project to succeed we have a clear answer on our current plan with the risks the threats and the opportunities that we currently know about and other sources of uncertainty with the responses that we've already baked into the plan there's a 23% chance of meeting the 2.2 million target and we also know that the expected value where we think we're going to come out on our current plan is 7% over that now that's useful information in terms of how risky is the overall project we are a budget is at risk by at least 7% that's what we expect on our current plan and the other question in terms of variation we can take that 595 spread and say the range there is half a million which is 22 percent of the project value so we're expecting to be 7% over and the range on that is plus or minus 11% 22 percent which means that the very best case might be 4% under budget but the worst case might be 18% over budget now these are useful numbers that we can start to have a discussion with our project sponsor or business owner about because they're quantitative they're not just all there's a lot of red in my risk map it's not there's just lots of risks on the list and what are we going to do about them now we have something which is about the overall risk exposure of the project budget and obviously we could do the same I was going to say schedule because my good friend David's here but for schedule this obviously depends what what what surely went to of course so you could do the same for schedule you could do an integrative model as well but here we have from our standard Monte Carlo output a direct readout of overall project risk in quantitative terms and these are the sorts of things we could or we might say we should be reporting to our project owner our project sponsor very clear information okay let's just have one other output which you might see from your quantitative risk analysis I hope you do and I'm going to use this because I'm going to come back to this towards the end of my presentation on reporting risk obviously if we start our project at some point in time we've not spent any money and if we were to plot then as time passes how much money we expect to have spent we could end up with one output that says a prediction of where we think will be in terms of cost and time and of course that's useless isn't it just saying well here's our spot value it's gonna cost this much it's gonna take this long that's our answer the reality is there's a whole range of answers and from our Monte Carlo simulation we can get a range of possible answers of how much it will cost on this vertical dimension and how long it will take on this horizontal dimension and it generally gives us this kind of scatter plot of potential results of variation in cost and time together from an integrated cost and time risk model which I know some of the other speakers will talk about and what we could do is draw some kind of best fit plot around this which might be useful at some later point in time and then we could say how do we get from here we haven't started yet we haven't spent any money to this range of possible outcomes and we could say well there's a very best case where we spend the minimum which is this value so down the lowest of these dots here and we take the shortest time which is the leftmost of the dots so to get to this point on the best-fit ellipse is the best possible way to complete the project and of course there's a worst-case which is up on the top right hand corner this is where we take loads of time and spend loads of money and then reality what we expect is going to be somewhere in between and ends up with the expected value of this cluster of potential cost time points now my good friend David Hewlett who you'll hear from the end of the day comes from the other side of the pond for some reason they call this a football plot I have no idea why you would call it a football plot we call it an eye ball plot which is sort of much more obvious on this side but you know over here it's rugby that it's played by men with odd shaped balls and and not football exactly that well it might be round of course but it usually isn't so this is a standard output from integrated cost and time risk model and I did my first one of these in 1984 this is not new this is not clever this is not sort of way advanced out there this is a standard output of looking at the variation of cost and time together on our projects and we should know how to do this why should we know how to do it well it looks pretty this is very impressive but actually gives us really useful information to report on which I'll come back to in just a moment well I'll come back to it now so talking about reporting and what we want to do is to obviously tell people once we've done this sort of analysis what have we found and what does it mean so obviously you don't just do your quantitative risk analysis once we update it as we go along at key milestones or at key phase points or key changes in the scope or major major rework in the project we rework our quantitative risk analysis because we want to know what is the overall risk exposure of the project at this point and now we've changed this and now we've redirected in this way and so we're monitoring the change not just of individual risks as we go along but the overall risk exposure of the project and so as we do these updates we should be talking to our stakeholders our project sponsors the business owner maybe the customer and saying this is the level of overall project risk using probability of success and variation around key targets and then we can say this is what drives it and David all come to this at the end of the day and this is what we're going to do about it and what we have done if this is where it's taking us and this is where we're going to be next time I come back and talk to you so it's really important that we tell people what we found because analysis must lead to action the point of doing analysis is so that we can shape their project strategy and tactics in order to make give ourselves the best chance of getting to the end where we want to be so we're going to monitor changes and trends I'm going to tell people about them well how how do you tell people about changes in the probability of success and in the variation of your overall project outcomes I don't think there are any standard reporting format so I've never come across any and I'm kind of looking at the speaker table and nobody's going oh yeah we've got those so you know we kind of make stuff up don't we and that's what we thought leaders do and so I've made a couple of things up that that you might like I'm just going to offer them to you and if there's any supplies in the room then you might want to you can borrow this it's not copy so here are a couple of components you might like to think about one is what I call the project risk barometer could have been a thermometer but I I thought I caught a barometer so if we just said here let's plot the depressing the probability of success probability of project success from out guaranteed failure to guaranteed success and we might have some kind of zones where we said if we're below a 50% probability of success that's not so good if we're in the West we'll count that as red because in the East they would think red is good and lucky and so on but that's what we do over here radies is a warning so there's less than 50% chance of succeeding bad news if it's in that kind of 50 to 80% well okay but what we really want to be is above an 80% chance of succeeding and then what we can do is actually plot overtime at our various various review points and milestones how our probability of success is changing this isn't really rocket science is it so we might say well when we do our first review there's only something like a 15% chance of succeeding but as we go through and we manage the risk on the project this probability of success hopefully is increased seeing here we've got time now at review number seven and we can see that we're into that nearly into that green zone is about a 75% chance of succeeding and if we were to do a kind of best fit trendline it looks like we should be okay by the end of the project so the barometer will give you that sense of plotting changes in our overall probability of success either on project duration project budget or some kind of combined project performance parameter well that's the right for the probability of success what about variation well we could actually put variability into this plot as well and so what we do is we say if we're on target that's kind of okay what we really want to be is ahead of target if we're below target or over target we're late or we're overspent that's that's a problem and so in addition to these kind of probability of success measures we could also plot something here that says this is where we expect to be best case um worst case this is actually in kind of this dimension coming out an s-curve right so you can see this is our 5% expected a 95 maybe and we can say well this is where it was oh dear this isn't very good but as time progresses maybe we get a bit more variability in the middle but it'll tighten up as we come towards the end and now we're on the sort of expecting to be okay which ties in with here and that and the variability is tightening up so we've got some kind of visual components of ways that we might report on this probability of success and overall variation and you might like to consider something like this and it will be very easy to produce those automatically from your software let me come back to the the football plot oh and a trend line on variability the eyeball football plot you remember this thing what could we do with that as I say I started producing these in about 1980 four or five and I thought they were very pretty and I was very impressed when I first saw one I think it might have been in one of your papers actually David but then I thought well okay so what good is it you know what can I do with this and I realized that this best fit line was actually quite useful so if we forget all the dots and the lines and the way that we got there we just focus on this end point then if my target is to finish on this date and to finish with this budget then that s curve can tell me where I expect or sorry that's a best-fit ellipse can tell me where I expect to be in how much off in terms of overall cost and overall time I expect to be and what the range of variation is in both dimensions so on my first review point I might say oh dear you know there's only a few of these results which actually fall at the target or within it most of them are on the wrong side I'm in trouble what we want to see is how does this change with time and I started plotting these ellipses as we went through the reviews in our projects 30 years ago and I was able to show this kind of thing that would come back for another review the ellipses got smaller because there's less variability we're managing the overall answer and see in the project and our expected value is getting closer to the target and of course what you want to see is as each review goes by we get closer and closer to the target of course what sometimes you see is it kind of wanders off into the middle distance but that's all obviously useful information as well so the the other dashboard components the barometer with the variability can give you information about individual milestones or individual budget targets or overall project cost or time but here we've got something which is two-dimensional and will give us that two-dimensional information the size of the ellipse is the variability in the project and the distance of the spot the expected spot from the target is something about our probability of success or failure and I've been doing these as I say for a long time it's a very interesting sort of visual target as as this is the kind of you know the focus point of the of the project and you do actually see it wandering around quite a lot we want to see it tightening in and eventually it'll be a spot we want it to be somewhere close to this one so what can we do with these football plots well the orientation of the plot so the angle of the cost time major axis of the ellipse tells you something about the correlation between cost and time so if you had an ellipse that was at 45 degrees what it tells you is that as time goes up cost goes up they're positively correlated it doesn't tell you which causes which but they're correlated and if it's in 45 degrees it tells you they're more or less equally balanced so when you get an increase in time you get an increase in cost there's obviously a scatter around these so that's an interesting extra piece of information in terms of the type of risk that we're exposed to and we could actually use that when we're plotting those variations in the ellipse as time goes by to see where it's taking us so if we've got these in a cost and time are positively correlated it goes up with a slope and it's about 45 degrees they're equally balanced I've shown it here in the middle but of course they could be anywhere around that kind of cost time quadrant here we're under spent in late or we're over spent and early or we're sort of in a in a good place it's the orientation we're looking at this time then actually we could think about other types of orientation and in my projects I've seen all sorts so here's one which says cost and time are negatively correlated as time goes up cost goes down that's interesting so it might be to do with our the way that we engage with our suppliers it might be to do with the way that we release our resources but this is an interesting case or maybe if we get a kind of more or less horizontal set of ellipses it tells you that most of the risk is to time because here's the variability we don't get very much vertical variability so it's really a time risk project or obviously if the ellipse is more upright then it's exposed to cost risk so there's a number of really interesting things that we can get from these kind of very simple standard outputs from our project quantitative risk analysis okay well I want to leave some time for questions so let me just wrap this up with a few thoughts about what we need to do as practitioners and maybe what the profession ought to be doing as well we know in this room in the APM risk seek in the in the PM PMI and and other professional bodies the Institute of risk management and so on that risk is important we wouldn't be here we wouldn't want to know about it otherwise but it matters at both the individual case what are the biggest threats that and the biggest opportunities the worst things that might happen the best things that might happen and what we can do about it but it also matters at the level of the overall project how risky is the project and what can I do about it that the overall project level it matters both of these things at the individual level and the overall level need to be met to be managed proactively and we need to evolve our practice to cover both of them now as I said right at the beginning the good news is that the guidelines have been talking about this for 10 years so there is guidance in the APM pram guide or in the PMI practice standard the latest edition of the PMI PIM Bock guide which I'm on the update team forum the vice-chair it's coming out in September has a lot more on how to manage overall project risk in Chapter 11 of the pinball guide so we are moving this forward but we need to do it and so the standards need to provide guidance and those of us who are in practice need to implement it and show people how to do it we have to deal with the risks in the project but we also have to deal with the risk of the project and depending on the audience that that we're speaking to then we need to be talking about both of these at different levels so my final questions to you are these two how risky is your project how do you know and what are you going to do about it once you find out and if this isn't part of your overall thinking and practice then maybe we do need to just modify and adjust the way that we approach risk management to include this really important bigger picture element the information is available in our standard quantitative risk analysis outputs we just need to think at that bigger picture level and then start talking about it so I hope you found that useful I will take have I got time for questions Keith is it ten minutes or something like that yeah so there's ten minutes for questions and I'll repeat the question back for the recording if anybody has any but thank you for your attention first of all can we put our hands together and just thank David

Channel: RiskDoctorVideo

Views: 21,591

Rating: 4.8125 out of 5

Keywords: risk, risk management, risk doctor, david hillson, quantitative risk analysis, monte carlo

Id: tSQvZlUdRTM

Length: 27min 45sec (1665 seconds)

Published: Wed Mar 23 2016