From our discussions with many project managers (PM) and business analysts (BA), there are several topics that seem to strike terror into their souls and sour the looks on their faces. In this column over the next several months, we are going to deal with one of those topics – the use of Monte Carlo simulations (MCS) in the improvement of estimation activities for projects. OK, that we have probably lost 50% of the readers that began this paragraph, and before we lose the remainder, let us say that this will be a very EASY foray into the world of quantitative simulations. We are not going to make you learn large statistical or probability density functions. Learning how to improve your estimation processes using quantitative techniques is quite enjoyable and has the benefit of making your projects easier to manage given that you improve on the accuracy of your estimates of time, cost, and even resource allocation. So do not fear the Monte Carlo – it can be quite illustrative and effective. It also is not too bad when impressing your colleagues and key stakeholders.
We are going to take this article and present MCS is a very easy and understandable format. In the subsequent monthly presentations, we will take this introduction and build on it until you will be able to apply MCS and will most likely see MCS everywhere in your work. Actually, seeing MCS everywhere once you understand them is a form of the frequency illusion bias where something that has recently come to your attention (this article on MCS) suddenly seems to appear with improbable frequency shortly after the awareness is realized. OK, we will stop with the cognitive biases for now.
What precisely is a Monte Carlo simulation? Want the academic definition? OK – here it is: “Monte Carlo simulations or methods are a class of computational algorithms that rely on repeated random sampling to obtain numerical results.” In other words, Monte Carlo simulations repeatedly solve equations using a random range a values for the equations unknowns and then look to see if patterns start to form in the results. Before computers, doing MCS was mind numbing and time consuming. Computers even desktop and laptops using powerful spreadsheet and mathematical tools such as Excel and Mathematica can now easily solve for thousands and even millions of “repeated runs” using randomly generated values for an equations inputs while the system tracks and plots the results using very easy to interpret graphical or tabular formats.
Before dissecting just how a MCS works, let us take a look at a very simple example to aid us in understanding how MCS can help us with our project estimation requirements. In this example, taken from a blog post on the very interesting and useful Excel web site: chandoo.org, you can see that the MCS is not an intimidating as you may have first thought. This example simply shows that the model creator wants to know how his profit from a gold mine is going vary given changes (random and undeterminable) in the exchange rates between Australian Dollars and US Dollars. A very useful question to answer if you are mining down under, but selling up yonder.
Figure 5 [Figure 1] Simple Monte Carlo Simulation
Do not get too lost in the above example, but suffice to say that the model shows how the profit from the gold mine will vary if we vary the exchange rate of A$ to U$ from a value of 0.86 to 0.98. The MCS is a perfect tool for answering this question or other question like:
1. What is the average project completion time if we vary the resource utilization from 80% to 90% for our chief Java developer?
2. What impact on the project budget will a variation in funding release date of plus or minus 2 weeks have?
3. What will the impact be to the schedule if the contractor on-boarding process takes between 4.5 and 5.8 weeks?
All of these questions can be guessed at (the normal way of producing estimates on projects), or we can build – which we will over the next 4-5 articles – a very simple simulation model based on the concepts and principles of MCS. It is very easy yet looks quite powerful and significant. When using estimation results from MCS, we have yet to have anyone question or dispute our results, and our “after action reviews” showed that our MCS results versus the “just pull it out of your ear” guessimations by even the so called subject matter experts (SME) were more accurate over 67% of the time.
MCS is used in many areas of project management:
- Risk analysis,
- Estimation analysis for durations, costs, and resources,
- Schedule impact analysis,
- Quality control and defect profile analysis, and even
- Project on-time completion probability analysis
Think of the improvement to your project management analysis and reporting as they begin to show how you are taking your project management and business analysis to the next level of sophistication and effectiveness not to mention how you are improving the estimations for time, cost, and resources thus supporting better budgeting of all your project consumables. This will make your project sponsors, organizational management, and resources managers smile with satisfaction – all through the learning and utilization of MCS techniques.
MCS are not difficult once you get past all the academic and statistical scare tactics. Trust us, we have your backs on this technique, and would be very happy to have you come back and rejoin us over the next few issues as we show you exactly how you and your project team can use the powerful MCS technique to improve your estimation processes. We will be providing examples, detailed graphics, and even the Excel spreadsheets that you can use to help you harness this effective and interesting estimation tool.
Monte Carlo Simulations, Part II
In our last article, the first in a long, yet gentle exploration of Monte Carlo type simulations for use within the project management and business analysis disciplines, we discussed the value of considering more advanced less infantile analysis techniques that can be used in the solution of:
- Duration, costing, and resource availability estimation,
- Risk probability and impact estimations,
- Portfolio and ideation prioritizations,
- Funding stability simulations,
- Requirements instability estimations,
- Sensitivity, variance, and gap analysis,
- Schedule and completion probability analysis,
- Stakeholder analysis,
- Issue costing and impact simulation,
- Mitigation planning, and
- Task interdependency impact analysis
to name just a few applications of advanced statistical analysis techniques that can definitely add value to our professions.
Many asked after last month’s article, why use Monte Carlo Simulations (MCS) in applications such as estimation, and completion analysis? The main reason is that most companies or organizations do NOT complete post-project completion de-briefings (lessons learned) nor archive the important completed project information that would provide historical data points for future similar projects to utilize. So without such valuable and much more appropriate data, the project manager (PM) and business analyst (BA) is left with either querying their cadre of subject matter experts (SME), search the Internet, or do what they have always done – guess!
MCS based models can take a place along with these other valuable techniques of estimation and/or measurement determinations as you improve your estimation skills and accuracy. Remember the whole purpose of simulations is to more accurately estimate a future value of interest that may be needed to provide valuable information to decision makers about the efficacy of funding your project over those in competition with your organization’s limited resources. Decision makers make decisions when they are comfortable about the underlying assumptions and accuracy of the estimates that comprise the business case on which their decisions are requested. MCS based models can help to improve your accuracy in providing these estimates.
What are the strengths and weaknesses of MCS? All techniques have them, and we need to understand those that impact and result from the use of MCS so we do not misuse them or attempt application where none is effective.
MCS typically provide the following advantages or strengths:
- MCS can improve insight into how uncertainty impacts a model,
- What-if scenarios can be easily and timely investigated,
- They can provide support for most types of input variable distribution functions,
- Account for many variable correlations and impacts,
- Can be used to assess input variability and uncertainty, and
- Can be accomplishes on most desktop computers without expert assistance
MCS does have shortcomings or weaknesses as well:
- Simulations must be able to define and quantify input variables,
- Input variables distribution probabilities are not guaranteed accurate,
- May require more computing power than available on the desktop (if MCS is complex),
- Interpretations of the output probability distribution needs to be reduced to graphics or images for most decisions makers to make sense of or use in their decision making
- MCS may intimidate the uninitiated
The basics of MCS is that a model of interest, say a duration that varies according to percentage of availability for the responsible resource, the skill of the responsible resources, the number of working days in the task accomplishment period, the difficulty of the task assignment, and novelty of the task (well-known task versus unknown). These variables can all have various input values (distribution of values) that can be expressed as a probability distribution function (PDF) where the possible values of a particular input are associated with the probability of that value occurring in reality. A PDF can take many different shapes or profiles:
- Normal (the well-known Gaussian or bell-shaped curve),
- Binomial (yes/no answers)
- Uniform (all possibilities are equally likely),
- Chi and Chi-squared distributions,
- Exponential distribution,
and the list is almost endless as the above link will attest which is why one of the weaknesses of the MCS is the assumptions that are made about the correctness of the PDF that most accurately models a particular input variable. If the incorrect PDF is used to model an input variable, well, ‘garbage in, garbage out.’
The next major concept of MCS is that the inputs to the model, say Profits = Revenue – Expenses (P=R-E), can varying either in sync or not with other input variables. In other words, some input variables may have correlative association so that they are not truly independent. For most MCS used in project management this concept can be ignored, but just be aware that if there is correlative input association the results may be a bit less accurate. But moving on, all input variables can vary through their distribution ranges with corresponding probability of each value determining the shape of the distribution. Varying input variables one at a time and then recalculating the resultant outcomes is why the MCS based models were not very popular before the desktop computer revolution. Now most desktop computer running Microsoft’s Excel can easily run thousands if not millions of simulations without breaking a sweat. It is the power of using a random generator to ‘choose’ a value of an input variable according to its distribution function, and plugging that value along with the other input variables chosen randomly from their distribution functions into the model (P=R-E) to determine a particular value or scenario result, in our simple model – profit. Doing this thousands or millions of times and tracking the outcome of the above, Profit, begins to show a tendency of the model when calculated with the provided randomized input values to approach a probability function of its own. This simply means that if your inputs are chosen according to a PDF your output will also be a probability density function as well – no single answer here. In the above model, profit will take on the shape of a PDF from which you can use other techniques to determine an effective equation or pattern.
The resulting model outputs are tracked and plotted to show how they are trending, and with some simple mathematics even a predictive equation can be determined for future situations where the inputs are known and the equation can then provide a quick, easy solution value – estimation improvement in action.
In our next episode, we will develop an actual model for the duration estimation of a task within a construction project where the model will use 4 to 5 variables from which a task duration can be estimated, but with the additional advantage that the model will allow the PM or BA to determine ‘what-if’ scenarios given the variability of the input values according to either probability or actual deterministic fixations. Think of how valuable such a simulator would be when reality of the input variables such as temperature, humidity, concrete mixture, and lane shutdown availability all begin to vary at the same time. A MCS simulator allowing for the variability of all these factors plus more could now produce an output from which real time decisions on ‘if to go ahead and pour the ramp tomorrow’ are made.