Published in the Project Post-Gazette, February 2014
by Paul Lohnes, MBA, PMP
In our last article, the first in a long, yet gentle exploration of Monte Carlo type simulations for use within the project management and business analysis disciplines, we discussed the value of considering more advanced less infantile analysis techniques that can be used in the solution of:
- Duration, costing, and resource availability estimation,
- Risk probability and impact estimations,
- Portfolio and ideation prioritizations,
- Funding stability simulations,
- Requirements instability estimations,
- Sensitivity, variance, and gap analysis,
- Schedule and completion probability analysis,
- Stakeholder analysis,
- Issue costing and impact simulation,
- Mitigation planning, and
- Task interdependency impact analysis
to name just a few applications of advanced statistical analysis techniques that can definitely add value to our professions.
Many asked after last month’s article, why use Monte Carlo Simulations (MCS) in applications such as estimation, and completion analysis? The main reason is that most companies or organizations do NOT complete post-project completion de-briefings (lessons learned) nor archive the important completed project information that would provide historical data points for future similar projects to utilize. So without such valuable and much more appropriate data, the project manager (PM) and business analyst (BA) is left with either querying their cadre of subject matter experts (SME), search the Internet, or do what they have always done – guess!
MCS based models can take a place along with these other valuable techniques of estimation and/or measurement determinations as you improve your estimation skills and accuracy. Remember the whole purpose of simulations is to more accurately estimate a future value of interest that may be needed to provide valuable information to decision makers about the efficacy of funding your project over those in competition with your organization’s limited resources. Decision makers make decisions when they are comfortable about the underlying assumptions and accuracy of the estimates that comprise the business case on which their decisions are requested. MCS based models can help to improve your accuracy in providing these estimates.
What are the strengths and weaknesses of MCS? All techniques have them, and we need to understand those that impact and result from the use of MCS so we do not misuse them or attempt application where none is effective.
MCS typically provide the following advantages or strengths:
- MCS can improve insight into how uncertainty impacts a model,
- What-if scenarios can be easily and timely investigated,
- They can provide support for most types of input variable distribution functions,
- Account for many variable correlations and impacts,
- Can be used to assess input variability and uncertainty, and
- Can be accomplishes on most desktop computers without expert assistance
MCS does have shortcomings or weaknesses as well:
- Simulations must be able to define and quantify input variables,
- Input variables distribution probabilities are not guaranteed accurate,
- May require more computing power than available on the desktop (if MCS is complex),
- Interpretations of the output probability distribution needs to be reduced to graphics or images for most decisions makers to make sense of or use in their decision making
- MCS may intimidate the uninitiated
The basics of MCS is that a model of interest, say a duration that varies according to percentage of availability for the responsible resource, the skill of the responsible resources, the number of working days in the task accomplishment period, the difficulty of the task assignment, and novelty of the task (well-known task versus unknown). These variables can all have various input values (distribution of values) that can be expressed as a probability distribution function (PDF) where the possible values of a particular input are associated with the probability of that value occurring in reality. A PDF can take many different shapes or profiles:
- Normal (the well-known Gaussian or bell-shaped curve),
- Binomial (yes/no answers)
- Uniform (all possibilities are equally likely),
- Chi and Chi-squared distributions,
- Exponential distribution,
and the list is almost endless as the above link will attest which is why one of the weaknesses of the MCS is the assumptions that are made about the correctness of the PDF that most accurately models a particular input variable. If the incorrect PDF is used to model an input variable, well, ‘garbage in, garbage out.’
The next major concept of MCS is that the inputs to the model, say Profits = Revenue – Expenses (P=R-E), can varying either in sync or not with other input variables. In other words, some input variables may have correlative association so that they are not truly independent. For most MCS used in project management this concept can be ignored, but just be aware that if there is correlative input association the results may be a bit less accurate. But moving on, all input variables can vary through their distribution ranges with corresponding probability of each value determining the shape of the distribution. Varying input variables one at a time and then recalculating the resultant outcomes is why the MCS based models were not very popular before the desktop computer revolution. Now most desktop computer running Microsoft’s Excel can easily run thousands if not millions of simulations without breaking a sweat. It is the power of using a random generator to ‘choose’ a value of an input variable according to its distribution function, and plugging that value along with the other input variables chosen randomly from their distribution functions into the model (P=R-E) to determine a particular value or scenario result, in our simple model – profit. Doing this thousands or millions of times and tracking the outcome of the above, Profit, begins to show a tendency of the model when calculated with the provided randomized input values to approach a probability function of its own. This simply means that if your inputs are chosen according to a PDF your output will also be a probability density function as well – no single answer here. In the above model, profit will take on the shape of a PDF from which you can use other techniques to determine an effective equation or pattern.
The resulting model outputs are tracked and plotted to show how they are trending, and with some simple mathematics even a predictive equation can be determined for future situations where the inputs are known and the equation can then provide a quick, easy solution value – estimation improvement in action.
In our next episode, we will develop an actual model for the duration estimation of a task within a construction project where the model will use 4 to 5 variables from which a task duration can be estimated, but with the additional advantage that the model will allow the PM or BA to determine ‘what-if’ scenarios given the variability of the input values according to either probability or actual deterministic fixations. Think of how valuable such a simulator would be when reality of the input variables such as temperature, humidity, concrete mixture, and lane shutdown availability all begin to vary at the same time. A MCS simulator allowing for the variability of all these factors plus more could now produce an output from which real time decisions on ‘if to go ahead and pour the ramp tomorrow’ are made.