The big debate on best practice Monte Carlo modelling
THE BIG DEBATE: BEST-PRACTICE APPLICATION OF MONTE CARLO SIMULATION IN PROBABILISTIC COST RISK ESTIMATION
It’s not surprising the Monte Carlo method is often a topic of vibrant discussion amongst our Aquenta planning, estimating and risk teams. As you’ll see below, the process itself looks relatively simple at first (as most things do in risk management and project controls). Although, there are various tricks and tactics often used along the way that lead to different approaches and most significantly, different outcomes – there is a great debate around best practice Monte Carlo modelling and simulation, particularly for probabilistic scheduling and cost estimation.
From memory, the first time I used the Monte Carlo method was to assess the range of possible afflux impacts (in hydrology, afflux is defined as a rise in the water level immediately upstream of and due to a natural or artificial obstruction, like a new bridge) while completing my first master degree in Hydraulic Design. It then took me another 10-15 years before I realised I needed to go back to UNSW to fully understand the quantification of risks and simulation modelling in delivery of major projects. You should never underestimate the need for long hours of manual iterations to fully understand the process.
So what is the Monte Carlo method? It is best described as a problem solving technique used to approximate the probability of certain outcomes by running multiple trial runs, called simulations, using random variables. Typically, a simulation will consist of 2,500 to 10,000 iterations in order to reach a steady-state result. The results of the simulation include risk-adjusted estimates and corresponding statistical estimate distributions; these provide the decision maker with a range of possible outcomes with a minimum and maximum value. The method has been successfully used in scientific applications for about 70 years in various applications, approaches and interpretations, and is recognised by ISO 31000 as a standard in risk management and risk assessment.
Stanislaw Ulam is often credited with inventing the Monte Carlo method in 1946 while pondering the probabilities of winning a card game of solitaire! The Polish born mathematician worked for John von Neumann on the United States’ Manhattan Project during World War II, and is primarily known for designing the hydrogen bomb with Edward Teller in 1951. He published his first paper on the Monte Carlo method in 1949.
In 2017, our schedule and cost risk assessments are vastly more complex, including economic factors such as rate uncertainties, cost estimating errors, and statistical uncertainty inherent in the estimate/schedule. Cost estimating risk assessment takes into account the cost, schedule, and contingent risks that are then planned back into the cost estimate.
A number of good industry practices state that the contingency element of any cost estimate needs to account for the likelihood and cost impact of three factors: