Using Monte Carlo Simulation to Evaluate Performance of Forecasting Models in Project Control

Abstract

The construction sector is one of the largest in the world economy. Billions of dollars are spent every year to execute and deliver projects serving the public. However, internal and external factors – such as unexpected weather, financial difficulties, manpower shortages, and excessive change orders – can impact the performance of these projects and cause deviations from planned performance. Consequently, researchers are constantly developing methods to improve project performance during execution. Although the models are developed using advanced techniques, the validation techniques applied to assess their accuracy are flawed. One example of this can be found in schedule- and cost-forecasting models. These models are crucial in ensuring that project progress is regularly tracked and reviewed, areas requiring changes are identified, and the subsequent changes are initiated accordingly. However, most researchers use validation techniques that rely on comparing the developed method to other well-known methods using actual completed project data. This has several drawbacks that render the validation experiments inconclusive. Accordingly, this thesis focuses on designing a Monte Carlo simulation-based empirical experiment to improve the process of assessing and comparing the effectiveness of different methods, with a focus on forecasting methods. The experiment involves generating a large sample set of random activity-on-node networks that resemble actual construction projects. The actual cost and schedule performance of these projects is simulated using a Markov chain approach that mimics the uncertainty affecting the execution of these projects. To test the experiment, four forecasting methods are applied to predict the final project cost and duration given the randomly-generated planned and actual performance data: earned value analysis (EVA), earned schedule (ES), non-linear regression Gompertz growth model (NLR-GGM), and the Kalman filter forecasting model (KFFM). These methods are evaluated in terms of forecasting accuracy, timeliness, and stability under varying project complexities and lengths. By providing an in-depth analysis, researchers and practitioners can develop an enhanced understanding and insight into these methods. This research contributes to the current state-of-the-art by developing a means for validating and assessing cost- and schedule-forecasting models in a controlled, unbiased, and simulated environment.