A HYBRID MACHINE LEARNING AND DEEP LEARNING APPROACH FOR PREDICTING COST OVERRUNS AND DELAYS IN IT PROJECTS
Keywords:
IT project management, cost overrun prediction, schedule delay prediction, machine learning, deep learning, hybrid ensemble, Random Forest, gradient boosting, SVR, MLP, explainable AI.Abstract
Cost overruns and schedule delays remain persistent challenges in information technology (IT) project management, often resulting in increased expenses, missed deadlines, and reduced project performance. These issues commonly stem from inaccurate cost and time estimates, changing project requirements, inadequate resource allocation, technical complexity, and ineffective stakeholder communication. Despite advances in project management methodologies and tools, these challenges are still difficult to manage because of the dynamic and unpredictable nature of IT projects. This paper implements and empirically evaluates a hybrid machine-learning and deep-learning framework that jointly predicts percentage cost overrun and percentage schedule delay from information available at project initiation. Using a dataset of 5,000 software projects, we train a Random Forest (RF), a gradient-boosted tree model (a scikit-learn implementation of the XGBoost approach), Support Vector Regression (SVR) with an RBF kernel, and a three-hidden-layer Multilayer Perceptron (MLP), and we combine them through a validation-weighted hybrid ensemble. The models are compared against a median baseline on a held-out test set (n = 750) using the mean absolute error (MAE), root mean squared error (RMSE), and coefficient of determination (R²), with project-level bootstrap confidence intervals. The hybrid ensemble reduces the test MAE by roughly 35% (cost) and 36% (schedule) relative to the baseline, reaching R² values of 0.655 and 0.718, respectively. The hybrid ensemble is statistically indistinguishable from the best single model (Random Forest) for both targets. Permutation importance shows that the predictive signal comes almost entirely from the planned budget and planned duration, whereas project size, team size, requirement changes, developer experience, and complexity contribute essentially nothing. A data audit further shows that the planned and actual values in this dataset are uncorrelated (|r| < 0.02) and that the binary Delay and Budget Overrun labels are exact functions of the actual values. These findings indicate that the high R² is largely a mechanical consequence of the ratio-based target definition and that the dataset is likely synthetic.
