Predictive accuracy of index volatility: GARCH family models vs. XGBoost Model

Abstract

Volatility signifies the degree of price variation in financial markets influenced by multiple factors, prompting the development of models and statistical tools to forecast its inherently chaotic nature. The main aim of the paper is to do a comparative analysis of prominent models, GARCH type models and the XGBoost machine learning model, to analyze the suitability of models and to measure and predict the volatility of selected international indices, including the S&P 500 (United States), TAIEX (Taiwan), and MSE TOP-20 (Mongolia). Furthermore, this research explores the applicability and implementation of models using the garch and xgboost package in the R programming language. The empirical result shows that: (1) The sGARCH model was chosen for the S&P 500 and Taiwan indices, while the eGARCH model was found to be more suitable for capturing the volatility dynamics of the MSE Top 20 index due to its optimal performance across RMSE, MSE, MAPE and SMAPE error term metrics. Moreover, the S&P 500 index was modeled with a student-t distribution, TWII with a generalized error distribution, and the MSE Top 20 index with a skew-student t-distribution, leveraging skewness for enhanced accuracy and robustness. (2) The XGBoost shows the superior performance compared to the fitted GARCH models based on the error terms.