Skip to content

How to compare GARCH volatility forecast performance? #607

@BdeBuman

Description

@BdeBuman

I have a question regarding the evaluation of volatility forecasts using the arch_model class. I am very new, so I sincerely apologise if this is obvious, or if this is the wrong place to ask.

I want to forecast the daily volatility (for a test set of roughly 1000 obs) of stock returns using a GARCH model, and concretely using the arch package. After doing so, I want to measure the forecasting performance using the mean squared error.

I construct a model using mean adjusted and scaled returns arch_model(x_var, mean="Zero", vol="Garch", p=1, q=1) and I fit the model using model.fit(last_obs="2018-07-03", update_freq=10) to produce a fixed window forecast as in the tutorial in the documentation.

What I am having trouble with is:

  1. Correctly understanding the ARCHModelForecast object. Is it correct that the variance attribute is the prediction for the daily volatility?
  2. If yes, what value do I compare it to to calculate the MSE? My understanding for GARCH models was that you use squared returns as a proxy for volatility, but my model forecasts are nowhere near the returns, squared returns, or any other feature I have in my data set. Is it correct that I would compare the forecast values to the squared returns at the according time step to evaluate prediction performance?
  3. If yes, is the model I fitted just garbage? I copied the output for model_fit.summary() below. Any sugestions for improvements?

Before I go fiddle with different parameters, I want to make sure that my understanding of the concepts is correct.

I can post more code or an example of my dataset if this helps. Many thanks for your help!

Zero Mean - GARCH Model Results                         
=================================================================================
Dep. Variable:     Returns mean adjusted   R-squared:                       0.000
Mean Model:                    Zero Mean   Adj. R-squared:                  0.000
Vol Model:                         GARCH   Log-Likelihood:               -8545.06
Distribution:                     Normal   AIC:                           17096.1
Method:               Maximum Likelihood   BIC:                           17115.1
                                           No. Observations:                 4110
Date:                   Thu, Sep 01 2022   Df Residuals:                     4110
Time:                           11:40:24   Df Model:                            0
                              Volatility Model                             
===========================================================================
                 coef    std err          t      P>|t|     95.0% Conf. Int.
---------------------------------------------------------------------------
omega          0.0454  6.394e-02      0.710      0.478 [-7.994e-02,  0.171]
alpha[1]       0.0612  5.064e-02      1.209      0.227 [-3.804e-02,  0.160]
beta[1]        0.9319  6.049e-02     15.406  1.500e-53    [  0.813,  1.050]
===========================================================================

Covariance estimator: robust

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions