Adjusted Evaluation Measures for Asymmetrically Important Data

George-Jason Siouris; Despoina Skilogianni; Alex Karagrigoriou

doi:10.33119/ERFIN.2019.4.1.3

George-Jason Siouris Department of Statistics and Actuarial-Financial Mathematics, University of the Aegean, Greece
Despoina Skilogianni Department of Statistics and Actuarial-Financial Mathematics, University of the Aegean, Greece
Alex Karagrigoriou Department of Statistics and Actuarial-Financial Mathematics, University of the Aegean, Greece http://orcid.org/0000-0002-4919-2133

DOI: https://doi.org/10.33119/ERFIN.2019.4.1.3

Keywords: PVaR, Violation Ratios, Low Price Effect, Low Price Correction, Backtesting, Evaluation Measures

Abstract

In this paper we introduce adjustments for standard evaluation measures appropriate for the analysis of data with asymmetrical importance. In risk analysis, it is understood that the returns of an asset do not all provide the same amount of information. This asymmetry of information is crucial for choosing the most appropriate model and evaluating its forecasting ability. In risk analysis, measures like value at risk (VaR) and expected shortfall (ES) concentrate on the left tail of the distribution of returns so that failures in fitting a model on the right tail are not important. Therefore, when we estimate the VaR of an asset, the days of violations are more important than the days of non-violations. The proposed adjustments take into consideration the asymmetry in importance and are filling the gap in the theory of evaluation of percentiles measures. The measures are divided into fixed partition, based on prior information or the goal of forecasting, and non fixed partition, based on the time proximity of the model failure. The performance of the proposed measures is illustrated with the use of a stock from the industrial metals and minerals index of the American Stock Exchange (NYSE MKT), as well as a warrant, from the Athens Exchange (ATHEX).

References

Acerbi, C. and Tasche, D. (2002). Expected shortfall: a natural coherent alternative to value at risk. Economic Notes by Banca Monte dei Paschi di Siena SpA, 31(2):379–388.

Al-Hawamdeh, S. (2008). Knowledge Management: Competencies and Professionalism, volume 7. World Scientific, Singapore.

Aladag, C. H., Egrioglu, E., Gunay, S., and Basaran, M. A. (2010). Improving weighted information criterion by using optimization. Journal of computational and applied mathematics, 233(10):2683–2687.

Artzner, P., Heath, D., Delbaen, F., and Eber, J.-M. (1997). Thinking coherentlyg. Risk, 10:68–71.

Artzner, P., Heath, D., Delbaen, F., and Eber, J.-M. (1999). Coherent measures of risk. Mathematical Finance, 9(3):203–228.

Asadabadi, M. R., Saberi, M., and Chang, E. (2018). Targets of Unequal Importance Using the Concept of Stratification in a Big Data Environment. International Journal of Fuzzy Systems, pages 1–12.

Broda, S. A. and Paolella, M. S. (2011). Expected shortfall for distributions in finance. In Statistical tools for finance and insurance, pages 57–99. Springer, London.

Christoffersen, P. (1998). Evaluating interval forecasts. International Economic Review, 39:841–862.

Creal, D., Koopman, S. J., and Andre, L. (2013). Generalized autoregressive score models with applications. Journal of Applied Econometrics, 28:777–795.

Das, K., Jiang, J., and Rao, J. (2004). Mean squared error of empirical predictor. Annals of Statistics, 32(2):818–840.

Gonzalez-Rivera, G., Lee, T.-H., and Mishra, S. (2004). Forecasting volatility: A reality check based on option pricing, utility function, value-at-risk, and predictive likelihood. International Journal of Forecasting, 20:629–645.

Harvey, A. (2013). Generalized autoregressive score models with applications. Dynamic models for volatility and heavy tails: with applications to financial and economic time series, 52.

Imbens, G. W., Newey, W. K., and Ridder, G. (2005). Mean-square-error calculations for average treatment effects. https://dx.doi.org/10.2139/ssrn.954748.

Kupiec, P. (1995). Techniques for verifying the accuracy of risk measurement models. Journal of Derivatives, 3:73–84.

Singh, S., Singh, D. S., and Kumar, S. (2014). Modified Mean Square Error Algorithm with Reduced Cost of Training and Simulation Time for Character Recognition in Backpropagation Neural Network. In Proceedings of the International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA) 2013, pages 137–145. Springer, London.

Siouris, G.-J. and Karagrigoriou, A. (2017). A low price correction for improved volatility estimation and forecasting. Risks, 5(3):45.

Siouris, G.-J., Skilogianni, D., and Karagrigoriou, A. (2019). Post model correction in value at risk and expected shortfall. International Journal of Mathematics, Engineering and Management Sciences, 4(3):542–566.

Sokolova, M. and Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4):427–437.

Wang, Z. and Bovik, A. C. (2009). Mean squared error: Love it or leave it? A new look at signal fidelity measures. IEEE Signal Processing Magazine, 26(1):98–117.