Constructing models for credit ratings and default probabilities

Talking about coefficients of variables, it can be concluded, that their sign is different from our expectation in some cases. Still, signs of the largest coefficient (for variable NIM) and of the coefficient of the only variable connected with liquidity of a bank (CL_E) are consistent with our expectations. Opposite sign of coefficient of lnA can be explained by the opposite coefficient of lnA2, as in total their effect is as expected (in the Model 3 lnA has a negative sign). While opposite signs of measures connected with capital adequacy can be interpreted as the problem with holding too much capital, which in turn decreases competitive positions of a bank.

3.2 Modelling probabilities of the default separately

banking credit financial

Same modelling techniques could be applied to the Probability of the Default model. Firstly, Step-Wise procedure should be made in order to obtain best combination of initial explanatory variables. Thereafter macroeconomic and quadratic variables can be added. Finally, if new variables have low significance, we can apply Principle Component Analysis to increase the overall significance and get rid of strongly correlated variables.

Models received during Step-Wise procedure are summarized below. Model 1 there is the initial model with maximum number of individual variables, Model 2 excludes some insignificant ones and Model 3 has only significant and uncorrelated variables, which still can be used for estimation of each part of CAMELS.

Table 7. Step-Wise procedure for CR model

* - significant at 10%, ** - significant at 5%, *** - significant at 1%.

Together with information on coefficients significance (can be found in annex 2), AIC/BIC show that the third model should be the selected one for estimating PD basing solely on individual information. As it was with the model for ratings, macro and quadratic variables should be added. For the Probability of the Default model those variables are even more important, as event of default strongly depends on the systematic risks and operational environment. Again, macro coefficients would be very correlated and, because of that, we need to apply PCA here. In total we received following three models.

Table 8. Models for estimation of PD

* - significant at 10%, ** - significant at 5%, *** - significant at 1%.

Before proceeding to the final choice of the Probability of the Default model, another issue connected with defaults should be examined. Due to relatively low number of defaulted banks in our sample (unbalanced sample), models tend to underestimate default probabilities. In order to overcome this issue, we can change the sample and increase number of observations with the dependent variable equal to 1 (He and Garcia, 2009).

This can be done by the two ways. Firstly, number of non-defaulted banks can be randomly reduced. For this purpose, random numbers should be assigned to every solvent bank and 10% of them should be excluded. Sample with excluded banks would be called Balanced (-). Another option is to, on the contrary, add defaulted banks. This is done by randomly adding observations with dependent variable equal to 1 in our sample. This sample would be called Balanced (+). After estimating all three models (3, 4, 5) on the three samples, following results were received.

Table 9. Comparison of PD models

From AIC/BIC and information about significance and correlation of variables, we can conclude that the Model 5 estimated on the Balanced (-) sample has the highest significance among all three models. Therefore, this model should be chosen.

3.3 Forecasting and measuring the forecast power of separate models

This section of the diploma paper would be focused on predicting values of credit ratings and PDs with the use of models estimated in the previous section. Both measures would be predicted basing on all available data for explanatory and dependent variables.

Starting with prediction of credit ratings, we estimate predicted ratings for each bank in each period and, afterwards, compare those figures to the actual average ratings given by agencies of our interest. Differences between actual and predicted ratings are in the table 10 and in the histogram below.

Table 10. Forecasting power of CR model

Graph 1

From these findings it can be seen that Credit Rating model has good forecasting power, as more than 80% of predicted values are less than 3 points different from actual values. It is a good result, as 3 points deviation is usually captured by one or two changes in the actual scale. So, if the actual rating is AA, predicted value would be somewhere between AA- and AA+ with large probability.

Another thing to mention here is the skewness of the histogram of deviations to the right. That means that the assumption about overfitting issues of rating models is proven. It is so because predicted ratings tend to be one or more points larger than actual ones, while larger rating means worse financial stability of a bank. That is the main reason, why this paper is aimed at combing Credit Rating model with Probability of the Default model, as PD model is expected to have underfitting issue and to underestimate risk of a bankruptcy.

Next, proceeding to the prediction of PDs, we estimate default probabilities by the use of the final PD model received. As a result, estimated default probabilities appear. However, for the main purpose of this paper, we need to convert those probabilities into numerical ratings. It is so, because the final combined model would use ratings as a dependent variable.

To make ratings from PDs, we need to know probabilities with which bank goes bankrupt given that it has a certain rating. This would be estimated on the base of a paper by Pomasanov et al., 2008. It has a table, which shows which PD corresponds to each scale of S&P Russian scale (table is in the annex 5). Using this information, we can extrapolate and understand which PD corresponds to a numerical rating in our scale (8-21 with lag equal to 0,5). Results can be seen in a graph below.

Graph 2

Using this information, we are now able to calculate deviations of ratings received from predicted default probabilities and actual average ratings for each bank in each period. Results are shown in the histogram below.

Graph 3

Histogram shows that PD model has lower forecasting power, as there is a large amount of serious deviations from actual values and the range of them is significantly higher than in the model for ratings. It is consistent with our expectations and can be explained by the fact that there is no perfect correlation between PD and a rating and it is hard to distinguish PDs between similar rating scales. Therefore, we can expect lower significance of PD explanatory variable in the final combined model. Moreover, the distribution is skewed to the left (showing PD underestimation), which again proves the practical applicability of a combination of two models.

4. Combined model estimation and forecasting

After constructing two models separately, the reason while the combined model is needed is clear. PD model tend to underestimate risks, while CR tend to overestimate it and assign “too low” ratings. It can be seen on the following chart, where black bars correspond to the CR model, while grey ones are from PD model.

Graph 4

Combined model can be a linear or a non-linear combination of ratings and PDs estimated previously. It is in the form Yit = c + a1 x Ratit + a2 x Rat_Dit. Where Yit is the actual average rating, Ratit is a rating estimated from the Credit Rating model and Rat_Dit is a rating received from the transformed PD estimated by the Probability of the Default model. As it was noted earlier, ratings received by transforming PDs are expected to have lower influence on the dependent variable (smaller coefficient).

To choose the best combined model, four types of models were estimated and compared by information criteria:

· linear estimation of actual ratings on ratings predicted by both models.

· linear estimation of actual ratings on ratings predicted by both models and squared ratings estimated by CR model.

· ordered logistic estimation of actual ratings on same two sets of explanatory variables.

Table 11. Combined models

* - significant at 10%, ** - significant at 5%, *** - significant at 1%.

From table 11 we can see that the significance of coefficients increases after adding squared ratings, estimated by Credit Rating model. This can be explained by the decreasing marginal effect of those ratings. Moreover, we can notice that ordered logistic model has much higher AIC/BIC, which can be used as an argument in favour of linear models. However, we still need to choose ordered logistic model for estimation of the combined model due to the potential biases in estimation of linear models and the nature of ratings.

All in all, final combined model is chosen as an ordered logistic estimation of actual ratings on ratings from the CR model, ratings from the CR model squared and ratings transformed from default probabilities predicted by the PD model. All coefficients are significant and both estimated ratings have coefficients with positive signs, which is consistent with expectations.

Graph 5

From the plot of deviations from actual values it can be noticed that the mode is in the 0 and the overall distribution is similar to a normal, as it has less skewness and flatter tails. That potentially shows that the combined model gives better predictions than two separate models, which is consistent with the main hypothesis of this diploma paper.

Next we can proceed to testing out-of-sample forecasting power of this model. For doing this, model should be estimated on the whole dataset, but for time periods up to 29 only, so that three years (12 quarters) are left “out-of-sample”. Next, predictions can be made for those time periods. Differences between these predicted values and actual ratings are summarized in the table below and showed in comparison with those figures received from the original Credit Rating model.

Table 12. Comparison of the forecasting power

From table 12 it can be concluded that research in this paper has achieved its main goal - the increase in the forecast power (out-of-sample). Comparing to the separate CR model, percentage of predictions which fall in the one-point interval near the actual value is increased by more than 7%, while percentage of observations which fall in the tree points interval near the actual value has reached 88%.

Conclusion

This paper focused on constructing models for credit ratings and default probabilities and, afterwards, combining them. From the results achieved, it can be argued that the main hypothesis was proven and the combined model, which increases the forecast power, was introduced.

It is important to mention, that this paper is aimed to introduce a methodology, rather than a single final model. It is so, because dataset continuously changes and it is not universal, as here it was based only on publically available information, while other researchers can add variables connected to other types of information, including qualitative expert opinions. Moreover, the set of variables used is not unique and can differ for sets of banks (f.e. it will be different for banks from other regions or of another size). In addition, models used for constructing separate predictions can be significantly better, like complex non-parametric models.

All of this means that there is a huge place for improvements in estimating financial stability of a bank. But the methodology introduced in this paper would have strong influence on future researches on this topic, as it can be successfully applied with other sets of data, models and variables

References

1. Ayvazian, S.A., Golovan, S.V., Karminsky, A.M., Peresetsky, A.A. (2011). Approaches to a comparison of rating scales. Prikladnaia ekonometrika (Applied econometrics), 3 (23), 13-40.

2. Bulatov, Y.D. (2016). Systematization of data for modeling credit ratings of banking organizations. Course paper, HSE.

3. Bulatov, Y.D. (2017). Modelling the relationship between credit ratings of Russian public financial institutions and credit cycle indicators. Course paper, HSE.

4. Central Bank of Russia (2016). Creation of a system of comparison of rating scales of credit rating agencies (mapping).

5. Dyachkova, N.F., Karminsky, A.M. (2016). Comparison of rating scales for financial institutions. Upravlenie finansovimi riskami (Financial risk management), 4(48).

6. Karminsky, A.M., Khromova, E., (2016). Extended Modelling of Banks' Credit Ratings, Procedia Computer Science, (91), 201-210.

7. Karminsky, A.M., Khromova, E., (2016). Modelling banks' credit ratings of international agencies, Eurasian Economic Review, (6), 341-363.

8. Karminsky, A.M., Kostrov, A.V. (2014). The probability of default in Russian banking. Eurasia Business and Economics Society, 110-125.

9. Karminsky, A.M., Kostrov, A.V. (2017). The back side of banking in Russia: forecasting bank failures with negative capital. International Journal of Computational Economics and Econometrics, 7(1/2), 170-209.

10. Karminsky, A.M., Kostrov, A.V., Murzenkov, T.N. (2012). Modeling the probability of default of Russian banks using econometric methods. Financial Economics, Preprint WP7.

11. Karminsky, A.M., Morgunov, A.V., Bogdanov, P.M. (2015). The Assessment of Default Probability for the Project Finance Transactions. Journal of the New Economic Association, 2(26), 99-122.

12. Karminsky, A.M., Peresetsky, A.A. (2007). Models of ratings of international rating agencies. Prikladnaia ekonometrika (Applied econometrics), 1, 3-19.

13. Karminsky, A.M., Sosurko, V.V. (2010). Features of modeling the international ratings of banks. Upravlenie finansovimi riskami (Financial risk management), 4, 292-305.

14. Karminsky, A.M., Sosurko, V.V. (2011). The unified rating mapping: a step from the myth to reality. Bankovskoe delo (Banking), 6, 58-63.

15. Peresetsky, A.A., Karminsky, A.M. (2008). Models for Moody's Bank Ratings. Bank of Finland, BOFIT Discussion Papers, 17.

16. Peresetsky, A.A., Karminsky, A.M. (2011). Models for Moody's Bank Ratings. Frontiers in Finance and Economics, 8(1), 88-110.

17. Peresetsky, A.A., Karminsky, A.M., Golovan, S. (2004). Probability of default models of Russian banks. Bank of Finland BOFIT Discussion paper 21/2004.

18. Pomasanov, M., Vlasov, A. (2008). Calibration of national rating systems.Rynokcennichbumag (Security market), 74-79.

19. Solovjova, I. (2016). New Approaches to Regulating the Activities of Rating Agencies: a Comparative Analysis. Procedia, Social and Behavioral Sciences, 229, 115 - 125.

20. Totmyanina, K.M. (2011). Overview of probability models of default. Upravlenie finansovimi riskami (Financial risk management), 1(25).

21. Vasilyuk, А.А., Karminsky, А.М. (2011). Modeling credit ratings of domestic banks based on Russian accounting standards. Upravlenie finansovimi riskami (Financial risk management), 3, 194-205.

22. Bedendo, M., Cathcart, L., El-Jahel, L. (2017). Reputational shocks and the information content of credit ratings. Journal of Financial Stability, 34, 44-60.

23. Bissoondoyal-Bheenick, E., Treepongkaruna, S. (2011). An analysis of the determinants of bank ratings: comparison across ratings agencies. Australian Journal of Management, 36(3), 405-424.

24. Calabrese, R., Giudici, P. (2015). Estimating bank default with generalised extreme value regression models. Journal of the Operational Research Society, 66(11), 1783-1792.

25. Caporale, G.M., Matousek, R., Stewart, C. (2012). Ratings assignments: Lessons from international banks. Journal of International Money and Finance, 31, 1593-1606.

26. Florez-Lopez, R., Ramon-Jeronimo, J.M. (2014). The Journal of the Operational Research Society, 65(3), 416-434.

27. Godlewski, C.J. (2007). Are Ratings Consistent with Default Probabilities? Empirical Evidence on Banks in Emerging Market Economies. Emerging Markets Finance & Trade, 43(4), 5-23.

28. Gogas, P., Papadimitriou, T., Agrapetidou, A. (2014). Forecasting bank credit ratings. The Journal of Risk Finance, 15(2), 195-209.

29. Hбjek, P., Olej, V. (2010). Credit rating modelling by kernel-based approaches with supervised and semi-supervised learning. Springer, 5.

30. He, H., Garcia, E. (2009), Learning from Imbalanced Data, IEEE Trans. Knowledge and Data Engineering, 21(9), 1263-1284.

31. Lazarides, T., Drimpetas, E. (2016). Defining the factors of Fitch rankings in the European banking sector. Eurasian Economiv Review, 6, 315-339.

32. Pompella, M., Dicanio, A. (2016). Ratings based Inference and Credit Risk: Detecting likely-to-fail Banks with the PC-Mahalanobis Method. Economic Modelling, 67, 34-44.

33. Rцsch, D., Scheule, H. (2014). Forecasting probabilities of default and loss rates given default in the presence of selection. The Journal of the Operational Research Society, 65(3), 393-407.

Annex 1

CR models

Model 1

Model 2

Model 3

Model 4

Model 5

Annex 2

PD models

Model 6

Model 7

Model 8

Model 9

Model 10

Model 11

Annex 3

Correlation matrixes

Model 12

Model 13

Annex 4

Principal Component Analysis

PCA in CR model

Model 14

PCA in PD model

Model 15

Annex 5

Table 13. Transformation of PDs into the rating scale of S&P

Размещено на Allbest.ru