Partial orderings of default predictions

Abstract

We compare and generalize various partial orderings of probability forecasters according to the quality of their predictions. It appears that the calibration requirement is quite at odds with the possibility of some such ordering. However, if the requirements of calibration and identical sets of debtors are relaxed, comparability obtains more easily. Taking default predictions in the credit rating industry as an example, we show for a data base of 5333 (Moody’s) and 6505 ten-year default predictions (S&P), that Moody’s and S&P cannot be ordered neither according to their grade distributions given default or non-default or to their Ginicurves, but Moody’s dominate S&P with respect to the ROC-criterion.