Combining Model Selection Procedures for Online Prediction

Abstract

Here we give a technique for online prediction that uses different model selection principles (MSP's) at different times. The central idea is that each MSP in a class is associated with a collection of models for which it is best suited so that the the data can be used to choose an MSP. Then, the MSP chosen is used with the data to choose a model, and the parameters of the model are estimated so that predictions can be made. Depending on the degree of discrepancy between the predicted values and the actual outcomes one may update the parameters within a model, reuse the MSP to rechoose the model and estimate its parameters, or start all over again rechoosing the MSP. Our main formal result is a theorem which gives conditions under which our technique performs better than always using the same MSP. We also discuss circumstances under which dropping data points may lead to better predictions.