Sankhya: The Indian Journal of Statistics

2001, Volume 63, Series A, Pt. 2, pp. 229--249

COMBINING MODEL SELECTION PROCEDURES FOR ONLINE PREDICTION

By

B. CLARKE, University of British Columbia, Vancouver, Canada

SUMMARY. Here we give a technique for online prediction that uses different model selection principles (MSP's) at different times. The central idea is that each MSP is associated with a collection of models for which it is best suited. This means one can use the data to choose an MSP. Then, the MSP chosen is used with the data to choose a model, and the parameters of the model are estimated so that predictions can be made. Depending on the degree of discrepancy between the predicted values and the actual outcomes one may update the parameters within a model, re-use the MSP to rechoose the model and estimate its parameters, or start all over again rechoosing the MSP. Our main formal result is a theorem which gives conditions under which our technique performs better than always using the same MSP. We also discuss circumstances under which dropping data points may lead to better predictions.

AMS (1991) subject classification. Primary 62M20; secondary 62L99.

Key words and phrases. Prequential statistics, model selection, model mis-specification, model meta-selection, online prediction.

Full paper (PDF)