Regularization and Model Selection in the Context of Density Estimation

Abstract

We propose a new information theoretically based optimization criterion for the estimation of mixture density models and compare it with other methods based on maximum likelihood and maximum a posterio estimation. For the optimization, we employ an evolutionary algorithm which estimates both structure and parameters of the model. Experimental results show that the chosen approach compares favourably with other methods for estimation problems with few sample data as well as for problems where the underlying density is non-stationary.