Comparison between the regression depth method and the support vector machine to approximate the minimum number of misclassifications

Abstract

The minimum number of misclassifications achievable with afine hyperplanes on a given set of labeled points is a key quantity in both statistics and computational learning theory. However, determining this quantity exactly is essentially NP-hard, c.f. Simon and van Horn (1995). Hence, there is a need to find reasonable approximation procedures. This paper compares three approaches to approximating the minimum number of misclassifications achievable with afine hyperplanes. The first approach is based on the regression depth method of Rousseeuw and Hubert (1999) in linear regression models. We compare the results of the regression depth method with the support vector machine approach proposed by Vapnik (1998), and a heuristic search algorithm.