Risk Minimization and Minimum Description for Linear Discriminant Functions
Haldun Aytug, Gary J. Koehler, Ling He
INFORMS Journal on Computing · 2008 · Volume 20 · Issue 2
Abstract
Statistical learning theory provides a formal criterion for learning a concept from examples. This theory addresses directly the trade-off in empirical fit and generalization. In practice, this leads to the structural risk-minimization principle where one minimizes a bound on the overall risk functional. For learning linear discriminant functions, this bound is impacted by the minimum of two terms—the dimension and the inverse of the margin. A popular and powerful learning mechanism, support vector machines, focuses on maximizing the margin. We compare this to methods that focus on minimizing the dimensionality, which, coincidentally, fulfills another useful criterion—the minimum description length principle.
Keywords
linear discriminant functions, structural risk-minimization principle, the minimum description length principle, VC-dimension, margin, risk bound, empirical risk, genetic algorithm, support vector machines
Enable JavaScript for citation tools and additional paper interactions.