Sparse logistic regression utilizing cardinality constraints and information criteria

In this paper we address the problem of estimating a sparse parameter vector that defines a logistic regression. The problem is then solved using two approaches: i) inequality constrained Maximum Likelihood estimation and ii) penalized Maximum Likelihood which is closely related to Information Criteria such as AIC. For the promotion of sparsity, we utilize a nonlinear constraint based on the ℓ0 (pseudo) norm of the parameter vector. The corresponding optimization problem is solved using an equivalent representation of the problem that is simpler to solve. We illustrate the benefits of our proposal with an example that is inspired by a gene selection problem in DNA microarrays.