Greedy feature construction

Oglic, Dino; Gaertner, Thomas

Abstract

We present an effective method for supervised feature construction. The main goal of the approach is to construct a feature representation for which a set of linear hypotheses is of sufficient capacity -- large enough to contain a satisfactory solution to the considered problem and small enough to allow good generalization from a small number of training examples. We achieve this goal with a greedy procedure that constructs features by empirically fitting squared error residuals. The proposed constructive procedure is consistent and can output a rich set of features. The effectiveness of the approach is evaluated empirically by fitting a linear ridge regression model in the constructed feature space and our empirical results indicate a superior performance of our approach over competing methods.

Citation

Oglic, D., & Gaertner, T. Greedy feature construction. Presented at 30th Conference on Neural Information Processing Systems (NIPS 2016)

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 30th Conference on Neural Information Processing Systems (NIPS 2016)
• End Date: Dec 10, 2016
• Acceptance Date: Aug 12, 2016
• Publication Date: Dec 5, 2016
• Deposit Date: Nov 9, 2016
• Publicly Available Date: Dec 5, 2016
• Journal: Advances in Neural Information Processing Systems
• Electronic ISSN: 1049-5258
• Publisher: Massachusetts Institute of Technology Press
• Peer Reviewed: Peer Reviewed
• Volume: 29
• Public URL: https://nottingham-repository.worktribe.com/output/836488
• Publisher URL: http://papers.nips.cc/paper/6557-greedy-feature-construction
• Related Public URLs: http://papers.nips.cc/
• Contract Date: Nov 9, 2016