Implications on feature detection when using the benefit–cost ratio

dc.contributor.authorJagdhuber, Rudolf
dc.contributor.authorRahnenführer, Jörg
dc.date.accessioned2022-03-29T09:38:14Z
dc.date.available2022-03-29T09:38:14Z
dc.date.issued2021-06-03
dc.description.abstractIn many practical machine learning applications, there are two objectives: one is to maximize predictive accuracy and the other is to minimize costs of the resulting model. These costs of individual features may be financial costs, but can also refer to other aspects, for example, evaluation time. Feature selection addresses both objectives, as it reduces the number of features and can improve the generalization ability of the model. If costs differ between features, the feature selection needs to trade-off the individual benefit and cost of each feature. A popular trade-off choice is the ratio of both, the benefit–cost ratio (BCR). In this paper, we analyze implications of using this measure with special focus to the ability to distinguish relevant features from noise. We perform simulation studies for different cost and data settings and obtain detection rates of relevant features and empirical distributions of the trade-off ratio. Our simulation studies exposed a clear impact of the cost setting on the detection rate. In situations with large cost differences and small effect sizes, the BCR missed relevant features and preferred cheap noise features. We conclude that a trade-off between predictive performance and costs without a controlling hyperparameter can easily overemphasize very cheap noise features. While the simple benefit–cost ratio offers an easy solution to incorporate costs, it is important to be aware of its risks. Avoiding costs close to 0, rescaling large cost differences, or using a hyperparameter trade-off are ways to counteract the adverse effects exposed in this paper.en
dc.identifier.urihttp://hdl.handle.net/2003/40833
dc.identifier.urihttp://dx.doi.org/10.17877/DE290R-22690
dc.language.isoende
dc.relation.ispartofseriesSN computer science;Vol. 2. 2021, Issue 4, Art. No 316
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectFeature costsen
dc.subjectFeature detectionen
dc.subjectBenefit–cost ratioen
dc.subjectFeature selectionen
dc.subjectCost-sensitive learningen
dc.subject.ddc310
dc.titleImplications on feature detection when using the benefit–cost ratioen
dc.typeTextde
dc.type.publicationtypearticlede
dcterms.accessRightsopen access
eldorado.secondarypublicationtruede
eldorado.secondarypublication.primarycitationSN computer science. Vol. 2. 2021, Issue 4, Art. No 316en
eldorado.secondarypublication.primaryidentifierhttps://doi.org/10.1007/s42979-021-00705-6de

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Jagdhuber-Rahnenführer2021_Article_ImplicationsOnFeatureDetection.pdf
Size:
1.31 MB
Format:
Adobe Portable Document Format
Description:
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
4.85 KB
Format:
Item-specific license agreed upon to submission
Description: