Authors: Duivesteijn, Wouter
Thaele, Julia
Title: Understanding Where Your Classifier Does (Not) Work - the SCaPE Model Class for Exceptional Model Mining
Language (ISO): en
Abstract: FACT, the First G-APD Cherenkov Telescope, detects air showers induced by high-energetic cosmic particles. It is desirable to classify a shower as being induced by a gamma ray or a background particle. Generally, it is nontrivial to get any feedback on the real-life training task, but we can attempt to understand how our classifier works by investigating its performance on Monte Carlo simulated data. To this end, in this paper we develop the SCaPE (Soft Classifier Performance Evaluation) model class for Exceptional Model Mining, which is a Local Pattern Mining framework devoted to highlighting unusual interplay between multiple targets. In our Monte Carlo simulated data, we take as targets the computed classifier probabilities and the binary column containing the ground truth: which kind of particle induced the corresponding shower. Using a newly developed quality measure based on ranking loss, the SCaPE model class highlights subspaces of the search space where the classifier performs particularly well or poorly. These subspaces arrive in terms of conditions on attributes of the data, hence they come in a language a domain expert understands, which should aid him in understanding where his/her classifier does (not) work. Additional experiments are carried out on nine UCI datasets. Found subgroups highlight subspaces whose difficulty for classification is corroborated by astrophysical interpretation, as well as subspaces that warrant further investigation.
Subject Headings: astrophysics
exceptional model mining
Cherenkov radiation
soft classifier
Subject Headings (RSWK): Astrophysik
Issue Date: 2014-09
Appears in Collections:Sonderforschungsbereich (SFB) 876

Files in This Item:
File Description SizeFormat 
duivesteijn_thaele_2014a.pdfDNB3.83 MBAdobe PDFView/Open

This item is protected by original copyright

If no CC-License is given, pleas contact the the creator, if you want to use thre resource other than only read it.