Active Learning for Classification With Abstention
Classification with abstention refers to the classification problems in which the learner can also abstain from declaring a label, i.e., a “don’t know” option. It models several applications such as medical diagnostic systems, voice assistant devices and content filtering. In (Shekhar et al., 2019), we proposed and analyzed the first active learning algorithm for this problem motivated by the approach used in our GP bandits work (Shekhar and Javidi, 2018). The scheme works for two abstention models: fixed-cost and bounded-rate and is general enough to work under several active learning paradigms (pool-based, stream-based and membership query). The algorithm proposed in (Shekhar et al., 2019) performs better than prior passive methods, both theoretically and in experiments. Furthermore, we also demonstrate the optimality of our algorithm by deriving matching lower bounds on excess risk.