Active Learning for Classification With Abstention

Histogram of the samples queried by our proposed algorithm for simple 1D functions. See the slides above for further details of this example.

Classification with abstention refers to the classification problems in which the learner can also abstain from declaring a label, i.e., a “don’t know” option. It models several applications such as medical diagnostic systems, voice assistant devices and content filtering. In (Shekhar et al., 2019), we proposed and analyzed the first active learning algorithm for this problem motivated by the approach used in our GP bandits work (Shekhar and Javidi, 2018). The scheme works for two abstention models: fixed-cost and bounded-rate and is general enough to work under several active learning paradigms (pool-based, stream-based and membership query). The algorithm proposed in (Shekhar et al., 2019) performs better than prior passive methods, both theoretically and in experiments. Furthermore, we also demonstrate the optimality of our algorithm by deriving matching lower bounds on excess risk.

Shubhanshu Shekhar
Shubhanshu Shekhar
Ph.D. Candidate

Interested in sequential learning and optimization.