Learning from Positive and Unlabeled Data with a Selection Bias
International Conference on Learning Representations (ICLR), 2019.
- PU learning
- selection bias
- partial identification
- density ratio
In one sentence. PU learning normally assumes labeled positives are distributed like unlabeled positives. When labeling is selective that assumption fails; the classifier is then only partially identified, and what can be learned is a scoring function that preserves the order of the class posterior.
The assumption that usually breaks
Methods for learning a binary classifier from positive and unlabeled data typically assume that the labeled positives are identically distributed to the positives hiding in the unlabeled pool — labeling is "completely at random". Real labeling processes are rarely like that. Which positives get recorded depends on who was looked at, which channel they came through, or how obvious the case was, and that dependence is exactly a selection bias.
Under such a bias, the Bayes optimal classifier is not recoverable by conventional PU learning methods.
What the paper does instead
Rather than assuming the bias away, the paper characterizes what remains identifiable. Under mild assumptions the classifier is partially identified, and the proposed algorithm learns a scoring function that preserves the order induced by the class posterior. Ordering is enough for most uses: setting an appropriate threshold on the score yields a classifier, and ranking by the score is often the operational goal anyway.
Where it is used
The setting matches applications such as identifying prospective customers from a record of existing ones, where the customers on file were acquired through a selective process rather than sampled at random from the population of people who would have bought.