Density Ratio Estimation, Weak Supervision, and Anomaly Detection

• density ratio estimation
• PU learning
• anomaly detection
• Bregman divergence

The common object

Estimating two densities and dividing is a bad idea in high dimensions: the ratio is usually far better behaved than either density, and estimating it directly avoids solving a harder problem on the way to an easier one. Direct density ratio estimation is therefore the shared engine behind several problems that look unrelated at first — importance weighting under distribution shift, classification from partially labeled data, and anomaly detection framed as a comparison against a normal-data distribution.

Making direct estimation work with deep models

The standard formulation minimizes a Bregman divergence between a model and the true ratio. With flexible models this fails in a specific way: non-negative Bregman divergence minimization (ICML 2021) identifies train-loss hacking, where the empirical objective can be driven down without learning the ratio, and proposes a non-negative correction with a generalization error bound and strong results on inlier-based outlier detection.

Related work gives a unified view of probability divergences through maximum likelihood density ratio estimation (AISTATS 2023), bridging KL divergence and integral probability metrics.

Weakly supervised learning

• Learning from positive and unlabeled data with a selection bias (ICLR 2019) drops the usual assumption that labeled positives are representative of unlabeled positives, shows the classifier is then only partially identified, and learns a score that preserves the order of the class posterior.
• PUATE (NeurIPS 2025) carries the same observation scheme into causal inference: efficient ATE estimation from treated and unlabeled units.
• Double PU learning for potential-customer identification (CDEF 2025, arXiv:2506.00436) applies the framework where the labeling process is selective by construction.

Anomaly detection

Anomalies are characterized by how their distribution differs from that of normal data, which makes the density ratio a natural score when labeled anomalies are scarce or absent. Applied work in this direction includes audit sampling with statistical guarantees, where the question is not only which items look unusual but how much assurance a given sample provides.