A Unified Framework for Debiased Machine Learning: Riesz Representer Fitting under Bregman Divergence

The bias that debiased machine learning removes

When a causal or structural parameter is estimated by plugging in a flexible machine learning fit, the regularization that makes the fit possible also leaves a first-order bias in the target parameter. Debiased machine learning removes it with a Neyman-orthogonal score, whose correction term is built from the Riesz representer of the target functional — inverse propensity weights in the case of the average treatment effect, and its analogues for other functionals.

Everything therefore rests on estimating that representer well. Several proposals exist — Riesz regression, tailored loss minimization, covariate balancing weights, density ratio fitting — and they have been developed largely in parallel, with separate motivations and separate analyses.

Generalized Riesz regression

This paper estimates the representer by fitting a representer model under a Bregman divergence. The choice of divergence is what distinguishes the existing methods:

• Squared distance recovers Riesz regression.
• Kullback–Leibler divergence recovers tailored loss minimization.
• Other members of the family give estimators that were not previously available.

The framework also generalizes Bregman-divergence density ratio fitting from density ratios to Riesz representers, so density ratio estimation becomes a special case rather than an analogy.

Two automatic properties

• Automatic covariate balancing. For suitable pairs of divergence and model specification (link function), the dual of the representer-fitting problem is a covariate balancing problem. Balance is not imposed as a side constraint; it falls out of the fitting objective.
• Automatic Neyman orthogonalization. Under the same specifications, the sample average of outcomes weighted by the estimated representer satisfies Neyman orthogonality without estimating the regression function. This reduces the estimation error of the orthogonal score, and it clarifies a concrete distinction between debiased machine learning and targeted maximum likelihood estimation.

Convergence is analyzed for both reproducing kernel Hilbert space and neural network model classes.

Software: genriesz

The framework is released as genriesz, a Python package. The user specifies the target functional, the representer model, and the Bregman divergence; the package constructs the appropriate link function, performs cross-fitting, and returns estimates with inference. Supported targets include average treatment effects, treatment-specific means, and marginal (derivative) effects.