Research / BFI Working PaperOct 16, 2020

How Well Generative Adversarial Networks Learn Distributions

This paper studies the rates of convergence for learning distributions implicitly with the adversarial framework and Generative Adversarial Networks (GAN), which subsume Wasserstein, Sobolev, MMD GAN, and Generalized/Simulated Method of Moments (GMM/SMM) as special cases. We study a wide range of parametric and nonparametric target distributions, under a host of objective evaluation metrics. We investigate how to obtain a good statistical guarantee for GANs through the lens of regularization. On the nonparametric end, we derive the optimal minimax rates for distribution estimation under the adversarial framework. On the parametric end, we establish a theory for general neural network classes (including deep leaky ReLU networks), that characterizes the interplay on the choice of generator and discriminator pair. We discover and isolate a new notion of regularization, called the generator-discriminator-pair regularization, that sheds light on the advantage of GANs compared to classical parametric and nonparametric approaches for explicit distribution estimation. We develop novel oracle inequalities as the main technical tools for analyzing GANs, which is of independent interest.

Additional Materials

More Research From These Scholars

BFI Working Paper Oct 16, 2020

Mehler’s Formula, Branching Process, and Compositional Kernels of Deep Neural Networks

Tengyuan Liang, Hai Tran-Bach
Topics:  Technology & Innovation
BFI Working Paper Apr 6, 2021

Universal Prediction Band via Semi-Definite Programming

Tengyuan Liang
Topics:  Uncategorized
BFI Working Paper Oct 16, 2020

A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-L1-Norm Interpolated Classifiers

Tengyuan Liang, Pragya Sur
Topics:  Technology & Innovation