Generative Modeling With Adversarial Training

A key limitation of existing generative models, specifically explicit density models such as autoregressive models, normalizing flows, VAEs, and energy-based models (EBMs), is that they tend to assign higher likelihood values on out-of-distribution data than on in-distribution data. In this work we investigate an adversarial training-based generative model that overcomes this limitation. Inspired by recent work that shows adversarially robust classifiers learn high-level, interpretable features, we investigate training a binary classifier to discriminate in-distribution data from adversarially perturbed out-of-distribution data. Our analysis shows that in this setup, the binary classifier learns the support of the in-distribution data, and the learning process is closely related to MCMC-based maximum likelihood learning of EBMs. The training objective of the binary classifier can also be interpreted as a maximin two-player zero-sum game, and is related to GANs' minimax game. Based on the above analysis, we propose improved training techniques for generative modeling with adversarial training (AT), and show that this AT generative model is capable of generating diverse and realistic images, and at the same time has the expected behavior on (normal and worst-case) out-of-distribution inputs. We further investigate the AT generative model's applications to image restoration (denoising and inpainting), image-to-image translation, detecting adversarial examples, (worst-case) out-of-distribution detection, and generative classification.