Abstract
Advancements in data collection technology have led to the emergence of large-scale data in real-world applications, necessitating scalable and flexible statistical tools to address complex problems. Scalable nonparametric learning methods have become essential for capturing intricate relationships without imposing rigid parametric assumptions. This thesis introduces novel nonparametric statistical techniques designed to improve scalability while preserving flexibility and robustness in both Bayesian Statistics and Frequentist Statistics.
The first project focuses on developing flexible posterior approximations that account for skewness, multimodality, and bounded support. Automatic Differentiation Variational Inference (ADVI) is efficient in learning probabilistic models. Classic ADVI relies on the parametric approach to approximate the posterior. In this project, we develop
a spline-based nonparametric approximation approach that enables flexible posterior approximation for distributions with complicated structures, such as skewness, multimodality, and bounded support. Compared with widely-used nonparametric variational inference methods, the proposed method is easy to implement and adaptive to various data structures. By adopting the spline approximation, we derive a lower bound of the importance weighted autoencoder and establish the asymptotic
consistency. Experiments demonstrate the efficiency of the proposed method in approximating complex posterior distributions and improving the performance of generative models with incomplete data.
Classic Variational Inference (VI) methods often are developed based on mean-field assumptions, which will ignore the underlying latent dependent structures. For the second project, we propose Structured Nonparametric Variational Inference (SN-VI), a novel framework for modeling complex dependencies among latent variables in posterior approximation, leveraging multivariate spline techniques. Unlike traditional methods that rely on the mean-field assumption, SN-VI preserves intricate latent variable dependencies, thus provides flexible approximation of posteriors with arbitrary shapes. We establish theoretical guarantees, including the lower bound of the importance-weighted autoencoder objective and asymptotic consistency in posterior estimation. Additionally, we introduce a scalable algorithm that automatically identifies dependent
latent variables and their dependency structure without manual specification. Simulation studies illustrate SN-VI’s effectiveness in approximating posterior distributions with bounded support and complex dependencies. Applications to real-world data demonstrate SN-VI’s potential to enhance the performance of generative models,
particularly in scenarios with incomplete or noisy data.
The rapid advancement of modern technology has led to the generation of vast amounts of large-scale spatial-temporal datasets, which offer valuable insights into human behavior. The task of identifying when and where changes occur in spatial-temporal processes has gained significant attention in recent years. In this project, we propose a generalized spatial-temporal modeling framework that captures the trends in the spatiotemporal process and identifies regions with quick changes. To achieve this, we utilize tensor product splines over triangular prismatic partitions to approximate the unknown spatial-temporal trend. A piecewise-penalty function is then imposed to efficiently identify the regions with dynamic changes, employing a computationally efficient algorithm. We conduct simulation studies and apply our proposed method to mobile location data in Baltimore, demonstrating its effectiveness in efficiently recovering regions with dynamic changes over time.
