Spring 2023 Data Science Seminar Series: Bobby Gramacy: Deep Gaussian Process Surrogates for Computer Expirements

Speaker: Robert B. Gramacy, Ph.D., Professor of Statistics, Virginia Tech

Faculty Host: Rui Tuo, ISEN

Abstract: Deep Gaussian processes (DGPs) upgrade ordinary GPs through functional composition, in which intermediate GP layers warp the original inputs, providing flexibility to model non-stationary dynamics. Recent applications in machine learning favor approximate, optimization-based inference for fast predictions, but applications to computer surrogate modeling — with an eye towards downstream tasks like calibration, Bayesian optimization, and input sensitivity analysis — demand broader uncertainty quantification (UQ). We prioritize UQ through full posterior integration in a Bayesian scheme, hinging on elliptical slice sampling the latent layers. We demonstrate how our DGP’s non-stationary flexibility, combined with appropriate UQ, allows for active learning: a virtuous cycle of data acquisition and model updating that departs from traditional space-filling design and yields more accurate surrogates for fixed simulation effort. But not all simulation campaigns can be developed sequentially, and many existing computer experiments are simply too big for full DGP posterior integration because of cubic scaling bottlenecks. For this case we introduce the Vecchia approximation, popular for ordinary GPs in spatial data settings. We show that Vecchia-induced sparsity of Cholesky factors allows for linear computational scaling without compromising DGP accuracy or UQ. We vet both active learning and Vecchia-approximated DGPs on numerous illustrative examples and a real simulation involving drag on satellites in low-Earth orbit. We showcase implementation in the deepgp package for R on CRAN.

Biography: Dr. Robert B. Gramacy is Professor of Statistics at Virginia Tech and affiliate faculty in VT’s Computational Modeling and Data Analytics program, and a Fellow of the American Statistical Association (ASA). He currently serves as Editor-in-Chief for Technometrics, and as President for the ASA’s Uncertainty Quantification Interest Group. Recently he completed tours as President of the ASA’s Section on Physical and Engineering Sciences, and as Treasurer for the International Society of Bayesian Analysis. Dr. Gramacy’s research interests include Bayesian modeling methodology, statistical computing, Monte Carlo inference, nonparametric regression, sequential design, and optimization under uncertainty. He recently published a fully reproducible (and open source) textbook called “Surrogates: Gaussian process modeling, design and optimization for the applied sciences.”

Link to PDF Version

You can also click this link to join the seminar

For more information about TAMIDS Seminar Series, please contact Ms. Jennifer South at jsouth@tamu.edu