Alternatives to Gaussian Processes for Model Calibration

Rapid growth in computing power has improved the ability to simulate complex systems. In some applications, interest lies in combining simulations and field data (i.e., model calibration; Kennedy and O'Hagan, 2001). Gaussian processes (GPs) are used here because they (i) are a good non-parametric regression estimators; and, most importantly, (ii) they provide a foundation for statistical inference for deterministic simulators. In the latter case, since there is no randomness in the simulator observations, the predictive uncertainty is the result of possible sample paths the GP can take after conditioning on the data. In this work, we will investigate using an over-specified set of basis functions (say Legendre polynomials - e.g., see Xiu and Kamiadakis, 2003) to emulate the GP. Here, "over-specified" means the candidate set of bases can be much larger than the number observations. Model fitting will be set within a Bayesian framework, and thus the simulator can also be viewed as a random function a priori. The chosen set of bases can be as larger than the number of runs of the simulator to encourage interpolation. To fit the model, the selection of basis functions from a candidate set will be done using stochastic search variable selection (George and McCulloch, 1993) to switch between different sets of bases, of possibly different sizes - each iteration of the variable selection procedure can consider different selected basis functions from a candidate set. From a foundational view, the randomness in predictions at new locations comes from the different sets of basis functions that represent the observations - and any lack of interpolation can be attributed to the inability to resolve high frequency variation - this is a new viewpoint. The proposed work will implement this method (we have worked out the math) and compare the predictive performance of the proposed method with the GP in the model calibration context. The result of this methodology will be an approach the computationally faster than the GP, with similar predictive ability.