Sieve Estimators For Spatial Data

Estimation of a regression function, linking a set of features to an outcome of interest, is a fundamental statistical task. This dissertation focuses on the application of sieve estimators in modern statistical learning problems. The method of sieves, or estimation via basis expansion, has its roots in Fourier analysis. In the past decades, it has achieved much success in smaller sample size, lower dimensional data science problems. In this dissertation, we will demonstrate its effectiveness in modern statistical learning settings. Sieve estimators can achieve statistical and computational optimality (almost) simultaneously, which makes them very suitable for online and/or large scale nonparametric estimation tasks. Sieve estimators can also be applied to high-dimensional nonparametric problems. They can effectively alleviate the “curse of dimensionality” by leveraging additional structures such as feature sparsity. For each topic covered in this dissertation, we will present both theoretical discussion and a variety of numerical examples.

By extending the ideas of Ibragimov & Hasminski in the finite dimensional parameter estimation a large deviation inequality for a sieve estimator estimating a Hilbert space valued parameter is obtained. This sieve estimator corresponds to a sieve which consists of finite dimensional, compact, convex sets. The inequality suggests a procedure of consistent estimation of Hilbert space valued parameters and naturally provides the convergence rates of the resultant estimators. The usefulness of this approach is demonstrated by applying it to two examples; the first one deals with the estimation of the drift function in a linear stochastic differential equation and the second problem is of the intensity estimation of a nonstationary Poisson process. A detailed discussion of the convergence rates of our estimators and how they compare with the other estimators proposed in the literature is given in both cases. (sdw).

The random field model has been applied to model spatial heterogeneity for spatial data in many applications. The purpose of this dissertation is to explore statistical properties of noisy spatial data through estimation of the Gaussian random field. Large sample properties of the Maximum Likelihood Estimator (MLE) of an Onrstein-Uhlenbeck process model with measurement error are studied. The effect caused by adding measurement error, or "nugget," is revealed by the fixed region asymptotics of the MLE. The kriging predictor with estimated covariance is discussed under such models. An extension to regression models is proposed and its asymptotic properties are examined. The Gaussian random field is characterized by its corresponding covariance function. By means of constructing the multi-dimensional covariance function from one-dimensional covariance functions, some spatial process models applicable to both spatial and regression data are proposed. The estimation of covariance functions for these models is studied. Large sample theory for some estimators is provided.

This paper studies the two-step sieve M estimation of general semi/nonparametric models, where the second step involves sieve estimation of unknown functions that may use the nonparametric estimates from the first step as inputs, and the parameters of interest are functionals of unknown functions estimated in both steps. We establish the asymptotic normality of the plug-in two-step sieve M estimate of a functional that could be root-n estimable. They asymptotic variance may not have a closed form expression, but can be approximated by a sieve variance that characterizes the effect of the first-step estimation on the second-step estimates. We provide a simple consistent estimate of the sieve variance and hence a Wald type inference based on the Gaussian approximation. The finite sample performance of the two-step estimator and the proposed inference procedure are investigated in a simulation study.

Explosive growth in the size of spatial databases has highlighted the need for spatial data mining techniques to mine the interesting but implicit spatial patterns within these large databases. This book explores computational structure of the exact and approximate spatial autoregression (SAR) model solutions. Estimation of the parameters of the SAR model using Maximum Likelihood (ML) theory is computationally very expensive because of the need to compute the logarithm of the determinant (log-det) of a large matrix in the log-likelihood function. The second part of the book introduces theory on SAR model solutions. The third part of the book applies parallel processing techniques to the exact SAR model solutions. Parallel formulations of the SAR model parameter estimation procedure based on ML theory are probed using data parallelism with load-balancing techniques. Although this parallel implementation showed scalability up to eight processors, the exact SAR model solution still suffers from high computational complexity and memory requirements. These limitations have led the book to investigate serial and parallel approximate solutions for SAR model parameter estimation. In the fourth and fifth parts of the book, two candidate approximate-semi-sparse solutions of the SAR model based on Taylor's Series expansion and Chebyshev Polynomials are presented. Experiments show that the differences between exact and approximate SAR parameter estimates have no significant effect on the prediction accuracy. In the last part of the book, we developed a new ML based approximate SAR model solution and its variants in the next part of the thesis. The new approximate SAR model solution is called the Gauss-Lanczos approximated SAR model solution. We algebraically rank the error of the Chebyshev Polynomial approximation, Taylor's Series approximation and the Gauss-Lanczos approximation to the solution of the SAR model and its variants. In other words, we established a novel relationship between the error in the log-det term, which is the approximated term in the concentrated log-likelihood function and the error in estimating the SAR parameter for all of the approximate SAR model solutions.