Outer Products and Stochastic Approximation Algorithms in a Heavy-tailed and Long-range Dependent Setting

Abstract

Classical time-series theories are mainly concerned with the statistical analysis of light-tailed and short-range dependent stationary linear processes. Applications in network theory and financial mathematics lead us to consider time series models with heavy tails and long memory. Heavy-tailed data exhibits frequent extremes and infinite variance, while positively-correlated long memory data displays great serial momentum or inertia. Heavy-tailed data with long-range dependence has been observed in a plethora of empirical data set over the last fifty years and so. Methodological and theoretical results as well as a considerable portion of applied work in this thesis address long-range dependence and heavy-tailed types of the data. The first contribution of this thesis is the development of Marcinkiewicz strong law of large numbers for outer products of multivariate linear processes while handling long-range dependent and heavy-tailed data structure. This result is used to obtain Marcinkiewicz strong law of large numbers for non-linear function of partial sums, sample auto-covariances and linear processes in a stochastic approximation setting. The next part of the result is on developing almost sure convergence rates for linear stochastic approximation algorithms under some assumptions that are implied by Marcinkiewicz strong law of large numbers. Finally, we verify our results experimentally in the stochastic approximation setting while handling all gains, long-range dependence and heavy tails and addressing the optimal polynomial rate of convergence by establishing results akin to the Marcinkiewicz strong law of large numbers.