First-Order AR(1) Noise Statistics

• First-order auto-regressive noise statistics is a time series model defined by xₜ = φxₜ₋₁ + eₜ, offering clear insights into mean, variance, and autocovariance structures.
• It employs estimation methodologies like least squares and Yule–Walker equations to robustly capture stationarity and short-term dependencies even under colored or long-memory noise.
• AR(1) models find practical applications across fields such as biology, astrophysics, and finance by effectively modeling temporal dependencies and dynamic stochastic behavior.

A first-order autoregressive process (AR(1)) constitutes a foundational model in time series analysis, describing both the persistence and stochastic behavior of noise and signals in diverse scientific domains. The defining feature is the linear recursion with coefficient $\phi$ acting on the previous value, often driven by noise that may possess a wide range of distributional and dependence properties. The AR(1) formalism enables direct calculation of statistical characteristics such as mean, variance, autocovariance, and spectral density, with precise implications for stationarity, memory, and model fitting. Modern research further addresses colored noise, long memory, time-varying autoregression, non-Gaussian innovations, and degenerate settings, elucidating robust inference procedures and important exceptions.

1. Formal Definition and Stationarity

An AR(1) process is given by:

$x_t = \phi x_{t-1} + e_t,$

where $e_t$ is a stochastic noise sequence. Classical AR(1) assumes $e_t$ to be zero-mean, independent, identically distributed (iid) white noise with variance $\sigma_e^2$ (Wafi, 2023, 0808.1021, 0709.2963), and $\phi \in (-1,1)$ ensures (weak) stationarity. More generally, processes with colored or dependent noise $e_t$ require separate consideration of their autocovariance structure, which governs stationarity and estimation (Proïa, 2013, Voutilainen et al., 2018, Chen et al., 2020).

For strictly stationary $x_t$ with zero mean,

$$E[x_t]=0, \quad \mathrm{Var}[x_t] = \frac{\sigma_e^2}{1-\phi^2}, \quad \gamma(k) = \mathrm{Cov}[x_t, x_{t-k}] = \frac{\sigma_e^2}{1-\phi^2} \phi^{|k|}.$$

These results are preserved when the noise retains zero mean and suitable covariance decay (Wafi, 2023, 0709.2963).

2. Autocovariances and Power Spectral Density

The AR(1) autocovariance function decays geometrically:

$$\gamma(k) = \gamma(0) \phi^{|k|}, \quad \gamma(0) = \frac{\sigma_e^2}{1-\phi^2}.$$

The autocorrelation is thus $\rho(k) = \phi^{|k|}$, signifying exponentially fast loss of memory. The power spectral density (PSD) in the frequency domain is given by:

$$S_x(e^{j\omega}) = \frac{\sigma_e^2}{|1 - \phi \, e^{-j\omega}|^2}$$

or, equivalently, for discrete frequency $v$,

$$S(v) = \frac{\sigma_e^2}{2\pi} \frac{1}{1 + \phi^2 - 2\phi\cos(2\pi v)}.$$

Importantly, AR(1) spectra show a pronounced plateau at low frequencies, a quasi-linear log-log decay over intermediate frequencies (with slope −2 for large $\phi$), and flatten at high frequencies, distinguishing AR(1) from true $1/f$ (power-law) spectra (0808.1021).

In colored and long-memory noise scenarios, the innovations $e_t$ possess autocovariance $\gamma_e(k)$, e.g., $\gamma_e(k) \sim |k|^{2H-2} L(|k|)$ for fractional Gaussian noise or ARFIMA contexts, with $H\in(1/2,1)$ (Chen et al., 2020). The spectral density near frequency zero is then nontrivial, e.g., $$h(\lambda)\sim C_H L(|\lambda|^{-1}) |\lambda|^{1-2H}$$ as $\lambda\to0$.

3. Estimation Procedures and Finite Sample Properties

Parameter estimation for AR(1) commonly employs least squares or moment-based methods:

$$\hat\phi = \frac{\sum_{t=1}^n x_t x_{t-1}}{\sum_{t=1}^n x_{t-1}^2}$$

which, under iid noise, is strongly consistent and asymptotically normal (Wafi, 2023, Chen et al., 2020, 0709.2963). In presence of noise autocorrelation, e.g., AR(1)-driven innovations, least squares estimators converge to a bias-adjusted value $\phi^*$ that depends on the noise autocorrelation parameter $\rho$ (Proïa, 2013, Bercu et al., 2011):

$\phi^* = \frac{\phi + \rho}{1 + \phi \rho}.$

For model fitting, it is essential to account for transient effects in finite length series. If initialization matches stationary variance ($\mathrm{Var}[x_1] = \sigma_e^2/(1-\phi^2)$), sample moments and autocorrelations closely track theoretical values; otherwise, transients decay as $|\sigma_1^2 - \sigma_x^2| \phi^{2(n-1)}$ (0709.2963). This motivates "warm start" conditions in simulation.

Batch least-squares identification generalizes to colored noise and time-varying AR(1) frameworks, with the accuracy governed by sample size $N$ and noise variance (Wafi, 2023, Gruber et al., 2022). For time-varying AR(1) processes:

$X_{t,T} = a(t/T) X_{t-1,T} + \varepsilon_{t,T},$

estimation rates depend not only on smoothness $\beta$ of $a(\cdot)$ but also on the shape parameter $\alpha$ of innovation tails; minimax rates interpolate between parametric ($T^{-1}$) and regular ($T^{-\beta/(2\beta+1)}$) regimes (Gruber et al., 2022).

4. Noise Properties: Colored, Long-Memory, and Degenerate Cases

When the innovation process deviates from white noise, characterization of noise statistics becomes central to AR(1) analysis. For strictly stationary $X_t$:

$X_t = \phi X_{t-1} + Z_t,$

with colored noise $Z_t$ determined from the autocovariances of $X_t$ (Voutilainen et al., 2018):

$$\mathrm{Var}(Z_t) = \gamma_X(0) - 2\phi \gamma_X(1) + \phi^2 \gamma_X(0),$$

$$R_Z(k) = \gamma_X(k) - \phi [\gamma_X(k+1) + \gamma_X(k-1)] + \phi^2 \gamma_X(k).$$

Estimation via quadratic Yule–Walker equations may fail in degenerate cases when covariance recursions reduce to pure two-term forms; such processes exhibit rank-deficient (periodic or dense) covariance structures, "degenerate" in the sense of limited principal components (Voutilainen et al., 2018).

Long-memory noise, as in fractional Gaussian or ARFIMA models, induces non-standard estimation behavior: asymptotic normality holds only for $H<3/4$, variance grows with memory strength ($H \to 1$), and Berry–Esseen bounds for CLT decay polynomially with $n$, via fourth-moment theorems (Chen et al., 2020).

5. Statistical Tests for Serial Correlation

Detection of correlated noise requires tailored test statistics. For AR(1) models with AR(1) innovations, the Durbin–Watson statistic admits a sharp asymptotic analysis:

$$D_n = \frac{\sum_{k=1}^n (\hat\varepsilon_k - \hat\varepsilon_{k-1})^2}{\sum_{k=0}^n \hat\varepsilon_k^2} \to 2(1 - \rho^*),$$

with asymptotic normality and explicit variance (Bercu et al., 2011). A bilateral test achieves correct level and power even with lagged regressors. For higher-order tests, portmanteau-type statistics correct for lagged structure and residual estimation, outperforming Box–Pierce, Ljung–Box, and Breusch–Godfrey in finite samples (Proïa, 2013):

$$T_m = n \hat{\rho}_n' [I_m - P_n S_n^{-1} P_n'/(n \hat{\sigma}_n^2)]^{-1} \hat{\rho}_n \to \chi^2_m.$$

6. AR(1) Noise Statistics in Extended and Nonlinear Models

Extending AR(1) noise frameworks, nonlinear and multiplicative update rules can alter tail and spectral behavior. For example, in GARCH(1,1) and NGARCH models, embedding into continuous time yields stochastic differential equations whose noise statistics determine power-law behaviors and genuine $1/f$ noise only in the nonlinear case (e.g., $\mu=3$, $\eta=3/2$) (Kononovicius et al., 2014):

$$S(f) \sim 1 / f^\beta, \quad \beta = 1 + (\mu-3)/(\mu-2),$$

achieving $\beta=1$ exactly for $\mu=3$. Linear GARCH fails to produce $1/f$ spectra due to scaling breakdown ($\eta=1$).

7. Practical Applications and Representative Results

First-order autoregressive models describe short-range dependencies in physically, biologically, and psychologically motivated phenomena. AR(1) fits explain protein backbone mobility, red blood cell fluctuations, galaxy X-ray light curves, and human-generated random series (0808.1021). In many empirical cases, AR(1) statistics closely mimic—but are distinct from—power-law spectra over limited frequency ranges. Correct identification and estimation require spectral averaging and residual analysis to avoid confounding short and long-range dependencies.

Typical fitted $\phi$ values range from $\sim0.9-0.99$ (strong coupling, e.g., proteins, galaxies), $\sim0.5-0.7$ (moderate coupling, e.g., RBCs), to low/negative values for weak dependence (e.g., cognitive data). 30–40% of tested series suffice with AR(1); higher-order models capture cases with more complex correlation structure (0808.1021).

---

References:

• (0808.1021): Autoregressive description of biological phenomena (Morariu et al.)
• (0709.2963): Order 1 autoregressive process of finite length (Vamos et al.)
• (Wafi, 2023): System Identification on Families of Auto-Regressive with Least-Square-Batch Algorithm
• (Proïa, 2013, Bercu et al., 2011): Testing for residual correlation/autocorrelation statistics (Proïa, Bercu & Proïa)
• (Voutilainen et al., 2018): AR(1)-characterisation with coloured noise (Voutilainen–Viitasaari–Ilmonen)
• (Chen et al., 2020): Second Moment Estimator for AR(1) Driven by Long Memory Gaussian Noise (Chen–Tian–Li)
• (Gruber et al., 2022): Time-varying AR(1) with irregular innovations
• (Kononovicius et al., 2014): Nonlinear GARCH model and $1/f$ noise