Autocorrelation, also known as serial correlation in the discrete time case, is a mathematical concept that measures the correlation of a signal or a random process with a delayed copy of itself. It quantifies the similarity between observations of a variable at different points in its domain, most commonly time. Autocorrelation is a fundamental tool in statistics, signal processing, and time series analysis for identifying repeating patterns, hidden periodicities, or temporal dependencies within data.
Definition in Statistics
In statistics, the autocorrelation of a real or complex random process is defined as the Pearson correlation between values of the process at different times, as a function of the two times or of the time lag. For a random process ${X_t}$, the autocorrelation function between times $t_1$ and $t_2$ is:
$$ R_{XX}(t_1, t_2) = \operatorname{E}\left[X_{t_1} \overline{X}_{t_2}\right] $$
where $\operatorname{E}$ is the expected value operator and the bar represents complex conjugation. Subtracting the mean yields the auto-covariance function:
$$ K_{XX}(t_1, t_2) = \operatorname{E}\left[(X_{t_1} - \mu_{t_1})\overline{(X_{t_2} - \mu_{t_2})}\right] $$
For a wide-sense stationary (WSS) process, the mean and variance are time-independent, and the autocorrelation depends only on the lag $\tau = t_2 - t_1$:
$$ R_{XX}(\tau) = \operatorname{E}\left[X_{t+\tau} \overline{X}_t\right] $$
The normalized autocorrelation coefficient (the correlation coefficient) lies in the range $[-1, 1]$, where 1 indicates perfect positive correlation, -1 indicates perfect negative correlation (anti-correlation), and 0 indicates no linear correlation.
Key Properties
- Symmetry: For real-valued signals, the autocorrelation is an even function: $R_{XX}(\tau) = R_{XX}(-\tau)$. For complex signals, it is Hermitian: $R_{XX}(-\tau) = \overline{R_{XX}(\tau)}$.
- Maximum at zero: The autocorrelation function attains its maximum value at lag zero: $|R_{XX}(\tau)| \leq R_{XX}(0)$.
- Wiener–Khinchin Theorem: The autocorrelation function and the power spectral density form a Fourier transform pair, linking time-domain and frequency-domain representations.
Autocorrelation of Deterministic Signals
In signal processing, the autocorrelation of a continuous-time signal $f(t)$ is defined as:
$$ R_{ff}(\tau) = \int_{-\infty}^{\infty} f(t+\tau) \overline{f(t)} , dt $$
For discrete-time signals $y(n)$:
$$ R_{yy}(\ell) = \sum_{n \in \mathbb{Z}} y(n) , \overline{y(n-\ell)} $$
Estimation
For a discrete process with $n$ observations ${X_1, X_2, \ldots, X_n}$, the autocorrelation coefficient at lag $k$ can be estimated as:
$$ \hat{R}(k) = \frac{1}{(n-k)\sigma^2} \sum_{t=1}^{n-k} (X_t - \mu)(X_{t+k} - \mu) $$
When the true mean and variance are unknown, the sample mean and sample variance are typically used, which introduces bias. The Hassani −1/2 theorem states that for any finite time series of length $T \geq 2$, the sum of the sample autocorrelations over all positive lags equals exactly $-1/2$.
Applications
Autocorrelation has broad applications across many fields:
- Fluorescence correlation spectroscopy: Analyzing molecular diffusion and chemical reactions.
- Optics: Measuring light pulse durations via optical autocorrelators; determining the degree of coherence of an electromagnetic field.
- Dynamic light scattering: Determining particle size distributions of nanoparticles suspended in fluid.
- GPS systems: Correcting for propagation delay by matching satellite signal codes.
- Astronomy: Determining the frequency of pulsars.
- Music and audio: Pitch detection algorithms (e.g., instrument tuners, Auto-Tune) and beat/tempo determination.
- X-ray crystallography: The Patterson function (spatial autocorrelation) helps recover phase information for atom positions.
- Econometrics and regression analysis: Detecting autocorrelation in regression residuals using the Durbin–Watson statistic or Breusch–Godfrey test; applying corrections via generalized least squares or Newey–West HAC estimators.
- Medical ultrasound: Visualizing blood flow.
- Markov chain Monte Carlo: Correcting for autocorrelation in error determination.
Regression Analysis and Autocorrelation
In ordinary least squares (OLS) regression, autocorrelation of the error terms violates the assumption that errors are uncorrelated. While OLS coefficient estimates remain unbiased, standard errors tend to be underestimated (and t-scores overestimated) when positive autocorrelation is present, meaning the Gauss–Markov theorem no longer applies and OLS estimators are not the Best Linear Unbiased Estimators (BLUE). Common remedies include autoregressive (AR), moving average (MA), ARMA, and ARIMA models, as well as vector autoregression (VAR) for multiple interrelated series.
