Wangsheng's World
Notes
/
Chapter 4
Spectral Analysis
October 29, 2024
1. Spectral Densities
  • Suppose that \(\{X_t\}\) is a zero-mean stationary time series with ACVF \(\gamma(\cdot)\).
  • To define spectral density, we first assume that \(\gamma(\cdot)\) satisfies \(\sum_{h = - \infty}^{\infty} \mid \gamma(h) \mid < \infty\).
    Summability of \(\gamma(\cdot)\); holds true for ARMA
  • In this case, the spectral density of \(\{X_t\}\) is defined as \[ f(\lambda) = \frac{1}{2 \pi} \sum_{h = - \infty}^{\infty} e^{-i h \lambda} \gamma(h) \] for \(-\infty < \lambda < \infty\).
  • It follows that \(f(\lambda) = \frac{1}{2 \pi} \sum_{h=-\infty}^\infty \cos(h\lambda) \gamma(h)\).
Examples
  • White noise, \(\{Z_t\} \sim WN(0, \sigma^2)\)
    \(\gamma(h) = \begin{cases} \sigma^2 & h=0 \\ 0 & \text{otherwise}\end{cases}\)
    So, \(f(\lambda) = \frac{1}{2 \pi} \sum_{-\infty}^\infty e^{-ih\lambda} \gamma(h) = \frac{1}{2 \pi} e^0 \cdot \gamma(0) = \frac{\sigma^2}{2\pi}\)
  • AR(1): \(X_t - \phi X_{t-1} = Z_t, \mid \phi \mid < 1\) & \(\{Z_t\} \sim WN(0, \sigma^2)\)
    \(\gamma(h) = \frac{\sigma^2 \phi^{\mid h \mid}}{1 - \phi^2}\)
    So, \( f(\lambda) = \frac{1}{2\pi} \sum_{-\infty}^\infty e^{-ih\lambda} \gamma(h) = \frac{\sigma^2}{2 \pi (1 - \phi^2)} [1 + \sum_{h=1}^\infty \phi^h e^{-ih\lambda} + \sum_{l=1}^{\infty} \phi^{l} e^{il\lambda}] = \frac{\sigma^2}{2 \pi (1 - \phi^2)} [1 + \frac{\phi e^{-i\lambda}}{1 - \phi e^{-i \lambda}} + \frac{\phi e^{i\lambda}}{1 - \phi e^{i \lambda}}] = \frac{\sigma^2}{2 \pi} \frac{1}{1 - 2 \phi \cos(\lambda) + \phi^2} \) for \(\lambda \in (-\infty, \infty)\).
    numerical example in R
    (see photo - 15:43 Oct 22)
  • MA(1): \(X_t = Z_t + \theta Z_{t-1}, \{Z_t\} \sim WN(0, \sigma^2)\)
Basic Properties of \(f(\cdot)\)
  • \(f(\lambda + 2 \pi) = f(\lambda)\)
    • So, it suffices to study \(f(\cdot)\) on the interval \((-\pi, \pi]\).
  • \(f(\lambda) = f(-\lambda)\)
    • So, it suffices to study \(f(\cdot)\) on the interval \([0, \pi]\).
  • \(f(\lambda) \ge 0\) for all \(\lambda \in (-\pi, \pi]\)
  • \(\gamma(k) = \int_{-\pi}^{\pi} e^{i k \pi} f(\lambda) d \lambda = \int_{-\pi}^{\pi} \cos(k\lambda) f(\lambda) d \lambda \)
    • \(\gamma\) determines \(f\) and vice versa.
Definition of Spectral Density
A function \(f\) is the spectral density of a stationary time series \(\{X_t\}\) with ACVF \(\gamma(\cdot)\) if
  • \(f(\lambda) \ge 0\) for all \(\lambda \in (-\pi, \pi]\), and
  • \(\gamma(h) = \int_{-\pi}^{\pi} e^{i h \lambda} f(\lambda) d \lambda\) for all integers \(h\).
Proposition 4.1.1
A real-valued function \(f\) on \((- \pi, \pi]\) is the spectral density of a real-valued stationary time series if and only if
  • \(f(\lambda) = f(-\lambda)\)
  • \(f(\lambda) \ge 0\)
  • \(\int_{-\pi}^{\pi} f(\lambda) d\lambda < \infty \)
Corollary 4.1.1
An absolutely summable function \(\gamma(\cdot)\) is the ACVF of a stationary time series if and only if
  • \(\gamma(h) = \gamma(-h)\)
  • \(f(\lambda) = \frac{1}{2 \pi} \sum_{h = -\infty}^\infty e^{-ih\lambda} \gamma(h) \ge 0\) for all \(\lambda \in (-\pi, \pi]\).
Example
Show that the function defined by
\( \kappa(h) = \begin{cases} 1 & h = 0 \\ \rho & h = \pm 1\\ 0 & \text{otherwise} \end{cases} \)
is the ACVF of a stationary time series if and only if \(\mid \rho \mid \leq 1/2\).
Spectral Representation of the ACVF
  • A function \(\gamma(\cdot)\) is the ACVF of a stationary time series if and only if there exists a right-continuous, nondecreasing, bounded function \(F\) on \([-\pi, \pi]\) with \(F(-\pi) = 0\) such that \[ \gamma(h) = \int_{-\pi}^\pi e^{ih\lambda}dF(\lambda) \] for all integers \(h\).
  • \(F\) is called the spectral distribution function of \(\gamma(\cdot)\).
Remarks
  • Every stationary process has a spectral distribution function \(F\), and \(\gamma(h) = \int_{-\pi}^\pi e^{ih\lambda} dF(\lambda)\).
  • But not every stationary process has a spectral density \(f\).
  • If the spectral density \(f\) exisits, then
    • \(F(\lambda) = \int_{-\pi}^\pi f(\omega) d\omega\)
    • \(\gamma(h) = \int_{-\pi}^\pi e^{ih\lambda}f(\lambda)d\lambda\)
    • Time series is said to have a continuous spectrum.
Example
\(X_t = A \cos(\omega t) + B \sin(\omega t)\) where \(\omega \in (0, \pi)\) and \(A\) & \(B\) are uncorrelated random variables with mean 0 & variance \(\sigma^2\).
Preliminaires:
  1. Riemann-Stieltjes integral
    \( \int_I g(x) d F(x) = \lim_{\max_i(x_i - x_{i-1})\rightarrow 0} \sum_{i=1}^N g(\xi_i)[F(x_i) - F(x_{i-1})] \)
    A result: if \(F(x) = \sum_{i=1}^N a_i \mathbb{1}_{\{x \ge x_i\}}\), then \(\int_I g(x) dF(x) = \sum_{i=1}^N g(x_i)a_i\)
  2. Stochastic integral / Itô calculus
    (see Appendix D.2.4)
2. Periodogram
  • Suppose \(\{x_1, \dotsm, x_n\}\) is a realization of a stationary time series \(\{X_t\}\) with ACVF \(\gamma(\cdot)\) and spectral density \(f(\cdot)\).
  • The periodogram of \(\{x_1, \dotsm, x_n\}\) is defined as \[ I_n(\lambda) = \frac{1}{n}\mid \sum_{t=1}^n x_t e^{-it\lambda}\mid^2. \]
  • To derive the properties of periodogram, we start with \(\mathbb{C}^n\), vector space of all \(n\)-tuples of complex numbers.
Fourier Frequencies
Foourier frequencies associated with sample size \(n\) are defined as \[ \omega_k = \frac{2 \pi k}{n}, k = -[\frac{n-1}{2}], \dotsm, [\frac{n}{2}], \] where \([y]\) denotes the largest integer less than or equal to y.
A Basis for \(\mathbb{C}^n\)
  • Moreover, we define \[ \boldsymbol{e}_k = \frac{1}{\sqrt{n}}\begin{bmatrix}e^{i \omega_k} \\ \vdots \\ e^{ni \omega_k}\end{bmatrix}, \] \[ k = -[\frac{n-1}{2}], \dotsm, [\frac{n}{2}] \]
  • Then, \(\{e_k\}\) is a basis for \(\mathbb{C}^n\) because \[ \boldsymbol{e}_j^* \boldsymbol{e}_k = \begin{cases}1 & \text{if } j = k \\ 0 & \text{if }j \ne k \end{cases} \] where \(\boldsymbol{e}_j^* = \frac{1}{\sqrt{n}}(e^{-i\omega_j}, \dotsm, e^{-ni \omega_j})\).
Discrete Fourier Transform
  • So, \(\boldsymbol{x} = (x_1, \dotsm, x_n)'\) can be expressed as \[ \boldsymbol{x} = \sum_{k = -[(n-1)/2]}^{[n/2]}a_k \boldsymbol{e}_k \] where \(a_k = \boldsymbol{e}_k^* \boldsymbol{x} = \frac{1}{\sqrt{n}} \sum_{t=1}^n x_t e^{-it\omega_k}\).
  • \(\{a_k\}\) is called the discrete Fourier transform of the sequence \(\{x_1, \dotsm, x_n\}\).
Remarks
  • For \(t = 1, \dotsm, n\),
    \(x_t = \frac{1}{\sqrt{n}} \sum_{k = -[(n-1)/2]}^{[n/2]} a_k [\cos(\omega_k t) + i \sin(\omega_k t)] \)
  • \(I_n(\omega_k) = \mid a_k \mid^2\)
  • \(\sum_{t=1}^n \mid x_t \mid^2 = \sum_{k = -[(n-1)/2]}^{n/2} I_n(\omega_k)\)
Proposition 4.2.1
  • If \(x_1, \dotsm, x_n\) are any real numbers and \(\omega_k\) is any of the nonzero Fourier frequencies \(2 \pi k /n\) in \((-\pi, \pi]\), then \[ I_n(\omega_k) = \sum_{\mid h \mid < n} \hat{\gamma}(h) e^{-ih \omega_k} \] where \(\hat{\gamma}(h)\) is the same ACVF of \(x_1, \dotsm, x_n\).
  • Recall that \(2 \pi f(\lambda) = \sum_{h = -\infty}^\infty \gamma(h) e^{-ih\lambda}\) for absolutely summable ACVF.
  • So, we may view the periodogram \(I_n(\lambda)\) as a sample analogue of \(2 \pi f(\lambda)\).
Estimation of Spectral Density
  • Note that, \(I_n(\lambda)/ 2\pi\) is not a consistent estimator of \(f(\lambda)\).
  • For a large class of stationary processes including the ARMA processes, consistent estimation of \(f(\lambda)\) can be achieved by the discrete spectral average estimator as defined in Definition 4.2.2.
    • See Remark 3 underneath Definition 4.2.2.
Example
  • The sunspot numbers
3. Time-Invariant Linear Filters
  • \(\{Y_t\}\) is the output of a time-invariant linear filter (TLF) \(\Psi = \{\psi_k, k = 0, \pm1, \dotsm \}\) applied to an input process \(\{X_t\}\) if \[ Y_t = \sum_{k = -\infty}^\infty \psi_k X_{t-k}. \]
    • Time-invariant: \(\psi_k\)'s are independent of \(t\)
    • Linear: \(\psi(B)(aU_t + bV_t) = a \psi(B)U_t + b \psi(B) V_t\)
  • A linear process, \(X_t = \sum_{k=-\infty}^\infty \psi_k Z_{t-k}\), is the output of \(\Psi\) applied to a white noise input series.
Proposition 4.3.1
  • Let \(\{X_t\}\) be a stationary time series with mean 0 and spectral density \(f_X(\lambda)\).
  • Suppose that \(\Psi = \{\psi_k, k = 0, \pm 1, \dotsm\}\) is an absolutely summable TLF.
  • Then the time series \[ Y_t = \psi(B)X_t \] is stationary with mean 0 and spectral density \[ f_Y(\lambda) = \mid \psi(e^{-i\lambda})\mid^2 f_X(\lambda). \]
Transfer Function & Power Transfer Function
  • \(\psi(e^{-i\lambda}), \lambda \in [-\pi, \pi]\), is called transfer function of \(\Psi\).
  • \(\mid \psi(e^{-i\lambda}) \mid^2, \lambda \in [-\pi, \pi]\), is called power transfer function of \(\Psi\).
    • The power transfer function relates to the spectral densities of the output and input.
    • If \(\mid \psi(e^{-i\lambda}) \mid^2 = 0\), then the filter eliminates the component of the input with frequency \(\lambda_0\).
Example
  • Differencing filter, \(\nabla_{12} = 1 - B^{12}\)
Filter in Series
If two absolutely summable TLFs \(\{\psi_j\}\) and \(\{\xi_j\}\) are applied to \(\{X_t\}\) with spectral density \(f_X(\lambda)\), then \[ f_W(\lambda) = \mid \xi(e^{-i\lambda})\mid^2 \mid \psi(e^{-i\lambda})\mid^2 f_X(\lambda) \] where \(f_W(\lambda)\) is the spectral density of \[ W_t = \xi(B)\psi(B)X_t. \]
4. Spectral Density of an ARMA Process
  • Let \(\{X_t\}\) be a causal-invertible ARMA(\(p, q\)) process satisfying \(\phi(B) X_t = \theta(B)Z_t, \{Z_t\} \sim WN(0, \sigma^2)\).
  • Then, \[ f_X(\lambda) = \frac{\sigma^2}{2\pi} \frac{\mid \theta(e^{-i\lambda})\mid^2}{\mid \phi(e^{-i\lambda})\mid^2}, \lambda \in [-\pi, \pi]. \]
Examples
  • AR(1): \(X_t - \phi X_{t-1} = Z_t\)
  • AR(2): \(X_t - \phi_1 X_{t-1} - \phi_2 X_{t-2} = Z_t\)
  • ARMA(1, 1): \(X_t - \phi X_{t-1} = Z_t + \theta Z_{t-1}\)
5. Linear Combination of Sinusoids
  • Sinusoid is a curve like the sine function but possibly shifted in phase, period, amplitude, or any combination thereof.
  • The general sinusoid of amplitude \(a\), angular frequency \(\omega\) (and period \(2 \pi /\omega\)), and phase \(c\) is given by \(f(x) = a \sin(\omega x + c)\).
  • Consider harmonic process \[ X_t = \sum_{j=1}^K [A_j \cos(\omega_j t) + B_j \sin(\omega_j t)] \] where \(0 < \omega_1 < \dotsm < \omega_k < \pi\) and \(A_1, B_1, \dotsm, A_k, B_k\) are uncorrelated random variables with \(\mathbb{E}(A_j) = \mathbb{E}(B_j) = 0\) & \(Var(A_j) = Var(B_j) = \sigma_j^2, j = 1, \dotsm, k\).
Spectral Representation of \(\{X_t\}\)
  • \(\{X_t\}\) can be expressed as
    \( X_t = \sum_{j=1}^k [A_j \frac{e^{i \omega_j t} + e^{-i \omega_j t}}{2} - i B_j \frac{e^{i \omega_j t} - e^{-i \omega_j t}}{2}] = \int_{-\pi}^\pi e^{i \lambda t} dZ(\lambda) \)
    where \( dZ(\lambda) = \begin{cases} (A_j + i B_j) / 2 & \text{if } \lambda = - \omega_j, \\ (A_j - i B_j) / 2 & \text{if } \lambda = \omega_j, \\ 0 & \text{otherwise}. \end{cases} \)
Spectral Representation of \(\gamma_X(h)\)
  • Moreover,
    \( \gamma_X(h) = \sum_{j=1}^k \sigma_j^2 \cos(\omega_j h) = \sum_{j=1}^k \sigma_j^2 \frac{e^{i \omega_j h} + e^{-i \omega_j h}}{2} = \int_{-\pi}^\pi e^{i\lambda h} dF(\lambda) \)
    where \( dF(\lambda) = \begin{cases} \sigma_j^2 / 2 & \text{if } \lambda = - \omega_j, \\ \sigma_j^2 / 2 & \text{if } \lambda = \omega_j, \\ 0 & \text{otherwise}. \end{cases} \)