Draft ## Introduction Climate scientists track global sea levels to understand long-term trends. Financial analysts monitor stock prices to predict market movements. Medical researchers study patient data to identify treatment patterns. All of these professionals face the same fundamental challenge: how do we separate meaningful signals from random noise in time-varying data? Fourier analysis provides a powerful mathematical lens for this problem. By decomposing complex signals into simple sine and cosine waves, we can identify dominant frequencies, remove unwanted noise, and build predictive models. But here's the key insight: Fourier analysis isn't just about frequencies—it's fundamentally a least squares problem in disguise. ## Project Description Starting with global sea level data spanning three decades, you'll discover how linear algebra transforms our understanding of temporal patterns. This project bridges time series analysis with the regression techniques you've already mastered. **1 Point - Foundation**: Implement Fourier regression using ordinary least squares to model sea level trends. Verify your decomposition matches built-in FFT results and interpret the physical meaning of discovered frequency components. **2 Points - Exploration**: Extend your analysis to compare linear, quadratic, and Fourier models across different time periods. Investigate how model performance varies with data length and examine the trade-offs between trend fitting and periodic pattern detection. **Key Deliverable**: A comprehensive analysis demonstrating how least squares principles underlie both regression and frequency domain analysis, with practical insights about climate data interpretation. ## Mathematical Background: Fourier Regression as Linear Algebra The key insight is that Fourier analysis can be formulated as a standard least squares problem. Instead of fitting $y = \beta_0 + \beta_1 x$, we fit a model of trigonometric functions. ### The Fourier Regression Model Given time series data $y(t_1), y(t_2), \ldots, y(t_n)$, we seek to approximate the signal as: $y(t) = a_0 + \sum_{k=1}^{m} a_k \cos(2\pi f_k t) + b_k \sin(2\pi f_k t)$ where $f_k$ represents the $k$-th frequency component. This is a linear combination of known functions, making it a perfect candidate for least squares regression. ### Design Matrix Construction The design matrix for Fourier regression becomes: $X = \begin{bmatrix} 1 & \cos(2\pi f_1 t_1) & \sin(2\pi f_1 t_1) & \cos(2\pi f_2 t_1) & \sin(2\pi f_2 t_1) & \cdots \ 1 & \cos(2\pi f_1 t_2) & \sin(2\pi f_1 t_2) & \cos(2\pi f_2 t_2) & \sin(2\pi f_2 t_2) & \cdots \ \vdots & \vdots & \vdots & \vdots & \vdots & \ddots \ 1 & \cos(2\pi f_1 t_n) & \sin(2\pi f_1 t_n) & \cos(2\pi f_2 t_n) & \sin(2\pi f_2 t_n) & \cdots \end{bmatrix}$ The parameter vector contains our Fourier coefficients: $\boldsymbol{\beta} = \begin{bmatrix} a_0 \ a_1 \ b_1 \ a_2 \ b_2 \ \vdots \end{bmatrix}$ ### Worked Example: Sea Level Analysis Using the global mean sea level data from 1993-2024, we construct a simple two-frequency model. The dominant patterns emerge from the normal equations $X^T X \hat{\boldsymbol{\beta}} = X^T \mathbf{y}$. From the lecture analysis, we found: - $a_0 = 14.55$ mm (mean sea level offset) - $a_1 = 1.03$, $b_1 = -4.84$ (annual cycle coefficients) - $a_2 = -1.59$, $b_2 = 0.39$ (semi-annual cycle coefficients) The resulting model captures the seasonal oscillations while revealing the underlying trend through the residuals. ### Connection to FFT The Fast Fourier Transform provides the same frequency decomposition but uses a complete orthogonal basis. Our least squares approach allows selective frequency modeling and direct statistical interpretation of coefficients. ## Implementation Guidelines ### Part 1: Exact Replication and Verification #### Step 1: Data Preparation and Basic Fourier Regression **MATLAB Implementation:** ```matlab % Load and prepare sea level data data = readtable('global_mean_sea_level_19932024.csv'); time = data.YearPlusFraction; sea_level = data.SmoothedGMSLWithGIA; % Construct design matrix for 2-frequency model n = length(time); f1 = 1; f2 = 2; % Annual and semi-annual frequencies (cycles per year) X = [ones(n,1), cos(2*pi*f1*time), sin(2*pi*f1*time), ... cos(2*pi*f2*time), sin(2*pi*f2*time)]; % Solve normal equations beta_hat = (X'*X) \ (X'*sea_level); y_fitted = X * beta_hat; ``` **Python Implementation:** ```python import numpy as np import pandas as pd # Load and prepare sea level data data = pd.read_csv('global_mean_sea_level_19932024.csv') time = data['YearPlusFraction'].values sea_level = data['SmoothedGMSLWithGIA'].values # Construct design matrix for 2-frequency model n = len(time) f1, f2 = 1, 2 # Annual and semi-annual frequencies X = np.column_stack([ np.ones(n), np.cos(2*np.pi*f1*time), np.sin(2*np.pi*f1*time), np.cos(2*np.pi*f2*time), np.sin(2*np.pi*f2*time) ]) # Solve normal equations beta_hat = np.linalg.solve(X.T @ X, X.T @ sea_level) y_fitted = X @ beta_hat ``` #### Step 2: Verification Against FFT Results - Compare your Fourier coefficients with frequency domain analysis - Verify reconstruction accuracy using $R^2$ values - Interpret the physical meaning of each frequency component ### Part 2: Systematic Model Comparison Analysis **Scenario A: Trend Model Comparison** Compare linear, quadratic, and Fourier models on residuals after removing periodic components: **MATLAB:** ```matlab % Remove periodic components to analyze trend residuals = sea_level - y_fitted; % Linear trend model X_linear = [ones(n,1), time]; beta_linear = (X_linear'*X_linear) \ (X_linear'*residuals); ``` **Python:** ```python # Remove periodic components to analyze trend residuals = sea_level - y_fitted # Linear trend model X_linear = np.column_stack([np.ones(n), time]) beta_linear = np.linalg.solve(X_linear.T @ X_linear, X_linear.T @ residuals) ``` _Prediction_: The quadratic model should capture acceleration in sea level rise, yielding higher $R^2$ values than linear trends. **Scenario B: Time Window Sensitivity** Analyze how model performance varies with data length: **MATLAB:** ```matlab % Test different time windows windows = [5, 10, 15, 20, 25]; % years of data r_squared_values = zeros(length(windows), 3); % linear, quad, fourier for i = 1:length(windows) % Extract subset ending at most recent data subset_indices = (n - windows(i)*12 + 1):n; % approximate monthly data % Fit models and compute R^2 values end ``` **Python:** ```python # Test different time windows windows = [5, 10, 15, 20, 25] # years of data r_squared_values = np.zeros((len(windows), 3)) # linear, quad, fourier for i, window in enumerate(windows): # Extract subset ending at most recent data subset_indices = slice(-window*12, None) # approximate monthly data # Fit models and compute R^2 values ``` _Prediction_: Longer time series should favor Fourier models as seasonal patterns become more apparent. **Scenario C: Frequency Resolution Trade-offs** Compare models with different numbers of frequency components: **MATLAB:** ```matlab % Test models with 1, 2, 3, and 5 frequency components max_frequencies = [1, 2, 3, 5]; aic_values = zeros(size(max_frequencies)); % Akaike Information Criterion for i = 1:length(max_frequencies) % Construct design matrix with specified frequencies % Compute AIC = n*log(RSS/n) + 2*p where p is number of parameters end ``` **Python:** ```python # Test models with 1, 2, 3, and 5 frequency components max_frequencies = [1, 2, 3, 5] aic_values = np.zeros(len(max_frequencies)) for i, max_freq in enumerate(max_frequencies): # Construct design matrix with specified frequencies # Compute AIC = n*log(RSS/n) + 2*p where p is number of parameters ``` _Prediction_: Adding frequencies should improve fit initially, but overfitting will increase AIC for too many components. #### Student Tasks For each scenario: 1. **Predict** the expected relationship between model complexity and performance 2. **Implement** the analysis using both matrix methods and built-in functions 3. **Compare** results across different models and time periods 4. **Interpret** the implications for climate data analysis ## Analysis Framework ### Mathematical Insights - How does the **condition number** of the design matrix change as you add more frequency components? A high condition number indicates that small changes in the data could dramatically affect the coefficient estimates, suggesting potential overfitting. - What is the relationship between data length and frequency resolution? Can you identify the minimum time series length needed to reliably detect annual cycles? ### Practical Applications - How do different models perform for prediction versus explanation? - When would you choose Fourier regression over polynomial trends, and vice versa? ### Statistical Considerations - How does the $R^2$ metric behave differently for trend versus periodic components? - What does the residual structure tell us about model adequacy? ## Real-World Context Understanding how to decompose signals into trend and periodic components has practical implications across many fields: ### Climate Science Applications - **[Sea Level Monitoring](https://climate.nasa.gov/evidence/sea-level-rise/)**: Separating seasonal cycles from long-term trends to assess climate change impacts - **[Temperature Analysis](https://www.noaa.gov/education/resource-collections/climate/climate-change-impacts)**: Distinguishing natural variability from anthropogenic warming signals - **[Precipitation Patterns](https://www.usgs.gov/mission-areas/water-resources/science/precipitation-frequency)**: Identifying changing rainfall patterns for water resource management ### Financial Applications - **[Economic Indicators](https://www.federalreserve.gov/econres/feds/seasonality-in-macroeconomic-data.htm)**: Seasonal adjustment of employment, sales, and production data - **[Market Analysis](https://www.investopedia.com/terms/c/cyclical_industry.asp)**: Identifying cyclical patterns in stock prices and commodity markets - **[Risk Assessment](https://www.bis.org/publ/qtrpdf/r_qt1509e.pdf)**: Modeling temporal volatility patterns in financial instruments ### Medical Applications - **[Circadian Rhythms](https://www.nigms.nih.gov/education/fact-sheets/circadian-rhythms)**: Analyzing daily cycles in physiological measurements - **[Epidemic Modeling](https://www.cdc.gov/flu/about/season/flu-season.htm)**: Understanding seasonal disease patterns for public health planning - **[Treatment Response](https://www.nature.com/articles/s41598-019-56961-w)**: Monitoring patient recovery patterns over time ### Technical Challenges The intersection of time series analysis and linear algebra presents several computational considerations: - **Numerical Stability**: Large design matrices with oscillatory functions can become ill-conditioned - **Frequency Selection**: Choosing appropriate frequencies without prior knowledge requires spectral analysis - **Overfitting**: Balancing model complexity with predictive accuracy, especially for short time series ### Business Impact Accurate trend and cycle identification drives decision-making across industries. Climate scientists use these techniques to provide policy-relevant assessments of sea level rise rates. Financial institutions employ similar methods for risk modeling and algorithmic trading. The ability to separate signal from noise in temporal data translates directly to competitive advantages and better-informed decisions. ## Deliverable Checklist - [ ] **Code Implementation** - [ ] Fourier regression using normal equations with proper matrix construction - [ ] Verification against FFT results with coefficient comparison - [ ] Systematic model comparison across multiple scenarios - [ ] Clear documentation of design matrix construction and interpretation - [ ] **Analysis Report** - [ ] Verification that manual implementation matches FFT decomposition - [ ] Comparison of linear, quadratic, and Fourier model performance - [ ] Investigation of time window effects on model accuracy - [ ] Discussion of frequency selection trade-offs and overfitting detection - [ ] Physical interpretation of discovered patterns in sea level data - [ ] **Communication** - [ ] One-page synthesis connecting Fourier analysis to least squares principles - [ ] Five to seven minute video walkthrough of implementation and climate science insights - [ ] Clear explanation of how trigonometric regression reveals both trends and cycles in temporal data