# MATH310 Spring 2026 - Day 25 **Date**: March 16, 2026 **Topic**: Random Walks, Covariance and Correlation **Lecture**: 25 ## Administrative Notes - Workday on Friday (before [spring break](https://en.wikipedia.org/wiki/Spring_break)) - No Low-Stakes Feedback (LSF) this week - Version 1.0 snapshot due Friday - Not meant to be onerous, just a check-in - Prevents "heroic GPT-based efforts" - Shows progress: math derivations, draft codes, working components - Spring break plans discussed (Golden Gate Canyon cabin for instructor) ## Recall: Random Walk Framework From [[MATH310S26-Day23-Notes|Day 23]] and [[MATH310S26-Day24-Notes|Day 24]]: The stochastic process: $S_{n+1} = S_0 + \sum_{i=0}^{n} X_i$ Where $X_i$ are [i.i.d. random variables](https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables) with: - Events: $L_1, L_2$ (step left/right) - Probabilities: $p_1, p_2$ Key results for the random walk process: - $E[S_n] = S_0 + nE[X]$ - $V[S_n] = nV[X]$ - For symmetric walk: $E[X] = 0$ **Critical insight**: "This all falls onto the steps" - the statistics of walkers depend entirely on step statistics. ## Covariance Properties When two [random variables](https://en.wikipedia.org/wiki/Random_variable) $X$ and $Y$ are not [independent](https://en.wikipedia.org/wiki/Independence_(probability_theory)), we examine their [covariance](https://en.wikipedia.org/wiki/Covariance): $\text{Cov}(X,Y) = E[(X - \mu_X)(Y - \mu_Y)] = E[XY] - E[X]E[Y]$ ### Five Essential Properties **Property 1**: Covariance with itself is variance $\text{Cov}(X,X) = E[(X - \mu_X)^2] = V[X]$ **Property 2**: Independent variables have zero covariance If $X$ and $Y$ are independent: $E[XY] = E[X]E[Y]$ This uses the factorization of [joint probability density function](https://en.wikipedia.org/wiki/Joint_probability_distribution): $\rho_{X,Y}(x,y) = \rho_X(x) \cdot \rho_Y(y)$ Leading to (by [Fubini's theorem](https://en.wikipedia.org/wiki/Fubini%27s_theorem)): $\int\int xy \rho_{X,Y}(x,y)\,dx\,dy = \left(\int x\rho_X(x)\,dx\right)\left(\int y\rho_Y(y)\,dy\right)$ Therefore: $\text{Cov}(X,Y) = 0$ when independent. **Property 3**: [Symmetry](https://en.wikipedia.org/wiki/Symmetric_relation) $\text{Cov}(X,Y) = \text{Cov}(Y,X)$ **Property 4**: [Linearity](https://en.wikipedia.org/wiki/Linearity) in first argument $\text{Cov}(c_1X_1 + c_2X_2, Y) = c_1\text{Cov}(X_1,Y) + c_2\text{Cov}(X_2,Y)$ (By symmetry, also linear in second argument - this is [bilinearity](https://en.wikipedia.org/wiki/Bilinear_form)) **Property 5**: Variance of sums For independent $X$ and $Y$: $V[X + Y] = \text{Cov}(X+Y, X+Y) = V[X] + V[Y]$ Since $\text{Cov}(X,Y) = 0$ when independent. ## Visualizations of Random Walks ### One-Dimensional Random Walk - 50 out of 1000 walkers shown (to avoid visual clutter) ![[RW_01.png]] - Horizontal axis: time (0 to 1000 steps) - Vertical axis: position - Each walker follows independent coin flips - Spread grows as $\sqrt{n}$ (sublinear) ![[Growth_super_linear_sub.png]] ### Two-Dimensional Random Walk - "Jiggly puffs" moving in 2D space ![[2D.png]] - Can decompose into $x$ and $y$ components - Random step: choose random angle and radius - Still exhibits $\sqrt{n}$ spreading behavior ### Growth Rates Comparison - **[Ballistic](https://en.wikipedia.org/wiki/Ballistic_conduction)**: $y = n$ (linear) - **Random walk**: $y = \sqrt{n}$ ([sublinear](https://en.wikipedia.org/wiki/Time_complexity#Sub-linear_time)) - **Quadratic**: $y = n^2$ ([superlinear](https://en.wikipedia.org/wiki/Time_complexity#Polynomial_time)) Random walks exhibit characteristically slow, [diffusive](https://en.wikipedia.org/wiki/Diffusion) spreading. ## Covariance Between Random Walks at Different Times ### Setup Consider two random variables from the same walk: - $S_n$: position at time $n$ - $S_{n+k}$: position at time $n+k$ (future) ### Calculation Starting with: $\text{Cov}(S_{n+k}, S_n) = \text{Cov}\left(S_0 + \sum_{i=0}^{n+k-1} X_i, S_n\right)$ Using linearity (Property 4): $= \text{Cov}(S_0, S_n) + \sum_{i=0}^{n+k-1} \text{Cov}(X_i, S_n)$ Expanding $S_n$: $= \text{Cov}\left(S_0, S_0 + \sum_{j=0}^{n-1} X_j\right) + \sum_{i=0}^{n+k-1} \text{Cov}\left(X_i, S_0 + \sum_{j=0}^{n-1} X_j\right)$ ### Key Insight When computing $\text{Cov}(X_i, X_j)$: - If $i = j$: $\text{Cov}(X_i, X_i) = V[X]$ - If $i \neq j$: $\text{Cov}(X_i, X_j) = 0$ (independence) ### Double Sum Structure The calculation leads to: $\text{Cov}(S_{n+k}, S_n) = \sum_{i=0}^{n+k-1} \sum_{j=0}^{n-1} \text{Cov}(X_i, X_j)$ Only diagonal terms ($i = j$) survive, giving: $\text{Cov}(S_{n+k}, S_n) = nV[X]$ ## Important Results For a random walk with i.i.d. steps: 1. **[Autocorrelation](https://en.wikipedia.org/wiki/Autocorrelation)**: $\text{Cov}(S_n, S_n) = nV[X]$ 2. **Cross-time covariance**: $\text{Cov}(S_{n+k}, S_n) = nV[X]$ 3. **Key property**: Covariance depends on the earlier time only ## Check Your Understanding 1. Why does covariance between $S_{n+k}$ and $S_n$ only depend on $n$, not $k$? 2. What would happen if steps were not independent? 3. How does this connect to the [Markov property](https://en.wikipedia.org/wiki/Markov_property)? 4. How does variance (from [[MATH310S26-Day24-Notes|Day 24]]) relate to autocorrelation? ## Mathematical Connections - Builds on [covariance](https://en.wikipedia.org/wiki/Covariance) theory from [statistics](https://en.wikipedia.org/wiki/Statistics) - Continues from [[MATH310S26-Day23-Notes|Day 23]] introduction and [[MATH310S26-Day24-Notes|Day 24]] variance analysis - Connects to [[MATH310S26-Day22-WorkdayMaterials (Cross-correlation and signal detection)|Day 22 cross-correlation]] - Links back to [[MATH310S26-Day5-Notes|Day 5-6]] covariance matrices in PCA - Foundation for [Brownian motion](https://en.wikipedia.org/wiki/Brownian_motion) theory - Links to [stochastic processes](https://en.wikipedia.org/wiki/Stochastic_process) and [time series analysis](https://en.wikipedia.org/wiki/Time_series) ## Next Class Preview - Workday on Friday (similar to [[MATH310S26-Day22-WorkdayMaterials (Cross-correlation and signal detection)|Day 22 workday]]) - After break: likely connection to [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem) - Possible topics: [continuous-time limits](https://en.wikipedia.org/wiki/Continuous-time_stochastic_process), [Brownian motion](https://en.wikipedia.org/wiki/Brownian_motion)