# MATH310S26 - Introduction to Mathematical Modeling (Lecture Notes) ## Spring 2026 - Day 5 Notes ### Date: January 23, 2026 (01/23/26) --- ## Administrative Updates ### Low-Stakes Feedback (LSF) - **Returned today** with grades of "Complete" or "Incomplete" - Complete: Has stamp in upper right and smiley face (hopefully) - Incomplete: Feedback provided for corrections - **Action required**: Submit PDF to Canvas - Complete: Submit as-is for full credit - Incomplete: Make corrections based on feedback, then submit ### Check Your Understanding Problem - Added to lecture PDF on the last page - 2x2 eigenvalue problem for review of eigen concepts - Will be discussed on Monday ### Upcoming on Monday - Another low-stakes feedback covering last lecture and today's content - More coding exercises with the data set from last time - Extended discussion on modeling approaches based on student bios --- ## Main Topic: Mean Centering, Eigenanalysis, and Dimensional Reduction ### Part 0: Reminder of our goal (Make flat like pancake into something ) ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_01.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_02.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_03.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_04.png]] ### Part 1: Mean Centering Review #### Definition Given 2D data, we create mean-centered data: - $\tilde{x} = x - \bar{x}$ where $\bar{x}$ is the mean of x-values - $\tilde{y} = y - \bar{y}$ where $\bar{y}$ is the mean of y-values ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_05.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_06.png]] #### Physical Interpretation - Mean centering shifts the coordinate system so the center of mass is at the origin - Analogy: Balancing point of a mass distribution - The line of best fit remains the same, just shifted ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_07.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_08.png]] ### Part 2: Covariance Matrix #### Construction For mean-centered data $\tilde{x}$ and $\tilde{y}$, we construct matrix $S$: $S \propto \begin{bmatrix} \tilde{x} \cdot \tilde{x} & \tilde{x} \cdot \tilde{y} \\ \tilde{y} \cdot \tilde{x} & \tilde{y} \cdot \tilde{y} \end{bmatrix}$ Where: - Diagonal elements: Proportional to sample variances - Off-diagonal elements: Proportional to covariances - Note: Full covariance matrix requires dividing by $m$ or $m-1$ (for unbiased estimate) #### Properties of the Covariance Matrix 1. **Symmetric**: $S^T = S$ (due to symmetry of dot product) 2. **Real eigenvalues**: Guaranteed by symmetry 3. **Orthogonal eigenvectors**: Eigenvectors form an orthogonal coordinate system 4. **Non-negative eigenvalues**: Due to the squaring in the matrix construction ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_09.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_10.png]] ### Part 3: Eigenanalysis and Principal Components #### Geometric Interpretation - Eigenvectors define natural coordinate directions in the data - Eigenvalues represent variance along each eigendirection - Normalized eigenvectors scaled by $\sqrt{\lambda}$ define axes of ellipse (2D) or ellipsoid (3D) ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_11.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_12.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_12.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_13.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_14.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_15.png]] #### Toy Example Results For our 4-point dataset: - $\lambda_1 \approx 7.8956$ (eigenvalue for major axis) - $\lambda_2 \approx 0.0211$ (eigenvalue for minor axis) - $\sigma_1 = \sqrt{\lambda_1} \approx 2.81$ (standard deviation along major axis) - $\sigma_2 = \sqrt{\lambda_2} \approx 0.145$ (standard deviation along minor axis) ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_16.png]] #### Variance Explained - Direction 1: $\frac{\sigma_1^2}{\sigma_1^2 + \sigma_2^2} \approx 99.73\%$ - Direction 2: $\frac{\sigma_2^2}{\sigma_1^2 + \sigma_2^2} \approx 0.27\%$ ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_17.png]] ![[S26_01.Linear Regression and Dimensional Reduction_1-23_Page_18.png]] **Key insight**: When one eigenvalue dominates, the data is nearly one-dimensional! ### Part 4: Application to 3D Data #### The Data01 Dataset - 7544 rows with three columns (measured in inches) - Shows both central tendencies and clustering patterns - Three distinct clusters identified in previous analysis #### Principal Component Analysis Results For the 3D data: 1. **First principal component**: ~58% of variance 2. **Second principal component**: ~41% of variance 3. **Third principal component**: <1% of variance **Conclusion**: The third dimension can be eliminated with minimal information loss (less than 1% of variance). #### Ellipsoid Visualization - In 3D, eigenvectors define axes of ellipsoid - Multiple standard deviations create nested ellipsoids - Data capture within ellipsoids: - 1σ: ~20% of data points - 2σ: Higher percentage (following 3D normal distribution) - 3σ: Most data points captured ### Part 5: Mathematica Demonstrations #### Circle vs. Ellipse Example - Uniform sampling on circle perimeter → poorly defined eigen-directions - Breaking symmetry (e.g., scaling y by 0.5) → well-defined eigen-directions - **Lesson**: Symmetry breaking helps eigenanalysis identify meaningful directions --- ## Weekend Challenge ### Mystery Data Contest - **The Data**: Columns 1 and 2 represent real-world measurements in inches - **The Challenge**: Identify what the data represents - **Hint**: "I guarantee you have data available to you that is consistent with columns 1 and 2" - **Tools Provided**: Tape measures available for borrowing - **Prizes**: - Various forms of candy - Math book collection (for those interested) - Stickers (not AI-generated) - Possible slap bracelets ### Instructions 1. Use your linear regression model from previous work 2. Take measurements in the real world 3. Compare against your model predictions 4. Self-discovery encouraged (don't ask someone who knows!) --- ## Looking Ahead ### Monday's Activities - Low-stakes feedback assessment - Code implementation for dimensional reduction - Begin serious modeling discussions based on student interests - Review of Check Your Understanding problem from today ### Key Concepts to Review - Eigenvalue/eigenvector calculations (2x2 matrices) - Geometric interpretation of covariance matrices - Relationship between variance and dimensional reduction --- ## Important Takeaways 1. **Covariance matrices encode data geometry**: Eigenanalysis reveals natural coordinate systems 2. **Dimensional reduction is quantitative**: We can measure exactly how much information each dimension carries 3. **"Flat like a pancake"**: When eigenvalues differ greatly, data lies approximately in lower-dimensional space 4. **Physical intuition helps**: Center of mass, moment of inertia analogies make abstract concepts concrete --- *Note: Remember to check the last page of the lecture PDF for the eigenvalue practice problem!*