NOTES, SLIDES, AND PAPERS

Date: 01-09-2025

Introduction: Slides

Date: 01-14-2025

"You can always recognize truth by its beauty and simplicity"―Richard P. Feynman 

• Singular Value Decomposition (SVD) 

• Example 1

Papers

1. Lieberman-Aiden, Erez, ..., Groudine Mark, ..., Lander Eric. "Comprehensive mapping of long-range interactions reveals folding principles of the human genome." Science 326.5950 (2009): 289-293. 

2. Turk, Matthew, and Alex Pentland. "Eigenfaces for recognition." Journal of cognitive neuroscience 3.1 (1991): 71-86. (Classic! just browse)

3. Martin CD, Porter MA. The extraordinary SVD. The American Mathematical Monthly. 2012 Dec 1;119(10):838-51.  (Excellent Review!)

Date: 01-16-2025

"If you don’t believe in yourself why is anyone else going to believe in you?" ― Tom Brady

• Eckhart-Young Theorem and Low Rank Approximations 

• Poincare Diagram: Stability diagram classifying Poincaré maps as stable or unstable according to their features

• Image Compression with Low-Rank SVD: Example

• My talk yesterday (1/15/2025)

Date: 01-21-2025

Quote of the Day

"Young man, in mathematics you don't understand things. You just get used to them" ― John von Neumann 

• Guest Lecture: Dr. Cleve Moler

MathWorks Co-Founder and Chief Mathematician Cleve B. Moler

• 1. How SVD Saved the Universe

  2. 1976 Matrix Singular Value Decomposition Film

  3. Direct link to YouTube:  https://www.youtube.com/watch?v=R9UoFyqJca8

Date: 1-23-2025

Quote of the Day

"Everything is practice" ― Pele 

• SVD and PCA

• Scree Plots and Optimal Hard Threshold (OHT)

• Low Rank Approximations 

• OHT and Clustering

• Mouse Data

Additional Reading

1. Gavish, Matan, and David L. Donoho. "The optimal hard threshold for singular values is 4/sqrt(3)." IEEE Transactions on Information Theory 60.8 (2014): 5040-5053.  (Amazing Paper!)

2. Excellent book chapter (Please read pages 31 - 32)

https://databookuw.com/

3. Donoho, David, Matan Gavish, and Elad Romanov. "ScreeNOT: Exact MSE-optimal singular value thresholding in correlated noise." The Annals of Statistics 51, no. 1 (2023): 122-148. 

4. Udell, Madeleine, and Alex Townsend. "Why are big data matrices approximately low rank?." SIAM Journal on Mathematics of Data Science 1.1 (2019): 144-160. 

5. Thibeault, Vincent, Antoine Allard, and Patrick Desrosiers. "The low-rank hypothesis of complex systems." Nature Physics 20, no. 2 (2024): 294-302. 

Date: 1-28-2025

Quote of the Day

"Nature has a great simplicity and therefore a great beauty" ― Richard Feynman 

• Poincare Diagram: Stability diagram classifying Poincaré maps as stable or unstable according to their features

• Inverse and Pseudoinverse of a Matrix

• Dynamic Mode Decomposition (DMD)

• DMD Demo Codes

Book

• Kutz, J. Nathan, et al. Dynamic mode decomposition: data-driven modeling of complex systems. Society for Industrial and Applied Mathematics, 2016. 

Chapter 1: Dynamic Mode Decomposition: An Introduction

Additional Reading

1. Schmid, Peter J. "Dynamic mode decomposition of numerical and experimental data." Journal of fluid mechanics 656 (2010): 5-28. 

2. Schmid, Peter J. "Dynamic mode decomposition and its variants." Annual Review of Fluid Mechanics 54 (2022): 225-254. 

3. Tu, Jonathan H. "Dynamic mode decomposition: Theory and applications." PhD diss., Princeton University, 2013.

Date: 1-30-2025

Quote of the Day

"Be constantly on the lookout for hype"―David Heckerman

• Guest Lecture: Causal discovery from data (Dr. David Heckerman from Amazon)

• Please visit and like these GitHub:

  1. https://github.com/Jpickard1/BRAD

  2. https://github.com/Jpickard1/BRAD-Video

Date: 2-04-2025

Quote of the Day

"I think one of the things about creativity is not to be afraid of saying the wrong thing " ― Sydney Brenner 

• Data and Laplacian Matrix

• Edify Image

Papers

1. Von Luxburg, Ulrike. "A tutorial on spectral clustering." Statistics and computing 17.4 (2007): 395-416.  (Excellent Review!)

2. Belkin, Mikhail, and Partha Niyogi. "Laplacian eigenmaps and spectral techniques for embedding and clustering." Advances in neural information processing systems. 2002. 

3. Ng, Andrew Y., Michael I. Jordan, and Yair Weiss. "On spectral clustering: Analysis and an algorithm." Advances in neural information processing systems 2 (2002): 849-856.

4. Eliasof, Moshe, Eldad Haber, and Eran Treister. "Graph neural reaction diffusion models." SIAM Journal on Scientific Computing 46, no. 4 (2024): C399-C420. 

Date: 2-06-2025

Quote of the Day

"You have to spend some energy and effort to see the beauty of mathematics" ― Maryam Mirzakhani  

• Working with Data (PS2: 5 and 6)

  • Joshua Pickard, Cooper Stansbury, Amit Surana, Anthony Bloch, and Indika Rajapakse. "Biomarker Selection for Adaptive Systems." arXiv preprint arXiv:2405.09809 (2024). 

• Data Collections

• Graph Representation Learning

• Cooper Stansbury will present a Bioinformatics PhD Defense

Date: 2-11-2025

Quote of the Day

"Imagination will often carry us to worlds that never were. But without it we go nowhere" ― Carl Sagan

• Marc's DMD Slides

  • Joshua L. Proctor, Philip A. Eckhoff. "Discovering dynamic patterns from infectious disease data using dynamic mode decomposition." International Health, Volume 7, Issue 2, March 2015

• OHT revisited: Slides from Matan Gavish

• K-Means Clustering

• The leading eigenvector

Date: 2-13-2025

Quote of the Day

"I believe in my game, and I believe in me. At the end of the day, I'm my biggest fan" ― Serena Williams 

• Stories light the path to mastery: Professor Lee Hartwell

• Low rank matrices (Let's do it one more time!)

Papers

1. Udell, Madeleine, and Alex Townsend. "Why are big data matrices approximately low rank?." SIAM Journal on Mathematics of Data Science 1.1 (2019): 144-160.

Date: 02- 18-2025

"Stay hungry, stay foolish" ― Steve Jobs 

• Least-squares problems 

• Data-guided Control (DGC)

Papers

1. Ronquist S, Patterson G, Muir LA, Lindsly S, Chen H, Brown M, Wicha MS, Bloch A, Brockett R, Rajapakse I. "Algorithm for cellular reprogramming." Proceedings of the National Academy of Sciences. 2017 Nov 7;114(45):11832-7. Data-guided Control (DGC) Supporting Information and Slides from Scott Ronquist 

2. Proctor, Joshua L., Steven L. Brunton, and J. Nathan Kutz. "Dynamic mode decomposition with control." SIAM Journal on Applied Dynamical Systems 15, no. 1 (2016): 142-161. 

Date: 02- 20-2025

"I don’t think limits”  ― Usain Bolt

1. Spectral Clustering and Data-Guided Control (DGC)

2. Mathematics of DGC

3. Some additional materials

Date: 02- 25-2025

"The only way to learn mathematics is to do mathematics ” ― Paul Halmos 

Linear Dimension Reduction

• PCA and MDS

• Mathematics of DGC

• Some additional materials

Date: 02- 27-2025

"Simplicity makes me happy ” ― Alicia Keys 

Nonlinear Dimension Reduction

• Data Background for Motivation 

• Kullback–Leibler divergence 

  • Other Loss functions 

  • Good post that explains some common ones: https://www.geeksforgeeks.org/ml-common-loss-functions/ 

• Stochastic Neighbor Embedding and UMAP

Papers

1. UMPAP: McInnes, Leland, John Healy, and James Melville. "Umap: Uniform manifold approximation and projection for dimension reduction." arXiv preprint arXiv:1802.03426 (2018).

2. t-SNE: Van der Maaten, Laurens, and Geoffrey Hinton. "Visualizing data using t-SNE." Journal of machine learning research 9.11 (2008).

3. Wang, Y., Huang, H., Rudin, C. and Shaposhnik, Y., 2021. Understanding how dimension reduction tools work: an empirical approach to deciphering t-SNE, UMAP, TriMAP, and PaCMAP for data visualization. The Journal of Machine Learning Research, 22(1), pp.9129-9201. 

Remark: This is a really nice review in general, but the most relevant parts are pages 8, 9, and 10

Winter Vacation (March 1 - March 10, 2025): Suggested Reading!

1. Strogatz S. Sync: The emerging science of spontaneous order. 

2. uring, Alan Mathison. "The chemical basis of morphogenesis." Bulletin of mathematical biology 52.1-2 (1990): 153-197. 

Date: 03- 12-2025

"Study hard what interests you the most in the most undisciplined, irreverent and original manner possible ” ― Richard P. Feynman

The Pleasure of Finding Things Out

Nonlinear Dimension Reduction

• Data Background for Motivation 

  • SLIDES

• Matlab Toolbox for Dimensionality Reduction

Papers

1. Wang, Y., Huang, H., Rudin, C. and Shaposhnik, Y., 2021. Understanding how dimension reduction tools work: an empirical approach to deciphering t-SNE, UMAP, TriMAP, and PaCMAP for data visualization. The Journal of Machine Learning Research, 22(1), pp.9129-9201. 

Remark: This is a really nice review in general, but the most relevant parts are pages 8, 9, and 10

Date: 3-13-2025

Quote of the Day

"The true delight is in the finding out rather than in the knowing"―Isaac Asimov  

Tensors = Multi-Dimensional Arrays, Graphs represented with adjacency matrix and Hypergraphs represented with adjacency tensor 

• Tensors and Hypergraphs (Notes)

• Tensors (Slides) and Hypergraphs (Slides)

• Hypergraphs: STEPHEN WOLFRAM  and Wolfram Physics Project Launch

Other Resources:  Dr. Charles Van Loan

Papers

1. Kolda, Tamara G., and Brett W. Bader. "Tensor decompositions and applications." SIAM review 51.3 (2009): 455-500. (Excellent Review!)

2. Williams, Alex H., et al. "Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor component analysis." Neuron 98.6 (2018): 1099-1115.

3. Wolf, Michael M., Alicia M. Klinvex, and Daniel M. Dunlavy. "Advantages to modeling relational data using hypergraphs versus graphs." 2016 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2016.

Date: 3-18-2025

Quote of the Day

"The best way to have a good idea is to have a lot of ideas"― Linus Pauling

• Decomposing a Tensor: Examples

• HAT: Hypergraph Analysis Toolbox

Links for Programming Demo:

• HAT Demo Codes

• Neuron Data and Pore-C Data

Papers

1. Pickard J,  Can C, Salman R, Stansbury C, Kim S, Surana A, Rajapakse I. “HAT: Hypergraph Analysis Toolbox,” PLOS Computational Biology, 2023 

Date: 3-27-2025

"No great discovery was ever made without a bold guess" ― Isaac Newton

• Tensors and Hypergraphs (Notes)

• Marc's Slides

• Slides: Hypergraph Similarity

Papers

1. Donnat, Claire, and Susan Holmes. "Tracking network dynamics: A survey using graph distances." The Annals of Applied Statistics 12.2 (2018): 971-1012. 

2. Surana A, Chen C, Rajapakse I. "Hypergraph Similarity Measures."  IEEE Transactions on Network Science and Engineering,  2022

Date: 4-01-2025

Quote of the Day

"The only way to do great work is to love what you do" ― Steve Jobs 

• 1. Topological Data Analysis 2. The Shape of Data:Nice introduction by Gunnar Carlsson

• Chapter 2: Charles Pugh. Real mathematical analysis. (Excellent Exposition!)

• Introduction: These slides deck includes slides from my good friend Yuan Yao

Papers

1. Carlsson G. Topological methods for data modelling. Nature Reviews Physics 2, no. 12 (2020): 697-708. 

2. Wasserman L. Topological data analysis. Annual Review of Statistics and Its Application. 2018 Mar 7;5:501-32. 

3. Lum PY, Singh G, Lehman A, Ishkanov T, Vejdemo-Johansson M, Alagappan M, Carlsson J, Carlsson G. Extracting insights from the shape of complex data using topology. Scientific reports. 2013 Feb 7;3(1):1-8. 

Date: 4-03-2025

Quote of the Day

"The more we know, the more we realize there is to know" ― Jennifer Doudna

1. Introduction: These slides deck includes slides from my good friend Yuan Yao

2. Matrix Completion in the Nuclear Norm

3. Convex Optimization Tools: CVX

Remark: Condition Number and Hierarchical Clustering 

Papers

1. Koren Y, Bell R, Volinsky C. Matrix factorization techniques for recommender systems. Computer. 2009 Aug 7;42(8):30-7. 

2. Van Dijk D, Sharma R, Nainys J, Yim K, Kathail P, Carr AJ, Burdziak C, Moon KR, Chaffer CL, Pattabiraman D, Bierie B. Recovering gene interactions from single-cell data using data diffusion. Cell. 2018 Jul 26;174(3):716-29.  MAGIC

3. Davis, George J., and Cleve B. Moler. Sensitivity of matrix eigenvalues. International Journal for Numerical Methods in Engineering 12, no. 9 (1978): 1367-1373.  Blog post