NOTES, SLIDES, AND PAPERS 

Date: 1-10-2023

Quote of the Day

"Ideas only realize their power when people understand them" ― Small Worlds: The Dynamics of Networks between Order and Randomness (Book) 

• Singular Value Decomposition (SVD) 

• Slides

Papers

1. Lieberman-Aiden, Erez, ..., Groudine Mark, ..., Lander Eric. "Comprehensive mapping of long-range interactions reveals folding principles of the human genome." Science 326.5950 (2009): 289-293. 

Date: 1-12-2023

Quote of the Day

"It is not enough to be in the right place at the right time. You should also have an open mind at the right time " ― Paul Erdos

• Eckhart-Young Theorem and Low Rank Approximations 

• Poincare Diagram: Stability diagram classifying Poincaré maps as stable or unstable according to their features

• SVD Image Denoising MATLAB Code (sample image)

Date: 1-17-2023

Quote of the Day

"Everything is practice" ― Pele 

• Eckhart-Young Theorem and Low Rank Approximations 

• Optimal Hard Threshold

• Clustering

Additional Reading

1. Gavish, Matan, and David L. Donoho. "The optimal hard threshold for singular values is 4/sqrt(3)." IEEE Transactions on Information Theory 60.8 (2014): 5040-5053.  (Amazing Paper!)

2. Udell, Madeleine, and Alex Townsend. "Why are big data matrices approximately low rank?." SIAM Journal on Mathematics of Data Science 1.1 (2019): 144-160. 

3. Turk, Matthew, and Alex Pentland. "Eigenfaces for recognition." Journal of cognitive neuroscience 3.1 (1991): 71-86. (Classic! just browse)

Date: 1-19-2023

Quote of the Day

"Optimism is the faith that leads to achievement. Nothing can be done without hope and confidence" ― Helen Keller 

• Guest Lecture: Dr. Cleve Moler

  1. How SVD Saved the Universe

  2. 1976 Matrix Singular Value Decomposition Film

  3. Direct link to YouTube:  https://www.youtube.com/watch?v=R9UoFyqJca8

Date: 1-23-2023

Quote of the Day

"Nature has a great simplicity and therefore a great beauty" ― Richard Feynman 

• Dynamic Mode Decomposition (DMD)

• Poincare Diagram: Stability diagram classifying Poincaré maps as stable or unstable according to their features

Book

• Kutz, J. Nathan, et al. Dynamic mode decomposition: data-driven modeling of complex systems. Society for Industrial and Applied Mathematics, 2016. 

Chapter 1: Dynamic Mode Decomposition: An Introduction

Date: 1-26-2023

Quote of the Day

"Science and everyday life cannot and should not be separated" ― Rosalind Franklin 

• DMD Demo Codes

• Dynamic Mode Decomposition (DMD): Jamboard

Additional Reading

1. Schmid, Peter J. "Dynamic mode decomposition of numerical and experimental data." Journal of fluid mechanics 656 (2010): 5-28. 

2. Schmid, Peter J. "Dynamic mode decomposition and its variants." Annual Review of Fluid Mechanics 54 (2022): 225-254. 

3. Tu, Jonathan H. "Dynamic mode decomposition: Theory and applications." PhD diss., Princeton University, 2013. 

Date: 1-31-2023

Quote of the Day

"I think one of the things about creativity is not to be afraid of saying the wrong thing " ― Sydney Brenner 

• Data and Laplacian Matrix

Papers

1. Von Luxburg, Ulrike. "A tutorial on spectral clustering." Statistics and computing 17.4 (2007): 395-416.  (Excellent Review!)

2. Ng, Andrew Y., Michael I. Jordan, and Yair Weiss. "On spectral clustering: Analysis and an algorithm." Advances in neural information processing systems 2 (2002): 849-856.

Date: 2-2-2023

Quote of the Day

"If you don’t believe in yourself why is anyone else going to believe in you?" ― Tom Brady

• Norms and Distances

Papers

1. Belkin, Mikhail, and Partha Niyogi. "Laplacian eigenmaps and spectral techniques for embedding and clustering." Advances in neural information processing systems. 2002. 

2. Cheeger J. A lower bound for the smallest eigenvalue of the Laplacian. In Problems in analysis 2015 Mar 8 (pp. 195-200). Princeton University Press. 

3. Chen, Pin-Yu, et al. "Fast incremental von neumann graph entropy computation: Theory, algorithm, and applications." International Conference on Machine Learning. PMLR, 2019.

Date: 2-7-2023

Quote of the Day

"The only way to do great work is to love what you do" ― Steve Jobs 

• A Taste of Topology 

Chapter 2: Charles Pugh. Real mathematical analysis. (Excellent Exposition!)

Papers

1. Wasserman L. Topological data analysis. Annual Review of Statistics and Its Application. 2018 Mar 7;5:501-32. 

2. Nicolau M, Levine AJ, Carlsson G. Topology based data analysis identifies a subgroup of breast cancers with a unique mutational profile and excellent survival Proceedings of the National Academy of Sciences. 2011 Apr 26;108(17):7265-70. 

3. Carlsson G. Topology and Data. Bulletin of the American Mathematical Society. 2009;46(2):255-308. 

4. Lum PY, Singh G, Lehman A, Ishkanov T, Vejdemo-Johansson M, Alagappan M, Carlsson J, Carlsson G. Extracting insights from the shape of complex data using topology. Scientific reports. 2013 Feb 7;3(1):1-8. 

5. Derenick, Jason, Alberto Speranzon, and Robert Ghrist. "Homological sensing for mobile robot localization." In 2013 IEEE International Conference on Robotics and Automation, pp. 572-579. IEEE, 2013. 

Date: 2-9-2023

Quote of the Day

"I wasn’t the fastest guy in the world. I wouldn’t have done well in an Olympiad or a math contest. But I like to ponder. And pondering things, just sort of thinking about it and thinking about it, turns out to be a pretty good approach " ― James Simons  

• Spectral Clustering

  • Example1 and Example 2

• K-Means Clustering

• Jamboard

Papers

1. Von Luxburg, Ulrike. "A tutorial on spectral clustering." Statistics and computing 17.4 (2007): 395-416.  (Excellent Review!)

2. Ng, Andrew Y., Michael I. Jordan, and Yair Weiss. "On spectral clustering: Analysis and an algorithm." Advances in neural information processing systems 2 (2002): 849-856.

3. Chen H, Chen J, Muir LA, Ronquist S, Meixner W, Ljungman M, Ried T, Smale S, Rajapakse I. "Functional Organization of the Human 4D Nucleome. " Proceedings of the National Academy of Sciences 112.26 (2015): 8002-8007. Supporting Information 

4. Chen J, Hero A, and Rajapakse I. "Spectral Identification of Topological Domains." Bioinformatics 32.14 (2016): 2151-2158.

Date: 2-14-2023

Quote of the Day

"Order and simplification are the first steps toward the mastery of a subject" ― Thomas Mann 

• Review topics covered so far..

• Problem Set 2 and 3

Beautiful Talk!

Digital twins: A personalized future of computing for complex systems | Karen Willcox | TEDxUTAustin

Date: 2-16-2023

Quote of the Day

"No great discovery was ever made without a bold guess" ― Isaac Newton 

• DMD + Control and Data Guided Control (DGC) 

Papers

1. Liu, Yang-Yu, Jean-Jacques Slotine, and Albert-László Barabási. "Controllability of complex networks." Nature 473.7346 (2011): 167-173. Slides: Courtesy of Yang Liu

Movie: Controllability of Complex Networks - Data Visualization

Date: 2-21-2023

Quote of the Day

"I never, never in my life took a course in economics " ― Lloyd Shapley ( 2012 Nobel Prize for Economics) 

• Jamboard

• Controllability Gramian

• DMD with Control (DMDc) and Model Reduction

Slides: Magnus Egerstedt

Papers

Empirical Gramian

1. Lall, Sanjay, Jerrold E. Marsden, and Sonja Glavaški. "Empirical model reduction of controlled nonlinear systems." IFAC Proceedings Volumes 32, no. 2 (1999): 2598-2603. 

2. Rowley, Clarence W. "Model reduction for fluids, using balanced proper orthogonal decomposition." International Journal of Bifurcation and Chaos 15, no. 03 (2005): 997-1013. 

3. Chen C, Surana A, Bloch A, Rajapakse I. "Data-Driven Model Reduction for Multilinear Control Systems via Tensor Trains." arXiv preprint arXiv:1912.03569 (2020)

Controllability

1. Proctor, Joshua L., Steven L. Brunton, and J. Nathan Kutz. "Dynamic mode decomposition with control." SIAM Journal on Applied Dynamical Systems 15, no. 1 (2016): 142-161. 

2. Rajapakse I, Groudine M, Mesbahi M. Dynamics and control of state-dependent networks for probing genomic organization. Proceedings of the National Academy of Sciences. 2011 Oct 18;108(42):17257-62. 

3. Pasqualetti, Fabio, Sandro Zampieri, and Francesco Bullo. "Controllability metrics, limitations and algorithms for complex networks." IEEE Transactions on Control of Network Systems 1, no. 1 (2014): 40-52. 

Book

• Kutz, J. Nathan, et al. Dynamic mode decomposition: data-driven modeling of complex systems. Society for Industrial and Applied Mathematics, 2016. 

Chapter 6: DMD with Control

Date: 2-23-2023

Quote of the Day

"Be constantly on the lookout for hype"―David Heckerman

Guest Lecture: Methods for Causal Discovery (Dr. David Heckerman from Amazon)

Jamboard

Paper: 

1. Heckerman, David. "Heckerthoughts." arXiv preprint arXiv:2302.05449 (2023).  (Dr. Heckerman will cover sections 3 and 4.5, but your may wish to browse the whole manuscript and are free to ask  him  questions!) 

Date: 3-07-2023

Quote of the Day

"You can always recognize truth by its beauty and simplicity"―Richard P. Feynman 

Jamboard

• Tensors and Hypergraphs (Notes)

• Tensors (Slides)

• Hypergraphs (Slides)

• Decomposing a Tensor: Examples

• POD

Other Resources:  Dr. Charles Van Loan

Papers

1. Kolda, Tamara G., and Brett W. Bader. "Tensor decompositions and applications." SIAM review 51.3 (2009): 455-500. (Excellent Review!)

2. Benzi, Michele, Dario Bini, Daniel Kressner, Hans Munthe-Kaas, Charles Van Loan, and Charles F. Van Loan. "Structured matrix problems from tensors." Exploiting Hidden Structure in Matrix Computations: Algorithms and Applications: Cetraro, Italy 2015 (2016): 1-63. 

3. Chen C, Rajapakse I. "Tensor Entropy  for Uniform Hypergraphs." IEEE Transactions on Network Science and Engineering 7.4 (2020): 2889-2900.

4. Williams, Alex H., et al. "Unsupervised discovery of demixed, low-dimensional neural dynamics across multiple timescales through tensor component analysis." Neuron 98.6 (2018): 1099-1115.

5. Wolf, Michael M., Alicia M. Klinvex, and Daniel M. Dunlavy. "Advantages to modeling relational data using hypergraphs versus graphs." 2016 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2016.

Books

• Eldén, Lars. Matrix methods in data mining and pattern recognition. Society for Industrial and Applied Mathematics, 2007.

Chapter 8: Tensor Decomposition

Date: 3-09-2023

No Class: 

Date: 3-14-2023

"When you reach the end of your rope, tie a knot in it and hang on" ― Franklin D. Roosevelt  

• Matlab Script for Tensors and Hypergraphs

• Tensor Decomposition (Examples with Solutions)

• An application of HOSVD for signals processing (see Figure 4): Multimodal tensor-based method for integrative and continuous patient monitoring during postoperative cardiac care

Software

• Tensor Toolbox

• HAT: Hypergraph Analysis Toolbox

Papers

1. Pickard J,  Can C, Salman R, Stansbury C, Kim S, Surana A, Rajapakse I. “HAT: Hypergraph Analysis Toolbox,” arXiv:2211.11166, 2022 

2. Valdivia, Paola, et al. "Analyzing Dynamic Hypergraphs with Parallel Aggregated Ordered Hypergraph Visualization." IEEE Transactions on Visualization and Computer Graphics (2019).

Date: 3-16-2023

Guest Speaker: Ed Pagani (Ethics of Data Interpretation)

Date: 3-21-2023

"Imagination will often carry us to worlds that never were. But without it we go nowhere" ― Carl Sagan

• SVD existence and uniqueness 

• Lakmal Jayasinghe Slides! (Amazing Technology!)

Books

• • Eldén, Lars. Matrix methods in data mining and pattern recognition. Society for Industrial and Applied Mathematics, 2007.

Chapter 8: Tensor Decomposition

Papers

• 1. De Lathauwer, Lieven, Bart De Moor, and Joos Vandewalle. "A multilinear singular value decomposition." SIAM journal on Matrix Analysis and Applications 21.4 (2000): 1253-1278. (Existence and Uniqueness page 1265: ''The first property implies that the HOSVD shows essentially the same uniqueness properties as the matrix SVD')

Date: 3-23-2023

"No great discovery was ever made without a bold guess" ― Isaac Newton

• Norms and Distances

• Entropy: Slides from ICML 2019 and Hypergraph Entropy

Slides: Hypergraph Similarity

Papers

1. Donnat, Claire, and Susan Holmes. "Tracking network dynamics: A survey using graph distances." The Annals of Applied Statistics 12.2 (2018): 971-1012. 

2. Surana A, Chen C, Rajapakse I. "Hypergraph Similarity Measures."  IEEE Transactions on Network Science and Engineering,  2022

3. Chen C, Rajapakse I. "Tensor Entropy  for Uniform Hypergraphs." IEEE Transactions on Network Science and Engineering 7.4 (2020): 2889-2900

Date: 3-28-2023

"I don't have any magical ability. I look at a problem, play with it, work out a strategy" ― Terence Tao

Matrix Completion and CVX

1. Matrix Completion in the Nuclear Norm

2. Mathematics of Optimization

Papers

1. Koren Y, Bell R, Volinsky C. Matrix factorization techniques for recommender systems. Computer. 2009 Aug 7;42(8):30-7. 

2. Van Dijk D, Sharma R, Nainys J, Yim K, Kathail P, Carr AJ, Burdziak C, Moon KR, Chaffer CL, Pattabiraman D, Bierie B. Recovering gene interactions from single-cell data using data diffusion. Cell. 2018 Jul 26;174(3):716-29. 

3. Candès, Emmanuel J., et al. "Robust principal component analysis?." Journal of the ACM (JACM) 58.3 (2011): 1-37.

Date: 3-30-2023

"Always work hard on something uncomfortably exciting" ― Larry Page 

1. Data and Stories: Minimum number of images required to tell the story 

2. Data from Class

Date: 4-04-2023

"Nothing is more practical than a good theory" ― Ludwing Boltzmann 

Compressive Sensing

1. "Magic" Reconstruction: Compressed Sensing (MATLAB Code) 

2. Backslash

3.  The following two chapters are good references on Compressive Sensing [2, 3].

Sparsity and Compressed Sensing

Basics of Compressed Sensing

Papers

1. Candès EJ, Wakin MB. An introduction to compressive sampling. IEEE signal processing magazine. 2008 Mar 21;25(2):21-30.  (Excellent Review!)

Date: 4-06-2023

"Basically, our goal is to organize the world's information and to make it universally accessible and useful " ― Larry Page 

1. PageRank

2. From Cleve Moler: 1: The World’s Largest Matrix Computation 2. Google PageRank 

Papers

1. Gleich DF. "PageRank beyond the Web." SIAM Review. 2015;57(3):321-63.

2.  Brin, Sergey, and Lawrence Page. "The anatomy of a large-scale hypertextual web search engine." (1998). 

PatentsUS6285999B1

1. US6285999B1 Method for node ranking in a linked database. Lawrence Page: 1998-01-09

2. 115-008219-US-PS1 Network approach to navigating the human genome. Indika Rajapakse: submitted November 2020.

Date: 4-11-2023

"One never notices what has been done; one can only see what remains to be done" ― Marie Curie 

Jamboard

1. Matrix Completion and CVX

2. Compressive Sensing: Beautiful Talk!

3. The Extraordinary SVD

Papers

1. Wang, Y., Huang, H., Rudin, C. and Shaposhnik, Y., 2021. Understanding how dimension reduction tools work: an empirical approach to deciphering t-SNE, UMAP, TriMAP, and PaCMAP for data visualization. The Journal of Machine Learning Research, 22(1), pp.9129-9201. 

Date: 4-11-2023

"You teach me, I forget. You show me, I remember. You involve me, I understand" ― E. O. Wilson 

Jamboard

1. Smallest positive eigenvalue of the graph laplacian 

2. Delay Coordinates and Hankel Matrix

3. Koopman Analysis 

Papers

1. Cheeger, Jeff. "A lower bound for the smallest eigenvalue of the Laplacian, Problems in analysis (Papers dedicated to Salomon Bochner, 1969)." (1970): 195-199. 

2. Rajapakse I, and Smale S. "Emergence of Function from Coordinated Cells in a Tissue." Proceedings of the National Academy of Sciences 114.7 (2017): 1462-1467. 

3. Mezic I. Koopman operator, geometry, and learning. arXiv preprint arXiv:2010.05377. 2020 Oct 12. 

4. Champion K, Lusch B, Kutz JN, Brunton SL. "Data-driven discovery of coordinates and governing equations." Proceedings of the National Academy of Sciences. 2019 Nov 5;116(45):22445-51. 

5. Brunton, Steven L., Bingni W. Brunton, Joshua L. Proctor, Eurika Kaiser, and J. Nathan Kutz. "Chaos as an intermittently forced linear system." Nature communications 8, no. 1 (2017): 19. 

Software: PyDMD

Video: Koopman Operator Theory for Dynamical Systems, Control and Data Analytics by Prof. Igor Mezic