Instructor: Prof. INDIKA Rajapakse

Teaching Assistant: COOPER Stansbury

Class Time: Tuesday and Thursday, 8:30 AM - 10:00 AM

Office Hours:

• Tuesdays (in-person), from 10:00 AM - 11:00 AM EST in DCMB (Palmer Commons) Rm. 2017A

• Thursday and Friday (remote), after 2pm EST by appointment via Google Meet: meet.google.com/dnm-zipr-tsb (Links to an external site.) (note that this is a different link)

Syllabus

Course Reading List

Simplicity, Rigor, and Magic

References of Interest

• Digital Library

• Dr. Cleve Moler's Blog

• Matrix World

• SVD and Image Compression

• Basic Notation

Recommended Reading Material

• Dr. Cleve Moler's Book: Numerical Computing with MatLab

• Jon Kleinberg's Book: Networks, Crowds, and Markets

PROBLEM OF THE DAY

POD1: Class Expectations

POD2: Symmetric Stein Divergence

PROBLEM SETS

Problem Set 1

Problem Set 2

• Hint for Problem Set 2 part C

• Cleve Moler's answers PDF

• Cleve Moler's MATLAB Files

Problem Set 3

• Data files

• Helpful code

Problem Set 4

• Data files

• A tutorial on spectral clustering

• Solution

Problem Set 5

• MATLAB Solution

• Solution Explanation

Problem Set 6

• Data files

• Algorithm for cellular reprogramming

• MATLAB Helpful Code

Problem Set 7

• Data files

• A global genetic interaction network maps a wiring diagram of cellular function

• A reference map of the human binary protein interactome

• Centrality Example (Data for Example)

Problem Set 8

• MATLAB Data Files

• Helpful Code

Final Problem Set

• Overleaf Template

• Final Project Ideas

NOTES, SLIDES, AND PAPERS

Date: 12-09-2021

• Mathematics of Optimization

• Network Systems: Synchrony and its Breakdown

• Biological Controllability and Templating

• CHALLENGE Problem

Papers

• 1. P. J. Mucha, T. Richardson, K. Macon, M. A. Porter, and J.-P. Onnela, “Community Structure in Time-Dependent, Multiscale, and Multiplex Networks,” Science, vol. 328, no. 5980, pp. 876–878, May 2010, doi: 10.1126/science.1184819.

  2. Stankovski T, Pereira T, McClintock PV, Stefanovska A. Coupling functions: universal insights into dynamical interaction mechanisms. Reviews of Modern Physics. 2017 Nov 6;89(4):045001.

  3. Too much Coupling

Date: 12-07-2021

• Models of Networks

• Turing System

Supplementary materials that used in today's lecture

• The Best is Yet to Come

• On Distributed Communication Networks

• Displaying Correlation Matrices

Papers

1. Strogatz SH. "Exploring complex networks." nature. 2001 Mar;410(6825):268-76.

2. Barabási, Albert-László, and Réka Albert. "Emergence of scaling in random networks." science 286, no. 5439 (1999): 509-512.

3. Rajapakse, Indika, and Stephen Smale. "Emergence of function from coordinated cells in a tissue." Proceedings of the National Academy of Sciences 114.7 (2017): 1462-1467.

Date: 12-02-2021

• Laplacian

  1. • Spectral Graph Drawing

• Optimal Hard Threshold

  1. • MATLAB Example Notebook

     • Image for MATLAB Example

     • Examples of Optimal Hard Threshold

Papers

1. Gavish, Matan, and David L. Donoho. "The optimal hard threshold for singular values is 4/sqrt(3)." IEEE Transactions on Information Theory 60.8 (2014): 5040-5053.

2. Excellent book chapter

Date: 11-30-2021

• Mathematics of Optimization

Papers

3. Gavish, Matan, and David L. Donoho. "The optimal hard threshold for singular values is 4/sqrt(3)." IEEE Transactions on Information Theory 60.8 (2014): 5040-5053. (Amazing Paper!)

   • Supplementary Code

   • Python Code

FUN readings.....

1. Buchanan, Mark. "The best is yet to come." Nature 447.7140 (2007): 39-39.

2. More papers: Connections in 2007

Date: 11-23-2021

• Controlling and Observing Networks

• Network Motifs

• Data and Laplacian Matrix

• Diseasome

• Final Project Ideas

Please read!

1. Benson, Austin R., David F. Gleich, and Jure Leskovec. "Higher-order organization of complex networks." Science 353.6295 (2016): 163-166.

URL: https://snap.stanford.edu/higher-order/

2. D. L. Barabási and A.-L. Barabási, “A Genetic Model of the Connectome,” Neuron, vol. 105, no. 3, pp. 435-445.e5, Feb. 2020, doi: 10.1016/j.neuron.2019.10.031.

Date: 11-18-2021

• Network Dynamics: Stability and Bifurcations cont.....

Additional Information:

• Inequalities between the Jenson-Shannon and Jeffreys divergences

• Community Structure

Papers

1. Tero, Atsushi, et al. "Rules for biologically inspired adaptive network design." Science 327.5964 (2010): 439-442.

2. Rajapakse I, Groudine M, Mesbahi M. "What can systems theory of networks offer to biology?" PLoS Comput Biol. 2012 Jun 28;8(6):e1002543.

3. J. Gao, B. Barzel, and A.-L. Barabási, “Universal resilience patterns in complex networks,” Nature, vol. 530, no. 7590, pp. 307–312, Feb. 2016, doi: 10.1038/nature16948.

Date: 11-16-2021

• Network Dynamics: Stability and Bifurcations

  • Beautiful Biology!

• Network Visualization: (Gephi)

  • Tutorial

  • Data: Node Data, Edge Data

Papers

1. Donnat, Claire, and Susan Holmes. "Tracking network dynamics: A survey using graph distances." The Annals of Applied Statistics 12.2 (2018): 971-1012

2. Surana A, Chen C, Rajapakse I. "Hypergraph Similarity Measures." arXiv preprint arXiv:2106.08206 (2021)

Date: 11-11-2021

• Network Neuroscience (slides)

• Why Hypergraphs?

TED Talk: I am my connectome

Papers

• 1. Restrepo, Juan G., Edward Ott, and Brian R. Hunt. "Characterizing the dynamical importance of network nodes and links." Physical review letters 97.9 (2006): 094102.

  2. Sweeney P, Chen C, Rajapakse I, Cone R. "Network Dynamics of Hypothalamic Feeding Neurons." Proceedings of the National Academy of Sciences, 118.14 (2021).

Date: 11-09-2021

• Network Robustness: Network robustness is the ability of a network to survive from random failures or deliberate attacks (e.g., removal of nodes or edges). It is intimately related to the underlying network structure/topology.

• Tensors and Hypergraphs

Papers:

• 1. Kitano, Hiroaki. "Biological robustness." Nature Reviews Genetics 5.11 (2004): 826-837.

  2. Chen P, Wu L, Liu S, Rajapakse I. "Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications." 2019 International Conference on Machine Learning (2019).

  3. Chen C, Rajapakse I. "Tensor Entropy for Uniform Hypergraphs." IEEE Transactions on Network Science and Engineering 7.4 (2020): 2889-2900.

Book:

Network Science: Network Robustness

Date: 11-04-2021

• Nonlinear Dynamics of Networks

  • • DMD Example

    • MATLAB Code

• Network Centrality

• Tensors and Hypergraphs

Date: 11-02-2021

• Nonlinear Dynamics of Networks

  • • DMD Example

    • MATLAB Code

• Network Centrality

• Tensors and Hypergraphs

Date: 10-28-2021

• Dynamic Mode Decomposition (DMD)

• Poincare Diagram: Stability diagram classifying Poincaré maps as stable or unstable according to their features

• Controllability Metrics for Networks: Controllability Gramian

Papers

Data-guided Control (DGC)

1. Ronquist S, Patterson G, Muir LA, Lindsly S, Chen H, Brown M, Wicha MS, Bloch A, Brockett R, Rajapakse I. "Algorithm for cellular reprogramming." Proceedings of the National Academy of Sciences. 2017 Nov 7;114(45):11832-7.

Date: 10-26-2021

• Dynamic Mode Decomposition (DMD)

Papers

4. Gavish, Matan, and David L. Donoho. "The optimal hard threshold for singular values is 4/sqrt(3)." IEEE Transactions on Information Theory 60.8 (2014): 5040-5053. (Amazing Paper!)

   • Supplementary Code

   • Python Code

Book

• Kutz, J. Nathan, et al. Dynamic mode decomposition: data-driven modeling of complex systems. Society for Industrial and Applied Mathematics, 2016.

Chapter 1: Dynamic Mode Decomposition: An Introduction

Date: 10-21-2021

• Bifurcations and Genetic Networks

• Controllability Metrics for Networks: Controllability Gramian

Papers

1. Timothy Gardner, Charles Cantor, and James Collins. "Construction of a genetic toggle switch in escherichiacoli." Nature, 403(6767): 339, 2000.

2. Michael Elowitz and Stanislas Leibler. "A synthetic oscillatory network of transcriptional regulators." Nature, 403(6767): 335, 2000.

3. Spencer, Sabrina L., et al. "The proliferation-quiescence decision is controlled by a bifurcation in CDK2 activity at mitotic exit." Cell 155.2 (2013): 369-383.

4. Rajapakse I, and Smale S. "Mathematics of the Genome." Foundations of Computational Mathematics 17.5 (2017): 1195-1217.

Can We Control the Brain Networks?

1. Controllability of structural brain networks

2. Warnings and Caveats in Brain Controllability

Date: 10-14-2021

• Data-Guided Controllability

• Network Controllability Part 2

• Dynamic Mode Decomposition (DMD)

Papers

Data-guided Control (DGC)

2. Ronquist S, Patterson G, Muir LA, Lindsly S, Chen H, Brown M, Wicha MS, Bloch A, Brockett R, Rajapakse I. "Algorithm for cellular reprogramming." Proceedings of the National Academy of Sciences. 2017 Nov 7;114(45):11832-7.

Date: 10-12-2021

• Network Controllability Part 1

• Network Controllability Part 2

• K-Means Clustering

  • • • Example K-Means Code

Papers

1. Liu, Yang-Yu, Jean-Jacques Slotine, and Albert-László Barabási. "Controllability of complex networks." Nature 473.7346 (2011): 167-173.

2. Rajapakse I, Groudine M, Mesbahi M. "Dynamics and control of state-dependent networks for probing genomic organization." Proceedings of the National Academy of Sciences. 2011 Oct 18;108(42):17257-62.

3. Gurdon, J. B., and D. A. Melton. "Nuclear reprogramming in cells." science 322.5909 (2008): 1811-1815.

Date: 10-07-2021

• Google Matrix and Perron

• Norms

Extra Notes: Prof. David Bindel's Fall 2019 6210 course notes

Papers

1. Brin, Sergey, and Lawrence Page. "The anatomy of a large-scale hypertextual web search engine." (1998).

2. Bryan, Kurt, and Tanya Leise. "The $25,000,000,000 eigenvector: The linear algebra behind Google." SIAM review 48.3 (2006): 569-581.

3. Ng AY, Jordan MI, Weiss Y. "On spectral clustering: Analysis and an algorithm." In Advances in neural information processing systems (2002) (pp. 849-856).

Date: 10-05-2021

• Eckhart-Young: The Closest Rank k Matrix to A

Papers

3. Ng AY, Jordan MI, Weiss Y. "On spectral clustering: Analysis and an algorithm." In Advances in neural information processing systems (2002) (pp. 849-856).

4. Chen H, Chen J, Muir LA, Ronquist S, Meixner W, Ljungman M, Ried T, Smale S, Rajapakse I. "Functional Organization of the Human 4D Nucleome. " Proceedings of the National Academy of Sciences 112.26 (2015): 8002-8007.

Please read!

5. Benson, Austin R., David F. Gleich, and Jure Leskovec. "Higher-order organization of complex networks." Science 353.6295 (2016): 163-166.

URL: https://snap.stanford.edu/higher-order/

Date: 9-30-2021

• Network Centrality

• Norms

Slides

• Hi-C and SVD

Papers

1. Lieberman-Aiden, Erez, et al. "Comprehensive mapping of long-range interactions reveals folding principles of the human genome." Science 326.5950 (2009): 289-293.

2. Misteli, T., 2011. The inner life of the genome. Scientific American, 304(2), p.66.

Date: 9-28-2021

• Notes on the Laplacian

• Turing System

• KULLBACK-LEIBLER Divergence

Papers

1. Bapat, R. B. "The Laplacian matrix of a graph." Mathematics Student-India 65, no. 1 (1996): 214-223.

Book (Great book!): Chapter 1

1. Spielman, Daniel A. "Spectral and Algebraic Graph Theory." (2019)

Date: 9-23-2021

• Inverse and Moore–Penrose inverse

• Turing System

• KULLBACK-LEIBLER Divergence

Papers

1. Rajapakse, Indika, and Stephen Smale. "Emergence of function from coordinated cells in a tissue." Proceedings of the National Academy of Sciences 114.7 (2017): 1462-1467.

2. Too much Coupling

Date: 9-21-2021

• Inverse and Moore–Penrose inverse

• Turing System

• Slides on Covariance Matrices

Papers

1. Rajapakse, Indika, and Stephen Smale. "Emergence of function from coordinated cells in a tissue." Proceedings of the National Academy of Sciences 114.7 (2017): 1462-1467.

FUN readings.....

1. Alon, Uri. "Biological networks: the tinkerer as an engineer." Science 301.5641 (2003): 1866-1867.

2. Too much Coupling

Date: 9-16-2021

• Eigenvalues of Graphs

Papers

1. Watts, Duncan J., and Steven H. Strogatz. "Collective dynamics of ‘small-world’networks." Nature 393.6684 (1998): 440-442.

Date: 9-14-2021

• Models of Networks

• MATLAB Examples

Papers

4. Strogatz SH. "Exploring complex networks." nature. 2001 Mar;410(6825):268-76.

5. Barabási, Albert-László, and Réka Albert. "Emergence of scaling in random networks." science 286, no. 5439 (1999): 509-512.

Date: 9-09-2021

Guest Lecture: Dr. Cleve Moler

• How SVD Saved the Universe

Date: 9-07-2021

Topics

• Review of Eigenvalues and Eigenvectors

• Introduction to Singular Value Decomposition (SVD)

Slides

• Hi-C and SVD

Papers

3. Lieberman-Aiden, Erez, et al. "Comprehensive mapping of long-range interactions reveals folding principles of the human genome." Science 326.5950 (2009): 289-293.

4. Turk, Matthew, and Alex Pentland. "Eigenfaces for recognition." Journal of cognitive neuroscience 3.1 (1991): 71-86.

Date: 9-02-2021

Topics

• Networks: (1) Random, (2) Small-world, (3) Scale-free, (4) Ramanujan

• Review of Adjacency, Degree matrices

• Laplacian, Eigenvalues and Eigenvectors

Slides

• Matrix Operations

Date: 8-31-2021

Topics

• Introduction to course logistics

• Nodes and Edges

• Introduction to Adjacency and Degree matrices