ReportofContributions StatisticalMechanicsofInference

(1)

Statistical Mechanics of Inference

Report of Contributions

https://indico.fysik.su.se/e/2718

(2)

Statistical Mech … / Report of Contributions (no title)

Contribution ID: 250 Type: not specified

(no title)

Primary author:

Prof. SOLLA, Sara (Northwestern University)

October 20, 2022 Page 1

(3)

(no title)

Primary author:

(4)

(no title)

Presenter:

(5)

Statistical Mech … / Report of Contributions Statistical mechanics for a network …

Statistical mechanics for a network of real neurons

Monday, 5 September 2011 12:00 (1 hour)

In most areas of the brain, information is encoded in the correlated activity of large populations of neurons. Here we build probabilistic models of such population codes using maximum entropy principle from new recordings of more than 100 retinal ganglion cells from a dense patch on the salamander retina. We illustrate how the pairwise maximum entropy (Ising-like) models can be extended to capture better the experimental data. We analyze the qualitative features of these codes and report on their emerging critical behavior. These results can be put into context by a theoretical examination of information- maximizing codes for noisy spiking neurons.

Presenter:

TKACIK, Gasper (IST, Austria)

(6)

Statistical Mech … / Report of Contributions On the criticality of the inferred m …

On the criticality of the inferred models

Advanced inference techniques allow one to reconstruct the pattern of interaction from high dimensional data sets. We focus here on the statistical properties of inferred models and argue that inference procedures are likely to yield models which are close to a phase transition. On one side, we show that the reparameterization invariant metrics in the space of probability distributions of these models (the Fisher Information) is directly related to the model’s susceptibility. As a result, distinguishable models tend to accumulate close to critical points, where the susceptibility diverges in infinite systems. On the other, this region is the one where the estimate of inferred parameters is most stable. In order to illustrate these points, we discuss inference of interacting point processes with application to financial data and show that sensible choices of observation time-scales naturally yield models which are close to criticality.

Presenter:

Dr MARSILI, Matteo (Abdus Salam ICTP)

(7)

Statistical Mech … / Report of Contributions Approximate inference for contin …

Approximate inference for continuous time Markov processes

Wednesday, 7 September 2011 12:00 (1 hour)

Continuous time Markov processes (such as jump processes and diffusions)

play an important role in the modelling of dynamical systems in many

scientific areas ranging from physics to systems biology.

In a variety of applications, the stochastic state of the system as a function of time is not

directly observed.

One has only access to a set of nolsy observations taken at discrete

times.

The problem is then to infer the unknown state path as best as possible.

In addition, model parameters (like diffusion constants or transition rates) may also be

unknown and have to be estimated from the data.

Since Monte Carlo sampling approaches can be time consuming

one is interested in efficient approximations.

I will discuss variational approaches to this problem which are based

on methods developed in statistical physics and machine learning and

which have also interesting relations to stochastic optimal control.

Applications to transcriptional regulation in systems biology will

be given.

Presenter:

OPPER, Manfred (TU Berlin)

(8)

Statistical Mech … / Report of Contributions An approximate forward- …

An approximate forward-backward algorithm

Tuesday, 6 September 2011 11:00 (1 hour)

In the presentation we propose computationally feasible approximations to binary Markov random fields. The basis of the approximation is the forward-backward algorithm. This exact algorithm is computationally feasible only for fields defined on small graphs. The forward part of the algorithm computes a series of joint marginal distributions by summing out each variable in turn. We construct an approximate forward-backward algorithm by adapting approximation results for pseudo-Boolean functions. By using an approximation of the energy function which minimises the error sum of squares we construct a forward- backward algorithm which is computationally viable. We also show how our approach gives us upper and lower bounds as well as an approximate Viterbi algorithm. Through examples we demonstrate the accuracy and flexibility of our

approximate algorithm.

Presenter:

TJELMELAND, Hakon (NTNU)

(9)

(no title)

(10)

(no title)

Presenter:

TRUCCOLO, Wilson (Brown University)

(11)

Statistical Mech … / Report of Contributions Disentangling individual and gro …

Disentangling individual and group properties in complex networks

Tuesday, 6 September 2011 12:00 (1 hour) Not all nodes in a network are created equal. Differences

and similarities exist at both individual node and group levels. Disentangling single node from group properties is crucial for network modelling and structural inference.

Based on unbiased generative probabilistic exponential random graph models and employing distributive message passing techniques, we present an efficient algorithm that allows one to separate the contributions of individual nodes and groups of nodes to the network structure. This leads to improved detection accuracy of latent class structure in real world data sets compared to models that focus on group structure alone. Furthermore, the inclusion of hitherto neglected group specific effects in models used to assess the statistical significance of small subgraph (motif) distributions in networks may be sufficient to explain most of the

observed statistics. We show the predictive power of such generative models in forecasting putative gene-disease associations in the Online Mendelian Inheritance in Man (OMIM) database. The approach is suitable for both directed and undirected uni-partite as well as for bipartite networks.

Reichardt J, Alamino R, Saad D (2011) The Interplay

between Microscopic and Mesoscopic Structures in Complex Networks. PLoS ONE 6(8): e21282.

doi:10.1371/journal.pone.0021282

Presenter:

SAAD, David (Aston University Birmingham)

(12)

Statistical Mech … / Report of Contributions The binary garrote

The binary garrote

In this talk, I present a new model and solution method for sparse regression. The model introduces binary selector variables sifor the features i in a way that is similar to the original garrote model. The posterior probability for s_iis computed in the variational approximation. I refer to this method as the Variational Garrote (VG). The VG is compared numerically with the Lasso method and with ridge regression. Numerical results on synthetic data show that the VG yields more accurate predictions and more accurately reconstructs the true model than the other methods. The naive implementation of the VG requires the inversion of a modified covariance matrix which scales cubic in the number of features. We indicate how for sparse problem the solution can be computed linear in the number of features.

Presenter:

KAPPEN, Bert (Radboud University)

(13)

(no title)

Presenter:

Prof. ZECCHINA, Riccardo (Politecnico di Torino)

(14)

Statistical Mech … / Report of Contributions Inference in an over-complete, …

Inference in an over-complete, ultra-sparse model of olfaction

The general problem faced by the sensory system is to translate from spikes at sensory receptors to stimuli in the outside world. Here we consider how the brain might do this for olfaction, possibly the simplest of the senses. Our starting point is a linear generative model in which the response of each odorant receptor neuron is a linear combination of odors in the outside world; the problem, then, is to find the probabilistic mapping from those responses to odors. This is equivalent to a sparse, overcomplete model, but inference is especially hard because we use an ultra-sparse prior in which odors are either absent or present. We discuss our current progress on this problem, prospects for future directions, and what olfaction tells us about other sensory systems.

Presenter:

LATHAM, Peter (University College London)

(15)

(no title)

(16)

(no title)

Presenter:

ROUDI, Yasser (Nordita)

(17)

(no title)

Presenter:

Prof. MEZARD, Marc (Universite de Paris Sud)

(18)

(no title)

Presenter:

BIEHL, Michael (University of Groningen)

(19)

Statistical Mech … / Report of Contributions Unconventional and Adaptive Dis …

Unconventional and Adaptive Distance Measures:

applications in life sciences

Wednesday, 7 September 2011 09:30 (1 hour) An introduction to distance based classification of multi-

dimensional data is given.The popular Learning Vector Quantization (LVQ) will serve as the main example in this talk. Here, typical representatives of the classes

(prototypes) are determined from labelled example data in a supervised training process. In the working phase,the prototypes parameterize a classifier which can be applied to novel, unlabelled data.

A key issue in LVQ and many related methods is the choice of a suitable similarity or distance measure. So- called relevance learning schemes employ parameterized distance measures which are optimized in the data-driven training process. The recently introduced Matrix Relevance LVQ, based on generalized Euclidean distances, will be discussed in greater detail. It is straightforward to extend the framework beyond Euclidean measures. As an

important example, the use of statistical divergences in LVQ is introduced.

Divergences can serve as generalized distances when data correspond to positive or normalized measures, as for instance in the histogram based classification of image data.

Matrix Relevance Learning and Divergence based LVQ are illustrated in terms of a number of real world applications from the biomedical domain. These include adrenal tumor classification based on steroid excretion values and the detection and classification of plant diseases using color histograms.

Presenter:

BIEHL, Michael (University of Groningen)

(20)

Statistical Mech … / Report of Contributions Cascading failures in complex- …

Cascading failures in complex-networks: Power Blackouts and the Domino Effect

Our contemporary societies rely more and more on a steady and reliable power supply for their well-functioning. During the last few decades a number of large-scale power

blackouts have been witnessed around the world, and this has caused major concerns among politicians and citizens.

In this talk we will mention a few major power blackouts and discuss the sequence of events and why they occurred.

These empirical examples show that major power blackouts often are results of a cascading of failures (a “Domino effect”). We will introduce a generic (random walk) model for the study of cascading failures in networks, and investigate the impact of transient dynamics caused by the redistribution of loads after an initial network failure (triggering event). It is found that considering instead the stationary states, as has been done in the past, may dramatically overestimate (by 80-95%) the robustness of the network. This is due to the transient oscillations or overshooting in the loads, when the flow dynamics adjusts to the new (remaining) network structure. Consequently, additional parts of the network may be overloaded and therefore fail before the stationary state is reached. The dynamical effects are strongest on links in the neighborhood of broken links. This can cause failure waves in the network along neighboring links, while the evaluation of the

stationary solution predicts a different order of link failures.

Presenter:

SIMONSEN, Ingve (NTNU)

(21)

(no title)

Presenter:

ALAVA, Mikko (HUT, Espoo, Finland)

(22)

Statistical Mech … / Report of Contributions Fluxes and chemical potential dist …

Fluxes and chemical potential distributions in biochemical networks

Tuesday, 6 September 2011 09:30 (1 hour) The analysis of non-equilibrium steady states of biochemical

reaction networks relies on finding the configurations of fluxes and chemical potentials satisfying stoichiometric (mass balance) and thermodynamic (energy balance) constraints. Efficient methods to explore such states are crucial to predict reaction directionality, calculate physiologic ranges of variability, estimate correlations, and reconstruct the overall energy balance of the network from the

underlying molecular processes. While different techniques for sampling the space generated by mass balance

constraints are currently available, thermodynamics is generically harder to incorporate. Here we introduce a method to sample the free energy landscape of a reaction network at steady state.

(in collaboration with Daniele De Martino, Matteo Figliuzzi, Andrea De

Martino)

Presenter:

MARINARI, Enzo (University of Rome La Sapienza)

(23)

Statistical Mech … / Report of Contributions Classifying Mycobacterium Tuber …

Classifying Mycobacterium Tuberculosis Complex by Message Passing

Thursday, 8 September 2011 11:00 (1 hour)

Presenter:

FRANZ, Silvio (University of Paris-Sud)

(24)

(no title)

Presenter:

SOLLICH, Peter (Kings College London)

(25)

(no title)

Presenter:

SOLLA, Sara (Northwestern University)

(26)

Statistical Mech … / Report of Contributions Stochastic modeling of neuronal e …

Stochastic modeling of neuronal ensemble dynamics

I will review past and ongoing work on the statistical modeling of neuronal ensemble point processes based on conditional intensity functions. These functions incorporate intrinsic single-neuron and network dynamics, as well as extrinsic inputs. I will illustrate the approach and current challenges with applications to single and dual 96-microelectrode cortical- array recordings from human and non-human primates.

Presenter:

TRUCCOLO, Wilson

(27)

Statistical Mech … / Report of Contributions Nonequilibrium network reconstr …

Nonequilibrium network reconstruction

Presenter:

ROUDI, Yasser (Kavli Inst./Nordita)

(28)

Statistical Mech … / Report of Contributions Mean field inference in asymmetr …

Mean field inference in asymmetric Ising systems

Presenter:

Prof. MEZARD, Marc (Universite de Paris Sud)

(29)

Statistical Mech … / Report of Contributions Network inference using asynchr …

Network inference using asynchronously updated kinetic Ising model

Wednesday, 7 September 2011 16:30 (20 minutes)

In the presentation, I will focus on the reconstruction of the network structure based on the synthetic data produced by asynchronously update Ising model. The inferred structure can be obtained by three different approaches: naive mean field (nMF) approximation, Thouless-Anderson-Palmer (TAP) approximation and an exact learning method. I will explain these approaches briefly for asynchronous case and give a primary conclusion of the application ability of these methods.

Presenter:

ZENG, HongLi (Aalto University)

(30)

Statistical Mech … / Report of Contributions Dynamic cavity method and dyna …

Dynamic cavity method and dynamic mean-field for diluted Ising systems

The stationary state of Ising models with Glauber dynamics is studied. In case of fully connected networks, naive mean field approximation is investigated to describe long-time limit of magnetizations. For diluted networks dynamic cavity method is used for a wide range of parameters. The comparison between these two methods show that dynamic BP outperforms naive mean field in diluted networks.

Presenter:

MAHMOUDI, Hamed (Aalto University)

(31)

Statistical Mech … / Report of Contributions Kinetic Ising models with delayed …

Kinetic Ising models with delayed interactions for neural data

Presenter:

WITOELAR, Aree (Kavli Inst./NTNU)

(32)

Statistical Mech … / Report of Contributions Finding influential sets of nodes in …

Finding influential sets of nodes in dynamical processes over networks

Presenter:

Prof. ZECCHINA, Riccardo (Politecnico di Torino)

(33)

Statistical Mech … / Report of Contributions Introductory Lecture: Statistical i …

Introductory Lecture: Statistical inference on biological networks

Thursday, 8 September 2011 12:00 (1 hour)

Coupling large numbers of relatively simple elements often results in networks with complex computational abilities.

Examples abound in biological systems - from genetic to neural networks, from metabolic networks to immune systems, from networks of proteins to networks of economic and social agents. Recent and continuing

increases in the experimental ability to simultaneously track the dynamics of many constituent elements within these networks challenge the theorists to provide conceptual frameworks and develop theoretical tools for the analysis of such vast data. The subject poses great challenges, as the systems of interest are noisy and the available information is incomplete. The techniques and approaches of statistical physics have proven remarkably useful, but need to be further developed in their application to non equilibrium dynamical systems. In this talk, I will review the currently available theoretical tools, describe a few applications to the analysis and characterization of biological networks, and discuss the limitations of these techniques and some of the directions along which novel approaches and

implementations are needed.

Presenter:

SOLLA, Sara A.

(34)

Statistical Mech … / Report of Contributions Generalized Linear Models for ne …

Generalized Linear Models for neural activity

Generalized Linear Models provide a framework for the systematic description of neural activity. The formulation of these models is based on the exponential family of

probability distributions; the case of Bernoulli and Poisson distributions are relevant to the case of stochastic spiking.

In this approach, the time-dependent firing rate of individual neurons is modeled in terms of experimentally accessible correlates of neural activity: patterns of activity of other neurons in the network, inputs provided through various sensory modalities or by other brain areas, and outputs such as muscle activity or motor responses. Model parameters are fit to the maximum of a likelihood function that is everywhere convex. In this talk, I will present the theory of generalized linear models, derive equations for likelihood maximization, and briefly discuss applications of this approach to a variety of problems: the incorporation of refractory effects in Poisson models, the mapping of spatiotemporal receptive fields of individual neurons, the characterization of network connectivity through directed time-dependent pairwise interactions, and the monitoring of plasticity.

Presenter:

SOLLA, Sara A. (Northwestern University)

(35)

Statistical Mech … / Report of Contributions Can projection techniques be used …

Can projection techniques be used to understand subnetwork dynamics?

Thursday, 8 September 2011 09:30 (1 hour) In systems biology we are encouraged to think in terms of

networks to try and understand the complex behaviour of cells. There is much uncertainty in the identification process of proteins, so often the complete network of reactions in a protein interaction network (PIN) is unknown. Even in cases where the whole network is known relatively accurately, it is typically very large and therefore the dynamics hard to analyse in detail; also reliable information on the requisite reaction rates is difficult to obtain. Finally, experimental measurements of the dynamics of protein concentrations, e.g. using optical techniques, are typically possible only for a limited number of protein species simultaneously.

These considerations motivate us to try to find descriptions for the dynamics of moderately sized subnetworks of much larger (”bulk”) reaction networks. Eventually we would like to solve the inverse problem, i.e. infer from observed subnetwork dynamics something about the bulk network properties. As a first step towards this, we address the question of what form the dynamical equations for a subnetwork should take. We propose to use the Mori- Zwanzig projection formalism, which allows one to derive a set of dynamical equations for selected variables from a network using information from the whole network. We point out some ways in which this differs from more familiar applications of projection techniques in mode coupling theory, and give some toy examples to illustrate the approach.

Presenter:

SOLLICH, Peter (Kings College London)