Critical Branching Regulation of the E-I Net Spiking Neural Network Model

Net Spiking Neural Network Model

Oskar Öberg

Teknisk fysik och elektroteknik, master 2019

Luleå tekniska universitet Institutionen för system- och rymdteknik

A

BSTRACT

Spiking neural networks (SNN) are dynamic models of biological neurons, that commu-nicates with event-based signals called spikes. SNN that reproduce observed properties of biological senses like vision are developed to better understand how such systems function, and to learn how more efficient sensor systems can be engineered. A branching parameter describes the average probability for spikes to propagate between two different neuron populations. The adaptation of branching parameters towards critical values is known to be important for maximizing the sensitivity and dynamic range of SNN. In this thesis, a recently proposed SNN model for visual feature learning and pattern recognition known as the E–I Net model is studied and extended with a critical branching mecha-nism. The resulting modified E–I Net model is studied with numerical experiments and two different types of sensory queues. The experiments show that the modified E–I Net model demonstrates critical branching and power-law scaling behavior, as expected from SNN near criticality, but the power-laws are broken and the stimuli reconstruction error is higher compared to the error of the original E–I Net model. Thus, on the basis of these experiments, it is not clear how to properly extend the E–I Net model properly with a critical branching mechanism. The E–I Net model has a particular structure where the inhibitory neurons (I) are tuned to decorrelate the excitatory neurons (E) so that the visual features learned matches the angular and frequency distributions of feature detec-tors in visual cortex V1 and different stimuli are represented by sparse subsets of the neurons. The broken power-laws correspond to different scaling behavior at low and high spike rates, which may be related to the efficacy of inhibition in the model.

P

REFACE

This paper is published for the work concerning my M.Sc. thesis in engineering physics and electrical engineering at Lule˚a Tekniska Universitet. The work was done at the Cognitive Mechanics Lab of UC Merced in Merced, CA, between May and August of 2016. The work concerned a spiking neural network, called the E–I Net, and the stud-ies/implementation of critical branching and self-organized criticality. The goal of the thesis was to search for a method to make the spiking neural network more efficient. First I want to thank my supervisors, Prof. Fredrik Sandin, of LTU SRT, and Prof. Christopher T. Kello, of UCM Cogmech, for supervising and supplying me with tools and their expertise in the field. Second, I acknowledge support from the Swedish Foundation for International Cooperation in Research and Higher education (STINT), grant number IG2011- 2025, for the scholarship that I received which enabled this work and the visit to Merced, CA. Last, I want to thank my parents and siblings for supporting me while I was studying in a foreign country.

Oskar ¨Oberg

C

ONTENTS

Chapter 1 – Introduction 1

1.1 Background . . . 1

1.1.1 Spiking neurons . . . 2

1.1.2 Spiking neural networks . . . 3

1.1.3 Criticality and meta-stability . . . 4

1.2 Motivation . . . 5

1.3 Goals . . . 5

1.3.1 Scope . . . 6

Chapter 2 – Methods 7 2.1 Spiking neuron model . . . 7

2.2 E–I Net model . . . 10

2.3 Critical branching . . . 11 2.3.1 Recurrent connections . . . 13 2.4 Self-organized criticality . . . 15 2.4.1 Avalanches . . . 16 2.4.2 Allan factor . . . 18 2.5 Network accuracy . . . 19

2.5.1 RMS error and memory . . . 20

2.5.2 Branching ratio . . . 21

Chapter 3 – Results 23 3.1 Queues of stimuli used as input . . . 24

3.2 Branching ratio . . . 24

3.3 Avalanches . . . 26

3.3.1 Avalanche size in the excitatory populations . . . 27

3.3.2 Avalanche size in the inhibitory populations . . . 29

3.3.3 Avalanche lifetime in the excitatory populations . . . 31

3.3.4 Avalanche lifetimes of the inhibitory populations . . . 33

3.4 Allan factor . . . 35

3.4.1 Allan factor of the excitatory population . . . 35

3.4.2 Allan factor of the inhibitory population . . . 37

3.5 RMS error . . . 39

3.6 Spike ratio . . . 40

3.7 Summary of results . . . 41

C

HAPTER

1

Introduction

As the information transmission of sensory systems increase when monitoring more com-plex environments, it seems natural to research sparse ways to represent this information. This would reduce the power needed from systems, and sophisticated pattern recognition models with high temporal resolution could be implemented. Spiking neural network models have been found to learn sparse representations of stimuli, so it is natural to investigate their properties and limitations. There are several concepts with the intent to improve the accuracy or “memory” of neural network models, depending on the task of the model. If neural networks could learn to represent signals sparsely and sufficiently accurately, this could be a great benefit for wireless sensor systems. As Internet of Things has become a discussed topic, neural networks should be of big interest as it could yield sophisticated monitoring. During this thesis, a sparse coding neural network has been studied and an established method to try to optimize its capacities has been used.

1.1

Background

Currently used digital sensor systems are usually constrained to the sampling rate needed to monitor a system, as higher sampling rates require more power. Systems using wireless devices or pattern recognition systems, for example, usually requires more power than is available. Some models have taken biology as an inspiration and tried to mimic the properties of neurons and cortices, called neural network models, as the sparse and energy efficient way neurons encode information [1] can reach low power consumption as sophis-ticated systems for monitoring. The development of processor units using neural network models, neuromorphic chips, has led to research on efficient methods to teach these chips. With the use of neuromorphic chips, the power consumption of sensory systems might

decrease greatly and the performance might increase as well [2]. There have been studies on learning techniques to use on neural networks so that they can learn to recognize certain signal features in an accurate and sparse way. Some known learning methods use supervised learning, which uses a read-out function to interpret the response from a population of cells and a teacher signal used as input to said population in order to learn the teacher signal’s features. Other learning methods use unsupervised learning, which tunes the network to learn these features on their own. This would be a great achieve-ment for sensory systems if neuromorphic chips could manage to sparsely and effectively learn to recognize features on their own, as some features are hard to distinguish and usually requires time to study. Therefore, developing a sufficient self-regulating learning rule could be a huge benefit for many applications. The activations of a neural network can be seen through the perspective of statistical mechanics, which in theory could yield a critical point where the memory of information in the fluctuations, or the dynamic correlation, is at its highest [3]. The critical point discussed in this thesis however, is not the same critical point as the one when referring to equilibrium thermodynamics, as it is a point far from equilibrium. This critical point is said to act more like an attractor, which the system self-organizes towards [4]. If the fluctuations are indeed at a critical point through self-organization is highly debated and why these models self-organize to criticality is not known. Models that self-organize to criticality have shown similar char-acteristics like power-law distributed activity [5] and avalanches [4], which have been used to identify self-organizing dynamics. Critical branching models have been associ-ated with generating power-laws [6] as well as reaching critical branching, which on its own has shown to maximize mean accuracy over time [7]. The critical branching models have been applied to different types of neural network models and have shown promising results.

This thesis has used a homeostatic spiking neural network using leaky-integrate-and-fire (LIF) neurons, called the E–I Net model [8]. This model has been used for unsupervised learning of visual features investigating how a critical branching model would affect the learning and if the recurrence improves the accuracy for structured sequences of stimulus. Methods to analyze criticality are also be investigated in order to determine if the network fluctuates with power-law distributions.

1.1.1

Spiking neurons

Since the discoveries of the brain’s structure of neurons, studies with the purpose to understand and replicate neurons and their structure have grown in number. From simple models of neural hierarchy to rather accurate representations of neurons’ prop-erties [9]. Either using a local field potential representation, mimicking how voltage

1.1. Background 3

potential spreads in cortices, or a spiking neuron representation, mimicking how neu-rons excite each other to spread this potential. There are models with cells that mimic how neurons get excited and transfer currents, called spikes, between each other using ion channels, called synapses. An illustration of this is shown in Fig. 2.1. Studies have shown that there are different classes of synaptic connections between cortical neurons, for example, inhibitory and excitatory and that the distribution of connections between the cells is sparse. At this point, there is no fully accurate representation of these prop-erties, but there have been several suggestions as to how it can be used to solve practical tasks.

1.1.2

Spiking neural networks

The human brain consists of about one hundred billion neurons, coupled together with hundred trillions of synapses; so it is not surprising that describing networks of neurons interaction with each other is a challenging task. Research have to some extent been able to describe how neurons and synapses function and have constructed neural network models that solve practical problems like pattern recognition and making predictions [9]. How synapses distribute this current has been modeled in different ways, depending on the application. There are two main types of synapses, inhibitory and excitatory, which inhibits or excites the affected neurons’ activations respectively [9]. These properties have shown to be of great significance, as the excitatory synapses pass on “information” in the form of voltage and inhibitory constrains coupled neurons probability to activate. How synapse strengths change over time has been studied and has led to increased under-standing of the process, but as natural neural systems contain millions if not billions of synapses and neurons it is hard to distinguish the underlying dynamics. The timings of neuron activations have however shown to play a role in the change [9]. From this knowl-edge, the spike timing dependent plasticity (STDP) model was developed, a model where the current transferred by synapses increase with the temporal correlation of spikes [10]. The amount transferred, implemented as a weight [9], increases when a spike transferred from a presynaptic neuron causes its coupled postsynaptic neuron to spike as a response. If the timings are reversed it decreases the weights instead. This results in neurons acti-vating with high probability when another neuron with temporal correlation has spiked, and low probability if they are uncorrelated. STDP based models have been used to learn patterns and make predictions when presented with spatio-temporal inputs, recordings of conversations or real-time videos as an example. As spiking neural networks are being applied to neuromorphic hardware [11], they might soon be used for actual/real sensory systems instead of CPUs. Thus this thesis has focused on studying general methods to make spiking neural networks more effective for sensory purposes.

1.1.3

Criticality and meta-stability

What computational principles, if any, characterize an efficient spiking neural network? What characterizes a spiking neural network with optimal information processing prop-erties? These are currently open questions and there have been suggestions, but any general methods have yet to be accepted. Studies of neurons in cortices indicate that these networks may operate near a critical point where the dynamic range, information transmission, and information capacity are optimal [12, 13]. In equilibrium statistical mechanics, the critical point is a state with unique properties. For example, it is a phase-transition state where water can coexist with ice and steam, and where the correlation length of the interactions of particles is maximum. This critical point is usually reached by tuning parameters to some particular values, depending on the system; for example, the temperature and pressure of water. The Ising model is a textbook example in statis-tical mechanics with a cristatis-tical point and associated two-phase transition that appears at a specific temperature. The two phases are typically called the disordered phase, where the orientations of the particles are random, and the ordered phase where the orienta-tion of adjacent particles tend to be the same and non-varying. At the phase transiorienta-tion, these two phases coexist and regions of particles with aligned orientations appear and disappear across sizes of all scales of the system. This is an example of a meta-stable state, where the fluctuation of the particles’ orientations affects a maximum number of neighboring particles leading to maximum correlation length of the interacting particles. It has also been argued that at this state the “memory capacity” of the system is at max-imum because the system is effectively susceptible to influences. This is when the system is subject to external stimulation and the particles change their orientations even if the stimulation is relatively small, and if the stimulus is relatively big the orientation of the whole system does not align. With this property, the information of the fluctuations does not fade away and each fluctuation makes the most change relative to its “significance”. Some systems have been found to self-organize their configuration to a critical point, which has led to the theory of self-organized criticality. The Abelian sand pile model [4] has been a way to demonstrate this property. The Abelian sand pile model is an exper-iment where when new grains of sand are sprinkled on a sand pile, then the “unstable” grains slide down, relieving force applied to themselves but increasing the force on their neighbors. If the neighbors are “unstable” or the force strong enough it might cause neighbors to slide with it, creating a chain reaction; this phenomenon is usually referred to as an avalanche and will be called so henceforth. When plotting the observed avalanche sizes of the model versus their probabilities on a log-log scale, the density distribution shows a linear slope. This phenomenon is called a power-law distribution. Power-law distributed avalanches have been found in several known systems that self-organize to criticality, but only showing power-law distributed avalanches is not enough for telling if a system self-organizes to criticality. The sand pile model also displays 1/f noise, claimed to be the effect of a dynamic that self-organize to criticality [4]. This topic is further

1.2. Motivation 5

discussed in Sec. 2.4. It should be stated that the term self-organized criticality refers to something fundamentally different from the critical point found in equilibrium statistical mechanics, as the self-organized critical point is rather an attractor of the dynamical system and is found far from equilibrium.

Methods that regulate the dynamics of neural networks to critical branching have been found to exhibit self-organized criticality. Critical branching is a state where the interac-tions between populainterac-tions of cells have a 1 : 1 ratio. That is, if an input sends a signal to an output layer, the output layer responds with a signal of the same size. As the response is of the same size as the input, the response contains the same amount of information as the input. When applied to neural network models, increased response fidelity and power-law distributed interactions have been observed. This has led to suggestions that critical branching mechanisms can regulate neural networks to criticality [5, 6, 7]. For these reasons, a critical branching mechanism is modeled and applied to a spiking neural network in this thesis.

1.2

Motivation

Neural network models poised near criticality have shown properties like exhibiting opti-mal memory and information processing capabilities. This is due to the models dynamics being influenced by earlier activations for a long time and maximizing the response ac-curacy to inputs [3]. Critical branching neural models are examples of models observed to exhibit properties associated with self-organized criticality, such as increased memory capacity and power-law distributed avalanches [7]. This thesis has implemented a critical branching method based on C. Kello’s model [6] to tune the dynamics of a spiking neural network model, the E–I Net [8], to critical branching. An analytic demonstration of the network model performance is presented together with a discussion of the observations made.

1.3

Goals

• Study critical branching and self-organized criticality for a better understanding of the subject.

• Investigate if the resulting network have properties associated with self-organized criticality.

• Investigate and discuss methods to evaluate the activity of the resulting network. • Investigate how the critical branching mechanism affects the reconstruction

accu-racy of the neural network model.

• Investigate and discuss methods to evaluate the reconstructions.

1.3.1

Scope

These following points are constraints regarding the goals of this thesis: • The thesis course should correspond to twenty weeks of full-time work.

• The project focuses on the E–I Net spiking neural network model with the addition of a critical branching mechanism. No other network models are considered in order to make the project feasible in the twenty-week time-frame.

C

HAPTER

2

Methods

This chapter introduces the methods used in the thesis; it contains an introduction to how networks of neurons are modeled and a description of the selected spiking neural net-work model and its properties. The selected model is extended with a critical branching mechanism, which is explained with its associated properties. Also, it has been noticed that adding feedback to models has resulted in networks gaining a spatio-temporal mem-ory. This is an interesting feature which might be a necessity for self-organized criticality and how this is implemented on the model is explained with a motivation. This thesis investigates mechanisms to regulate an SNN’s dynamics to criticality, thus methods to evaluate the products of criticality and how the performance of the network have been altered are proposed.

2.1

Spiking neuron model

When a neuron spikes a current is sent through its axon, and this current is distributed to the postsynaptic neurons. When a neuron receives a spike, its cell membrane potential increases, depending on the electrical resistance between the neurons. An illustration of this is presented in Fig. 2.1, the figure was collected from earlier work [14]. Here u is the membrane potential of the neuron, θ the membrane potential threshold, z is the neurons spike state, i, j are indices for the post- and presynaptic neurons, respectively, and wij is a weight capturing the plasticity of the synapse between neuron i and j. These variables are explained in more detail in this section.

A common model of the neuron membrane potential is called the leaky integrate-and-fire model, where the membrane potential increases when receiving spikes but decays with

ui(t) uj(t) Wij Axon zj(t) Axon Synapses u_i(t+1) uj(t+1) Wij Synapses Axon zi(t+1) ui > ui <

Figure 2.1: An illustration of a neural network with two neurons at time t and t + 1.

time. The decay is a way to resemble a leakage found in the cell membranes of neurons. The membrane potential of leaky integrate-and-fire is modeled with a simple differential equation, presented with Eq. 2.1,

τm· du

dt = −u(t) + R · I(t). (2.1)

Here u(t) is the membrane potential of a neuron at time t, τm is a leakage constant of the neuron’s membrane potential and R is the resistance between the neurons. I is the current of received spikes from either a presynaptic neuron or input cell. For example, an input cell could be the retina of an eye, a light-sensitive layer that sends impulses to the brain when presented with light. The differential equation in Eq. 2.1 has been approximated for this thesis using an exponential decay for the leakage of membrane potential, proposed by P. King [8]. This is presented with Eq. 2.2,

u(C)_i (t + 1) = ui(t) exp(−η/τ(C)) + X C∗ βC∗→CX j Z_jC∗(t)W_ijC∗→C, (2.2)

where the membrane potential state, u(C)_i , of each cell, i, in the postsynaptic population C is updated each time step t. The indices i and j refer to cells of the postsynaptic population C and the presynaptic population C∗, respectively. The parameter η is the simulation time step length and τ is the membrane potential decay rate of population C. The coupling constant of the synaptic connections, β, is −1 if the presynaptic population is inhibitory and +1 if it is excitatory. ZC∗ _{represents the spike state of each cell in the} presynaptic population at the former time step, it is 1 if the neuron is spiking and 0

2.1. Spiking neuron model 9

if not. WC∗→Cij is a weight matrix resembling the contribution of membrane potential caused by presynaptic spikes. The neuron emits a spike to its postsynaptic neurons when the neuron membrane potential reaches its threshold, expressed as Eq. 2.3,

Zi(t) =

1, if ui(t) ≥ θi,

0, otherwise, (2.3)

and the membrane potential is then reset to its resting potential, according to Eq. 2.4, ui(t) ⇐ ur, iff Zi(t) = 1. (2.4) Here Z(t) is the spike state of one cell, which is 1 if it spikes and 0 otherwise. The neurons membrane resting potential is denoted by ur and the threshold is denoted as θ.

In computational neuroscience, several learning rules have been created with the aim to capture the changes of synapse strengths observed in experiments and to have the network “learn” by using weight matrices that resemble synapse strengths. One well-known example, a modification of Hebb’s rule, is the Hebbian-Oja (HO) learning rule;

∆Wij ∝ αC

∗_→C

· yi· xj− y2i · Wij
, (2.5) where i,j denotes postsynaptic and presynaptic cells, respectively. Wij is the synaptic strength between the cells i and j, yi is the spike rate vector of postsynaptic cells and xj of the presynaptic cells. This weight is then scaled with a learning rate constant, αC∗→C_{, depending on the class of the presynaptic and postsynaptic cell population. When} changing the weight matrix this way, the weights get tuned so neurons that spike together continue to spike together. This satisfies Hebb’s postulate that ”Cells that fire together, wire together” [15], which is a characteristic found in neurons. Another well-known learning rule is F¨oldi´ak’s rule (F), which is a combination of Hebbian learning and anti-Hebbian learning. Anti-anti-Hebbian learning is a rule that counteracts the anti-Hebbian learning, by reducing the synapse weight to cells that spike repeatedly to redundant inputs; this has been used to teach models sparse codes [16]. By combining these two rules the F¨oldi´ak rule, presented with Eq. 2.6, teach models to emphasize on correlation between cells whilst keeping the response to input sparse;

∆Wij ∝ αC∗→C· (yi · xj − hyii · hxji) , (2.6) where hyii and hxji are the mean spike rates of the presynaptic and postsynaptic cell populations, respectively. The Correlation Measuring (CM) learning rule, presented with Eq. 2.7 is an extension of F¨oldi´ak’s rule, introduced by P. King. It is a rule that regulates the synaptic weights between cells to increase if they are correlated and decrease if they are uncorrelated. The rule was modeled for populations of homeostatic spiking neurons

and results in an increased learning rate, ∆Wij ∝ αC

∗_→C

· (yi· xj− hyii · hxji · (1 + Wij)) . (2.7) The E–I Net model, proposed by P. King [8], is a spiking neural network model using the Correlation Measuring rule used to learn features when presented with visual stimuli, similar to ones found in cortical neurons. A description of the E–I Net model is given in Sec. 2.2.

2.2

E–I Net model

The E–I Net is a model of spiking neurons, inspired by the primary visual cortex (V1) [8], which through a simple and elegant structure manage to learn features in visual stimuli without supervision. By regulating the thresholds of each cell, a constant spike rate of the populations is reached; this phenomenon is called a homeostasis and is referred to as such henceforth. How the membrane potential threshold is regulated is presented with Eq. 2.8,

∆θ_j(C)∝ hZ_j(C)i − p(C)_{. (2.8)} Here j denotes the cell number, C denotes the cell’s population, θ is the threshold of the cell, Z is the mean spike rate of the cell and p is a predetermined spike rate constant. With this property, the hierarchy, and the learning rules, a population of excitatory cells learns to respond to features in sparse representations of visual stimulus while keeping low statistical dependency between cells. A representation of the E–I Net’s hierarchy is presented in Fig. 2.2, showing how the populations of neurons are connected and where which learning rule applies. Whitened images are used as input to the excitatory population with weights adjusted by Hebbian-Oja’s learning rule (HO) (Eq. 2.5). When neurons in the excitatory population spike, it propagates to the inhibitory population with weights adjusted by the Correlation Measuring learning rule (CM) (Eq. 2.7). Lastly, when neurons in the inhibitory population spike it propagates back to the population and to the excitatory population, both with weights adjusted with the CM rule. The separation of excitatory and inhibitory cells differs with the commonly used models in reservoir computing, resulting in some interesting properties. By having two separate populations this way, the inhibitory population is used to decorrelate the excitatory cells from responding together, causing the excitatory cells to respond uniquely to features in visual stimuli. By “reverse engineering” the response of the excitatory population, a reconstruction of the stimulus can be produced; a representation of a reconstruction is presented in Fig. 2.2.

2.3. Critical branching 11

I

E

Figure 2.2: A representation of the E–I Net model and how patches of stimuli are presented in the form of whitened natural images, where “E” stands for the excitatory population and “I” for the inhibitory population.

So, considering the E–I Net’s properties, does a critical branching mechanism improve the memory and hence the reconstruction of stimuli? This thesis tests an implementation of a critical branching mechanism on the E–I Net model and evaluates the differences in performance. How this model was designed and how it affects the networks dynamic is further explained in Sec. 2.3.

2.3

Critical branching

Population neural network is a branch of neural networks using a reservoir of excitatory and inhibitory cells recurrently connected to each other, processing input through the spikes resonating within the reservoir. From the reservoir’s propagation, a read-out func-tion is used to decipher the input. The branching ratio (σ) has shown to be an important property of reservoir computing, as it’s a measure of how information propagates within the reservoir. The branching ratio is collected by the relation of “ancestor” spikes and their “descendant” spikes. When a cell spikes, the ratio describes how many cells that spike as a result of that spike. If the branching ratio is σ < 1 the information in the reservoir diminishes, such that for each cell that spikes, fewer will spike as a response.

If σ > 1, the information dissipates, as each cell that spikes causes more cells to spike as a response. When the configuration of the network is tuned right, the branching ratio can reach σ ∼ 1, called critical branching, which is said to be a state where the spikes contain maximal information. This is because a cells response to stimulus is carried by other cells responses for a long time without turning the networks fluctuations into chaos. From these arguments, spiking neural network models have been proposed with mechanisms that tune the models to critical branching. Studies of such spiking neural network models have shown that the memory capacity and the accuracy of the models are maximum when tuned to critical branching [6, 7]. Neural network models with critical branching have had fluctuations with properties associated with self-organized critical-ity, such as power-law distributed neural avalanches and power-law distributed spectral power densities of a populations fluctuations.

One spiking neural network model, formulated by C. Kello [6], uses a mechanism which self-regulates the activity of a reservoir to critical branching by enabling unblamed synap-tic weights and disabling blamed synapsynap-tic weights. The model presented in this thesis includes a similar mechanism, except that instead of enabling or disabling synaptic con-nections it updates the synaptic weight values. This mechanism increments a synaptic weight that is unblamed and decrements one that is blamed for each spike per time step, thereby continuously regulating the branching of spikes according to “relevance”. For each spiking cell, one pre- and one postsynaptic weight are updated. The presynaptic weight with the lowest weight-change has its blame state checked, and if it is blamed the weight is decremented with a small constant. If it is unblamed it gets blamed. The postsynaptic weight can only be updated if the spiking cell is unblamed, and if the weight with the highest amount of change is updated. If it is already blamed, it gets unblamed. An illustration of this mechanism is presented in Fig. 2.3 and algorithms (A) and (B), in Eq. 2.9, defines the computations involved.

(A) Choose the presynaptic weight for which: ∆wk,i = min(∆wk,i), ∆wk,i< 0, then execute the following:If bk = 1 → Wk,i= Wk,i− C,

Set bk = 1.

(B) Choose the postsynaptic weight for which: ∆wi,j = max(∆wi,j), ∆wi,j > 0, then execute the following:If bi = 0 → Wi,j = Wi,j + C,

Set bi = 0.

(2.9)

Here W is the synaptic weight between cells; i, j, k is indices for the spiking cells, the postsynaptic cells and the presynaptic cells respectively; b is the blame state of cells, which is 1 when blamed and 0 when unblamed; and C is a predetermined, small constant. This critical branching mechanism regulates the spike activity between two populations to

2.3. Critical branching 13 k1 k2 k3 i1 j1 j2 j3 For each time step (A) and (B) occurs to each spiking cell (A) (B)

Figure 2.3: An illustration of how the critical branching mechanism functions. In this illustra-tion, the cell i1 spikes at the current time step and a pre- and postsynaptic weight is updated according to (A) and (B). This process occurs for each spike at every time step.

critical branching, and hence the spike rate of the populations. The E–I Net’s threshold regulation is also a mechanism that regulates the spiking activity of the populations, and these two mechanisms do not function as expected when combined. Therefore, the model using the critical branching mechanism includes a constant threshold. This mechanism was implemented in the E–I Net, and an illustration of which connections are affected by this mechanism is presented in Fig. 2.4. The red arrows indicates the connections that are the same as the original E–I Net model and unaffected by the critical branching mechanism. The solid blue arrows indicates the connection altered to the critical branching mechanism, and the striped blue arrow symbolize a connection that has been added in this thesis and is regulated by the critical branching mechanism. This recurrent connection was added because the critical branching mechanism is usually applied to models with recurrent connections [6, 17] and because of curiosity of how it would affect the learning. The same model but without the recurrent connections was simulated to evaluate how this feedback affects the learning process. This topic is furthered discussed in Sec. 2.3.1.

2.3.1

Recurrent connections

Models with recurrent connections, or feedback, have shown to give rise to networks with not only memory of the spatial properties of the stimulus, but also a memory of earlier inputs. By adding feedback to a population, spikes carrying information oscillate in the population some duration after the spikes occurred, giving the models a temporal

I

E

Figure 2.4: An illustration of how the E–I Net model was extended with a critical branching mechanism in this work. The green and red boxes indicate the excitatory and inhibitory popu-lation, respectively. Connections marked with HO refers to Hebbian-Oja’s weight adaption, CM refers to the Correlation Measuring weight adaptation and CB refers to connections affected by the critical branching mechanism.

memory of the input sequence. This is essential for models with the task of predicting the next words in a sentence, for example. In order to make a fair prediction of words about to come, the preceding words play a huge role as they give context to the sentence. Similar analogies can be made for different kinds of stimuli as well, for example a video of a ball being thrown. The trajectory of the ball would be determined by observing the change of position over several preceding frames. There are many neural network models that use feedback, of which liquid state machines [18] and echo state networks [19] are examples. These models have been known to encode inputs that vary with time into spatio-temporal responses. This is a response that is sensitive to the spatial properties of the stimulus, for instance the pixels of an image, and to when the input is presented relative to earlier inputs. However, using feedback has some limitations. If the excitation from the feedback is too high the activation of cells escalates and oscillates, and if there is not enough excitation the temporal information is quickly lost. There are methods to balance the excitation from feedback. For example, by making the recurrent connections sparse [20]; by using a portion of the population’s connections as inhibitory [8] and by keeping the spectral radius of the synaptic weight matrix slightly below unity [21]. This balance of excitation and inhibition in the feedback is similar to the mechanism presented in this work, but this mechanism tunes the balance iteratively and not by predetermining some parameters. In order to evaluate how feedback affects the learning of the model and if it grants a temporal memory, the model was simulated with and without feedback using two kinds of sequences of stimuli. The first sequence has a predetermined order, presented repeatedly during the simulation. The second sequence uses the same stimuli as the first, but for each repetition the order of the stimuli is shuffled. Comparing the results of these simulations, an evaluation can be made of how feedback affects the learning and if it grants a temporal memory.

2.4. Self-organized criticality 15

With these additions, the dynamics of the model need to be analyzed: Firstly if there are any indications of self-organized criticality with methods discussed in Sec. 2.4 and secondly how these additions affect the performance of the model with methods discussed in Sec. 2.5.

2.4

Self-organized criticality

Systems exhibiting self-organizing criticality (SOC) have shown dynamics that act scale– invariant, such as 1/f noise, power-law distributed avalanches and more [4]. Systems not showing these characteristics are therefore not at a critical point. This topic is furthered discussed in Chap. 4. Scale invariance is said to be self-similar in size scales. For example, if a relation of a curve is found in the range of 100–103, that same relation is found in the scale 10−6–106_{. In other words, if you “zoom in” on the curve, the shape of curve} looks the same as if you would “zoom out”. Models tuned to a critical point of a phase transition have shown to exhibit scale–invariant dynamics, for instance the spin-spin correlation function of the two-dimensional Ising model [22]. When the system is not at the critical point the correlation function follows the relation

G(r) ∼ 1

rηexp(−r/ξ). (2.10)

Here r is the distance between particles, η is a critical exponent and ξ is the correlation length. When the system is located at the critical point, ξ goes towards infinity and the correlation function can be written as

G(r) ∼ 1

rη. (2.11)

This is a power-law distribution, which is scale–invariant. When plotted on a log-log graph it has a linear slope of −η, which is called the critical exponent. Close to the critical point, several relations act like power-laws with their unique critical exponents. It is believed that the critical exponents of systems are universal, that is that the exponents describing a certain relation can be found in systems with different dynamical details [23, 24].

Common methods to evaluate if a neural network model is exhibiting SOC dynamics include studying the neural avalanches of the model, the spectral power densities of spike timings, and the study of the Allan factor of the spike timings. Two of these, the study of neural avalanches and the Allan factor, are considered in this thesis to evaluate if the model’s fluctuations indicate SOC. These methods are described in more detail in Sec. 2.4.1 and 2.4.2, respectively.

2.4.1

Avalanches

The term self-organized criticality originates from Per Bak, et al.’s paper [4] where a pile of sand, when sprinkled with new grains of sand, self-organizes its configuration to criticality with avalanches. The size and lifetime of the avalanches were found to follow power-law distributions, indicating that the model is poised close to a critical point. It was found that the lifetime and size of an avalanche followed a relationship resulting in fluctuations with a frequency spectrum following a 1/f power-law relation. From these findings, the authors argued that the self-organizing property of the model gave rise to the 1/f noise. Natural systems with long-term correlations have for a long time been observed to exhibit 1/f noise, but there has been no explanation for this characteristic. Since this discovery, physical systems with fundamental differences have shown power-law distributed event sizes, such as earthquakes, forest fires, etc. This has led to investigations of how neural networks distribute bursts of activity, and power-law distributed neuronal avalanche have been observed in the cerebral cortex of awake monkeys [25] as well as other animals. This has led to suggestions that the brain operates while poised close to a critical point.

How neural avalanches have been collected for this thesis is described below with an example and illustration with Fig. 2.5. The collection of a neural avalanche starts when any amount of cells spike after at least one time step without activity, the avalanche is concluded when a time step comes without any cells spiking. Avalanches are characterized by the number of cells that spiked during the avalanche and the number time steps the avalanche lasted, called its size and lifetime respectively. In Fig. 2.5, four avalanches are presented, where S is their size and T is their lifetime. When the system is close to a critical point, small fluctuations cause avalanches of all sizes, S, with respective lifetimes T . When the probability densities of these quantities are plotted on a log-log graph against their size or lifetime, the relations follows a power-law distribution. The distributions can be expressed as Eq. 2.12 for the size distribution,

D(S) = kSS−β, (2.12)

and Eq. 2.13 for the lifetime distribution,

D(T ) = kTT−κ. (2.13)

Here D(S) is the probability function of observing an avalanche of size S, D(T ) is the probability function of observing an avalanche with lifetime T , kS and kT are constants, β is the critical exponent of avalanche sizes and κ is the critical exponent of avalanche lifetimes. The self-organizing, critical sand pile model in two spatial dimensions produced avalanches with the critical exponents β ≈ 0.98 and κ ≈ 0.42, which are the anticipated

2.4. Self-organized criticality 17 Spik es of n eur ons 1-5 1 2 3 4 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Time S=9 L=4 S=4 L=2 S=7 L=3 S=11 L=4

Figure 2.5: Raster plot of five cells’ activities. If a neuron spikes during a time step, a black frame is presented for that neuron at that time step. If no spike occurred a white frame is presented instead.

values associated with self-organizing criticality [4]. However, these critical exponents on their own are not sufficient to affirm that a system exhibits criticality and additional analyses are required. Physical systems close to the critical point have several relations which can be expressed as power-law distributions, and at the critical point the exponents of these power-laws can be expressed with a mathematical relationship. For the case of neuronal avalanches, it has been shown that the exponents of avalanche size and lifetime are related to the average spike count evolution – called avalanche shape – given the avalanches’ lifetimes [26]. The avalanche shape is expressed as Eq. 2.14,

hSi(T ) = kcT−γ, (2.14)

and the relation between the quantities exponents can be expressed as Eq. 2.15, γ = κ − 1

β − 1, (2.15)

where kcis a constant and γ is the critical exponent of avalanche shapes. This shows that the relation between the avalanche size and duration can be expressed mathematically and results with a third critical exponent, which is a property of criticality and all of which has been found in neuronal data [27, 28]. However, analyzing neural avalanches might give an insight to if the system is poised at a critical point, but is not sufficient to confirm criticality. As 1/f noise is a product of self-organizing criticality, a method to evaluate 1/f noise is necessary. For this, the Allan factor is considered.

2.4.2

Allan factor

The Allan factor [29] was developed to measure the stability of atomic clocks, which had proved to be difficult by using traditional analytical tools as the estimators did not converge. This was found to be the result of the atomic decay behaving like flicker frequency noise, called 1/f noise, in addition to the expected white frequency noise, which led to the noise being divergent. A way to describe the Allan factor is that it reflects the stability of the event variance in a time series by using the variance in number of events when binning the time series. By varying the size of the bin, an estimate of how the time series evolves with time can be made. The Allan factor is expressed with Eq. 2.16,

A(T ) = h(N (T )j− N (T )j+1) 2_i 2 · hN (T )i =

hd(T )2_i

2 · hN (T )i, (2.16) where A is the Allan factor, T is the size of the time bin. N (T ) is the number of events in the time bin j, j is the index of the time bin, d(T ) is the difference of events between two adjacent, non-overlapping time bins and h·i denotes the expectation value. As the Allan factor distinguishes between different power-law noises, it has been used to identify power-law noise in signals. Because of this property and its convenience it has been used in different fields, from signal processing and the measuring of stability in heart rate [30] to studying the fractal properties of spike timings in rat cortex [31, 32]. To affirm that a distribution of spike timings are scaling in time, the Allan factor can be studied. An example with a calculation of the Allan factor is presented with Fig. 2.6. With the time bins given in the example in Fig. 2.6, the Allan factor of the spike trains can be calculated with Eq. 2.16, which gives:

T = 1 →    hd(1)2_{i = 31/15,} hN (1)i = 31/16, A(1) = 0.533, T = 2 →    hd(2)2_{i = 54/7,} hN (2)i = 31/8, A(2) = 0.995, T = 4 →    hd(4)2_{i = 50/3,} hN (4)i = 31/4, A(4) = 1.08, T = 8 →    hd(8)2_{i = 25/1.} hN (8)i = 31/2. A(8) = 0.807. (2.17)

An interpretation of an incrementing power-law distributed Allan factor is that there is relatively low variance for small time steps. When observing one time step after another, there are no significant difference in the number of events, but when observing a wider time frame there are notable differences. This relation has been used as an argument for self-organized criticality, because for small time scales there are small differences in the organization but for larger time scales there is an apparent structure. The interesting values of the Allan factor occur when the slope shows a positive power-law relation,

2.5. Network accuracy 19 Spik es of n eur ons 1-5 1 2 3 4 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Time T=8 T=4 T=2 T=1 13 18 9 4 7 11 6 3 2 2 5 2 3 8 3 3 2 1 0 2 2 0 3 2 2 0 1 2 4 4

Figure 2.6: Raster plot for an example of an Allan factor calculation. If a neuron spikes during a time step, that neuron is represented with a black frame at that time step. If no spike occurred the neuron is represented with a white frame.

A(T ) ∼ Tα_{, with the exponent in the interval 0 < α < 3. Within this interval, the} Allan factor is an indicator of a fractal process, and the exponent is related to the Hurst exponent [32, 31]. The Hurst exponent is used as a method to evaluate long-term memory of signals but is not discussed in this thesis.

Methods to evaluate if the models exhibit properties associated with self-organized crit-icality have been presented in this section. In the coming section methods to evaluate how the models’ performances have been affected are introduced.

2.5

Network accuracy

In Sec. 2.3 methods to improve the memory, and hence performance, of a spiking neural network model were proposed and applied to the E–I Net model. In this section methods to evaluate the performance of the new model are proposed.

Firstly, as the performance of the E–I Net model is measured from the reconstruction error of the model, this method is considered for the extended model. The reconstruction

error is determined by the root mean square (RMS) error between the input patch and the model’s activation to input patches, the same method was used for the extended model. Sec. 2.5.1 describes how the RMS error estimates the response accuracy and how memory is evaluated.

Secondly, the mechanism applied to the E–I Net model regulates the branching ratio between the excitatory and the inhibitory populations, thus the branching ratio is mea-sured in the numerical simulations of the model. This is furthered discussed in Sec. 2.5.2. Lastly, in order to evaluate how feedback, introduced in Sec. 2.3.1, affects learning, the models are evaluated both with and without feedback. By comparing how the models performing, insight can be drawn of how the feedback affected the models. For each evaluation in this section of the extended model, an evaluation of the original E–I Net model is also presented as a reference.

2.5.1

RMS error and memory

The performance of the network can be determined by the root mean squared (RMS) error between the input and the reconstructed patch of the networks fluctuations, where lower RMS error means higher accuracy of the networks reconstruction. This is a measurement of the networks “memory” in the sense of how accurate the cells encoding of information is. The RMS error between the input patch and the reconstructed patch is calculated after the stimulation period but before the input patch is exchanged for a new one. During that period the spikes of the excitatory population are collected and after the stimulation period the spike count for each cell is individually multiplied by the weights between the input and the excitatory cells. This is a way to reconstruct the feature each cell was activated by, which is referred to as the “activation feature”, expressed with Eq. 2.18. The reconstructed signal is calculated by summing the activation features of each cell and then normalizing the sum by its standard deviation, Eq. 2.19. The RMS error is then calculated by the difference between the input signal and the reconstructed signal, expressed in Eq. 2.20,

ri(n) = n·Tstim. X τ =(n−1)·Tstim.+1 Wi,j(τ ) × Zj(τ ), (2.18) rrecon._i (n) = ri(n) std(ri(n)) , (2.19) RM S(n) = v u u t Ninput X i=1 rinput_i (n) − rrecon. i (n)
2 Ninput . (2.20)

2.5. Network accuracy 21

Here ri is the signal, i, j are the indices of input and excitatory cells respectively, n is an integer referring to a specific time period of stimulus, Tstim is the length of the time period and τ is the time step of the simulation. Wi,j is the synaptic weight between input and excitatory cells, Zj is the spike state of the excitatory population, rrecon.i is the reconstructed signal, r_iinput is the input signal and Ninput is the number of input cells. The model that inspired the extension of the E–I Net model is known to improve the classification accuracy of spiking neural networks by regulating the model’s branching ratio [6]. It was found that a spiking neural network had optimal performance when the branching ratio was tuned to unity, which motivated the implementation. What the branching ratio is and how it can be interpreted will be explained further in the next section.

2.5.2

Branching ratio

The branching ratio is a measurement of the networks susceptibility to spikes; that is, how many activations will be caused per spike. The branching ratio of the model in this thesis is measured by the number of spikes in the inhibitory population per spike in the excitatory population. This is expressed with Eq. 2.21,

σ(t) = NI X i=1 ZI,i(t) NE X j=1 ZE,j(t − 1) , (2.21)

where σ(t) is the branching ratio at time step t, ZC(t) is the spike state of cells with class C and NC is the size of population C. In this framework the excitatory population can be called the“ancestor” population, and the inhibitory the “descendant” population. A branching ratio close to unity indicates that for each transmitted spike from a pop-ulation results with a spike in the targeted poppop-ulation, the transmitted “information” has neither increased or decreased but remains the same. This, σ ∼ 1, is called critical branching, a point where the information transmission between populations are maximal which leads to an optimal performance for population network models [7]. The branch-ing ratio was in this thesis collected by first low-pass filterbranch-ing the spike counts of each population, and then dividing the value of the inhibitory with the excitatory population. The evaluations proposed in this chapter were gathered with numerical experiments and are presented in Chap. 3, followed with a discussion of the results in Chap. 4.

C

HAPTER

3

Results

This chapter presents the results from numerical experiments performed to investigate the ideas and questions raised in Sec. 1.3. It contains a measurement of the network models’ branching ratio, two methods to evaluate characteristics of self-organized criticality and a measurement of stimuli reconstruction accuracy. Results of the E–I Net model and the extended model are presented, both with and without excitatory feedback. Additional analyses of the models’ behavior were made, but as no conclusions were drawn from these were they disregarded in this paper. The stimuli used as input for the models have been collected from whitened images of natural scenes [33] and presented with patches in two manners: The first with a predetermined order of patches presented repeatedly and the second with a randomized order in the sequence of patches. Each model used cell dynamics presented with Eq. 2.2, Eq. 2.3 and Eq. 2.8, and with a structure either as Fig. 2.2 or Fig. 2.4. The values of ∆Wi,j used for the critical branching mechanism in this thesis were calculated using the CM rule. The extended model does not use the threshold regulation as the original E–I Net model does, but uses a static value for each cell potential threshold. The population sizes of the models are 64 excitatory cells and 9 inhibitory cells; they were constrained to small populations to decrease computation time. Results were gathered from simulations of 2.50 · 108 _{time steps, with 2.50 · 10}5 sequences of 10 image patches as stimuli.

Descriptions of key elements of the simulations are presented in this chapter with exam-ples and figures. An illustration of how input sequences were collected is presented with Fig. 3.1 and a description of how each result was gathered is explained with an example before each result. To summarize the collected results, Tab. 3.1 compares the results from the simulations.

3.1

Queues of stimuli used as input

Figure 3.1: Illustration of how the two kinds of queues used as stimuli in this work are generated from whitened natural images [33].

The stimuli for the simulations were presented with queues of patches of 10 × 10 pixels, taken from whitened natural images [33]. Ten patches were collected at random from ten images of 512 × 512 pixels in one of two ways. Either with the queue of patches repeated with a predetermined order or randomly ordered for each queue. An illustration of how these queues were collected is shown in Fig. 3.1.

3.2

Branching ratio

The branching ratio between “ancestor” and “descendant” spikes were collected from simulations of the models, each model simulated for 2.50 · 108 time steps with stimuli of patches drawn from whitened images of natural scenes [33]. The branching ratio is the ratio between “ancestor” and “descendant” spikes, where the excitatory spikes are defined as “ancestor” spikes and inhibitory spikes as “descendant” spikes. The stimuli for the network model were presented in two ways, described in Sec. 3.1. The results from the predetermined ordered queue are presented in Fig. 3.2a,c and Fig. 3.3a,c and the results from the queue with randomized order are presented in Fig. 3.2b,d and Fig. 3.3b,d.

3.2. Branching ratio 25

t

b) d)

a) c)

Figure 3.2: The branching ratio of the E–I Net model’s activity. a,c) With an ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

t

b) d)

a) c)

Figure 3.3: The branching ratio of the extended model’s activity. a,c) With an ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

Fig. 3.2a,b and Fig. 3.3a,b show the branching ratios of the models without excitatory feedback. Fig. 3.2c,d and Fig. 3.3c,d show the branching ratios of the models with

excitatory feedback. The average branching ratio of the models are presented in Tab. 3.1. The branching ratios of the models have the same trend for the ordered queue as the one with the randomly ordered queue of stimuli, except that the ratios of randomly ordered queues are not as steady.

3.3

Avalanches

In the coming sections the distribution of avalanche sizes and lifetimes for the excitatory and inhibitory populations of the models are presented. For each power-law distribution the exponent of the slope of the distribution is presented in Tab. 3.1 and discussed in Chap. 4. The avalanches were collected from numerical experiments of the models. Each was simulated for 2.50 · 108 _{time steps with stimuli of patches taken from whitened} images of natural scenes [33]. Each simulation of the models used stimuli in one of two ways, the first with a predetermined ordered queue that repeats presented in Fig. 3.4a,c and Fig. 3.7a,c, and the second with the order of the queue randomized presented in Fig. 3.4b,d and Fig. 3.7b,d. Fig. 3.4a,b, Fig. 3.5a,b, Fig. 3.6a,b and Fig. 3.7a,b present the distributions of avalanche sizes of the models without excitatory feedback, and Fig. 3.4c,d, Fig. 3.5c,d, Fig. 3.6c,d and Fig. 3.7c,d present the distributions of avalanche sizes of the models with excitatory feedback. The sizes of the avalanches were restricted to the size of the population, as studies have shown that the distribution of avalanche sizes suffer sub-sampling if the avalanche size is larger than the populations size [34]. The size distribution of the avalanches is constrained to the size of the population and thus only the lifetimes of these avalanches are collected.

The avalanche lifetime distributions are presented below. Fig. 3.8a,b, Fig. 3.9a,b, Fig. 3.10a,b and Fig. 3.11a,b present the distributions of avalanche lifetimes of the models without excitatory feedback, and Fig. 3.8c,d, Fig. 3.9c,d, Fig. 3.10c,d and Fig. 3.11c,d present the distributions of avalanche lifetimes of the models with excitatory feedback.

3.3. Avalanches 27

3.3.1

Avalanche size in the excitatory populations

S

D(S) b) d)

a) c)

Figure 3.4: The avalanche size distributions of the E–I Net model’s excitatory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback. The dashed red line is calculated with the least squares method (LSM) over the whole interval and the solid lines over a divided interval, separated where the mean square error (MSE) is the lowest. c) The MSE for the dashed line is 3.6 · 10−2 and 1.8 · 10−3 for the solid lines. d) The MSE for the dashed line is 1.6 · 10−1 and 2.0 · 10−3 for the solid lines. As the MSE is remarkably lower when dividing the interval, this is presented for these cases.

S

D(S) b) d)

a) c)

Figure 3.5: The avalanche size distributions of the extended model’s excitatory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback. The red lines are calculated with the least squares method (LSM) over a divided interval, separated where the mean square error (MSE) is the lowest.

3.3. Avalanches 29

3.3.2

Avalanche size in the inhibitory populations

S

D(S) b) d)

a) c)

Figure 3.6: The avalanche size distributions of the E–I Net model’s inhibitory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with exci-tatory feedback. No regression was made for these, as the inhibitory population only contains nine cells.

S

D(S) b) d)

a) c)

Figure 3.7: The avalanches size distributions of the extended model’s inhibitory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback, and c,d) the model with exci-tatory feedback. No regression was made for these, as the inhibitory population only contains nine cells.

3.3. Avalanches 31

3.3.3

Avalanche lifetime in the excitatory populations

T

D(T) b) d)

a) c)

Figure 3.8: The avalanche lifetime distributions of the E–I Net model’s excitatory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback. The orange lines are the least squares method (LSM) over the whole interval. The mean square error for a) and b) are significant, with the values 2.6 and 1.66, respectively. This is likely due to the low values for the small time windows.

T

D(T) b) d)

a) c)

Figure 3.9: The avalanche lifetime distributions of the extended model’s excitatory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback, and c,d) the model with excitatory feedback. The orange lines are the least squares method (LSM) over the whole interval. The mean square error for c) is significant, with a value of 10.

3.3. Avalanches 33

3.3.4

Avalanche lifetimes of the inhibitory populations

T

D(T) b) d)

a) c)

Figure 3.10: The avalanche durations of the E–I Net model’s inhibitory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback. No regression model was fitted to this data because the inhibitory population only contains nine cells, resulting in too few data points.

T

D(T) b) d)

a) c)

Figure 3.11: The avalanche durations of the extended model’s inhibitory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback. No regression was made for these, as the inhibitory population only contains nine cells and was deemed insufficent.

3.4. Allan factor 35

3.4

Allan factor

In the coming sections the Allan factor of the spike trains in the excitatory and inhibitory populations are presented respectively. The Allan factor was calculated from the spikes within a time window of the last ten sequences, that is the last 104 _{time steps. For the} E–I Net model with excitatory feedback, however, the Allan factor was calculated from a time window of ten sequences slightly before the activations turn to chaos, shown in Fig. 3.18. Each simulation of the models used stimuli in one of two ways, the first with a predetermined ordered queue that repeats presented in Fig. 3.12a,c and Fig. 3.15a,c, and the second with the order of the queue randomized presented in Fig. 3.12b,d and Fig. 3.15b,d. Fig. 3.12a,b, Fig. 3.13a,b, Fig. 3.14a,b and Fig. 3.15a,b present the Allan fac-tor of the models without excitafac-tory feedback and Fig. 3.12c,d, Fig. 3.13c,d, Fig. 3.14c,d and Fig. 3.15c,d present the Allan factor of the models with excitatory feedback.

3.4.1

Allan factor of the excitatory population

T

A(T) b) d)

a) c)

Figure 3.12: The Allan factor of the E–I Net model’s excitatory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

T

A(T) b) d)

a) c)

Figure 3.13: The Allan factor of the extended model’s excitatory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

3.4. Allan factor 37

3.4.2

Allan factor of the inhibitory population

T

A(T) b) d)

a) c)

Figure 3.14: The Allan factor of the E–I Net model’s inhibitory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

T

A(T) b) d)

a) c)

Figure 3.15: The Allan factor of the extended model’s inhibitory population’s activity. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

3.5. RMS error 39

3.5

RMS error

The RMS error of the models reconstruction was collected when simulating the models for 2.50 · 108 _{time steps with patches from whitened images of natural scenes as} stim-uli [33]. Each simulation of the models used stimstim-uli in one of two ways, the first with a predetermined ordered queue that repeats, presented in Fig. 3.16a,c and Fig. 3.17a,c, and the second with the order of the queue randomized, presented in Fig. 3.16b,d and Fig. 3.17b,d. The average RMS error of each simulation is presented in Tab. 3.1. Fig. 3.16a,b and Fig. 3.17a,b present the RMS error of the models without excitatory feedback and Fig. 3.16c,d and Fig. 3.17c,d present the RMS error of the models with excitatory feedback.

When adding excitatory feedback to the E–I Net model the reconstruction error exhibits a drastic change after a short period of the simulation due to over-excitation that results from redundant responses. The activations are counteracted by the E–I Net model’s threshold regulation, resulting in the dissipation of activations. The spike rate of the E–I Net is presented in Fig. 3.18 and is used as a reference for this behavior.

b) d)

a) c)

Figure 3.16: The RMS error of the E–I Net model’s stimuli patch reconstruction. The red line is the moving mean error and the gray shaded area covers the moving mean error ± the standard deviation. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

b) d)

a) c)

Figure 3.17: The RMS error of the extended model’s stimuli patch reconstruction. The red line is the moving mean error and the gray shaded area covers the moving mean error ± the standard deviation. a,c) With a predetermined ordered sequence of stimuli. b,d) With a randomly ordered sequence of stimuli. a,b) The model without excitatory feedback and c,d) The model with excitatory feedback.

3.6

Spike ratio

Time

Spike ratio

Time

Figure 3.18: Spike ratio from a simulation of two models. To the left is the E–I Net model and to the right is the E–I Net model with excitatory feedback.

3.7. Summary of results 41

For reference, the moving mean spike rate from the simulations of the E–I Net and E–I Net with excitatory feedback are presented in Fig. 3.18. Theses spike rates were collected when simulating the models with predetermined queues of stimuli as input.

The model without excitatory feedback reaches homeostasis quickly, whereas the model with feedback reaches its desired spike rate but gets unstable shortly after that and dis-sipates to zero with occasional activity. The reconstruction error measured in this thesis becomes extremely large when the activity dissipates, as no information is transmitted as output. It should be stated that the learning rules, as well as threshold regulation used for these models, are dependent on the spike rates of the cells. Therefore the occurring instability when introducing excitatory feedback might remove any “ learned” behavior.

3.7

Summary of results

In this section the obtained results are presented in Tab. 3.1, summarizing the results. Each result has been collected from simulations of 2.50 · 108 _{time steps except for the} “Branching ratio,” which is the average branching ratio during the last 1.00 · 108 time steps of the simulations, and the Allan factor, which has been collected during the last 104 _{time steps. The “Lowest RMS error” refers to the lowest value of the moving mean} reconstruction RMS error.

In the coming chapter, the work in this thesis is discussed and then compiled with a con-clusion. As additional insight on the subject was gained during this work, recommended studies for further work are presented after the conclusions.

XX XX XX XX XX XX Model Stimuli

Ordered sequences of stimuli Disordered sequences of stimuli a) E–I Net No feedback With feedback No feedback With feedback

Branching ratio 0.26 0.77 0.26 0.98 β (Excitatory) N C [1.5, −4.3] N C [0.13, −4.3] β (Inhibitory) N C N C N C N C κ (Excitatory) −7.6 −3.7 −6.1 −3.2 κ (Inhibitory) −13 −6.4 −12 −6.9 Lowest RMS error 0.05 0.11 0.05 0.05

b) CB model No feedback With feedback No feedback With feedback

Branching ratio 1.2 0.88 1.1 1.0 β (Excitatory) [0.50, −4.3] [0.26, −3.8] [0.78, −3.9] [0.16, −3.5] β (Inhibitory) N C N C N C N C κ (Excitatory) −3.9 −4.2 −3.8 −3.5 κ (Inhibitory) −7.8 −4.7 −7.8 −4.2 Lowest RMS error 0.26 0.24 0.21 0.22

Table 3.1: Table with the gathered results for a) the E–I Net model and b) the extended model. For each model a comparison of the model with feedback is presented, and for each result, two types of stimuli have been used, one with an ordered queue and one randomly ordered queue of stimuli. The Branching ratio refers to the spike relation between the inhibitory and excitatory populations, β is the power-law slope of the avalanche sizes, κ is the power-law slope of the avalanche durations and Lowest RMS error refers the lowest point of the moving-average reconstruction RMS error. NC stands for Not Clear, i.e. that the exponent value was not clear hence not extracted from that analysis.