Implementation and Simulation Study of Methods for the Evolution of Interdependent Networks

IT 17 032

Examensarbete 30 hp Juli 2017

Implementation and Simulation

Study of Methods for the Evolution of Interdependent Networks

Ismail Elouafiq

Institutionen för informationsteknologi

Department of Information Technology

Teknisk- naturvetenskaplig fakultet UTH-enheten

Besöksadress:

Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0

Postadress:

Box 536 751 21 Uppsala

Telefon:

018 – 471 30 03

Telefax:

018 – 471 30 00

Hemsida:

http://www.teknat.uu.se/student

Abstract

Implementation and Simulation Study of Methods for the Evolution of Interdependent Networks

Ismail Elouafiq

The purpose of this work is to study different methods for the evolution of interdependent networks. Different systems such as social networks, protein interactions or transportation systems can be represented using multilayer

networks, which can provide a simple and unified means of expression. In this work, we start by selecting a representation of multilayer networks and outlining the pre- existing methods to simulate their evolution. We then propose a model of multilayer network formation that considers target measure for the network to be generated and focuses on the case of finite multiplex networks. Thus, before defining the model we propose different measures and properties that should enable us to differentiate between a generated network and a real life counterpart. We simulate network formation following the previous method and show in which cases we get closer to our target measures.

Tryckt av: Reprocentralen ITC IT 17 032

Examinator: Mats Daniels

Ämnesgranskare: Michael Ashcroft Handledare: Matteo Magnani

v

“I was born not knowing and have had only a little time to change that here and there.”

Richard P. Feynman

“Reading, after a certain age, diverts the mind too much from its creative pursuits. Any man who reads too much and uses his own brain too little falls into lazy habits of thinking.”

Albert Einstein

Acknowledgements

Well, first of all, thank you. This work only does become a bit valuable when you read it. So thank you for reading this, thank you for giving it value, and time.

Today, I am able to ride a bike. One month ago, however, I could not.

Let me tell you a funny story. I was in Sweden, it was summer, the sun was up, nothing seemingly special nor exciting about that day. Until Fadi and Shady, two friends of mine and some of my favorite humans on this planet, asked me to join them by bike. There was only one issue: I did not know how to bike. I was about to finish this thesis, I was twenty four years old and I did not know how to bike. So, of course, after forcing me to try it out, making fun of my reactions and watching me struggle with what should be a basic skill, especially in a city like Uppsala, I finally realized how amazing it was to ride on two wheels. How can I ever claim that I learned to bike on my own? How can I ever claim that I could’ve started biking on my own?

After all, I had spent a whole year in a city full of bikes and never gave it a thought.

So, thank you Fadi, thank you Shady and thank you beloved reader for going through this all while wondering "how is this even related to your thesis in any way Ismail?". If I cannot claim that I can bike because I learned it, how can I ever claim that this work you’re reading is in any way my own?

It is true that I spent a reasonable amount of time to get results. It is true that I read lots of research articles before finding one that I could make use of, only to realize later that I need to make a U-turn and change directions.

It is also true that I loved doing this, that I enjoyed it. However, it is not true that this work is entirely self-made.

We all depend on people who support us during the good days, and during the other ones too. This work was built on a foundation of mentors, teachers, parents; of kind souls who offered smiles and knowledge; of fun human beings with whom I shared not just lunch, but also wisdom and advice; of love from people who are alive, people who are not; of love from pets and imaginary friends. Without these people I wouldn’t have heard of Uppsala, without them I wouldn’t know what the words Fika or Lagom meant. So how can I ever claim to have done this on my own? How can I ever claim that I had learned or achieved anything on my own? I cannot.

And odds are, you cannot either. We all need fuel from others.

So let me start with a big thank you to all the fun human beings in the

department that all went on a different journey; thank you Sobhan, Thành

and Minpeng for the fun conversations during lunch. To Victor, my friend

and companion in both networks and rock climbing, thank you for sharing

ideas and chocolate, thank you for making the whole experience fun. To

my mentor Matteo who showed me not only the technical basics but also

the importance of doing something you love, thank you for the guidance

and for teaching me the joy of finding things out. Thank you Michael for

reviewing my thesis, thank you for showing me the importance of listening

and paying attention to detail, after all, how we do the little things is how

we do everything. My beloved family, my parents, my brother Ali and

sister Sara, thank you for just being who you are. And thank you again,

reader, for being generous with your time and attention.

xi

Contents

Acknowledgements

1 Introduction

1

1.1 Network Formation . . . . 2

1.2 Goals . . . . 2

1.3 State of the Art . . . . 3

2 Preliminaries

5 2.1 Single Layer Networks . . . . 5

2.2 Multilayer Networks . . . . 6

2.3 Graph Properties . . . . 7

2.3.1 Intralayer properties . . . . 7

2.3.2 Interlayer Properties . . . . 8

Property Matrix Representation . . . . 8

Measures . . . . 9

3 Generating Single Layer Networks

11 3.1 Single Layer Network Properties . . . . 11

3.2 Formation Models . . . . 12

4 Multilayer Network Formation

15 4.1 Characteristics of Multilayer Networks . . . . 15

4.2 Existing Models for Multilayer Networks . . . . 17

5 Proposal - The Adjusting Model

19 5.1 Goals . . . . 19

5.2 Empirical Data Exploration . . . . 19

5.2.1 Datasets . . . . 20

5.2.2 Observations on Empirical Data . . . . 20

Node Jaccard . . . . 20

Edge Jaccard . . . . 25

Triangle Jaccard . . . . 29

Interlayer Assortativity . . . . 29

Choice of measures . . . . 33

5.3 Adjusting Model Framework . . . . 33

5.3.1 Description . . . . 33

5.3.2 Basic Operations . . . . 33

5.3.3 Basic Operations Characteristics . . . . 35

Back consistency . . . . 35

Completeness . . . . 35

xii

Soundness . . . . 36

Non Redundancy . . . . 37

5.3.4 Derived Operations . . . . 37

5.3.5 Problem Reformulation . . . . 40

Reassignment operations R . . . . 40

Balancing operations B . . . . 40

Layer edit operations L . . . . 41

5.3.6 Effects of the Operations on the Target Measures . . . 41

5.4 Adjusting Copy Model . . . . 43

5.4.1 Description . . . . 43

5.4.2 Full Copy Adjusting Algorithm . . . . 44

Edge Jaccard and Node Jaccard as targets . . . . 44

Interlayer Assortativity as a target . . . . 45

5.5 Other Possible Approaches . . . . 47

6 Results and Discussion

49 6.1 Node Jaccard . . . . 49

6.2 Edge Jaccard . . . . 50

6.3 Interlayer Assortativity . . . . 52

6.4 Error Refinement . . . . 54

7 Conclusion and Future Work

57

Bibliography

61

xiii

List of Figures

2.1 Graph Rep Example . . . . 6

2.2 Multilayerpres Example . . . . 6

4.1 JSD Example . . . . 16

5.1 Swap Example . . . . 21

5.2 Swap Example . . . . 21

5.3 Swap Example . . . . 22

5.4 Swap Example . . . . 22

5.5 Swap Example . . . . 23

5.6 Swap Example . . . . 23

5.7 Swap Example . . . . 24

5.8 Swap Example . . . . 24

5.9 Swap Example . . . . 25

5.10 Swap Example . . . . 26

5.11 Swap Example . . . . 26

5.12 Swap Example . . . . 27

5.13 Swap Example . . . . 27

5.14 Higgstriangle Example . . . . 29

5.15 Noordintriangle Example . . . . 30

5.16 Noordintriangle Example . . . . 30

5.17 Noordintriangle Example . . . . 31

5.18 Noordintriangle Example . . . . 31

5.19 Noordintriangle Example . . . . 32

5.20 Noordintriangle Example . . . . 32

5.21 Noordintriangle Example . . . . 33

5.22 Swap Example . . . . 38

5.23 Copy Example . . . . 38

5.24 Edge Example . . . . 39

5.25 State Machine . . . . 41

5.26 allops Example . . . . 47

6.1 Swap Example . . . . 50

6.2 Swap Example . . . . 51

6.3 Swap Example . . . . 51

6.4 Swap Example . . . . 51

6.5 Swap Example . . . . 53

6.6 Swap Example . . . . 53

6.7 Swap Example . . . . 53

6.8 Swap Example . . . . 54

6.9 Swap Example . . . . 54

6.10 Swap Example . . . . 55

6.11 Swap Example . . . . 55

6.12 Swap Example . . . . 55

xiv

7.1 allops Example . . . . 58

xv

List of Tables

2.1 Property matrix where the network structure is the actors and each value corresponds to whether an actor is present on a layer . . . . 8 2.2 Property matrix that can be used for computing the node jac-

card: (the network structure is the actors and each value cor- responds to whether an actor is present on a layer) . . . . 9 3.1 Summary of the Single Layer Network Formation Model Prop-

erties . . . . 12 5.1 Effect of Elementary Operations on The Interlayer Properties. 43 5.2 Example of degrees of selected actors, an actor swap in this

case will decrease the assortativity . . . . 46

7.1 Node Jaccard Measure Between Three Layers . . . . 58

xvii

List of Abbreviations

BA Barabási Albert EA Edge Attachment NG Network Growth

SNA Social Network Analysis

JSD Jensen Shannon Divergence

AC Adjusting Copy

1

Chapter 1

Introduction

“The first follower is what transforms a lone nut into a leader.”

Derek Sivers

“It took thirty-eight years before 50 million people gained access to radios. It took television thirteen years to earn an audience that size. It took Instagram a year and a half.”

Gary Vaynerchuk Around four years ago, Evan Spiegel, a Stanford University student at the time, and his friends Bobby Murphy and Reggie Brown had the idea of making a phone app with disappearing photos, which is now known as Snapchat (Xu et al., 2016). What was an idea back then is now used by over 100 million users worldwide. Online social networks like Snapchat, Facebook or Twitter are of growing popularity (Jin, 2013). But social net- works are not the only phenomenon that can be represented by networks.

In fact, we are surrounded by complex systems, from the activity of neurons in our brain to the routing of communications between multiple platforms (Musella, 2015). The study of these systems has induced the emergence of network science.

Multilayer networks intend to generalize the study of even more complex situations. These can include interactions between different transportation networks (Gallotti, 2015), protein-protein interactions (Shinde, 2014) or dy- namics on multiple social networks (Boccaletti, 2014). Consider for exam- ple a group of people present in the same time on Facebook and Instagram.

Each network of connections on Facebook and Instagram taken seperately could be represented by a network. But other characteristics might arise if the relationships between these seperate networks are considered. For instance, it could be the case that someone who joins Instagram would be more likely to follow the people he was previously connected with on Face- book. While these systems can be studied seperately in each different field, multilayer networks can provide a simple and unified way to express these different systems (Dickison, 2016).

Representing complex phenomena as multilayer networks can enable us

to measure and, hence, get a deeper understanding of the underlying dy-

namics and characteristics. For example, in the case of single layer net-

works, by introducing the principle of cumulative advantage (Price, 1976),

Price revealed an underlying phenomenon in networks known as preferen-

tial attachment (Barabási, 1999). Moreover, we might need to test different

algorithms that operate on different types of networks to conclude which

strategy would best suit certain networks or we might need to decide which

network structure to favour. Thus, we should be able to generate networks

2 Chapter 1. Introduction

that exhibit properties that are representative of what happens in the real world.

1.1 Network Formation

An understanding of how networks operate and how they form is impor- tant to get a grasp of the dynamics within a network, both on a higher and a lower level (Jackson, 2003). Models for network formation are also essential to enable the generation of networks similar to networks in the real world.

But what do we mean by similar to a network in real life? An important step before is to define which characteristics and properties networks tend to exhibit. So that we can decide which of these properties we are trying to achieve when we want to generate a network. These properties might range from the size, degree to degree correlations, or the clustering coeffi- cient that characterizes how the nodes in a network gather together. Gener- ating a network with some target properties in mind will not only help us get closer to a network in real life, but also achieve desired properties for the structure and size that we might aim for depending on the application.

1.2 Goals

Our main focus will be on multiplex networks. These are multilayer net- works where the edges exist only inside the same layer. Hence nodes that are in different layers will not be attached by an edge. This could for exam- ple be a multilayer social network where connections do not exist between one social network and another.

With that in mind, we want to define a network formation model that will enable us to generate a multilayer network that has certain target prop- erties. In the scope of this work, we limit the number of layers to two.

Hence, we should be able to generate a network with one or two layers where: each single layer can exhibit single layer network properties when taken apart (i.e.: if we only take one of the layers of the network, the sin- gle layer network that corresponds to it has the properties of a single layer network. We specify these properties later in Chapter 5) and the multilayer network as a whole should achieve some desired target properties.

This means that we will first need to define which properties we want our model to achieve. To do so, we will start by investigating existing prop- erties in the state of the art and explore empirical data to have an idea of which features to consider.

The proposed model, which we named the Adjusting Model, provides

a general framework that includes previous work on network formation

and enables generating different types of networks by applying a set of

operations on a pre-existing network. We then define the Adjusting Copy

Model, where we apply the Adjusting Model framework by generating a

single layer network first, then copying the layer to get a second layer with

matching characteristics, and then adjusting the second layer until we get

our desired target properties. We will mainly prioritize two propertiesthat

our model will be expected to achieve, these are: the node jaccard and the

1.3. State of the Art 3

edge jaccard, both of which will be defined later in the preliminaries (Chap- ter 2). In the same chapter, we define the interlayer assortativity for which we will give an example of how it can be approached as a target, we then discuss how more properties can be considered later.

1.3 State of the Art

In many real life networks the degree distribution seems to be heavy-tailed which usually indicates the presence of compelling underlying processes (Mitzenmacher, 2004). This power-law degree distribution was pointed out by Price when he introduced the mechanism of cumulative advan- tage in networks (Price, 1976). In his work on citations, Price presents his model based on the assumption that new papers cite randomly selected pre-existing papers chosen by a probability proportional to the latter’s num- ber of citations. The resulting network shows that a process, that will later get known as preferential attachment, might be responsible for the power- law degree distribution. The process of preferential attachment was later introduced by Barabási and Albert (Barabási, 1999). In the Barabási-Albert (BA) model, each new node connects to a number of pre-existing nodes with a probability proportional to their current degree, which is undirected in this case. The BA model is an example of a Network Growth model.

In Network Growth models, nodes are added or deleted as well as edges.

Other examples of this include the model introduced by Toivonen et al. (Toivo- nen, 2006), the Growing Network by Copying model (Krapivsky, 2005) and the Kronecker graph model (Leskovec, 2010). The Toivonen model also uses preferential attachment but in an implicit way where nodes are attached to achieve a certain average. The Copying model on the other hand is a model for directed networks where each newly created node selects a node at ran- dom and links to it as well as to all of its ancestor nodes. In the Kronecker graph, the main idea is to use the Kronecker product matrix operation on the adjacency matrix representation of a network graph (Leskovec, 2010).

Besides Network Growth models, there are also Edge Attachment (EA) models. In EA models, the graph size is fixed at the beginning and the nodes are connected to achieve a desired state. A standard example of EA models is the Erdös-Rènyi (ER) model where we simply define a probabil- ity p for a pair of nodes to be connected (Erd˝os, 1959). If for example this probability is equal to 1, we will obtain a fully connected graph. Another example of EA models is the Configuration model where instead of defining the probability of nodes to be connected, we give a desired degree distribu- tion (Kang, 2008).

In order to investigate the networks generated by the latter formation

models we must check how well the generated networks compare to real

networks. Some of the properties that are found to characterize real world

networks are: the presence of a giant component, the scale-free property,

the clustering coefficient and assortativity (Musella, 2015). The giant com-

ponent in a graph (as defined in chapter 2) is basically a connected com-

ponent (part of the graph where all nodes are connected by a set of edges)

containing a constant fraction of the graph (Bradonji, 2007). The scale-free

property refers to the fact that, in a number of networks, the probability dis-

tribution of the number of connections of nodes over the network (degree

4 Chapter 1. Introduction

distribution) follows a power-law degree distribution (Musella, 2015, Price, 1976, this will be detailed later in Chapter 3). On the other hand the cluster- ing coefficient and assortativity will be defined in Chapter 2. The clustering coefficient gives insights on how nodes of a network that are connected to the same node are more likely to get connected together. While the assor- tativity can provide insights on the structure of the networks. For example, social networks are often seen to be assortative, which means that vertices of similar degree tend to be connected, and hence people who are popular (i.e.: have a relatively high number of connections) will have a tendency to connect to similarly popular people .

Unlike single layer network formation models, described above, where we can expect certain properties in the resulting networks, characteristics that multilayer networks should achieve are not as clear (Dickison, 2016).

Although we can expect some properties to arise in the case of multiple layers, based on our understanding of real life phenomena, more accurate and measurable properties are yet to be found.

By considering a single layer network formation model, we can general- ize to the case of multiple layers. However, we need to consider the fact that new features will arise. As an example, (Lee, 2012) established a formation model for a network with two layers by considering interlayer correlations and combining them with an ER model. The interlayer correlations in the Lee, 2012 model show the important impact of the correlations between the two layers on the other people of the mechanisms of spreading in networks (Dickison, 2016). Wile the latter is similar to an EA model, other models are similar to NG models. For example, (Magnani and Rossi, 2013) established a model where the link overlap was taken into account, given that the over- lap is not expected to be random in social networks. Another multilayer NG model was the one introduced by (Podobnik, 2012) where preferential attachment is taken to a two layer case including the interlayer correlations.

In Chapter 2, we cover the preliminary notions and concepts that are necessary for the rest of the document, we also give a mathematical repre- sentation of the multilayer networks as defined in Magnani and Br, 2016.

Then in Chapter 3 we give an overview of network formation models for

single layer networks by presenting their main properties followed by the

models of formation in the single layer network case. This is extended in

Chapter 4 to the case of networks with multiple layers. In Chapter 5 we

propose our formation model, the Adjusting Model, that enables generat-

ing networks with some selected target properties, in defining our model

we considered the case of a multiplex networks with two layers. The re-

sults of this model are presented and discussed later in Chapter 6, where

we also investigate the case of more than two layers and how we could

target other properties. Then we give a brief conclusion to summarize this

work in Chapter 7.

5

Chapter 2

Preliminaries

In this section we are going to define the notions necessary for the rest of the documents as well as some will enable us to formulate properties and proofs in a unified language.

Graph theory provides a uniform terminology to approach the study of networks in mathematical terms (Barrat, 2012). However, defining a uni- form representation for multilayer networks, that have been studied in sep- arate and independent fields, remains a challenge. Yet, significant develop- ment in the social network analysis (SNA) field can simplify the represen- tation of a multilayer network. The node is usually seperated from the real world entity it represents, referred to as the actor. The terminology we used takes this into account and was introduced in Dickison, 2016.

2.1 Single Layer Networks

Generally, single layer networks are represented by graphs. A graph repre- sents a set of elements (nodes) and the relationships between them (edges).

Definition 2.1.1 An undirected graph G is a tuple (V, E) where V is the set of vertices (nodes) and E is the set of edges i.e.: E = {{n, m}/n ∈ V, m ∈ V } (E is the set of unordered pairs of elements of V )

Definition 2.1.2 Given a graph G = (V, E). An adjacency matrix A is a square matrix with a length equal to the cardinality of V and where an element is equal to 1 if an edge exists and 0 if the edge does not exist, i.e:

• A = (a

)

where N = |V | and:

– a

= 1 if the edge {i, j} is in E – a

= 0 Otherwise.

Example 2.1.2.1 Figure 2.1 shows an example representation of a real life network as a graph as well as the corresponding adjacency matrix representation.

Definition 2.1.3 A directed graph G is a tuple (V, E) where V is the set of ver- tices (nodes) and E is the set of edges i.e.: E = {((n, m)/n ∈ V, m ∈ V } (E is the set of ordered pairs of elements of V )

Note that we will not consider the directed case in the rest of the document

and we will refer to unordered pairs using the same notation we used for

ordered pairs, i.e.: given two elements u and v, a pair (u, v) can be consid-

ered the same as (v, u).

6 Chapter 2. Preliminaries

FIGURE 2.1: Network representation as a graph with the corresponding adjacency matrix.

FIGURE2.2: Multilayer network representation of a real life network (Note that the network is multiplex in this case, the lines between layers are only there to indicate the same

actor).

2.2 Multilayer Networks

Definition 2.2.1 A multilayer network M can be defined as a quadruple (A, L, V, E) where A is a set of actors, L a set of layers, V a set of nodes and E a set of edges such that:

(V, E) is a graph and V ⊆ A × L.

We can hence introduce the following definitions:

Definition 2.2.2 • An actor a refers to the real world entity represented by nodes.

• A layer l is a part of a multilayer network that groups a set of nodes and edges together. Taken seperatly, one layer of a multilayer network can repre- sent a single layer network.

• A node (or vertex) n is a representation of an actor on a certain layer, for- mally a node is an ordered pair where the first element is an acotr and the second is a layer.

• An edge (or connection) e is the relationship between two nodes of a net- work. Formally, an edge is a pair of two nodes (Note that as stated earlier we will only consider the undirected case and hence we do not take the order of the pair into account).

Example 2.2.2.1 Figure 2.2 shows a multilayer network M where the actors are

labeled as a1, a2, a3, a4, a5, a6 the layers as l1, l2. In this case, the actor a1, that

2.3. Graph Properties 7

represents alice, is present on both layers, hence both nodes (a1, l1) and (a1, l2) are in the set of nodes. On the other hand, the actor a6, that corresponds to oscar, is only present on layer l2 and hence only (a6, l2) exists, while (a6, l1) is not among the nodes of the network.

Definition 2.2.3 Let M=(A, L, V, E) be a multilayer network. We say that M is a multiplex network if there are no edges between nodes in different layers. i.e:

∀(a, a

) ∈ A

and ∀(l, l

) ∈ L

(if ((a, l), (a

, l

)) ∈ E then l = l

)

When defining our model we will only consider the case of a finite multi- plex network. Hence, in the rest of this document, we will interchangeably refer to such a network as network or multiplex network unless specified otherwise. We denote by M the set of finite multiplex networks.

Definition 2.2.4 • We will refer to the properties of a network that only con- cern one single layer of the network as intralayer properties.

• We will refer to the properties of a network that indicate the relationship between at least two of its layers as interlayer properties.

2.3 Graph Properties

2.3.1 Intralayer properties

Let M = (A, L, V, E) ∈ M be a multilayer network.

Definition 2.3.1 The degree of a node n is the number of edges from n to any other node, the latter is said to be a neighbor of n.

More specifically, the degree deg is a function from the set of nodes N to the set N, such that deg(n) (also referred to in this document as d

) is the number of neighbors of n.

If the number of neighbors of a node n is q, then in a setting where all these neighbors are connected to each other the number of edges between them would be

which would be the maximal number of possible edges between the neighbors of n. From this we can deduce the definition of the clustering coefficient of a node.

Definition 2.3.2 Given a node n that has q neighbors, and where p is the actual number of neighbors of n that are connected to each other, the clustering coeffi- cient of the node n is defined as the ratio between p and

, i.e.:

2

Definition 2.3.3 The clustering coefficient of a layer l with N

nodes is equal to the average clustering coefficients of all the nodes. i.e.:

C(l) =

P_Nl

i=1

N_l

Note that this is the clustering coefficient of the single layer network de- fined by the layer l separately.

Now, we are going to define the assortativity for a layer l. We call this

the intralayer assortativity. We only focus on the assortativity regarding

the degrees for the purpose of this document. To get an intuition about

8 Chapter 2. Preliminaries

how the assortativity behaves, it represents how nodes of similar degrees have a tendency to connect to each other. For example, social networks are usually assortative, this means that the assortativity value is relatively high and this is due to the fact that popular nodes tend to connect to other similarly popular nodes.

For the degree assortativity we are going to use the pearson correlation coefficient ρ which is defined for two random variables X, Y as follows:

ρ(X, Y ) = cov(X, Y )

σ

σ

(2.1)

Now, given that X and Y are two random variables that represent the de- grees of the nodes corresponding to an edge selected at random, the pear- son correlation coefficient between X and Y gives the intralayer assortativ- ity, which can be defined as follows:

Definition 2.3.4 The intralayer assortativity a

of a layer l with N

nodes is equal to :

a

(l) = P

i,j=1

(deg(i) − µ

)(deg(j) − µ

) P

i=1

(deg(i) − µ

)

(2.2) 2.3.2 Interlayer Properties

Before we proceed with the main measures used to quantify the relation- ships between different layers. We are going to define the property matrix as introduced by (Magnani and Br (2016)). The property matrix represen- tation will provide us with a uniform way of quantifying the similarity be- tween different layers of a multilayer network.

Property Matrix Representation

Definition 2.3.5 Given a multilayer network M = (A, L, V, E), the property matrix p is defined by:

• Columns that indicate the chosen network structure (actor, node, edge, con- nected triple, etc.) we will compare to quantify the similarity between layers.

• Rows that indicate for each layer the value observed for every corresponding network structure.

a

a

a

a

a

a

l

1 1 1 1 1 0

l

1 1 0 1 0 1

TABLE2.1: Property matrix where the network structure is the actors and each value corresponds to whether an actor

is present on a layer

Example 2.3.5.1 Given the previous network on figure 2.2 If we want to compare

the existence of nodes for each actor on different layers. Table 2.1 shows the corre-

sponding property matrix.

2.3. Graph Properties 9

Measures

Given the property matrix where each network structure is an actor (which we call the actor degree property matrix), and the value is the correspond- ing degree on each layer, we can define how similar the degrees of actors are from a layer to another using the pearson correlation coefficient defined in equation 2.1.

Definition 2.3.6 The interlayer assortativity a

between two layers l and l

of a network M is the pearson correlation of the corresponding rows on the actor degree property matrix, i.e.:

a

=

P

i=1

(d

− µ)(d

− µ

) q

P

i=1

(d

− µ)

q

P

i=1

(d

−

)

(2.3)

Where N

is the total number of actors, µ and µ

are the means of degrees on layers l and l

respectively, while d

and d

are the degrees of actor i on l and l

respectively.

Now we will introduce measures that will enable us to compare how nodes, edges and triangles are shared between different layers. The mea- sure we are going to use for that is the jaccard measure j which is defined for two sets A and B as:

j(A, B) = |A ∩ B|

|A ∪ B| (2.4)

And for multiple sets:

j(A

, ..., A

) = | ∩

A

|

| ∪

A

| (2.5)

We will refer to the property matrix where the values are 1 if the chosen structure exists on a layer and 0 otherwise as existence property matrix.

Definition 2.3.7 Given a multilayer network M and its corresponding node ex- istence property matrix. The node jaccard j

between a set of layers is defined as the fraction of the of scalar product of the rows, corresponding to the set of lay- ers, divided by the number of total actors present in at least one layer (sum of the elements of the bitwise OR of the matrix rows). I.e.:

j

= number of shared nodes

number of all nodes (2.6)

If the property matrix is like the one in table 2.2, then the node jaccard is the a

a

a

a

a

a

l 0 1 0 1 1 0

l

1 1 0 1 0 1

TABLE2.2: Property matrix that can be used for computing the node jaccard: (the network structure is the actors and each value corresponds to whether an actor is present on a

layer)

fraction of the scalar product of the two rows (equal to 2 in this case), divided by

10 Chapter 2. Preliminaries

the sum of the elements of the bitwise OR of the two rows (the vector of the bitwise OR between the two rows is in this case: (1, 1, 0, 1, 1, 1), which means that the sum of its elements is 5). Hence the node jaccard will be 2/5 in this case.

Definition 2.3.8 Given a multilayer network M and its corresponding edge ex- istence property matrix. The edge jaccard j

between a set of layers is defined as the fraction of the of scalar product of the rows, corresponding to the set of lay- ers, divided by the number of total actors present in at least one layer (sum of the elements of the bitwise OR of the matrix rows). I.e.:

j

= number of shared edges

number of all edges (2.7)

Definition 2.3.9 Given a multilayer network M and its corresponding triangle existence property matrix. The triangle jaccard j

between a set of layers is de- fined as the fraction of the of scalar product of the rows, corresponding to the set of layers, divided by the number of total actors present in at least one layer (sum of the elements of the bitwise OR of the matrix rows). I.e.:

j

= number of shared triangles

number of all triangles (2.8)

Note: the jaccard measures are only defined if the number of elements of the union is not null.

Example 2.3.9.1 For the network on figure 2.2.

• The interlayer assortativity is 0

• The node jaccard is 0.5

• The edge jaccard is

• The triangle jaccard is 0

11

Chapter 3

Generating Single Layer Networks

In this section, we investigate the algorithms used to generate a single layer network. We will start by outlining some of the main properties that these network structures exhibit in real life. Then we will describe some of the many commonly used models.

3.1 Single Layer Network Properties

Research on social networks has resulted in a deeper understanding of the underlying properties and structures of real life social networks. The main characteristics of real life social networks can be observed on the degree dis- tribution, the clustering coefficient, the degree to degree correlations (assor- tativity for example) and the small world property. The small world prop- erty simply refers to the fact that the average shortest path connecting the nodes is usually low.

Structural organization in real life networks results in the presence of significant degree correlations (Serrano, 2007). For example, social net- works are assortative (have a relatively high assortativity) while other types of networks (e.g.: biological networks) are dissortative (have a low negative assortativity value). Thus, different empirical network structures exhibit different properties. Let us analyze two examples:

Social Networks

Milgram’s small world experiment (Milgram, 1963) introduced the 6 de-

grees of separation in social networks by showing that the mean length of

a path in such a network is around 6.5. While it is true that the individu-

als were not randomly chosen (which adds bias to the experiment) and the

paths found were not necessarily the shortest, more recent studies (Dodds,

2003) have shown that the small world property is omnipresent in social

networks. Another interesting concept is the fact that individuals who are

connected to the same person (or have more mutual connections) will be

more likely to get to know each other. This transitivity is observed in real

life social networks by a significantly high clustering coefficient (Clauset,

2013). Moreover, the distribution of degrees and can usually be approxi-

mated by a power-law degree distribution (Barabási, 1999). To get a better

sense of this, we can interpret it in the following way: there are very few

popular members in a social networks (members with a high number of

connections, and consequently a high degree) while less popular members

are more present on the network.

12 Chapter 3. Generating Single Layer Networks

Protein Interactions Networks

The network of protein interactions that represents the proteome (proteins in a cell and the interactions between them) can be used in biological appli- cations to represent simple cells of which we have a better understanding, such as the bacteria E. Coli (Serrano, 2007). Some of the main observations on these networks are the fact that they are scale-free, the clustering coef- ficient is relatively high compared to a random graph (i.e.: a graph gener- ated using the ER model described below in section 3.2. Although generally lower than a social network), and the network is mainly dissortative (with a relatively low assortativity) (M. Á. Serrano, 2007).

3.2 Formation Models

Edge Assignment Network Growth

Model E.R. Con-

figuration B.A. Toivonen Copy-

ing Kronecker Degree

Distribution Poisson Arbit- rary

Power- law

Power- law

Poisson (out) Power- law (in)

Multi- nomial Clustering

Coefficient low low low high high high

Small World

Property no yes yes yes yes yes

Degree to Degree Correlations

no no no assor-

tative no no

TABLE3.1: Summary of the Single Layer Network Forma- tion Model Properties

Let us consider the case of a random graph. In other words, a graph that is generated by having nodes that are already present and connecting them with a probability p. We can easily observe that the resulting graph does not correspond to a real world network like the ones described above. For instance, such a random graph will consequently have an assortativity close to 0. Such an approach is taken in the Erdös-Renyi (ER) model (Erd˝os, 1959) where the number of nodes and the probability of attachment are given as parameters to generate a network. Such a model can be used to generate random graphs and derive relevant properties by comparing them to other graphs. This is an example of an edge attachment model. On the other hand, some models such as the Barabasi-Albert (BA) model (Barabási, 1999) start with an empty network and adds nodes and edges step by step. This is an example of a network growth model. Table 3.1 shows the different properties of the networks generated by some of these models. We give a description of both types (edge attachment and network growth) below:

Edge Attachment (EA) models

In edge attachment models, the nodes are all created beforehand, and then

edges are created to connect the latter nodes. The ER model is an example

3.2. Formation Models 13

of an edge attachment model. Another example of an EA model is the Con- figuration model (Kang, 2008). To generate a network, the Configuration model requires the total number of nodes and the degree distribution of the entire network. The network is then constructed by selecting random degrees from the chosen distribution. Note that, if random sampling is not possible, the degree sequence can be drawn from empirical data. The edges are then positioned using a matching construction in such a way that the degrees are handled in order so that the network can be written with one single pass over the selected degrees. This means that we can obtain the scale-free property observed in empirical networks. However, the networks generated by the Configuration model have a low clustering coefficient and an assortativity close to 0.

Network Growth (NG) models

The BA model (Barabási, 1999) is one of the most widely used models to generate a network. It makes use of the preferential attachment, that was investigated in Price, 1976, by considering that new nodes are more likely to get attached to nodes that have a high degree. Hence the BA model operates by first creating a fully connected graph of a given initial number of nodes. At each step a new node is added and attached to the present nodes with a probability proportional to their degrees. Given that N is the number of nodes already present, the probability p

to get attached to i is:

p

= d

P

j=1

d

(3.1)

The resulting degree distribution is a power-law distribution which is rep- resentative of the heavy-tailed property in empirical network datasets. An important aspect of NG models where nodes are added recursively is that these can indicate the underlying properties of a growing network such as the preferential attachment as an example.

The Network Growth by Copying model (Krapivsky, 2005) is also an NG model where a node is added at each step. In the Copying model the newly added node selects a node to connect to, then links to all of its connections (hence the name copying). Note that this generates a directed graph, intu- itively this can be compared to a citation network where a if a paper is cited by another, citations present in this paper are likely to be cited as well.

Another example of an network growth model is the one introduced by Toivonen (Toivonen, 2006). As opposed to the BA model and as shown in table 3.1, the networks generated by the Toivonen model are assortative.

Instead of connecting nodes on a probability proportional to their degrees, the new nodes are connected to a number of randomly selected nodes (with a predefined average) and then connected to a number of their neighbors (with a predefined average).

The Kronecker (Leskovec, 2010) model on the other hand makes use of re- cursively applying the Kronecker product on the adjacency matrix of an initial graph to generate a network. The Kronecker product of two matri- ces A = (a

)

and B = (b

)

is defined as the block matrix of length mp × nq equal to (a

B)

.

While some generated networks can have a low clustering coefficient or

a low assortativity, they can be made more assortative or more transitive

by performing edge-swaps. For instance, Xulvi-Brunet and Sokolov, 2004,

14 Chapter 3. Generating Single Layer Networks

shows a way to increase assortativity in the case of a network generated

using the BA model by reconnecting two pairs of randomly chosen nodes

such that the higher degree nodes are connected to those of a lower degree

with a probability p.

15

Chapter 4

Multilayer Network Formation

Before trying to define models that will generate a multilayer network, we first need to attempt to establish characteristics of these networks in the real world that we might want to achieve. While these characteristics have long been studied in the case of a single layer network, the properties that a multilayer network should achieve are not as clear (Kivelä, 2014).

4.1 Characteristics of Multilayer Networks

Unlike the case of a single layer network, there are no properties that accu- rately characterize networks with multiple layers. On one side, each layer of a multilayer network is, independently, a single layer network and hence exhibits the different single layer properties. However, different layers of a real life multilayer network are usually correlated. Consider as an example two people who are already connected on multiple social platforms. If we consider a new platform where those same people are present, it is possi- ble these people can be more likely to be connected there as well (but not necessarily). We present some measure that can give an insight into the dif- ferent aspects of measuring similarities between layers and getting hold of properties for a multilayer network.

Shared and Overlapping Network Structures One simple way of inves-

tigating the similarity between two layers would be considering how many

nodes are shared between these layers. This can be done using the jaccard

measure as defined in the preliminaries section where we divide the num-

ber of shared nodes by the number of all nodes. A high node jaccard would

mean that the corresponding set of layers shares most of its nodes which

can be expected from a social network. If for example we take the network

of friends and coworkers for the same group of people and the members

are present on both networks, then the node jaccard will be maximal (equal

to 1). Besides, when this measure is low, very few nodes are shared be-

tween the layers. This can be the case in networks that do not share nodes,

and where the node jaccard will be equal to 0. For example, if we take an

airport network and its relationship with a network constituted of people,

each layer will have a different kind of nodes, and consequently the node

jaccard measure will be equal to 0. The same thing can be said for edges

and triangles of a network. As we will see later, in Chapter 5, these proper-

ties show significant patterns in empirical networks compared to random

networks. Coverage and overlap are also measures that can be taken into

account. Overlap is simply the number of shared nodes, edges or triangles.

16 Chapter 4. Multilayer Network Formation

FIGURE 4.1: Jensen-Shannon Divergence heatmap of a protein-genetic interaction network with corresponding re- ducibility dendrogram (obtained by hierarchical clustering)

and entropy q (Domenico,2015).

Coverage is similar to the jaccard measures, but this time the number of shared elements (nodes, edges or triangles) is divided by the number of el- ements present on one of the networks. This means that the coverage gives the percentage of nodes of a layer l that are shared with a layer l

.

Entropy While investigating structural reducibility, Domenico, 2015 shows that Jensen-Shannon divergence (JSD) between different layers shows sig- nificant patterns in empirical data (Figure 4.1). If we have two discrete vari- ables X and Y , the mutual information of X and Y is defined as:

I(X; Y ) = X

y∈Y

X

x∈X

p(x, y)log( p(x, y)

p(x)p(y) ) (4.1)

where p(X) and p(Y ) are the marginal probability distribution functions of X and Y respectively while p(X, Y ) is the joint probability distribution function of X and Y . The JSD between two probability distributions can be interpreted as the mutual information between the random variable used to produce the mixture distribution and the random variable of the mixture distribution itself. In other words we can interpret the JSD between two layers as being opposite to how much information is shared between the layers (That is why this measure is null between a layer and itself).

Assortativity As we will see later, in Chapter 5, computing the inter- layer assortativity between different layers can reveal that some layers in empirical networks can be assortative while others can be dissortative. If two layers are assortative (the value computed for the assortativity is high), then the actors will tend to have similar degrees on both layers. On the other hand, if the two layers are dissortative (the assortativity value is low) then the actors that have a high degree on one layer will be more likely to have a low degree on the other and vice versa.

Clustering Coefficient Since both the concept of neighborhood itself be-

comes more ambiguous in the case of a multilayer network and can be de-

fined in multiple ways (Kivelä, 2014), the clustering coefficient is not as easy

to define as it is in the case of a single layer. If, for example, we want to de-

fine it by using the fraction of connected triples on triangles, we need to

define what a triangle is and what connected triples are exactly. The reason

is that we can, for instance, consider an edge from one layer to another as

part of a triangle. Despite this difficulty, there has been significant effort

4.2. Existing Models for Multilayer Networks 17

and development in investigating the clustering coefficient in the case of multiple layers (Bródka, 2010, Bródka, 2012).

4.2 Existing Models for Multilayer Networks

One approach to defining a model of formation for multilayer networks is generalizing some of the single layer network models. Such an approach can for example be applied to the EA models to generate a multilayer net- work. The advantage of such an approach is that EA models do not operate by a step by step node addition. Lee et al. (Lee, 2012) presented such mod- els to generate a network with two-layers, in their work one can observe that the giant component appears faster but grows slower when a positive cross-layer degree correlation is considered.

However, generalizing an NG model to more layers is not as obvious due to

node insertion. Other questions arise such as whether to add an actor that

will be present on only one layer, add a node for an actor that is already

present, connect actors on only one or multiple layers etc. With that being

said, there have been several proposals in this case. The model proposed in

Podobnik, 2012, generalizes preferential attachment to a two-layer case by

considering a probability of attachment that depends on the overall degree

on the two layers. Magnani and Rossi, 2013, present a model where action

on one layer can induce action on another while there is also a possibility

for no action to be taken at a certain step. While other models assimilate ex-

plicit overlap of the edges, such as the one proposed by Bianconi (Bianconi,

2013) which makes use of both microcanonical and canonical ensembles for

the case of a multiplex network.

19

Chapter 5

Proposal - The Adjusting Model

“What gets measured, gets managed.”

Peter Drucker

“It is vain to do with more what can be done with less.”

William of Occam In this chapter we suggest a new model for the generation of multiplex networks which we will refer to as the Adjusting Model.

The general idea behind the model is the following:

• We define a set of target properties that we want our generated net- work to have in the end.

• With the previous properties defined, we start from a network M and try to adjust M until we achieve the desired properties.

Hence, we will first outline the main goals and assumptions necessary to define our model.

5.1 Goals

As mentioned before, we will focus on finite multiplex networks as defined in the preliminaries (Chapter 2). To define the properties that we need our model to achieve we are going to start by exploring empirical multilayer network data. We will then compare its results with some randomly gener- ated multilayer networks that will be described in section 5.2.

The model should be valid for the case of a single layer network.

We will start by defining a model for generating a two layer network. We will try to generalize to the case of more layers in the last chapter (Chap- ter 7). This generalization might result in more issues. For example, if we have a set of target properties between three layers that we need to achieve and start by generating two of these layers, once we add one more layer we need to consider the properties between this layer and all the layers that were previously in the network.

5.2 Empirical Data Exploration

In this section we are going to investigate the characteristics that networks

have in real life so that we can decide which properties we want to aim for

when generating a network.

20 Chapter 5. Proposal - The Adjusting Model

5.2.1 Datasets

We will operate on the following datasets (more information on the datasets is available on the Appendices A):

• Real Life Networks:

– Higgs: The Higgs Twitter Dataset for the discussions on Twitter about the discovery of a particle with the features of a Higgs boson, the dataset contains three layers. One for retweets, one for replies and one for mentions.

– TwYtFf: A three layer network with layers from Twitter, Friend- feed and Youtube.

– Celegans:

– Arxiv:

– Noordin:

– FAO:

– transp_Airlines:

– Padgett:

– Wiring:

• Generated Networks:

– ER random: Two layers generated independently using the ER model, then the actors are matched uniformly at random.

– BA random: Two layers generated independently using the BA model, then the actors are matched uniformly at random.

– ER copy: One layer is generated using the ER model and the second is a copy of the first.

– BA copy: One layer is generated using the BA model and the second is a copy of the first.

5.2.2 Observations on Empirical Data

Node Jaccard

Node Jaccard gives an idea of how nodes are shared between every two lay- ers. Note that the node jaccard is symmetric. In the case of the generated datasets (ER random, BA random, ER copy and BA copy) this measure will always be equal to 1 since every node on each layer is matched with a node on another layer. Let us see how this measure behaves in the real world.

Figures from 5.1 to 5.8 show that depending on how the layers can be cor-

related, this measure can range from 0 to 1, depending on how the actors

are present on the multiple layers. For example, on the Padgett dataset 5.5

since most people are present on both layers (marriage alliances and busi-

ness relationships) the value of the node Jaccard is high. On the other hand,

for the Celegans dataset, the node jaccard is very low.

5.2. Empirical Data Exploration 21

FIGURE5.1: Node Jaccard for the Arabidopsis dataset, the xaxis and y axis indicate the layers of the network.

FIGURE 5.2: Node Jaccard for the Celegans dataset, the x axis and y axis indicate the layers of the network.

22 Chapter 5. Proposal - The Adjusting Model

FIGURE5.3: Node Jaccard for the FAO dataset, the x axis and y axis indicate the layers of the network.

FIGURE5.4: Node Jaccard for the Higgs dataset, the x axis and y axis indicate the layers of the network.

5.2. Empirical Data Exploration 23

FIGURE5.5: Node Jaccard for the Padgett dataset, the x axis and y axis indicate the layers of the network.

FIGURE5.6: Node Jaccard for the TwYtFf dataset, the x axis and y axis indicate the layers of the network.

24 Chapter 5. Proposal - The Adjusting Model

FIGURE5.7: Node Jaccard for the Wiring dataset, the x axis and y axis indicate the layers of the network.

FIGURE5.8: Node Jaccard for the transp_Airlines dataset, the x axis and y axis indicate the layers of the network.

5.2. Empirical Data Exploration 25

FIGURE 5.9: Edge Jaccard for the Celegans dataset, the x axis and y axis indicate the layers of the network.

Edge Jaccard

The Edge Jaccard is shown on figures from 5.9 to

generally be lower than the node jaccard. This is due to the fact that: for nodes to have a shared edge, the nodes need to be shared themselves. We can observe if we take the Higgs dataset for example that on Figure 5.4 the layers for which the node jaccard is the lowest (between layer 2 and the other layers), the edge jaccard is almost equal to 0 as shown in Figure 5.11.

The same goes for the TwYtFf dataset.

For the ER copy and BA copy datasets, the edge jaccard is equal to 1

since the layers are exactly the same. But on the ER random and BA ran-

dom datasets we find that the edge jaccard is close to 0 (0.004 and 0.001

respectively for ER random and BA random).

26 Chapter 5. Proposal - The Adjusting Model

FIGURE5.10: Edge Jaccard for the FAO dataset, the x axis and y axis indicate the layers of the network.

FIGURE5.11: Edge Jaccard for the Higgs dataset, the x axis and y axis indicate the layers of the network.

5.2. Empirical Data Exploration 27

FIGURE 5.12: Edge Jaccard for the Noordin dataset, the x axis and y axis indicate the layers of the network.

FIGURE5.13: Edge Jaccard for the TwYtFf dataset, the x axis and y axis indicate the layers of the network.

28 Chapter 5. Proposal - The Adjusting Model

Example]Edge Jaccard for the Wiring dataset, the x axis and y axis indicate

the layers of the network.

5.2. Empirical Data Exploration 29

FIGURE 5.14: Triangle Jaccard for the Higgs dataset, the x axis and y axis indicate the layers of the network.

Triangle Jaccard

The Triangle Jaccard is shown to be smaller than the Edge Jaccard in figures from 5.14 to 5.16 when compared with figures from 5.9 to

that, given two layers, the groups formed on each layer tend to differ from one layer to another. For the BA random dataset, due to a low clustering coefficient, there are no triangles and hence the value given by simulation is Nan. On the other hand the value returned by the ER random dataset is 0, which means that there is at least a triangle but none of the triangles are shared between the layers. For the triangles to be shared their corre- sponding edges need to be within the edges that are shared, the fact that the edge jaccard is close to 0 makes the number of these edges too small.

Hence, when the edge jaccard is close to 0, given that there are very few triangles on each single layer itself, the fact that the two layers are matched randomly can explain the fact that the triangle jaccard is null. However, the event where two triangles would be shared, although highly unlikely, might have a greater impact on the triangle jaccard (For instance, if we only have one triangle on each layer, and these two happen to be shared, the tri- angle jaccard will be equal to 1). However, in real world networks we can see that some of the datasets do show the presence of a non null triangle jaccard in figures 5.14 to 5.16.

Interlayer Assortativity

The ER random and BA random are close to 0 (a value of -0.01 for the BA

random and a value of 0.04 for the ER random). However, real life networks

can show both dissortative and assortative layers. In figure 5.18 for example

we can see that some of the layers of the FAO dataset are assortative while

others are dissortative. While on the social network datasets 5.20 and 5.21

we observe that the layers are mostly assortative and hence people who are

popular on a certain layer are mostly also popular on the other.

30 Chapter 5. Proposal - The Adjusting Model

FIGURE5.15: Triangle Jaccard for the Noordin dataset, the xaxis and y axis indicate the layers of the network.

FIGURE5.16: Triangle Jaccard for the TwYtFf dataset, the x axis and y axis indicate the layers of the network.

5.2. Empirical Data Exploration 31

FIGURE 5.17: Interlayer Assortativity for the Celegans dataset, the x axis and y axis indicate the layers of the net-

work.

FIGURE 5.18: Interlayer Assortativity for the FAO dataset, the x axis and y axis indicate the layers of the network.

32 Chapter 5. Proposal - The Adjusting Model

FIGURE 5.19: Interlayer Assortativity for the Noordin dataset, the x axis and y axis indicate the layers of the net-

work.

FIGURE 5.20: Interlayer Assortativity for the TwYtFf dataset, the x axis and y axis indicate the layers of the net-

work.

5.3. Adjusting Model Framework 33

FIGURE 5.21: Interlayer Assortativity for the Padgett dataset, the x axis and y axis indicate the layers of the net-

work.

Choice of measures

We can see that the measures chosen here can enable us to make the dif- ference between a network that has been generated randomly and real life networks. Hence, for simplicity reasons we will focus on the node jaccard, the edge jaccard and the interlayer assortativity in the rest of the document.

5.3 Adjusting Model Framework

In this section we define the general framework of the Adjusting Model.

5.3.1 Description

Given a set of target properties, a set of possible operations and a starting multiplex network M , an Adjusting Model will be a process that will apply some of the operations on the network M to achieve the target properties.

We will define a set of operations in section 5.3.2 and 5.3.4.

Notes:

• The starting network M can be empty.

• The generated networks should verify the goals listed in Section 5.1.

Particularly, these should be valid in the case of a single layer net- work.

5.3.2 Basic Operations

We are going to define a set of operations on multiplex networks. These operations should enable us to transform any multilayer network to any other multilayer network. We later prove that the defined operations are complete, sound, non-redundant and back consistent.

Let M = (A, L, V, E) ∈ M be a multiplex network. Let A, L be respectively