Approximating multi-commodity max-flow in practice

IN

DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS

STOCKHOLM SWEDEN 2016 ,

Approximating multi- commodity max-flow in practice

KRISTOFFER EMANUELSSON

KTH ROYAL INSTITUTE OF TECHNOLOGY

Degree Project in Computer Science and Communication, Second Cycle DA222X

Approximating multi-commodity max-flow in practice

Author:

Kristoffer Emanuelsson kriema@kth.se

Supervisor:

Danupon Na Nongkai danupon@kth.se

March 24, 2016

Kungliga Tekniska Högskolan

Examensarbete inom Datateknik, Avancerad nivå, DA222X

Approximera multi-commodity max-flow i praktiken

Författare:

Kristoffer Emanuelsson kriema@kth.se

Handledare:

Danupon Na Nongkai danupon@kth.se

24 mars 2016

mum flow in a graph, later called a multiplicative weight update framework. Mądry

used this framework and exchanged Dijkstra’s algorithm to a dynamic graph algo-

rithm for approximating the shortest paths through the graph. With this approach

he developed the fastest algorithm to date for calculating the multi-commodity max-

imum flow, with a running time of O(mn

). This project have implemented the

algorithm and compared it with a slightly modified version of the former fastest

algorithm by Fleischer with a time complexity of O(m



). The results show that

Mądry’s algorithms is slower than Fleischer’s algorithm in practice for graph with less

than 100 million edges. This project also computed the space needed for the dynamic

algorithms used in Mądry’s algorithm and can show a resulting space complexity of

O(n(n + m) log

n), compared to the space complexity of Fleischer’s algorithm of

O(m). For a graph with 100 million edges, 50 million Gb of space is needed to use

Mądry’s algorithm, which is more than our test computers had. We can therefore

conclude that Mądry’s algorithm is not usable in real life today, both in terms of

memory usage and time consumption.

Sammanfattning

Garg and Könemann utvecklade ett framework för att beräkna multi-commodity maximum flöde i en graf sedan kallat ett multiplicative weight update framework.

Mądry använde detta framework och bytte ut Dijkstra’s algoritm mot en dyna-

misk grafalgoritm för att kunna approximera kortaste vägen i grafen. Med detta

angeppssätt utvecklade han den idag snabbaste algoritmen för beräkning av multi-

commodity maximum flöde med en tids komplexitet på O(mn

). Det här projektet

har implementerat hans algoritm och jämfört den med den tidigare snabbaste algo-

ritmen skapad av Fleischer med en tidskomplexitet på O(m



). Resultatet visar att

Mądrys algoritm är långsammare än Fleischers algoritm i praktiken för grafer med

färre än 100 miljoner kanter. Detta projekt beräknade också minnesåtgången för de

dynamiska algoritmerna i Mądrys algorithm och kan visa en resulterade minneskom-

plexitet på O(n(n+m) log

n), jämfört med Fleischers algoritm på O(m). För en graf

med 100 miljoner kanter så behövs 50 miljoner Gb av minne för att kunna använda

Mądrys algoritm, vilket var mer än våra testdatorer hade. Vi kan därför konstatera

att Mądrys algoritm inte är användbar i praktiken idag, både när det kommer till

minnesanvändning och till tidsåtgång.

Contents

1 Introduction 1

1.1 Maximum flow and multi-commodity flow problems . . . . 1

1.2 Known multi-commodity maximum flow algorithms . . . . 1

1.3 Mądry’s algorithm . . . . 2

1.4 Research Question and Goals . . . . 3

1.5 Our results regarding time . . . . 3

1.6 Our contribution regarding space . . . . 4

1.7 Our conclusion . . . . 4

1.8 Outline . . . . 4

2 Background 7 2.1 Graphs . . . . 7

2.1.1 Examples of graphs . . . . 8

2.2 Dynamic Graphs . . . . 8

2.3 Shortest Path Algorithms . . . . 9

2.3.1 Breadth First Search and Depth First Search . . . . 9

2.3.2 Dijkstra’s Algorithm . . . . 10

2.4 Flow networks . . . . 10

2.4.1 Multi-commodity flow . . . . 11

2.4.2 Example of flow networks . . . . 12

2.5 Approximate multi-commodity Maximum Flow . . . . 12

2.6 Approximate decremental shortest path data structure . . . . 14

2.7 Single Source Shortest Paths . . . . 14

2.8 Approximate Single Source Shortest Path . . . . 15

2.9 (r, Q)-DSPP data structure . . . . 16

2.9.1 (r, Q)-DSPP with a single vertex . . . . 17

2.9.2 (r, Q)-DSPP with multiple vertices . . . . 18

2.10 Sampling center nodes . . . . 18

2.11 (δ, M

, M

, Q)-ADSP data structure . . . . 19

2.12 Logarithmic (δ, M

, M

, Q)-ADSP . . . . 21

2.13 Aggregated (δ, M

, M

, Q)-ADSP . . . . 22

2.14 Using (δ, M

, M

, ˆ P)-ADSP in the framework . . . . 22

2.15 Space complexity . . . . 23

3 Method 27 3.1 Language . . . . 27

3.2 Implementing the dynamic graph structure . . . . 27

3.2.1 Implementing (r, P({s}))-DSPP . . . . 28

3.2.2 Implementing (r, P(S

))-DSPP . . . . 28

3.2.3 Implementing linear (δ, M

, M

, P)-ADSP

. . . . 28

3.2.4 Implementing logarithmic (δ, M

, M

, P)-ADSP

. . . . 28

3.2.5 Implementing aggregated (δ, M

, M

, P)-ADSP

. . . . 29

3.3 Implementing the naïve Dijkstra data structure . . . . 29

3.4 Implementing the maximum flow algorithm . . . . 29

3.5 Testing the algorithms . . . . 30

3.5.1 Setting up the tests . . . . 30

3.5.2 What have been tested . . . . 30

4 Results 33 4.1 Variants of algorithm in the results section . . . . 33

4.1.1 Naïve Dijkstra . . . . 33

4.1.2 Aggregated ADSP . . . . 33

4.1.3 Log ADSP . . . . 34

4.2 Dependency on the number of edges . . . . 34

4.3 Dependency on the number of vertices . . . . 43

4.4 Dependency on  . . . . 46

4.5 Dependency on Space . . . . 50

5 Discussion 53 5.1 Summary of the results . . . . 53

5.2 Correctness and approximation claim . . . . 54

5.2.1 Approximation claim . . . . 54

5.2.2 Dependency on the number of edges m . . . . 54

5.2.3 Dependency on the number of vertices n . . . . 54

CONTENTS

5.2.4 Dependency on the error parameter  . . . . 55

5.2.5 Conclusion . . . . 55

5.3 Slow practical times . . . . 56

5.4 Slow update cycles . . . . 56

5.5 Attempts to solve the cycle problem . . . . 58

5.6 Memory limits . . . . 59

5.7 When Mądry and the naïve approach is equally fast . . . . 60

6 Conclusion and future work 63

7 Bibliography 65

8 Appendix 69

List of Algorithms

1 Fleischers algorithm for calculation the multi-commodity maximum flow 13 2 Implementation of the constructor of the (r, P({s}))-DSPP data struc-

ture . . . . 70 3 Implementation of the Distance, SSrcDist and Path methods of the (r,

P({s}))-DSPP data structure . . . . 71 4 Implementation of the Increase method of the (r, P({s}))-DSPP data

structure . . . . 72 5 Implementation of the SSrcDist methods of the (r, P(S

))-DSPP data

structure . . . . 73 6 Implementation of the constructor, SSrcDist and Increase methods of

the linear (δ, M

, M

, P)-ADSP data structure

. . . . 74 7 Implementation of the constructor, Increase and getB methods of the

logarithmic (δ, M

, M

, P)-ADSP data structure

. . . . 75 8 Implementation of the naïve (δ, M

, M

, P)-ADSP data structure

76 9 Implementation of the improved multi-commodity maximum flow al-

gorithm . . . . 77 10 Implementation of FindAdmissiblePair for the improved multi-commodity

max-flow algorithm . . . . 78

In theory there is no difference between theory and practice.

But, in practice, there is.

JAN L. A. VAN DE SNEPSCHEUT

Chapter 1

Introduction

1.1 Maximum flow and multi-commodity flow problems

This thesis studies recent algorithms for computing the multi-commodity maximum flow problems. Maximum flow is an optimization problem in graph theory that involves finding the maximum flow that can be routed from a source to a sink through a graph. Each edge in the graph has a capacity of maximum amount of flow that can be routed through that edge. This problem can be solved by the use of a residual graph and algorithms like Ford-Fulkerson [FF56] or Edmonds-Karp [EK72]

in polynomial time. The maximum flow problem have multiple real life applications where the Baseball Elimination problem [Sch66] and matching a bipartite graph are among them.

The multi-commodity maximum flow problem is a generalization of the former problem with multiple sink-source pairs. The problem then involves finding the flow that maximizes the sum of flow that can be routed between each pair of source-sink pairs, while still keeping the capacity constraints. This problem can also be found in real life applications like the amount of information computers can send to each other through the Internet.

1.2 Known multi-commodity maximum flow algorithms

Finding an exact integer flow solution to the multi-commodity maximum flow prob-

lem is shown to be NP-complete [EIS75] even with only two commodities, however

by allowing fractional flows a polynomial solution can be found using Linear Pro-

gramming [Iri71]. To make the solution faster researchers have focused their energy

to different approximation schemes starting with Shahrokhi and Matula in 1990 [SM90]. This was followed up by Young in 1995 [You95], whose algorithm was improved by Garg and Könemann in 1998 [GK07]. Garg and Könemann both im- proved and simplified the ideas from Young and made a framework for computing the multi-commodity maximum flow.

The idea behind the framework is to compute the single source shortest paths between each commodity repeatedly. The resulting paths are then used to both route flow and increase the edge lengths. The updated graph is then used for the next iteration of computing shortest paths. The framework has later been called a multiplicative weight update framework and works as follows:

1. Find the shortest path between all source-sink pairs 2. If the length of the found path is shorter then 1:

(a) Route flow through the path,

(b) Increase the length of each edge included in the path, (c) Go back to (1)

3. Else, scale down the flow to satisfy the capacity constraints.

4. Return the flow

Due to the fact that (1) in the framework need to check every commodity to find the shortest distance between them, this algorithm is dependent on the number of commodities.

Fleischer improved the framework in 2000 [Fle00] by realizing that an approx- imation of which commodity has the shortest path could be made in (1). By this realization she was able to develop an algorithm with a running time independent of the number of commodities in the problem instance. Her algorithm has a running time of O(m



), where m is the number of edges in the graph, and  is an error constant.

1.3 Mądry’s algorithm

In 2010 Mądry [Mąd10] improved the framework further by realizing that not even the distances on the paths need to be exact and can be approximated as well.

He changed the shortest paths algorithm, which until now has consisted of Dijk-

stra’s algorithm, to a dynamic algorithm developed by Roditty and Zwick [RZ04b].

1.4. RESEARCH QUESTION AND GOALS

His algorithm is the theoretically fastest algorithm to date with a running time of O(mn

), where n is the number of vertices in the graph.

A dynamic graph is a graphs that may undergo modifications, like removing an edge or updating the length of an edge. If one uses Dijkstra’s algorithm to find the shortest path in a dynamic graph, everything need to be recomputed from scratch after each update. This is a very naïve approach, but it would work.

Mądry uses a structure originally developed by Even and Shiloach in 1981 [SE81], that was later improved by Roditty and Zwick [RZ04b]. The structure is called an ES-tree and supports both the shortest path queries in the frameworks (1) and the update made in (2:b). More information on this structure and how it is used by Mądry can be found in Section 2.9 to 2.12.

1.4 Research Question and Goals

Theoretical guarantees suggest that Mądry’s algorithm provides the fastest way to compute approximate multi-commodity maximum flow. In this thesis, we try to answer the following question

Is the algorithm developed by Mądry in [Mąd10] usable in practice in order to approximate the multi-commodity maximum flow of a graph?

We have implemented two versions of Mądry’s algorithm, one exactly as Mądry developed it, and one with a dynamic graph that has a slightly worse time complexity.

We compared the two algorithms from Mądry with a version very similar to the algorithm proposed by Fleischer, that uses Dijkstra’s algorithm for finding shortest paths. Our goal was see if Mądry’s algorithm is faster in a practical setting, and if the space it needs is of a size that makes it runnable on a standard computer with 12 Gb of RAM.

1.5 Our results regarding time

We measured the time it took for the three algorithms to solve the multi-commodity

maximum flow problem on generated random graphs. As non of the algorithms

tested was dependent on the number of commodities, we decided to test it on the

single commodity setting to make it easier to validate and construct. More on this

can be read in Section 3.5. By changing the number of vertices, edges and the error

constant , while keeping the other two constant, we could see how the different

algorithms depended on the three parameters. More information on how the tests where created can also be read in Section 3.5.

The results, which can be be seen in Chapter 4, show that both algorithms developed by Mądry is multiple magnitudes slower than the algorithms similar to Fleischer’s algorithm if the graphs tested are small. The results also show that a graph needs at least 100 million edges for Mądry’s algorithms to be faster, but this can not be tested due to memory constraints.

1.6 Our contribution regarding space

The space complexity of Mądry’s algorithm is discussed in Section 2.15 and shows a surprisingly high complexity of O(n(n + m) log

n). We calculated the hidden constants in Mądry’s algorithm and for a small graph with 100 vertices and 300 edges, the algorithm will need at least 1 Gb of RAM, compared to Fleischers algorithm that would need about 1,2 Kb of RAM. For a graph where Mądry’s algorithm theoretically would perform faster than Fleischer’s algorithm (i.e., a graph with at least 100 million edges), 50 Pb (i.e., 50 million Gb) of memory is needed, compared to 3,2 Gb for Fleischer. The computer that performed the tests for this project only had 12 Gb of memory, which is common in computers nowadays, but made it impossible to test Mądry’s algorithm on bigger graphs.

1.7 Our conclusion

Based on the results in this report, the time it would take for either of the three algorithms to solve a flow network with 100 million edges would be around 2000 years.

Our conclusion therefore is that the algorithm is unusable with today’s standard computers, both concerning time and memory. However, in a future with faster computers and more memory, Mądry’s algorithm may be able to solve the multi- commodity maximum flow faster than any other algorithm even in practice.

1.8 Outline

This paper continues with a background in Chapter 2, with basic information about

the problem, and related subjects. The end of the chapter will focus on the algorithms

implemented and tested in the project. Chapter 3 will talk about the decitions made

during implementation and how the tests where made. The reuslts are shown in

1.8. OUTLINE

Chapter 4 and discussed in Chapter 5. Chapter 6 will conclude the thesis, and the

references can be found in Chapter 7. The paper ends with an appendix in Chapter

8 including pseudo code of the algorithms implemented.

Chapter 2

Background

This chapter starts by explaining graphs in general and dynamic graphs in particular in Sections 2.1 and 2.2. Section 2.3 explains the most com- mon shortest paths algorithms, after which the chapter continue to de- scribe flow network in Section 2.4 and how to approximate their solutions in Section 2.5. Section 2.6 gives an overview of the dynamic structures used in Mądry’s algorithm and Sections 2.7 to 2.14 explains all the parts in more detail. The chapter finishes with a calculation of the space com- plexity of Mądry’s algorithm in Section 2.15.

2.1 Graphs

A graph G = (V, E), is a mathematical concept that describes a relationship between a set of vertices (or nodes), V . If two vertices u, v ∈ V fulfills the relationship property, an edge (also known as an arc) e = (u, v) is created. The set E contains all of these edges [Vol09]. In this paper the number of vertices are denoted by n = |V | and the number of edges are denoted by m = |E|.

A u − v path p from u to v, in G, is an ordered list of vertices [x

, x

, ..., x

] with (x

, x

) ∈ E, x

= u and x

= v for any 0 < i < k.

A graph can be undirected implying that (u, v) = (v, u), or directed, which

differentiates an edge (u, v) and an edge (v, u). In the directed case an edge (u, v) is

called an outgoing edge with respect to the node u and an incoming edge with respect

to the node v. A graph can also have a length function, l : E → R

, associated with

each edge, where l

returns the length of the edge e. The length of a path l

, is

defined as the sum of the lengths of all the edges in the path

(l

) if a length

1 2

3

4 1

4 2

7

13 7

Figure 2.1: A simple graph with 4 vertices and 6 edges. The number on each edge denotes their length.

function exists, or, if no such function exists, as the number of edges in the path (e.i., |p| − 1).

The shortest path between the vertices u and v, is the path that minimizes the sum

(l

) for every possible path p between u and v. The distance dist(u,v) is defined as the length of the shortest path between u and v. If no such path exists, the distance is defined as ∞.

2.1.1 Examples of graphs

Many real life problems and concepts can be represented as graphs.

• On Facebook a vertex could be a person with an edge between two vertices if they are friends.

• Each vertex could be a published article, and an edge between paper a and b could represent paper a citing paper b.

• Each vertex could represent a different city, and an edge between two neigh- bouring cities could have a length equal to the distance between those cities.

• The vertices could be real companies in the world with edges between them if one is a supplier to the other.

2.2 Dynamic Graphs

In static graphs, as the cases explained above, each edge will remain as it is during

the problem instance, and no lengths will be updated. In some real world problems

2.3. SHORTEST PATH ALGORITHMS

this might be true, but in others not. A road might be closed, a computer might lose connection to the Internet or be moved. Algorithms that can handle these kind of modifications on graphs are called dynamic algorithms. If only edge deletions are allowed, the algorithms are called decremental, if only edge insertions are allowed, they are called incremental and if both are allowed, they are called fully dynamic algorithms. If the graph has a length function associated to it, a length increase and a length decrease function, might also be available. This paper will focus on decremental algorithms with a length increase function.

A dynamic graph algorithm generally have two functions:

• Update(u,v): modifies the edge (u, v).

• Query(u,v): queries the graph with the nodes u and v.

The Update(·,·) function can be either delete, insert, increase or decrease, in the latter two a weight parameter is also applied. The Query(·,·) function can ask questions about the graph, like the shortest path between u and v or if u and v are connected.

Two time complexity measures are associated with a dynamic algorithm, the time to calculate each answer for the query function and the total time to complete all update requests on the graph. Throughout this paper, if the context is clear, the time will refer to the total update time of the algorithm.

2.3 Shortest Path Algorithms

2.3.1 Breadth First Search and Depth First Search

Two common algorithms are used to find the path between vertex u and v in a graph with no length function: Breadth First Search (BFS) and Depth First Search (DFS).

A BFS algorithm starts at vertex u and checks all the neighbours on distance 1 from u, to verify if one of them is v. If not, the algorithm checks all vertices on distance 2 from u, to verify of one of them is v, and so forth until v is found. This algorithm performs well if v is located close to u. The algorithm can guarantee that the shortest path is found.

A DFS algorithm starts at u and continues to one of the neighbours z. If that

vertex isn’t v, it continues to one of the neighbours of z, not yet visited, going as

deep as it can. When only visited neighbours exists, the algorithm backtracks until

it ends up at a vertex with a neighbour not yet visited, and continues from there.

The algorithm guarantees finding a path between u and v if such a path exists, but it doesn’t have to be the shortest one.

2.3.2 Dijkstra’s Algorithm

When a graph has a length function associated with it, the basic BFS algorithm can not find the shortest path in the graph, but Dijkstra suggested a way to modify it.

1. Start the algorithm at vertex u and put all neighbours in a priority list, pri- oritised on the distance to u with respect to the length function.

2. Go to vertex z, with shortest distance in the priority list.

3. If z = v: Return.

4. Else add all neighbours of z to the priority list, prioritized on their distance to u, with respect to the length function.

5. Repeat 2 - 5.

This algorithm will guarantee that the path u − v is found and that its distance is minimum. In the worst case every edge needs to be visited once, and at each visit a vertex in the priority queue might be updated. Each update takes O(log(n)) time, resulting in a O(m log(n)) total time complexity of Dijkstra’s algorithm.

2.4 Flow networks

Let the directed graph G = (V, E) have a capacity function c : E → R

associated with each edge, such that c(e) represents the capacity of flow that can be routed through edge e ∈ E. Let’s define two special vertices: s ∈ V the source, and t ∈ V the sink. This setup is called a flow network. The flow F in a flow network, is a set of non-negative real numbers f (e) associated to each edge e ∈ E, that satisfies the following constraints:

1. Capacity constraint: f (e) ≤ c(e)

2. Flow conservation: For each u ∈ V \ {s, t} we require

v∈V

f (v, u) =

v∈V

f (u, v)

2.4. FLOW NETWORKS

t s

(5/5) (10/15)

(5/5) (0/5) (5/5) (5/5)

(10/15) (5/5)

Figure 2.2: A flow network with capacity c and optimal routed flow f on each edge in the parentheses as (f /c). The slash (/) is just a separator and doesn’t denote division.

Intuitively as the capacity is the maximum amount of flow that can be routed through an edge, the flow through that edge needs to be less than or equal to that amount. The flow conservations says that the amount that flows into a vertex, has to be the same amount that flows out of it. Except from the source node and to the sink node. [Cor09]

Let the size of the flow |F | be the amount of flow routed from s to t in the graph.

This can be defined as the amount of flow leaving s:

f (s, v).

In the maximum flow problem we are given a flow network and want to return the flow F that maximizes |F | among all possible flows F .

2.4.1 Multi-commodity flow

Instead of a single source-sink pair it is possible to be given a set of k source-sink pairs {(s

, t

)}

. The task then is to find the flows (F

, F

, ..., F

) where F

is the flow from s

and t

, which maximizes the sum

|F

|, where |F

| is the amount of flow routed from s

to t

, while each flow still satisfies flow conservation and capacity constraint

f

(e) ≤ c(e) [Mąd10].

Definition 1 ([Mąd10]) For 1 ≤ i ≤ k, we define P

, the set of all s

− t

paths in

G. Let P =

P

. For a given subset U ⊆ V of vertices, let P(U ) be a set of all

paths in P that pass through at least one vertex in U . Finally, for a given j > 0, let

P(U, j) be the set of all the paths from P(U ) that consist of at most j edges.

2.4.2 Example of flow networks

Single commodity flow networks are seen in multiple real life areas like the amount of water that can flow through pipes in a city or for matching people together on a dating site. Real life multi-commodity flow networks are mainly focused on com- munication networks. Examples includes data routed through computers on the Internet or wavelengths routed through an optical network.

2.5 Approximate multi-commodity Maximum Flow

It has been shown that computing multi-commodity maximum flow (MMF) with integer values, even with only two commodities, is NP-complete [EIS75], implying that it is very hard to compute. Allowing fractional flow values enables the MMF problems to be solved using linear programming [Iri71] in polynomial time. To make the computations even faster different approximation schema’s has been the target for recent research.

Shahrokhi and Matula made the first successful approximation algorithm in [SM90] 1990. Many papers [Gol92, KP95, LMP

95] followed improving on their result, but using the same basic schema of creating an initial flow and redistributing the flow between less congested paths.

In 1995, Young was the first to deviate from this idea in [You95], by building up the flow from scratch. At each time step he queried the shortest path, routed some flow through it and updated the length of that path in relation to the flow routed. In the end the final flow is achieved by scaling down the result to fit the capacity. Garg and Könemann improved upon this solution 1998 in [Gar98] by simplifying Young’s algorithm and provide a framework for solving MMF problems. The framework which later have been called multiplicative weight update, is as follows:

1. Find the shortest path between all source-sink pairs 2. If the length of the found path is shorter then 1:

(a) Route flow through the path,

(b) Increase the length of each edge included in the path, (c) Go back to (1)

3. Else, scale down the flow to satisfy the capacity constraints.

4. Return the flow

2.5. APPROXIMATE MULTI-COMMODITY MAXIMUM FLOW

This algorithm is dependent on the number of commodities k with a total running time of O(km



), where O(·) notation hides poly-logarithmic factors.

2000 Fleischer [Fle00] used this framework to significantly improve the algorithm and managed to develop a (1+)-approximation algorithm for the MMF problem, in- dependent of the number of commodities to satisfy, with a running time of O(m



).

Her algorithm approximates which of the commodities has the shortest distance be- tween them, by checking the distance against a value α. Flow is routed through the path as long as the distance is shorter than α. Her updated algorithm can be seen in Algorithm 1.

1

Procedure Fleischer()

2

while α < 1 do

3

foreach s-t ∈ commodities do

4

while dist(s, t) < α do

5

p ← path(s, t);

6

Route flow through p;

7

Increase the length of p;

8

end

9

end

10

Increase α;

11

end

12

Scale down flow;

13

return flow;

Algorithm 1: Fleischers algorithm for calculation the multi-commodity max- imum flow

2010 Mądry improved Fleischers algorithm further in [Mąd10] by combining the

techniques from dynamic graph algorithms and approximate single source shortest

path problems [RZ04a]. He realized that even dist(s,t), the distance from s to

t, can be an approximation and exchanged what earlier was a call to Dijkstra’s

algorithm to a request to a dynamic algorithm. The request is answered in constant

time, but instead the dynamic structure need to be updated after the length of p

have been updated in the inner loop of Fleischer’s algorithm. His algorithm have

an improved running time of O(mn

), and is explained in detail in the coming

sections, as it is the main focus in this report.

2.6 Approximate decremental shortest path data struc- ture

Mądry improved a dynamic data structure developed by Roditty and Zwick [RZ04b].

The data structure is supposed to answer shortest path requests between the com- modities, in place of Dijkstra’s algorithm in the maximum flow framework (i.e., the call to dist(s,t) and path(s,t) in Algorithm 1). The data structure can do this in constant time, however it needs to be updated every time the graph is modified (i.e., each time Line 7 is executed in Algorithm 1). The data structure is created in multiple layers, which makes it a fairly complicated algorithm.

In the bottom most layer is a simple dynamic data structure called an ES-tree.

The ES-tree can solve the single source shortest path problem for a dynamic graph and is explained in Sections 2.7 and 2.8. Two such trees are used to approximate solutions for the all pairs shortest paths problem, and a set of these paired ES-trees will create a better approximation. The data structure that is created out of a set of paired ES-trees are called a (r, Q)-DSPP data structure and is explained in detail in Section 2.9 and 2.10.

Mądry developed a data structure called (δ, M

, M

, Q)-ADSP that is the data structure that can be plugged in to the multi-commodity maximum flow framework. It uses multiple instances of the (r, Q)-DSPP data structure for an- swering the shortest path requests as accurately as possible. The (δ, M

, M

, Q)-ADSP data structure exists in three versions, all developed by Mądry and called the linear (Section 2.11), logarithmic (Section 2.12) and aggregated (Section 2.13) (δ, M

, M

, Q)-ADSP data structure in this report. Each version uses one or more versions of the former to each improve the theoretical running time of the whole algorithm.

How to modify the multi-commodity maximum flow framework, to be able to use the (δ, M

, M

, Q)-ADSP data structure in it, is explained in Section 2.14.

2.7 Single Source Shortest Paths

Given a graph G = (V, E), a length function l, and a node s, the goal of the single

source shortest path (SSSP) problem is to answer queries for the distance dist(s,u)

and path path(s,u) from the node s to any other node u ∈ V with respect to

the length function l. In the static graph case, this problem is easily solved with

Dijkstra’s algorithm producing an O(m log n) algorithm, but has later been improved

2.8. APPROXIMATE SINGLE SOURCE SHORTEST PATH

by exchanging the min-priority queue to a Fibbonachi Heap for an O(m + n log n) running time [FT87].

Moving into decremental dynamic algorithms, one can naïvely use Dijkstra’s algorithm after each edge deletion. This would result in a O(m(m + n log n)) time algorithm (or O(m

log n) using a min-priority queue), as there are at most m edges to be deleted.

Even and Shiloach [SE81] made the first non trivial SSSP algorithm, by first running Dijkstra’s algorithm once and creating a tree with node s as root. By remembering the distance to each node, the algorithm can answer distance queries in O(1) time. After each edge deletion, the algorithm needs to update each distances that have changed, which can be done in total time O(mn). This structure is called an ES-tree throughout this paper.

Roditty and Zwick showed in [RZ04b] that it is very difficult to improve this time while keeping the distances exact

, which has led many researchers into the field of approximation.

2.8 Approximate Single Source Shortest Path

To be able to compare approximation algorithms to each other, it’s common to make a guarantee of how close an algorithms answer will be to the optimal value. A common annotation is to say that an algorithm is an (α, β)-approximation algorithm, where α is a multiplicative error and β is an additive error on the optimal solution to the problem. This means that if the optimal value to a problem is OP T and the algorithm returns the value d, then the algorithm guaranties that OP T ≤ d ≤ α ∗ OP T + β. If an algorithm is an (α, 0)-approximation of the problem (i.e., the additive part is 0), then we simply call it an α-approximation.

In 1995 Henzinger and King showed that the ES-tree could be generalized to di- rected graphs [HK95], and King later extended the ES-tree into a O(mnL) algorithm on directed weighted graphs [Kin99], where L is the maximum possible length of an edge. These techniques was used by both Bernstein [Ber13, Ber09] and Mądry [Mąd10] to achieve a (1 + )-approximate algorithm in O(mn log L) time.

This was later improved by Henzinger, Krinninger and Nanongkai in [HKN14b]

to achieve a (1+)-apprioximate O(mn

log

L)-time algorithm for SSSP on decre-

1This result comes from the use of combinatorial algorithms assuming that Boolean matrix mul- tiplication does not admit a combinatorial algorithm with so-called truly subcubic time (Henzinger et al. recently showed in [HKNS15] similar lower bound for any algorithms assuming another con- jecture called OMV).

mental directed graphs. In the latest paper [HKN14a] from the same authors a new approach has been visited, to achieve a total update time of O(m

√

log log n/ log n)

log L), although in undirected graphs.

2.9 (r, Q)-DSPP data structure

Given a graph G = (V, E) and a length function l, the goal of the all-pair shortest paths (APSP) problem is to answer queries for the distance dist(u,v) and path path(u,v) between each nodes u ∈ V and v ∈ V with respect to the length function l.

Roditty and Zwick generalized the ES-tree to an approximate all-pairs shortest path algorithm in [RZ04b] by having two trees rooted at the same node s and letting one tree be constructed with only incoming edges and the other with only outgoing edges. By doing so, one tree can answer questions in the form dist(u,s) while the other in the form dist(s,v). By adding the two results together an approximation to the dist(u,v) ≈ dist(u,s) + dist(s,v) can be formed.

Let’s call the structure that uses the generalized ES-tree a (r, Q)-DSPP data structure, which will be used in the dynamic data structure that Mądry uses in his algorithm. We will now formally define the (r, Q)-DSPP data structure that can answer path and distance queries and increase edges on a graph G, so that it later can be use in the algorithms:

Definition 2 ([Mąd10]) For any integer r ≥ 0 and a set of paths Q ⊆ P, let the decremental (r, Q)-shortest path problem ( (r, Q)-DSPP for short) be a problem in which one maintains a directed graph G with positive integral weights on its edges, and that supports four operations:

• Distance(u,v), for u, v ∈ V : returns the length of the shortest u-v path in Q if this length is at most r, and ∞ otherwise.

• Increase(e,t), for e ∈ E and integer t ≥ 0: increases the length of the edge e by t.

• Path(u,v), for u, v ∈ V : return a u-v path of length Distance(u,v), as long as Distance(u,v) 6= ∞.

• SSrcDist(u), for u ∈ V : returns Distance(u,v) for all v ∈ V .

2.9. (R, Q)-DSPP DATA STRUCTURE

2.9.1 (r, Q)-DSPP with a single vertex

Mądry gives a first guarantee of the time to maintain such a data structure in Lemma 1, if the parameter Q is set to P({s}). Remember from Definition 1 that P({s}) is the set of all source-sink paths going through the fixed node s.

Lemma 1 ([Mąd10]) For any s ∈ V and positive integer r, a (r, P({s}))-DSPP data structure can be maintained in total time O(mr) plus additional O(log n) per

Increase(·,·) request. Each Distance(·,·) query can be answered in O(1) time and each Path(·,·) and SSrcDist(·) query in O(n) time.

To show that Lemma 1 holds, let’s maintain a (r, P{s})-DSPP structure as in Definition 2, by using the generalized ES-tree from [RZ04b] and return dist(u,s) + dist(s,v) to each Distance(u,v) query, if the sum is ≤ r and ∞ otherwise. By storing the distances from s to z dist

[z] and the distances from z to s dist

[z], for each node z ∈ V , these queries can be answered in constant time. We also let each node in the tree keep track of it’s incoming and outgoing edges with two priority queues Q

and Q

respectively. The queues is prioritized by the shortest distance to the root node using that edge. By walking on the first edge in each queue one can find the shortest path back to the root node in O(n) time, and can answer Path(u,v) queries by returning path(u,s) + path(s,v). The SSrcDist(u) query can be answered by performing n number of Dist(u,v) queries, one to each v ∈ V and return all of the answers, which clearly will take O(n) time. The Increase(e,t) queries can be answered by first increasing the length of edge e with the set amount t and then make a scan with a depth first search (DFS) in each of the two ES-trees (i.e., the one created from incoming edges and the one created from outgoing edges).

Let’s assume that the distance to node v through edge f has been increased, after the Increase(e,t) query was performed, in the tree handling incoming edges. The scan will update edge f in Q

[v] with the new distance. If the shortest distance to v is greater than dist

[v] (i.e., the previous shortest value), the value in dist

[v] is updated and the DFS continues on all outgoing edges of v. The same thing happens on the other tree but with outgoing and incoming edges switched. The update of a priority queue will take O(log n) time and the scan will only continue if dist

[v]

has been increased with at least one. This limits the number of times a scan can be

performed to r times for each edge, which results in a time of O(rm log n), where r

is the maximum length of an edge in the data structure.

2.9.2 (r, Q)-DSPP with multiple vertices

It’s easy to realize that a distance and path though a graph that have to go through a fixed node s, can not be guaranteed to give a satisfactory answer. However by changing the parameter of Q to P(U ) (i.e., all source-sink paths that go through at least one of the vertices in the set U ), we create multiple so called oracles, with different center nodes. By returning the best answer after querying all of them, a better approximation can be gained.

Lemma 2 ([Mąd10]) For any U ⊆ V and positive integer r, a (r, P(U ))-DSPP data structure can be maintained in total time O(mr|U |) plus additional

O(|U |) per

Increase(·,·) request. Each Distance(·,·) query can be answered in O(|U |) time, each Path(·,·) request in O(n) time, and SSrcDist(·) query in O(m + n log n) time.

To show that Lemma 2 holds, let’s maintain a (r, P{s})-DSPP data structure R

as in Lemma 1, for each s ∈ U . We can answer Distance(u,v) queries by querying each R

and return the minimum result in O(|U |). The Path(u,v) query can be answered by first determine which R

have the shortest distance between u and v and then forward the path request to that R

. This will take O(n) time. Each Increase(e,t) can be forwarded to each R

in O(mr|U |) time, and finally each

SSrcDist(u) query can be answered by creating a new graph G

, exactly as G, but with one extra node u

. Let u

have an edge (u

, s) for each s ∈ U with the length R

.Distance(u,s). Then let’s use Dijkstra’s algorithm on G

starting at node u

, and then return the resulting distances. The time for Dijkstra’s algorithm will limit the request to O(m + n log n) time, using a Fibbonachi Heap.

2.10 Sampling center nodes

Letting the set U be the complete set V would create an exact decremental dynamic all-pairs shortest path algorithm, although we aren’t interested in all paths through the graph, only the s

-t

paths, for each 1 ≤ i ≤ k, where {(s

, t

)}

is the k source-sink pairs in the multi-commodity maximum flow (MMF) problem. Namely the set of paths P. In fact, for the algorithm to work, we do not need all of those paths, just the ones included in the optimal solution to the MMF problem [Mąd10].

In Definition 1 we defined P(U ) to be the set of all paths in P going though at least

one of the nodes in U . That means that if we can find a set U such that P(U )

2.11. (δ, M

, M

, Q)-ADSP DATA STRUCTURE

includes all the paths in the optimal solution to the MMF we can use that set as a parameter for the (r, Q)-DSPP data structure.

Definition 3 ([Mąd10]) For j = 1, ..., dlog ne, let S

be a random set obtained by sampling each vertex of V with probability p

= min{

, 1}. Define P :=

j=1

P(S

, 2

), where P(U, j

) for a given U ⊆ V , and j

> 0 is the set of all paths in P that consist of at most j

edges and pass through at least one vertex of U .

Intuitively we sample log n sets of vertices, each times with a smaller probability to include a vertex in the set, and want the paths going through the nodes in the sets to have continuously longer lengths. It turns out that this set actually includes all the paths in the optimal solution with high probability, stated in Lemma 3, and can therefore be used as parameter Q.

Lemma 3 ([Mąd10]) For any fixed multi-commodity flow F = (F

, ..., F

) solution, with high probability, all the flow paths of F are contained in P.

2.11 (δ, M

, M

, Q)-ADSP data structure

To glue the set P, from Definition

3, together with our previously defined data structure (r, P(U ))-DSPP, and make it easier to use in the multi-commodity flow problem, we define a new data structure:

Definition 4 ([Mąd10]) For any δ ≥ 0, M

≥ 2M

> 0, and a set of paths Q ⊆ P, let the δ-approximate decremental (δ, M

, M

, Q)-shortest path prob- lem ( (δ, M

, M

, Q)-ADSP for short) be a problem in which one maintains a directed graph G with a length function l on its edges that supports four operations:

• Distance(u,v,β), for u, v ∈ V , and β ∈ [M

, M

/2]: let d

be the length of the shortest (with respect to l) u-v path in Q; if d

≤ 2β then the query returns a value d such that d ≤ d

+ δβ. If d

> 2β, the query may return either d as above, or ∞.

• Increase(e,ω), for e ∈ E and ω ≥ 0: increases the length l

of the edge e by ω.

• Path(u,v,β), for u, v ∈ V , and β ∈ [M

, M

/2]: return a u-v path of

length at most Distance(u,v,β), as long as Distance(u,v,β) 6= ∞.

• SSrcDist(u,β), for u ∈ V , and β ∈ [M

, M

/2]: returns Distance(u,v,β) for all v ∈ V .

Intuitively β is our guess of the length of the shortest path between u and v, and the distance returned will have an additive error of δβ which implies a (1 + δ) multiplicative error guarantee [Mąd10].

Definition 4 of the (δ, M

, M

, Q)-ADSP data structure already looks sim- ilar to Definition 2 of the (r, Q)-DSPP data structure, but to make use of the later in the former a few changes are needed. The largest difference is that the length function l operating on the graph G accepts real numbers, while the length function operating on the (r, Q)-DSPP data structure only accepts integers. This can be mended by converting the real number into a rounded multiple of ρ, a concept first introduced to the world of dynamic graph algorithms by Bernstein in [Ber09].

Definition 5 ([Mąd10]) For a given length function l, and real number ρ, let us define l

to be a length function with the length of an edge e ∈ E be l

= dl

/ρeρ.

Intuitively, dl

/ρe will return an integer describing how many multiples of ρ that will fit into l

. By using this as a length function, and multiply the answer with ρ again before returning it, the result is a rounded answer with an additive error of at most ρ.

Lemma 4 ([Mąd10]) For any δ > 0, and M

≥ 2M

> 0 we can maintain a (δ, M

, M

, P)-ADSP data structure in total expected time

O(mn

min

) plus additional O(n) per Increase(·,·) request. Each Distance(·,·,·) and Path(·,·,·)

query can be answered in O(n) time, and SSrcDist(·,·) query in O(m) time.

For each 1 ≤ j ≤ dlog ne let’s maintain a (dM/ρ

e,P(S

))-DSPP data structure R

with respect to the rounded length function l

:= l

/ρ

, where l is the length function of the graph G, ρ

:= δM

/2

, and M := M

+ δM

. Intuitively we create a data structure for each set of sampled nodes in P, as in Definition

3, each bounded with respect to their length function, M

and M

.

To answer Distance(u,v,β) queries, issue a Distance(u,v) query to each R

and return the minimum result after multiplying the answer with each corresponding

ρ

. The Path(u,v,β) query can be answered by first determining which R

has the

shortest distance between u and v and then forward the Path(u,v) request to that

R

. Both these requests will take O(n) time. Similarly each SSrcDist(u,β) request

is forwarded to each R

, and each answer is multiplied by it’s corresponding ρ

. The

2.12. LOGARITHMIC (δ, M

, M

, Q)-ADSP

answer returned to the request is then the minimum distance to each v ∈ V among all the answers, which results in a O(m) time request. For the Increase(e,ω)

requests, increase each R

with respect to their own length function by issuing an Increase(e, d(l

+ ω)/ρ

e-dl

/ρ

e) to each R

. This can be done in O(mn

min

) time.

2.12 Logarithmic (δ, M

, M

, Q)-ADSP

The earlier version of the data structure have linear time dependency on

min

for maintenance, which needs to be improved to logarithmic.

Lemma 5 ([Mąd10]) For any δ > 0, and M

≥ 2M

> 0 we can maintain a (δ, M

, M

, P)-ADSP data structure in total expected time

O(mn

) plus additional O(n log

) per Increase(·,·) request. Each Distance(·,·,·) and Path(·,·,·) query can be answered in O(n) time, and SSrcDist(·,·) query in O(m)

time.

To show that Lemma 5 holds, let’s split up the [M

, M

] range into smaller slightly overlapping exponentially increasing ranges [M

, M

], where M

= 2

M

and M

= 2

M

, for each 0 ≤ b ≤ log

min

. For each b let’s maintain a (δ, M

, M

, P)-ADSP data structure R

as in Lemma 4.

Now for each Distance(u,v,β), Path(u,v,β) and SSrcDist(u,β) request, we just need to forward it to the corresponding R

that handles the unique range with M

≤ β ≤ M

/2.

For each Increase(e,ω) request, it is possible to forward it to each R

. However, Mądry developed a better way, by observing that if the length function of an edge in any R

would be increased to exceed 2M

, it can safely be set to ∞. He also observes that it isn’t necesary to forward an increase to a R

where the increase wouldn’t increase the multiple of the length function ρ even one step. Define b

to be the largest b such that 2M

< l

+ ω, and define b

to be the largest b such that l

+ ω ≥

2^{dlog ne}

. For each b ≤ b

we increase the edge to ∞. For each b

< b < b

we send an Increase(e, ω

) with ω

set to the sum of all the increases done on the

edge, since the last Increase that was issued on that R

. For b ≥ b

we do nothing.

2.13 Aggregated (δ, M

, M

, Q)-ADSP

A last improvement was developed by Mądry by making the additional time for each Increase(·,·) request constant.

Theorem 6 ([Mąd10]) For any δ > 0, and M

≥ 2M

> 0 we can maintain a (δ, M

, M

, P)-ADSP data structure in total expected time

O(mn

) plus additional O(1) per Increase(·,·) request. Each Distance(·,·,·) and Path(·,·,·) query can be answered in O(n) time, and SSrcDist(·,·) query in O(m) time.

To make Theorem 6 hold, let’s maintain a (δ/2, M

, M

, P)-ADSP data

structure R as in Lemma 5, and forward all Distance(u,v,β), Path(u,v,β) and SSrcDist(u,β) request to it. The answer is then returned with

added to each of them. The Increase(e,ω) requests will also be forwarded but in an aggregated manner. Let ˆ l be a length function of R, that initially is equal to the length function l of the graph G. As we increase l we only increases ˆ l

to l

for a certain edge e when l

> max{(1+δ/8)ˆ l

, δM

/4n}. When this happens we forward a Increase(e,l

− ˆ l

) to R, which makes R work with respect to ˆ l instead of l.

2.14 Using (δ, M

, M

, ˆ P)-ADSP in the framework

The data structure can be queried for paths and distances, and edges can be in- creased. Let’s see how to use it to approximate the multi-commodity maximum flow problem.

Garg and Könemann [Gar98] first came up with the idea to initialize the length of each edge in the graph to γ, where γ = (1 + )/((1 + )n)

is very small.

Then as long as there is a path in the graph between any source-sink pair that has a length smaller than 1, pick the shortest such path. First, augment flow along that path equal to the smallest capacity c along that path, and secondly increase each edge along that path by a factor of (1 +

), where c(e) is the capacity of the edge e.

When the algorithm terminates, the flow routed though the graph might violate

the capacity constraint, which easily can be mended by scaling down the flow by

the maximum congestion ratio in the solution. The maximum congestion is defined

as max

f (e)/c(e), where f (e) is the flow routed through the edge e. This gives

a (1 − 2)-approximation in O(km



) time for any  < 1, given that Dijkstra’s

algorithm is used to find the shortest path [Gar98].

2.15. SPACE COMPLEXITY

Fleischer [Fle00] later came up with an improvement to the described algorithm where she realized that we only need a (1 + )-approximation to the shortest path to get the correct solution. With that in mind she doesn’t need to go though all of the k source-sink pairs to find the shortest path, but can look for a path with a length at most (1 + )α, where α = γ in the beginning. After going through all source-sink pairs and no path can be found with a length of at most (1 + )α, α is updated to (1 + )α. When α exceeds 1, the algorithm terminates.

She also realized that she could bundle together all source-sink pairs sharing the same source, by using Dijkstra’s algorithm for finding a shortest path, as Dijkstra’s algorithm effectively calculates all of them at the same time.

For any 0 ≤  ≤ 0.15 this algorithm will give a (1 − 4)-approximation of the maximum flow in time O(m



), given that Dijkstra’s algorithm is used, resulting in the first multi-commodity maximum flow algorithm not dependent on the number of commodities. [Fle00]

With just a few modifications, Mądry showed in [Mąd10] that it is possible to use the δ-approximate decremental (δ, M

, M

, Q)-shortest path data structure R, from Definition 4, instead of recalculating the shortest path using Dijkstra’s algorithm every time, where M

is set to 1 and M

is set to γ.

The first modification was to update R every time the length of an edge in the graph G is updated. This is done by issuing an Increase(e,ω) query to R. The second modification was that the bundled version with pairs that are sharing the same source, that Fleischer uses to find all possible pairs to augment at the same time, can not be used. Instead, Mądry suggest selecting a sink at random in a bundle, and if that doesn’t have a path between the source and sink short enough, pick an other one at random, at most log n times. If no possible pair was found to augment, then make a SSrcDist(s,α) query to R, and remove all source-sink pairs that have a length greater than (1 + )α.

When asking for a distance, issue a Distance(s,t,α) request to R and when to find a path, issue a Path(s,t,α) request to R.

Theorem 7 ([Mąd10]) For any 0 ≤  ≤ 0.15, the algorithm should find, with high probability, a (1 + 3)-approximation to the MMF problem in time O(mn

)

2.15 Space complexity

When developing an algorithm there is sometimes a trade-off between space and

time. Every modification that improves the time complexity by adding an extra