BitTorrent Traffic Measurements and Models

Blekinge Institute of Technology

Licentiate Dissertation Series No. 2005:13 School of Engineering

BITTORRENT TRAFFIC MEASUREMENTS AND MODELS

David Erman

BITT ORRENT TRAFFIC MEASUREMENTS AND MODELS

The Internet has experienced two major revolu- tions. The fi rst was the emergence of the World Wide Web, which catapulted the Internet from being a scientifi c and academic network to beco- ming part of the societal infrastructure. The second revolution was the appearance of the Peer-to-Peer (P2P) applications, spear-headed by Napster.

The popularity of P2P networking has lead to a dramatic increase of the volume and complexity of the traffi c generated by P2P applications. P2P traf- fi c has recently been shown to amount to almost 80% of the total traffi c in a high speed IP backbone link. One of the major contributors to this massive volume of traffi c is BitTorrent, a P2P replication system. Studies have shown that BitTorrent traf- fi c more than doubled during the fi rst quarter of 2004, and still amounts to 60% of all P2P traffi c in 2005.

This thesis reports on measurement, modelling and analysis of BitTorrent traffi c collected at Ble- kinge Institute of Technology (BIT) as well as at a local ISP. An application layer measurement in- frastructure for P2P measurements developed at BIT is presented. Furthermore, a dedicated fi tness assessment method to avoid issues with large sample spaces is described. New results regarding BitTorrent session and message characteristics are reported and models for several important cha- racteristics are provided. Results show that seve- ral BitTorrent metrics such as session durations and sizes exhibit heavy-tail behaviour. Additionally, previously reported results on peer reactivity to new content are corroborated.

ABSTRACT

ISSN 1650-2140

Da vid Erman 2005:13

BitTorrent Traffic

Measurements and Models

David Erman

October 2005

Department of Telecommunication Systems, School of Engineering,

Blekinge Institute of Technology

Blekinge Institute of Technology

Licentiate Dissertation Series No. 2005:13 ISSN 1650-2140

ISBN 91-7295-071-4

Published 2005

Printed by Kaserntryckeriet AB Karlskrona 2005

Sweden

This publication was typeset using L

TEX.

For my Family

past, present and future

Abstract

The Internet has experienced two major revolutions. The first was the emer- gence of the World Wide Web, which catapulted the Internet from being a scientific and academic network to becoming part of the societal infrastructure.

The second revolution was the appearance of the Peer-to-Peer (P2P) applica- tions, spear-headed by Napster.

The popularity of P2P networking has lead to a dramatic increase of the volume and complexity of the traffic generated by P2P applications. P2P traffic has recently been shown to amount to almost 80 % of the total traffic in a high speed IP backbone link. One of the major contributors to this massive volume of traffic is BitTorrent, a P2P replication system. Studies have shown that BitTorrent traffic more than doubled during the first quarter of 2004, and still amounts to 60 % of all P2P traffic in 2005.

This thesis reports on measurement, modelling and analysis of BitTorrent

traffic collected at Blekinge Institute of Technology (BIT) as well as at a local

ISP. An application layer measurement infrastructure for P2P measurements

developed at BIT is presented. Furthermore, a dedicated fitness assessment

method to avoid issues with large sample spaces is described. New results re-

garding BitTorrent session and message characteristics are reported and models

for several important characteristics are provided. Results show that several

BitTorrent metrics such as session durations and sizes exhibit heavy-tail be-

haviour. Additionally, previously reported results on peer reactivity to new

content are corroborated.

Acknowledgements

Several people have contributed in various ways, directly or indirectly, to the work culminating in this thesis. I extend my gratitude to them all. However, I would like to thank a few people in particular.

• My advisor, Docent Adrian Popescu for his attention to detail and cor- rectness, motivation and encouragement.

• My fellow graduate students at BIT. In particular Dragos Ilie and Doru Constantinescu for valuable criticism and encouragement.

• Dr. Markus Fiedler. His enthusiasm and tenacity is an inspiration to any PhD student.

• Prof. Arne Nilsson for accepting me as a PhD student.

• My parents, for performing above and beyond the call of duty and teaching me to question the unquestionable.

• My immediate family, Maria, for putting up with me during the writing of this thesis.

David Erman

Karlskrona, October 2005

Contents

Page

1 Introduction 1

1.1 Motivation . . . . 2

1.2 Related Work . . . . 3

1.3 Main Contributions . . . . 4

1.4 Thesis Outline . . . . 5

2 Peer-to-peer Protocols 7 2.1 Evolution . . . . 7

2.2 Definitions . . . . 9

2.3 P2P and File Sharing . . . . 12

2.4 Summary . . . . 14

3 The BitTorrent Protocol 15 3.1 BitTorrent Encoding . . . . 16

3.2 Resource Meta-data . . . . 17

3.3 Network Entities and Protocols . . . . 18

3.4 Peer States . . . . 22

3.5 Sharing Fairness and Bootstrapping . . . . 23

3.6 Data Transfer . . . . 24

4 Traffic Measurements 29

4.1 Measurement Approaches . . . . 30

4.2 Application Level Traffic Analysis . . . . 34

4.3 Measurement Infrastructure . . . . 36

4.4 Measurement Software . . . . 36

4.5 Summary . . . . 42

5 Traffic Modelling 43 5.1 Heavy-tailed Traffic Models . . . . 44

5.2 Hypothesising Distributions . . . . 51

5.3 Mixture Distributions . . . . 54

5.4 Parameter Estimation . . . . 58

5.5 Summary . . . . 63

6 Fitness Assessment 65 6.1 Graphical Methods . . . . 66

6.2 Hypothesis Testing . . . . 66

6.3 The Case of Large Sample Spaces . . . . 71

6.4 Relative and Absolute Fitness . . . . 71

6.5 Summary . . . . 72

7 Modelling Methodology 73 7.1 Distribution Selection . . . . 73

7.2 Parameter Estimation . . . . 74

7.3 Fitness Assessment . . . . 74

7.4 Summary . . . . 77

8 BitTorrent Measurements 79

8.1 Traffic Metrics . . . . 80

8.2 Traffic Measurements . . . . 82

8.3 Summary Statistics . . . . 84

8.4 Swarm Size . . . . 90

8.5 Session Sizes . . . . 92

8.6 Summary . . . . 93

9 BitTorrent Models 95 9.1 Session Characteristics . . . . 95

9.2 Message Characteristics . . . 105

9.3 Summary . . . 117

10 Conclusions and Future Work 119 10.1 Future Work . . . 120

A BitTorrent Protocol Details 123 A.1 Bencoding Types . . . 123

A.2 Peer Wire Protocol Messages . . . 124

A.3 Tracker Request Parameters . . . 125

A.4 Scrape Response Keys . . . 128

B BitTorrent XML Log File 129 B.1 BitTorrent Application Log DTD . . . 134

Bibliography 137

List of Figures

Figure Page

3.1 BitTorrent handshake procedure . . . . 19

3.2 Example tracker announce GET request . . . . 20

3.3 Compact tracker response . . . . 21

3.4 Example tracker scrape GET request . . . . 22

3.5 BitTorrent protocol exchange . . . . 25

4.1 BIT measurement setup . . . . 36

4.2 Measurement procedures . . . . 37

4.3 Sample BitTorrent log file . . . . 42

5.1 Pareto, Weibull, Log-normal and Exponential Hill plots . . . . . 46

5.2 Pareto, Weibull, Log-normal and Exponential α-estimator plots . 47 5.3 Pareto, Weibull, Log-normal and Exponential CCDF . . . . 49

5.4 Skewness . . . . 52

5.5 A finite mixture distribution . . . . 54

6.1 AD weighting function for a uniform distribution . . . . 69

8.1 Temporal structure of measurements 1–12 . . . . 83

8.2 Connected peers during seed phase for measurements 4 and 6 . . 90

9.1 Fitness assessment plots . . . . 97

9.2 Session size-duration scatter plot . . . 100

9.3 α-estimates and CCDF for measurement 3 . . . 102

9.4 Upstream request rate during leech phase . . . 106

9.5 Modelling results for request rate during leech phase . . . 107

9.6 Modelling results for request inter-departure times during leech phase . . . 108

9.7 Modelling results for downstream piece rate during leech phase . 110 9.8 Modelling results for downstream piece inter-arrival times during leech phase . . . 111

9.9 Dual Weibull modelling results for downstream request rate dur- ing seed phase . . . 113

9.10 Modelling results for request inter-arrival times during seed phase 115 9.11 Dual Weibull modelling results for upstream piece rates during seed phase . . . 116

9.12 Modelling results for piece inter-departure times during seed phase117

B.1 Extract from BitTorrent XML log file . . . 130

List of Tables

Table Page

2.1 P2P and CS content models . . . . 11

6.1 EDF statistic percentage points . . . . 71

7.1 Fitness quality boundaries . . . . 76

8.1 Measurement summary . . . . 83

8.2 Content summary . . . . 84

8.3 Download time and average download rate summary . . . . 85

8.4 Session and peer summary . . . . 86

8.5 Downstream protocol message summary . . . . 88

8.6 Upstream protocol message summary . . . . 89

8.7 Share ratio during leech phase . . . . 92

8.8 Correlation coefficients for session sizes . . . . 92

9.1 Fitted hyper-exponential parameters . . . . 98

9.2 Correlation coefficients for session duration and sizes . . . . 99

9.3 Percentages of session sizes exceeding 0 bytes and 1 piece size . . 100

9.4 Session α-estimates . . . 101

9.6 Log-normal parameter estimates and errors for upstream session durations during seed phase . . . 104 9.7 Gaussian parameter estimates and errors for upstream request

rate during leech phase . . . 107 9.8 Exponential parameter estimates and errors for request inter-de-

parture times during leech phase . . . 107 9.9 Exponential and Uniform parameter estimates and errors using

alternative model for request inter-departure times during leech phase . . . 108 9.10 Weibull parameter estimates and errors for downstream piece rate

during leech phase . . . 109 9.11 Exponential parameter estimates and errors for piece inter-arrival

times during leech phase . . . 110 9.12 Weibull parameter estimates and errors for downstream request

rate during seed phase . . . 112 9.13 Dual Weibull parameter estimates and errors for downstream re-

quest rate during seed phase . . . 112 9.14 Exponential parameter estimates and errors for request inter-

arrival times during seed phase . . . 114 9.15 Weibull parameter estimates and errors for upstream piece rate

during seed phase . . . 115 9.16 Dual Weibull parameter estimates and errors for upstream piece

rate during seed phase . . . 116 9.17 Exponential parameter estimates and errors for piece inter-depa-

rture times during seed phase . . . 117

Acronyms

AD Anderson-Darling

BIT Blekinge Institute of Technol- ogy

CCDF Complementary Cumulative Distribution Function

CDN Content Delivery Network CS Client-Server

CVM Cram´er-von Mises DHT Distributed Hash Table DNS Domain Name System DTD Document Type Definition DOM Document Object Model EDF Empirical Distribution Func-

tion

EPDF Experimental Probability Den- sity Function

IID Independent and Identically Distributed

ISP Internet Service Provider KS Kolmogorov-Smirnov LRD Long-Range Dependence MLE Maximum Likelihood Estima-

tion

ML Maximum-Likelihood

MPAA Motion Picture Association of

America

NAT Network Address Translation NFS Network Filesystem

NNTP Network News Transfer Proto- col

P2P Peer-to-Peer

PDF Probability Density Function PIT Probability Integral Transform PSTN Public Switched Telephone Net-

work

QQ Quantile-Quantile QoS Quality of Service

RIAA Recording Industry Association of America

RMON Remote Monitoring SAX Simple API for XML SHA-1 Secure Hash Algorithm One SMTP Simple Mail Transfer Protocol SNMP Simple Network Management

Protocol

SRD Short-Range Dependence

URI Uniform Resource Indicator

UUCP Unix to Unix Copy Protocol

VoIP Voice over IP

Chapter 1

Introduction

The prisoner falls in love with his chains.

– Edsger W. Dijkstra

The global Internet has emerged to become an integral part of everyday life.

It is now as fundamental a part of the infrastructure as the telephone system or the road network. The initial driving factor pushing the acceptance and widespread usage of the Internet was the introduction of the World Wide Web (WWW) by Tim Berners-Lee in 1989. The WWW provided a way of accessing information in a novel and intuitive way, and quickly became the Internet “killer application” [6].

In May 1999, ten years after the advent of the WWW, Shawn Fanning introduced Napster, arguably the first modern Peer-to-Peer (P2P) application [53]. The Napster application and protocols were the first to allow users to share files among each other without the need of a central storage server. Very quickly, Napster became immensely popular, and the P2P revolution had begun.

Since the advent of Napster, P2P systems have become wide-spread with

the emergence of file-sharing applications such as Gnutella [46], KaZaA [71] and

eDonkey [27]. These systems generated headlines across the globe when the

Recording Industry Association of America (RIAA) [56] and Motion Picture Association of America (MPAA) [55] started filing law suites against file-sharing users suspected of copyright infringement. The law suites are partly responsible for the embrace of the term P2P as a euphemism for illegal file-sharing. Fortu- nately, the concept of P2P networking is broader than that and P2P systems have many useful legitimate applications.

The P2P paradigm is the logical and functional antithesis of the Client- Server (CS) paradigm that has been the predominant paradigm for IP-based networks since their inception. This is however only true to a certain degree, as the idea of sharing among equals has been part of the Internet since the early days of the network. Two examples are the e-mail system employed in the Internet and the Domain Name System (DNS). Both protocols are so tightly connected to the inner workings of the Internet, that it is impossible to imagine the degree of usage that the Internet sees today without them. Once an e-mail has left the user’s mail software, it is routed among mail transfer agents (MTAs), all acting as equally valued message forwarders. The DNS is the first distributed information database, and implements a hierarchical mapping scheme, which is comparable to the multi-layered P2P systems.

The fundamental difference between “legacy” P2P systems such as DNS and e-mail and the new Peer-to-Peer (P2P) systems such as Gnutella, Napster and eDonkey is that the older systems work as part of the network core, while the new applications are typically application-layer protocols run by edge-node applications. The shift of the edge nodes from acting as purely service users to additionally taking the role as service providers has significantly changed the characteristics of the network traffic.

1.1 Motivation

Measurement studies and analysis of P2P traffic have been rather limited so far.

This is because of the complexity of this task, which involves answering hard

questions related to data retrieval and content location, storage, data analysis

and modelling of traffic and topological characteristics as well as privacy and

1.2. RELATED WORK

copyright issues.

There are two major points in motivating the work performed for this thesis:

• BitTorrent has become extremely popular over the last years. According to Cachelogic, the BitTorrent traffic volume has increased from 26 % to 52 % of the total P2P traffic volume during the first half of 2004 [9].

The increase of the amount of BitTorrent traffic indicates that understand- ing the characteristics of BitTorrent would also help in understanding the overall Internet behaviour.

• There are few measurement studies performed on BitTorrent [24, 38, 39].

This is because the protocol is quite new, only a few years old, but also because of the general complexity of the task. In the few studies that do exist, traffic has been collected from ”trackers” as well as with the help of modified clients. However, there have been no dedicated measurement studies on a message-level so far.

The main goals of this thesis are to understand the characteristics of the BitTor- rent system and, based on that, to develop models suitable for a P2P simulation environment. To that end, a dedicated measurement system for P2P system traffic measurements [36] has been designed and implemented.

1.2 Related Work

In general, measurement studies of P2P systems are limited in number. Saroiu et al. performed a measurement study of Napster and Gnutella in 2002 [69].

Active and passive measurements were performed on both systems. Their results show the non-cooperativity of peers involved in the systems and several other characteristics such as estimated peer bandwidths, number of shared files and resilience.

The present work is one of a few works investigating the properties of the

BitTorrent system. For instance, in [38], the authors use tracker and client

logs to evaluate performance on both global and session scales. They note the efficiency of the tit-for-tat-policy employed in BitTorrent, and the flexibility and scalability of the protocol.

Qiu and Srikant present a fluid flow model for BitTorrent-like file-sharing P2P networks. They assume a Poisson peer arrival process and exponentially distributed download times. Additionally, the authors assume that seeds remain in the network according to an exponentially distributed time and identical download rates for all peers. Their results state that the number of seeds and leechers are Gaussian random variables [24] when in steady state.

Nicoll et al. have analysed tracker log files with regards to session sizes (denoted by file sizes in the paper), peer bandwidth and share ratios. They note that up to 20 % of peers do not download any data at all, and 20-25 % of peers connect but do not upload any data. Additionally, the authors point to that 80 % of peers have a share ratio less than 1, i.e., they download more than they upload.

The measurement infrastructure developed at Blekinge Institute of Technology (BIT) is capable of detecting and measuring application layer messages with link layer accuracy. One drawback with the infrastructure is that it cannot yet do so in real-time. In [44], Karagiannis et al. present a novel method for identifying P2P traffic without resorting to application payload decoding. Their method is based on observing connection patterns of source and destination IP addresses.

To verify the method, the authors also present a payload identification method using protocol-specific bit strings.

1.3 Main Contributions

The main contributions of this thesis are related to providing accurate models of several BitTorrent key characteristics. To the best of our knowledge, this is the first study of this kind.

The reported models include session duration, size and inter-arrival times

as well as rates and inter-arrival times for the two most relevant BitTorrent

1.4. THESIS OUTLINE

application messages.

From a traffic engineering and control viewpoint, the session models re- ported in Section 9.1 provide incentive for controlling the amount of concurrent BitTorrent flows. The message characteristics reported in Section 9.2 indicate that some form of per-message control may also be beneficial to decrease the burstiness of the network traffic.

Other contributions include the development of a modular P2P measurement infrastructure. This infrastructure is currently used to measure Gnutella and BitTorrent traffic with high accuracy on the link layer.

Additionally, the method for assessing model fitness in the case of large sam- ple spaces may prove useful for other modelling scenarios as well (Section 7.3).

It has performed well during the current work, as well as in other published work [21].

Parts of the the work presented in this thesis has been previously been published in [29–31, 36, 37].

1.4 Thesis Outline

This thesis contains nine chapters and two appendices. The current chapter has presented the motivation for and main contributions of the thesis, along with a brief presentation of the state of the art in P2P research.

Chapter two contains a short history and description of P2P systems, with

special focus on the most popular application, i.e., file sharing. This is followed

by a detailed description of the BitTorrent system and the associated protocols

in chapter three. Chapter four gives an introduction to traffic modelling and a

brief description of the measurement infrastructure used for this work. Chapter

five discusses traffic modelling in general, and heavy-tailed modelling in partic-

ular. Also, tools for describing empirical distributions are presented. Chapter

six summarises some of the most common methods of determining the fitness

of specific distributions. Chapter seven builds on the two preceding chapters to

present the modelling methodology used for the work performed for this thesis.

In Chapter eight, the actual measurements performed are presented, together with some of the more salient results of these measurements. Chapter nine re- ports on the models for BitTorrent session and message characteristics. Chap- ter ten concludes the thesis, with conclusions and implications of the presented work. Potential future work is also presented.

The appendix contains implementation details for the BitTorrent protocols,

and a description of the XML log format used for the application measurements

presented in Chapter eight.

Chapter 2

Peer-to-peer Protocols

Tvertimot!

– Henrik Ibsen

The concept of P2P protocols, systems and applications is quite broad. The term P2P commonly refers to applications and systems that share resources in a distributed and decentralised manner. Participants in these systems are viewed as logical and functional equals. This is in contrast to pure Client- Server (CS) protocols, where participants either serve resources or are being served resources. A more formal definition of these is provided in Section 2.2.

2.1 Evolution

The earliest recorded use of the term “peer-to-peer” was in 1984. It was related to the IBM Advanced Peer to Peer Networking (APPN) Architecture [28], which was the result of multiple enhancements to the Systems Network Architecture (SNA).

Although early networking protocols such as the Unix to Unix Copy Protocol

(UUCP) [35], Network News Transfer Protocol (NNTP) [43] and Simple Mail

Transfer Protocol (SMTP) [45] were working in a P2P fashion – indeed, the original ARPANET was designed as a P2P system – the term P2P did not become mainstream until the appearance of Napster in the fall of 1999.

Napster was the first popular file-sharing P2P service. The main goal of the service was to provide users with easy means of finding music files encoded in the MP3 format. The architecture of Napster was built around a central server that was used to index music files shared by client nodes. This approach is called a centralised directory. The centralised directory allowed Napster to give a very rapid reply to which hosts stored a particular file. The actual file transfer occurred directly between the node looking for the file and the node storing the file.

The success of Napster quickly became a source of serious concern for ma- jor record companies who rapidly filed a lawsuit against Napster on grounds of copyright infringement. The lawsuit made Napster immensely popular, at- tracting additional millions of users to the system. However, Napster could not withstand the pressure of the lawsuit and in July 2001 they were forced to shut down the central server. Without the central server the client nodes could no longer search for files. Thus, the fragility of a centralised directory system became apparent.

Napster is one of the first generation P2P applications as defined by [28]. Fol- lowing the advent of Napster, several other P2P applications appeared. These applications were similar in appearance, but altogether different beasts in de- tail. Gnutella [18], which was released by Justin Frankel of Winamp fame in early 2000, opted to implement a fully distributed system with no central au- thority. The same year saw the emergence of the Freenet system, which was the brainchild of Ian Clarke. Clarke wrote his Master’s thesis on a distributed, anonymous and decentralised information storage and retrieval system. This system later became Freenet [17, 65]. Freenet’s major difference to previous P2P systems was the complete anonymity it offered to users.

The fully distributed architecture was resilient to node failures and was also

immune to service disruptions of the type experienced by Napster. However,

experience with Gnutella has shown that fully distributed P2P systems may

2.2. DEFINITIONS

lead to scalability problems due to the massive amounts of signalling traffic they generate [68].

By late 2000 and early 2001, the P2P boom had started and applications such as KaZaA [71], DirectConnect [54], SoulSeek [73] and eDonkey [27] appeared.

These new systems usually provided some form of community-like features such as chat rooms and forums, in addition to the file-sharing services provided by previous systems.

KaZaA, which uses the FastTrack protocol, introduced the concept of su- pernodes in order to solve scalability problems similar to those experienced by Gnutella. Each supernode manages a number of regular nodes and exchanges information about them with other supernodes. Regular nodes upload file lists and search requests to their supernode. The search requests are processed solely among the supernodes. Regular peers establish direct HTTP connections to download files.

Gnutella resolved the scalability problem in a similar way. In Gnutella, supernodes are called ultrapeers.

During the last few years, the old P2P systems have evolved to better utilise network resources. New systems have emerged with the specific focus of efficient bandwidth utilisation. The most significant example of this development is the BitTorrent system. Furthermore, new systems tend to focus on using Distributed Hash Tables (DHTs). DHTs force network topology and data storage to follow specific mathematical structures in order to optimise various parameters (e.g., minimise delay or number of hops). They are considered as being a promising alternative to the flooding algorithms required by routing in unstructured P2P networks.

2.2 Definitions

There is no clear consensus regarding an exact definition of a P2P system.

Schollmeier makes an attempt to define a P2P network in [70]. In general, the

notion of a P2P network seems to be leaning towards some form of utilisation

of edge node resources by other edge node resources. The resource in question is commonly accepted to be files, and much research is focusing on the efficient localisation and placement of files. There also seems to be some consensus regarding the idea of pure and hybrid systems.

A P2P network is defined in [70] as a network in which the service provided by the system is provided by the participating nodes. The participating nodes share part of their local resource pool, such as disk space, files, CPU processing time to the common resource pool. A pure P2P network is one in which any given participant may be removed without the system experiencing loss of service.

Examples of this type of network are Gnutella, FastTrack and Freenet. A hybrid P2P network is one in which a central authority of some sort is necessary for the system to function. Note that, in contrast to the CS model, the central authority in a hybrid network rarely shares resources – it is still the participating peers that share resources. The central authority is commonly an indexing server for files or provides a peer localisation service. Examples of this type of network are Napster, eDonkey and DirectConnect.

It is also possible to take a resource view of the two types of P2P networks described above. Consider the three functions of content insertion, distribution and control and how they are performed in P2P and CS networks (Table 2.1).

Insertion Insertion is the function of adding content to the resource pool of a network. Insertion is here referred to in the sense of providing the content, so that in both pure and hybrid content is inserted by the participating peers. This is anal- ogous with the peers sharing content. In a CS system how- ever, content is always provided by the server, and thus also

“shared” by the server.

Distribution This is the function of retrieving content from a network re-

source pool. Again, P2P systems lack central content local-

isation, thus content is disseminated in a decentralised fash-

ion. This does not necessarily mean that parts of the same

content is retrieved from different sources, i.e., swarming,

2.2. DEFINITIONS

but rather that the parts (e.g., files) of the total resource pool are retrieved from different sources.

Hybrid CS systems refer to redundant server systems and content delivery networks (CDNs), such as Akamai [1]. Re- dundant server systems are systems in which several servers provide the same content, but are accessed by the request- ing client from a single Uniform Resource Indicator (URI).

This is a common model for WWW servers in the Internet today.

Control Control is the function of managing the resource pool of a network, such as admission control and resource locali- sation. This is the function that separates the two types of P2P networks. The peers participating in fully decen- tralised networks are required to assist in the control mech- anisms in the network, while hybrid systems may rely on a central authority for this. Of course, the clients in CS systems have no responsibility towards the network control functionality.

Table 2.1: P2P and CS content models. C denotes centralised and D de- notes decentralised.

Pure P2P Hybrid P2P Hybrid CS Pure CS

Insertion D D C C

Distribution D D C/D C

Control D C C C

In addition to the definitions provided above, P2P systems can also be clas-

sified according to their “generation” [28]. In this classification scheme, hybrid

systems such as Napster are considered to be first generation systems, while

fully decentralised systems such as FastTrack and Gnutella are second genera-

tion systems. A third generation is discussed as being the improvement upon the

two first with respect to features such as redundancy, reliability or anonymity.

2.3 P2P and File Sharing

File sharing is almost as old as operating systems themselves. Early methods for sharing files include protocols such as the UNIX remote copy (rcp) command and the File Transfer Protocol (FTP) [64]. They were quickly followed by full- fledged network file systems such as NFS [52,72] and CIFS [2]. A common trait of these protocols (with the exception of rcp) is that they were designed around the CS paradigm, with the servers being the entity storing and serving files. A client that wants to share files must upload them to the server to make them available to other clients.

Instant messaging systems such as ICQ [3], Yahoo! Messenger [7] and MSN Messenger [4] attempted to provide file sharing service by implementing a mech- anism similar to rcp. Users could thus share file with each other without having to store them on a central server. In fact, this was the first form of P2P file sharing. Napster further extended this idea by implementing efficient file search facilities.

In the public eye, P2P is synonymous with file sharing. However, other applications that may be termed P2P have become fairly popular as well, such as the SETI@home project [75], distributed.net [26] and ZetaGrid [84]. These applications have been fairly successful in attracting a user-base, but none of them come close to the number of users that the file sharing services have.

These services are examples of altruistic systems. The participating peers provide CPU processing power and time to a common resource pool without deriving personal benefit from this. The pooled CPU resources are then used to perform various complex calculations such as calculating fast Fourier transforms of galactic radio data, code-breaking or finding roots of the Riemann Zeta- function.

A possible reason for the difference in number of users could be that the

incentive to altruistically share resources without gaining anything other than

2.3. P2P AND FILE SHARING

some virtual fame or feel-good points of having contributed to the greater good of humanity seems to be low. Most file sharing P2P systems employ some form of admission scheme in which peers are not allowed to join the system or download from it unless they are sharing an adequate amount of files. This provides a dual incentive: first, a peer wanting to join the network must

provide sort of an entry token in the form of shared files, and second, peers joining the system know that there is a certain amount of content provided to them once they join.

The BitTorrent P2P system is one of the most prominent networks in enforcing incentive.

As not all files are equally desirable in every system, files not belonging to the general category of files handled in a specific P2P network should not be allowed in. For instance, users of a network such as Napster, which only manages digital music files, might not be interested in peers sharing text files.

For systems that require a large amount of file data to be shared as an admission scheme, this becomes a problem. Peers may share “junk files” just to gain access to the network. Junk files are files that are not really requested or desired in the network. These practices are usually scorned upon, but are hard to get to grips with. Some systems, such as eDonkey, have implemented a rating system, in which peers are punished for sharing junk files.

Similar to junk files, there are also “fakes” or “decoys”. Fakes are files in- serted in the network that masquerade under a filename that does not represent the actual content, or files that contain modified versions of the same content.

By adding fakes into the network, the real content is made more difficult to find. This problem is alleviated by using various hashing techniques for the files instead of only relying on the filenames to identify the content. An example of this is the insertion of a faked Madonna single, in which the artist had overlaid the phrase “What the hell do you think you’re doing?” on top of her newly released single. Often, fakes are not as immediately apparent as this, and some form of user feedback is useful. For instance, the eDonkey system implements a reputation system for files. Decoys are often automatically generated from incoming queries to pollute the P2P networks. While decoys do not pollute the actual resource pool of the network, they can have the effect of valid queries

1

Not in all systems, but in most hybrid systems.

being ignored or de-emphasised.

While file sharing in and of itself is not an illegal technology and has several non-copyright infringing uses, the ease with which peers may share copyrighted material has drawn the attention of the MPAA and RIAA. These organisations consider the sharing of material under the copyrights of their members as seri- ously harming their revenue streams, by decreasing sales. In 2004, the MPAA and RIAA began suing individuals for sharing copyrighted material. However, not all copyright holders and artists agree on this course of action, nor do they agree on the detrimental effect file sharing has on sales or artistic expression.

Several smaller record labels have embraced the distribution of samples of their artists’ music online, and artists have formed coalitions against what they feel is the oppressive behaviour of the larger record labels.

More recently, P2P systems have been used by corporations to distribute large files such as Linux distributions, game demos and patches. Many compa- nies make use of the BitTorrent system for this, as it provides for substantial savings in bandwidth costs.

2.4 Summary

This chapter has discussed the history and evolution of P2P systems. The first P2P systems were classic Internet services such as the DNS or e-mail systems.

More modern systems include Gnutella and eDonkey. Currently, P2P systems are usually categorised as either pure or hybrid systems.

Additionally, the most popular P2P service, file-sharing, has been discussed.

File-sharing is the major application for P2P protocols, and is used in both

commercial and personal applications.

Chapter 3

The BitTorrent Protocol

Anyone who considers protocol unimportant has never dealt with a cat.

– Robert A. Heinlein

BitTorrent is a P2P protocol for content distribution and replication designed to quickly, efficiently and fairly replicate data [15,19]. The BitTorrent system may be viewed as being comprised of two protocols and a set of resource meta-data.

The two protocols are for communication among peers and for the communica- tion with a central network entity called the tracker. The meta-data provides all information needed for a peer to join a BitTorrent distribution swarm and to verify correct reception of the resource.

The following terminology is used in this thesis: a BitTorrent swarm refers to all network entities partaking in a distribution of a specific resource. When referring to the the peer–peer protocol, the BitTorrent protocol or protocol in singular is used, while explicitly referring to the tracker protocol for the peer–

tracker communication. The collection of protocols (peer, tracker and meta- data) is referred to as the BitTorrent protocol suite or protocol suite.

In contrast to many other P2P protocols such as eDonkey, DirectConnect,

KaZaA, the BitTorrent protocol suite provides neither resource query or lookup

functionality, nor chat, messaging or topology formation facilities. The protocols rather focus on fair and effective distribution of data. The signalling is geared towards an efficient dissemination of data only.

Fairness in the BitTorrent system is implemented by enforcing tit-for-tat exchange of content between peers. Non-uploading peers are only allowed to download very small amounts of data, making the download of a complete re- source very time consuming if a peer does not share downloaded parts of the resource.

With one exception (Section 3.3.2), the protocols operate over TCP and use swarming, i.e., peers simultaneously downloading parts of the content, called pieces, from several peers. The rationale for this is that it is more efficient in terms of network load, as the load is shared across links between peers. This results in a more evenly distributed network utilisation than in the case of conventional CS distribution systems.

The size of the pieces is fixed on a per-resource basis and may not be changed without generating a new meta-data file. The default piece size is 2

bytes, i.e., 256 kB. The selection of an appropriate piece size is a fairly important issue. If the piece size is small, re-downloading a failed piece is fast, while the amount of extra data needed to describe all the data grows. On the other hand, larger piece sizes means less meta-data, but longer re-download times.

3.1 BitTorrent Encoding

BitTorrent uses a simple encoding scheme for most protocol messages and asso- ciated data. This encoding scheme is known as bencoding. The scheme allows for data structuring and type definition, and currently supports four data types:

strings, integers, lists and dictionaries. These are detailed in Section A.1 in the

Appendix.

3.2. RESOURCE META-DATA

3.2 Resource Meta-data

A peer interested in downloading some content by using BitTorrent must first obtain a set of meta-data, the so-called torrent file, to be able to join a set of peers engaging in the distribution of the specific content. The meta-data needed to join a BitTorrent swarm consists of the network address information (in BitTorrent terminology called the announce URL) of the tracker and resource information such as file and piece size. The torrent file itself is a bencoded version of the associated meta information.

An important part of the resource information is a set of Secure Hash Algo- rithm One (SHA-1) [8, 57] hash values

, each value corresponding to a specific piece of the resource. The hash values are used to verify the correct reception of a piece. When rejoining a swarm, the client must recalculate the hash for each downloaded piece. This is a very intensive operation with regards to both CPU usage and disk I/O, which has resulted in certain alternative BitTorrent clients storing information regarding which pieces have been successfully downloaded within a specific field in the torrent file.

A separate SHA-1 hash value, the info field, is also included in the meta- data. This value is used as an identification of the current swarm, and the hash value appears in both the tracker and peer protocols. The value is obtained by hashing the entire meta-data (except the info-field itself). Of course, if a third-party client has added extra fields to the torrent file that may change intermittently (such as the resume data or cached peer addresses), these should not be taken into account when calculating the info-field hash value.

The meta-data as defined by the original BitTorrent design does not contain any information regarding the peers participating in a swarm, though this in- formation is added by some alternative clients to lessen strain on trackers when rejoining a swarm. This feature allows the peer to continue the download in case of tracker failure.

1

These are also known as message digests.

3.3 Network Entities and Protocols

A BitTorrent swarm is composed of peers and at least one tracker. The peers are responsible for content distribution among each other. Peers locate other peers by communicating with the tracker, which keeps peer lists for each swarm.

A swarm may continue to function even after the loss of the tracker, but no new peers are able to join.

To be functional, the swarm initially needs at least one connected peer to have the entire content. These peers are denominated as seeds, while peers that do not have the entire content, i.e., downloading peers, are denominated as leechers.

The BitTorrent protocols (except the meta-data distribution protocol) are the tracker protocol and the peer protocol. The tracker protocol is either a HTTP-based protocol or a UDP-based compact protocol, while the peer pro- tocol is a BitTorrent-specific binary protocol. Peer-to-tracker communication usually takes place using HTTP, with peers issuing HTTP GET requests and the tracker returning the results of the query in the returning HTTP response.

The purpose of the peer request to the tracker is to locate other peers in the distribution swarm and to allow the tracker to record simple statistics of the swarm. The peer sends a request containing information about itself and some basic statistics to the tracker, which responds with a randomly selected subset of all peers engaged in the swarm.

3.3.1 The Peer Protocol

The peer protocol, also known as the peer wire protocol, operates over TCP, and uses in-band signalling. Signalling and data transfer occur in the form of a continuous bi-directional stream of length-prefixed protocol messages over a common TCP byte stream.

A BitTorrent session is equivalent with a TCP session, and there are no

protocol entities for tearing down a BitTorrent session beyond the TCP tear-

3.3. NETWORK ENTITIES AND PROTOCOLS

down itself. Connections between peers are single TCP sessions, carrying both data and signalling traffic.

Once a TCP connection between two peers is established, the initiating peer (Peer A in Figure 3.1) sends a handshake message containing the peer id and info field hash (Figure 3.1). If the receiving peer (Peer B) replies with the corresponding information, the BitTorrent session is considered to be opened and the peers start exchanging messages across the TCP streams. Otherwise, the TCP connection is closed. Immediately following the handshake procedure, each peer sends information about the pieces of the resource it possesses. This is done only once, and only by using the first message after the handshake. The information is sent in a bitfield message, consisting of a stream of bits, with each bit index corresponding to a piece index.

Peer B Peer A

info

info,peer_id B peer_id A

bitfield exchange

message exchange

Figure 3.1: BitTorrent handshake procedure

The BitTorrent peer protocol messages are described in Section A.2 of the Appendix.

3.3.2 The Tracker Protocol

The tracker is accessed by HTTP or HTTPS GET requests. The default listen-

ing port is 6969. The tracker address, port and top-level directory are specified

in the announce url field in the torrent file for a specific swarm.

Tracker Queries

Tracker queries are encoded as part of the GET URL, in which binary data such as the info_hash and peer_id fields are escaped as described in RFC1738 [14].

The query is added to the base URL by appending a question-mark, ?, as described in RFC2396 [13].

The query itself is a sequence of parameter=value pairs, separated by am- persands, &, and possibly escaped. An example of a tracker request is given in Figure 3.2. The \-characters indicate that the line continues on the following line.

GET /announce?info_hash=n%05hV%A9%BA%20%FC%29%12%1Ap%D4%12%5D%E6U%0A%85%E1&\

peer_id=M3-4-2--d0241ecc3a07&port=6881&key=0fcca260&uploaded=0&downloaded=0&\

left=663459840&compact=1&event=started HTTP/1.0

Figure 3.2: Example tracker announce GET request

A complete list of parameters is given in Section A.3 of the Appendix.

Tracker Replies

The tracker HTTP response is, unless the compact parameter is 1, a bencoded dictionary. The contents of the reply are listed in Section A.3.3 of the Appendix.

If the compact parameter is set to 1, then the reply is a binary list of peer addresses and ports. This list is encoded as a six-byte datum for each peer, in which the first four bytes are the IP address of the peer, and the last two bytes are the peer’s listening port (Figure 3.3). This saves bandwidth, but is only usable in an IPv4 environment. There is no equivalent compact format for IPv6.

If the request fails for some reason, the dictionary contains only a single key:

failure reason, indicating the reason for the failed request.

3.3. NETWORK ENTITIES AND PROTOCOLS

0 32 48

Peer

IP address Peer

port Peer

IP address Peer

port

.. .

Peer

IP address Peer

port

Figure 3.3: Compact tracker response

Tracker UDP Protocol Extension

To lower the bandwidth usage for heavily loaded trackers, a UDP-based tracker protocol has been proposed [77].

The UDP tracker protocol is not part of the official BitTorrent specification, but has been implemented in some of the third-party clients and trackers.

Compared to the standard HTTP-based protocol, the UDP protocol uses about 50 % less bandwidth. It also has the advantage of being stateless, as opposed to the stateful TCP connections required by the HTTP scheme. This means that a tracker is less likely to run out of resources due to for instance half-open TCP-connections.

The Scrape Convention

BitTorrent trackers commonly include simple HTTP servers to provide informa- tion on the swarms they track. Web scraping denotes the procedure of parsing a Web page to extract information from it. It is a fall-back method of obtaining information when other methods fail or are not available. The BitTorrent vari- ant is a bit different, as it is a way for peers to gain information on a specific swarm without actually joining the swarm.

Trackers can implement functionality to allow peers to request information regarding a specific swarm without resorting to error-prone Web-scraping tech- niques.

If the last name in the announce URL, i.e., the name after the last /-character

is announce, then the tracker supports scraping by using the announce URL with the name announce replaced by scrape.

The scrape request may contain an info_hash parameter, as shown in Fig- ure 3.4, or be completely without parameters.

GET /scrape?info_hash=n%05hV%A9%BA%20%FC)%12%1Ap%D4%12%5D%E6U%0A%85%E1 HTTP/1.0

Figure 3.4: Example tracker scrape GET request

The tracker responds with a bencoded dictionary containing information about all the swarms that the tracker is currently tracking. The dictionary has a single key, named files. This key contains another dictionary whose keys are the 20-bit binary info_hash values of the torrents on the specific tracker.

Each value of these keys contains another dictionary with information about the specific swarm. The contents of this dictionary is given in Section A.4 of the Appendix.

3.4 Peer States

A peer maintains two states for each peer relationship. These states are known as the interested and choked states. The interested state is imposed by the requesting peer on the serving peer, while for the case of the choked state the opposite is true. If a peer is being choked, then it will not be sent any data by the serving peer until unchoking occurs. Thus, unchoking is usually equivalent with uploading.

The interested state indicates whether other peers have parts of the sought

content. Interest should be expressed explicitly, as should lack of interest. That

means that a peer wishing to download notifies the sending peer (where the

sought data is) by sending an interested message, and as soon as the peer no

longer needs any other data, a not interested message is issued. Similarly,

for a peer to be allowed to download, it must have received an unchoke message

from the sending peer. Once a peer receives a choke message, it will no longer

be allowed to download. This allows the sending peer to keep track of the

3.5. SHARING FAIRNESS AND BOOTSTRAPPING

peers that are likely to immediately start downloading when unchoked. A new connection starts out choked and not interested, and a peer with all data, i.e., a seed, is never interested.

In addition to the two states described above, some clients add a third state – the snubbed state. A peer relationship enters this state when a peer purports that it is going to send a specific sub-piece, but fails to do so before a timeout occurs (typically 60 seconds). The local peer then considers itself snubbed by the non-cooperating peer, and will not consider sub-pieces requested from this peer to be requested at all. The snubbed state is reconsidered from time to time.

3.5 Sharing Fairness and Bootstrapping

The choke/unchoke and interested/not interested mechanism provides fairness in the BitTorrent protocol. Since it is the transmitting peer that decides whether to allow a download or not, peers not sharing content tend to be reciprocated in the same manner.

To allow peers that have no content to join the swarm and start sharing, a mechanism called optimistic unchoking is employed. Optimistic unchoking means that from time to time, a peer with content will allow even a non-sharing peer to download. This will allow the peer to share the small portions of data received so far and thus enter into a data exchange with other peers.

This means that while sharing resources is not strictly enforced it is strongly

encouraged. It also means that peers that have not been able to configure

their firewalls and/or Network Address Translation (NAT) routers properly will

only be able to download the pieces altruistically shared by peers through the

optimistic unchoking scheme.

3.6 Data Transfer

Data transfer is performed in parts of a piece (called subpiece, block or chunk) at a time, by issuing a request message. Subpiece sizes are typically of size 16384 or 32768 bytes. The subpiece size is not part of the protocol, and may be chosen at the discretion of the requesting peer.

To allow TCP to increase throughput, several requests are usually sent back- to-back. Each request should result in the corresponding subpiece to be trans- mitted. If the subpiece is not received within a certain time (typically one minute), the non-transmitting peer is snubbed, i.e., is punished by not being allowed to download, even if unchoked. Data transfer is performed by sending a piece message, which contains the requested subpiece (Figure 3.5). Once the entire piece, i.e., all subpieces, has been received, and the SHA-1 hash of the piece has been verified to the corresponding hash value in the meta-data, a have message is sent to all connected peers.

The have message allows other peers in the swarm to update their internal information on which pieces are shared by specific peers in the swarm.

3.6.1 End-game Mode

When a peer is approaching completion of the download, it sends out requests for the remaining data to all currently connected peers to quickly finish the download. This is known as the end-game mode. Once a requested subpiece is received, the peer sends out cancel-messages to all peers that have not yet sent the requested data.

Without the end-game mode, there is a tendency for peers to download the

final pieces from the same peer, which may be on a slow link [20].

3.7. BITTORRENT PERFORMANCE ISSUES

request(piece,subpiece) request(piece,subpiece)

request(piece,subpiece) request(piece,subpiece)

Peer A Peer B Peer C

interested

interested

unchoke unchoke

piece(subpiece) piece(subpiece)

piece(subpiece) piece(subpiece)

have

have

Figure 3.5: BitTorrent protocol exchange

3.7 BitTorrent Performance Issues

Even though BitTorrent has become very popular among home users, and widely deployed in corporate environments, there are still some issues currently being addressed for the next version of BitTorrent.

The most pressing issue regards the load on the central tracker authority.

There are two main problems related to the tracker: peak load and redundancy.

Many trackers also handle more than a single swarm. The most popular trackers

handle several hundred swarms simultaneously. It is not uncommon for popular

swarms to contain hundreds or even thousands of peers. Each of these peers

connect to the tracker every 30 minutes by default to request new peers and

provide transfer statistics. An initial peer request to the tracker results in

about 2-3 kilobyte of response data. If these requests are evenly spread out

temporally, the tracker can usually handle the load. However, if a particularly

desired resource is made available, this may severely strain the tracker, as it will

be subject to a mass accumulation of connections akin to a distributed denial

of service attack by requesting peers. This is also known as the flash-crowd effect [39].

It is imperative for a swarm to have a functioning tracker if the swarm is to gain new peers as, without the tracker, new peers have no location to receive new peer addresses. Tracker redundancy is currently being explored and two alternatives are studied: backup trackers and distributing the tracking functionality in the swarm itself. An extension exists to the current protocol that adds a field, announce-list to the meta-data, which contains URLs to alternate trackers. No good way of distributing the tracking in the swarm has yet been found, but a network of distributed trackers has been proposed.

Proposals of peers sending their currently connected peers to each others have also cropped up, but again, no consensus has been agreed on. Additionally, Distributed Hash Table (DHT) functionality has been implemented in third party clients to address this problem [11]. A beta version of the reference client also has support for DHT functionality.

Another important problem is the initial sharing delay problem. If a torrent has large piece sizes, e.g., larger than 2 MB, the time before a peer has down- loaded an entire piece and can start sharing the piece might be quite substantial.

It would be preferable to have the ability to have varying verification granulari- ties for the data in the swarm, so that a downloading peer does not have to wait for an entire piece to begin calculating the hashes of the data. One way to do this would be to use a mechanism known as Merkle trees [16], which allow for varying granularity. By using this mechanism, a peer may start sharing after having downloaded only a small amount of the data (on about the same order as the subpiece sizes).

3.7.1 Super Seeding

When a swarm is fairly new, i.e., there are few seeds in the swarm and peers have

little of the shared resource, it makes sense to try to evenly distribute the pieces

of the content to the downloading peers. This will speed up the dissemination

of the entire content in the swarm. A normal seed would announce itself as

3.8. SUMMARY

having all pieces during the initial handshaking procedure, thus leaving the piece selection up to the downloading peer. Seeds have usually been in the swarm longer. This means that they are likely to have a better view on which pieces are the most rare in the swarm, and thus most suitable to be first inserted.

As soon as peers start receiving the rare pieces, other peers can download them from other peers instead of seeds. This further balances the load in and increases the performance of the swarm.

A seed that employs super seeding does not advertise having any pieces at all during handshake. As peers connect to the in effect hidden seed, it instead sends have-messages on a per-peer basis to entice specific peers to download a particular piece.

This mechanism is most effective in new swarms, or when there is a high peer-to-seed ratio and the peers have little data. It is not recommended for everyday use.

As certain peers might have heuristics governing which swarms to be part of, a swarm containing only super seeders might be discarded. This is because peers cannot detect the super seeder as a seeder, thus assuming that the swarm is unseeded. This decreases the overall performance of the swarm.

3.8 Summary

This chapter has discussed the BitTorrent system in detail. It is a swarming content replication and distribution system. The BitTorrent system consists of peers and trackers. Peers are either leechers, i.e., downloading peers, or seeds, i.e., uploading peers. Data content is described by a torrent file, using a spe- cific binary encoding known as bencoding. Data transfer occurs among peers in a swarming fashion, i.e., peers share parts of the content among themselves.

Each content part has an associated SHA-1 hash to enable verification of down- loaded data. Peers that do not share data are punished by not being allowed to download.

Furthermore, protocol performance issues and current developments have

been discussed. Primarily, the load on the tracker has been identified as a

potential bottleneck.

Chapter 4

Traffic Measurements

Science is the observation of things possi- ble, whether present or past; prescience is the knowledge of things which may come to pass, though but slowly.

– Lenoardo da Vinci

Depending upon the domain, traffic measurements may serve different purposes.

For example, an Internet Service Provider (ISP) may benefit from measuring the amount of outgoing traffic to estimate pricing and the services provided. A company providing Voice over IP (VoIP) services may want to measure latencies with high accuracy to ensure a certain degree of Quality of Service (QoS), while a Web hosting provider may be more interested in metrics such as the number of requests per time unit. On the other hand, manufacturers of network hardware such as routers and switches use real-world measurements to test the behaviour of the hardware under realistic conditions without deploying them.

There are four main reasons for the usefulness of network traffic measure-

ments: network troubleshooting, protocol debugging, workload characterisation

and performance evaluation [82]. For the present work, only the last two are

considered, with a strong emphasis on workload characterisation.

4.1 Measurement Approaches

There are two main approaches to traffic measurements: active and passive mea- surements. Active measurements entails actively probing a network with either artificially generated traffic or having a node join in the network as an active participant. Probing with artificial traffic is somewhat analogous to system identification using impulses in for instance vibration experiments or acousti- cal environments. A passive measurement is one where the network is silently monitored without any intrusion.

4.1.1 Passive Measurements

Passive measurements are commonly used when data on “real” networks is desired, for instance for use in trace-driven simulations, model validation or bottleneck identification. Essentially, this technique is used to observe a live network without interacting with it.

Depending on the level of accuracy desired, different measurement options are available. For coarse-grained measurements, on a time-scale on the or- der of seconds, there is the possibility of using Simple Network Management Protocol (SNMP) and Remote Monitoring (RMON) to gather information from networking hardware. This provides only very rough information granularity and no packet inspection capabilities. It is usually used as part of normal network operations, and is not very useful for protocol evaluations and per-flow performance evaluation. Per-flow information is available in for instance Cisco’s NetFlow, but still without packet inspection.

Application Logging

Application logging is commonly used in server software to enable traceability

of errors and client requests. In certain server applications, such as critical

business systems and other high-security systems, server logs are very important

for detecting intrusion attempts and for estimating severity of security breaches.

4.1. MEASUREMENT APPROACHES

In other applications, logs are a useful tool for performance analysis.

However, client applications do not usually provide for much in terms of log- ging. If logging is made available, it usually provides rather coarsely grained in- formation, such as application start and other very high-level application events.

It is unusual that an application provides the amount of log detail needed to analyse the network performance of the application.

To provide adequate detail in application logs, it is necessary to modify the application in such a way that the application both provides the detailed event information needed and a way to store this information in a log file or database.

In applications that are based on an event-loop with a central managing component, obtaining the relevant information is a fairly easy task, as the events being handled contain all information relevant to the specific event. By adding a timestamp, these may then be ejected to a log file or database. On the other hand, in a threaded and less centralised application, this becomes a more difficult task, as events may not be handled through a single component.

An additional issue with client-side logging is deployment of the modified clients. It is important to have a large enough number of users to provide representative data. Also, not all users may agree to running a modified client.

One of the most difficult problems relates to the non-availability of client- source code. For example, most proprietary software does not provide the source code for the application, making modification impossible without substantial reverse engineering.

Log storage may become an issue if, for instance, the application is running

on an embedded system where there is no storage available except for internal

memory. Also, if measurements are performed over a long period of time and/or

there is a large number of events, the application logs may grow prohibitively

large.

Packet Capture

Application logs rarely provide network specific information, such as IP packet arrival times. Dedicated packet capture or packet monitoring units have the possibility of capturing every packet on a physical link. The packets may then be associated with application level events and messages, resulting in higher granularity of the captured messages.

Packet capture may be performed using only software or by employing spe- cialised hardware. Software configurations provide measurement accuracies on the order of tens of microseconds, while dedicated measurement hardware pro- vides nanosecond accuracy.

The arguably most commonly used passive software measurement tool for capturing live network traffic is tcpdump [40]. tcpdump is based on the pcap- library which includes the Berkeley Packet Filter (BPF) [51]. This allows tools using the library to set up complex filter rules on which packets to capture. This library is the basis for many other measurement and general network tools.

Two important issues regarding passive measurements are related to storage and computing power requirements [44]. In the case of complete flow reconstruc- tion, the entire data payload portion of the captured packets must be retained.

For each packet captured, there is an additional capture header containing a timestamp and other meta-data. To capture large flows such as P2P down- loads, the storage requirements are correspondingly large. In the case of traffic measurements on backbone networks, the amount of off-line storage needed is prohibitively large. For instance, large BitTorrent trackers measure the amount of downloaded data among their peers in peta-bytes and the number of messages to a single peer often count in the millions. The complexity of computing statis- tical measures with this amount of data makes it a challenging, if not daunting, task. It is often necessary to create specialised software to calculate statistics of interest.

Other important issues regarding capturing live traffic relate to privacy, de-

ployment and cost. Since a capturing unit has the potential of capturing all

traffic, including user data, security and privacy implications must be consid-