STOCKHOLM SWEDEN 2017,
Accurate Traffic Generation in the CheesePi Network
SAGAR SHARMA
KTH ROYAL INSTITUTE OF TECHNOLOGY
First and foremost, I express my gratitude to the Almighty who has guided me at all times during the project.
I would first like to thank my thesis advisors Dr. Ian Marsh and Dr. Liam Mc- Namara at SICS, Swedish ICT for giving me this opportunity, facilities and support needed to complete this thesis. I wish to express my sincere gratitude to Research Engineer Robert Olsson at KTH Royal Institute of Technology for being my supervi- sor and providing necessary guidance throughout the project. I am always thankful to them for giving me the intellectual freedom, supporting my ideas and demanding a high quality work in all my endeavors.
With deep sense of gratitude I thank Associate Professor Markus Hidell at KTH Royal Institute of Technology for offering me the support and keeping the door open whenever I had question regarding my research. I am gratefully indebted to him for the very valuable comments on this thesis.
I am always thankful to Mom, Dad, Yogesh and Megha for the everlasting love and support. I am thankful to Jeren for the unyeilding support, encouragement and belief in me throughout the study. This accomplishment would not have been possible without my friends and I owe a debt of gratitude to all my friends.
Sagar Sharma March 17, 2017
Abstract
Generating traffic to characterize large capacity network links with high accuracy in transport backbones is important for Internet service providers.
Using a network application (Iperf) to measure throughput (and indirectly congestion) on links within a network can be used to perform traffic engi- neering. Using short bursts at high utilisation can ascertain whether a con- nection can support capacity sensitive applications, such as video streaming.
Repeating the process to capture day-night effects can categorise links for management decisions.
CheesePi is an Internet quality monitoring system that aims to charac- terise the services obtained from the home Internet connections of the users and establish an open measurement infrastructure in Sweden. The thesis eval- uates a study of performance and characterization of network links within the CheesePi network at high data rates (Gbits/sec) and will extend the work done in the CheesePi software base.
Keywords - Network characterization, Traffic generation, Throughput es- timation
Att kunna generera trafik för karakterisering av stora nätförbindensler med hög noggrannhet är en mycket viktigt faktor för internetleverantörer. Med hjälp av en nätverksapplikation (Iperf) så kan man mäta bithastigheten samt den indirekta överbelastningen på länkar i ett nätverk. Man kan även mäta performansen av nätet för ytterligare analys. Genom att ha intensiv trafik i korta perioder så kan man få ett hum om en uppkoppling kan stödja kapacitets känsliga applikationer så som videouppspelning. I och med att internet användningen varierar så kan man även kategorisera länkarna ytterligare så som dag och natt performansen.
CheesePi är ett övervakningssystem som försäkrar kvaliteten och karakteris- erar de tjänster som erhållits för hemanslutningar samt infrastrukturs mätningar i Sverige. Examensarbetet utvärderar prestandan och karakteriseringen av nät- förbindelser med hög datahastighet (Gigabit/sec) igenom att använda CheesePi pro- gramvaran.
Keywords - Nätverkskarakterisering, Trafikgenerering, Genomströmnings upp- skattning
Abstract i
Contents v
List of figures vii
List of tables viii
1 Introduction 1
1.1 Domain problem: minimising measurement impact . . . 2
1.2 Research Questions . . . 3
1.3 Methodology . . . 3
1.4 Benefits, Ethics and Sustainability . . . 3
1.5 Thesis Outline . . . 4
2 Technical Background 5 2.1 Terminology . . . 5
2.2 Internetworking . . . 6
2.2.1 TCP/IP . . . 6
2.2.2 Transmission Control Protocol . . . 8
2.3 Bandwidth Related Metrics . . . 9
2.3.1 Capacity . . . 9
2.4 Available bandwidth . . . 10
2.5 Bulk Transfer Capacity . . . 11
2.6 Key factors affecting bulk transfer capacity . . . 11
2.6.1 Round trip time . . . 11
2.6.2 Bandwidth Delay Product . . . 12
2.6.3 Traffic . . . 14
2.7 Previous work on Traffic generation . . . 14
2.7.1 Evolution of Realistic Traffic Generation . . . 15
2.8 Current Traffic Generation Systems . . . 15
2.8.1 The Harpoon Model . . . 15
2.8.2 Tmix model . . . 16
2.8.3 The Swing Model . . . 17
2.8.4 Impact of Background Traffic . . . 18
3 Project background 19 3.1 CheesePi . . . 19
3.2 Measurement agent hardware availability . . . 20
3.2.1 Raspberry Pi . . . 20
3.2.2 ODROID . . . 21
3.3 Measurement approach for this thesis . . . 22
3.3.1 Active-Controlled hardware-based Measurements . . . 22
3.3.2 End-to-End Monitoring . . . 22
3.3.3 Wide area network traffic measurements . . . 23
3.4 Tools used for the study . . . 23
3.4.1 Iperf . . . 23
3.4.2 Ping . . . 23
3.4.3 NetEm . . . 24
3.4.4 Tcpdump packet capture . . . 24
3.4.5 Wireshark packet capture . . . 24
4 Experimental outline 25 4.1 Measurement motivation . . . 25
4.2 Throughput-based measurements . . . 25
4.2.1 Throughput measurements between MA and test machine . . 26
4.2.2 Throughput between the MAs connected using Ethernet cable 27 4.2.3 Throughput measurement between MAs connected via 100M switch . . . 27
4.2.4 Student residential network . . . 28
4.3 Throughput measurements within the CheesePi . . . 29
4.3.1 Throughput between MAs at SICS and Karlstad . . . 30
4.3.2 Throughput between MAs at SICS and Luleå path . . . 31
4.3.3 Throughput between MAs at SICS and Västerås . . . 32
4.4 Round trip time based measurements . . . 32
4.5 Network Emulation based measurement . . . 33
4.6 TCP Throughput Estimation within CheesePi network . . . 33
5 Results and analysis 35 5.1 Throughput measurement between MA and test machine . . . 35
5.2 Throughput measurements between two MAs connected using Eth- ernet cable . . . 36
5.3 Throughput measurements between MAs connected using 100M switch 40 5.3.1 Student residential network . . . 42
5.4 Throughput measurements within the CheesePi network . . . 44
5.4.1 Throughput measurements between MAs at SICS and Karlstad 44 5.4.2 Throughput measurements between MAs at SICS and Luleå . 45 5.4.3 SICS - Västerås path . . . 46
5.5 Round trip time measurements . . . 46
5.5.1 Bidirectional RTT between MAs at SICS and KTH . . . 47
5.5.2 Bidirectional RTT between MAs at Karlstad and Västerås . . 48
5.5.3 Bidirectional RTT between MAs at SICS and Luleå . . . 49 5.6 Network emulation . . . 50 5.7 Throughput Estimation within the CheesePi network . . . 52 5.7.1 Throughput estimation between MAs at SICS and Karlstad . 52 5.7.2 Throughput estimation between MAs at SICS and Luleå . . . 54 5.7.3 Throughput estimation between MAs at SICS and Västerås . 55
6 Recommendations 56
6.1 Hardware . . . 56 6.2 Performance measurements . . . 56
7 Discussion 57
8 Conclusions 58
9 Future work 59
10 Lessons Learnt 60
2.1 Conceptual layers of TCP/IP . . . 7
2.2 TCP Sliding Window Mechanism . . . 12
2.3 Sliding mechanism with two different window sizes . . . 13
2.4 An x-y-t diagram illustrating a sequential connection . . . 17
2.5 An x-y-t diagram illustrating concurrent connection . . . 17
3.1 Geographical location of nodes within CheesePi network . . . 19
4.1 Test Machine and MA connected using Ethernet cable . . . 27
4.2 Back-to-back Ethernet connectivity between two MAs . . . 27
4.3 Two MAs connected through a switch . . . 28
4.4 MA at SICS connection to MA at Student Accommodation . . . 28
4.5 Geographical location of MA installed at location, Kista and Karl- stad. The green marker represents SICS and the orange marker rep- resents Karlstad University. . . 30
4.6 Geographical location of MA installed at location, Kista and Luleå. The green marker represents SICS and the red marker represents Luleå University of Technology. . . 31
4.7 Geographical location of MA installed at location, Kista and Västerås. The green marker represents SICS and the blue marker represents Mälardalen University University, Västerås . . . 32
5.1 Maximum TCP throughput between MAs connected back-to-back . . 37
5.2 Applied Bandwidth versus Achieved Throughput using UDP between two MAs connected back-to-back . . . 38
5.3 Applied Bandwidth versus Jitter between two MAs connected back- to-back . . . 39
5.4 Applied Bandwidth versus Jitter between two MAs connected back- to-back . . . 40
5.5 Applied bandwidth vs achieved bandwidth between two MA’s con- nected using a switch . . . 41
5.6 Applied bandwidth versus jitter between two MAs connected using a switch . . . 42
5.7 Applied bandwidth vs achieved throughput within student network using Raspberry Pi 2 . . . 43
5.8 Applied bandwidth and Jitter between MAs at SICS and Student Accommodation . . . 43
5.9 24 hours (12pm to 12pm) of Iperf TCP throughput measurements between MAs at SICS and Karlstad on Saturdays of weeks 22 (blue) and 23 (red). Each hour was sampled 3 times with the median ±1 median absolute deviation. . . 44
5.10 24 hours (12pm to 12pm) of Iperf TCP throughput measurements between MAs at SICS and Luleå on Saturdays of week 36. Each hour was sampled 3 times with the median ±1 median absolute deviation. . 45 5.11 24 hours (12pm to 12pm) of Iperf TCP throughput measurements
between MAs at SICS and Västerås on Saturdays of week 37. Each hour was sampled 3 times with the median ± median absolute deviation. 46 5.12 12 hour bidirectional RTT measurement between MAs at SICS and
KTH. . . 47 5.13 12 hour bidirectional RTT measurement between MAs at Karlstad
and Västerås. . . 48 5.14 12 hour bidirectional RTT measurement between MAs at SICS and
Luleå . . . 49 5.15 Emulated throughput vs actual throughput between MAs at SICS
and Karlstad . . . 50 5.16 Emulated throughput vs actual throughput between MAs at SICS
and Luleå . . . 51 5.17 Emulated throughput vs actual throughput between MAs at SICS
and Västerås . . . 51 5.18 Absolute error in throughput estimation between MAs at SICS and
Karlstad . . . 53 5.19 Absolute Error in throughput estimation between MAs at SICS and
Luleå . . . 54 5.20 Absolute Error Estimation between MAs at SICS and Västerås . . . . 55
1 Internetworking layer functionality . . . 7
2 Location of the measuring agent and connectivity . . . 20
3 Technical Specification of Hardware within CheesePi Project . . . 21
4 Technical specification of test machine (Laptop) . . . 26
5 Maximum throughput between test machine and MA . . . 35
6 Maximum TCP throughput between MAs connected back-to-back us- ing Ethernet cable . . . 36
7 Maximum TCP throughput between MAs connected through a 100M Switch . . . 40
8 Statistics pertaining to bi-directional Round Trip Time between MAs at KTH and SICS . . . 47
9 Statistics pertaining to bi-directional Round Trip Time between MAs at Karlstad - Västerås . . . 48
10 Statistics pertaining to bi-directional Round Trip Time between MAs at SICS - Luleå . . . 49
11 Absolute error in throughput estimation between MAs at SICS and Karlstad calculated for the duration of data transfer . . . 53
12 Absolute error in throughput estimation between MAs at SICS and Luleå calculated for the duration of data transfer . . . 54
13 Absolute error in throughput estimation between MAs at SICS and Västerås calculated for the duration of data transfer . . . 55
14 Pros/cons of the protocol used for throughput measurements. . . 57
1 | Introduction
The annual global Internet traffic is going to surpass the zettabyte threshold by 2016 [Cisb]. With the explosion of Internet traffic, characterization of the network with parameters such as the link capacity, round trip delay and available bandwidth is essential. Internet traffic measurement and characterization is an essential task in order to understand and solve performance-related issues of current and future networks. Many previous efforts have focused mainly on statistically characterising network traffic [Pax04].
In order to characterize network parameters, such as link capacity or link through- put, generally requires injection of traffic into the network path. One of the simplest ways to generate network traffic is to transfer a file between two endpoints and mea- sure the network parameters associated with that transfer. While an HTTP source approach provides some insights about the capabilities of the network under test, it does not provide the configurability of a packet by packet test. Also one is ex- posed to the complexities of TCP, the socket interface and the HTTP application itself. An alternative approach to producing network traffic is to generate a stream of packets separated at constant intervals A simple sequence of packets can be used to measure the delay, loss and throughput of the underlying network. One tool that make use of both TCP file transfer and a sequence of packets is iperf.
The purpose of this thesis is to study the achievable throughput on an end-to-end path using a dedicated end device over an extended period of time. The dedicated end devices generate traffic within the network by an active measurement approach.
By generating traffic between endpoints, we characterise the true networking per- formance.
The data transport protocols considered for the measurements are the Transmis- sion Control Protocol (TCP) and the User Datagram Protocol (UDP). By gradually increasing the applied data rates for a short amount of time at the sender and mea- suring the corresponding throughput at the receiver we will obtain data to quantify the throughput. With the quantification of networking capability of the hardware used, we aim to extend the measurements in a wide area environment.
Traffic can be generated using different hardware platforms:
• An Application Specific Integrated Circuit (ASIC)
• A Network Processor (NP) and specialized software
• A General Purpose Processor (GPP) and specialized software
Using ASICs it is possible to generate high bandwidths of traffic with high ac- curacy, however they introduce complexity in design of hardware and reusability. A
network processor achieves higher bandwidth of traffic but are generally uneconom- ical. For our study, a GPP with community software is best suited as they provide flexibility and reusability. However, a study of supporting data rates of the GPP is necessary, and thus is a core contribution of this work.
Network path measurements load the network path with additional traffic. There- fore to limit the extra (redundant) traffic we study the minimal time to run such tests. In other words loading an operational network unnecessarily is considered bad practice and may cause unwanted backoff behaviour for the exisiting flows.
Essentially, this study is an effort to infer the network path characteristics within the CheesePi network which consist of small (size, energy and cost) measurement devices.
1.1 Domain problem: minimising measurement impact
High capacity active measurements over extended periods of time are not consid- ered network ’friendly’. Minimal resource usage with representative (in terms of real usage) throughput tests is what we would like to achieve in this project. The following are the challenges faced while performing our study:
1. Measurement Duration: To study the effects of varying network conditions and its impact on network parameters, an extended set of measurements is necessary. One concern is to define the duration of throughput measurements.
2. Hardware: It is important to study the performance of the host hardware in terms of its capability to process/generate the incoming/outgoing data.
The hardware manufacturers specify the networking capability of the designed hardware, but generally the achievable rates differ from those specified rates.
A multicore based architecture adds complexity in achieving the highest rates.
3. Time of the day for measurements: It is important not to disrupt the existing traffic during the throughput measurements. As the project makes use of the resources provided by universities, we have taken care to schedule the measurements, essentially at night-time and weekends.
4. Traffic type: Performing throughput measurements during the busy-hours may appear as denial-of-service attack by intermediary network equipment.
The preferred choice for wide area throughput measurements is the TCP pro- tocol (however see Discussion) as it least implements traffic control.
1.2 Research Questions
The research questions addressed in this study are:
1. By generating traffic in a network for a short amount of time, what is a suitable measurement duration to make an estimation of the representative throughput values?
2. What is the trade-off (error in estimation) between the duration of throughput measurement and accuracy in estimation.
The limitations of this work are restricted to quantify the throughput mea- surements during the busy-hours. A weekday-weekend comparison of achievable throughput is worthwhile to study the influence of daily traffic load variations.
1.3 Methodology
We analysed the networking capability of the hardware to perform performance network measurements. An experimental study of the factors affecting the network throughput have been conducted. Incrementally increasing the complexity within the system under test, from benchmarking individual units to WAN deployment, the performance of hardware units and to some degree parts of the Swedish academic network have been evaluated.
Making use of the location of test devices, the distance between them, test du- ration, time of day, the performance measurements have been characterised on an end-to-end path. We have also emulated a network to carry out tests, so as not to use time setting up a real network. Importantly, we have identified the trade-off between throughput measurement duration and accuracy in estimation.
1.4 Benefits, Ethics and Sustainability
The social aspects of this project are beneficial because:
• Users are provided with a statistical summary plus a graphical view of an individual Internet connection.
• ISPs can diagnose network issues affecting the performance of several Internet connections
• Regulators can monitor SLA violations and hence review country-wide network performance
Regarding the ethical aspects, the data generation and collection is totally objec- tive to measuring the quality of service. The project introduces its own traffic within the network and measurements are done with respect to the same for performance measurements. No user generated traffic is captured and assuring the privacy of the customer. Since the measurement load the network with additional traffic, care is taken to prevent the additional use of the resources. We do not keep any user data and will remove the collected data after 6 months of analysis.
The average power consumption of Raspberry Pi 1 model B is 3.5 Watts and at this rate it would require 285 hours to use 1 kWh (kilo Watt hour) of power. The average CO2 emission per kWh is 0.45 kg . In 24 hours Raspberry Pi 1 model B emits upto 0.03 kg/day of CO2. Similarly Raspberry pi 2 model B (4 Watts) and ODROID-XU4 (10 Watts) emits about 0.04 kg/day and 0.1 kg/day respectively of CO2. A typical desktop computer with 100 W power rating, emits about 1 kg/day of CO2. The environmental aspects of the project are comparatively insignificant.
Also, there are no negative ecological impacts caused due to the project.
1.5 Thesis Outline
The organization of the report is as follows.
• Chapter 2: The necessary technical background to understand the thesis work
• Chapter 3: A detailed description of the project background, measurement approach and the tools used for the measurements
• Chapter 4: An outline of the measurements conducted
• Chapter 5: The results from the measurements are interpreted and analysed
• Chapter 6: In this chapter, we recommend network performance measurement hardware
• Chapter 7: The outcomes of this study are discussed
• Chapter 8: In this chapter we draw some conclusions
• Chapter 9: We suggest future work.
• Chapter 10: the lessons learnt during the thesis are stated
2 | Technical Background
This chapter provides the technical background. In short, this chapter explains the relevant TCP/IP protocols behaviours, and the network metrics used in this work. Additionally, a review of previously conducted research, and their outcomes is described. First, a short description of the terminology used is outlined.
2.1 Terminology
• IP address - This is an address used to identify a network interface and its location on a TCP/IP network [KR12].
• Network Packet - A unit of data that is routed between a source and desti- nation on the Internet [KR12].
• TCP flow - A TCP flow, or a TCP connection, consists of setting up a virtual end-to-end connection dedicated to a pair of source and destination hosts which request to communicate, transferring data segments through that connection and terminating that same connection when communication ends [KR12].
• Bandwidth - quantifies the rate the network link or network path can transfer data. It has two characteristics -physical and available, and both of them are independent of end hosts and protocol type [LoEOoSI03].
• Physical Bandwidth or Capacity -is the maximum number of bits per second a network element can transfer. The physical bandwidth of an end- to-end path is determined by the slowest network element along the path [LoEOoSI03].
• Available Bandwidth - is the difference between capacity and utilization over a given time interval. This is applicable to paths, links, routers or switches [LoEOoSI03].
• Throughput - the amount of data that is successfully sent from one host to another via a network path. It may be limited by every component along the path from source host to destination host, including all hardware and software [LoEOoSI03]. Throughput also has two variants achievable and maximum.
• Maximum Throughput - is the best transfer rate that can be success- fully performed between two end hosts when they are connected back-to-back [LoEOoSI03].
• Achievable Throughput - is the throughput between two end points under a completely given set of conditions, such as transmission protocol, end host hardware, operating system, tuning method and parameters. This characteris- tic represents the performance that an application in this specific setting might achieve. Since the bottleneck could be in an end host, achievable throughput may or may not correlate with available bandwidth [LoEOoSI03].
• Bottleneck Bandwidth - refers to the lowest bandwidth along the complete path [SFGC11].
• Jitter - is defined as the variation in the delay of received packets if the sending side transmits packets in a continuous stream spaced evenly apart. Because of network congestion, improper queuing, or configuration errors, the delay between packets can then vary instead of remaining constant.
• Median and Median Absolute Deviation (MAD) - To aggregate the results obtained from the measurements the study makes use of median and median absolute deviation (MAD) [LLK+13]. Median is the central represen- tation of the values and is insensitive to the outliers. Outliers affect the mean if the data set is small. To calculate median the data is arranged in numerical order and the middle value is the median if the data set consist of odd numbers and mean of two middle values if the data set has even numbers. The median absolute deviation is median of absolute deviation from the median. For a data set [X1, X2, X3, ..., Xn] the MAD is defined as follows:
M AD = median(|Xi− median(X)|)
2.2 Internetworking
The Internet is a global computer network consisting of interconnected networks.
There was a need to interconnect many physical networks and make them look like one functional unit. The technology, Internetworking, accommodates intercon- nection of heterogeneous networks hiding the network hardware and allowing these networks to communicate with each other [Com91].
2.2.1 TCP/IP
TCP/IP forms the base technology that connects the global Internet and was funded by the Advanced Research Project Agency [Com91]. TCP/IP includes a set of conventions for interconnecting networks and routing traffic. The TCP/IP protocol suite software has five conceptual layers, four belonging to software and a layer of hardware as shown in figure 2.1.
Figure 2.1: Conceptual layers of TCP/IP
Layer Description
Application Provides a service to a user enabling communication over TCP/IP and interacts with the transport layer protocols to send/receive data.
Transport End-to-End communication between two hosts regulating the flow of information, two variants exist TCP/UDP.
Internet Communication from one machine to another, accepting requests from transport layer with the identification of machine to
which the packet has to transport layer with the identification of machine to which the packet has to be delivered and
encapsulating the datagram to appropriate network interface Network Responsible for accepting IP datagram and transmitting them
over a specific network. The physical address of the current and next hop hardware are registered to enable data transmission on the medium.
Table 1: Internetworking layer functionality
2.2.2 Transmission Control Protocol
The Transmission Control Protocol (TCP) is a communication protocol providing connection-oriented, error checked and byte ordered stream service. TCP allows application programs to communicate concurrently. TCP establishes a virtual circuit connection between two application programs by defining endpoints consisting of the host and TCP port number.
TCP provides the following facilities:
• Stream Data Transfer: When two application programs transfer data, the stream delivery service transfers the stream of bytes into segments which are transmitted onto the IP network. The segmentation of the data is decided by TCP itself and it may forward the data as per its own convenience.
• Reliability: The fundamental technique for TCP reliability is positive ac- knowledgement with retransmission. The technique requires a recipient to communicate with the source, sending back an acknowledgement on the suc- cessful reception of data. The sender keeps a record of the transmitted packet waiting for acknowledgement before sending the next packet. When the packet is sent, the sender starts a timer during which it expects an acknowledgement and retransmits the packet if the timer expires before receiving the acknowl- edgement.
• Flow Control: During data transfer the receiver TCP, along with acknowl- edgements also advertises the number of bytes it can accept. In response to an increased window advertisement, the sender increases the size of its win- dow to send. In response to decreased window size advertisement, the sender decreases the size of its window to send. The variable window size provides an advantage by providing a flow control needed for reliable data transfer. The idea of variable window size is essential in Internet as it is required to commu- nicate with various machines operating at various speed and capacities.
• Congestion Control: There is a need of control mechanism for the inter- mediate sources (e.g. routers) forwarding the traffic, to limit traffic more than that they are capable of handling. The mechanism to control the traffic is called congestion control. When the intermediate resources are overloaded and experiences a packet loss, the sender decreases its transmission rate during data transfer.
• Multiplexing/Demultiplexing: To allow multiple application processes running on a source host machine and simultaneously requiring to communi- cate with the same destination machine, TCP at source host machine combines the multiple data streams into a single multiplexed stream for transmission.
At the destination host the received stream is demultiplexed to its correct ap- plication. Within each host, TCP assigns a set of addresses and concatenates with network and host drivers to form a socket. Each socket then identifies a connection.
• Full Duplex Byte Stream Transfer: TCP allows concurrent transfer in both directions called full duplex connection. Full duplex consist of two inde- pendent streams flow in opposite directions. There is no interaction between the two independent streams. TCP also supports flows in one direction known as half duplex and allowing termination of one flow. TCP reduces traffic by sending control information for one stream back to the source in datagram carrying information in the opposite direction. The sender segments data on a byte stream and assigns them a sequence number. The receiver extracts the bytes from the segments and transfers it to the application process [Com91].
2.3 Bandwidth Related Metrics
In this section we introduce three bandwidth related metrics: capacity, available bandwidth and bulk transfer capacity. These metrics measure the rate of information transfer and are measured in bytes/bits per second.
2.3.1 Capacity
Consider two hosts: Source (S) and Sink (R). The source sends packets to the sink through a packet network infrastructure P(S,R). The maximum throughput pro- vided by a path to a flow is known as the capacity, given the underlying network conditions and the forwarding elements. Each link on the path can forward packets with a maximum throughput depending on its link capacity. Each link is further de- fined by its physical hardware and computational resources. A link that determines the capacity of the path is referred to as the bottleneck link of the path. Forwarding elements within the network due to processing limitations, have different throughput values for different packet sizes. Forwarding elements also include traffic shapers, which only allow certain transmission burst sizes [PDMC03].
The following are the uses of capacity metrics:
1. The path capacity information can be used by the ISPs to help diagnose the problems faced at the intermediate network elements.
2. The path capacity information can be used for service level agreements (SLA) verification by the user or ISPs.
Measurement Approaches for path capacity
1. Loading the path with back-to-back packets: This method is considered to be intrusive in the presence of cross traffic, as the packets gets multiplexed with packets from cross traffic in the network queues.
2. Packet-Pair dispersion: In this method two packets are dispersed within the network back-to-back to estimate the capacity of a path. This method is not considered to be statistically robust as it is difficult to estimate the path capacity over a heavily loaded path. Few tools that use this approach are bprobe, nettimer, pathrate.
3. Variable Packet Size: In this technique estimates of each link in the path are made by measuring the variation of the RTTs as a function of probing packet size and therefore inferring the overall path capacity. However, this method results in underestimation errors if the path consists hidden layer 2 forwarding elements. Some tools that use this technique are pathchar, pipechar and nettimer.
2.4 Available bandwidth
The available Bandwidth A of a link in a time interval T is the part of link’s capacity that is unused. The available bandwidth of a path is dictated by the link with minimum available bandwidth along the path. The link in a path with minimum available bandwidth is referred as a bottleneck link. The available bandwidth is a progressively shifted entity. The variance in the available bandwidth is a function of the burstiness within the traffic. Available bandwidth at a time interval T 1 can be used as a predictor for available bandwidth in future time for time interval T 2, given the underlying traffic is stationary.
The available bandwidth is often misinterpreted as the throughput. A saturated link with fixed number of buffers can introduce losses and queuing delays. Hence, it is desirable for the flow to not set the transmission rate to that of available bandwidth.
If there are no queuing delays or packet losses it is possible to achieve throughput greater than the available bandwidth during that time interval[PDMC03].
From a network operator point of view it is important to monitor the network load. This is especially true as protocols such SNMP is not accessible for network users. One of the ways to measure the available bandwidth of a path is by using SNMP data from the routers but unfortunately many network providers do not advertise the loads on their network (for confidentiality purposes). Tools such as cprobe and nettest attempt to measure the available bandwidth using end-to-end packet dispersion. However it has been shown in a study that this methodology
measures the dispersion rate that is not relative to available bandwidth, because of the bottleneck link within the path [DRM01].
2.5 Bulk Transfer Capacity
The Bulk Transfer Capacity (BTC) of a path is a maximum throughput obtained during a TCP connection and is influenced by the underlying network conditions of the path. BTC is influenced by many factors (see section 2.6) and is dynamic varied over time. It is important to note that the BTC is TCP specific only and must not be misinterpreted as available bandwidth which is independent of transport protocol.
To measure BTC, the connection implementation should follow all the description of RFC 2581 [APS99].
BTC is determined by several links in the path unlike available bandwidth and capacity. The throughput is influenced by the RTT, end-to-end loss rate, further any link that adds queuing delay can affect the BTC measurement. BTC bottleneck may or may not correspond to the bottleneck link as an under-buffered link can cause multiple packet losses and it is not necessary that BTC is influenced available bandwidth. If the path is completely saturated (available bandwidth = 0) but the BTC will still have a throughput as TCP shares the capacity of the path with other connections. However, it is important not to affect the cross traffic during measurements and maintaining utilization of the link [PDMC03].
The following are the uses of BTC metric:
1. provides important file transfer information about the path and the BTC is the primary metric the user is interested in.
2. helps in determining the throughput in an end-to-end path using TCP’s con- gestion control algorithm and can be used by non-TCP applications (stream- ing).
3. can be used for network monitoring, debugging and SLA verification, provided there is no cross traffic.
2.6 Key factors affecting bulk transfer capacity
The following are the key factors affecting the bulk transfer capacity.
2.6.1 Round trip time
Round trip time (RTT) is the time interval between the sending of a datagram from source host to destination host and the receipt of the response packet from the destination host to source host. RTT essentially quantifies the propagation delay,
processing of the datagram at the source/destination host and the acknowlegdement.
The RTT is computed at the source host by subtracting the timestamp of the out- going datagram with the respective incoming timestamp acknowledgement and does not require any clock synchronisation [KP87]. A low RTT makes a significant differ- ence on the throughput. When using TCP data transfer, the achievable throughput, can be roughly calculated from the window size and the RTT [LoEOoSI03]:
T hroughput(bits/sec) = W indowSize(bits) RT T (sec)
As per RFC 6349 [SFGC11] the RTT measurements should be based during the off-peak hours to prevent the additional delay introduced by the network during the peak hours.
2.6.2 Bandwidth Delay Product
The Bandwidth Delay Product (BDP) of a path defines the number of unacknowl- edged data packets TCP can have in-flight in order to fully utilise the available bandwidth. BDP is a metric to measure the capacity of a network pipe. BDP refers to the product of a data link’s bandwidth (in bits per second) and its end-to-end delay (in seconds). During a TCP data transfer mechanism, a client can send a window of packets and has to wait for an ACK from the server.
Figure 2.2: TCP Sliding Window Mechanism
In figure 2.2 above, the client (C) sends 4 packets as the window size and has to wait for an acknowledgement for the first packet sent from the server (S) before it
can send more packets to the server. After receiving the acknowledgement for the first packet, it can send the fifth packet and so on. Considering a single link, the time to receive an ACK depends on the packet size (Data and ACK) that is the RTT, rate at which data is being transmitted and propagation delay (propDelay).
RT T = Datarate + propDelay +ACKrate + propDelay
During the ideal conditions, i.e no packet loss, the receiving rate is identical to the sending rate and more over if we ignore the packet headers, the receive rate is identical to the sending rate. We know that the TCP window size is dependent on the flow control and the congestion control. Ignoring the congestion control, we denote the available buffer size at the receiver as awnd. Furthermore, if the ACKs are of a very small value, we have the above equation as,
RT T = Datarate + 2(propDelay)
Figure 2.3: Sliding mechanism with two different window sizes
The above figure 2.3 shows two cases of different window sizes in the TCP sliding mechanism. On the left side of the figure 2.3 the window size is 4 and on the right side the window size is 8. We are interested in determining the conditions during which the throughput is maximum. In the case of TCP window sliding mechanism with window size as 4, the client transmits the smaller packets and waits for the ACK from the server to transmit more as the time to transmit the window is smaller than the time to receive the first ACK. On the other hand, with a window size as 8,
considering the time to transmit the window is greater than time to receive the first ACK from the server. Soon after the client has transmitted the ACK, it already has the first ACK and ready to transmit the next packet and so doesn’t wait for the ACK. Therefore, the conditions for maximum throughput is said to be when the time to transmit the window, awnd, is equal to or greater than the time to receive the first ACK i.e the RTT.
RT T ≤ awndrate
Rearranging the above equation, awnd ≥ rate ∗ RT T , notice the right hand side is the BDP we described earlier .
2.6.3 Traffic
The traffic on an Internet link may not be stationary (statistical properties such as mean, median variance are constant over time) and is likely to be non-stationary over a period of time. For a shorter period of time it may be stationary based on the traffic on the link. These cause interesting impacts on the throughput characterisation.
2.7 Previous work on Traffic generation
Traffic generation has remained a key challenge in the field of experimental network- ing during the last two decades. The key goal of traffic generation is to test and ensure the developed mechanisms such as congestion control before deploying it to the Internet. This is achieved by running experiments in a laboratory and producing realistic traffic to produce reliable results. The state-of-art in generating realistic traffic today consists of measuring traffic on a real link and using of many ways to replay this traffic in the laboratory. Traffic generation is used in network simulators and emulators. Traffic generation in experimental networking is broadly classified into environments: simulation and emulation. Emulation is further classified into:
• Controlled and repeatable experiments in laboratory
• Live Internet Experimentation
Initially, the network research community who had developed simulators had very narrow and specific goals and a standard framework was needed by set of diverse network researchers to increase the reliability and acceptance of results. Network simulator (NS) was one of the efforts in creating this framework [BEF+00]. More recently emulation testbeds such as Emulab [Emu], Wan-in-lab [iLp] and ModelNet [Mod] were developed.
2.7.1 Evolution of Realistic Traffic Generation
One of the most common challenges of simulation is the generation of traffic which emulates the characteristics of the network path during the experimentation. Floyd and Paxson [FP01] analyse this problem in simulating the network traffic to promote the research in this area. One of the simplest methods of generating traffic is done by injecting packets into network and the approximation is called packet-level traffic generation [DPV06]. Packet-level characterization includes the reproduction of exact packet-size and arrival times of the packets noticed during the experimentation on a real production link.
However, packet-level traffic generation has a few limitations such as inflexibility.
In this method, there is no way to modify the packet size during the experiments.
Another shortcoming of packet-level traffic generation is that it does not preserve the properties such as congestion control and flow control. On the other hand capturing the application level traffic is essential for realistic traffic experimentation.
Application-level traffic modelling is used on top of the network and captures traffic by implementing flow control and congestion control. Previously, application-level modelling was unidirectional i.e for each TCP connection, the sender and receiver establish a connection and sender constantly sends data and receiver constantly receive data. The modelling is simple with no parameters.
The drastic growth of the web changed the traffic characteristics of network links.
The web is dominated by the request-response exchanges type of connections on the links. The unidirectional was no longer an interpretation of realistic traffic as most of traffic on the network is now bidirectional.
2.8 Current Traffic Generation Systems
In the early 1990’s, most of the traffic generation were limited to certain type of application protocol such as FTP, SMTP and HTTP [BC98]. The real links carries a mixture of continuously evolving traffic and this was one of limitation of the traffic generation models in 1990’s.
We are going to discuss three models in this section.
• The Harpoon Model
• Tmix Model
• Swing model
2.8.1 The Harpoon Model
The Harpoon model [PW00], was a landmark contribution in the field of traffic generation as it was the first one to address the issue of representing a complete
set of applications empirically using the TCP and UDP transport protocol without the knowledge of application protocol or port usage. Harpoon modelling basically used the NetFlow [Cisa] records for all the flows traversing within a network and representing the source-destination IP addresses pair, relative start time, total bytes transferred in each direction as seen at the router and protocol used. Harpoon uses this data to generate TCP and UDP flows with the same characteristics as observed at the router.
Each connection in this model is defined by file size transferred and time between file transfer. The connections are based on the 5-tuple flow: (source IP address, destination IP address, source port, destination port, protocol used). The sessions are further divided to TCP and UDP types that conduct data transfers using the protocol defined. The session flows are 3-tuple flow: (source IP address, destination IP address, protocol used).
Harpoon produces the temporal patterns which are similar to real traffic noticed on production links by matching the byte, packet and flow volumes from the original data (NetFlow). One significant limitation of the Harpoon model is that they ignore the time dimension and model the size dimension. The time dimension is a very crucial role in the outcome of experiments. Harpoon model further discards smaller flows such as ACKs. Also Harpoon model discards the incomplete flows where the SYN or RST/FIN markers were not observed. The Harpoon model does not recreate the trace characteristics of RTT, receiver window size or losses observed [AJS12].
2.8.2 Tmix model
Tmix[WAHC+06], uses the information about the TCP sequence and ACK number exchanged present in the TCP header trace to characterise connections as a sequence of request-response exchanges between endpoints. The number of exchanges, the time between the request and response and the data in each direction are the repre- sentation of the request-response exchanges. The Tmix model reproduces the RTT, start-time, receiver window size and packet loss found during the original trace. A model of all TCP connection is built including the headers. x-y-t are a set of connec- tion vectors that are used by the traffic generator to reproduce the characteristics of the application-level modelling. The x’s and y’s are the application data units (ADUs) which are recorded using the trace from the real production link. The t’s represent the elapsed time between a request and its response. The response from server represents intra-epoch latency and the response between requests represents inter-epoch latency. Tmix essentially captures the application behaviour between connections that exchange data sequentially and concurrently as shown in figure 2.4 and figure 2.5. Every request-response represents an epoch within connection and every connection consist of one or many epochs. Although Tmix model is flexible, it cannot reproduce the characteristics of UDP flows[AJS12].
Figure 2.4: An x-y-t diagram illustrating a sequential connection
Figure 2.5: An x-y-t diagram illustrating concurrent connection 2.8.3 The Swing Model
The swing model [VV06], observes traffic at a single point in the network and auto- matically extracts details about the network behaviour, link behaviour, link capacity and loss rates. The swing model then generates traffic similar to the original trace corresponding to fundamental models in network emulation running the commodity network protocol stacks. Swing defines the request-response exchanges as RREs, where the base request for a web page which is accompanied by several image down- loads as a part of the request and its responses are of the same RRE. Although they are considered as different connections within same RRE. Swing emulates the network path using ModelNet [Mod], where every packet to routed to the ModelNet core. Swing reproduces traffic similar to the burstiness of the original traffic on
the real production link for both bytes and packets bidirectionally and shown for a variety of applications. The generated traffic also corresponds to burstiness in inter- packet arrival process with the original trace at a variety of timescales. The authors of swing wanted to generate traffic which is representative of the real traffic within the network and independent of a pattern of communication in the original trace.
Swing was intended to permit experimentation with changing load and application characteristics. Although swing is also very flexible it has limitations such as swing is not application independent unlike the Tmix and Harpoon. Swing also does not consider incomplete connections [AJS12].
2.8.4 Impact of Background Traffic
Despite the traffic modelling tools there also have been attempts to show that simply the presence of background traffic makes a significant difference in the outcome of experiments. In the studies [HLRX07], the authors show the effect of background traffic in experimental evaluations of networks. The authors state ’it is insufficient to take simple models of Poisson in experimental evaluations and the effects of background traffic on the TCP protocol evaluation’. The authors also demonstrate stability, link utilisation, fairness of protocols are influenced by the RTT, flow size and background traffic competing for higher speed flows in a bottleneck element.
3 | Project background
In this chapter we introduce the project and traffic monitoring background employed by this project. This study adapts a quantitative methodology[Håk13] and various networking tools to perform measurements. A quantitative approach helps to sta- tistically generalize a finding and look at the relationship between variables as we move from the domain of knowns to unknowns. In this chapter we mention the hard- ware used for measurements, location of the hardware, type of connectivity. The measurement approach for this thesis is active- controlled measurement. Finally the tools used for this thesis is described.
3.1 CheesePi
CheesePi is an Internet quality monitoring system that aims to characterise the service users obtain from their home Internet connection and establish an open measurement infrastructure in Sweden. The hardware used for the measurements are Raspberry Pi and ODROID. The hardware used for measurements are referred as the measuring agents (MAs). A measuring agent enables continuous and reliable monitoring of the user’s Internet connection. Table 2 indicates the hardware used at different locations across Sweden and the connectivity of the hardware. Figure 3.1 shows the geographical location of the nodes within the CheesePi network. All the measurements for this thesis were conducted at the following nodes within the CheesePi network.
Figure 3.1: Geographical location of nodes within CheesePi network
Measuring Agent Connection type Location Raspberry Pi 1 model B Ethernet SICS (Kista) Raspberry Pi 2 model B Ethernet SICS (Kista) Raspberry Pi 1 model B Ethernet Karlstad University,
Karlstad Raspberry Pi 1 model B Ethernet Luleå University
of Technology, Luleå Raspberry Pi 2 model B Ethernet Mälardalen University,
Västerås Raspberry Pi 1 model B Ethernet KTH (Kista) Raspberry Pi 2 model B Ethernet Student
Accommodation, Kista
ODROID XU4 Ethernet SICS (Kista)
ODROID XU4 Ethernet SICS (Kista)
Table 2: Location of the measuring agent and connectivity
3.2 Measurement agent hardware availability
In this section, we mention the hardware used for this thesis and their technical specification.
3.2.1 Raspberry Pi
A Raspberry Pi is a small, cheap, credit card-sized computer and was developed with the intention of creating enthusiasm in computer science related topics and aptitudes in young people. The Raspberry Pi caters to an extensive variety of programming from simple to more complex hardware projects. The operating system used for the study is Raspbian, based on the Debian Linux distribution with CheesePi image pre- installed and configured. The distribution comes with pre-installed programming languages such as Python and Scratch. Raspberry Pi accommodates a number of general purpose input/output pins that reads input from variety of sensors and outputs to hardware such as servo motors and LED’s. Furthermore, the Raspberry Pi has a large user base community developing projects and resources making it possible to explore different applications. Raspberry Pi has many hardware models [BFT15]. The Raspberry Pi models used for this project are Raspberry Pi 1 Model B and Raspberry Pi 2 Model B. For technical specification of please refer the following table 3.
3.2.2 ODROID
ODROID is a small, powerful, energy efficient, single board computers. ODROID of- fers support from open source communities and can run various distribution of Linux.
ODROID was developed at Hardkernel, an open source community. ODROID sup- ports high transfer speeds to support advanced processing power on ARM devices.
ODROID supports faster booting, faster web browsing, networking and 3D games.
The operating system used for the study is Ubuntu mate 15.10, GNU Linux distri- bution with CheesePi image pre-installed and configured. For technical specification of please refer the following table 3.
Model Raspberry Pi 1 Raspberry Pi 2 ODROID-XU4
Model B Model B
Price 270 SEK 300 SEK 620 SEK
Processor Type Broadcom, Broadcom, Samsung
BCM2835 BCM2835 Exynos5422
ARMv6 SoC ARMv7 SoC Cortex A7
Processor Speed Single Core Quad Core Octa Core
700 MHz 900 MHz 2GHz
Memory Speed 256 MB 1 GB 2 GB LPDDR
Storage SD card MicroSD eMMC 5.0
Flash Storage Ethernet Port 100 Mbits/sec 100 Mbits/sec 1 Gbits/Sec
USB Port 2 X USB 2.0 4 X USB 2.0 2 x USB 3.0, 1 USB 2.0
GPIO Port 30 pin 40 pin 30 pin
Video HDMI, HDMI, 1080p via
Composite RCA Composite RCA HDMI cable Audio Multi-channel HD Multi-channel HD
via HDMI, via HDMI, None
stereo 3.5mm jack stereo 3.5mm jack
Power 5V, 700mA 5V Micro USB 5V, 4A via
Micro USB or 800 mA Micro USB or
GPIO header GPIO header
Size 85 x 56 x 21 mm 85 x 56 x 21 mm 82 x 58 x 22 mm Table 3: Technical Specification of Hardware within CheesePi Project
3.3 Measurement approach for this thesis
For the purpose of network performance monitoring, a software-based measurement technique is adopted using Python scripts to automate the measurement process at different times of the day. The Python scripts reside on user space of the MAs and initiate the measurement tools used for the network performance measurement. The measurement tests include the data transfer between two MAs using the TCP/UDP transport protocol and round-trip time measurements. The observed results are simultaneously stored in a comma separated file (CSV). From the observed results inferences about the network state is made. Further, the following methodologies are adopted in this study:
3.3.1 Active-Controlled hardware-based Measurements
Active measurements are done by generating packets that are sent over the network to determine the network path characteristics. Due to the added load on the network the measurements utilizes the available bandwidth. The traffic generated by the ac- tive measurements may disrupt the normal functioning of the traffic flows within the network. This leads to intrusiveness caused within the network due to performance measurements and is undesirable. Although intrusive, active measurements enable ease of monitoring at both the end points and provide measurement accuracy of the obtained results. Active measurements provides control over the scheduling of performance measurement during the non busy hours of the day [MRK].
In this study Active-controlled hardware-based measurement methodology is fol- lowed. A hardware-based measurement methodology enables continuous and non- intervened measurements. The active measurements are controlled by performing the active measurements during the non-busy hours with minimal traffic on the link.
The measurements are further sampled to avoid intrusiveness.
3.3.2 End-to-End Monitoring
In a network of unknowns (delay, packet loss, throughput, etc.) it is desirable to monitor the end-to-end characteristics to perform traffic engineering efficiently.
An end-to-end monitoring provides a view of aggregate conditions of the network characteristics. An end-to-end monitoring is a direct indicator of the quality of service the user perceives in terms of their Internet connection. An end-to-end monitoring organizes itself in a pair of client and server machines that monitor the traffic which is traversing through a network path and determine the network condition. This methodology also enables the ease of monitoring and reduces the complexity of understanding the path conditions at each network element.
3.3.3 Wide area network traffic measurements
In a wide area network (WAN), the traffic within the network is not stationary and there is very little information available about the congestion within the network path. Monitoring the traffic over WAN links enables to easily notice the congestion variations over a period and helps to allocate resources within the network.
3.4 Tools used for the study
3.4.1 Iperf
Iperf [ipe07], originally developed by National Laboratory for Applied Network Re- search (NLANR), is a cross platform tool for active measurements of the maximum achievable throughput on IP networks. It allows tuning of various parameters re- lated to timing, buffers and protocols (TCP, UDP with IPv4 and IPv6). For each test it reports the bandwidth, loss, and other parameters.
Jitter in Iperf - Jitter calculations are continuously computed by the server, as specified by RTP in RFC 1889 [GSC+96]. The client records a 64 bit second/mi- crosecond time-stamp in the packet. The server computes the relative transit time as (server’s receive time - client’s send time). The client’s and server’s clocks do not need to be synchronized; any difference is subtracted out in the relative jitter calculation. Essentially, jitter is presented as the smoothed mean of differences be- tween consecutive transit times. In the case of packet losses, Iperf simply excludes the measurements with an undefined delay and computes the jitter with a reduced sample space.
3.4.2 Ping
Ping is the best known delay measurement utility and is already implemented in majority of operating systems. Generally making use of the ICMP protocol, ping probes the host destination with ICMP packets and in response receives the ICMP Reply packet and calculates from the timestamps of the ICMP packet pairs to de- termine the round trip delay. However, generation of ICMP packets happens at network layer of the TCP/IP stack (see figure 2.1) and also handled at the network layer. Ping does not reflect the ICMP packet processing at an end host. Also ping is susceptible to packet loss at routers due to priority scheduling. Few host may just disable the reply for security purposes. The flood ping options in ping im- plementations generates high load conditions and may be used for denial-of-service attack.
3.4.3 NetEm
Often, it is costly and difficult to reproduce network behavior in a controlled envi- ronment. There are tools available for testing, but they are either expensive hard- ware solutions, proprietary software, or limited research projects. NetEm provides the necessary statistical options to emulate real world network response. NetEm is a network emulator to emulate the properties of wide area networks. Network emulation is a technique for testing the performance of real applications over a vir- tual network. This is different from network simulation where purely mathematical models of traffic, network models, channels and protocols are applied. The aim is to assess performance, predict the impact of change, or otherwise optimize technology decision-making. NetEm is a recent enhancement of the traffic control facilities of Linux that allows adding delay, packet loss and other scenarios. NetEm is controlled by the command line tool ’tc’ which is part of the iproute2 package of tools [Hem].
3.4.4 Tcpdump packet capture
Tcpdump is a well known packet analyser, enabling capture and parsing incom- ing/outgoing packets to sniff the packets on the host it resides. Tcpdump captures all the essential headers with time-stamp information making use of libpcap library.
However, major drawbacks of using tcpdump: if the output is stored to a file the resulting size of the file is large. Command line tcpdump requires manual interven- tion with no user friendly graphical user interface. Tcpdump does not provide any performance analysis of the network.
3.4.5 Wireshark packet capture
Wireshark is an open source and cross platform packet analyzer generally used for protocol development and network troubleshooting. Wireshark uses pcap for packet capture and supports graphical user interface using Qt widget toolkit. The functionality of Wireshark is similar to that of tcpdump with additional sorting and filtering options. Wireshark captures all the traffic on the Network Interface Controller (NIC) including the broadcast/multi-cast traffic. Wireshark is capable of parsing, dissecting different fields of the packet and display the meanings specified by different protocols.
4 | Experimental outline
In this section we describe setup of the experiments and the motivation for perform- ing the measurements conducted in this study. We describe different measurements configurations and the instrumentation to measure the throughput and delay in lo- cal and wide area networks. The factors affecting BTC are studied and efforts are made to reduce the load on the network caused due to performance measurements.
4.1 Measurement motivation
In order to ensure reliability of the results, the traffic must be monitored on a real production link. As mentioned earlier a controlled and incremental approach is adapted to monitor the network behavior. Although the performance measurements are active care must be taken not to impact the existing traffic within the links.
In order to perform throughput measurements at high data rates, it is important to study the networking capability of hardware. As mentioned earlier the MAs in this study are listed in table 3 (Raspberry Pi 1/2 model B and ODROID-XU4), it is essential to determine the practical networking capability of the MAs. The maximum achievable throughput between the MAs is determined followed by the throughput measurements in local and wide area networks.
The factors affecting the network throughput such as round trip delay time are analysed across different paths in the CheesePi network. The measurements were motivated to study the time of the day impacts on the measurements.
In order to reduce the intrusiveness caused due to performance measurement we make an attempt to emulate the network characteristics and verify the results with measurements on real production links. From the previously conducted measure- ments we statistically estimate the throughput during the duration of data transfer in an end-to-end connected network.
4.2 Throughput-based measurements
Although the throughput measurements generalized, to be dependent on the net- working capability of the hardware. This study adapts to an incremental approach of introducing complexity within the system and is an effort to understand the be- havior of the system with increased complexity. From the specification of MAs as seen in table 3, suggest both the Raspberry Pis with a networking capability of 100 Mbits/sec and ODROID with 1000 Mbit/sec. However, do the MAs in practice sup- port the specified data rates? How much do they differ? This was the motivation to perform these experiments. For all throughput-based measurements in this study we make use of Iperf to generate traffic between the client and server. In Iperf the
client initiates the data transfer. The traffic traverses through the network interface controller (NIC) and traverses through various networking elements (if present) to the server. Iperf enables to choose between the TCP or UDP transport protocols between the client and server. The measurements and data monitoring are auto- mated using python scripts. The python scripts reside on both the client and server machines. The achieved results are store in comma separated file for analysis.
4.2.1 Throughput measurements between MA and test machine
For the following test, the throughput measurements are conducted between the MA and a test machine. The test machine used for measurements is a laptop and the specification for the laptop are listed below in table 4. The two machines are connected using an Ethernet crossover cable as shown in figure 4.1.
For throughput measurements using TCP, the duration of the data transfer is 60 seconds. Iperf at the server, reports the throughput information at an interval of 1 second and is recorded at the server. To aggregate the measurements we calculate the median and median absolute deviation of the measurements to reduce the impact of outliers on the values.
For throughput measurements using UDP, the data transfer duration is 10 sec- onds between the client and server. Initially applied bandwidth at the client is 10 Mbits/sec and at the end of each data transfer duration the applied bandwidth is incremented by 10 Mbits/sec up to the maximum throughput is noticed. The packet size for the UDP throughput measurement is 1450 bytes.
Test machine (Laptop) Specifications
Processor Intel i5 3337U, 1.8 GHz, Quadcore Operating system Ubuntu 14.04 LTS
Memory DDR3 1600 MHz SDRAM,
on board 4GB Networking Integrated 802.11 b/g/n
1000 Mbits/sec Base T
Manufacturer Asus
Table 4: Technical specification of test machine (Laptop)
Figure 4.1: Test Machine and MA connected using Ethernet cable
4.2.2 Throughput between the MAs connected using Ethernet cable For the following test, the MAs are connected back-to-back using an Ethernet cable as shown in the figure 4.2. The MAs used used in this study are as described in section 3.1. We perform throughput measurement using the TCP and the UDP protocol as mentioned in previous section 4.2.1.
Figure 4.2: Back-to-back Ethernet connectivity between two MAs
4.2.3 Throughput measurement between MAs connected via 100M switch In the previous section, the throughput is measured in the absence of any network element except an Ethernet crossover cable between MAs. To increase the com- plexity incrementally a 5 port non blocking 100M switch is introduced between the two MAs as shown in figure 4.3 and observe the effects of network elements on the throughput measurements. We repeat the tests as mentioned in previous section 4.2.1 in the presence of a switch between the two MAs.
Figure 4.3: Two MAs connected through a switch
4.2.4 Student residential network
The MAs installed at various locations as mentioned in table 2, mainly utilizing the network resources of Swedish academic links. Although we incrementally want to measure the network throughput across the CheesePi network, it is a requirement not to overload the network links. However, we performed throughput measure- ments using a MA which is installed at student accommodation. The measurements provide insights of throughput on a real production link.
Figure 4.4: MA at SICS connection to MA at Student Accommodation
The two MAs are separated with a total of 12 hops as shown in figure 4.4, including internal routers and public Internet nodes forming a network path. For this test, we made the MA at student accommodation as client and the MA at SICS as server using Iperf. Using UDP, we measure the throughput with respect to each applied bandwidth. The data transfer duration is 10 seconds for each corresponding applied bandwidth. At the end of each data transfer duration, the applied bandwidth is incremented by 10 Mbits/sec and the throughput is recorded at the server MA.
For this experiment, we have used Raspberry Pi 2 model B as the MA at both locations.
4.3 Throughput measurements within the CheesePi
The CheesePi nodes are mostly installed on the academic links. To perform through- put measurements between different network paths of CheesePi it is important to not to over load the network. We follow the following steps to perform throughput measurements within the CheesePi network:
1. The throughput measurements are performed only during the weekends for 24 hours. During the weekends the traffic over the academic links is assumed to be comparatively less than the weekdays and has minimal impact on the other users of the network.
2. Use only the TCP data transfer to perform throughput measurements as it offers congestion control.
3. The duration for each TCP flow is 10 seconds.
4. The throughput measurements are conducted every 20 minutes in the entire 24 hour duration. This sampling further reduces the intrusiveness within the network.
5. The MA at location SICS is assigned as client and the MA at remote location is assigned as server.
6. The throughput values at the server are reported at an interval of 1 second using Iperf.
7. The throughput is recorded at the server in a comma separated file for analysis.
4.3.1 Throughput between MAs at SICS and Karlstad
The MA at location Karlstad is Raspberry Pi 1 model B and the MA used for measurements at location SICS is Raspberry Pi 2 model B. Refer figure 4.5 for the geographical location of SICS and Karlstad University. We follow the steps as described in the section 4.3 to perform throughput measurements between the MAs at these two locations.
Figure 4.5: Geographical location of MA installed at location, Kista and Karl- stad. The green marker represents SICS and the orange marker represents Karlstad University.
The MAs are connected to the internal routers of the SICS and the Karlstad University. The hop count between the MA at SICS and MA at Karlstad University is 12 obtained using the tool MTR. The hops include internal routers and public Internet nodes. The distance between the two MAs is 295 km.
4.3.2 Throughput between MAs at SICS and Luleå path
The MA at location Luleå University of Technology is Raspberry Pi 1 model B and the MA at location SICS is Raspberry Pi 2 model B. Refer figure 4.6 for the geographical location of SICS and Luleå University of Technology. We follow the steps as described in the section 4.3 to perform throughput measurements between the two MAs.
Figure 4.6: Geographical location of MA installed at location, Kista and Luleå.
The green marker represents SICS and the red marker represents Luleå University of Technology.
These MAs are connected to the internal routers of the SICS and the Luleå University of Technology. The hop count between the MA at SICS and MA at Luleå University of Technology is 13. The hops include internal routers and public Internet nodes. The distance between the two MAs in 891 km. We repeat the throughput measurements as discussed in section 4.3.1, with the MA at Luleå University of Technology.
