i Master Thesis
Electrical Engineering April, 2012
School of Computing
Blekinge Institute of Technology 371 79 Karlskrona
Sweden
Evaluation of Smartphones Network
Performance
Chane, Mekides Abebe
ii
This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Electrical Engineering. The thesis is equivalent to 20 weeks of full time studies.
Contact Information:
Authors:
Chane, Mekides Abebe
E-mail: mekdiab3@gmail.com
Tekelmariam, Hailay Mezgebo
E-mail: hmezgebo@gmail.com University Advisor: Dr. Patrik Arlos School of Computing, BTH E-mail: patrik.arlos@bth.se School of Computing
Blekinge Institute of Technology 371 79 Karlskrona
Sweden
Internet : www.bth.se/com
Phone : +46 455 38 50 00
i
A
BSTRACT
Nowadays most Desktop based softwares, operating system (OS) and applications features are being adapted to Smartphones (SMPs). The simplicity and mobility of SMPs are some of the qualities which make them interesting to run different network applications. In order to develop mobile applications with efficient functionality and competitive in marketability, application developers need to have knowledge on network performance of SMPs.
In this thesis, an experimental based methodology is provided to evaluate the effect of transmission patterns on One Way Delay (OWD), throughput and packet loss across designed setup for different SMPs. Based on these metrics the SMPs have been compared with each other under the same experimental setting with relatively higher accuracy of measurement techniques. For accurate measurement DAG 3.6E card together with GPS for synchronization is used to capture traffic for further analysis. To avoid undeterministic inputs in the network, the experiment is made to be in a controlled and continuously monitored wireless local area network. Moreover the nodes or sub networks constituting the entire network are evaluated to conceive the effects on the estimated output. From this work application developers will have the opportunity to design their application according to the network performance of SMPs and users are also able to select suitable applications and SMPs.
ii
A
CKNOWLEDGMENT
Many thanks to our supervisor Dr. Patrik Arlos for his continuous guidance throughout the thesis work. We would also like to thank Dr. David Erman for his feedback and all the people who participate in the thesis work by sharing their knowledge and experience.
Thank you families and friends for your support
iii
C
ONTENTS
ABSTRACT ...I ACKNOWLEDGMENTS... II CONTENTS ...III LIST OF FIGURES ... IV LIST OF TABLES ... V LIST OF ACRONYMS... VI 1 INTRODUCTION ... 8 1.1 Smartphone Overview... 9 1.2 Related Works ... 10 1.3 Motivation ... 11 1.4 Contributions ... 12 1.5 Research Questions ... 12 1.6 Research Methodology ... 131.6 Outline of the Thesis ... 13
2 EXPERIMENTAL SETUP AND IMPLEMENTATION ... 14
2.1 Experimental Setup ... 14
2.1.1 Sender and Receiver Workstation ... 15
2.1.2 GW Workstation ... 15 2.1.3 Access Point ... 15 2.1.4 Measurement Point ... 15 2.2 Method ... 16 2.2.1 OWD Estimation... 16 2.2.2 Throughput Estimation ... 19
2.2.3 Packet Loss Estimation ... 20
2.2.4 OWD of GW... 21
2.2.5 OWD of AP ... 21
2.2.6 OWD of USB Link ... 21
2.2.7 Throughput measurement using DPMI and Iperf ... 23
2.2.8 Experimental Configuration ... 24
2.2.9 Controlling the Experimental Environment ... 26
3 RESULTS AND ANALYSIS ... 28
3.1 Effect of GW on OWD ... 28
3.2 Effect of AP on the OWD... 32
3.3 Effect of USB Link on the OWD ... 32
3.4 Evaluation and Comparisionof OWD of SMPs. ... 32
3.4.1 Packet Trace Analysis of OWD. ... 45
3.4.2 Comparison of OWD Distribution of the Network using the SMPs. ... 47
3.5 Throughput Analysis ... 48
3.6 Packet Loss Ratio Analysis ... 58
3.7 Throughput difference between DPMI and Iperf. ... 64
4 CONCLUSTION AND FUTURE WORK ... 68
iv
L
IST OF
F
IGURES
Figure 2.1 Experimental setup for network performance measurement of SMP. ... 14
Figure 2.2 MP special wiring scheme ... 16
Figure 2.3 Network trace excerpt inside Mp ... 16
Figure 2.4 OWD estimation of GW. ... 21
Figure 2.5 OWD estimation of AP. ... 22
Figure 2.6 OWD estimation of USB Link. ... 14
Figure 2.7 Experimental setup for DPMI vs. Iperf comparison. ... 23
Figure 2.8 Flow chart for statistical analysis process. ... 23
Figure 3.1a OWD distribution for R=2Mbps at PL=1400 bytes (Zoomed) ... 32
Figure 3.1b OWD distrubution exc. initial spike as a function of sequence number ... 32
Figure 3.2 Minimum OWD of the network at R=8kbps ... 37
Figure 3.3 Minimum OWD of SMPs in a network at R=1Mbps. ... 38
Figure 3.4 Minimum OWD of SMPs in a network at R=2Mbps. ... 43
Figure 3.5 OWD vs. Seq. No. at PL= 1472 bytes and R= 1Mbps. ... 46
Figure 3.6 ECDF of OWD of the network at R=1Mbps and PL= 1472byte. ... 48
Figure 3.7 Sender and Receiver average Throughput at R=1Mbps ... 49
Figure 3.8 Sender and Receiver average Throughput at R=2Mbps ... 49
Figure 3.9 Sender and Receiver average Throughput at R=5.5Mbps ... 53
Figure 3.10 Throughput vs Time at PL=1472 and R=5.2Mbps ... 56
Figure 3.11 CDF graph for PL=1472 byte at R=5.2Mbps. ... 57
Figure 3.12 Packet Loss Ratio at the R=1Mbps. ... 58
Figure 3.13 Packet Loss ratio at R=2Mbps ... 61
Figure 3.14 Packet loss ratio at R=5.5Mbps. ... 62
Figure 3.15 Throughput variation for DPMI vs. Iperf on SMP1. ... 65
Figure 3.16 Throughput variations for DPMI vs. Iperf on SMP2. ... 66
Figure 3.17 Throughput variation for DPMI vs. Iperf on SMP3. ... 67
Figure B.1 Minimum OWD of a Network at IFG=1s . ... 75
Figure B.2 Minimum OWD of a Network at IFG=2s ... 75
Figure B.3 Minimum OWD of a Network at IFG=8ms.. ... 76
v
L
IST OF TABLES
Table 2.1 Experimental parameter setting ... 25
Table 3.1 Effect of GW on OWD of the network at R 8kbps, 1Mbps and 2Mbps... 30
Table 3.2 Effect of AP on OWD of the network at R 8kbps,1Mbps and 2Mbps ... 33
Table 3.3 Effect of USB Link on OWD of the network at R 8kbps ... 33
Table 3.4 Statistical results of OWD at R=8kbps for PL=400bytes to PL=1150bytes. ... 39
Table 3.5 Statistical results of OWD at R=8kbps for PL=1200bytes to PL=1450bytes ... 40
Table 3.6 Statistical OWD delay at R=1Mbps for PL=400 bytes to PL=900 bytes ... 41
Table 3.7 Statistical OWD delay at R=1Mbps for PL=900 bytes to PL=1450 bytes ... 44
Table 3.8 Statistical OWD delay at R=2Mbps for PL=400 bytes to PL=1450 bytes ... 44
Table 3.9 Sender and Receiver average Throughput at R=1Mbps ΔT=1S ... 50
Table 3.10 Sender and Receiver average Throughput at R=2Mbps ΔT=1S... 52
Table 3.11 Sender and Receiver average Throughput at R=5.5Mbps ΔT=1S. ... 55
Table 3.12 Packet Loss Ratio at R=1Mbps ... 59
Table 3.13 Packet Loss Ratio at R=2Mbps ... 60
Table 3.14 Packet loss ratio at R=5.5Mbps... 59
Table 3.15 DPMI vs Iperf average throughput output at R=1Mbps ... 64
Table 3.16 Standard deviation DPMI vs. Iperf at R=1Mbps. ... 65
Table A.1 SMPs Specification overview ... 74
Table B.1 Stastistical results of OWD at IFG=1s for PL=400bytes to PL=1450bytes ... 77
Table B.2 Stastistical results of OWD at IFG=2s for PL=400bytes to PL=1450bytes. ... 78
vi
L
IST OF ACRONYMS
1. A.P: Access Point 2. ACK: Acknowledgment
3. API : Application Programming Interface 4. Corr. Coef : Correlation Coefficient 5. CPU : Central Processor Unit
6. DAG : Data Acquisition and Generation 7. DHCP : Dynamic Host Configuration Protocol 8. DIFS : DCF Interframe Space
9. DPMI : Distributed Passive Measurement Infrastructure
10. Exp. No. : Experiment Number
11. GPS : Global Positioning System 12. GUI : Graphical User Interface 13. GW : Gateway
14. IFG : Inter Frame Gap 15. IP : Internet Protocol
16. Mar : Measurement Area
17. MArC : Measurement Area Controller
18. NTP : Network Time Protocol
19. OS : Operating System
20. PDA : Personal Digital Assistant
21. PDU : Packet Datagram Unit 22. pps : packets per second 23. QoE : Quality of Experience 24. RTT : Round Trip Time
25. Seq. No. : Sequence Number
26. SIFS : Short Interframe Space 27. SMP : Smartphone
28. SMP1 : HTC Windows 29. SMP2 : HTC Android 30. SMP3 : Xperia X1 Windows
31. SMP4 : XPERIA Neo android
vii 34. TCP : Transport Control Protocol
35. UDP : User Datagram Protocol
36. UMTS : Universal Mobile Telecommunication System 37. USB : Universal Serial Bus
8
C
HAPTER
1
INTRODUCTION
The demand for Smartphone (SMPs) in the market is increasing from day to day due to addition of new features and improvements in hardware, software and applications. To cope up with the market demand, SMP manufactures are continuously improving the hardware capacity and OS efficiencies to match the processing capacities of personal computers, laptops and other advance computing devices. Currently numerous mobile applications are being developed for different purposes. In order to develop competent mobile applications with good user experience, application developers need to have an insight into the network performance of the SMPs.
The survey conducted by AdMob Mobile Metrics show that 46% of mobile Internet traffic worldwide is generated by SMPs compared to feature phones and mobile Internet data cards [1]. And most applications which require internet access are dependent on the metrics OWD, throughput and packet loss ratio. Hence, it is important for the application developers to understand how these metrics affect SMPs performance on networks.
9
1.1 Smartphone Overview
A special mobile phone that combines the features of Personal Digital Assistant (PDA) is called SMP. A PDA is a computer based small mobile hand held device that provides computing, personal organizer tool, information storage and retrieval capacity for personal and business purpose. SMP are called “Smart” for the reason that they have the functionality of a computer. There functionalities and easy to carry along in our small pocket or purse make them important in our day to day life.
Mobile phones support the features such as organizers, camera, games, and browsers. Beside the features that mobile phones have, SMP has functionalities to act as a mobile camera with high resolution, GPS navigation, GB of mass storage, radio and wireless interface i.e. UMTS and WLAN. They also have their own mobile OSs, some of the most common and well known ones are, Google Android, Nokia Symbian, Microsoft Windows phone 7 and Apple iOS [33].
The first mobile device, that somehow fulfils the term SMP definition, was IBM SIMON manufactured by IBM and BellSouth, which was shown in trade in 1992 [35]. Most currently available SMPs such as Blackberry, iPhone and Android have similar featured applications with IBM SIMON. The common used applications today in these SMPs are email, fax, world clock, calendar, calculator, address book and even a touch screen. The next notable chronologically known SMP was Nokia Communicator 9210 which was introduced by Nokia in 1996. It was the first SMP to have open OS unlike the previous predecessors. Inspired by the fact that third party applications was becoming more popular, in 2001 Handspring released Palm OS Treo. Some of the popular applications on this SMP introduced are wireless web browsing, email and contact organizer.
10
IPHONE was easy to use, aesthetically attractive and very good touch screen feature; it changes the course of market acceptance for the SMPs. Moreover in 2008 APPLE has established apps store which we can download different apps freely or fee based. This trend was followed by different SMP manufactures and OSs developers such as Google apps store, Nokia Ovi store, Microsoft windows mobile apps store and so on.
The full version release of an open source Android OS by Google and Open Handset Alliance in 2008 took into the highest level in market competition of OS, apps and other web applications. Currently different version of operating OS, hardware releases, and millions of apps are available on the market. Vendors and applications developers are striving to bring about new products and feature to satisfy their customers. Hence evaluation of network performance of SMPs from the user’s perspective keeps them in the open market game.
1.2 Related Works
Many network performance comparisons of SMPs have been reported in different papers, their center of attention is mainly based variety of application based measurement softwares. Micheal et al. performances a study on some well known mobile OSs (Android, iPhone OS, Symbian, Windows mobile, palm OS and Blackberry) to determine which OS are the most efficient and convenient for users, developers, mobile gaming and business applications [18]. Jonathan et al. [19] compares two mobile OSs windows and symbian focusing on CPU management. The researchers from Rysavy research perform a test on the efficiency comparison of blackberry 6.0 versus Apple iPhone iOS3 and Android 2.1, the test was made on applications including e-mail, social networking, instant messaging and web browsing. From the test the researchers concluded that blackberry phones consume far less data than iPhone iOS and Android [7].
11
comparison. As reported in [12], application level measurement tools are less accurate in describing network behaviors. The author further showed that hardware measurement tools offer more accuracy in time measurement than software tools. In [4, 5] papers, One Way Delay (OWD) measurement was undertaken using measurement setup that give timestamp accuracy of less 100ns for uplink and downlink on 3G networks connecting data card as a modem. The authors pointed out that payload size and data rate affects OWD. In [6], the authors studied the processing delay of IP routers on best-effort Internet traffic generated in their experimental setup based on specification of IETF RFC2679 for OWD metric. They have also shown that the processing delay in IP routers is influenced by traffic characteristics, link status and hardware specification. The variables payload size, protocol and inter-packet time collectively described as transmission pattern in [2, 4, 5, 6, 12 and 14], are important input parameters to be considered in our thesis work.
The authors in [2] have compared cross SMP and cellular carrier of TCP and UDP throughput. Their experimental result shows that there is a difference in throughput between different SMPs platforms. In [8] two SMP platforms are studied by monitoring the traffic using passive sniffers. After measuring TCP throughput they conclude that, connection throughput is not only a function of Round Trip Time (RTT) and loss rate but also a function of application lever factors such as amount of data.
1.3 Motivation
12
1.4 Contributions
Some of the basic contributions of the thesis are,
To provide knowledge for application developers on OWD, throughput
and packet loss ratio of a SMPs network performance for UDP protocol.
An experimental methodology to evaluate network performance of
SMPs in a designed network in uplink direction.
Comparison of the network performance of SMP based on OWD,
throughput and Packet Loss with respect to sending rate and payload size.
Estimation of throughput difference between the Distributed Passive
Measurement Infrastructure (DPMI) and existing network performance
measurement tool (Iperf) for SMPs.
1.5 Research Questions
RQ1: How does transmission pattern affect OWD of SMPs and what are the factors for OWD variation?
RQ1.1: How can we model the comparison of SMPs based on OWD?
RQ2: How does transmission pattern affect the throughput of SMPs?
RQ2.1: What is the difference in throughput of different SMPs and what factors affect this difference?
RQ3: How does transmission pattern affect packet loss ratio of SMPs?
RQ3.1: What is the difference in packet loss ratio among different SMPs and what factors affect the difference?
13
1.6
Research Methodology
In this section the research methodology and approach used to address the research questions will be discussed. To answer the research questions, we studied previous research papers related to network performance of SMPs that give us good perceptive for our thesis work. In addition we studied software tools and hardware equipments that are required in commencing our experimental method. The choice of experimental methodology in this thesis instead of other methodologies such as Simulation, mathematical modeling, and others, is due the fact that the experimental results
represent true network performance in real networks. To do the experiment we
designed experimental setup in the laboratory where the setup was verified and validated by executing pre-experimental tests comparing with theoretical results. Our methodology is intended to be replicable for similar experimental tests. To conduct the experiment we clearly indentified and controlled the variables in the experimental process; these variables are sending rate, payload size, Inter Frame Gap (IFG), time of the day (TOD) and distance of the SMP from the directly connected wireless media. The transmission patterns sending rate and payload size are varied independently to see the effect on OWD, throughput and packet loss ratio. The experimental runs are repeated for reducing the randomization behavior of the networks and measurement processes. Moreover for external random and interference effects, the networks are controlled and monitored throughout the experimental process. Using the above experimental approaches it was possible to answer the challenges raised in our research questions.
1.7
Outline of the Thesis
14
C
HAPTER
2
Experimental Setup and Implementations
In this chapter, the hardware and software tools used in the experimental setup will be discussed in section 2.1. In section 2.2 the methods used to approximate OWD, throughput, packet loss and experimental configuration setting will be discussed.
2.1 Experimental Setup
To observe the effect of transmission pattern on OWD, throughput and packet loss in
network performance of SMPs, we designed experimental setup shown in Figure 2.1. Using existing UDP traffic generator tool, traffic is sent in uplink direction from sender to receiver. Since it’s not possible to wiretap USB cable to capture data at the MP, Gateway (GW) workstation is connected to the SMP via its USB interface. Using tethering software, the SMP was made to share wireless network of the Access Point (AP) to the GW to transfer traffic at the receive workstation.
Sender SMP Receiver MP GW AP Wiretap A Wiretap B
15
Traffic sent from the sender is captured and time stamped near the sender workstation at Wiretap A and upon arrival at the receiver their copy is capture and time stamped at Wiretap B using MP.
2.1.1 Sender and Receiver Workstation
Two workstations installed with Linux OS were used as sender and receiver end points as shown in Figure 2.1. Each of them has CPU processor of Dual core 2.6GHz AMD Athlon. A UDP Traffic generator tool at the sender and UDP traffic sink at the receiver written in C++ program was installed at both ends. The sender is connected to the GW by 10Mbps Ethernet via Wiretap A. On the other side the receiver is connected to the AP through 10Mbps Ethernet via Wiretap B. Moreover at the receiver a DHCP server was configured to assign IP address to the SMPs through the AP.
The transmission pattern variables which are used by the UDP generator parameters at the sender and receiver are IFG and payload size (PL) for different experimental
configuration settings it will be discussed in section 2.2.8. Additional parameters used by the UDP generator to differentiate the experiments are IP address, experimental number (e), run number (r), key id (k) and port (p) which will be varied accordingly for each experiment independent of the transmission patterns.
2.1.2 Gateway workstation
The GW is Intel Core 2 Duo CPU P8600 2.40GHZ with 4GBytes RAM and Ubuntu 10.10 version OS. Since it is not possible to wiretap USB cable of the SMP to capture packets in the MP, the sender and the GW workstations are made to be connected through the Ethernet link. Hence by doing so it is possible to wiretap the link to send copy of packets to the MP. The GW forwards traffic from the sender to the SMP using Internet Connection Sharing (ICS) through the USB cable.
2.1.3 Access Point
16
WLAN interface supports this mode. It operates at four half duplex data rates which are 1Mbps, 2Mbps, 5.5Mbps and 11Mbps [17, 26, and 27]. The AP passes traffic from the SMP to receiver through its Ethernet interface.
2.1.4 Measurement Point
The MP is a Linux workstation with two Endace network monitoring DAG 3.6E cards [15]. It collects the copy of the original data near the sender and receiver wiretaps
Wiretap A and Wiretap B respectively as shown in Figure 2.1 [12, 34].
2.2 Method
2.2.1 OWD Estimation
OWD is the time it takes for a packet to travel from sender to receiver across a given network [4, 5 and 6]. For the packet designate as index i captured and time stamped at the sender (Tαi) and receiver (Tβi), the OWD is estimated by subtracting the ith packet
at the sender (Tαi) from ith packet at the receiver (Tβi) [4, 5].
OWDi= Tβi – Tαi 2.1
In order to calculate OWD from the given parameters at the sender and receiver, the accuracy of the absolute time should be achieved. The location of the sender and receiver could be in two different places geographically. Hence in this thesis, DAG 3.6E card is used which captures packet with timestamp resolution of 60 ns at both sender and receiver. Moreover the DAG cards clocks are synchronized to a common GPS clocking system.
17
Figure 2.2 MP special wiring scheme
The ith indexed single packet is indentified by combination of network trace parameters
captured inside the MP. These parameters comprised of traffic input parameters feed to the traffic generator as depicted in section 2.1.1, unique sequence number, IP headers, PDU length, timestamp (TS), MP ID, DAG ID, and so son. Excerpt of the network trace parameters at sender and receiver are shown in Figure 2.3.
Pkt No DAG ID MP
ID TS PL ………. SRC IP DST IP EXP ID RUN ID KEY ID SEQ No
Figure 2.3 Network Trace Excerpt inside MP
SRC IP, DST IP, EXP Id, DAG ID, RUN ID and KEY ID are important selected parameters to match the specific packet SEQ No and TS at the sender and receiver. Hence PERL Script is employed to extract and estimate OWD of each packet. Ultimately by repeating the process the statistical mean, maximum (max), minimum (min) and standard deviation (Std.dev) are estimated. The procedure is illustrated in flow chart format in Figure 2.7.
For single packet the OWD of the network is estimated using equation 2.1. For N
number of samples the min, max, mean and std.dev values for the ith packet inside the
18
OWDmean = 2.2
OWDmin= i=1 N 2.3
OWDmax= i=1N 2.4
OWD standard deviation describes how the OWD values varies from the mean and is estimated as equation 2.5,
OWDStd.dev= 2 2.5
Using linear regression fitness of curve we can formulate the dependency of payload size (x) and the minimum OWD (y) of the network for each SMP as,
= a + bx 2.6
‘a’ is computed as and ‘b’ is given by b = (s/byte)
Where: = average of y’s and = average of x’s. Q = total number of selected payload size ranges.
Hence the OWD ( ) function can be approximately dependent on payload size variable ( ). The correlation coefficient (Corr.Coeff) is the relative fitness factor between the measured OWD versus payload size plot and the theoretical linear regression formula.
19
2.2.2 Throughput Estimation
Throughput is defined as the ratio of an amount of data passing a point of reference and the total elapsed time [13]. Throughput serves as Quality of Service (Qos) indicator for the network [21]. The measure of network performance QoS as view by the end user termed as Quality of Experience (QoE), covers the all the communication layers [13, 16 and 21].
For the given kth interval, bitrates for sampling interval time of (ΔT) at a reference point S is estimated as,
DK = 2.8
Where: DK= the bitrate at (kth) interval,
dk- = number of bits started arriving in prior kth interval
dk+= number of bits started arriving in kth interval but not completed
dj=bits of M packets completely inside kth interval
Each (DK) throughput value at position A is calculated as the average bitrate of
duration of kth interval captured at MP in the interval [(k-1)ΔT, kΔT] specifies one sampling interval which includes one or more packets [21]. The combination arrival time series (timestamp) and payload size are the parameters collected from the captured data at the MP to estimate the throughput. The average throughput during the time window (W) is given as,
Dave= 2.9
Where Z= , defines the total number Z throughput values during W time
Window. The standard deviation of the average throughput is given by:
20
The Coefficient of Throughput Variation (CoTV) for a network at point ‘M’ is given by:
CoTVM = 2.11
The CoTV comparison at the sender and receiver indicates the extent of the burstiness in the network [13].
From the above equations estimation of the throughput depends on selection of
sampling interval (ΔT), time window (W) and number of samples. For accuracy in
throughput results the accuracy of timestamp and consideration of fractional packets (PDU) in the sampling processing are determinant factors [12].
Bitrate C++ program tool is used to convert the binary cap file to textual stream of samples. This program handles timestamp resolution of picoseconds. The input parameters for the program are sampling frequency (m), speed of the link (l), layer of interest (q), source IP address, destination IP address, and port number. The sampling frequency is the inverse of sampling interval. The throughput of the network including the SMPs is estimated from the average bitrate in the streams of samples [12]. For most of the throughput analysis section the sampling frequency is selected 1Hz which is equivalent to 1s sampling time interval.
2.2.3 Packet Loss Ratio Estimation
Packet loss can be due to corruption in the IP fields of the packet after it is received on the receiver or completely lost in the network. The designated packets captured at the sender and receiver of the MPs is compared for each packet to analyze the packet loss in the network.
21
λ = 2.12 Where: λ= packet loss ratio
Sp = Total number of sent packets Rp = successfully received packets
2.2.4 OWD of Gateway
To evaluate the effect of GW on OWD estimation of the network; the GW connects the
sender and receiver workstations using internet connection sharing (ICS) the same as
the complete setup for SMPs. The SMP and AP from Figure 2.1 were replaced with D-link DUB-E100 fast Ethernet USB adaptor to connect the GW and receiver workstation as show in the setup Figure 2.4. Selecting USB-Ethernet adapter in this case would assure the same treatment of packets as SMP USB interfaces as in Figure 2.1 with a difference in driver. The two ends of the GW are wiretapped to the MP DAG 3.6E interfaces for capturing the traffic at the sender and receiver workstations.
Sender Receiver MP GW Wiretap A Wiretap B usb Ethernet DUB-E100.
Figure 2.4 OWD estimation of the GW.
2.2.5 OWD of Access Point
22
Figure 2.5 OWD estimation of AP.
2.2.6 OWD of USB Link
To estimate the effect of the USB link and interface on OWD, experimental setup shown in Figure 2.6 is used. The SMP is connected to GW through USB 2.0 cable which has interfaces speed of 480Mbps. The transfer of packets to and from SMP is captured and timestamp recorded using a special hardware USB protocol analyzer called Beagle USB 480 Protocol Analyzer [23, 40]. The packets that appear on the bus are copied to the analysis port. The captured ‘TDC’ format is converted to a readable ‘csv’ file in the datacenter software. PERL script is employed to separate payload sizes and estimate the duration of time accordingly.
Mostly for all SMPs every packet there is 86 bytes of header appended to application packet during transition which is greater than USB-Ethernet adapter i.e. 46bytes. Packets greater than or equal to (450+86) bytes are fragmented. The maximum MSS is 512bytes on the bus and as payload size increase the packets are divided in more segments.
23
Figure 2.6: OWD estimation of USB Link.
2.2.7 Throughput Measurement using DPMI and Iperf
The main aim of this section is to compare the throughput measured using available network performance measurement tools for SMPs and DPMI system. In this thesis Iperf is selected because it is commonly used tool in different type of devices and platforms. Iperf estimates throughput of a network between two end point devices [10].
Receiver Sender MP Access point Wiretap B
Figure 2.7 Experimental setup for DPMI vs. Iperf comparison.
24
through the AP. While the data pass through wiretap B the copy of the data is captured at the MP before reaching the receiver. The Iperf tool installed at the SMP and receiver side captures and calculates the throughput for each sampling interval of 1 s. With the experimental trace found at the MP, the throughput for DPMI case is analyzed using existing bit rate evaluation tool at the sampling interval of 1 s. The statistical results are discussed in section 3.7.
2.2.8 Experimental Configuration
In this experimental configuration three SMPs, HTC HD2 Microsoft Windows
Mobile 6.5 Professional OS (SMP1), HTC Desire HD Android OS v2.2 (SMP2) and
Sony Ericsson Xperia X1a Microsoft Windows Mobile 6.1 Professional (SMP3) [22, 23], have been connected in the experimental setup. In addition to the above mentioned SMPs, Nokia N8 (Symbian^3 OS) and Apple iPhone iOS3 experimental runs were tried, but due to the limitation on tethering software they are not dealt here.
The AP operates at four data rates i.e. 1Mbps, 2Mbps, 5.5Mbps and 11Mbps. In our experiment the sending rates (R) in UDP traffic generator were tuned to 8kbps, 1Mbps, 2Mbps and 5.5Mbps where data rate of AP link was fixed at R=11Mbps as shown in Table 2.1. The payload sizes (PL) are made to vary starting from 400 bytes to 1450
bytes with step size of 50 bytes at different R. The UDP traffic generator uses the IFG and PL to calculate R using equation 2.13.
25
Start capturing traffic inside MP
using bitrate estimation tool with averaging interval of 1 s for a given sample size and total observation time as shown in Figure 2.8. By using Perl script and MATLAB program, the textual stream file is used to estimate the statistical average, min, max and CoTV for each metrics.
Figure 2.8: Flow chart for statistical analysis process.
Binary file (.cap)
Bitrate or OWD
Conversation to text format (.txt)
Network Traces
OWD Analysis Tool
(PERL andMATLAB)
Report OWD, packet loss
statistics
Conversion text format (.txt)
Stream of avg. bitrate samples
Bitrate
Analysis Tool
(PERL and MATLAB)Report Throughput, loss,
statistics
26
Table 2.1 Experimental Parameter setting.
Exp.
No. Metrics PL [Byte] R
Number of sample packets (N) Duration (s) 1 OWD 400 -1450 with step size of 50 8kbps 10,000 - 2 1Mbps 100,000 3 2Mbps 4 1472 1Mbps - 5 Throughput and Packet Loss Ratio 400 -1450 with step size of 50 1Mbps 320-1160 6 2Mbps 160-580 7 5.5Mbps 64-232 8 Throughput 1472 5.2Mbps 80,000 226 9 Throughput for DPMI vs. Iperf 1470 1Mbps - 100
2.2.9 Controlling the Experimental Environment
27
28
C
HAPTER
3
Results and Analysis
In this chapter the results found from the experimental processes will be discussed in detail. The OWD analysis will be discussed in section 3.1, section 3.2 and section 3.3, the average throughput and packet loss ratio will be discussed in section 3.4 and section 3.5. Finally comparison of SMPs throughput with other available network performance evaluation tool will be discussed in section 3.6.
3.1 Effect of GW on the OWD
Exp. No. 1-3 configuration setting is employed to see the effect of the GW on OWD of the network as shown in Figure 2.4. For R 8kbps, 1Mbps and 2Mbps the estimated OWD statistical min, mean, max and standard deviations are tabulated as shown in Table 3.1.
From the table as the payload sizes increases the minimum and mean OWD of the GW increase for each increase in payload size as expected. Moreover the standard deviation value ranges from 46µs to 65µs for the specified range of payload size and the maximum values for the entire payload sizes are also at certain limit; increase in payload size increases the maximum value also in non strict sense. The typical
maximum OWD occurrence was identified to be 5.77ms at PL=1400 byte and
29
Table 3.1 Effect of GW on OWD of the Network at R 8kbps, 1Mbps and 2Mbps
30
This incident is plotted in Figure 3.1a initial version (above) and zoomed version (below) as function of sequence number. The scaled version of randomly selected sample distribution for sequence number from 5000 up to 5010 in the plot resembles flatted top and sharp bottom edge (trapezoid) structure which repeats every 28ms interval. Further enlarging the traces might also resemble triangular seesaw at some parts. This pattern appears in most of the payload size ranges. This effect might be partly contributed due to USB-Ethernet driver. The full original Distribution plot without the initial OWD spike is as show in Figure 3.2b. Hence the GW effect
contributes a small OWD variation to the overall network as shown for all PL ranges.
Moreover the packet losses were also investigated. There were no packet losses observed for lower R i.e. R=8kbps but at R=1Mbps and R=2Mbps total packet losses
from one million packets is not more than 6 and 10 packets at PL=400byte as shown in
Table 3.1. The losses are mostly at shorter PL’s. Packet losses have been observed
31 0 1 2 3 4 5 6 7 8 9 10 x 104 1.4 1.6 1.8 2 2.2 2.4 2.6 Sequence Number O WD (ms )
OWD Distribution For Gateway R=2Mbps PL=1400byte
Figure 3.1a: OWD distribution of the first 30 packets (above) and 100 packets zoomed randomly selected packets from the middle of the same trace (below) for the same transmission pattern.
-
32
3.2 Effect of Access Point on the OWD
To evaluate the effect of AP on OWD of the network similar to the GW, experimental setup shown in Figure 2.5 is used. The effect of AP on OWD of the network at R=8kbps, 1Mbps and 2Mbps are estimated in Table 3.2. The results confirm that there is no large deviation in the observed OWD of the AP between measured sample packets. The maximum and minimum are almost equal to the mean in effect the standard deviation has very small values from 1µs to 12.3µs. As payload size increase the mean and minimum OWD increase linearly and so does the standard deviation.
33
Table 3.2 Effect of AP on OWD of the network at R 8kbps, 1Mbps and 2Mbps
34
3.3 Effect of USB link on OWD
The effects of USB link on OWD for each SMP is shown in the Table 3.3. From the experimental results we can see that as payload size increases the duration of packets in the USB bus increases. When data transferred over the USB bus to or from SMP there is an extra delay besides propagation delay and transmission delay. These delays are due the USB protocol which is periodically generated synchronization frames (signals) to control data communication and power consumption on the bus.
35
Table 3.3: Effect of USB Link on OWD of the network at R 8kbps PL
[Bytes]
SMP1 [us] SMP2 [us] SMP3 [us]
Mean Min Max Std.dev Mean Min Max Std.dev Mean Min Max Std.dev
36
3.4 Evaluation and Comparison of OWD of SMPs
The aim of this section is to see how transmission patterns affect the OWD of the network and based on that to compare the network performance of SMPs. To observe the effect we have done experiments using experimental configuration as shown in Table 2.1. From all the experiments Exp. No. 1 up to Exp. No. 3, we observe that as payload size increase the minimum OWD of the network increases across all SMPs. This is expected since longer payload size takes longer time to transmit.
From the experimental results we also observe that, the minimum OWD of the network at low R i.e. 8kbps has higher minimum OWD value than at relative ly higher data rates i.e. 1Mbps, 2Mbps especially for higher payload sizes. In this
section the results for R=8kbps, R=1Mbps and R=2Mbps are presented in Figure
0.3, 0.5 and 0.7, additional low data rate results can be found in Appendix B.
The linear regression function and Corr.Coeff are evaluated for network with each SMP by using equation 2.6 and 2.7. The constant values and function are shown at Table 3.4, Table 3.6 and Table 3.7 for data rates 8kbps, 1Mbps and 2Mbps. The
intercept constant discussed in section 2.2.1 equation 2.6 may be used as
comparator for each SMP, which has empirically inversely proportional to the size of RAM and processing speed where other internal software and applications processes have been assumed relatively smaller effect. The intercept constant can
be defined the initial OWD value at the initial PL value. In this thesis consideration
the initial PL value is 50byte for each SMP’s. The initial PL=50bytes value is
chosen close to the minimum possible PL=40bytes set by the traffic generator. The
intercept constant is considered as the OWD at relatively zero PL coordinate. From
37
size) capability are different. Since they have same processing speed their difference in the intercept constant is very small as compared with SMP3. In contrast SMP3 has lower RAM size and processing speed compared to SMP1 and
SMP2, in effect the intercept constant is comparatively higher both of them.
The Corr.Coeff for minimum OWD versus payload size shows mostly greater than 99% as shown in Table 3.5, Table 3.6 and Table 3.7 for each SMP. Hence for low R the minimum OWD and payload sizes have linearity relationship in loose sense. But the OWD versus payload size graph for higher R might deviate from linearity behavior due to loss of packets and excessive queuing in the network.
Figure 3.2 Minimum OWD of the network at R=8kbps
Based on OWD and PL dependency criteria, OWD of the network using SMP2 has
mostly more smooth linearity (Corr.Coeff = 99.99%) than the other SMP1 and
SMP3.For SMP3 the MTU has been estimated to be 1366 byte where as for SMP1
and SMP3 they have MTU of 1500 bytes each. Hence for SMP3 at PL’s of 1400
38
deviated from the linear curve as shown in the Figure 3.2, 3.3 and 3.4 at the respective R’s.
Figure 3.3 Minimum OWD of SMPs in a network at R=1Mbps.
39
Table 3.4 Statistical results of OWD at R=8kbps for PL=400bytes to PL=1150bytes.
PL [Bytes]
SMP1 [ms] SMP2 [ms] SMP3 [ms]
Mean Min Max Std.dev Mean Min Max Std.dev Mean Min Max Std.dev
40
Table 3.5 Statistical results of OWD at R=8kbps for PL=1200bytes to PL=1450bytes
PL [Bytes]
SMP1 [ms] SMP2 [ms] SMP3 [ms]
Mean Min Max Std.dev Mean Min Max Std.dev Mean Min Max Std.dev
41
Table 3.6 Statistical OWD delay at R=1Mbps for PL=400 bytes to PL=900 bytes.
PL
[bytes]
SMP1[ms] SMP2[ms] SMP3[ms]
Mean Min Max Std.dev Mean Min Max Std.dev Mean Min Max Std.dev
42
Table 3.7 Statistical OWD delay at R=1Mbps for PL=950 bytes to PL=1450 bytes.
PL [Bytes]
SMP1 [ms] SMP2 [ms] SMP3 [ms]
Mean Min Max Std.dev Mean Min Max Std.dev Mean Min Max Std.dev
43
Figure 3.4 Minimum OWD of SMPs in a network at R=2Mbps.
As R increase the minimum OWD of the network decrease slightly mostly for longer
PL’S. But for shorter PL the minimum OWD decrease. This can be seen from Table 3.6,
44
Table 3.8 Statistical OWD delay at R=2Mbps for PL=400bytes to PL=1450byte
SMP1(ms) SMP2 (ms) SMP3 (ms)
PL (byte) Mean Min Max
Std.
dev Mean Min Max
Std.
Dev Mean Min Max Std.dev
45
Based on the regression equation the slope of each line denotes an inverse of the bandwidth of the network. This slope varies between each SMP and selected R’s due to loss, queuing, total number of samples (N), PHY overhead, MAC overhead, the nature of each SMP and other factors. From the above tables SMP2 has lowest slope especially for lower R at 8kbps and SMP1 has lower slope than SMP3. For relatively higher R i.e. 1Mbps and 2Mbps, SMP2 still has lower slope than both SMPs but SMP3 has lower slope than SMP1. Moreover there is a slight tendency in increment of the intercept constant as R increase especially from 1Mbps to 2Mbps. This can be interpreted for lower R the minimum OWD of the network has relatively less values than higher R’s especially for shorter PL.
3.4.1 Packet Trace Analysis of OWD distributions
To examine how the network is performing as function of sequence number, the network traces collected from Exp. No 4 are plotted for PL
1472 byte at R=1Mbps. OWD of the network for the three SMPs connected cases as a function of sequence numbers are shown Figure 0.5. The OWD across the network experience loss of packets, variations and delay spikes. The effect of OWD variations might be due to the combination or independent actions of underlying MAC and PHY parameters or higher layer protocols involving in the transmission of data packets across the 802.11b link.
46
wait”. The packet can be retransmitted without discarded until the maximum seven threshold number of attempts is repeated. Due to this random retransmission or loss of packet the variation of OWD across the network increases in effect some packets might experience huge delay spikes [26, 29, 31, and 32].
Figure 3.5 OWD vs. Seq. No. at PL= 1472 bytes and R= 1Mbps.
47
the SMPs. From the traces SMP1 has highest max and std.dev OWD at most packets while SMP2 has lowest max and std.dev OWD. SMP1 and
SMP3 have higher max OWD and std.dev compared to SMP2 at most
packets. This might be because of the reason that they are using the same tethering software and windows OS version. The tethering software might be responsible for keeping the packets in the queue in the presence of congestion in the network. This in effect might contribute to nominal loss of packets and increase in throughput as it will be discussed in sections 3.5 and 3.6.
The effects of OWD variation and spikes due to retransmission and other effects mentioned above are clearly noticeable in the zoomed Figure 3.5 for the three SMPs. The maximum OWD of the network for SMP1, SMP2 and SMP3 are 298ms, 55ms and 169ms respectively. The delay spikes are randomly distributed throughput the total sequence number ranges.
3.4.2 Comparison of OWD Distribution
The OWD Empirical Cumulative Distribution Function (ECDF) of network with three SMPs is plotted in Figure 3.6. The OWD probability distribution of the network using SMP2 is greater than SMP1 and SMP3 but the distribution for SMP1 lies in between SMP2 and SMP3. This comparison based on the OWD distribution of the network holds mostly true for relatively smaller and lossless R values at given PL’s.
48
Figure 3.6 log scaled ECDF OWD of the network at using the three SMP’s
R=1Mbps and PL= 1472bytet
3.5 Throughput Analysis
To study how the transmission patterns affect the average throughput of a
network having SMP, We perform experiments by varying R’s and PL‘s
using the same experimental configurations as previous section 3.4. The experimental configuration of Exp. No. 5, 6 and 7 are applied to the
network to observe the throughput of the network.
49
the small distance between the generated traffic and the wiretap. From these tables we can also observe that as the length of payload size increase the sender and receiver side throughput also increases slightly except at some payload sizes, this is due to the random loss of packets in the network [39]. The CoTV at the receiver is greater than the sender side for the respective PL. As PL increase the CoTV at the receiver decrease mostly. The CoTV
describes the burstiness in the network [16]. The packets loss ratio for the
respective R and PL can be seen from Table 3.9, Table 3.10 and Table 3.11
but it will be discussed insection 3.6 in more detail.
Figure 3.7 Sender and Receiver average throughput at R=1Mbps.
50
Table 3.9 Sender and Receiver average Throughput at R=1Mbps ΔT=1s.
PL [Bytes]
SMP1 SMP2 SMP3
Sender [Mbps] Receiver [Mbps]
λ [%] Sender [Mbps] Receiver [Mbps] λ [%] Sender [Mbps] Receiver [Mbps] λ [%]
Mean CoTV Mean CoTV Mean CoTV Mean CoTV Mean CoTV Mean CoTV
51
As we increase the sending data rate to R=2Mbps the average throughput of network with SMP1 attains better throughput than the network with SMP2 and SMP3 as shown in Table 3.10 and Figure 3.8. From experimental results at 1Mbps and 2Mbps, the average throughput at the sender side is comparable with the receiver side throughput since the packet loss is relatively small. And the standard deviation at the receiver side is higher than the sender side, this is expected because of the bottleneck in the network might create random arrival of packets at the receiver [13].
52
Table 3.10 Sender and Receiver average Throughput at R=2Mbps. ΔT=1s
PL [Bytes]
SMP1 SMP2 SMP3
Sender [Mbps] Receiver [Mbps]
λ [%] Sender [Mbps] Receiver [Mbps] λ [%] Sender [Mbps] Receiver [Mbps] λ [%]
Mean CoTV Mean CoTV Mean CoTV Mean CoTV Mean CoTV Mean CoTV
53
For data rate of 5.5Mbps as tabulated in Table 3.11, the average received throughput increases as payload size increases for all the three networks as stated above except at some payload sizes. Network with SMP1 has the highest average received throughput compare to the network with SMP2 and SMP3. At higher payload size they all have comparable throughput as shown in the Figure 3.9.
Figure 3.9: Sender and Receiver average Throughput at 5.5Mbps.
From Figure 3.9 we can also observe that at lower payload size for SMP1 and SMP2 from 400 bytes up to 800 bytes, and for SMP3 from 400 bytes to 1200 bytes the network experiences packet loss in which results the network throughput at the range of PL’s is relatively smaller. The reason could be the
narrow IFG between packets at the sender creates more strain in the short distant network [31, 39]. Furthermore the data rate at the lower layer (layer 1 and 2) for shorter PL are higher than longer PL’s which ultimately might
54
be due to the tethering software treatment of packets as previously discussed in section 3.4.2. The tethering software of SMP1 holds packets for longer period of time and keeps them from getting lost in the network. SMP3 has lowest throughput this is due to it is limitation in CPU RAM and processing speed.
As reported in [13] it has been evaluated for WLAN based setup as transparent network, i.e. the sender and receiver Application-perceived throughput values have very small difference. In this thesis it has been shown similar trend for R=1Mbp and R=2Mbps with some packets losses i.e. the sender is slightly greater than the receiver throughput. But for R=5.5Mbps and shorter PL’s the sender and receiver throughput have at
worst case of 46% difference due to packet losses as show in Figure 3.14 . This setback can be accounted due to the efficiency of packet processing and buffer management of the SMPs [2]. The packets enter the SMP USB interface (480Mbps) and leave the WLAN (11Mbps) interface, i.e. the incoming packets are pumping into high speed and exit with relatively
bottleneck at the exit WLAN interface. Hence for shorter PL (400-800bytes
55
Table 3.11 Sender and receiver average throughput at R=5.5Mbps ΔT=1s
PL [Bytes]
SMP1 SMP2 SMP3
Sender [Mbps] Receiver [Mbps]
λ [%] Sender [Mbps] Receiver [Mbps] λ [%] Sender [Mbps] Receiver [Mbps] λ [%]
Mean CoTV Mean CoTV Mean CoTV Mean CoTV Mean CoTV Mean CoTV
56
Another experiment was performed for longest PL=1472bytes without
fragmentation in the WLAN link at R=5.2Mbps using Exp. No. 8. As shown in Figure 3.10 the average throughput at the receiver are estimated 5.1185Mbps, 5.1187Mbps and 5.099Mbps for each network having SMP1, SMP2 and SMP3 respectively. From this we can see that the received throughput for the networks of using SMP1 and SMP2 have more or less same average throughput but they have greater average throughput than SMP3. This decrease in throughput at the receiver side is contributed due to loss of packet roughly 1.9%.
Figure 3.10: Throughput vs Time at PL=1472 for R=5.2Mbps.
57
difference shows burstiness of traffic in the network [13]. The statatstical throughput difference of the network might be contributed to by the difference in hardware and software capabilty of each SMP. The network throughput ECDF in Figure 3.11 using SMP1 and SMP2 also show close probability distribution between them. But ECDF distribution for SMP3 lies beneath the two SMP’s.
Figure 3.11: CDF graph for probabilty (P(X>x)) versut throughput
PL=1472byte at R=5.2Mbps.
58
3.6 Packet Loss Ratio Analysis
To observe the effect of transmission pattern variation on packet loss the three experiments performed at R 1Mbps, 2Mbps and 5.5Mbps are used here. For R=8kbps there were not noticeable packet losses in the network traces. Hence the packet loss ratio for lower R are considered negligible. From Table 3.12, Table 3.13 and Table 3.14, it’s observed that when the data rate increases from 1Mbps to 5.5Mbps the packet loss incresease. And for each increase in the payload size the packet loss decreases for each data rate.
Figure 3.12 Packet loss ratio at R=1Mbps.
59
packets [31].
Table 3.12 Packet loss ratio at R=1Mbps.
Packet Loss Ratio at R=1Mbps PL [Byte] SMP1 [%] SMP2 [%] SMP3 [%] 400 0.02160 0.20658 0.01071 450 0.03009 0.15491 0.00602 500 0.00602 0.14001 0.00201 550 0.00100 0 0.00802 600 0.00739 0.00902 0.00902 650 0.01303 0 0.53007 700 0.00301 0.00100 0.02705 750 0.00401 0.00739 0.01302 800 0.01202 0.41569 0.01903 850 0.00300 0.10616 0.15324 900 0.00300 0.09414 0.01702 950 0.00130 0.08354 0.01102 1000 0.13420 0.21127 0 1050 0.00901 0.23453 0.00991 1100 0.00400 0.07509 0.00400 1150 0.00501 0.06407 0.01629 1200 0.00767 0.07520 0.00300 1250 0.00544 0.05706 0.05033 1300 0.01201 0.06000 0.09009 1350 0.18690 0.06846 0 1400 0.02102 0.06105 0.01101 1450 0.00300 0.05387 0.00200
60
Table 3.13 Packet Loss Ratio at R=2Mbps
Packet Loss Ratio at R=2Mbps
61
Figure 3.13 Packet Loss ratio at R=2Mbps.
From the Table 3.14 at R=5.5Mbps for network with SMP1, payload size range from 400 byte to 550 byte the packet loss ratio decrease from 17% to 7.7% and for the network with SMP2 payload sizes range of 400 bytes to 850 bytes the packet loss ratio decrease from 29.4% to 0.1%. For network having SMP3 the loss rate become higher and the range of payload size with loss become wider than network of SMP1 and SMP2 as shown in Figure 3.14.
The loss rate decreases with increasing payload sizes for each SMP. This is due to the fact that shorter payload sizes have higher ratio of lower layers headers (42bytes) in addition to the extra MAC and PHY wireless headers such as DIFS (50µs), PLCP Preamble (144bits) and Header
(48bits), SIFS (10µs), ACK (304µs) and so on [29, 31]. Moreover for the
same R the IFG for shorter length packets is smaller hence packets
leaving the sender are more denselyand have high probability of causing
62
Figure 3.14 Packet Loss Ratio at the R=5.5Mbps.
Even though it is difficult to completely rule the loss effect inside the network is solely due to SMPs. But as previous section show GW and AP have less chance of losing packets at the given range of PL‘s. The
loss of packets in the network might be most likely contributed due to sender (SMP) hardware, OS, and application software activity and in addition to the wireless link losses. By repeating same experimental processing with same setting the losses using different SMPs show differently with trend of losses in the end to end network. As media have
less interference from the surrounding, the packet losses contributed by
63 Table 3.14 Packet Loss Ratio at R=5.5Mbps
Packet Loss Ratio at 5.5Mbps PL [bytes] SMP1 [%] SMP2 [%] SMP3 [%] 400 17.55803 29.47766 45.92078 450 12.0986 24.30755 42.10361 500 7.7947 19.71775 37.79728 550 3.54861 15.42569 35.35209 600 0.46063 15.39278 31.33775 650 0.00023 11.22914 29.88684 700 0.00566 10.66113 28.2638 750 0.00848 9.42066 27.09566 800 0.00724 0.25684 22.32198 850 0.0064 0.10889 16.79361 900 0.00819 0.0036 12.60205 950 0.00155 0.00694 8.93296 1000 0.00101 0.00743 12.89224 1050 0.00088 0.00417 8.48915 1100 0.0068 0.00024 1.75552 1150 0.00669 0.0021 5.10814 1200 0.00805 0.01002 0.0192 1250 0.0041 0.0034 0.01207 1300 0.00171 0.00915 0.01736 1350 0.00442 0.00686 0.00329 1400 0.00348 0.00012 0.01042 1450 0.0067 0.00669 0.04222
64
3.7 Throughput difference between DPMI and Iperf.
To observe the estimated throughput difference between DPMI and Iperf, the Iperf at the SMP and the receiver are made to display the bitrate at each interval of 1 s throughout the observation time of 100 s. From Exp. No. 9 as shown in Table 3.15 and Table 3.16, the throughput of DPMI is greater than the Iperf measured output at the receiver. However, the calculated throughput and the Iperf output reported result for both SMP and receiver shows small differences.
In addition the standard deviation of DPMI is less than the receiver workstation and sender SMPs. This shows a sign of loss or burtiness in the network but under the given experimental condition this is not the case because the sending rate (1Mbps) is much less than the available data rate (11Mbps). Furthermore the standard deviation comparison between DPMI and Iperf depend on the type of SMP. SMP1 and SMP2 have less throughput standard deviation as show in Table 3.16. SMP3 has higher standard deviation throughput compared with the two SMPs.
Table 3.15 DPMI vs. Iperf average throughput output at R=1Mbps
SMPs Average Throughput at R=1Mbps Sender (SMP) DPMI Receiver (PC) SMP1 1000070 1000188 999835.2 SMP2 1000070 999956.4 999952.8 SMP3 996025 997389.1 991368
65
Iperf which is higher than the DPMI (<100ns) [15, 36]. Most importantly SMPs have very limited resource in hardware and OS. Longer period traffic injection into SMPs would deteriorate the network performance of SMPs (by more than 50%) hence using SMPs as a MP would lead to erroneous result.
Table 3.16 Standard deviation DPMI vs. Iperf at R=1Mbps.
SMP Std.dev [bps] SMP DPMI Receiver SMP1 4397.01 7142.91 7282.04 SMP2 2856.25 3303.96 3903.93 SMP3 85081.1 26245.9 90584.8
As show in Figure 3.15, Figure 3.16 and Figure 3.17, the SMPs throughput graph and the DPMI show a similar trend while the receiver show deviated from the sender. The throughput of SMP2 has smaller upper hand compared with SMP1 but the standard deviation is vice versa. SMP3 has lowest average throughput than the two SMPs.
66
As reported in [2] et. al. they are able to compare the network performance of 3G network with Iperf like measurement tool. But application level measurements are seriously affected by the OS, primarily scheduling and
network stack process [11, 12]. In comparing the SMPs, the hardware,
software, OSs, network stack architecture and interference play important roles. Hence due to the above mentioned reason it is difficult to get accurate results when we measure network performance of SMPs using application software measurement tools.
In the next chapter the paper will be finalized by discussing the key points and contributions that the thesis provide. In addition, future works will be proposed.
67
68
C
HAPTER
4
Conclusions and Future work
4.1 Conclusion
In this paper we discussed the effect of traffic pattern on OWD, throughput and packet loss on SMP network performance and their comparison with each other based on the selected metrics. The estimation of these parameters was conducted using hardware based experiment which can give timestamp accuracy of less than 100ns. The experimental setup was controlled to ensure less effect in underwent experimental procedure.
From experimental data we were able to answer our research questions, the
first RQ1: aimed to answer the effect of transmissionpattern on OWD, from
the experimental results as payload size increase the minimum OWD of the network also increase for each SMP. Using simple linear regression curve-fitting the minimum OWD and payload size have approximately direct relationship. Based on the minimum OWD comparison, the network with SMP2 has the lowest OWD and SMP3 has the highest OWD. The minimum OWD of the SMPs directly relates with the efficiency of SMP in forwarding packets. The effects of GW and AP on OWD of the network have been evaluated. The contribution of GW and AP on the overall OWD variation of the network is evaluated to be insignificant. Moreover it has been investigated that the OWD variation and huge spikes are due to the retransmission of 802.11 algorithms in MAC and PHY layer. In this course of action particularly the stop and wait ARQ for 802.11b technique plays a great role in packet loss and OWD distribution of the network.
69
becomes wider for a given R, so the probability of packet loss decrease
which results in higher throughput but for shorter PL the opposite
phenomenon was observed i.e. the network tends to lose packets. Comparing the SMPs based on throughput, we have observed that in addition to the SMPs hardware performance the underlying tethering application has advantage in handling buffer of the SMPs during forwarding of the packets. Using the third research question RQ 3: the effect of transmission patterns on packet loss ratio of the network is evaluated, the experimental result shows that for each increase in data rate the packet loss ratio also increases for all the SMPs.
In our last RQ 4: The difference in SMPs throughput between the common network performance evaluation tool Iperf and DPMI have been evaluated. From the estimated result we have seen that there is a throughput difference between DPMI and Iperf. In our analysis the DPMI result is compared with
the receiver side throughput and DPMI gives better accuracy inthroughput
measurement. The estimated throughput of the network is affected due to the accuracy of the timestamp of SMPs, consideration of fractional PDU,
the limitation of hardware and software resources of the SMPs. From each
experiment and comparison of SMPs using higher precision experimental setup, application developers can tune their design based on the network performance of SMPs to enhance the QoE for end users.
4.2 Future Work
To extend the work with more SMPs having variety of platforms.
To analyze the performance of the SMPs on a controlled 3G
network.
In our work we used UDP protocol as a transfer protocol, as a future
70
B
IBLIOGRAPHY
[1] AdMob Mobile Metrics (2010. May) [online. Accessed December. 2010]. Available: http://www.filesocial.com/mbuynk
[2] J. Huang, Q. Xu, Z. M. Mao, M. Zhang, P. Bahl, “Anatomizing Application Performance Differences on smartphones,” from Microsoft Research, University of Michigan, 2010.
[3] R. Gass, C. Diot, "An Experimental Performance Comparison of 3G and Wi-Fi, " “From Passive and Active Measurement Conference (PAM).” Zurich, Switzerland, April 7-9, 2010.
[4] P. Arlos, M. Fiedler , "Influence of the packet size on the one-way delay on the down-link in 3G networks," from 5th IEEE International Symposium on Wireless Pervasive Computing (ISWPC), 5-7 May 2010, pp.573-578.
[5] R. Kommalapati, “Performance Of 3G data Services over Mobile Network in Sweden,” MSc Thesis, Dep. of Telecommunication System at School of Computing (COM), Blekinge Institute of Technology (BTH), Karlskrona, Sweden, January 2010.
[6] P. Carlsson, D. Constantinescu, A. Popescu, M.Fiedler, A.A.Nilsson, “Delay Performance in IP Routers,” From 2nd International Working Conference (HET-NETs '04), Ilkley, England, 2004.
[7] E. E. Reber, R. L. Michell, and C. J. Carter, “Smartphone Efficiency Report,” Rysavy Research, LLC, Hood River, USA, Jan. 2011.
[8] H. Falaki, et.al. “A First Look at Traffic on Smartphones,” from international communication conference, Melbourne, Australia, Nov. 2010.
[9] Gigaom, [online Accessed August, 2011]
Available: http://gigaom.com/mobile/smartphone-wi-fi-usage-on-the-rise/
[10] A. Kogan, “On Porting Iperf to Windows Mobile and Adding BlueTooth Support, “ Department of Computer Science, Teknion, Israel Institute of Technology, Technion, Israel, April 2010.
