Master Thesis Electrical Engineering October 2013 QoE of Video Streaming over LTE Network

(1)

Master Thesis

Electrical Engineering

October 2013

QoE of Video Streaming over LTE Network

Pradeep Uppu, Sushanth Kadimpati

School of Computing

Blekinge Institute of Technology 37179 Karlskrona

(2)

This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfilment of the requirements for the degree of Master of Science in Electrical Engineering. The thesis is equivalent to 20 weeks of full time studies.

This Master Thesis is typeset using LA_TEX

Contact Information

Author(s): Pradeep Uppu

Address: Karlskrona, Sweden E-mail: pradeep.uppu@gmail.com Sushanth Kadimpati

Address: Karlskrona, Sweden E-mail: sushanthkad@gmail.com

University advisor(s):

Dr. Adrian Popescu, Prof. COM/BTH

School of Computing

Blekinge Institute of Technology 371 79 KARLSKRONA SWEDEN

(3)

Abstract

In recent years, the mobile Internet has increased dramatically with the development of 3G and 4G technologies. Especially the usage of mobile broadband internet on the devices like cellular mobiles, Tablets and Laptops has skyrocketed. Among the multimedia applications video streaming is the most popular mobile application. But, making these services available to users in a cost effective way without compromising quality is a big challenge. The development of Long Term Evolution (LTE) technology in the mobile world made this task achievable. The features of LTE technology provide effective services in multimedia applications with high data rates and low latency.

In this paper, we study and analyze the Quality of Experience (QoE) at the end user for Video on Demand (VoD) over the LTE network. To achieve this, we streamed High Definition (HD) videos based on H.264/AVC and these videos are delivered from source to destination using Transport Control Protocol (TCP) and User Datagram Protocol (UDP). Specifically, our study is about QoE evaluation in terms of delay variation, packet loss metrics and provides performance evaluation to characterize the impact of transport layer protocol in video streaming over radio networks like LTE. In order to know the performance of video streaming over LTE network, we also evaluate the LTE performance in terms of one-way delay, packet loss and inter packet delay for the generated UDP and TCP packets.

Keywords: QoE, Video Streaming, H.264/AVC, LTE, One-way Delay, Packet loss.

(4)

Acknowledgements

It gives us great immense joy in acknowledging Prof Adrian Popescu for his diligent support and extending the opportunity to pursue our master thesis under his immaculate supervision. We would like to thank Dr. Patrik Arlos for providing the experimental test bed and valuable suggestions in the evolution of this thesis. We would also like to thank Wowza Media Systems for providing the software. A word of thanks to Mr. Tahir Minhas Nawaz for his valuable suggestions and tips. We are indebted to our friends and family for their constant support and prayers that helped us complete the thesis.

This thesis is dedicated to our parents who stood beside us through thick and thin in making this thesis substantial.

Pradeep Uppu Sushanth Kadimpati

(5)

List of Figures

2.1 Quality of Experience Measurement . . . 10

2.2 Evolution of LTE . . . 15

2.3 LTE Radio Access Network . . . 18

3.1 Detailed Experimental Set up . . . 21

3.2 Experimental Set up for Video Streaming . . . 25

3.3 Screen Shot of QoE Evaluation Tool . . . 27

4.1 Minimum One way Delay for Generated UDP Packets for Uplink . . . 30

4.2 Maximum One way Delay for Generated UDP Packets for Uplink . . . 31

4.3 Mean One way Delay for Generated UDP Packets for Uplink 32 4.4 Minimum Inter Packet Delay for Generated UDP Packets for Uplink . . . 33

4.5 Maximum Inter Packet Delay for Generated UDP Packets for Uplink . . . 33

4.6 Mean Inter Packet Delay for Generated UDP Packets for Uplink 34 4.7 Packet Loss for Generated UDP Packets for Uplink . . . 34

4.8 Minimum One way Delay for Generated TCP Packets for Uplink 35 4.9 Maximum One way Delay for Generated TCP Packets for Uplink . . . 36

4.10 Mean One way Delay for Generated TCP Packets for Uplink 36 4.11 Minimum Inter Packet Delay for Generated TCP Packets for Uplink . . . 37

4.12 Maximum Inter Packet Delay for Generated TCP Packets for Uplink . . . 38

4.13 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38 4.14 Packet Loss for Generated TCP Packets for Uplink . . . 39

4.15 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink . . . 40

4.16 Minimum One Way Delay for Generated TCP and UDP packet size of 1500 bytes for Uplink . . . 41

4.17 MOS for TCP videos subjected to Delay Variation . . . 43

(9)

4.18 MOS for UDP videos subjected to Delay Variation . . . 44

4.19 Standard Deviation for Delay Variation . . . 45

4.20 Confidence Interval for Delay Variation . . . 46

4.21 MOS for TCP videos subjected to Packet loss . . . 47

4.22 MOS for UDP videos subjected to Packet Loss . . . 47

4.23 Standard Deviation for Packet Loss . . . 48

4.24 Confidence Interval for Packet Loss . . . 49

(10)

List of Tables

2.1 Five-level scale for rating overall quality of video . . . 13

3.1 Test Video Parameters . . . 26

4.1 MOS for Delay Variation . . . 43

4.2 MOS for Packet Loss . . . 46

(11)

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska H¨ogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

(12)

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union, Radio Communication Sector

ITU-T International Telecommunication Union, Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

(13)

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

(14)

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

(15)

Introduction

(16)

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited to provide voice communications alone, but also used for the video and data applications as well. Due to the rapid development of technology used in telecommunication systems and consumer electronics, network operators are now able to provide better Internet services over radio networks. After the development of 2G and 3G technologies, the Internet based services are available on mobile systems, namely mobile broadband.

According to CISCO report, mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and 2017, and it is the highest growth rate of any other mobile application [1]. Meeting this demand, maintaining the Quality of Service and user satis-faction has become a big challenge to network operators. To achieve this a radio interface is needed, which can provide the best Quality of Services with design parameters like Data rates, Delay and Capacity.

One of the main reasons for LTE evolution is to provide the IP based services to people on mobile devices with better QoS [2]. Some services have been already provided by 3G networks, but providing HD video streaming, interactive video gaming and other multimedia services without degrading the Quality of Services is a major challenge. Among these multimedia ser-vices, video streaming over mobile Internet is the most popular one. In the burst growth of data rates and services, being aware of user experience is important to maintain the service quality and the application performance. The Quality of Experience is defined as “The process of understand-ing the actual performance of services, as delivered to the customer, for the purpose of ensuring those services meet customer expectations and re-quirements”. Understanding user experience is very critical for the network operators in managing the QoS of the network. Quality of Experience mea-surements are made at the point of delivery directly from the subscriber’s smart phone or PC. QoE measurements deals with how well applications (Video streaming, VOIP and Web browsing) work in the hands of subscriber

(17)

CHAPTER 1. INTRODUCTION 3

[3]. QoE measurements require an understanding of the Key Performance Indicators (KPI) that impact on the user perception. KPI’s vary with the service type and services like VoIP, Video streaming, On-line gaming, In-ternet browsing has unique performance indicators to measure [4]. QoE considers the individual subscriber experience with a service unlike network conditions in QoS. The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and then operators can concentrate on issues to prevent the churn.

The mobile communication technologies are tracked back to various gen-erations 1G, 2G, 3G and 4G. 1G which started in 1980 stands for first gener-ation of wireless telecommunicgener-ations popularly known as Cellular phones, in which analog radio signals are used. In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2]. The sec-ond generation mobile technologies provide low bandwidth services, which are suitable for voice traffic. The packet data over cellular systems started with the introduction of GPRS (General Packet Radio Services) in GSM (Global System for Mobile Communications). With the introduction of 3G technology, network operators can provide better and more advanced ser-vices like video calls and mobile broadband serser-vices.

In our thesis, we worked on user quality assessment for video streaming over LTE network. We report on user perception of video quality degrada-tion of selected videos, which are subjected to different delays and packet losses. This thesis is specifically aimed to understand the experience of High Definition videos with H.264/AVC. We report the user experience by con-ducting Subjective Assessment as per the International Telecommunications Union (ITU) [5].

In addition to this, we also studied the Uplink behaviour of the LTE network by calculating the One-way Delay, Inter Packet Delay and Packet losses with different payloads at different data rates. We analyzed the Qual-ity of Service of video packets over LTE network with OWD, IPD and Packet losses as QoS metrics.

1.1 Motivation

LTE is the latest technology in the telecommunications world using which network operators are able to provide advanced multimedia applications to users maintaining Quality of service [2].

(18)

Definition videos on the mobile devices and at the same time the mobile systems are being manufactured to access advanced services.

In this current day of huge resource demand applications, understanding the user experience is very critical for the network operators to manage the QoS of the network and hence our motivation to measure and analyze it. We choose High Definition videos with 1280 × 720p and H.264/AVC, since H.264 codec is the widely used codec for video streaming. As compared to the previous codec like MPEG-2 and MPEG-4 Visual, AVC provides good quality of video for wide range of services like broadcast multimedia streaming and Video on Demand.

The QoS and QoE are so interdependent and they have to be studied and managed with common understanding. So, as a part of QoS evaluation we conducted experiments to measure the OWD, IPD and packet loss for gener-ated TCP, UDP packets over LTE network for Uplink. We conducted these experiments to know the behaviour of TCP and UDP packets for different data rates with different payloads. We chose uplink, since video sharing became popular with the emerging of websites like YouTube, Facebook and Vimeo. According to YouTube statistics 72 hours of video are uploaded to YouTube every minute [6].

1.2 Related Works

In paper [7], the authors proposed QoE assessment models for video stream-ing services like IPTV by usstream-ing QoS parameters in network layer. This helps the network service provider in providing multimedia services with improved QoE by using the suggested QoE assessment process. This also helps the network service provider to prevent the unnecessary expenses for mainte-nance and repair of the network.

In paper [8], the authors evaluated the QoE measurement in a live 3G network with automatic data capturing tools are used in the experiment. Evaluation is carried out by subjective methods relevant to a set of objective parameters. The author concluded that the QoE of mobile video streaming was influenced by the QoS and with the context also.

(19)

and proposed a flexible framework well capable to correlate with both packet loss ratio and subjective quality degradation. The proposed metric and the framework provide help for performing easy and accurate QoE evaluations on streaming applications.

In paper [12], the authors presented a new model for non-intrusive pre-diction of H.264 encoded video quality over UMTS networks. The test bed was evaluated on the NS2 based UMTS simulation network. The proposed model was based on combination of a set of objective parameters in the physical and network layers, in terms of Mean Opinion Score (MOS).

In paper [13], the author proposed a conceptual model of QoE, cor-responding to the hourglass model of Internet architecture based on the streaming video quality of experience and its factors. This model of QoE has four layers, similar to the Internet hourglass model, and each layer has a role towards the user perceived quality.

In paper [14], the author analyses the user perception towards the QoE of video, which is encoded with H.264 baseline profile and streamed through an emulated network with packet loss and packet delay variation. The ex-periment was conducted both on a laptop and a mobile device.

In paper [15], the authors analyzed the effect of QoS parameters like end-to-end delay, packet loss and packet delay on the performance of video conferencing in the LTE network. The authors carried out their experiment on OPNET 16.0 [16] simulator. The results are evaluated by taking three network scenarios namely low load, medium load and high load.

In paper [17], the authors analyzed the impact of payload size and data rate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden. The measurement in the network are carried out by using Endace DAG [18] cards and the Endace DAG are synchronized with GPS to get an accurate measurement.

In paper [19], the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements. The experimental setup is designed in such a way to get perfect time synchronization and accurate results. The authors concluded that the one-way delay of uplink is much higher than the one-way delay of downlink.

(20)

1.3 Contribution

The experiments were conducted in two phases. In the first phase, we con-ducted the video quality assessment using subjective method. Videos are transmitted over LTE network and subjected to different delays and packet losses. There has been previous works [21] [22] done on video streaming over 3G networks and other wireless networks. However most of the works are based on simulations and objective analysis using Peak Signal-to-Noise Ratio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) tools. PEVQ tool is recommended by ITU, but it has limitation of video length not more than 20 seconds [23]. We adopted subjective analysis method, which gives better user opinion than objective analysis. In wireless radio networks it is hard to predict the network behaviour with less than one minute video [24]. So we choose the videos, which are having a duration of four minutes. The Quality of end user experience affected by the technical factors of QoS (network quality and network coverage) and non technical Subjective factors (service content, ease of service setup and pricing). So, we attempted to know the performance of LTE network by measuring QoS metrics that are OWD, IPD and Packet loss for uplink. We measured these metrics on an experimental test bed where OWD is calculated with the packet traces collected at the link level. It gives the possible precise measurements up to 60 nano seconds [25]. With the collected traces we calculated the OWD, IPD and Packet losses using Perl program.

1.4 Aims and Objectives

This Thesis aims at studying the user experience on video quality with the parameters packet loss and delay/delay variance over LTE network and H.264 as video codec. Another aim is to know the LTE network behaviour in terms of OWD, IPD and Packet loss for generated traffic on UDP and TCP protocols.

The objectives are as follows:

• To understand the user perception of video quality for high definition videos with packet losses and delay variations over LTE network, with H.264 as codec.

• To understand the user perception of video quality for different pro-tocols.

(21)

1.5 Research Questions

1. What is the effect of TCP and UDP protocols on OWD, IPD and Packet loss in LTE network uplink with different payloads at different data rates ?

2. How is the user perception affected by the delay variation in video streaming over LTE network using TCP and UDP protocol ?

3. How is the user perception affected by the packet loss in video stream-ing over LTE network usstream-ing TCP and UDP protocol ?

1.6 Thesis Outline

(22)

Technical Background

(23)

Chapter 2

Technical Background

2.1 Quality of Experience

QoE is also defined [23] as “The overall acceptability of an application or service, as perceived subjectively by the end user”. It mainly deals with how an individual is satisfied with the provided service in terms of usability, accessibility, retain ability and integrity of the service. Quality of Experience considers the complete end-to-end system effects like, the effects of the client, network and infrastructure of the services. It also takes into consideration the end user’s mood, emotions and physical status, which encompasses the psychology of the end user. So, there is no particular defined statement for QoE, that is accepted universally [26].

QoE considers the individual subscriber experience with a service unlike network conditions in QoS. The knowledge of actual user experience is very important for the operators to meet the customer’s satisfactory levels. In doing so, operators can turn their attention on issues to prevent the churn.

2.2 Video Streaming

Today video sharing is the most effective way of entertainment and gaining knowledge. In order to share a video it must be stored and transmitted over a communication channel, but it is expensive to transmit a raw video over a communication channel because of the huge amount of data size. Even to store a raw video, it requires a lot of space on the data storing devices. Usually the video taken from camera footage contains lot of redundant data. So, there is a need to reduce the size of the redundant data in the raw video also considering the quality of the video. Here video compression comes into the picture, to reduce the redundant data by considering the quality of the video [27].

(24)

CHAPTER 2. TECHNICAL BACKGROUND 10

Figure 2.1: Quality of Experience Measurement

2.3 Video Compression

Video Compression is a technique where the raw video is compressed using mathematical models and algorithms to reduce the size of the video to lower bit rates. Video compression is an essential process for video applications like Internet video streaming, mobile TV, video conferencing, digital televi-sion and DVD-Video [28]. Video comprestelevi-sion is mainly classified into two types lossless compression and lossy compression. In lossless compression no information is lost in the compression process. So it is possible to recover the original raw video from a compressed video. In practical cases, the amount of data reduced is less. Quality of the video is maintained well in lossless compression [29]. This type of video compression is not used for streaming videos because even though video is compressed, it still maintains a large data size.

(25)

compres-CHAPTER 2. TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously considering the video quality.

2.4 Video Format

The video format consists of two distinct technology concepts: Video Codec and Video Container.

2.4.1 Video Codec

Video compression process involves a complementary pair of systems, a com-pressor (encoder), which converts the source video into compressed form before the transmission or storage and a decompressor (decoder) which con-verts the compressed data, back to represents the original data. So, this encoder-decoder pair is called as CODEC. Video codec is a software pro-gram that compresses a raw video into a compressed video of small data size in a way that the compressed video must play on the computer, since raw or uncompressed data requires a large bit rate, approximately 216 MB per second [28]. Video codec can only compress a video, similarly audio codec is used for audio file and played along with video files.

Different types of codes are available to compress raw video and the selection of video codec depends on various factors like size of video, type of video streaming and type of application [31]. Nowadays there are many video codes available for video compression, some of them are MPEG-1, MPEG-2, MPEG-3, MPEG-4, H.264, and Vorbis.

Among all available video codes H.264 is the most widely used codec. Main features of this codec are, providing better video quality at lower bit-rates, fast encoding speed and other advanced features make H.264/AVC better than its counterparts [28].

2.4.2 Video Container

Video Container describes the structure of the video, in which way it has been compressed and stored [32]. Video codec compresses the raw video into a format, which is considered as the video container. Examples of the video containers are .avi, .mp4, .mov, and .asf.

2.4.3 H.264/AVC Codec

(26)

applications, from low bit rate mobile video applications to high definition broadcast services.

2.5 Types of Video Streaming

Nowa-days different types of video streaming applications are in use. Some of them are point-to-point communication, broadcast communication and multicast communication. All these video streaming applications are mostly depends on two video streaming types. One of them is live video streaming, which is a process of real time transmission of a video over Internet [33]. So the streamed video file can be viewed on smart phones, mobiles and personal computers. The video application is mainly used in live sports broadcasting. Another type of video streaming is Video-on-Demand, this video stream-ing process is quite contrary to the previous type. In this video streamstream-ing, the selected video file is played whenever the viewer wishes to watch the video. In this type of streaming, one-to-many viewers can view the same or different video [34]. This process is mainly observed in video streaming websites like YouTube, Hulu and DailyMotion.

2.6 Supported Protocols for Video Streaming

There are several protocols that support media streaming. Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP), Session Description Protocol (SDP), Real-time Streaming Protocol (RTSP), Real-time Trans-port Protocol (RTP), Real Data TransTrans-port (RDT) and Real-time TransTrans-port Control Protocol (RTCP) [35]. Each protocol is used depending on the type of application in that particular point and also depends upon the require-ment of service. For most of the video streaming protocols, TCP and UDP serves as the underlying protocol. UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at a constant rate and it doesn’t care about the lost packets. But, TCP is also widely used in video streaming because of benefits of streaming with TCP, including its retransmission capabilities, congestion control and flow control [36]. On the same hand, TCP introduces delay due to the retransmission of data whenever data is lost. But in case of UDP, there is no point of retransmission.

2.7 Assessment of Videos

When it comes to the point of video quality assessment, there are mainly two types of methods. They are as follows.

(27)

• Subjective Assessment.

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equals to the subjective video quality assessment and it does not involve any hu-man grading. It is a software program designed to deliver results based on error signal ratio of the original and processed video. The most popular Objective methods are Mean Squared Error (MSE), Perceptual Evaluation of Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37] [38]. This method has few categories referred as Full-Reference (FR) and Reduced - Reference (RR) video quality metrics [39].

The other video assessment includes Subjective analysis, which is based on human perception. The subjective video quality assessment is considered as accurate way of measuring quality of video compared to objective assess-ment. For subjective video quality assessment, a set of videos are given to the subjects for rating the videos on a scale of five (5) and the grades for quality is given in Table 2.1. The rating given by the subjects are known as Mean opinion Score (MOS). The subjects include experts and non- expert observers.

MOS Rating User Opinion 5 Imperceptible

4 Perceptible, but not annoying 3 Slightly annoying

2 Annoying

1 Very annoying

Table 2.1: Five-level scale for rating overall quality of video

2.8 Overview of 3GPP Releases

(28)

3GPP has developed different mobile technologies and those technologies are differentiated by releases. A brief overview of different 3GPP releases from GSM to LTE-Advanced is as follows:

In Releases 99 [40], the GSM specifications of the Universal Terrestrial Radio Access Network (UTRAN) are developed. UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) in January 1998. Mobile technologies like EDGE (Enhanced Data rates for GSM Evolution) provides high data rate than GPRS, which offers services like messaging, email, etc. Majority of technologies in usage are based on release 99.

Release 4 [41], is provided with minor improvements in UMTS network. It mainly concentrates on Multimedia Messaging support, which includes development of the MMS Reference Architecture, MMS service features, streaming Support in MMS and a lot more.

In release 5 [42], the 3GPP featured a new technology called HSPDA, which helps the wireless operators to provide the customer need of high speed wireless data services with improved spectral efficiency. In the same release it also introduced the IP Multimedia Subsystem (IMS) architecture. IMS enhances the end user experience for multimedia application and also facilitates the use of IP (Internet Protocol) for packet communications in all wireless networks [43].

Release 6 [44], is mainly concentrated on Quality of Service (QoS) for multimedia applications. Enhanced Multimedia Broadcast/Multicast Ser-vices (MBMS), which is a user service, enables data to deliver to a set of users using same radio resources within a service area like High Speed Uplink Packet Access (HSUPA) for high uplink speed.

Release 7 [45], focuses on decreasing latency, improved QoS for the real-time applications such as gaming, VoIP. In this release the spectral efficiency of the HSPA is increased by the introduction of Input Multiple-Output (MIMO) antenna systems. Enhancements to GSM with Evolved EDGE which increases usual throughput rates, reduces the latency by two and improves spectral efficiency.

Release 8 [46], consists of evolution of HSPA features such as 64 QAM and simultaneous use of MIMO. Innovation of LTE technology, which is ex-pected to meet the high data rate requirements of the end user. Reduced user plane latency resulting highly improved user experience with full mo-bility. Using single-carrier frequency domain multiple access (SC-FDMA) for uplink and orthogonal frequency domain multiple access (OFDMA) for downlink. It introduces Evolved Packet System (EPS) to provide network architecture, which integrate common mobility, security and QoS mecha-nisms for fixed and mobile broadband accesses.

(29)

cells. It also initiated the study of Advanced LTE.

In Release 10 [48], new concepts in LTE-Advanced are introduced like Carrier Aggregation (CA), multi-antenna enhancements and relays. It also includes Self Organizing Networks (SON), Heterogeneous Networks (Het-Nets), quad-carrier operation for HSPA+, etc. Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced.

In Release 11 [48], will build on the advancements and refinements of capabilities of technologies that are developed in release 10. Enhancements to relays, Carrier Aggregation and MIMO etc.

Release 12 [48], includes the study of network optimization for Machine Type Communication (MTC), Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability.

Figure 2.2: Evolution of LTE

2.9 Technical Overview of LTE

(30)

• The downlink speed is up to 100 Mbps and the technique used is Orthogonal frequency-division multiplexing (OFDM).

• The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA).

• It uses 4x4 antennas for downloading rates and it uses a single antenna for uploading rates.

• The data transmission in LTE is significantly low for application that use low latency such as VoIP, Online Multi player gaming etc.

• The user plane latency offered in LTE is less than 10 ms and the control plane latency is less than 100 ms.

• In the case of mobility, LTE supports up to 500 kmph but like other mobile technologies it will be optimized for lower speed from 0 to 15 km/h.

• LTE supports higher flexibility in carrier bandwidths, the set of band-widths actually supported are 1.25, 2.5, 5, 10, 15, and 20 MHz. • LTE is capable of delivering optimum performance in a cell size up

to 5 km but it still can deliver its effective performance up to 30 km. Whereas with its limited performance, it can deliver up to 100 km radius.

2.10 Architecture of LTE

The enhancement features like higher packet data rates and significantly lower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE). This includes the Evolved Packet Core (EPC) network. The Evolved Packet System (EPS) consists of LTE and SAE. It was decided to have a ”Flat Architecture”. EPS [52] is defined to support only packet-switched traffic. It uses the concept of EPS bearers to direct the IP traffic from a gateway in the Packet Data Network (PDN) to the User Equipment (UE). EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE.

Likewise the HSPA architecture the LTE architecture also divided into two networks as radio access network and a core network. However the ultimate goal of the LTE is to minimize the number of nodes. As a result of this, the Radio Access Network (RAN) contains only one node.

2.10.1 Core Network

(31)

Policy Control and Charging Rules Function (PCRF): PCRF provides the service management and control of the LTE services. It is responsible for policy control decision making besides regulating the flow based charging functionalities in the Control Enforcement Function (PCEF). It helps to have dynamic control over QoS in turn help operators, to provide customers with a variety of QoS and charging options when opting for a service.

Home Subscriber Server (HSS): HSS is the master database, it con-tains the user subscription data to support call control and session manage-ment entities. This includes the subscribed QoS profile and restrictions to access, when the subscriber is in roaming. It provides the authentication and authorization of subscribers. It holds the information related to the location of subscriber and the information about the Packet Data Network (PDN) to which the subscriber is connected. HSS also supports multi-access, multi domain networks, which provides end to end traffic handling facilities for the subscribers moving between LTE and Wireless Local Area Network (WLAN).

PDN Gateway (P-GW): P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets. It provides the QoS enforcement and flow based charging as per the policies in the PCRF. PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as per the subscriber profile. It acts as an anchor for mobility between 3GPP and non-3GPPP technologies.

Serving Gateway (S-GW): S-GW directs data packets to all sub-scribers, which acts as a local mobility anchor for data bearer that moves between eNB hand overs. It also acts as the anchor between LTE and 3GPP technologies. It gets the information about the bearer when the UE is in an idle state. S-GW terminates the Downlink data path and also triggers paging when DL data arrives for the UE.

Mobility Management Entity (MME): MME is the key control node for LTE access network that processes the signaling between UE and the Core network. It is responsible for choosing the SWG for a UE at the initial registration process and also for intra-LTE handover including Core Network (CN) node relocation. The main functions that are carried by the MME are related to the bearer management and connection management. Bearer management includes maintaining, establishment and release of the bearers. And the connection management includes establishment of the connection as well as security between the network and UE.

2.10.2 Radio Access Network

(32)

Packet Core by using one component called eNodeB. Unlike normal NodeB, eNodeB does not have centralized controller system and hence the E-UTRAN is regarded as a flat architecture. eNodeB is responsible for Radio Resource Management, which includes radio bearer control, scheduling and dynamic allocation of links to UE for uplink and downlink. Also it compresses the IP packet header for efficient use of resources. Encrypted data is sent over the radio interface for better security.

eNodeB’s are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1 control plane part. ENodeB’s are interconnected by means of X2 interfaces [55].

(33)

Design and Implementation

(34)

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we used for conducting the experiments. The section consists of explanation of two experimental setups, one for the evaluation of LTE network performance with OWD, IPD and Packet loss as QoS metrics, using generated traffic for uplink and the other for video quality assessment over LTE network using subjective analysis.

Before proceeding to our aim of QoE evaluation of video streaming over LTE network, we measured the QoS KPI’s of live LTE network. Because QoE and QoS are integral part of each other and the evaluation of QoE would not be complete without addressing the QoS [56].

3.1 LTE Network Performance Evaluation with

Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed, measured and understood in networks and network elements, which usually has little concerned to user [56].

In this section we used Distributive Passive Measurement Infrastructure (DPMI) in our experimentation, which is a dedicated hardware to perform measurements at the link level, to evaluate the performance of LTE network in terms of OWD, IPD and Packet loss for Uplink. The traffic generating tool is used to generate the TCP and UDP packets from sender system (S) to receiver system (R). The test bed is configured in such a way that the sender is connected to LTE network using Huawei E398 LTE USB modem having business type subscription and the receiver is connected to the BTH Internet. We configured our setup in such a way that the sender is able to send the TCP and UDP packets over LTE network and the receiver is able to receive those packets via BTH Internet. In between sender and receiver, we placed Measurement Point (MP) to capture the traffic. This MP gives the accurate time stamp of packets when it leaves from sender and arrives

(35)

CHAPTER 3. DESIGN AND IMPLEMENTATION 21

at the receiver.

3.1.1 Experimental Procedure

The detailed experimental setup is shown in Figure 3.1. The packet flow in this experiment starts from the sender system, which generates the UDP packets with the help of Traffic generator and it passes to Gateway. This Gateway sends packets to receiver through LTE network. The receiver sys-tem connected to BTH Internet receives the packets. The transmitted traffic at the sender and receiver end is fed to the MP through wiretaps, which are connected to it by monitoring ports. The time stamping of packets at MP does not effect the actual packet flow, since the wiretaps duplicates the packets without disturbing the original packets.

Figure 3.1: Detailed Experimental Set up

The traffic generator tool consists of two programs, one is client program and the other is server program. The client program is installed in sender system and the server program is installed in receiver system. The client program consists [20] of Experiment number (E), Run Id (R), Key Id (K), destination IP, destination port number, number of packets to be sent, length of packets and inter frame gap in micro seconds. The server program consists of Experiment number (E), Run Id (R), Key Id (K) [20]. The collected traces at the consumer are analyzed using Perl program, based on the sequence numbers along with above mentioned E, R, K values respectively. These values are used to recognize the packets at source and destination [20]. By analyzing the collected traces we calculate the minimum, maximum and mean of OWD, IPD and packet loss percentage for different payloads at different data rates. Based on the calculated values we plotted the graphs.

(36)

3.1.2 Gateway

Gateway is a Linux based computer operating on Ubuntu 12.04 OS, consist-ing of AMD Athlon processor and 2 GB RAM. The gateway is connected between sender and receiver as shown in Figure 3.1. The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernet card and the LTE USB modem is connected to the gateway. We used this gateway to access the LTE network, since the sender with this LTE USB modem cannot be connected to the Measurement Point. We generate the traffic using existing traffic generators [20] at the sender system. The gen-erated packets are sent to receiver through Internet via gateway and the receiver also communicates in the same way.

3.1.3 Sender

Sender is a Linux based computer operating on Ubuntu 12.04 OS, consist-ing of AMD Athlon processor and 2 GB RAM. It is connected directly to the Gateway. The sender generates the UDP and TCP traffic using traffic generator tool and it sends to receiver through Internet via gateway.

3.1.4 Receiver

Receiver is a Linux based computer operating with Ubuntu 12.04 OS, con-sisting of AMD Athlon processor and 2 GB RAM. It is connected to BTH network using Ethernet. It is used as a sink, to receive the UDP and TCP traffic transmitted from the sender system using traffic generator tool.

3.1.5 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets. It is equipped with Endace DAG 3.5E cards and these are enabled in such a way to capture the traffic without loss. It is synchronized to Network Time Protocol (NTP) server and GPS (Global Positioning System) to achieve the most possible accuracy in time stamping. By this we can achieve time stamp accuracy of 60 nanoseconds. The MP filters the packets according to the rules set by Marc [20]. The wiretaps connected at the sender and receiver ends are connected to the DAG cards.

3.1.6 Consumer

(37)

3.2 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD), Inter-packet delay (IPD) and Packet loss over LTE network.

3.2.1 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source to destination. In our case, OWD of packet having the same sequence number (S) is calculated by subtracting the time stamp at receiver dag interface (T(S, Dr)) with the time stamp at sender dag interface (T(S, Ds)).

OW D = T (S, Dr) − T (S, Ds)

Dr—Dag at receiver side. Ds—Dag at sender side.

In order to get fair one-way delay measurements, the clocks should be synchronized. Some of the available synchronization methods are Network Time Protocol (NTP) and Global Positioning System (GPS). By using NTP we can able to synchronize the clocks within [10 ms, 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarscenar-ios. In our case, for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock of MP’s in order of 60 ns. In practical case, it is difficult to get high accurate time stamp with two DAG cards (one at sender and other at receiver). In order to avoid this minor inaccuracy we used special wiring system, where we capture traffic on singe DAG card and get time stamp from same clock [25].

3.2.2 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number of packets lost (L) to the total number of packets sent by the sender (N). The number of lost packets is the difference in the number of packets received by receiver and the total number of packets sent by sender with unmatched sequence numbers.

PL= L/N

3.2.3 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at sender or receiver. In our experiment we calculated at receiver end and it repre-sented by following equation.

(38)

T—Time of arrival. i—ith packet. r—at receiver side.

We calculated the IPD in these experiments at the receiver end and considering the no delay variation at the source side, since we observed from our traces that delay variation at source end showed a constant rate in most cases. We can also calculate the IPD at the source side, but we need further study and moreover we are more concerned about the client side (receiver).

3.3 Video Quality Assessment using Subjective

Analysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes, jerk-iness in them. This might be the experiment was conducted in a region of good signal coverage. Then arises the thought of conducting the experiment in the worst network performance conditions. To achieve this, traffic shaper is used to introduce different delays and packet loss in the video streaming. In this experiment, two types of videos are taken and they are streamed over a live LTE network by introducing different delay variations and packet losses. The streamed videos are saved for video quality assessment.

3.3.1 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure 3.2. It contains the server, which is operating on Ubuntu 12.04, with AMD Athlon processor, 2 GB RAM and it is connected to the BTH network. The gateway is operating on Ubuntu 12.04, with AMD Athlon processor, 2 GB RAM and it is connected to LTE network via USB modem. The client runs on Ubuntu 12.04, with AMD Athlon processor and 2 GB RAM. It is connected to gateway via Ethernet full duplex link, bandwidth of 100 Mbps. The client system receives packets from server through LTE network via gateway. The Measurement point is connected in between the sender and the client. The MP consisting of two DAG cards, one of them is connected to capture the traffic at the server side and the other DAG card is used to capture the traffic at the client side. The traces are collected and saved in the consumer system.

(39)

we used the VLC player to access the videos. The VLC media player has a feature to save different videos that are being streaming over network. With this option in VLC player we saved the streamed videos in the client system. These streamed videos are saved in the client system. The saved videos are further used to get the Mean Opinion Score from the users. We performed our experiments in months of January, February and March between 8 AM to 6 PM. We repeated the experiments to provide reliability in results and to reduce the chance errors.

Figure 3.2: Experimental Set up for Video Streaming

3.3.2 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files over public and private networks. It can stream Live video streaming, Video on Demand and Video recording over Adobe Flash player, Microsoft Sil-verlight player, Apple iPhone, iPad, iPod touch, Quick Time player, Smart phones devices, tablets and IPTV set-top boxes. The supporting protocols of Wowza Media Server are Real Time Messaging Protocol (RTMP), Microsoft Smooth Streaming, Apple HTTP Live Streaming (HLS), Real Time Stream-ing Protocol (RTSP), Adobe HTTP Dynamic StreamStream-ing (HDS), Real-time Transport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS) [57]. It is an alternative to Adobe Media Server, Darwin Streaming Server, Microsoft IIS Media Services, and other media servers. In our thesis we used Wowza media server and installed on server system, the encoded videos are saved in the server machine with a specific name. These videos are accessed from client system using VLC player which is installed on it.

3.3.3 Test Video Parameters

(40)

Buck Bunny animated video, which comes under slow moving video. We covered the two types of videos suggested by ITU-T [58].

With the availability of high speed data networks, users are more at-tracted to view high definition videos. There is a scope in this direction, to test the high definition videos over high speed data networks. So, we used 720p High Definition videos with resolution 1280 × 720 pixels and is coded with a Main Profile encoder at Level 3.1 [28]. These two raw videos used for video quality testing are taken from test media website [59]. These two videos are encoded to the required format using FFmpeg encoder [60].

The duration of selected video sequence is about 4 minutes. In [24], performance of 3G data services was observed by taking packet data streams for 5 minutes. To observe the user QoE for video streaming over radio network, a minimum 4 minutes length of video is considered as appropriate. We chose the .mp4 as a container, since it is supported by most of the latest smart phones based on Android and iPhone [61] [62].

We selected H.264/AVC as video codec, because it has more advantages than MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28]. H.264 offers greater flexibility in terms of transmission support and compression options.

Video sequences Rugby , Big Bucks Bunny

Codec Perceptible, H.264 Main Profile, Level 3.1 Resolution 720p, (1280 × 720)

Frame Rate 25fps

Container MP4

Table 3.1: Test Video Parameters

3.3.4 NetEm

(41)

3.3.5 Delay

To choose the packet delay variation values range, some set of laboratory tests were performed for final assessment. Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 ms of constant delay. In case of delay variations, we introduced ∆D from 0 to 50 ms with an increment of 5 ms and we could not find significant changes. We repeated the test with values from 0, 10, 25, 50 and 100. We observed significant changes in the range of constant delay 150 ± {0, 10, 25, 50, 100} and similar settings are used in [65].

3.3.6 Packet Loss

One or more packets that originate from the source being transmitted across the network, fail to reach the destination. This shows major impact on the performance of the network and causes significant problems in applications like Video conference, VoIP and Video streaming. The packet loss is mea-sured in percentage of packets lost from the overall transmitted packets. To choose the packet loss variation values range, some set of laboratory tests were performed similarly for delay. Significant variations in video were observed for values 2%, 4%, 6% and 8% and similar settings are used in [65].

3.3.7 Assessment of Videos

For the purpose of subjective video quality analysis of the users, we devel-oped an assessment tool. The tool is designed to have a graphical interface to the users to give a rating for each video based on its video quality. The tool contains front end developed in PHP and the back end in MySQL, which used to store the video ratings of the users. A screen-shot of the tool is given below.

Figure 3.3: Screen Shot of QoE Evaluation Tool

(42)

(43)

Results and Analysis

(44)

Chapter 4

Results and Analysis

4.1 Analysis of LTE Network Performance with

Generated Traffic

We analyzed the traces got from the experimental procedure. Firstly we calculated the OWD, IPD and Packet loss using the Perl code.

4.1.1 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found the minimum OWD, maximum OWD and mean OWD.

4.1.1.1 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator.

Figure 4.1: Minimum One way Delay for Generated UDP Packets for Uplink

(45)

CHAPTER 4. RESULTS AND ANALYSIS 31

The Figure 4.1 shows the minimum One-way Delay for the UDP packets for Uplink transmitted over LTE network. This minimum One-way Delay shows the best performance that can be expected for a particular size of packets at different payloads. The minimum OWD delay range for LTE network is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and data rates 64 kbps to 8 MB. While in 3G network the OWD in uplink varies from 80 ms to 250 ms by neglecting the rapid increase at the starting stage from 64 to 252 bytes, which occurred due to the technology shift in the network operator of WCDMA to HSDPA [11].

4.1.1.2 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant. There may be different reasons for this network behaviour, to explain this we need further investigation.

Figure 4.2: Maximum One way Delay for Generated UDP Packets for Uplink

Most of the maximum OWD does not exceed 200 ms, the remaining values that occurred are due to the disturbances in the network at that instant. The worst OWD is 1060 ms. These maximum values effect the mean of OWD calculations.

4.1.1.3 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD for different payloads at different data rates.

(46)

Figure 4.3: Mean One way Delay for Generated UDP Packets for Uplink

4.1.2 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at sender or receiver side. In our experiment we calculated it at the receiver side. We calculated the IPD from the collected traces and found the minimum, maximum and mean IPD from it. The variation of Inter Packet Delay is known as Jitter. There is always a little amount of jitter present in the network. The jitter preferences are varied with type of traffic class and type of applications.

The variation of Inter Packet Delay is known as Jitter. There is always a little amount of jitter present in the network. The jitter preferences are varied with type of traffic class and type of applications.

4.1.2.1 Minimum Inter Packet Delay

(47)

Figure 4.4: Minimum Inter Packet Delay for Generated UDP Packets for Uplink

4.1.2.2 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application to application, so the variation in IPD ie., jitter preferences are also varies with different applications. The jitter has almost no impact on performance in case of file downloading, but it has significant impact on applications like streaming and video sharing.

Figure 4.5: Maximum Inter Packet Delay for Generated UDP Packets for Uplink

(48)

4.1.2.3 Mean Inter Packet Delay

Figure 4.6: Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD values. In Figure 4.6 the mean IPD for UDP packets showed linear increase in IPD values with increase in packet size at lower data rates.

4.1.3 Packet Loss for UDP in Uplink

Figure 4.7: Packet Loss for Generated UDP Packets for Uplink

(49)

showing packet loss in the Figure 4.7 are not considerable loss compared to total number of packet sent and those are not even up to 1% of packet loss.

4.1.4 One Way Delay for TCP packets in Uplink

We calculated the OWD, IPD and Packet loss for generated TCP packets from the collected traces. Then we found the minimum, maximum and mean for the One-way delay, Inter-packet delay and Packet loss percentage. We drew graphs regarding each case for clear understanding.

4.1.4.1 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users.

Figure 4.8: Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 4.8. Its OWD values vary from 20 ms to below 30 ms. For the UDP packets also it showed a similar behaviour. We did not find any significant difference in its behaviour for minimum OWD.

4.1.4.2 Maximum One Way Delay

(50)

Figure 4.9: Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing. The maximum OWD value for TCP packets over LTE network is having a range of 50 ms to 500 ms for most of the values. It also have few extreme values like 1260 ms, 850 ms, 890 ms.

4.1.4.3 Mean One Way Delay

From the calculated OWD’s we found mean OWD, which gave the overall behaviour of network by considering the best and worst OWD values.

Figure 4.10: Mean One way Delay for Generated TCP Packets for Uplink

(51)

1.5 mbps data rate. It may be caused due to the worst network service at that particular time.

4.1.5 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDP packets. IPD is defined as the time interval between the two successive packets at sender or receiver. In our experiment we calculated it at the receiver side. We calculated the IPD from the collected traces and then we found the minimum, maximum and mean IPD from it.

4.1.5.1 Minimum Inter Packet Delay

The network having low and constant IPD is the best network, it is also known as the Delay-Jitter. In several multimedia applications this jitter performance is very crucial.

Figure 4.11: Minimum Inter Packet Delay for Generated TCP Packets for Uplink

(52)

4.1.5.2 Maximum Inter Packet Delay

Figure 4.12: Maximum Inter Packet Delay for Generated TCP Packets for Uplink

Similar to the UDP, the maximum IPD values for TCP packets in Figure 4.12 also shows uneven increase and drops, the reason for these behaviour need further investigation.

4.1.5.3 Mean Inter Packet Delay

Figure 4.13: Mean Inter Packet Delay for Generated TCP Packets for Uplink

(53)

in IPD values with increase in packet size at lower data rates. Along with that it also showed abnormal values at higher data rate 4 mb.

4.1.6 Packet Loss for TCP in Uplink

Figure 4.14: Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure 4.14. Similar to UDP, the packet loss effect for TCP packets at different data rates with varying payloads is much significant. The few instances showing packet loss in the Figure 4.14 are not considerable loss compared to total number of packet sent and those are not even up to 1% of packet loss.

4.2 Gateway Evaluation

(54)

4.3 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTE network, we calculate OWD from the collected traces for TCP and UDP packets. Taking the data rate as constant, which is 4 mbps and the size of packet is varied.

Figure 4.15: Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 4.15 the UDP packets experiencing the minimum delay vary from 22 ms to 25 ms, and graph showing a behaviour of sawtooth wave form. The TCP packets experience more variations in delay pattern compared to UDP packets. The minimum OWD vary from 23 ms to 28 ms.

4.4 One Way Delay for Video Streaming Over LTE

The two phases of our work, firstly the measuring OWD of generated packets over LTE and secondly with streaming video over LTE network. By analyz-ing the traces of video packets, we observed that the data payloads MTU (Maximum Transmission Unit) are 1500 bytes. We analyzed the OWD of 1500 data size packets at different data rates, we presented the OWD for multimedia application packets.

We plotted the graphs for the OWD for the packets of 1500 bytes at different data rates. We generated UDP packets using the traffic generator programs written C++. We also plotted the graphs for TCP and UDP protocols. Thus we can observe the behaviour of multimedia packets over LTE network.

The following graph shows the OWD of packets having unit size of 1500 bytes, which resembles the multimedia packets.

(55)

Figure 4.16: Minimum One Way Delay for Generated TCP and UDP packet size of 1500 bytes for Uplink

a standard delay up to 1.5 mb and after it was subjected to some linear decrease in delay for higher data rates. This behaviour is followed for all sets of payloads. We predict that this behaviour is due to Scalable Bandwidth, which is one of the significant feature of LTE technology.

Scalable Bandwidth is one of the excellent advantage of LTE network over 3G systems. LTE system provides the scalable bandwidth up to 20 MHz, covering 1.4, 3, 5, 10, 15 and 20 MHz.

4.5 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks, the facts provoked us to find the QoE of video streaming over mobile network like LTE [1]. The main idea behind this part of the thesis is, to present QoE of video streaming over the LTE network.

In this chapter the analysis of the results obtained from the subjective video quality assessment are presented. The mean, standard deviation and confidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted. In detail, this experiment has 30 subjects who were asked to give MOS for a sequence of 40 videos. These videos include the videos captured at different delay variations and different packet losses for both TCP and UDP. In turn for each protocol we chose two different videos, one was fast moving video and the other slow moving video was used. For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tool and the collected MOS are stored in database. From these collected MOS, the mean, standard deviation and confidence interval were calculated and presented in the graphs.

(56)

The Mean is defined as:

¯ Ujk = 1 N N X i=1 ¯ Uijk (4.1)

The Standard Deviation (SD) is defined as:

§_jk = v u u t N X i=1 ( ¯Ujk− ¯Uijk)2 N − 1 (4.2)

Where: N is the number of observers, Uijk is the score of ith observer for

test condition j, video sequence k.

Confidence Interval (CI):

As recommended by the International Telecommunications Union- Ra-diocommunications Sector (ITU-R) in the recommendation BT-500 we used 95% confidence interval. This means “With a probability of 95%, the ab-solute value of the difference between the experimental mean score and the ‘true’ mean score (for a very high number of observers) is smaller than the 95% confidence interval” [66].

The confidence interval consists of an upper and a lower limit for the mean ¯U . ( ¯Ujk− δjk, ¯Ujk+ δjk) (4.3) where δjk = 1.96 Sjk √ N

Sjk is Standard Deviation calculated from the equation 4.2

4.5.1 Packet Delay Variation

4.5.1.1 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delay variations are plotted for fast and slow videos of TCP and UDP protocol. The graph is plotted by considering the MOS on Y- axis from one to five. The delay variations are considered on the X- axis, the constant delay (D) is maintained at 150 ms and the ∆D varying delay is taken as mentioned in the Table 4.1. The units for both D and ∆D are in milliseconds.

(57)

Figure 4.17: MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150±0 4.15 4.62 4 4.04

150±10 3.41 4.19 4.04 3.92

150±25 3.31 4.19 3.85 3.84

150±75 2.92 4.08 3.65 3.31

150±100 1.96 2.5 3.31 3.08

Table 4.1: MOS for Delay Variation

not have any problem with the videos. For the next increased level at 10 ms the slow moving video is maintained with a value slightly below the previous MOS rating than the ∆D 0 ms and still maintains a user perception level more than good. But for the fast moving video the MOS drops below the perceptible level. For the ∆D at 25 ms the MOS obtained for both the videos show that they are nearly the same when compared to the previous values, this specifies that, in spite of the increased delay by 15 ms of the previous value does not affect the quality of the video. By further increase of ∆D to 75 ms shows little drop in MOS for slow video, still maintains a MOS of perceptible level. In case of fast video the MOS Drops little more amount when compared to the previous level and maintains below the slightly annoying level. By further increase of ∆D to 100 ms, at this value the delay shows significant impact on the video quality of both the fast and slow moving videos. From the graph, it shows a sudden drop in the MOS readings and the slow moving video maintains a little bit above the annoying level. But for the fast moving video the MOS drops to an annoying level.

(58)

TCP protocol resends the data, if ever the data is lost. So this in turn, introduces delays and decreases the video quality. Hence the MOS of a slow moving video performed well than fast moving video.

4.5.1.2 Packet Delay Variation for UDP

Figure 4.18: MOS for UDP videos subjected to Delay Variation

The Figure 4.18 shows the delay variations of fast and slow moving videos of UDP protocol. Similar to the previous graph same five values are maintained in this case also. For the constant delay 150 ms and ∆D at 0 ms both the fast and slow moving videos got same MOS of perceptible range and have good video quality. The ∆D at 10 ms both got almost equal MOS and are still maintain perceptible video quality and are good. By further increasing the ∆D to 25 ms the MOS still got nearly same for both fast and slow moving videos. MOS of those videos drops little less than the previous value and maintains slightly below the perceptible and are slightly annoying. For the ∆D value at 75 ms the MOS for both the videos falls to the region slightly above annoying. From this point the MOS of slow moving video starts degrading more when compared to the fast moving video. At ∆D is 100 ms the MOS for both the videos degrades further and the MOS for slow moving video degrades more than the fast moving video and tends to catch the slightly annoying level of video quality.

Master Thesis Electrical Engineering October 2013 QoE of Video Streaming over LTE Network

Master Thesis

Electrical Engineering

October 2013

QoE of Video Streaming over LTE Network

Abstract

Acknowledgements

Contents

List of Figures

List of Tables

Acronyms

Introduction

Chapter 1

Introduction

1.1

Motivation

1.2

Related Works

1.3

Contribution

1.4

Aims and Objectives

1.5

Research Questions

1.6

Thesis Outline

Technical Background

Chapter 2

Technical Background

2.1

Quality of Experience

2.2

Video Streaming

2.3

Video Compression

2.4

Video Format

2.5

Types of Video Streaming

2.6

Supported Protocols for Video Streaming

2.7

Assessment of Videos

2.8

Overview of 3GPP Releases

2.9

Technical Overview of LTE

2.10

Architecture of LTE

Design and Implementation

Chapter 3

Design and Implementation

3.1

LTE Network Performance Evaluation with

Gen-erated Traffic

3.2

Measurements

3.3

Video Quality Assessment using Subjective

Analysis

Results and Analysis

Chapter 4

Results and Analysis

4.1

Analysis of LTE Network Performance with

Generated Traffic

4.2

Gateway Evaluation

4.3

One Way Delay Comparison in TCP and UDP

4.4

One Way Delay for Video Streaming Over LTE

4.5

QoE Analysis of Video Streaming