Master Thesis
Electrical Engineering Thesis no:
09 2015
DESIGN AND IMPLEMENTATION
OF RICH COMMUNICATION
SERVICE SCENARIO REPLAYER
AND PERFORMANCE
EVALUATION OF APPLICATION
SERVICE
Amulya Yellakonda
Department of Communication Systems Blekinge Institute of Technology
This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Electrical Engineering. The thesis is equivalent to 40 weeks of full time studies.
Contact Information: Author(s): Amulya Yellakonda. E-mail: amye14@student.bth.se University advisor(s): Dr. Patrik Arlos
Department of Communication Systems Industrial advisor(s):
Katja Sjogren Marcus Bauer
Sony Mobile Communications ab University examiner(s):
Professor Kurt Tutschku
Department of Communication Systems School of Computing
Abstract
Rich Communication Services(RCS) program is a GSM Association (GSMA) initiative to create inter-operator communication services based on IP Multimedia Subsystem (IMS) . This initiative came up as the Global Telecom Operators´response to the decline in their revenues and to help compete ’Over The Top’(OTT) service providers such as Viber, whatsapp, etc. RCS is an universal standard, making it inter-operable between mul-tiple service providers unlike OTT services with closed communities. RCS services use IMS as the underlying architecture with a RCS stack imple-mented into Android background service which offers high level API. For the purpose of testing RCS stack functionality which is usually affected by external dependencies like third party vendors, or ISP customizations in real telecommunication scenario, there is a persistent demand for scenario replay tools that can recreate the range of test conditions similar to those experienced in live deployments. There is also a need to evaluate the per-formance of service provided by application servers in the network in-order to predict the factors affecting the RCS service in general.
In this work, we propose a tool to address the RCS scenario repro-duction in a test environment. The tool is implemented within an automated test environment with full control on interaction with the RCS stack, hence the ability to replay the scenario in a controlled fashion. To achieve the goal, the tool replays trace interactively with the RCS stack in a stateful manner , it ensures no improper packet generation which is critical feature for test environments where protocol semantics accuracy is fundamental. A detailed demonstration of how the tool can be deployed in test environ-ments is stated. A novel approach is used to validate the effectiveness of the replayed scenario, the sequence of events and states are compared to those from the recorded scenario using a call-back service to indicate the state. The replayed scenario showed strong relationship with the recorded RCS scenario. The paper also presents a performance evaluation of Application service by considering the request-reponse times of Network Registration procedure. The obtained results show that the average time taken for the Registration process is 555 milliseconds and in few instances there exists larger deviations from this average value showing the faulty behavior of the Server which is most crucial during the debugging process for the developers. Keywords: RCS,IMS,Replay, Performance
Acknowledgments
This thesis would not have been possible without the support of a number of wonderful individuals, my thanks and appreciation to all of them for being part of this thesis and making it possible. I owe my deepest gratitude to my supervisor Patrik Arlos for his support and valuable inputs. I express my warmest gratitude to my associate supervisor Katja Sjogren of Sony Mobile Communications where the research was performed.
Finally, my deep and sincere gratitude to my family and friends for their continuous love and support. They have always encouraged me to explore new directions in life and seek my own destiny.
–Amulya Yellakonda
Contents
Abstract i
Acknowledgments iii
List of Contents v
List of Figures viii
List of Tables x
1 Introduction 1
1.1 Motivation . . . 2
1.2 Scope of the thesis . . . 3
1.3 Research Questions . . . 3
1.4 Research Methodology . . . 4
1.5 Thesis Layout . . . 6
2 Background 7 2.1 IP Interconnection . . . 7
2.2 IMS Architectural Framework . . . 8
2.2.1 IMS Functional Architecture . . . 8
2.3 RCS protocols . . . 10
2.4 Network flows in RCS based scenarios . . . 13
2.5 Software tools used in the research process . . . 13
2.6 Related Works . . . 14
3 Purpose of the thesis 16 3.1 Problem Statement . . . 16
4 Design and Implementation of RCS scenario replayer and Per-formance Evaluation of RCS Application Service 18 4.1 Design and Implementation of Rich Communication Service sce-nario replayer . . . 18
4.1.1 RCS scenario replayer goals . . . 18
4.1.3 Test Bed . . . 20
4.1.4 Replay Validation . . . 22
4.1.5 Scenarios for testing . . . 23
4.2 Performance Evaluation of RCS Application service . . . 27
4.2.1 Performance Evaluation procedure . . . 27
4.2.2 Performance data collection . . . 29
5 Results 30 5.1 Design and Implementation of RCS based scenario record and re-play tool . . . 30
5.2 Validation of Replayed Scenario . . . 31
5.3 Performance Evaluation of Application Service . . . 32
6 Conclusion and Future work 35 6.1 Conclusion . . . 35
6.2 Future Work . . . 36
References 37
Apendix 40
List of Figures
1.1 Stack concept [17] . . . 2
2.1 IP Interconnection [19] . . . 8
2.2 IMS functional architecture [19] . . . 9
2.3 RCS Deployment in IMS . . . 10
2.4 SIP Request Message format [24] . . . 11
2.5 SIP Response Message format [24] . . . 12
4.1 Record and Replay tool Algorithm . . . 19
4.2 RCS stack as an interpreter . . . 20
4.3 Test Bed Set-up . . . 22
4.4 Replay tool functionality . . . 23
4.5 Trigger of one-to-one chat message intent . . . 24
4.6 Resend of failed messages . . . 26
4.7 Display Report sent despite of disabling display setting . . . 27
4.8 RCS registration scenario . . . 29
5.1 Confidence interval . . . 32
5.2 Request-Response times on hourly basis . . . 34
List of Tables
5.1 Request Response times statistics. . . 32
Acronyms
AS Application Server DNS Domain Name Space GPRS General packet radio service HSPA High Speed Packet Access HTTP Hypertext Transfer Protocol IETF Internet Engineering Task Force
IM Instant Messaging IMDN Instant Message Disposition Notification IMS IP Multimedia Subsystem IP Internet Protocol
LTE Long Term Evolution MNO Mobile Network Operator MSRP Message Session Relay Protocol NAT Network Address Translation NNI Network-to-Network Interface OS Operating System
P-CSCF Proxy Call Session Control Function PS Packet Switched
RCS Rich Communications Services RTCP Real-Time Transport Control Protocol RTP Real-Time Transport Protocol SDP Session Description Protocol
SIP Session Initiation Protocol TC Test Case
TCP Transmission Control Protocol UDP User Datagram Protocol
Chapter 1
Introduction
The global telecom industry, has always been witnessing continuously chang-ing technology and business environments since its initiation with early voice tele-phony. Traditionally, the main revenue returns for telecom operators have been through legacy voice and SMS with data services till recently. Telecom operators are facing new challenges to their revenues due to Over The Top (OTT) service providers which deliver media over Internet bypassing the underlying operators network. To be able to compete with these OTT services, the Rich Communi-cation Services initiative has come up. Albeit OTT services are cheaper means for user communication posing a serious threat to RCS initiative, the features of RCS are quite promising and might become the future core service for mobile. RCS also ensures unifying client identities in comparison to multiple server/client scenarios due to increasing number of OTT services.
The main goal of RCS is to provide inter-operable multimedia services to the customers that is regardless of the underlying Service provider’s network which are fully integrated into the RCS-enabled devices. The RCS architecture is de-fined by GSMA RCS collaborative project [18] with more than 100 operators, enterprises and telecommunication companies into it. RCS also facilitates multi-device handling [19] with a centralized data management system. Several services that fit into the RCS framework include enhancement of user’s messaging expe-rience, namely 1-to-1 chat, group chat, content sharing services allowing the user to share video or image file while on call, social presence, Geolocation service and enriched phonebook to give a rich communication experience to the user. RCS is an IMS (IP Multimedia Subsystem) based stack which offers services to the registered subscribers and uses SIP (Session Initiation Protocol) to initiate, cut off and control the multimedia sessions. RCS clients are usually configured to use SIP instances, with RCS client sending SIP requests to the server during the registration procedure.
RCS developers are on their way to develop working prototypes with a complete RCS enabled Android client. The RCS stack is an open source imple-mentation of the Rich Communication Suite standards for Google Android 2.x platform [17]. This implementation is compliant to GSMA RCS 1.1 standards. RCS stack offers the background RCS services on an android client. A RCS API
Chapter 1. Introduction 2
Figure 1.1: Stack concept [17]
on top of the stack is used by applications (native Address Book, Messaging, Dialer apps or any other third party apps) as shown in Figure 1.1. The RCS API permits to implement RCS application by hiding underlying RCS protocols complexity.
1.1
Motivation
Chapter 1. Introduction 3 The main interest of this research was to implement a tool able to replay the recorded scenario in a controlled test environment. It was interesting to analyze the RCS based scenarios at protocol level and to replay the recorded scenarios. One of the key challenges in the process of replaying the scenario was to handle the IP fragmentation. It is also very important to capture the network traces of complete scenario.
1.2
Scope of the thesis
This Thesis work involves recording of network traffic on android client and replaying the recorded scenario in a controlled test environment. The RCS sce-nario replay is a challenging task. Conceptually, the process of interactive scesce-nario replay in usual has one captured trace file of real RCS scenario as input. The captured packets are organized and replayed in terms of multiple flows, this in-teractive replay tool can be roughly modeled as a single-input/multiple-queue queuing system. But practically, the input trace, the network conditions (timing, delay and various third party service provider customizations in the network) are all varying and unpredictable in the replaying test environment. Hence the replaying tool is implemented in a very controlled test environment in order to simplify the complications which will be discussed in the implementation section of this thesis.Compared to the network traces recorded in a real RCS communica-tion, the reproduced traffic might have slight different timings due to background processes in the test environment. The lack of packet-replay control, real environ-ment condition emulation, or coordination between them could result in varied timing cases. However the main goal of replaying the scenario is maintained up to a larger extent by maintaining control over the sequence of events/states. The re-produced/ reconstructed traffic definitely do not match with the ones in recorded network trace because the objective is to replay the scenario and can only be achieved by reconstructing the payloads compatible with the test environment network configurations.
1.3
Research Questions
1. What are the possible approaches to record and replay the recorded RCS scenario in a controlled test environment?
2. What metric can be used for validation of the replayed scenario?
Chapter 1. Introduction 4
1.4
Research Methodology
In this Chapter, the methodology used for the research has been described. The reasons for choosing a specific method of implementation along with the logical method used to validate the design and implementation is explained in detail. It requires that the solution be implemented technically correct with assurance on maintaining protocol semantics in the replayed traffic. The research is carried out in an automated test framework namely aRCSine specific to Sony Mobile Communications. There are many phases involved in this work of research as listed below.
• Phase-I– In the first phase, a thorough literature review was performed on the RCS stack architecture, protocols involved and the enriched ser-vices provided by RCS to the mobile customers. The literature review is performed by referring to specification documents of RCS accredited by GSMA [18] to accumulate valid and quality information relevant to meet the objectives. The choice of specification document versions is specific to the RCS stack version used in the research.A thorough review of RCS spec-ification documents is necessary as the document includes complete details on the RCS related protocols, scenarios and technical details about RCS implementation in real networks.
Chapter 1. Introduction 5 the sequence of states considering the maintenance of environment settings as in recorded scenario such as RCS settings, features enabled, etc. This research is involved with recording and replaying chat scenarios in specific among all of the RCS services and can be extended for other services which is however out of the scope of this research.
• Phase-III– The third phase involves recording the required scenario on the android client and the extraction of the quality data from the recorded traffic/network traces specific to the involved parties. The trace files are captured on android client using Android Debug Bridge (ADB) which is a client-server program, where the client part runs in the PC and the android device can be invoked using the ADB commands. Tshark is used to record the traffic over the android client and captured trace is pulled on to the PC for further processing .Tshark is a powerful command line sniffer tool which has many advantages over other traffic capturing with capability to decode all the protocols. The extracted data undergoes the process of categorization of traffic related to Server and Mobile / inward and outward since the traffic is captured on the mobile device. The inward traffic is the Server traffic which is to be processed for the reconstruction of payloads while the mobile traffic is mapped to the triggers/actions to be performed on the mobile end for the specific traffic generation on the protocol level. • Phase-IV– The fourth phase involves the generation of test scenarios from
the extracted data categorized according to the actions performed in recorded scenarios. This phase involves figuring out possible ways to trigger start of activities on the mobile device (prototype) to repeat the recorded behavior and to generate actual payloads on the server end to repeat recorded behav-ior. The record and replay tool is implemented as a part of test suite. The interactive replay of the scenario involves synchronization of the activities on phone and Server according to the recorded scenario.
Chapter 1. Introduction 6
1.5
Thesis Layout
Chapter 2
Background
The Rich Communication Service (RCS) Initiative is a combined effort within GSM Association (GSMA) [18], to accelerate deployment of IMS based services into markets, with commitment from operators to deploy IMS. The IMS Network-to-Network Interface, NNI architecture forms an important part of RCS NNI since RCS heavily utilizes IMS core system as specified by 3rd Generation Partnership Project (3GPP) to perform various key functions such as handling of SIP signal-ing, authentication, authorization, etc. This initiative brings together operators, handset and network vendors, specifying the set of services made available in an RCS terminal. The most valuable feature of the IMS architecture is interoperabil-ity. Contrary to OTT players, RCS guarantees communication interoperability between device users whatever network subscription they have made or the ter-minal in use.
RCS relies on IMS core network for establishing and controlling sessions which rely on SIP signaling features. The key of RCS is the capability discovery framework which allows identifying the RCS services of any users even in case of multi-device usage. The RCS service is totally integrated into the already existing services of the phone. There are quite many RCS features and is totally up to the operators to decide which to deploy. RCS services are built using a set of stan-dardized features that use the IMS architecture to enable service integration to provide smoother user experience and interoperability between service providers.
2.1
IP Interconnection
RCS is an IP based service Figure 2.1 because the existing networks which are based on CS/TDM to transport voice interoperable [19] between Service providers cannot use SIP signaling or MSRP media functionalities. RCS marks the transi-tion of messaging and voice capabilities from Circuit Switched technology to an all-IP world and it shares the same IMS investment and leverages the same IMS capabilities as VoLTE and Video calls over LTE.
Chapter 2. Background 8
Figure 2.1: IP Interconnection [19]
2.2
IMS Architectural Framework
IP Multimedia Subsystem is a network framework to deliver multimedia traf-fic primarily designed by 3rd generation Partnership Project - 3GPP [19]. IP Multimedia Subsystem is based on SIP by 3GPP and IP protocols by IETF. IMS provides control and charge for services being delivered by network operator/ser-vice provider. IMS framework is a migration of CS, circuit switched domain network to IP Packet switched domain network and hence considered as all-IP network.
2.2.1
IMS Functional Architecture
IMS Functional Architecture: IP Multimedia core network subsystem is a col-lection of different functions, linked by standardized interfaces which are grouped to form one IMS administrative network as in Figure 3.
• HSS (Home Subscription Server) HSS is a database to store customer sub-scription profiles which are later downloaded to S-CSCF after authentica-tion procedure Applicaauthentica-tion Servers (AS) - Applicaauthentica-tion funcauthentica-tionality associ-ated with RCS specific applications
Chapter 2. Background 9
Figure 2.2: IMS functional architecture [19]
• Options-AS: Enables multi-device
• Handset notifications AS: Enables the handset asynchronous notification Other AS may also be included in the IMS domain e.g. VoLTE, Video Tele-phony. Access Session Border Controller (ASBC) ASBC controls the edge of the IMS network. Call Session Control Functions (CSCF) - SIP servers responsible for the enforcement of subscription profiles and authentication of customers.
• Proxy CSCF– performs access control
• Interrogating CSCF - top level authentication of the customer
• Serving CSCF - service control and integration Interconnect Session Border Controller (ISBC) Manages in-coming and out-going traffic from and to the IMS domain, and protects IMS from external attacks.
Chapter 2. Background 10
Figure 2.3: RCS Deployment in IMS
2.3
RCS protocols
RCS clients employ various protocols to support the above mentioned services and the choice among the protocols wouldn´t affect the interoperability of Service provider.
• SIP Protocol – "SIP is an application-layer control protocol that can es-tablish, modify, and terminate multimedia sessions (conferences) such as Internet telephony calls. SIP can also invite participants to already exist-ing sessions, such as multicast conferences. Media can be added to (and removed from) an existing session. SIP is constrained to only setup, modify and terminate sessions. All of the other key functions during the com-munication in session are done with other protocols. SIP is not a session description protocol, and does not do conference control. SIP works in a framework with other protocols such as SDP, HTTP, MSRP, etc to make sure the other roles are played out. Some of the protocols involved in the replay tool implemented are briefly stated. SIP is a text-based protocol de-veloped based on the HTTP protocol and functions like a request-response protocol, independent of the transport protocol. There are several concepts like transactions and dialogs on SIP session, which are initiated with cer-tain type of requests and terminated with other specific requests/responses, until its termination (dialog)". For further information, refer to [1].
Chapter 2. Background 11 description of the session. When the other client accepts call, a response is sent back.
Figure 2.4: SIP Request Message format [24]
The figure 2.4 is an example of an invite message and each of the fields are explained below:
1 = SIP request header gives information on the kind of SIP message. 2 = This header holds information of list of proxy servers involved. 3 = This header gives information on the SIP packetâĂŹs destination. 4 = This header specifies about the sender of packet.
5 = This is a SDP block containing description about the media involved in the session.
Chapter 2. Background 12 request-response messages are connected and identifiable to one-another using various fields in the messages.
Figure 2.5: SIP Response Message format [24]
• SDP Protocol [2] – This protocol can be used to describe media/data streams in the established SIP session and is a text-based protocol itself. The media type can be audio, video or messaging. It can be used to initiate the streams, modify the stream parameters (port, IP, codecs, etc.) or stop-ping one of them. SDP being a protocol is used to negotiate the mentioned properties between two endpoints. For further information, refer to [2]. • MSRP Protocol [3] – Message Session Relay Protocol (MSRP) defines a
Chapter 2. Background 13 • RTP protocol [4]– The real-time transport protocol, RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. For further information, refer to [4].
• HTTP Protocol [6]– The Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia infor-mation systems. It is a generic, stateless, protocol which can be used for many tasks beyond its use for hypertext, such as name servers and distributed object management systems, through extension of its request methods, error codes and headers. A feature of HTTP is the typing and ne-gotiation of data representation, allowing systems to be built independently of the data being transferred. For further information, refer to [6].
2.4
Network flows in RCS based scenarios
The term flow is used to depict the nature of network traffic. Flow is the se-quence of packets or a packet that belonged to certain network session(conversation) between two hosts. Network flow data is a summary of conversation between two entities involved in the conversation. The fields used to aggregate traffic typi-cally specify addresses at various levels of the protocol stack (e.g. IP addresses, IP protocol, TCP/UDP port numbers). Herein, it is used the term key to refer to the set of address attributes used to aggregate packets into a flow. The de-termination of flow in an RCS scenario over a state is used to identify the flow connections requiring a 5-tuple key representation. The tuple includes source IP address, source port number, destination IP address, destination port number and protocol information. Throughout the research, a bidirectional RCS flows involving various protocol traffic is considered.
2.5
Software tools used in the research process
• Tshark– TShark [15] is a network protocol analyzer. It lets you capture packet data from a live network, or read packets from a previously saved capture file, either printing a decoded form of those packets to the standard output or writing the packets to a file. TShark’s native capture file format is pcap format, which is also the format used by tcpdump and various other tools.
Chapter 2. Background 14 android client over the wlan interface in a pcap format for further investi-gation. The saved files are pulled on to the PC for further processing via the developed software tool.
• adb (Android Debug Bridge) Android Debug Bridge [22] (adb) is a versatile command line tool that helps to communicate with an android client as used in this research. The android client is connected to the PC using a USB interface and the client should be enabled to debugging mode. The ADB tool is of major importance in this research since it helps in provoking various actions on the android client. A connected android device is identified by the serial number assigned to the device connected over USB.
2.6
Related Works
There are various previous research studies performed on network traffic record and replay in order to evaluate the performance and working of systems in various domains like security systems, application testing, etc. They can be deployed to provide the background traffic to test the performance, but they are not capable of creating an interactive replay with proper payloads reconstruction to avoid ghost packets. For a proper RCS scenario replay it is important to have a technically proper reconstruction of protocol payloads and their synchroniza-tion with the acsynchroniza-tions from the phone which are involve the scenario replay.In this section we discuss the functionality of few open-source traffic replay tools and some related work in this field of research.
Chapter 2. Background 15 real traces and then produces traffic according to these payloads by establishing real connections.
Various other researches have been done on the upcoming RCS services, its implementation in the IMS core networks and protocols involved in the RCS re-lated network traffic, etc. The author in [11] has presented an analysis on the SIP signaling traffic generated by the upcoming RCS-e (Rich Communication Suite-enhanced) mobile service along with evaluation of the volume of the signal-ing and media traffic generated by the RCS-e services.Recently there have been many studies on the Performance evaluation of SIP based VoIP infrastructure and Proxy servers in various perspectives like load handling, security,etc. There are various open source tools in usage to benchmark the performance of SIP in-frastructures. For instance [23] is a free Open Source test tool / traffic generator for the SIP protocol.It features the dynamic display of statistics about running test scenarios.Though such tools help in producing performance statistics, do not suffice to assess the SIP server´s performance. To be able to determine the SIP server´s performance from the calculated statistics, there is a need to define the values of the limits for the chosen performance metrics.
Chapter 3
Purpose of the thesis
Generating workloads without investigation on real bugs or RCS stack crash scenarios would not enable developers to understand the full story. Therefore, it is crucial to implement replay techniques, which reproduce scenarios identified from real network traces. The main interest of this research was to implement a tool able to replay the recorded scenario in a controlled test environment. It was interesting to analyze the RCS based scenarios at protocol level and to replay the recorded scenarios. One of the key challenges in the process of replaying the scenario was to handle the IP fragmentation.
3.1
Problem Statement
This section explains the issues related to real RCS scenario replay and for-mulates the problem statement. RCS stack development faces a big challenge during the debugging stage to fix bugs and stack crash scenarios that are affected by external dependencies such as third party network implementation, specific timing or operator customizations. These bugs not being reproducible at the operator side turn out to be very costly for the developers to investigate and fix the bug/crash.Hence this paper presents the design and implementation of a record and replay tool to reproduce the crash scenarios in test environments. Some examples illustrating complex scenario replays with the real environment are discussed in the thesis in section 4.1.5.The validation of the replayed traffic in the illustrated complex scenarios is also provided. Though the main focus of the thesis is the design and implementation of the replayer, it would be worth evaluating the Performance of Application Service in order to find the possible location of the fault in the Service. Hence, the thesis presents a study of how the Request-Response times can be used to evaluate the Service performance. Hence the research aims to define normal request-response times in a ’IMS REGISTRA-TION’ scenario to base it as a metric for a desirable range of request-response times. An adaptive network flow knowledge base for the scenario is derived. By using the derived metric, it is observed that the performance of service can be analyzed and the possible flaw in the service can be detected.
Chapter 4
Design and Implementation
of RCS scenario replayer and Performance
Evaluation of RCS Application Service
This chapter explains the methodical work that is followed to replay the RCS scenario and provides information on the performance metric examined in a REGISTRATION scenario.
4.1
Design and Implementation of Rich
Commu-nication Service scenario replayer
4.1.1
RCS scenario replayer goals
This paper presents a scenario capture and replay tool that interactively replays the reconstructed traffic payloads in a live controlled test environment. The test environment is termed live because a real android client which is RCS enabled is being used during the replay with in-line communication happening between client and test server. With respect to this live interactive replay, there are several requirements the design must take into account for a successful record and replay of scenario.
• No improper packet generation– The reason why other traffic replay tools in usage might generate improper network packets is because of not being able to replay the TCP connections in a stateful manner. In these RCS based scenarios there are various protocols involved in the communication and a replay might be possible only with a stateful generation of protocol payloads by replacing specific fields during the run time. The specific fields to be replaced to reconstruct the payloads is done by checking the state of inward packets and extracting data from the inward packets to be reused in the outward reconstructed traffic. Hence, this is one of the fundamental requirements in replay tool design.
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service19 • Environmental transformations and configurations– One of the
chal-lenges in traffic replaying is to be able to maintain the test environment and network configurations and mapping of original network configurations like the IP addresses, port numbers, etc to those of test network since this is more than just address remapping. The replay tool must be able to route information between target device and the test Server on the PC.
• Extensibility– The replay tools put to real industrial use should be ex-tensible from the aspect of software engineering. The replay tool should be able to replay various scenarios without major changes in the design.
4.1.2
Replayer Architecture
The designed replay tool follows a trace-based replay approach. It extracts traffic properties like timing of packets from the IP packet streams and recreates new streams equivalent to captured packet streams. This helps in maintaining the timing of the replayed scenario as per the captured scenario.
Figure 4.1: Record and Replay tool Algorithm
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service20 RCS stack is a form of interpreter as shown in Figure 4.2. For a scenario to be
replayed, the RCS stack interpreter uses the RCS settings on the phone along with the inputs from the user and incoming traffic from the outside world. The traffic to be coming from the outside world is fed by the scripted servers in the test environment. The test servers are scripted so as to read the parsed sequence of packets and reconstruct the payloads as per the network configurations of test environment. The reconstruction of packet payloads which are to be injected to the device in test environment takes place following the semantics of each pro-tocol used. After the preprocessing stage is done, the replay tool sequences the events to happen by synchronizing both inward traffic injection from the Server and outward traffic by initiating triggers on the phone interactively.
Figure 4.2: RCS stack as an interpreter
4.1.3
Test Bed
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service21 controlled test framework. A Wi-Fi USB adapter is connected to the computer
to be able to connect to the WiFi network. A detailed description in support of the choice of the architecture of the test bed is provided in this section. The an-droid client used is a usual Sony vendor specific device running on Anan-droid 5.1.1 version with Qualcomm MSM8974PRO-AB processor. The additional hardware requirement includes USB cable connection from phone to PC as shown in Figure 4.2 Test Bed Set-up.
4.1.3.1 Android Client Set-up
The android phone used in this research consists of RCS-stack and Terminator application enabled on it. RCS stack is the main implementation into an android background service whose functionality is the main focus of our research. RCS stack offers a high level API which permits to implement RCS application (e.g. enhanced address book, content sharing, chat, widgets) by hiding RCS protocols complexity. Terminator application is a java based test application which mirrors Terminal API methods/callbacks for the Python script over an RPC interface i.e. request messages are sent for the execution of specific procedures with the given parameters. The communication through Terminator is a combination of HTTP and Android Intents. The Terminator application is used in our research to trigger the actions from the phone end in the interactive replay. The replay of scenario involves synchronizing the replay of reconstructed payloads from the server and trigger of actions from the client which actually created the traffic from the phone end.
4.1.3.2 PC Set-up
The setup of PC involves setting up aRCSine test framework which has com-plete control over the actions from the Server end. The replay tool is created as one of the test cases in the test framework.
4.1.3.3 aRCSine test framework
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service22
Figure 4.3: Test Bed Set-up
4.1.4
Replay Validation
The experiments are conducted based on the test-bed described in section 4.1.3. The major objective of the established test-bed is to record and replay the RCS based scenarios. The chosen test bed perfectly satisfies all the design goals mentioned in the section 4.1.1. The designed replayer replays the recorded scenario in an interactive fashion with proper packet generation and timely syn-chronization of events as explained in section 4.1.2. To validate the designed Record and Replay tool, three different scenarios are taken into consideration and are described as follows.
1. Basic receipt of one-to-one chat
2. Resend functionality of failed messages
3. Display Report sent despite of disabling display setting
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service23
Figure 4.4: Replay tool functionality
crashes or faulty behaviors is critical. This motivates to adopt the chosen scenar-ios since it considers complex cases of failed messages, enability and disability of display reports of sent messages, etc.
4.1.5
Scenarios for testing
In the following scenarios, a detailed description of the events on the protocol level and their reflection on the application level is provided. The occurrence of sequence of event intents and various other constraints are checked for effective-ness of the replayed scenario as follows:
4.1.5.1 Basic receipt of one-to-one chat
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service24 trigger of NewSingleChat() callback in the Terminal API of the RCS stack. A
’SIP ACK’ message is sent to acknowledge the received ’SIP INVITE’.
During the replay of the scenario,the events on the Terminator are checked for the correct sequence of happenings and the state wise validation is performed as shown in Figure 4-4. In the current scenario, the Terminator is checked for the callback of a specific intent
’Event::Intent::NewOneToOneChatMessage’ on receipt of ’SIP INVITE’ message. A successful receipt of callback means that the incoming ’SIP INVITE’ could trigger the initiation of new one-to-one chat session. Once the intent is received, the effectiveness of replay is measured by checking the direction of the intent, if it is incoming in this case and the mime type of the message, text/plain in this case.
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service25 4.1.5.2 Resend functionality of failed messages
Scenario:This is the scenario where an android client user performs one-to-one chat and tries to resend the messages which failed due to some network error as shown in the Figure 4-5. The scenario starts with initial registration to the network as explained in section 4.1.5.1. After a successful registration, a ’SIP INVITE’ is sent from the client to the Server to initiate a chat session. A ’SIP 408’ response is sent from the Server upon the invite request which indicates the failure of the sent message. A ’SIP INVITE’ message is sent which initiates the chat session. A ’SIP MESSAGE’ containing the delivery report is sent back to the client indicating the successful delivery of the sent message. A ’SIP MESSAGE’ containing a display report is also sent to the client to indicate the displayed status of the message.
During the replay of the scenario,the events on the Terminator are checked for the correct sequence of happenings and the state wise validation is performed as shown in Figure 4-5. In the current scenario, the RCS settings are set to replicate the settings as of the recorded scenario. The ’AutoAcceptChat’ setting is set to true which implies that the display report of the received message needs to be sent to the sender of the message. The Terminator is checked for the callback intent namely ’Event::ChatListener:OnMessageStatusChanged’ for a sequential change of the Message status on the callback intents.
4.1.5.3 Display Report sent despite of disabling display setting
Scenario: This is a scenario where an android client sends the display report of a message to the sender of the message despite of disabling the display setting in the RCS settings of the device.This is a complex scenario to describe a probable bug in the functionality of the RCS stack. The scenario starts with initial registration to the network as explained in section 4.1.5.1. After a successful registration procedure, a ’SIP INVITE’ message is sent to the android client indicating the invitation of a new chat message. A ’SIP ACK’ message is sent to the Server to acknowledge the receipt of the chat message. A ’SIP MESSAGE’ with a display report is sent from the client to the Server to indicate display of the received message.
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service26
Figure 4.6: Resend of failed messages
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service27
Figure 4.7: Display Report sent despite of disabling display setting
4.2
Performance Evaluation of RCS Application
service
Performance evaluation or Performance testing is a usual process in order to measure a system’s responsiveness, scalability and stability under various loads or environments. There are various metrics that can taken into consideration for such performance evaluations. In our research, the performance evaluation of the application service, specifically Registration service is of main concern.
4.2.1
Performance Evaluation procedure
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service28 communication capable and the Server is reachable. The registration
pro-cedure involves client sending a ’SIP REGISTER’ message to the Server in the network using configuration parameters. On receipt of valid ’SIP REGISTER’, the Server reponds by sending ’401 UNAUTHORIZED’ re-sponse with nonce for authentication and time validity. The device then goes through its own MD5 calculations to calculate random nonce and re-submits ’REGISTER’ request with authorization header. If the Server finds the response to be valid, it sends a ’200 OK’ response as an implication of successful authentication and registration. The time taken for the complete Registration transaction from first REGISTER request to final 200 OK re-sponse is always deterministic of the proper working condition of mobile and the Server in IMS core. Hence measurement of the delay in responding to a device’s ’REGISTER’ request i.e. request-response times is used for performance evaluation of the Application service.
• Identification of Performance Acceptance Criteria The obtained request response times is performed during several times of the by looping the REGISTRATION procedure for multiple iterations helps in determining the approximate range of acceptable request-response time which can be used as the performance acceptance criteria. This chosen metric will provide a better indication of performance with larger sample sets.
• Test environment configuration The key scenario chosen is the IMS Network Registration and to simulate the test environment, an android RCS enabled client is chosen and the request-response times are collected over an hourly time-frame. Registration request-response times (RRT) can be measured and reported only for successful REGISTER requests, while Ineffective Registration attempts with other reason codes will be reported as failures. This metric is measured at the originating end, ie on the an-droid device.The calculated value of request-response times is numerical and stated in units of milliseconds. The RRT is calculated using the following: RRT = (Time of Final ’200 OK’ Response from Server) - (Time of first ’REGISTER’ Request from the device)
Chapter 4. Design and Implementation of RCS scenario replayer and Performance Evaluation of RCS Application Service29
Figure 4.8: RCS registration scenario
• Analysis of Results The calculated request-response times are to be consolidated to obtain the accepted limits in order to set the thresholds. The individual obtained values are checked against the set threshold values inorder to evaluate the performance which might help in determination possible failures in the REGISTRATION scenario.
4.2.2
Performance data collection
Chapter 5
Results
This chapter is divided into two sections of results. Firstly, the observations on the implementation and validation of the RCS scenario based Record and Replay tool is analyzed. Secondly, the metric used for performance evaluation of RCS based Application Service is discussed.The discussion is structured in questioning-answering manner according to the three research questions raised in the thesis.
5.1
Design and Implementation of RCS based
sce-nario record and replay tool
RQ1→What are the possible approaches to record and replay the recorded RCS scenario in a controlled test environment?
This paper proposed a trace based approach in order to replay the recorded RCS based scenarios in a controlled test environment.In order to record the re-quired scenario, a command line tool namely tshark is used to capture the packet data on the android client. The captured network traces (trace file format: pcap) are processed to gather all the information necessary to replay the scenario. Be-fore the actual replay of the scenario in the controlled test environment, the RCS settings on the android client (device under test) are replaced with the RCS set-tings of the android client in real captured scenario. The replayer satisfies all the design goals mentioned for a successful replay of the scenario. The reconstructed packet payloads follow the syntax and semantics of the protocols involved in the actual replay. The sequence of events are replayed by maintaining the request-response times of involved transactions. A java based application specific to the android Vendor (Sony Mobile Communications ab) is used in order to trigger the actions from the android client based on the events as explained in Chapter 4. Based on the proposed approach, it was seen that the recorded scenarios can be successfully replayed in the proposed test environment with complete control over the traffic flows from the client and server ends. However, the replayer couldnot replay the scenarios with continuously changing RCS settings. The replayer is designed to replay the RCS based chat scenarios and cannot be used to replay
Chapter 5. Results 31 any other RCS based scenarios such as file transfer, video calling, etc. For a complete description about the design and implementation of approach refer to the appendix of this paper.
5.2
Validation of Replayed Scenario
RQ2→What metric can be used for validation of the replayed scenario?
This section presents the metric used for the validation of replayed sce-narios. With a view to validate the replayer,the scenarios explained in section 4.1.5 are examined. Prior to replay of any RCS based scenarios, the RCS settings database is changed as to replicate with the RCS settings in the real scenario.The intents/callbacks passed to the Terminator as explained in section 4.1.5 are used as a validation metric for the replayed scenarios.
Scenario: Basic Receipt of one-to-one chat In the current scenario, it is observed that when a reconstructed ’SIP INVITE’ message is being replayed from the Server, the terminator received a callback that is:
’Event::Intent::NewOneToOneChatMessage’.
This proves the successful replay of the captured scenario in this case. It is also observed that the time taken for the complete scenario replay is very close to that of the captured scenario.
Scenario: Resend Functionality of failed messages: In the current sce-nario, the Terminator is checked for a specific callback that is ’Event:ChatListener: OnMessageStatusChanged’. It is observed that when a reconstructed ’SIP 408’ message is replayed from the Server end, the Message status argument on the received callback is changed to fail. When the Terminator is used to trigger the resending of the failed packet, a ’SIP INVITE’ message is sent from the client to the Messaging Server. Now, a ’SIP MESSAGE’ with delivery report is being replayed from the Server, and is observed that the Message status has changed from ’sent’ to ’delivered’. A similar ’SIP MESSAGE’ with display report is be-ing replayed from the Server. The packets replayed from the Server end are the packets that were captured in the traces and reconstructed by varying all the parameters as to fit the in-line replay of the scenario.
’ChatRespondToDis-Chapter 5. Results 32 playReport’ is set to ’false’. This proves the successful replay of the captured scenario displaying a bug in the functioning of RCS stack.
5.3
Performance Evaluation of Application
Ser-vice
RQ3 →What metrics can be used to evaluate the performance of Application service of application servers?
The metric used to evaluate the performance of RCS application service is Request-Response times of the Registration procedure as explained in sec-tion 4.2.This secsec-tion also explains experimental results of the calculated request-response times of Registration service of Application Server in an IMS core net-work.Larger sample sets are gathered for a better performance evaluation by considering the Request-response times on an hourly basis for complete day.
Table 5.1: Request Response times statistics.
Statistical label Request-Response Time(ms) Average (ms) 551.85 Minimum (ms) 502.4 Maximum (ms) 680.08 Standard deviation (ms) 39.29 Half-length of 95% CI (ms) 12.28 Relative Half-length of 95 % CI 2.26
The performance evaluation is performed based on the chosen metric using a statistical method of approach. It is observed that the average request-response time taken for a successful Registration procedure is 551.85 milliseconds.
The confidence interval is calculated using the formula shown in following equation, where x is the sample mean calculated from the sample size n ( 40 in this case) iterations of the experiment and s is the square root of variance, which also called as standard deviation.
Chapter 5. Results 33 Greater deviations from this value might be due to ineffective Registrations performed due to faulty behavior either at the Server end or improper function-ality of the stack. It can be observed from the graph of sample data that at the specific clock times of 13 and 14 hours, there is a larger deviation from the aver-age value. On further investigation on the issue, it was noticed that there was a problem in RCS application service chain during those hours of the day and hence greater delay. Hence, such Performance metric values help in detecting failures or impairments causing the inability of a registrar to receive client´s REGISTER requests.
Chapter 5. Results 34
Chapter 6
Conclusion and Future work
6.1
Conclusion
In this chapter we conclude this thesis by summarizing our contributions and discussing directions for future work.
In this paper, we presented a traffic capture and replay system based on a trace- based scenario replay mechanism considering the interpreter character of RCS stack and then show the validation of the replayed scenarios. The design and implementation of the replayer satisfies all the design goals mentioned in this thesis as per industrial request. The replayer is implemented in a controlled test environment, and the settings are configurable by the user to test scenarios with desired RCS configurations.These parameters can be either reused to reproduce traffic or changed to create new scenarios. This work provides a real traffic re-play solution with consideration of event driven rere-play to reproduce RCS based scenario in a test environment. The implemented replayer uses the reconstructed payloads along with Terminator intents to achieve packet-replay control. To ex-amine how the replay tool approximates the captured scenario, a performance metric, Terminator java based application is used to quantify the performance of traffic replay. The control over the environment is obtained by replicating RCS settings in the test environment similar to real capture environment.The paper also presents a study on the performance evaluation of RCS based application service.
From the results presented in Chapter 5, it is found that the implemented re-player could successfully replay captured chat scenarios. There might be several discussions on the comparison of the implemented replay tool and similar existing tools in the market. The main difference between them is that the implemented replay tool doesnot consider a black box kind of replay like how other replay tools does. The RCS based replay tool considers a continuous interactive replay of the scenario with a two way flow of the traffic. However, there are several limitations in this kind of replay as it ignores the fact that the RCS settings do not always remain constant. There might also be the discussion about the changing trend of technology which might lead to outdation of the implemented replay tool. But
Chapter 6. Conclusion and Future work 36 the fact that even if the technology keeps changing the idea of considering the interactive replay rather than a black box testing remains worthily same. Such an interactive replay of the scenario by the consideration of the interpreter char-acteristic of RCS stack provides a great favour to the debugging of RCS stack. In addition the research study performed, it was found that the Application Service performance evaluation is a way to determine the possible location of faulty be-havior of the Application Service. The important task of this thesis is to design and implement a record and replay tool that can possibly replay complex scenar-ios as discussed in the paper. The main observations from the results are: For scenarios 1, 2 and 3 the replayer could replay the captured chat scenarios.
6.2
Future Work
This thesis presents a replayer that successfully handles replay of simple chat scenarios and is extensible to the file-transfer scenarios.For a successful replay of file transfer scenarios, there is requirement to recreate the media files from the captured network traces. Such a file recreation is out of the scope of this research.
References
[1] [RFC 3261] J. Rosenberg, H. Schulzrinne, G. Camarillo, A. Johnston, J. Peterson, R. Sparks, M. Handley, E. Schooler, “SIP: Session Initiation Pro-tocol," RFC 3261, June 2002.
[2] [RFC 2327] M. Handley, V. Jacobson, “SDP: Session Description Protocol," RFC 23227, April 1998.
[3] [RFC 4975] B. Campbell, Ed., R. Mahy, Ed., C. Jennings, Ed., “MSRP:The Message Session Relay Protocol," RFC 4975, September 2007.
[4] [RFC 1889]H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson, “RTP: A Transport Protocol for Real-Time Applications Audio-Video Transport Working Group," RFC 1889, January 1996.
[5] [RFC 6076]D. Malas, A. Morton, “Basic Telephony SIP End-to-End Perfor-mance Metrics," RFC 1889, January 2011.
[6] [RFC 1945] T. Berners-Lee, R. Fielding, H. Frystyk, “Hypertext Transfer Protocol – HTTP/1.0," RFC 1945, May 1996.
[7] [RFC 3432] V. Raisanen, G. Grotefeld, A. Morton, “Network performance measurement with periodic streams," RFC 3432,November 2002.
[8] Ku. C, Lin. Y Lai. Y, Li. P, Lin. K, Real traffic replay over WLAN with en-vironment emulation, in Proceedings of the IEEE Wireless Communications and Networking Conference, WCNC 2012, pp. 121-132.
[9] Chu. W, Guan. X, Cai. Z, Gao. L, Real-time volume control for interactive network traffic replay, in Proceedings of the Computer Networks, 2013, pp. 121-132.
[10] Shanghai, China, 2012. Yindar Lin, Poching Lin, Tsunghuan Cheng, Iwei Chen, Yuancheng Lai, Low-Storage Capture and Loss Recovery Selective Replay of Real Flows, IEEE Communication Magazine, 2012, pp. 114-121. [11] Boulle K Cleuziou O Rodrigues J, RCS-e: A signalling challenge, Proceed-ings of 2012 15th International Telecommunications Network Strategy and Planning Symposium, NETWORKS, 2012.
References 38 [12] Maguluri .S, "PERFORMANCE EVALUATION OF SIP
AUTHENTICA-TION AND TLS",2012
[13] Molavi Kakhki. A, Razaghpanah .A, Golani .R, Choffnes .D, Gill .P, "Iden-tifying Traffic Differentiation on Cellular Data Networks",2011
[14] Charles V. Wright, Christopher Connelly, Timothy Braje, Jesse C. Rabek, Lee M. Rossey, Robert K. Cunningham, Generating Client Workloads and High-Fidelity Network Traffic for Controllable, Repeatable Experiments in Computer Security, Recent Advances in Intrusion Detection, Ottawa, On-tario, Canada, 2010. pp. 218-237.
[15] G. C. et al., "tshark - dump and analyze network traffic.",[ONLINE] Available:<https://www.wireshark.org/docs/man-pages/tshark.html>. [16] Jitendriya Athavale. et al., "ASA: Using Packet Capture to
trou-bleshoot ASA Firewall : Configuration and Scenario’s",[ONLINE] Available: https://supportforums.cisco.com/document/69281/asa-using-packet-capture- troubleshoot-asa-firewall-configuration-and-scenarios, Au-gust,2011.
[17] Auffret J., "RCS-e stack API Specification",[ONLINE] Tech.Report, 2011. [18] "GSMA:Specs and Products",[ONLINE] Available:<http://www.gsma.com
/network2020/rcs/specs-and-product-docs/>.
[19] IP Communication Services project, "Rich Communication Suite 5.1 Ad-vanced Communications Services and Client Specification Version 4.0", Tech.Specification, November 2013.
[20] Lenders, V.; Martonosi, M., "Repeatable and Realistic Experimentation in Mobile Wireless Networks," in Mobile Computing, IEEE Transactions on , vol.8, no.12, pp.1718-1728, Dec. 2009
[21] Aaron Turner,"tcpreplay", [ONLINE]. Available: <http://tcpreplay. syn-fin.net/trac/wiki/tcpreplay>
[22] Android Developers,"ADB:Android Debug Bridge",[ONLINE], Available: <http://developer.android.com/tools/help/adb.html>
[23] "SIPp - performance testing tool for SIP protocol",Available: <http://sipp.sourceforge.net>,2010.
Apendix
In this appendix we describe the procedure followed for the implementation and the information about the softwares used in the thesis.
Implementation
This section includes the implementation details and explains the pref-erence of the method in this thesis. Starting from the programming language preferred for development, class structure details and the implementation steps are explained. Python is chosen as the development programming language. The replay tool is implemented as a test case in test framework specific to Sony mobile communications. Python coding language is used to implement the class hierar-chy. By following the class structure and using the Python language, steps that should be followed for running the software is defined.
Programming Language
This thesis can be divided into two level process. The first level is the traffic characterization, which requires pcap file traversing to get the packet level in-formation, storage of 5-tuple data into a MYSQL database. Second level is the replay of the scenario included in the aRCSine framework. The parsed data is obtained in CSV format, which are saved according to Protocol classification. Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and binding, make it very attractive for rapid application development, as well as for use as a scripting or glue language to connect existing components together. Python’s simple, easy to learn syntax emphasizes readability and there-fore reduces the cost of program maintenance. Another reason for the choice of Python is that it supports modularity and code reuse helping developers in the further go.
Python’s dpkt module is used to decode packets from pcap files and analyze them according to their header information. DPKT is a powerful python module for simple packet parsing, with definitions for the basic TCP/IP protocols. By using DPKT it is possible to get the source IP address, source port number, destination IP address, destination port number, packet size, payload ingredients and capture time and protocol information from each packet.
Apendix 41 Every .pcap file which is set as input to the replay tool is set as object. The replay tool named Test.py consists of class named Test. Initially after the connection of handset to the local PC using usb in the local environment, the handset is checked for detection over the usb ports. Using the Wlan method, both the handset and PC are connected to the local medium by enabling the wlan mode and all the related configuration data for the setup is used.
The local ip of PC is setup as the local medium and the obtained handset IP is used as the handset medium:
localmedium = local handsetmedium = handset setifacenames(localmedium, handsetmedium)
The setup() definition includes all the necessary servers and callbacks to be initiated. The choice over which servers to be initiated is taken from the data captured. If the data includes SIP and MSRP traffic, then respective servers are initiated. This initial setup mode uses various other methods to start up the RCS stack, to servers scripted in python, the database that the stack uses and the Terminator application for the callbacks initiation. The java application is started using the adb shell commands from the replay tool,
handset.shell(’am startservice variable’ TERMINATORSERVICENAME) where, ’TERMINATORSERVICENAME’ is replaced with the terminator service existing in the client.
