Is QUIC a Better Choice than TCP in the 5G Core Network Service Based Architecture?
PETHRUS GÄRDBORN
KTH ROYAL INSTITUTE OF TECHNOLOGY
SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
TCP in the 5G Core Network Service Based Architecture?
PETHRUS GÄRDBORN
Master in Communication Systems Date: November 22, 2020
Supervisor at KTH: Marco Chiesa
Supervisor at Ericsson: Zaheduzzaman Sarker Examiner: Peter Sjödin
School of Electrical Engineering and Computer Science Host company: Ericsson AB
Swedish title: Är QUIC ett bättre val än TCP i 5G Core Network Service Based Architecture?
Abstract
The development of the 5G Cellular Network required a new 5G Core Network and has put higher requirements on its protocol stack. For decades, TCP has been the transport protocol of choice on the Internet. In recent years, major Internet players such as Google, Facebook and CloudFlare have opted to use the new QUIC transport protocol. The design assumptions of the Internet (best-effort delivery) differs from those of the Core Network. The aim of this study is to investigate whether QUIC’s benefits on the Internet will translate to the 5G Core Network Service Based Architecture. A testbed was set up to emulate traffic patterns between Network Functions. The results show that QUIC reduces average request latency to half of that of TCP, for a majority of cases, and doubles the throughput even under optimal network conditions with no packet loss and low (20 ms) RTT. Additionally, by measuring request start and end times “on the wire”, without taking into account QUIC’s shorter connection establishment, we believe the results indicate QUIC’s suitability also under the long-lived (standing) connection model. In conclusion, from a performance perspective, QUIC appears to be a better candidate than TCP in the 5G Core Network Service Based Architecture.
Sammanfattning
Den snabba utvecklingen av det mobila nätverket 5G har medfört högre krav för 5G Core Network och dess protokoll. Under årtionden har TCP dominerat Internet som transportprotokoll. De senaste åren har dock stora aktörer som Google, Facebook och CloudFlare börjat använda det relativt nya transport- protokollet QUIC. Designantagandena för Internet (best-effort delivery) och 5G Core skiljer sig dock åt. Målet med denna studie är att undersöka huruvida QUIC är lika fördelaktigt att använda i 5G Core Network Service Based Ar- chitecture som på Internet. En testbädd sattes up för att emulera trafiken mel- lan två nätverksfunktioner. Resultaten visar att QUIC har halverar latensen jämfört med TCP i en majoritet av fallen, samt dubblerar genomströmning- en även under mycket goda nätverksförhållanden där inga paket förloras och tur-och-retur-tiden är låg (20 ms). Genom att mäta ett HTTP requests start- och sluttider “rakt på ledningen”, utan att inbegripa QUICs kortare förbin- delseetableringstid, menar vi att resultaten indikerar QUICs lämplighet också vid användande av långlivade kommunikationsförbindelser. Med avseende på prestanda, framstår QUIC som ett bättre alternativ än TCP också i 5G Core Network Service Based Architecture.
Acknowledgements
I would like to express my gratitude to several experts at Ericsson for the op- portunity to work on this interesting topic and for the enriching discussions that emerged. I would like to thank my industrial supervisor, Zaheduzzaman Sarker, for his knowledgeable guidance, help, and feedback during this project.
I would like to thank Patrick Sellstedt, for providing me the platform at Eric- sson for this work. I would also like to thank Magnus Westerlund and Mirja Kühlewind for additional very valuable feedback and insights.
I am also grateful to my KTH supervisor, Marco Chiesa, for his valuable feedback and insights, and to my examiner, Peter Sjödin, for reviewing and approving the initial thesis proposal and providing helpful academic feedback on the report.
I would also like to thank my parents for always being supportive, my father for the encouragement and support during all my studies, and my mother for believing in a positive outcome of this work. Finally, I want to thank my wife for enabling me to spend time, sometimes on nights and weekends, to track down bugs and finalize the report. Without your support, I would not be able to have both a Master Thesis and a beautiful family as well.
1 Introduction 1
1.1 Overview . . . 1
1.2 QUIC vs. TCP . . . 2
1.3 Problem Area . . . 3
1.4 Goals . . . 4
1.5 Methodology . . . 4
1.6 Delimitations . . . 5
1.7 Ethics and Sustainability . . . 5
1.8 Outline . . . 5
2 Background 7 2.1 Mobile Systems Architecture . . . 7
2.1.1 Common Logical Building Blocks . . . 7
2.1.2 5G Core Service Based Architecture . . . 8
2.2 Protocols . . . 9
2.2.1 Introduction . . . 9
2.2.2 TCP/TLS . . . 11
2.2.3 HTTP/1.1 and HTTP/2 . . . 12
2.2.4 QUIC and HTTP/3 . . . 13
2.3 QUIC on the Web vs. the 5G Core . . . 15
2.4 Related Work . . . 16
2.4.1 QUIC in the 5G Core Service Based Architecture . . . 16
2.4.2 QUIC on the Web . . . 17
3 Method 19 3.1 Testbed Requirements . . . 19
3.1.1 Overview . . . 19
3.1.2 Requirements to Ensure Comparability . . . 20
3.2 Testbed Components . . . 24
vi
3.2.1 Platform & Network Tools . . . 24
3.2.2 QUIC/TCP Client & Server Implementations . . . 25
3.3 Benchmark Parameter Settings . . . 26
3.3.1 Main Parameters . . . 26
3.3.2 Sub-parameters . . . 27
3.4 Key Performance Indicators . . . 27
3.5 Test Design . . . 28
4 Results 30 4.1 Sub-parameters . . . 30
4.1.1 Number of Multiplexed Requests . . . 31
4.1.2 Request Payload . . . 35
4.1.3 Response Payload . . . 39
4.2 Main Parameters . . . 46
4.2.1 Round Trip Time . . . 46
4.2.2 Packet Loss Rate . . . 47
5 Discussion 50 5.1 Faster Connection Establishment – Not the Only Factor for Improving Latency . . . 50
5.2 QUIC has a Significant Advantage Also in QoS Networks . . . 52
5.3 Performance Difference is Negligible With Few Multiplexed GET Requests . . . 52
5.4 Impacts of the Method used to Measure Request Duration . . . 53
5.5 Limitations & Future Work . . . 53
5.6 Conclusions . . . 54
Bibliography 56
Introduction
A short background overview of the research is presented followed by a brief comparison of QUIC and TCP. We then lay out the problem area, research question and hypotheses followed by the goals, methodology and a brief dis- cussion of ethical concerns and impacts on sustainability. Chapter 2 contains a more thorough background presentation.
1.1 Overview
Mobile telecommunication systems evolve continuously in order to meet new and increasing demands such as massive IoT deployment, time-sensitive re- mote control of robots over a network and increased throughput. Between 2010 and 2020, there was an expected capacity increase of a 1000 times more devices, 100 times higher data rate and 10 times lower latency. The fifth gener- ation (5G) mobile telecommunication system has been designed to meet these demands. [1]
3G, 4G and 5G architectures all have in common that they consist of two networks: the Radio Access Network and the Core Network. The role of the Radio Access Network is to provide a connection between the User Equipment – a cell phone or other device – and the Core Network. The Core Network is responsible for handling everything related to connection management and the forwarding of data to and from the Internet. It is logically divided in two parts, the Control Plane and the User Plane. The User Plane only handles the forwarding of data and the Control Plane handles all other functionality such as authentication, authorization, policy control and so forth. [2]
In order to modularize network components and make them independently scalable and evolvable, the new Service Based Architecture (SBA) was in-
1
troduced with 5G for the Control Plane of the Core Network [3]. SBA is a microservices-type architecture where the goals are to provide easier scaling and more rapid development [4, p. 12]. According to the "System architecture for the 5G System" specification [3], Network Functions (NFs) communicate over Representational State Transfer (REST) interfaces via request-response and subscribe-notify communication patterns. The authors argue that by iso- lating NFs and using generic protocols such as HTTP for communication, scal- ing and evolvability of individual components become easier; thereby, SBA also paves the way for eventual transitioning into a cloud-based environment where NFs run in containers in data centers.
The choice of protocols impacts all types of communication. With the introduction of REST interfaces in SBA, the protocol stack of the higher layers – layer 3 to layer 5 – becomes very similar to the protocol stack used on the Web [4, p. 22]. Therefore, any new protocol development in the Web stack could also be of use in the SBA protocol stack. In this thesis, we are interested in evaluating the transport protocol performance within the context of the 5G Core Network Control Plane SBA.
1.2 QUIC vs. TCP
The Transmission Control Protocol (TCP) has been the transport protocol of choice on the Internet for decades and provides reliability as well as congestion control [5]. In conjunction with the Transport Layer Security (TLS) protocol, authentication and encryption are also ensured [6]. However, in 2012 Google laid out the guidelines for a new transport protocol called QUIC [7]. The protocol was submitted to the Internet Engineering Task Force (IETF) which has since continued the development [8] and made substantial changes to the protocol. It is the IETF draft of QUIC [9] that this thesis is concerned with.
To understand why companies such as Google, CloudFlare and others opt for QUIC whenever possible [10, 11], we need to understand the design goals of QUIC and in what regards QUIC performs better than TCP. One of the main goals was to reduce latency in order to make web applications more responsive [7]. QUIC achieves this goal by reducing the connection establishment time by one Round Trip Time (RTT) when compared with TCP over TLS1.3 and two RTTs when compared with TCP over TLS1.2 [12].
Another goal was to eliminate the Head-of-Line blocking problem plagu- ing TCP [7]. The problem arises since TCP is not able to differentiate multiple HTTP/2 streams sent over the same TCP connection [13]. A packet loss in one stream therefore brings all other streams to a halt until the lost packet has been
recovered. With HTTP/3 over QUIC, however, streams are differentiated also at the transport layer and the Head-of-Line blocking problem eliminated [13].
1.3 Problem Area
Although QUIC brings certain improvements to the Web, these benefits do not necessarily translate to the Core Network. The design assumptions of the Internet are different from those of the Core Network. The Internet only pro- vides best-effort delivery guarantees which means that packets can be lost or reordered in the network [14]. In the Core Network, however, it is possible to have much more control over the network itself since it is often proprietary.
Quality of Service with regards to throughput, latency, Packet Loss Rate (PLR) and Packet Error Rate (PER) can be ensured.
Connection characteristics also need to be taken into consideration. After all, the transport protocol is only implemented in the end hosts (the NFs in our case) [15]; therefore, even if identical network characteristics are used across multiple tests, they may still yield different results depending on differences in the transport protocol design. What we investigate in this thesis is:
Does QUIC perform significantly better than TCP in the 5G Core Service Based Architecture?
In order to provide a framework for evaluating the research question, first we present three hypotheses followed by a motivation for each:
Hypothesis 1 QUIC allows the 5G Core Network to tolerate a significantly higher Packet Loss Rate than TCP.
Hypothesis 2 QUIC allows the 5G Core Network to tolerate a significantly higher degree of packet reordering than TCP.
Hypothesis 3 QUIC enables significantly better performance than TCP in the 5G Core Network when a majority of connections are short-lived.
The motivation for hypotheses 1 and 2 is due to the fact that some of QUIC’s greatest benefits are likely to come into play when packets are lost and reordered, since that is where QUIC’s HOL blocking elimination should enable higher throughput and lower latency than TCP [16, 17]. The motiva- tion behind Hypothesis 3 is that QUIC has faster connection establishment time than TCP and probably should benefit from an environment where many connections are short-lived [18].
It is also important to note that each hypothesis includes a parameter that can be varied across a spectrum. The Packet Loss Rate may be adjusted for Hy- pothesis 1, the amount of packet reordering for Hypothesis 2 and the fraction of long-lived connections as well as how long they should be in Hypothesis 3.
The idea is to identify the optimal performance region of QUIC under these premises and then see if it suits the specific use cases of SBA.
1.4 Goals
The first goal is to produce a test environment in which the performance of QUIC and TCP can be tested. This environment can then be used in future research, building on the work of this thesis. The second goal is to design tests in such a way that they will give an indication as to whether QUIC would increase overall performance in the 5G Core Network SBA. The third goal is to present data indicating whether QUIC performs significantly better than TCP in the 5G Core Network SBA with respect to latency and throughput.
1.5 Methodology
Chapter 3 gives a thorough description of the methodology used. In summary, the method of the thesis follows the steps listed below:
1. Perform a literature study.
2. Set up a test environment where it is possible to simulate different net- work characteristics such as throughput, packet loss rate and latency as well as including a number of end hosts (NFs).
3. Choose QUIC and TCP implementations and provide a framework in which it is possible to switch between QUIC and TCP in the same test scenarios.
4. Ensure that it is possible to log relevant data.
5. Test each hypothesis over a spectrum of measurements.
6. Find out under what circumstances QUIC/TCP performs the best.
1.6 Delimitations
The 5G Core Network SBA is a vast ecosystem and it is virtually impossible to perform a comprehensive comparison of QUIC and TCP in all aspects of the network. We therefore delimit this study to emulate communication between two virtual Network Functions within a controlled virtual environment using either QUIC or TCP. The focus is on comparing the performance of each proto- col, with respect to latency and throughput, under various multiplexing levels using message size ranges common to the 5G Core Network SBA. Although simplified, we consider our testbed to be a valid model of communication be- tween two Network Functions.
1.7 Ethics and Sustainability
From an individual viewpoint, we do not see any ethical issues with this project since there is no sensitive data collected on individuals. Neither do we see any ethical issues when it comes to potentially revealing trade secrets since all documents and source code used in this project has been publicly available.
The aim of the study is to find out whether QUIC is a better protocol than TCP in the 5G Core Network SBA. If the study proves helpful in selecting the most efficient protocol, it would lead to more efficiently designed mobile networks, consequently saving energy resources in 5G networks all over the world, which would be beneficial in creating a more sustainable technological infrastructure. Mobile communication infrastructure is an essential part of societies all over the world. A more efficient architecture could help bring down overall costs and raise the quality of mobile communication also in lesser developed parts of the world.
1.8 Outline
In Chapter 2, we provide the necessary background needed to understand mo- bile systems architecture in general and the 5G Core Network SBA in partic- ular. We continue with a thorough presentation of QUIC, TCP and related studies. In Chapter 3, we describe the testbed in detail and motivate the setup and test design. We then present the results of our study in Chapter 4. Fi- nally, in Chapter 5, we point out the most interesting findings from the results with a view towards our initial hypotheses, research question and related work.
We discuss the limitations of the work, give suggestions for future work and present our final conclusions.
Background
First, we present an overview of mobile systems architecture with an emphasis on the 5G Core Network and more specifically the Service Based Architecture.
Then follows a description of the protocols of interest for this project, mainly QUIC and TCP, and a discussion of related research. Finally, a brief overview of the tools used for the experiment is provided.
2.1 Mobile Systems Architecture
To begin with, the common traits of mobile systems architecture across differ- ent generations are accounted for and explained followed by a more detailed look at 5G, and more specifically the 5G Core Network Service Based Archi- tecture.
2.1.1 Common Logical Building Blocks
Even though each generation of mobile networks was made possible due to new technology and architectural designs, certain logical building blocks re- main the same throughout generations 2, 3, 4 and 5 [19]. These building
Figure 2.1: Common Traits of Mobile Architecture
7
Figure 2.2: 5G System Architecture
blocks are displayed in Figure 2.1. The User Equipment is any device that uses the mobile network. It connects to the mobile network via the Radio Ac- cess Network. The responsibility of the Radio Access Network is to establish a connection between the User Equipment and the Core Network [2]. The Core Network handles connection management – authorization, authentica- tion and so on – and the forwarding of data to an external network [2]. The Core Network is logically divided into the Control Plane, which is responsible for connection management, and the User Plane, which handles data forward- ing [2]. The User Equipment, Radio Access Network and Core Network have persisted from the first generations of mobile systems up until 5G [19] but the internals of each of these building blocks have changed radically.
2.1.2 5G Core Service Based Architecture
The general design principles that have guided the formation of 5G are recorded in the 3GPP standards [3]. They include separation between Control Plane and User Plane. The motivation for logically separating the Control Plane and User Plane is to enable independent scaling and evolution. Another motivation is to enable a more flexible deployment, centralized or distributed. Higher-level Network Functions could thus be moved to the cloud and serve many base stations [1]. Another design principle of the 3GPP was to take a modularized approach to Network Function design. Each Network Function provides a spe- cific service in the Control Plane, such as authentication, authorization and so on [3]. The Service Based Architecture (SBA) is the result of modularizing Network Functions and is, in essence, a microservices based design approach
[20]. Many of the Network Functions can be seen in the top of Figure 2.2.
Instead of having Network Functions communicating over point-to-point interfaces, in SBA services are exposed – offered – to other Network Functions and they can request the services as needed [4, p. 19]. Each Network Function is logically connected to a common networking infrastructure and the princi- ples of HTTP REST are used for communication [4, pp. 22-23]. Important principles in REST, as described by 3GPP [21], are that in a communication scenario with two entities, one acts as a client and another as a server. The client requests services from the server. Every request needs to be stateless, in other words, the server should not have to remember what the client did in a previous request to be able to process the next.
Even though the use of HTTP is a cornerstone also of Web traffic, in con- trast to the Web, the Core Network is not deployed on the Internet but resides between the Access Network and an External Network as seen in Figure 2.1.
Since most studies of QUIC have been done on the Web, it is of interest to note some important differences between the Web and Core Network since the underlying network will affect the choice of transport protocol. We will discuss these differences in-depth in Section 2.4.2 after we have presented the protocols.
2.2 Protocols
Protocols form the basis for any type of communication. They describe the format and order of messages communicated between two or more entities and what actions should be performed when sending and receiving messages [22, p. 37]. After a brief introduction on the role of different protocol layers, we will focus on describing the transport layer protocols, TCP and QUIC in particular, which are the focus of this work.
2.2.1 Introduction
The 5G Core Control Plane uses the HTTP protocol stack [4, p. 347] where each protocol corresponds to a certain layer. Each layer is shown in Figure 2.3 and their respective roles described below according to RFC 1122 [15].
End hosts implement the full protocol stack, as seen in Figure 2.3 – ap- plication, transport, network, link and physical layers – whereas routers in the network only implement the network, link and physical layers. The ap- plication layer protocol is what the application uses to communicate with an application at the other end. The application layer hands over its messages to a
Figure 2.3: Protocol Layers of 5GC Control Plane
transport layer protocol. The transport layer provides certain services to both ends which will be discussed in detail below. The transport protocol hands over the message to the network layer protocol which encapsulates the data, creates a packet and sends it out in the network. The network layer is used by routers in the network to route the packet to its destination. The link layer is responsible for handling communication between two physically adjacent nodes. [15]
Broadly speaking, there are two main types of transport protocols: connec- tionless and connection-oriented. Connectionless transport protocols encap- sulate application messages into datagrams and send them off without knowing anything about the receiving end or the network. Many factors could prevent a packet from reaching its destination. The receiving end might be down or overloaded and not able to receive the packet at the moment, the network might lose the packet and so on. The benefits of a connectionless service is that it is simple and straightforward.
The other main type is connection-oriented transport protocols which solve many of the problems of connectionless protocols. Connection-oriented pro- tocols establish a connection with the receiving end and send packets over that connection. Different parameters of the connection itself can be adjusted to provide reliable delivery and other services to the end hosts. [15]
Another important aspect that could belong to the transport protocol layer itself or put in between the transport and application layer is security [9][23].
Nowadays, we expect a secure channel of communication that provides con- fidentiality, authentication and integrity [23]. Confidentiality refers to data only being visible to the endpoints, hindering any eavesdropping between two communicating endpoints. Authentication is the ability to verify the sender of a message. Integrity is the ability to ensure that a message has not been tampered with.
Figure 2.4: RTTs before transmitting first data
2.2.2 TCP/TLS
The two most commonly used transport protocols are the User Datagram Pro- tocol (UDP) [24] and the Transmission Control Protocol (TCP) [25]. UDP offers a connectionless service, as described in Section 2.2.1, which does not guarantee that packets will arrive at their destination.
TCP, on the other hand, has several built-in features to enable applica- tions to transparently send and receive continuous reliable byte-streams over an unreliable network, as described in the original RFC [25]. Reliability is ensured by having the receiver acknowledging each packet it receives so that the sender knows it has reached its destination, otherwise the sender will re- send the packet. Flow control is the ability to ensure that neither endpoint will send more data than the receiver can handle. Congestion control refers to the ability of ensuring that network routers will not be overloaded by a sender and cause packets to be dropped. Simplified, TCP congestion control infers net- work congestion when ACKs are not received for packets sent [22, p. 298]. It is then taken as a sign that router buffers are full and the network is therefore starting to drop packets. Several congestion control algorithms can be used that are independent of the TCP implementation itself. Common congestion control algorithms used today include reno, cubic and bbr [26].
The connection establishment is the first thing that happens in a TCP con- nection and is referred to as a three-way handshake. The first step, as shown in the leftmost illustration of Figure 2.4, is when the client sends a SYN message to the server. If the server is ready to receive the connection, it will respond with a SYN,ACK, acknowledging the received SYN and indicating readiness to open a new connection. The third step is for the client to send an ACK ac-
knowledging the SYN received from the server. Each connection is identified by the source and destination IP and port numbers. [25]
Important to note for our purposes is that the TCP handshake induces one full Round-Trip-Time (RTT) before the client application can start to transmit data. However, TCP does not provide confidentiality, authentication or in- tegrity. For these purposes TLS (Transport Layer Security) is widely used as an additional layer in between the transport and application layers [12]. After TCP’s three-way handshake has taken place, TLS then needs to perform its crypto handshake to establish the shared key and other parameters [23]. TLS 1.2, which is still widely used, induces two additional RTTs as seen in the middle illustration of Figure 2.4, whereas TLS 1.3 reduces it to one additional RTT before the server can start transmitting data [12].
2.2.3 HTTP/1.1 and HTTP/2
After a brief overview of the development of HTTP, we will have laid the groundwork for understanding the interplay problems between the application and transport layers, which resulted in the inception of QUIC.
The authors of RFC 7231 define an HTTP message to be either a response or a request [27]. They further explain that the entity making a request is called client. Every request has a target identified by a Uniform Resource Identifier (URI). The entity receiving the request is called server and should respond to the requests of the client. Various HTTP methods can be used by the client to indicate the purpose of the request to the server. The methods of interest in this study is GET – which requests the current representation of a resource found at the server – and POST, where the client requests processing of data carried in its payload [27].
Saxcé et al. describe one of the major drawbacks with HTTP/1.1 to be that it is not possible to send more than one request at a time over a single TCP connection [28]. In order to send multiple requests concurrently, additional TCP connections need to be opened. Since TCP connections are identified through their port numbers and IP addresses, as mentioned in Section 2.2.2, it means that multiple ports would have to be used for a single client-server pair in order to achieve parallelization of requests. But as Roskind et al. noted:
“pairs of IP addresses and sockets are finite resources” [7]; in other words, it is possible to run out of port numbers if used excessively. Google therefore ini- tiated the development of SPDY, which later resulted in HTTP/2, and enabled multiplexing of several requests over a single connection [28]. Each HTTP request/response exchange is multiplexed onto its own stream [29]. HTTP/2
Table 2.1: Connection Establishment Latency Comparison QUIC TCP/TLS1.3 TCP/TLS1.2 Unknown server 1 RTT 2 RTTs 3 RTTs Known server 0 RTTs 1 RTTs 2 RTTs
provided additional improvements as well due to binary message framing – in- stead of text-based as in HTTP/1.1 – and a compression scheme of the HTTP headers called HPACK [29].
Currently, HTTP/2 over TCP is used by 3GPP in the 5G Core Control Plane [4, p. 348]. However, despite the improvements of HTTP/2, it still had some weaknesses and that is why the development of QUIC and the corresponding HTTP/3 started.
2.2.4 QUIC and HTTP/3
In 2012, Jim Roskind and his team at Google laid down the design goals for QUIC [7]. One of the main goals, according to Roskind, was to increase the responsiveness of Web applications by reducing connection establishment latency. Another important goal was to eliminate the Head-of-Line (HOL) blocking problem that had been plaguing HTTP/2 over TCP. The strategies used in QUIC to resolve these issues are described in more detail below. Google submitted QUIC to the IETF for consideration in 2016 [8] and several changes were made to the original protocol such as incorporating the TLS 1.3 hand- shake [30]. When referring to QUIC in this thesis henceforth, it is the IETF QUIC version that is meant.
Figure 2.5: The Head-of-Line Blocking Problem in HTTP2/TCP. A packet loss in one stream results in all streams being put to a halt until the packet has been recovered.
From a latency point of view, a major problem with connection-oriented protocols such as TCP [25] and SCTP [31] is the RTTs induced by the initial handshakes in order to establish a secure connection as discussed in Section
2.2.2. In Figure 2.4, we saw that TCP over TLS gives a delay of at least 2 RTTs before any data can start to be transmitted. This is because TCP and TLS require separate handshakes. The main way in which QUIC reduces the con- nection establishment latency is by combining the cryptographic handshake and the transport handshake [32]. In that way, QUIC never needs more than 1 RTT for setting up a secure connection. A comparison of connection estab- lishment latency between QUIC and TCP/TLS is presented in Table 2.1. If the server is previously known, then connection establishment latency could even be reduced to zero by sending a cryptographic cookie that identifies the client to the server [12].
The Head-of-Line blocking problem in HTTP/2 stems from the way that HTTP/2 interacts with TCP; HTTP/2 multiplexes requests by sending them on several individual streams – one stream per request-response interaction [29]. However, these streams are only visible at the application layer [32].
Once the application messages are handed over to TCP, they are treated as a single stream by TCP [32]. The advantage with HTTP/2 over TCP is that several requests can be sent in parallel over a single TCP connection. The disadvantage, however, is that if a packet loss occurs in one of the HTTP/2 streams, all of the other streams over that connection are put to a halt until the TCP has recovered the lost packet [32], as seen in Figure 2.5.
Figure 2.6: Elimination of the Head-of-Line Blocking Problem. With QUIC, only the stream that is experiencing the packet loss is put to a halt; packets in other streams will keep flowing.
In QUIC, the Head-of-Line blocking problem is eliminated by introducing streams also at the transport layer [30]. If a packet is lost in one of the streams, only that stream will be put to a halt until the packet is recovered, while packets will keep flowing on the other streams [32] as seen in Figure 2.6. This is one reason that makes QUIC more robust to packet loss than TCP, according to some studies [16, 17, 33].
Packet loss detection in QUIC is simplified since its packet number space is solely used to indicate order of transmission [34]. This is in contrast to TCP’s sequence number which is also used by the receiver to indicate the next ex-
pected segment in its ACK [25]. QUIC also enables more precise calculation of the RTT by having the endpoints providing the duration between receiving a packet and responding with an ACK [30]. The RTT is more accurate since it is now possible to deduct the time spent processing the packet.
The reason that QUIC necessitated the new HTTP/3 protocol and could not simply use HTTP/2 is largely because HTTP/2’s compression scheme HPACK requires total ordering of frames across streams while QUIC only requires or- dering within individual streams [10]. Therefore, a new HTTP header com- pression scheme needed to be created called QPACK [32]. Many other framing concepts from HTTP/2 are dropped in HTTP/3 since they are now taken care of at the transport layer instead [32].
Improving on existing TCP kernel implementations is described by Roskind as feasible albeit the implementation of these changes is cumbersome since the complete operating system need to be updated in order for the changes to take effect [7]. Therefore, he and his team decided to build a connection-oriented protocol on top of the ubiquitously available UDP protocol. Thus, QUIC pack- ets are encapsulated within UDP datagrams [30]. Moreover, all functionality provided by QUIC such as reliability, security, flow and congestion control etc.
is implemented in user space [12]. The advantages are that there is no need to update the operating system in order to experiment with the QUIC protocol.
The downside is that packet processing in user space could be less efficient than in the kernel [35].
Other features of QUIC not directly applicable to our study includes a new way of identifying a connection. Instead of identifying it based on the IP ad- dresses and ports of the parties communicating – like in TCP – QUIC gives each connection a unique ID which makes it easier to retain the connection when changing to new ports or IPs [30].
2.3 QUIC on the Web vs. the 5G Core
There are two major differences between the Web and the 5G Core Network that are of interest when comparing transport protocols:
1. The guarantees provided by the network layer.
2. The type of traffic sent over the network.
The reason that there even is a need for a transport protocol has to do with the characteristics of the underlying network. In the case of the Web, the Internet and Internet Protocol (IP) is used for its communications. IP is a best-effort
protocol; as the name implies, it will do its best to deliver data packets sent over its network. However, it does not give any guarantees. More specifically, it does not guarantee that a packet will actually be delivered, that a sequence of packets will be delivered in order, or that the content of the packet remains unchanged (integrity). [22, p. 220]
The Core Network, however, does offer Quality of Service guarantees on packet latency, throughput and error rate [4, ch. 9]. As we shall see in Section 2.4, most studies of QUIC have been done with the Web in mind. A different type of underlying network such as the 5G Core Network will also affect the performance of transport protocols.
The second difference has to do with the type of traffic sent over the net- work. When downloading a Web site, multiple objects of various sizes need to be retrieved such as images with sizes of several KBs. Most of the re- quest/response traffic in the 5G Core Network, however, consists of smaller payloads between 20–300 bytes as can be seen in the OpenAPI Specification Files for the 3GPP 5G Core Network [36]. These numbers have also been con- firmed verbally by people working in the industry; it has not been possible to retrieve other written sources on these numbers since many of those are pro- prietary. However, we believe the estimates used are certainly reasonable. In summary, the 5G Core Network is a more reliable network than the Internet and has a higher frequency of requests and responses with a low payload.
2.4 Related Work
To the best of our knowledge, there are no publicly available studies on QUIC within the 5G Core Network SBA that also present measurement data. The primary source of information regarding QUIC’s feasibility in the 5G Core Network SBA is the technical report TR 29.893 from the 3GPP which we dis- cuss below. However, this report mostly relies on comparing TCP and QUIC features from a theoretical point of view. Our work aims to provide real mea- surement results applicable for the 5G Core Network SBA.
Although there is a lack of studies done on QUIC within the 5G Core SBA, there are several studies on QUIC within the context of the Web; we also dis- cuss those studies and their findings as they relate to the 5G Core SBA context.
2.4.1 QUIC in the 5G Core Service Based Architecture
Although HTTP/2 over TCP is the current 3GPP standard in the 5G Core Net- work Control Plane [4, p. 348], in June 2019 the technical report TR 29.893
[37] was published by 3GPP on the potential of using QUIC in the 5G Core Network SBA. Five requirements for a transport protocol was listed on page 19. They were:
• Reliable message delivery.
• Flow control and congestion control mechanisms.
• Support of connection semantics (One HTTP connection maps to one TCP or QUIC connection).
• Failure to deliver one message shall not block subsequent messages (Head- of-Line blocking mitigation)
• The transport protocol supports mechanisms to authenticate the peer endpoint and secure the transfer of application messages.
All of the requirements are said to be fulfilled by QUIC. Yet, the authors con- cluded that QUIC is not yet ready to be used as a basis for the 5G Core Network Control Plane signaling due to the immaturity of the implementations at the time and the difficulty of using QUIC over a proxy. However, the IETF is cur- rently working on a draft describing discovery mechanisms for non-transparent proxies, which would be part in providing a solution for QUIC over proxies [38]. Moreover, one year is a long time for implementations to mature and there is a lot of work done by major Internet players such as Facebook and Cloudflare in the development of QUIC implementations [39, 40]. Still, these implementations require a lot of testing. Finally, the 3GPP concludes by en- couraging further studies on QUIC’s feasibility specifically in the 5G Core Network SBA [37]; the aim of this study is to make a contribution within this area.
2.4.2 QUIC on the Web
Most studies of QUIC are concerned with the Web. There seems to be a fairly general consensus that the setting where QUIC provides significant advan- tages, compared with TCP, is when network conditions are poor such as high RTT, high packet loss and low bandwidth. So did the authors of Does QUIC make the Web faster? note that 90% of web pages loaded faster with 2G com- pared to 60% when using 4GLTE [16]. The authors of QUIC: Better for what and for whom? [17] similarly conclude that “QUIC outperforms HTTP/2 over TCP/TLS in unstable networks”. The initial tests made by Google [33] also noted that QUIC outperformed TCP under “poor network conditions” and that
over the slowest connection, a Google Search page loaded a full second faster with QUIC.
A study made by Megyesi et al. [18] comparing Google QUIC with SPDY (predecessor of HTTP/2) and QUIC found that the “network conditions deter- mine which protocol performs the best”. They also noted that the condition where QUIC would shine the most is high RTTs. One weakness of Google QUIC was its performance on high-speed links. They attribute this weakness to its packet pacing mechanism not being able to reach the full capacity of a high speed link.
At a first glance, it seems like Kakhki et al. in their paper Taking a Long Look at QUIC [35] contradicts the previously mentioned studies by stating that “QUIC’s performance diminishes on mobile devices and over cellular net- works”. However, at a closer inspection, we find that it is not necessarily the features of the protocol itself that causes the worse performance over cellular network but rather QUIC’s reliance on application level processing and en- cryption makes it slower on mobile phones (from 2013 and 2014) and not as powerful as its desktop counterparts.
In summary, the consensus seems to be that QUIC has its most significant advantage on lossy networks with high RTTs. High bandwidth links could be a problem if the processing power of the device is poor.
Method
In the following chapter, we describe and motivate the procedure undertaken to identify whether QUIC performs significantly better than TCP in the 5G Core Network Control Plane SBA. First, a brief overview of the testbed architecture is presented followed by the requirements we have on the testbed. We then present the measurement/network tools and QUIC/TCP implementations that were used to adhere to our requirements. Finally, we describe and motivate the benchmark parameter settings, choice of key performance indicators and test design.
3.1 Testbed Requirements
In order to compare QUIC and TCP, we first need an environment that can run identical benchmarks using one protocol or the other. It is important to configure the QUIC and TCP setups in a way that will reduce the impact of other sources affecting the results. We begin with an overview of the testbed setup followed by a description of each requirement that we have on the testbed to ensure a fair comparison of the two protocols.
3.1.1 Overview
An overview of the testbed is shown in Figure 3.1. In the 5G Core Network SBA, Network Functions (NFs) make requests to other Network Functions over HTTP. As described in Section 2.2, HTTP runs over a transport layer pro- tocol. We needed one HTTP client/server pair (represented by NF A and NF B) running over TCP and another running over QUIC. Requests are sent from the client (NF A) and responses received from the server (NF B). In the case
19
Figure 3.1: Testbed Architecture
of TCP, we decided to use HTTP/2 since it is the current standard in 5G Core Network SBA [21]. The TCP setup also needs an additional protocol layer to provide encryption. TLS v1.3 was chosen for this purpose. The reasoning behind selecting this specific version is explained in Section 3.1.2. HTTP/3 is used over QUIC since it is the only HTTP version that works over QUIC. The client, bridge and server each resides in its own virtual network as described in Section 3.2.1. Finally, network emulation was needed in order to emulate real network characteristics such as link delay, packet loss, packet reordering and so forth. Emulation of packet loss happens on both NF interfaces (client and server) and emulation of RTT happens on the bridge interfaces.
3.1.2 Requirements to Ensure Comparability
The goal of this study is to determine whether QUIC gives any significant im- provements over TCP in the 5G Core Network Control Plane SBA. An over- arching guideline is to compare default QUIC and TCP implementations that would be widely available in various microservices rather than highly cus- tomized variants.
Also, when comparing results from two different setups, there is always the risk that characteristics other than the ones you want to compare affect the resulting data and creates unwanted noise. In our case, we wanted to compare the difference between the TCP and QUIC protocols and not, for instance, con- gestion algorithms. We have therefore strived towards, as far as possible, elim- inating other sources of noise in the measurement data by carefully consider the different configuration options in the TCP and QUIC setups respectively.
Each configuration option was set with the goal of providing a comparison that
Option QUIC TCP
QUIC Version IETF Draft 27 N/A
TCP Version N/A Linux Kernel 5.4.0
TLS Version 1.3 1.3
Hardware Acceleration N/A TSO, GSO, LRO,
GRO, TLS disabled.
Congestion Algorithm Cubic Cubic
TCPFastOpen N/A NO
0–RTT NO N/A
Connection Type Simplified Long-lived Simplified Long-lived
Multiplexing of Requests YES YES
Table 3.1: QUIC and TCP Configuration Option Decisions would be as fair and valid as possible.
In this section, we present the different configuration options that we deemed especially important to consider without pointing out specific implementa- tions; the tools and QUIC/TCP and HTTP2/HTTP3 implementations used are presented in Section 3.2. The configuration options considered and the choices made for each are summarized in Table 3.1. Some selections of the configu- ration options were not necessarily more right than others. Nevertheless, it was still important to make a conscious decision for each option in order to interpret the results accurately. We will now proceed to explain the reasoning behind the inclusion and setting of each option.
QUIC Version
As stated in Section 2.2, Google developed the initial QUIC protocol, now referred to as gQUIC, but submitted it to the IETF in 2016 to begin the stan- dardization process. It is the IETF QUIC protocol that we are interested to in- vestigate, and it is therefore important to ensure we are using the IETF version.
Since, at the time of writing, it is not yet standardized, it was also important to consider the draft version used. We wanted to use a draft that was as up to date as possible and are therefore using draft 27, the most current at the time of writing.
TCP Version
Even though there are specialized implementations of TCP running in user space, we wanted to use TCP from the Linux kernel since most microservices are not likely to implement their own TCP implementations. We use the Linux kernel 5.4.0.
TLS Version
In Table 2.1, we show that there is a full RTT difference between using version TLS 1.2 and 1.3 with TCP. Since we are using the most recent version of QUIC, and TLS 1.3 is already widely deployed [41], there is no reason for not using the most recent version of TLS also with TCP. Therefore, version 1.3 was selected in this study.
Hardware Acceleration
One of the major advantages with TCP is that it has been around for such a long time and therefore has a rich ecosystem supporting it. As discussed in Section 2.2 there are several hardware offloading mechanisms that can im- prove performance substantially. The offloading mechanisms identified by 3GPP [37] as able to increase TCP performance were: TCP Segmentation Offloading (TSO), crypto offloading (TLS), Generic Segmentation Offloading (GSO), Large Receive Offloading (LRO), Generic Receive Offload (GRO), and checksum offloading. By far, TCP Segmentation Offloading would have the most substantial impact with a reduction of up to 50 times on CPU usage [37].
All of these were disabled except for checksum offloading (due to limitations of the virtual machine environment) in order to compare the protocols them- selves and not hardware offloading capabilities. Checksum offloading is not expected to affect the result significantly.
Congestion Algorithm
Cubic was selected as the congestion algorithm on all implementations since it was the only congestion algorithm available on every implementation. How- ever, Cubic is a widely used congestion algorithm as mentioned in Section 2.2.2 and should therefore be a valid choice.
Connection Length
A connection could close immediately after a single request-response pair has been processed – a short-lived connection – or remain open – a long-lived con- nection. Due to time restrictions, we decided to only choose one connection model. Since the multiplexing aspect is such an essential part of both HTTP2 and QUIC because of its relation to the Head-of-Line blocking problem (as dis- cussed in Section 2.2), we decided to study the long-lived connection model.
Due to implementation restrictions, we chose a somewhat simplified vari- ant of the long-lived connection model in which all our requests are issued at once, albeit in a true long-lived connection model, they would be sent stochas- tically.
TCPFastOpen and QUIC 0–RTT
As discussed in Section 2.2, QUIC’s 0–RTT and TCP’s TCPFastOpen settings enable a client to immediately start sending data to a server that is previously known. Our QUIC implementation lacks the ability to use 0–RTT. However, since we opted to use long-lived connections with multiplexing of many re- quests, the effect that these early-data mechanisms would have on any of the protocols is reduced. As we shall see in Chapter 4, there is still a significant difference in request duration in QUIC’s favor even without taking into account the initial connection setup time. Worthy to note is also that the QUIC draft acknowledges that the 0–RTT feature is not always used since it has some se- curity drawbacks; increased latency at connection setup can thus be traded for stronger security protection against replay attacks that comes with the 1–RTT handshake [9].
Multiplexing
Some implementations that we tried were not able to perform multiplexing of requests. Multiplexing is an essential feature of both HTTP/2 and HTTP/3.
The multiplexing aspect was one of the most interesting ones to compare be- cause of its relation to the Head-of-Line blocking problem as discussed in Sec- tion 2.2. It was therefore necessary to choose implementations that could han- dle multiplexing. The ability to multiplex requests contributed to the choice of the long-lived connection model as well. All of the nodes are therefore able to multiplex requests.
3.2 Testbed Components
Apart from deciding on the requirements to ensure comparability, described in Section 3.1.2, another important aspect is the selection of components for the testbed; the implementations of QUIC/TCP should adhere to the requirements list in Table 3.1 and be stable and trustworthy. Moreover, good documenta- tion and ease-of-use are preferable and sometimes necessary in order to be able to perform the tests we desire. In the following sections, we present the components selected and the reasoning behind.
3.2.1 Platform & Network Tools
All benchmarks were run on the Ubuntu 18.04 operating system with four Intel i7 cores and 20 GB of memory. In order to emulate a real-world communi- cation scenario between networks, several tools described below were used to enable specific RTT and PLR selections.
Virtual Networks: Linux Network Namespaces
Running benchmarks locally poses some problems in providing the desired RTT and packet loss characteristics since only one interface is available – the loopback interface. However, with Linux Network Namespaces it is possible to have logically separate copies of the network stack with their own interfaces, firewall rules and routing tables [42]. This is very useful when using the tools for controlling the RTT and PLR described below.
Round-Trip-Time: netem
The netem Linux network emulator is a tool designed to emulate the charac- teristics of Wide Area Networks by controlling the RTT and PLR [43]. For our purposes, it is only used to apply a delay to packets to increase the RTT to a desired number. Even though netem is capable of also creating packet loss, the implementors do not guarantee accurate performance when running benchmarks locally [43]. Although we have a bridge in between the network namespaces, it is virtual and probably for that reason, we found that packet loss did not work as expected when using netem.
Packet Loss: iptables
The iptables program [44] is used to set up and maintain firewall rules and works very well in conjunction with netem. Since each network namespace has its own firewall rules, it is possible to set a desired amount of packet loss on the client and server interface respectively using iptables together with its statistical module; this is how it is used in our testbed.
Logging: tcpdump
The tcpdump program captures packets on a network interface and prints out that information to a .pcap file [45]. In our testbed, it is used to capture raw packets on the client interface for later higher-level processing.
Logging: Wireshark
Wireshark is a network protocol analyzer [46]. Its terminal-variant tshark [47]
is used in our testbed to interpret the different HTTP/2 and QUIC fields found in the .pcap file, generated by tcpdump, in order to identify the beginning and end of a request-response exchange, the number of bytes transmitted and the length of a complete connection.
3.2.2 QUIC/TCP Client & Server Implementations
There are two separate client programs for the TCP and QUIC implementa- tions. One server program is used that is able to run both TCP and QUIC.
QUIC Client: lsquic
The IETF QUIC working group maintains a website that lists all the available QUIC and HTTP/3 implementations [48]. Several good candidates are avail- able, and we also tried out several of them. The lsquic implementation [49], which is our choice, is provided by the LiteSpeed company, a company that specializes in server software design [50]. Their LiteSpeed QUIC server has been demonstrated to be highly performant [51]. The lsquic implementation also uses draft version 27 and enables multiplexing of requests with ease. The main downside of this implementation is that it does not offer the 0–RTT fea- ture. However, even though we are not able to see the reduction in connection setup offered by this feature, we are still able to get valuable information on request duration once the connection is opened as is demonstrated in Chapter 4.
TCP Client: nghttp2
The nghttp2 is a mature HTTP/2 implementation [52] offering the HTTP/2 terminal client nghttp which runs over the default Linux TCP stack (version 5.4.0). Since the default Linux TCP stack is used in our testbed, it was easy to turn off the hardware acceleration features listed in Table 3.1 by using the standard configuration tools provided in Ubuntu.
QUIC/TCP Server: nginx with quiche
nginx is a well-known server that profiles itself as a highly performant web server [53]. It has default support for HTTP/2 and runs over the default Linux TCP stack. The Content Delivery Network Cloudflare has implemented a QUIC extension that runs with nginx named quiche [54, 55]. Benefits of using nginx for both TCP and QUIC include ease of configuration, for instance only allowing TLS v.1.3 and so on.
3.3 Benchmark Parameter Settings
There are a multitude of parameters that should be considered in order to make a thorough and comprehensive comparison of TCP and QUIC. It is, however, not feasible to do a fully comprehensive comparison within the scope of this thesis and therefore some delimiting choices have been made. The focus is on QUIC and TCP performance within the 5G Core Network Service Based Architecture; that also affects the choice of what parameters to include in the benchmarks. We first present the main parameters of our study and then what we refer to as the sub-parameters.
3.3.1 Main Parameters
One of the main areas in which the future 5G Core Network SBA would differ from previous architectures is the physical distance between the nodes (Net- work Functions) communicating. 3GPP has clearly stated that the SBA micro- service-like architecture is intended to enable Network Functions to be de- ployed in different data centers far apart from each other rather than very close to each other as is the case today [3].
A more remote deployment of Network Functions will increase the RTT.
It is therefore important to investigate how an increasing RTT affects protocol performance. As Network Functions are deployed further from each other, the risk also increases of having a higher PLR. Therefore, it is also important to
look at the effect an increasing PLR would have on QUIC and TCP respec- tively.
Since both an increasing RTT and PLR are important implications of the future 5G Core Network SBA with remotely located Network Functions, these will be the main parameters that we focus on in our experiments.
3.3.2 Sub-parameters
The RTT and PLR, however, cannot be investigated without setting other pa- rameters that also affect protocol performance. The ones chosen to be included in this study are:
• Number of Multiplexed Requests
• Request Payload Size
• Response Payload Size
The number of multiplexed requests over a single connection is a param- eter that only needs to be set when the long-lived connection model is being used, since the short-lived connection model only sends one request at a time and then closes down the connection. Our implementation of QUIC did not have the 0–RTT feature available; hence, we only studied the long-lived con- nection model. The request payload and response payload are basic features of an HTTP request that always need to be set to zero or another value. In our implementation, the zero bytes payload request is represented by HTTP GET and as soon as payload is added, it is represented by HTTP POST.
It is important that the sub-parameters are set in a way that is as similar as possible to the 5G Core Network SBA traffic. Since the 5G Core Network SBA payload size varies between 20–300 bytes, as described in Section 2.4.2, we are not investigating the effect of trying to POST objects larger than 1000 bytes or several MB. The response payload, however, could be large for imple- mentational reasons and therefore we are investigating response payload sizes of up to 10 MB.
3.4 Key Performance Indicators
It is important to be able to measure the effect that the different parameters have on QUIC and TCP protocol performance. Two of the most important and somewhat lowest common denominators when evaluating the performance of
network communication in general is throughput and latency. Throughput basically measures the data per time unit ratio within a network. Latency, on the other hand, measures how fast data propagates. It is fully possible to have protocol characteristics with a high throughput but low latency or vice versa.
However, it is also possible to measure higher-level parameters such as Page Load Time (PLT) which is very important for the web where the end user is often loading a web page and reacting to how fast the web page loads.
Since we are not dealing with the loading of web pages in the 5G Core Network Control Plane SBA, we reasoned that it is most interesting to focus on latency of individual requests and throughput.
When evaluating a protocol, it could also be interesting to look at other aspects such as CPU usage that may also affect the final choice of protocol at the end. In this study, we decided to focus on pure network performance characteristics of the protocols, for which throughput and latency make good candidates.
There are also different ways of measuring throughput, but we use the model displayed in Equation 3.1. When measuring time, we always retrieve the timestamps from the client-side. The beginning of the connection is repre- sented by the very first packet that is sent over the connection and terminated by the very last packet, right before the client closes down the connection; thus, the full connection duration also includes the handshakes.
Throughput = Bytes From Server
Connection Duration (3.1)
When it comes to latency, we are mostly interested in how fast an HTTP re- quest completes and the KPI chosen for measuring latency is therefore request duration. The request duration measurement model is shown in Equation 3.2.
Request Duration = Response RX − Request TX (3.2) The request duration is the difference between the timestamp of the last response packet and the timestamp of the first corresponding request packet.
3.5 Test Design
Given five parameters to vary, it is unfeasible to vary them all to any greater extent because of the time it takes to run the benchmarks and perform the subsequent analysis. In order to have statistical significance, each identical benchmark is run 50 times. Thus, generating a plot with 10 points necessitates the running of 500 benchmarks. Because of the rapidly increasing number of
possible benchmarks that could be run when varying five different parameters, we have had to put some limitations and decide which parameters are the most important for the purpose of answering our research question.
As mentioned in Section 3.3, the focus is on primarily determining the protocol efficiency of QUIC and TCP as RTT and PLR increase due to more remote placement of Network Functions. At a first glance, it might seem straightforward to simply run benchmarks varying the RTT and PLR and see the performance of TCP and QUIC with regards to throughput and request du- ration. However, the other three parameters also influence the result. There- fore, measuring the performance of TCP and QUIC with regards to PLR and RTT is not just a one-dimensional measurement; we need to also look at the performance when we are multiplexing few requests or many requests, when the request/response payload is small or large and so on.
It is important to choose these values in a way that would reflect the 5G Core Network SBA as closely as possible. As discussed in Section 2.3, typ- ically, request/response payload within the 5G Core Network is between 20–
300 bytes. The response payload may be higher for implementational reasons, and we therefore try payloads up to 1 MB. The number of requests multiplexed over a single connection also varies and we have tried a whole spectrum, from 1–500 requests multiplexed over a single connection.
Results
In Section 3.3, we discuss in depth the reasoning behind the selection of pa- rameters to study. We point out that an increased RTT is a direct consequence of a futuristic deployment of the 5G Core Network SBA where Network Func- tions are placed in different data centers much further apart than what is the case today. It is also likely that the PLR would increase in this scenario. There- fore, it is of great interest to see if either protocol performs significantly better than the other with regards to request duration and throughput under varying degrees of RTT and PLR. We refer to these parameters as the main parameters.
It is equally important to also understand how the tuning of the number of multiplexed requests, the request payload and the response payload influence protocol performance; in this study, we refer to these parameters as the sub- parameters. The impact of these sub-parameter settings on protocol perfor- mance is valuable to understand on its own merits; additionally, an accurate and precise understanding of sub-parameter impact is important in order to make an informed decision on what their settings should be when evaluating PLR and RTT impact on protocol performance.
We begin by studying the impact of the sub-parameters on protocol per- formance in Section 4.1. Based on our findings, we then choose what the sub-parameter settings should be when we conclude with varying the main parameters – PLR and RTT – in Section 4.2.
4.1 Sub-parameters
Each main parameter and sub-parameter can only have one of two possible states at a time: static or dynamic. In a given benchmark, only one parameter can be dynamic; in other words, we only vary one parameter at a time in order
30
to isolate the impact of each parameter on protocol performance. Yet, the parameters that are not varied in a particular benchmark – the static parameters – also need to be set to some value. In order to have an overview of the different static values that were used in at least one benchmark throughout Chapter 4, we present Table 4.1. The RTT static value is set to 20 ms to emulate inter- regional deployments. The PLR static value is set to 0% to have a baseline of request duration and throughput results not being affected by congestion control mechanisms. The static values for each sub-parameter are selected based on the outcome of their results as presented throughout Section 4.1.
The reasoning behind sometimes selecting more than one value and why these particular values were chosen is also presented in the subsection of each sub- parameter. Important to know is also that when the request body is zero bytes, an HTTP GET request is used. When it is a non-zero value, an HTTP POST request is used.
We begin by varying the number of multiplexed requests in Section 4.1.1 followed by varying the request payload in Section 4.1.2 and the response pay- load in Section 4.1.3.
Parameter Value
Number of Multiplexed Requests 10; 25; 200
Round Trip Time 20 ms
Packet Loss Rate 0%
HTTP Request Body 0; 25 bytes
HTTP Response Body 200 bytes
Table 4.1: Even if a particular parameter is not being varied in a given bench- mark, it still needs to be set to a value. This table lists the various temporary values that a parameter can have in at least one of the benchmarks presented throughout Chapter 4.
4.1.1 Number of Multiplexed Requests
To begin with, we study the impact on request duration when varying the num- ber of multiplexed requests as seen in Figure 4.1. The number of multiplexed requests is varied between 10 and 500. The values of the other parameters during this benchmark is presented in Table 4.1 with the HTTP request body set to zero bytes.
At 10 multiplexed requests, the difference in request duration average be- tween QUIC and TCP is negligible; however, TCP’s request duration 95th
Figure 4.1: The request duration mean, 5th and 95th percentiles as a function of the number of multiplexed requests. The RTT is 20 ms, the PLR is 0%, the HTTP request body is zero bytes and the HTTP response body is 200 bytes.
percentile is twice as long as QUIC’s. Between 20–75 requests, QUIC re- duces average request latency to less than half of that of TCP. Between 50–75 requests, both TCP and QUIC request duration rises significantly. Between 75–175 requests, TCP average continues to rise until it reaches a peak average duration at 100 requests. At 75 requests, QUIC reaches a plateau and the re- quest duration average stays virtually the same until 175 requests after which it drops again. At 200 requests, QUIC request duration average is 60% of that of TCP and close to half when above 200 requests. Between 150–175 requests, the difference between QUIC and TCP average is very small; moreover, the 95th percentiles is not significantly better for one or the other.
To summarize, the number of multiplexed requests does have a significant impact on request duration. On average, QUIC enables requests to complete twice as fast compared to TCP when a low number of requests are multiplexed (20–75) and when a high number of requests are multiplexed (> 200). At very low intervals of multiplexed requests (< 10) the difference in request duration average is negligible. The difference is also very small when multiplexing 150–175 requests. However, as we shall see, the request payload also has a significant impact on request duration. An important detail in this result is that
Figure 4.2: The throughput mean, 5th and 95th percentiles as a function of the number of multiplexed requests. The RTT is 20 ms, the PLR is 0%, the HTTP request body is zero bytes and the HTTP response body is 200 bytes.
we are using an HTTP GET request with zero bytes request payload. Once we use an HTTP POST request with a payload of 25 bytes, we will see different results as presented in Section 4.1.2. Also, we should keep in mind that these results hold at an RTT of 20 ms and PLR of 0%.
Noteworthy is also that overall QUIC’s 95th percentile is further away from its average than TCP’s 95th percentile. In other words, QUIC’s spread is larger than TCP’s. Although a majority of QUIC requests will complete significantly faster than TCPs, there are 5% of the QUIC requests that have a significantly larger delay which can be important to be aware of for time critical applications for example.
In Figure 4.2, the throughput is displayed as a function of the number of multiplexed requests. Noticeably, TCP’s high increase in request duration av- erage between 10 and 25 requests is not reflected in the throughput as summa- rized by Table 4.2. At a first glance, one might think that an increasing request duration average should also lead to a decreasing throughput since fewer re- quests would be able to complete within the same time frame. Although coun- terintuitive at first, there is a valid explanation for this phenomenon. We will look at it in more detail since the same reasoning also applies when we look
No. of reqs. Req. Avg. Dur. Throughp. Avg. Tot. Number of Pac.
10 23 ms 55.3 KB/s 40
20 42 ms 88.7 KB/s 42
Table 4.2: Example of increase in request duration average and throughput average over TCP.
Request ID Response ID Req TX Resp. RX
1–10 47 ms
1–8 68 ms
9–10 89 ms
Table 4.3: Individual request TX and response RX timestamps when multi- plexing 10 requests over TCP.
at the request payload parameter in Section 4.1.2.
In Table 4.3, which shows a representative run of this benchmark with 10 multiplexed requests, we see that each TCP request receive the same times- tamp when sent out, 47 ms (they were all in the same packet). Corresponding responses 1–8 receive timestamp 68 ms. But the last two responses arrive quite a lot later and receive timestamp 89 ms. However, since only 2 out of 10 responses receive the latest time stamp, it does not affect the request duration average too much.
However, when increasing the number of multiplexed requests to 20, the situation changes as seen in Table 4.4. All 20 requests still receive the same request TX and the responses are received in two batches as in the example with 10 requests. But now, the latest response time account for 55% of all of the responses, thus increasing the request duration average substantially.
At the same time, as seen in Table 4.2, the total amount of packets is 40 Request ID Response ID Req TX Resp. RX
1–20 48 ms
1–9 70 ms
10–20 93 ms
Table 4.4: Individual request TX and response RX timestamps when multi- plexing 20 requests over TCP.
