Comparing the scalability of MQTT and WebSocket communication protocols using Amazon Web Services

(1)

IN

DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS

,

STOCKHOLM SWEDEN 2020

Comparing the scalability of MQTT

and WebSocket communication

protocols using Amazon Web

Services

MARCUS WERLINDER

KTH ROYAL INSTITUTE OF TECHNOLOGY

(2)

(3)

Comparing the scalability of

MQTT and WebSocket

communication protocols

using Amazon Web Services

MARCUS WERLINDER

Master in Computer Science Date: June 26, 2020

Supervisor: Hamid Faragardi Examiner: Elena Troubitsyna

School of Electrical Engineering and Computer Science Host company: Cepheid

(4)

(5)

iii

Abstract

(6)

iv

Sammanfattning

Denna studie jämförde skalbarheten mellan två populära kommunikations-protokoll inom IoT, MQTT och WebSocket. För att jämföra skalbarheten av kommunikationsprotokollen användes en utgivare-prenumerant-metodik. För MQTT användes meddelandehanteraren EMQX, och för WebSocket användes Socket.IO. Fem mätvärden dokumenterades; minnesanvändning, nätverksan-vändning, CPU-annätverksan-vändning, tur och retur - tid och meddelandeförlust. Medde-landehanterarna sattes i två olika scenarier. Första scenariot bestod av många utgivare och få prenumeranter. Det andra scenariot bestod av många prenume-ranter och få utgivare. AWS EC2 användes för att agera som värd för medde-landehanterarna, samt för utgivarna och prenumeranterna.

(7)

v

Acknowledgements

(8)

CONTENTS vii 4 Method 16 4.1 Brokers . . . 16 4.1.1 MQTT Broker . . . 16 4.2 WebSocket Broker . . . 18 4.3 Experimental Process . . . 18 4.4 Data collection . . . 20 5 Results 21 5.1 Scenario 1 . . . 21 5.1.1 MQTT Brokers . . . 22 5.1.2 WebSocket Broker . . . 25 5.2 Scenario 2 . . . 28 5.2.1 MQTT Broker . . . 28 5.2.2 WebSocket Broker . . . 31 6 Discussion 35 6.1 Scenario 1 . . . 35 6.2 Scenario 2 . . . 36

6.3 Validity of the Results . . . 37

6.3.1 Construct Validity . . . 37

6.3.2 Internal Validity . . . 37

6.3.3 Conclusion Validity . . . 38

6.4 Sustainability and Ethics . . . 38

7 Conclusions 40 7.1 Future Research . . . 40

Bibliography 41

(10)

(11)

Chapter 1 Introduction

1.1 Background

Internet of Things (IoT) is a constantly growing market [1] that consists of many different applications and devices. This entails that there are multitude of communication protocols used to communicate with all of these different types of IoT devices. Two of the most common communication protocols within IoT are MQTT (Message Queuing Telemetry Transport) and WebSocket.

IoT is a term coined back in 1999, but has grown in interest in the past decade. This is due to the growth of cloud computing. The main idea of cloud com-puting is that someone else is responsible for building and maintaining a large infrastructure of databases, that can be used for tasks such as large computa-tions. This allows for anyone to process large computations using these cloud infrastructures. The past couple of years, companies such as Amazon, Google and Microsoft have also released a large amount of different cloud services. These cloud services simplify the process of making use of the cloud com-puting infrastructure. This has led to different cloud models being defined [2]. Some of these are, infrastructure as a service (IaaS), software as a service (SaaS) and platform as a service (PaaS). Currently, not many research papers investigate the scalability of the communication protocols MQTT and Web-Socket. There is also a lack of studies looking at which cloud model would best fit an MQTT/WebSocket based application. Determining how MQTT compares to WebSocket, when looking at scalability, reliability and fault tol-erance, is of great interest as IoT and cloud computing continues to grow. It is also of importance to figure out which cloud models work best, and what advantages/disadvantages the cloud model imposes.

(12)

2 CHAPTER 1. INTRODUCTION

Figure 1.1: An overview of a smaller part of Cepheid’s C360 platform.

1.2 Research Question

Which one of the protocols between MQTT and WebSocket performs the best with regards to scalability.

1.3 Principals

By the end of this year, Cepheid is ending their contract with their current server provider, where their platform is hosted. Cepheid is now looking to move their platform to Amazon Web Services (AWS). The company is there-fore interested researching the different services Amazon offers, and what ad-vantages and disadad-vantages these services introduce. Cepheid is also inter-ested in figuring out what scalability possibilities AWS offer. As seen in Fig-ure 1.1, a part of their current platform deals with a large amount of persistent WebSocket connections with no possibility to horizontally scale the number of receivers.

1.4 Research Methodology

(13)

CHAPTER 1. INTRODUCTION 3

1. How can the current issue be solved?

2. How can a technique/product be developed to solve the problem in the most efficient way?

3. What basis/information is available and what are the requirements to develop a technique/product?

4. Develop the technique/product based on the bases/information gathered in step 3. If the technique/product is satisfactory, go to step 6.

5. Try creating a new technique/product.

6. Create a model/simulation with the suggested technique/product. 7. What are the consequences of the model/simulation from step 6? 8. Test the implementation of the model/simulation. If the results are not

satisfactory go to step 9 otherwise, continue to step 10.

9. Identify and correct the following flaws with the model/simulation. 10. Evaluate the results in relation to existing knowledge and practice, as

well as identify new problem areas for further research.

1.5 Scope and limitations

(14)

4 CHAPTER 1. INTRODUCTION

(15)

CHAPTER 1. INTRODUCTION 5

1.6 Disposition

This thesis is structured as followed: • Chapter 2 - Theoretical Background • Chapter 3 - Related Work

(16)

Chapter 2 Theoretical Background

This chapter covers the theoretical background of this degree project. The chapter starts with an introduction to cloud computing. Then comes the tech-nical background of the communication protocols, publish subscribe pattern and scalability. Finally, the technical background of the Amazon Web Services used in the degree project is explained.

2.1 Cloud Computing

According to the National Institute of Standards and Technology (NIST), “Cloud computing is a model for enabling ubiquitous, convenient, on-demand work access to a shared pool of configurable computing resources (e.g., net-works, servers, storage, applications and services) that can be rapidly pro-visioned and released with minimal management effort or service providerin-teraction” [2]. NIST has also identified five key aspects of cloud computing; On-demand self-service computing power, Broad network access, Resource pooling (computing resources can be adjusted to suit each client’s need), Rapid elasticity (the possibility to automatically scale and provisions computing re-sources) and Measured service (the possibility to measure resource usage).

2.1.1 Cloud Models

Within cloud computing there exist three different types of cloud computing models. Each of these models provide a different degree of control, manage-ment and flexibility.

(17)

CHAPTER 2. THEORETICAL BACKGROUND 7

• Infrastructure-as-a-service (IaaS): As defined by NIST, allows the user [2] “... provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbi-trary software, which can include operating systems and applications”. However, the user does not have control over the underlying cloud in-frastructure.

• Platform-as-a-Service (PaaS): Compared to IaaS, PaaS allows users to "... deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the cloud computing provider" [2]. This means that the user does not have control over the actual hardware running the applications.

• Software-as-a-Service (SaaS): This is about providing users with al-ready completed cloud applications, which are managed by the cloud service provider [2]. This leads to the user being more restricted on what they can control and change compared to the other cloud models.

2.2 Communication Protocols

Internet of Things (IoT) consists of many different applications and devices. This entails that there are a multitude of communication protocols used to communicate with all of these different types of IoT devices. Two of the more common communication protocols are WebSocket and MQTT.

2.2.1 WebSocket

(18)

8 CHAPTER 2. THEORETICAL BACKGROUND

WebSocket is located at layer 7 in the OSI [6] model and depend on TCP at layer 4 and uses a custom binary framing format, which splits each appli-cation message into one or more frames, transports them to the destination, reassembles them, and finally notifies the receiver once the entire message has been received. [7]

2.2.2 MQTT

MQTT (Message Queuing Telemetry Transport) is a communication protocol that makes use of the publish-subscribe pattern when sending messages (data) between client and server. MQTT is standardized by the technical committee of the Organization for the Advancement of Structured Information Standards (OASIS) MQTT runs over TCP/IP. MQTT can be supported by any network protocol that provides ordered, lossless, bi-directional connections [8]. The main usage of MQTT is within Machine-to-Machine communication, and it is designed to be used where network bandwidth is limited. MQTT is located at layer 4 in the OSI model and does not ensure that frames of the message is received in the correct order or reassembles of the frames.

MQTT offers three Quality of Services (QoS) levels for data delivery [9]: • QoS 0 (At most once): This service level sends the message once. Where

the sender does not store the message for possible re-transmission, and does not wait for an acknowledgement message from the receiver. This results in the message having the same guarantees as the underlying TCP protocol.

• QoS 1 (At least once): This ensures that the sent message is delivered at least one time to the receiver. The message is stored by the sender un-til an acknowledgement message is sent from the receiver. The sender resends the message if an acknowledgement message is not received within a certain time frame. This makes it possible that the receiver gets multiple duplicates of the same message.

(19)

2.2.3 Differences between MQTT and WebSocket

There exists a few differences between MQTT and WebSocket, that affect the scalability possibilities of the communication protocols. MQTT was specifi-cally designed for IoT devices with the aim to reduce bandwidth and provide assurance of delivery with its QoS. WebSocket was mainly developed for full bidirectional communication.

MQTT uses only the OSI layer 4 while WebSocket also makes use of the OSI layer 7. This enables WebSocket to ensure that messages that are split up in frames are put back together in the correct order when received by the receiver. MQTT does not offer this and the receiver has to ensure that the message is put back together correctly. However, MQTT has a much smaller overhead for each message sent compared to WebSocket. Leading to WebSocket creating a lot more overhead when a large amount of IoT devices communicate with each other.

MQTT was also designed for the publish-subscribe pattern and is majorly used in small devices, while WebSocket mainly works in an environment that sup-ports the publish-subscribe pattern. Meaning that WebSocket needs extra con-figurations/resources to operate a publish-subscribe pattern.

2.3 Publish-Subscribe Pattern

The publish-subscribe pattern consists of three parts; publishers, subscribers and a message broker. Messages are exchanged between the publishers and subscribers through the message broker. This is done by the use of topics, where subscribers specify which topics they want to listen to and publishers publish to those specific topics. The message broker filters and distributes messages according to the topic and identifier each message has. An example of the pattern is shown in Figure 2.1.

(20)

Figure 2.1: Subscriber only receives messages from the broker which it has subscribed to.

2.4 Scalability

One of the biggest features with cloud computing is the ability to scale one’s services with regard to the current demand from customers. There exist two types of scaling; vertical and horizontal [11].

• Vertical Scaling: This is about adding more power to a single instance of one’s application, by improving that instances hardware, CPU, IO, memory, etc. Vertical scalability is often the easiest way to improve scalability, since it does not require any change to the actual application and code. However, this does not mean the performance of the applica-tion will not be changing linearly with the improved hardware. It also does not offer the same fault tolerance as horizontal scaling does. • Horizontal Scaling: Instead of improving a single instance,

horizon-tal scaling is about creating additional duplicates of the same instance, which in theory should allow for almost infinite scaling. This requires additional configuration to the pre-existing application/code, to ensure that all the instances work together. This offers much better fault toler-ance. For example, if one of the instances fails, the application can still run due to the other instances.

2.5 Amazon Web Services

(21)

Figure 2.2: Structure of a AWS region, where availability zones are physically separated.

2.5.1 AWS Cloud computing infrastructure

The AWS cloud computing infrastructure is split up in multiple different re-gions that are spread across the world. An example of a region is "eu-north-1" located in Stockholm. Each region consists of two or more availability zones. An availability zone is a collection of data centers. To ensure fault tol-erance, each availability zone is physically separated from each other, as can bee seen in Figure 2.2. However, all availability zones are connected through low-latency links. Some regions even have edge locations to lower the latency even more.

2.5.2 AWS Elastic Compute Cloud

Elastic Compute Cloud (EC2) provides scalable computing capacity in the form of instances. EC2 instances are virtual servers that replace the need for on-site physical hardware and provide a virtual computing environment [12]. Users do not have any set limitations on the number of instances that they can launch, making it possible to scale up or down the number of instances. This makes it possible to start up an large number of instances during high traffic and remove instances during low traffic.

(22)

additional software.

2.5.3 AWS Elastic Load Balancer

Elastic Load Balancer (ELB) is a service that is used to evenly distribute in-coming traffic across EC2 instances, that are located in the same region. It is also possible to distribute traffic amongst multiple availability zones. There exist three different versions of elastic load balancers. Application load bal-ancer (ALB), network load balbal-ancer (NLB) and classic load balbal-ancer (CLB).

• ALB: Decides its routes based on layer 7 of the Open Systems Inter-connection (OSI). ALB allows HTTP/HTTPS traffic, but also supports WebSocket and HTTP/2.

• NLB: Compared to ALB, NLB decides its routing based on layer 4 of the OSI model. This includes procols such as TCP, TLS and UDP. • CLB: This is an older version, that allows HTTP, HTTPS and TCP

traf-fic.

2.5.4 AWS CloudWatch

AWS CloudWatch is a service that is used to monitor AWS resources, as well as applications ran on AWS. CloudWatch offers real-time monitoring of AWS resources, such as EC2. This can be used to find bottlenecks, pinpoint re-sources in high demand, improve performance and much more. For example, CloudWatch automatically provides metrics for Network usage, CPU utiliza-tion, latency and request counts.

(23)

Chapter 3 Related Work

As the Internet of Things (IoT) rapidly progresses, the range of applications has expanded in diverse fields. With this comes a wide range of different com-munication protocols in IoT. Two of the main comcom-munication protocols in IoT are MQTT and WebSocket. There have been studies about MQTT compati-bility with different IoT devices, different WebSocket architectures and how MQTT compares to WebSocket in response time and performance. However, there are no studies comparing scalability between MQTT and WebSockets. Also, there are no studies about which cloud model (SaaS, IaaS and PaaS) best fits a highly scalable application. This chapter introduces research related to this degree project, split up in three categories, IoT using MQTT, IoT using WebSocket and Comparison of MQTT and WebSocket.

3.1 IoT using MQTT

As MQTT only has one type of architecture (a MQTT broker), most papers focusing on MQTT only research about MQTT’s compatibility with different IoT devices or compare the performance of different MQTT broker providers. A study by Kang et al. [13] implements a MQTT broker on AWS using EC2. Another study by Jaya and Hossain [14] implemented a MQTT broker by us-ing AWS IoT core. However, the study conducted by Lohokare et al. [15] does look into implementing a cluster of MQTT brokers to handle a large amount of concurrent connections to support high scalability.

Even though, the study done by Lohokare et al. [15] focuses on scalability, no quantitative data was collected to confirm the scalability when using a cluster of MQTT brokers. This study will implement a MQTT broker using AWS IoT

(24)

14 CHAPTER 3. RELATED WORK

core but instead of looking into compatibility the main focus will be scalability.

3.2 IoT using WebSocket

The papers that only focus on the WebSocket protocol all look into differ-ent WebSocket architectures, and makes use of AWS. Kawazoe et al. [16] conducted a study where two different WebSocket architectures, mediation and forwarding, were tested. Mediation ensures that clients that communicate with each other connect to the same WebSocket server. In contrast to forward-ing, where communicating clients connects through two different WebSocket servers, that in turn are connected to each other. The paper also made use of AWS ELB, AWS autoscaling and AWS EC2. The focus of the paper was to test how the different architectures performed while handling a large amount of concurrent connections. To test this, a simulated environment was set up where 100 000 clients connected to the WebSocket servers at once. It was confirmed that the mediation architecture performed the best with regards to scalability.

Another study by Puripunpinyo and Samadzadeh [17] tested different scalable Push architectures. It was found that increasing the number of Web-Socket connections (1 to 4) did not make any difference compared to the choice of architecture used. Ganaputra and Paradamean [18] did a similar study where they implemented a publish-subscriber architecture. First, by only using WebSockets and another architecture using WebSockets together with AWS Simple Que Service. The goal of the study was to find which archi-tecture led to the least data loss. It was found that using AWS SQS together with WebSockets decreased the message loss but at the cost of a higher band-with usage as well as a higher response time.

This study will take a similar approach as the Kawazoe [16] study, but the focus will be on how well MQTT scales compared to WebSocket. This will also look into, at a theoretical level, which cloud model (SaaS, IaaS, PaaS) works best for a WebSocket architecture.

3.3 Comparison of MQTT and WebSocket

(25)

CHAPTER 3. RELATED WORK 15

and WebSocket for different payload sizes and found that the difference in RTT between the protocols is quite small, but that the WebSocket protocol is more suitable for applications which have a RTT of at least 1 ms. Similarly to Oliveira, the paper written by Silva et al. [20] compares the RTT for MQTT and WebSocket using servers between Italy and Brazil, where barely any dif-ferences where found even for varying sizes of payload.

The paper written by Kayal and Perros [21], compares how different com-munication protocols such as CoAP, MQTT and WebSocket perform during different server utilization (the amount of time the server worked). It was found that the WebSocket protocols mean response time stayed close to the same at different server utilization percentages while for MQTT it increased. Another paper by Sueda et al. [22] tested how lightweight MQTT actually is compared to other protocols such as WebSocket and HTTP. It was found that depending on topic complexity used in MQTT, it could surpass the WebSocket protocols overhead resulting in MQTT using more bandwidth than WebSocket.

(26)

Chapter 4 Method

This chapter presents the methodology used in this degree project. First the two different applications are described, which are used to analyze the Web-Socket and MQTT protocol. Then the different experimental processes will be described.

4.1 Brokers

To evaluate the scalability of MQTT and WebSocket, two message brokers were tested. One making use of MQTT and another using WebSocket. When deciding on what software to use for the brokers, two main factors were con-sidered. First, possibility to run on EC2, and, second the possibility to create a cluster with multiple brokers to achieve horizontal scaling.

All brokers ran were hosted on t3.xlarge instances, and ran in the Stockholm region (eu-north-1). The T3 instances feature either the 1st or 2nd genera-tion Intel Xeon Platinum 8000 series processor (Skylake-SP or Cascade Lake) [23]. The T3 instance consists of two cores and four virtual CPUs [24]. More details about the specification of the broker instances can be found in table 4.1.

4.1.1 MQTT Broker

For the MQTT broker it was decided to use EMQX [25], which is an open source MQTT message broker that is based on Erlang/OTP, and offers the possibility to create a cluster with other EMQX brokers. The free version of EMQX allows up to 100 000 concurrent connections and runs on most OSes,

(27)

CHAPTER 4. METHOD 17

Cloud Infrastructure AWS (eu-north-1) AWS (us-west-1)

Load Balancer ALB (WebSocket broker) NLB (MQTT broker) EC2 Instance Type: t3.xlarge Cores: 2 (4 vCPU) Memory: 16 (GB) OS: Ubuntu 18.04

Table 4.1: Specifications of system components.

making it compatible with EC2.

As seen in Figure 4.1, the MQTT broker system, consist of two parts, AWS NLB and EC2 instances. The network load balancer is configured to only lis-ten to port 1883, and ensures that the network traffic is evenly split between the EC2 instances.

Figure 4.1: The MQTT broker consists of multiple EC2 instances running EMQX, which are clustered together.

(28)

18 CHAPTER 4. METHOD

Then, to join a cluster, the command "emqx_ctl cluster join" is used, combined with specifying another EMQX node to join a cluster with.

4.2 WebSocket Broker

For the WebSocket broker, it was decided to program it in Javascript using Node.js [27] in combination with Redis [28]. Node.js is an open-source envi-ronment that executes Javascript code, and is most commonly used to create web servers and networking tools. Together with Node.js comes the package manager NPM [29]. NPM contains a large number of Javascript packages and helps resolve different packages dependencies.

For this study, the NPM package Socket.IO [30] was used to create the Web-Socket broker. Web-Socket.IO makes use of namespaces and rooms. Namespaces allows assigning sockets different endpoints or paths. This study made use of the default namespace "/", which is the namspace Socket.IO clients by default connect to. Within a namespace it is possible to define arbitrary channels that sockets can join and leave. This was used to create a pub/sub architecture, where subscribers join a specific channel, and publishers publish to a specific channel. This ensures that only subscribers that have joined that channel re-ceives the publishers message.

Redis is an open-source software, that can be used as a database, cache or even a message broker. Redis also supports a wide arrange of different data structures such as strings, hashes, lists sets and many more [28]. In this study, Redis was used to link together the EC2 instances hosting the WebSocket bro-ker, as seen in Figure 4.2. For this to work, the WebSocket broker needed to be configure to make use of the NPM package "Socket.IO-redis" [31], which is an socket.io adapter. That ensures that a publisher message will be broad-cast to every subscriber that is subscribed to that specific channel. This is done through the Pub/Sub mechanism of Redis. The code for the WebSocket broker can be found at the GitHub repository https://github.com/marcusWerKTH/kth-master-degree-project.

4.3 Experimental Process

(29)

CHAPTER 4. METHOD 19

Figure 4.2: The WebSocket broker consists of multiple EC2 instances running Socket.IO where Redis is used to enable clustering of the EC2 instances.

and few publisher. The publishers and subscribers for MQTT were programmed in Python 3.8 and used the package paho-mqtt [32]. While the publishers and subscribers for WebSocket were programmed in Javascript and used the NPM package Socket.io-client [33]. The code for the publishers and subscribers can be found in the GitHub repository https://github.com/marcusWerKTH/kth-master-degree-project.

The publishers and subscribers where hosted on t2.xlarge instance, which have the same specifications as the t3.xlarge. All publishers and subscribers were hosted in the Northern California region (us-west-1).

(30)

20 CHAPTER 4. METHOD

Figure 4.3: The sequence for the message sent by the publisher.

4.4 Data collection

(31)

Chapter 5 Results

This chapter first presents results obtained when performing simulation of a large number of publishers and few subscribers for both the MQTT and Web-Socket brokers. Then the results when simulating a large number of sub-scribers with a few publishers are presented. These results are then presented in graphs, where trend lines were plotted for each graph to be able to compare MQTT to WebSocket.

5.1 Scenario 1

(32)

22 CHAPTER 5. RESULTS

• 10000 publishers, 300 subscribers All publishers sent one message per second.

5.1.1 MQTT Brokers

The amount of memory each MQTT Broker uses for a specific amount of publishers and subscribers is shown in Table 5.1. Without any publishers or subscribers connected to the brokers, broker #1 had a base memory usage of 620 MB, and Broker #2 656 MB. Figure 5.1 shows the total memory usage for the cluster of brokers with N number of connections.

1,000 3,000 5,000 7,000 10,000 1 1.5 2 2.5 3

Total number of Connections (N)

Memor y Usag e (GB) y(x) 9.98 · 10−5· x + 1.27

(33)

CHAPTER 5. RESULTS 23

In Table 5.2 the total network usage (received and sent network traffic) for each broker is presented, and is measured in MB/s. With no publishers or subscribers connected, the network usage is zero. Figure 5.5 shows the total network usage for the cluster of brokers with N number of connections.

1,000 3,000 5,000 7,000 10,000 0

10 20 30

N etw ork Usag e (MB/s) y(x) 1.75 · 10−3· x + 4.73 · 10−2

Figure 5.2: The above graph illustrates the network usage in MB/s for specific numbers of connections. Connected Publishers/Subscribers 0/0 100/10 1000/50 3000/ 100 5000/ 150 7000/ 200 10000/ 300 Network Usage (MB/s) MQTT Broker #1 0 0.1 0.8 2.6 4.7 6.5 8.8 MQTT Broker #2 0 0.1 0.9 2.9 4.6 6.5 8.9 Total: 0 0.2 1.7 5.5 9.3 13.0 17.7

Table 5.2: The amount of network traffic received and sent for each broker for specific num-bers of connections.

(34)

24 CHAPTER 5. RESULTS 1,000 3,000 5,000 7,000 10,000 0 1 2 3

CPU

cores

used

y(x)

2.67 · 10−4· x + 2.72 · 10−2

Figure 5.3: The above graph illustrates the total number of cores used for specific numbers of connections.

Connected Publishers/Subscribers (P/S) 0/0 100/10 1000/50 3000/ 100 5000/ 150 7000/ 200 10000/ 300 CPU Utilization (%) MQTT Broker #1 0 1 6.6 19.3 39.8 47.3 67.9 MQTT Broker #2 0 1 7.4 22.5 37.8 51.9 66.9 Total (Cores): 0 0.04 0.28 0.836 1.552 1.984 2.696 Table 5.3: The CPU utilization percentage needed for each broker for specific numbers of connections.

In Table 5.4 the average round trip time (RTT) for messages sent by pub-lishers is presented and is measured in ms. The amount of messages sent and received by the publishers were recorded to measure message loss. The mes-sage loss for the MQTT brokers was close to zero (0.001%).

Number of Publishers/Subscribers (P/S) 100/10 1000/50 3000/ 100 5000/ 150 7000/ 200 10000/ 300 RTT (ms) 391.7 374.7 361.4 346.1 397.2 367.7

(35)

5.1.2 WebSocket Broker

The cluster of WebSocket brokers require a Redis instance to be functional. Table 5.5 presents the memory usage for both WebSocket brokers and Redis instance. Without any publishers or subscribers connected to the brokers, bro-ker #1 had a base memory usage of 1037 MB, brobro-ker #2 1038 MB and Redis 430 MB. Figure 5.4 shows the total memory usage in GB for the cluster of brokers and Redis instance.

1,000 3,000 5,000 7,000 10,000 1 1.5 2 2.5 3

Figure 5.4: The above graph illustrates the memory usage in GB for specific numbers of connections. Number of Publishers/Subscribers (P/S) 0 100/10 1000/50 3000/ 100 5000/ 150 7000/ 200 10000/ 300 Memory Usage (MB) WebSocket Broker #1 1037 1040 1089 1151 1289 1335 1380 WebSocket Broker #2 1038 1039 1080 1169 1289 1316 1394 Redis Instance 430 433 431 432 434 432 432 Total: 2505 2512 2600 2752 3012 3083 3206

(36)

26 CHAPTER 5. RESULTS

Table 5.6 presents the total network usage (received and sent network traf-fic) for each broker and Redis instance, and is measured in MB/s. With no publishers or subscribers connected, the network traffic usage is zero. Figure 5.5 shows the total network usage for the cluster of brokers and Redis instance with N number of connections.

1,000 3,000 5,000 7,000 10,000 0

10 20 30

N etw ork Usag e (MB/s) y(x) 4.45 · 10−3· x + 0.81

Figure 5.5: The above graph illustrates the network usage in MB/s for specific numbers of connections. Number of Publishers/Subscribers (P/S) 0 100/10 1000/50 3000/ 100 5000/ 150 7000/ 200 10000/ 300 Network Usage (MB/s) WebSocket Broker #1 0 0.1 1.6 4.6 7.3 8.9 9.5 WebSocket Broker #2 0 0.2 1.6 4.7 7.3 8.9 9.2 Redis Instance 0 0.2 2.4 7.0 10.9 13.0 13.5 Total: 0 0.5 5.6 16.3 25.5 30.8 32.2

(37)

Table 5.7 contains the CPU utilization for each WebSocket broker and Re-dis instance. Both brokers and ReRe-dis instance has two cores. The total number of CPU cores needed to run the WebSocket cluster is shown in Figure 5.6.

1,000 3,000 5,000 7,000 10,000 0

1 2 3

CPU cores used (C) y(x) 2.5 · 10−4· x + 6.93 · 10−2

Number of Publishers/Subscribers (P/S) 0 100/10 1000/50 3000/ 100 5000/ 150 7000/ 200 10000/ 300 CPU Utilization (%) WebSocket Broker #1 0 0.1 6.3 18.9 28 32.1 31 WebSocket Broker #2 0 0.2 6 21 29 31.9 31.5 Redis Instance 0 0 4 13.9 16.5 20.6 22 Total (Cores): 0 0.006 0.326 1.076 1.47 1.692 1.69

Table 5.7: The CPU utilization percentage needed for each broker and Redis instance for specific numbers of connections.

(38)

28 CHAPTER 5. RESULTS Number of Publishers/Subscribers (P/S) 100/10 1000/50 3000/ 100 5000/ 150 7000/ 200 10000/ 300 RTT (ms) 342.4 326.0 327.8 356.3 539.5 872.0

Table 5.8: The Round trip time for specific numbers of connections.

5.2 Scenario 2

Six simulations were conducted for the MQTT and WebSocket brokers were each simulation had an increase in subscribers and publishers. Each simulation was conducted for fifteen minutes. All values in the tables are the average over an eight minute peak of the simulation. The results presented in this section show the average memory usage, network usage and CPU utilization, both for the MQTT and WebSocket brokers. The number of publishers and subscribers for each simulation, were as follows:

• 10 publishers, 100 subscribers • 50 publishers, 1000 subscribers • 100 publishers, 3000 subscribers • 150 publishers, 5000 subscribers • 200 publishers, 7000 subscribers • 300 publishers, 10000 subscribers All publishers sent one message per second.

5.2.1 MQTT Broker

(39)

CHAPTER 5. RESULTS 29 1,000 3,000 5,000 7,000 10,000 1 1.5 2 2.5 3

Figure 5.7: The above graph illustrates the memory usage in GB for specific numbers of connections. Number of Publishers/Subscribers (P/S) 0/0 10/100 50/1000 100/ 3000 150/ 5000 200/ 7000 300/ 10000 Memory Usage (MB) MQTT Broker #1 625 630 661 676 746 756 835 MQTT Broker #2 668 672 694 727 787 804 866 Total: 1293 1302 1355 1403 1533 1560 1701

Table 5.9: The memory usage for each broker instance for specific numbers of connections.

(40)

30 CHAPTER 5. RESULTS 1,000 3,000 5,000 7,000 10,000 0 500 1,000 1,500

N etw ork Usag e (KB/s) y(x) 5.81 · 10−2· x + 13.38

Figure 5.8: The above graph illustrates the network usage in KB/s for specific numbers of connections. Number of Publishers/Subscribers (P/S) 0/0 10/100 50/1000 100/ 3000 150/ 5000 200/ 7000 300/ 10000 Network Usage (KB/s) MQTT Broker #1 0 11.7 52.8 88.3 159.4 212.4 305.2 MQTT Broker #2 0 13.0 52.0 86.2 152.1 201.7 321.3 Total: 0 24.7 104.8 174.5 311.5 414.1 626.5

Table 5.10: The amount of network traffic received and sent for each broker for specific num-bers of connections.

(41)

CHAPTER 5. RESULTS 31 1,000 3,000 5,000 7,000 10,000 0 5 · 10−2 0.1 0.15 0.2 0.25

CPU

cores

(C)

y(x)

2.18 · 10−5· x + 2.91 · 10−3

Number of Publishers/Subscribers (P/S) 0/0 10/100 50/1000 100/ 3000 150/ 5000 200/ 7000 300/ 10000 CPU Utilization (%) MQTT Broker #1 0 0 1 1.6 3.0 4.0 5.6 MQTT Broker #2 0 0 1 1.6 2.9 3.8 5.9 Total (Cores): 0 0 0.04 0.064 0.118 0.156 0.23 Table 5.11: The CPU utilization percentage needed for each broker for specific numbers of connections.

5.2.2 WebSocket Broker

(42)

32 CHAPTER 5. RESULTS 1,000 3,000 5,000 7,000 10,000 1 1.5 2 2.5 3

Figure 5.10: The above graph illustrates the memory usage in GB for specific numbers of connections. Number of Publishers/Subscribers (P/S) 0/0 10/100 50/1000 100/ 3000 150/ 5000 200/ 7000 300/ 10000 Memory Usage (MB) WebSocket Broker #1 1033 1036 1051 1106 1141 1170 1202 WebSocket Broker #2 1029 1030 1051 1103 1141 1166 1206 Redis Instance 431 436 431 431 428 429 435 Total: 2492 2502 2533 2640 2710 2762 2843 Table 5.12: The memory usage for each broker for specific numbers of connections.

(43)

CHAPTER 5. RESULTS 33 1,000 3,000 5,000 7,000 10,000 0 500 1,000 1,500

N etw ork Usag e (KB/s) y(x) 0.17 · x + 46.54

Figure 5.11: The above graph illustrates the network usage in KB/s for specific numbers of connections. Number of Publishers/Subscribers (P/S) 0/0 10/100 50/1000 100/ 3000 150/ 5000 200/ 7000 300/ 10000 Network Usage (KB/s) WebSocket Broker #1 0 19.7 88.6 180.1 235.4 364.0 543.5 WebSocket Broker #2 0 16.7 76.0 179.7 272.5 365.6 542.5 Redis Instance 0 25.6 123.3 245.5 365.1 488.6 729.7 Total: 0 62.0 287.9 605.3 873.0 1218.2 1815.7 Table 5.13: The amount of network traffic received and sent for each broker and Redis instance for specific numbers of connections.

(44)

34 CHAPTER 5. RESULTS 1,000 3,000 5,000 7,000 10,000 0 5 · 10−2 0.1 0.15 0.2 0.25

CPU

cores

(C)

y(x)

1.45 · 10−5· x − 6.78 · 10−3

(45)

Chapter 6 Discussion

This study aims to answer the research question mentioned in Section 1.2. The following sections are detailed discussion of the results (memory usage, network usage, CPU core usage, RTT and message loss) presented in Chapter 5. Then a validity discussion and ethical concerns regarding cloud computing are presented. Lastly, ideas for future research are proposed.

6.1 Scenario 1

For the scenario with many publishers, but few subscribers, When looking at memory usage for the cluster of MQTT brokers in Figure 5.1, the trend line slope is steeper than WebSockets trend line in Figure 5.4. The trend line for the cluster of MQTT brokers would be even steeper if QoS 1 and 2 were used, instead of just QoS 0. When QoS1 and 2 are used, the messages are stored until an acknowledgement message is received.

Important to note is that for the WebSocket cluster the 10 000 publisher data point was not considered when plotting the trend line for memory usage, net-work usage and CPU usage, due to the graph starting to resemble a logarithmic growth, indicating that something went wrong during simulation. When look-ing at the amount of messages sent by the publisher (Appendix A), it is clear that not near enough messages were sent from the publishers than what was expected. Most likely to Node.js being single threaded, so when simulating a lot of publishers on a single EC2 instance each publisher has to wait a long while before getting time to execute. As well as the Round Trip Time (RTT) (Table 5.8) drastically increasing. Showing that the instance simulating the publishers, could not handle the load created by the publishers.

(46)

36 CHAPTER 6. DISCUSSION

Comparing the network usage trend lines for the MQTT and WebSocket cluster (Figure 5.2 and Figure 5.5), the WebSocket clusters trend line has a five times steeper slope than the MQTT cluster. This is most likely due to the amount of messages needed to be sent between the WebSocket brokers and the Redis instance. As mentioned earlier, only QoS 0 was used when testing the MQTT brokers leading, to a lower network usage than if QoS 1 or 2 was used. In Figure 5.3 and Figure 5.6, the trend lines for the number of cores used for the MQTT and WebSocket cluster are shown. As shown, the trend line for MQTT is slightly steeper than the WebSocket trend line. This is surprising considering the amount of extra messages that is needed to be sent between the brokers and Redis instance. It could be argued that this is not a big difference, but on a larger scale, this could be the difference between thousands of con-nections. Finally when looking at the RTT between MQTT and WebSocket (Table 5.4 and Table 5.8), MQTT stays around 350-390 ms while WebSocket rapidly starts growing. However, it is difficult to draw any real conclusion from this data, as the instance simulating the WebSocket publishers had a hard time handling the load for larger amounts of publishers.

6.2 Scenario 2

In the next scenario, with many subscriber, but few publishers, all results gath-ered were used when plotting the trend line for memory usage, network usage and CPU usage due to there being a lot less publishers compared to the previ-ous scenario. Fewer publishers made it possible for the instances hosting the publishers to handle the load created by the publishers. Similar to the first sce-nario with few subscriber and many publishers, comparing the memory usage for the MQTT and WebSocket cluster (Figure 5.7 and Figure 5.10), MQTT has a steeper trend line than WebSocket.

(47)

CHAPTER 6. DISCUSSION 37

Surprisingly, there was a big difference between the RTT for MQTT and Web-Socket for this second scenario. MQTT had an average RTT of 163 ms and WebSocket had an average of 331 ms. In the previous scenario, the RTT stayed fairly the same for both the protocols. It is hard to explain why the RTT was 391 ms when there were 100 publishers and subscribers in the first scenario, since it is expect to be closer to the RTT for this second scenario.

6.3 Validity of the Results

Many factors can affect the validity of the results gathered in a study. This section will look at the construct, internal and conclusion validity [35].

6.3.1 Construct Validity

Construct validity [35] refers to how well an experiment measures what it was intended to measure. The simulations done in this study intended to com-pared the scalability of MQTT and WebSocket. To measure this, the mem-ory usage, network usage, CPU utilization, RTT and message loss were ana-lyzed. These are metrics commonly used when measuring the scalability of an application/system/protocol. Other studies mentioned in the related works chapter have also looked into these parameters when comparing MQTT and WebSocket. This strengthens the validity that the results gathered in this study measure the scalability of MQTT and WebSocket. Something which is an im-portant factor when measuring scalability, but was not looked into was the fault tolerance for the different protocols.

6.3.2 Internal Validity

(48)

38 CHAPTER 6. DISCUSSION

hosted in the Northern California region and broker instances were hosted in the Stockholm region.

6.3.3 Conclusion Validity

Conclusion validity [35] refers to how certain one can be that the correct con-clusions are drawn from the results of the study. The greatest risk to the conclu-sions drawn, as mentioned earlier, is the limitations of the instances simulat-ing the publishers and subscribers, resultsimulat-ing in questionable results for larger amounts of connections. To minimize the chance of drawing incorrect con-clusions, the 10 000 publisher data point was disregarded for the WebSocket simulation with many publishers and few subscribers. Also, only the results from an 8 minute peak period, instead of the whole 15 minute simulation, was recorded. To compensate inconsistencies in the workload produced from the publishers, detailed monitoring was enabled for the EC2 instances. This meant that EC2 posted their own instance data to CloudWatch each minute instead of every five minutes.

When deciding on what software to use to run the different brokers, it was en-sured that it was compatible with EC2. To ensure that the results were valid, i.e. not affected by the type of application architecture, network latency etc. It was also ensured that the brokers, Redis and publisher/subscriber instances had the same EC2 specifications, so that the MQTT/WebSocket clusters had the same simulated environments.

6.4 Sustainability and Ethics

(49)

CHAPTER 6. DISCUSSION 39

Another paper identified advantages and disadvantages with cloud computing [37]. One of the main advantages was the business benefits, as cloud services provide dynamically scalable resources. Making it possible for companies to avoid having to invest in large infrastructures, also makes it possible for smaller companies to compete as they are not hardware restricted when using cloud computing. However, the paper does identify that ownership responsibilities become much more unclear when relying on a third party provider.

(50)

Chapter 7 Conclusions

The objective of this study was reached in that the results enable a better under-standing of different communication protocols’ scalability possibilities. The focus was on two of the more popular communication protocols, MQTT and WebSocket. It has been shown in this study that WebSocket works better when it comes to CPU usage and memory usage, in both scenarios (many publish-ers, few subscribers; many subscribpublish-ers, few publishers) simulated in this study. However, MQTT is far more superior when it comes to network usage, since it uses five times less network than WebSocket in the scenario with many pub-lishers and few subscribers.

7.1 Future Research

For future research it would be interesting to see how well MQTT and Web-Socket scale for an even larger amount of connections that frequently send messages. This study only investigated one out of many MQTT brokers that are offered on the market today [39], and switching to another MQTT broker may yield other results. It would also be interesting to investigate a more fully developed WebSocket broker, as the one in this study was self-developed and kept as simple as possible. There was no focus on fault tolerance in this study which would be interesting to investigate. Due to the difficulties of trying to simulate the publishers and subscribers in the simulations, there was no time to test any other cloud models for high scalability than the IaaS model. It would be interesting to investigate how well MQTT and WebSocket performs for other cloud models. For example, AWS IoT [40] and AWS Gateway [41], are services that offer high scalability using MQTT and WebSocket that follow the cloud model SaaS.

(51)

Bibliography

[1] Louis Columbus. 2018 Roundup Of Internet Of Things Forecasts And

Market Estimates. url: https://www.forbes.com/sites/

louiscolumbus/2018/12/13/2018-roundup-of-internet-of-things-forecasts-and-market-estimates/#1f6553387d83. (accessed: 03.01.2020).

[2] Peter Mell, Tim Grance, et al. “The NIST definition of cloud comput-ing”. In: (2011).

[3] Niclas Andersson and Anders Ekholm. Vetenskaplighet - Utvärdering

av tre implementeringsprojekt inom IT Bygg & Fastighet 2002. swe.

Tech. rep. Institutionen för Byggande och Arkitektur, Lunds Universitet,

2002. url: http://www.lth.se/fileadmin/projekteringsmetodik/ research/Other_projects/utvarderingver1.pdf.

[4] Mario Bunge. Epistemology & Methodology I:: Exploring the World. Vol. 5. Springer Science & Business Media, 2012.

[5] I. Fette. The WebSocket Protocol. url: https : / / tools . ietf . org/html/rfc6455. (accessed: 04.4.2020).

[6] J. D. Day and H. Zimmermann. “The OSI reference model”. In:

Pro-ceedings of the IEEE 71.12 (1983), pp. 1334–1340.

[7] Ilya Grigorik. WebSocket: BROWSER APIS AND PROTOCOLS,

CHAP-TER 17. url: https://hpbn.co/websocket/. (accessed: 05.07.2020).

[8] A. Banks et al. MQTT Version 5.0. url: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html. (accessed: 04.4.2020).

[9] HiveMQ Team. Quality of Service 0,1 & 2 - MQTT Essentials: Part 6. url: https://www.hivemq.com/blog/mqtt-essentials-part- 6- mqtt- quality- of- service- levels/. (accessed: 04.5.2020).

(52)

42 BIBLIOGRAPHY

[10] Patrick Th Eugster et al. “The many faces of publish/subscribe”. In:

ACM computing surveys (CSUR) 35.2 (2003), pp. 114–131.

[11] A. Polak. Horizontal scaling with WebSocket – tutorial. url: https: / / tsh . io / blog / how - to - scale - websocket/. (accessed: 04.5.2020).

[12] AWS. What is Amazon EC2? url: https://docs.aws.amazon. com / AWSEC2 / latest / UserGuide / concepts . html. (ac-cessed: 04.7.2020).

[13] D. Kang et al. “Room Temperature Control and Fire Alarm/Suppression IoT Service Using MQTT on AWS”. In: 2017 International Conference

on Platform Technology and Service (PlatCon). Feb. 2017, pp. 1–5.

[14] N. Imtiaz Jaya and M. F. Hossain. “A Prototype Air Flow Control Sys-tem for Home Automation Using MQTT Over Websocket in AWS IoT Core”. In: 2018 International Conference on Cyber-Enabled Distributed

Computing and Knowledge Discovery (CyberC). Oct. 2018, pp. 111–

1116. doi: 10.1109/CyberC.2018.00032.

[15] J. Lohokare et al. “Scalable tracking system for public buses using IoT technologies”. In: 2017 International Conference on Emerging Trends

Innovation in ICT (ICEI). Feb. 2017, pp. 104–109.

[16] H. Kawazoe, D. Ajitomi, and K. Minami. “Design and evaluation of large-scale and real-time remote control architectures for home appli-ances”. In: 2015 12th Annual IEEE Consumer Communications and

Networking Conference (CCNC). Jan. 2015, pp. 820–825. doi: 10 .

1109/CCNC.2015.7158083.

[17] H. Puripunpinyo and M. H. Samadzadeh. “Design, Prototype Imple-mentation, and Comparison of Scalable Web-Push Architectures on Ama-zon Web Services Using the Actor Model”. In: 2017 25th International

Conference on Systems Engineering (ICSEng). Aug. 2017, pp. 301–308.

doi: 10.1109/ICSEng.2017.42.

[18] Jastian Ganaputra and Bens Pardamean. “Asynchronous Publish/Subscribe Architecture over WebSocket for Building Real-time Web Applications”. In: Internetworking Indonesia Journal 7 (Dec. 2015).

[19] G. M. B. Oliveira et al. “Comparison Between MQTT and WebSocket Protocols for IoT Applications Using ESP8266”. In: 2018 Workshop

on Metrology for Industry 4.0 and IoT. Apr. 2018, pp. 236–241. doi:

(53)

BIBLIOGRAPHY 43

[20] D. R. C. Silva et al. “Latency evaluation for MQTT and WebSocket Protocols: an Industry 4.0 perspective”. In: 2018 IEEE Symposium on

Computers and Communications (ISCC). June 2018, pp. 01233–01238.

doi: 10.1109/ISCC.2018.8538692.

[21] P. Kayal and H. Perros. “A comparison of IoT application layer pro-tocols through a smart parking implementation”. In: 2017 20th

Con-ference on Innovations in Clouds, Internet and Networks (ICIN). Mar.

2017, pp. 331–336. doi: 10.1109/ICIN.2017.7899436.

[22] Y. Sueda, M. Sato, and K. Hasuike. “Evaluation of Message Protocols for IoT”. In: 2019 IEEE International Conference on Big Data, Cloud

Computing, Data Science Engineering (BCD). May 2019, pp. 172–175.

[23] Amazon. Amazon EC2 T3 Instances. url: https://aws.amazon. com/ec2/instance-types/t3/. (accessed: 05.12.2020). [24] Amazon. Physical Cores by Amazon EC2 Instance Type. url: https:

//aws.amazon.com/ec2/physicalcores/. (accessed: 05.12.2020). [25] EMQ. Get Started. url: https://docs.emqx.io/broker/v3/

en/getstarted.html#emq- x- r3- 2- message- broker-introduction. (accessed: 04.10.2020).

[26] EMQ. Turning guide. url: https://docs.emqx.io/broker/ latest/en/tutorial/turn.html. (accessed: 05.01.2020). [27] Node.js. About Node.js. url: https://nodejs.org/en/about/.

(accessed: 05.05.2020).

[28] Redis. Introduction to Redis. url: https://redis.io/topics/ introduction. (accessed: 04.7.2020).

[29] Npm. What is npm? url: https://www.npmjs.com/get-npm. (accessed: 05.05.2020).

[30] Socket.io. Socket.io. url: https://www.npmjs.com/package/ socket.io. (accessed: 05.05.2020).

[31] Socket.io. socket.io-redis. url: https://www.npmjs.com/package/ socket.io-redis. (accessed: 05.05.2020).

[32] R. Light. paho-mqtt 1.5.0. url: https://pypi.org/project/ paho-mqtt/. (accessed: 04.10.2020).

(54)

44 BIBLIOGRAPHY

[34] Amazon. Monitoring memory and disk metrics for Amazon EC2 Linux

instances. url: https://docs.aws.amazon.com/AWSEC2/

latest/UserGuide/mon-scripts.html. (accessed: 04.10.2020). [35] Apollo M. Nkwake. “Validity in Analysis, Interpretation, and

Conclu-sions”. In: Credibility, Validity, and Assumptions in Program

Evalua-tion Methodology. Cham: Springer InternaEvalua-tional Publishing, 2015, pp. 97–

110. isbn: 978-3-319-19021-1. doi: 10.1007/9783319190211 _ 6. url: https : / / doi . org / 10.1007/97833191902110 . 10.1007/9783319190211007 / 978 3 310.1007/97833191902119 -19021-1_6.

[36] Cloud Computing. “Sustainability: The Environmental Benefits of Mov-ing to the Cloud”. In: Accenture in collaboration with WSP

Environ-ment & Energy (2010).

[37] Tomayess Issa, Vanessa Chang, and Theodora Issa. “The impact of cloud computing and organizational sustainability”. In: Annual international

conference on Cloud Computing and visrtualization. CCV& GSTF. 2010,

pp. 163–169.

[38] J. Timmermans et al. “The Ethics of Cloud Computing: A Conceptual Review”. In: 2010 IEEE Second International Conference on Cloud

Computing Technology and Science. 2010, pp. 614–620.

[39] Y. Jain. Server support. url: https : / / github . com / mqtt / mqtt.github.io/wiki/server-support. (accessed: 04.19.2020). [40] Amazon. What is AWS IoT? url: https://docs.aws.amazon.

com/iot/latest/developerguide/what-is-aws-iot. html. (accessed: 05.01.2020).

(55)

Appendix A

Appendix

(56)

46 APPENDIX A. APPENDIX MQTT sent MQTT received WebSocket sent WebSocket received 100 publishers/ 10 subscribers 95900 95854 96000 96000 1000 publihsers/ 50 subscribers 959000 958916 940724 940724 3000 publishers/ 100 subscribers 2877000 2876783 2847084 2847084 5000 publishers/ 150 subscribers 3836058 3835564 4745015 4745015 7000 publishers/ 200 subscribers 6713005 6712285 5880359 5880359 10 000 publishers/ 300 subscribers 9580249 9578806 6256930 6256930 100 subscribers/ 10 publishers 9561 9560 9600 9600 1000 subscribers/ 50 publishers 47949 47901 48000 48000 3000 subscribers/ 100 publishers 95900 95860 96000 96000 5000 subscribers/ 150 publishers 143850 143836 144000 144000 7000 subscribers/ 200 publishers 191800 191740 192000 192000 10 000 subscribers/ 300 publishers 287400 287345 288000 288000

(57)

TRITA -EECS-EX-2020:434

Comparing the scalability of MQTT and WebSocket communication protocols using Amazon Web Services

Comparing the scalability of MQTT

and WebSocket communication

protocols using Amazon Web

Services

MARCUS WERLINDER

Comparing the scalability of

MQTT and WebSocket

communication protocols

using Amazon Web Services

MARCUS WERLINDER

Abstract

Sammanfattning

Acknowledgements

Contents

Chapter 1

Introduction

1.1

Background

1.2

Research Question

1.3

Principals

1.4

Research Methodology

1.5

Scope and limitations

1.6

Disposition

Chapter 2

Theoretical Background

2.1

Cloud Computing

2.1.1

Cloud Models

2.2

Communication Protocols

2.2.1

WebSocket

2.2.2

MQTT

2.2.3

Differences between MQTT and WebSocket

2.3

Publish-Subscribe Pattern

2.4

Scalability

2.5

Amazon Web Services

2.5.1

AWS Cloud computing infrastructure

2.5.2

AWS Elastic Compute Cloud

2.5.3

AWS Elastic Load Balancer

2.5.4

AWS CloudWatch

Chapter 3

Related Work

3.1

IoT using MQTT

3.2

IoT using WebSocket

3.3

Comparison of MQTT and WebSocket

Chapter 4

Method

4.1

Brokers

4.1.1

MQTT Broker

4.2

WebSocket Broker

4.3

Experimental Process

4.4

Data collection

Chapter 5

Results

5.1