Datateknik C, Examensarbete, 15 högskolepoäng
Preventing Vulnerabilities and Mitigating
Attacks on the MQTT Protocol
Yara Ahmad
Bachelor of Computer Engineering Program, 180HP Örebro Fall Semester 2020
Examiner: Franziska Klügl
Örebro universitet Örebro University Institutionen för School of Science and
naturvetenskap och teknik Technology
Abstract
The purpose of this paper is to explore and understand how security breaches can be prevented and mitigated in the MQTT protocol to increase its overall security. I am particularly interested in techniques such as Fuzzing, Fuzzy Logic, and Machine Learning. To examine this purpose, I analyzed and discussed previous research on implementation of Fuzzing, Fuzzy Logic and Machine Learning in a MQTT protocol. This analysis shows that Fuzzing is considered to be an efficient prevention method for the MQTT protocol, and that both Fuzzy Logic and Machine Learning are efficient methods for mitigation purposes. In conclusion, the security level of the MQTT protocol could be further increased by implementing methods for both prevention and mitigation purposes. For instance Fuzzing can be used for finding and correcting vulnerabilities while the MQTT protocol still is in development and thereby be used as a method for prevention. Thereafter, when the MQTT protocol is in production, implementation of either Fuzzy Logic or Machine learning can be used to mitigate sudden ongoing attacks that might occur. This means that the developer can combine methods for both prevention and mitigation to ensure a higher level of security in the MQTT protocol.
Keywords: MQTT, Fuzzing, Machine Learning, DoS, DDoS
Sammanfattning
Syftet med denna studie är att undersöka och förstå hur säkerhetsöverträdelser kan förhindras och mitigeras i ett MQTT protokoll för att öka den överliggande säkerheten. Jag är särskilt intresserad av tekniker såsom Fuzzing, Fuzzy Logic och Machine Learning..
För att undersöka syftet, analyserade och diskuterade jag tidigare implementationer av Fuzzing, Fuzzy Logic och Machine Learning, i ett MQTT protokoll. Analysen visade att Fuzzing ansågs vara en väldigt effektiv metod för att förhindra säkerhetsöverträdelser samt att både Fuzzy Logic och Machine Learning var effektiva metoder för mitigering.
Sammanfattningsvis, kan säkerhetsnivån i ett MQTT protokoll öka genom implementering av metoder som används i syfte att förhindra och mitigera säkerhetsöverträdelser. Exempelvis kan man först använda Fuzzing för att hitta och korrigera sårbarheter och därigenom förhindra dem. Därefter kan man antingen använda sig av Fuzzy Logic eller Machine Learning för att mitigera plötsliga attacker på MQTT protokollet när den är i produktion. Detta betyder att att utvecklaren kan kombinera metoder för att både förhindra och mitigera överträdelser i syfte att öka säkerhetsnivån i ett MQTT protokoll.
Acknowledgement
I would like to express my deepest gratitude towards all of my supervisors that have been guiding me, helping me while stuck and for always being honest with me when I am at fault. Without you I would not have been able to finish this paper, and without you I would still be stuck figuring out what kind of paper this would be.
Jennifer Renoux, not only are you my supervisor at Orebro University, but during these weeks, with tough love, also a friend. You never hesitated to tell me when something was wrong nor did you hesitate to tell me when something was good, this gave me motivation to finish this paper. “There are never dumb questions” and indeed, I was able to ask you anything knowing that you would guide me to the answer. Thank you!
Dennis Karlberg, thank you for believing in my project, I would not have been able to write this project without you. When I reached out to you I had no idea what my aim would be other than “IoT and security”, yet, you took a leap of faith and presented my idea to what would become my supervising company, Headlight.
Having Headlight as my supervising company I got one more supervisor who I am grateful to, Emil Drugge. You were the one who realized how lost I was at the beginning as I was not being sure what the difference is between an attack is and what a security measure is. I appreciate and thank you for how humble you were about it and how you were ready to give me more knowledge and guide me whenever I needed help.
Last but not least, I would like to thank Orebro University for giving me this opportunity to write my paper and for making it possible for me to grow in my role as a student and future computer engineer.
Abstract 2 Sammanfattning 2 Acknowledgement 3 1. Introduction 5 1.1 Background 5 1.2 Project 5 2. Theoretical Background 7 2.1 Internet of Things 7
2.2 Message Queuing Telemetry Transport 8
3. Security breaches and Security measures 11
3.1 Brute Force Authentication 11
3.2 Denial of Service 11
3.3 Distributed Denial of Service 12
4. Prevention and Mitigation Method 13
4.1 Fuzzing 13
4.2 Fuzzy Logic 14
4.3 Machine Learning 15
5. Method and resources 16
6. Analysis and discussion 17
6.1 Prevention through Fuzzing 17
6.2 Mitigation through Fuzzy logic and Machine Learning 20
6.2.1 Mitigation through Fuzzy logic 20
6.2.2 Mitigation through Machine Learning 22
7. Conclusion and Discussion 26
7.2 Social and Economical Implications 27
7.3 Project Development Potential 28
8. Reflections on own Learning 29
8.1 Knowledge and Understanding 29
8.2 Skills and abilities 30
8.3 Evaluation ability and approach. 30
1. Introduction
In this section, I will present the overarching purpose of this research project. I will also briefly introduce the central parts of my project and give a short explanation to why doing this project is important.
1.1 Background
The Internet of Things (IoT) is a system that uses the internet to connect smart devices and allow them to communicate, and is often used within education, businesses and society in general [1]. IoT has expanded rapidly and has consequently become a part of daily life. For devices to connect and communicate with each other, protocols such as the Message Queuing Telemetry transport (MQTT) protocol can be implemented [1, 2].
Unfortunately, security breaches within both IoT and MQTT are common [3, 4] and can lead to privacy violations. For this reason, it is important to gain a deeper knowledge about different methods that can be implemented within the MQTT protocol to prevent and mitigate security breaches.
According to Headlight 1, an IT-consulting firm and supervisors of this paper, knowledge
about security measures within IT is important, which is one of the reasons why they wanted to partake in a paper that focuses on the MQTT protocol.
1.2 Project
The main purpose of this bachelor's thesis is to explore and understand how security breaches can be prevented and mitigated in the MQTT protocol with the help of Fuzzing, Fuzzy logic and Machine learning, to increase its overall security.
In this thesis, I use the word “prevention” to refer to methods that are developed for finding vulnerabilities before an attack, while the word “mitigation” will refer to detecting and dealing with ongoing attacks on the MQTT protocol. These words will be further introduced and explained in Section 6 of this paper.
In this paper, I used a theoretical approach. I selected, analyzed, and discussed recent research papers, about how Fuzzing, Fuzzy logic and Machine learning can be used to prevent and mitigate security breaches. Hopefully by understanding how prevention and mitigation methods can be implemented within the MQTT protocol, an increased level of security can be ensured to protect the legitimate users.
1
This paper is organized as follows: Section 2 will describe the theoretical background for IoT and MQTT. Section 3 will present the security breaches that are considered in the papers I selected. Section 4 will introduce some necessary background knowledge on Fuzzing, Fuzzy Logic and Machine learning. Section 5 will present the method I followed and the resources I used. Section 6 will present my analysis and discussion on Mitigation and Prevention though Fuzzing, Fuzzy logic, and Machine learning. In section 7, I will state the final conclusion of this thesis and discuss the implications and potential development related to this paper. Finally, in section 8, I will reflect over my own learning, skills and abilities that were obtained while working on this thesis.
2. Theoretical Background
In this section, I will introduce both the Internet of things (IoT) and Message Queuing Telemetry Transport (MQTT).
2.1 Internet of Things
During the late 90s, Kevin Ashton, a director at AUTO-ID center at the Massachusetts Institute of Technology (MIT), coined the term Internet of Things (IoT) [5]. Ashton explained that he wanted a system that “ empowers computers with their own means of gathering information, so they can see, hear and smell the world for themselves, in all its random glory”2. In other words, IoT can be used to link the physical world to the digital
world through the internet [5]. As of today, IoT is still developing as more objects are adopting this system [6]. The word “things” in the internet of things can be applied to all kinds of devices that can be used in daily life while the words “internet'' refers to the wireless connection made through the internet [7].
The main objective of IoT is to connect several devices to each other and this can be done with the help of different technologies, such as: Radio Frequency IDentification (RFID), Wireless Sensor Networks (WSN) and Cloud Computing [6]. Through IoT, individuals should be able to access and control their devices whenever and wherever they want as long as they are connected to the internet [3]. For example, in a smart home, IoT devices can be used to control the temperature and the light.
As mentioned, IoT as a system relies heavily on the internet for it to function and it is therefore highly prone to security breaches [3]. For example, a device can be hacked by individuals that should not have access to it. The actually authorized user can, because of these attacks, suffer from severe privacy violations. Therefore, implementation of security measures are essential to ensure that the privacy of the users are violated.
IoT can be attacked in different ways based on where the attack occurs. For instance, attacks can either be physical or dematerialized. With physical attacks, the attacker has access to the device itself and can therefore destroy it or temper with it physically [7]. However, security breaches through physical attacks are not relevant for this thesis and will therefore not be examined or considered throughout the rest of this paper.
Dematerialized attacks, on the other hand, do not require physical access. Instead, the attacker can focus on the software around the device to attack [7], meaning either the network layer, the application layer, or the transport layer.
2
Attacks on the network layer means that the attacker aims to disturb the internet communication between two devices, for instance by targeting the normal routing information or the traffic analysis [7]. These kinds of attacks can for example be executed with the help of spoofing where the attacker is able to swap the transmission of legitimate data to instead transmit malicious data.
Attacks on the application layer are executed to target the end-user software of the IoT devices [7]. The application layer can be attacked with the help of Denial of Service (DoS) attacks which are commonly used to exhaust the available resources of the server, such as the memory or the bandwidth of [8]. By exhausting the resources, the clients legitimate access to the server will be denied.
Attacks within the transport layer means that the delivering of data between devices might be at risk [7]. In order to deliver data between devices of IoT, transmission control protocols (TCP) are used. Similarly to the application layer, in the previous paragraph, the transport layer can also be attacked with the help of DoS attacks which in this case aim to exhaust and monopolize the data delivering function of the protocols.
In this paper, the main focus will be on attacks that occur within the Transport layer of IoT, in particular the Message Queuing Telemetry Transport (MQTT) protocol which is a very common standard that is implemented on top of the TCP [9]. This protocol will be introduced more thoroughly in Section 2.2.
2.2 Message Queuing Telemetry Transport
Message Queuing Telemetry Transport (MQTT) is a standard protocol that was developed during the late 90s by Arlen Nipper and Andy Standford [9]. It was developed to aid in the communication between smart devices that use IoT and it is used as a messaging protocol [9, 10]. To enable these communications between the devices of IoT, MQTT is implemented on top of the Transport Control Protocol (TCP) of IoT. Moreover, the MQTT protocol is especially useful in devices with networks that are considered unreliable and that have low-bandwidths.
The MQTT protocol is, as of 2021, still a very new protocol but it has become one of the most used protocols for messaging worldwide, for example, MQTT protocols are used in websites such as Facebook that allow their users to communicate with each other [4, 11].
MQTT uses pub-sub patterns for communication [9, 10]. There are three factors that have to be included in order for the pub-sub method to work: publishers, broker and subscribers, as demonstrated in Figure 1. Publishers contain data and their function is to send this data to topics which are managed by the broker. Subscribers are subscribed to the topics inside the broker. Data is sent by the publisher to the broker and this received data is then sent to the subscribers from the broker. This means that the publisher and the subscriber are not aware of each other and the broker functions as a middleware between them [4].
The Brokers is a fundamental part of the MQTT protocol and its function is to make sure that the pub-sub method is working as it should [9, 10]. To do that, the broker must, for example, ensure that clients are able to accept messages or that the client can both subscribe and unsubscribe from a device whenever it needs to.
Figure 1: The pub-sub pattern of a MQTT protocol
The latest version of the MQTT protocol, MQTT v5, has 15 different control packets, shown in Table 1, that are used to deliver the messages and thereby allow IoT devices to communicate with each other [12]. A control packet usually consists of a fixed header (containing the packet’s metadata) and payload (containing the actual body of the message) [9]. A fixed header has a size of 2-bytes while the Payload that can has a size up to 256 MB.
Table 1. Control packets of the MQTT protocol
To ensure the delivery of a message, there are three types of Quality of Services (QoS) that can be used: QoS0, QoS1, and QoS2 [12]. QoS0 is used to ensure that the message is delivered at most once. QoS1, is used to deliver the message at least once but more messages might be delivered. QoS3 is used to deliver messages exactly one time.
In terms of vulnerabilities, security breaches within the MQTT protocol have unfortunately been revealed to be common [4]. To counteract these security breaches, authentication technologies can be implemented [10]. For instance, an access control list (ACL) can be implemented within the broker to ensure that all access is authorized. ACL is mostly used to retain information of credentials, such as usernames and passwords, of the client that have legitimate access to the services of the broker.
However, ACL is not enough for ensuring the users privacy when using devices that communicate through the MQTT protocol. Indeed, ACL is a good security measure when it comes to attacks that target credentials, but the user's credentials is not always the attacker's target. For this reason, the implementation of methods for prevention and mitigation to tackle other types of attacks is essential.
Name Flow direction Description
CONNECT Client to server Connection request
CONACK Server to client Connection acknowledgment PUBLISH client to server or server to client Publish message
PUBACK client to server or server to client Publish acknowledgment (QoS1) PUBREC client to server or server to client Publish received (Qos2 delivery part1) PUBREL client to server or server to client Publish release (Qos2 delivery part2) PUBCOMP client to server or server to client Publish complete (Qos2 delivery part3) SUBSCRIBE client to server Subscribe request
SUBACK server to client Subscribe acknowledgment
UNSUBSCRIBE client to server Unsubscribe request
UNSUBACK server to client Unsubscribe acknowledgment
PINGREQ client so server PING request
PINGRESP server to client PING response
DISCONNECT client to server or server to client Disconnect notification AUTH client to server or server to client Authentication exchange
3. Security breaches and Security measures
In this section, I will be exploring different attacks that often occur within the MQTT protocol, and that will be investigated in the remainder of this paper. These attacks are Brute force Authentication, Denial of service, SlowITe and Distributed Denial of service.
3.1 Brute Force Authentication
Brute Force Authentication (BTA) is an attack that targets the authentication system of the MQTT broker. Its main purpose is to obtain the legitimate users credentials, such as their username and password [13]. To obtain these credentials, the brute force authentication attack is, for instance, used to attempt as many combinations of the password as possible until the right password is found [14]. Once the right password is found, the attacker can gain access to the services of the IoT.
3.2 Denial of Service
One of the most prevalent security breaches within both IoT and MQTT are called Denial of Service attacks (DoS) [15, 16]. These attacks function by making the actual users of the services not able to use them. This is done by creating interruptions in for example, the bandwidth of the network, which results in blocking the legitimate users from accessing the service that is provided by MQTT protocol. These interruptions are often caused by delivering attack packets. Packets are sets of data and they are used for delivering messages between the publisher and subscriber of the MQTT protocol. However, when a MQTT protocol is targeted, attack packets will be delivered instead. Attack packets are packets delivered from illegitimate sources and their main function is to exhaust available resources, thus withholding the legitimate user from the service. This is because the increased number of packets, both legitimate and attack, will overload the traffic to and from the broker, which in turn will result in a confusion about which packets are actually legitimate and which packets are attack packets.
Another kind of attack that might be used to target the MQTT protocol is SlowITe which is considered to be a novel DoS attack [13]. The main function of SlowITe is to occupy all connections, to and from the broker, that are available at the same time by creating as many connections as possible. The reason for executing a SlowITe attack is to exhaust the availability of the MQTT broker by making it busy which in turn will result in it not being able to work properly. The main difference between regular DoS attacks and SlowITe is that a SlowITe attack does not need as many resources to be carried out.
3.3 Distributed Denial of Service
Distributed Denial of Service attacks (DDoS), as illustrated in Figure 2, are attacks that target a system by sending attack packets from various sources, which in turn acts similarly to DoS by denying the actual user from being able to use the services [16]. The difference between Dos and DDoS is that malicious packets do not come from one source but multiple sources at the same time which will make it harder to mitigate since all of these malicious sources need to be stopped at once to mitigate the ongoing attack. In addition, a larger number of attack packets usually implies a larger overall interruption effect in the network.
4.
Prevention and Mitigation Method
In this section I will describe three different techniques that can be used to either prevent or mitigate vulnerabilities within the MQTT protocol. Section 3.1 will focus on Fuzzing, Section 3.2 on Fuzzy logic, and Section 3.3 on Machine Learning.
4.1 Fuzzing
Fuzzing, also known as fuzz testing, was originally developed during the late 90s by Miller et al. [17] and has recently become a very common technique for detecting various bugs and vulnerabilities that might occur within software systems such as IoT and MQTT [18, 19]. For example, Fuzzing was implemented by Google to detect bugs in their browser, Google Chrome, and during the past eight years they were able to find more than 16000 bugs [19].
The fuzzing is done by delivering data inputs that are altered into the system and monitoring the outcome, as demonstrated in Figure 3. There are two major outcomes, the system could either reject the input or accept it, however if the accepted input is malicious the system will crash [17]. There are three types of fuzzers: Greybox fuzzing, Blackbox fuzzing and Whitebox fuzzing.
Blackbox fuzzers are used to generate inputs constantly and then to observe the obtained data. The advantage of blackbox fuzzing is that it can be used in cases where there is no information about the program's internal structure .
In whitebox fuzzing, compared to blackbox, the program's internal structure is known and the main function and advantage of this fuzzer is to generate inputs to find new paths within the program.
A Greybox Fuzzer is a hybrid between a Blackbox Fuzzer and a Whitebox Fuzzer. The program's internal structure is used to gain feedback which is then used to generate new inputs from an initial seed input to increase the area of coverage within the program to find additional vulnerabilities. A greybox fuzzer can also be used to discover paths.
In terms of the security of the MQTT protocol, Fuzzing is a preventive method due to its ability to detect vulnerabilities. Once the vulnerabilities are detected the developer can choose to correct them and thereby increase the overall security.
4.2 Fuzzy Logic
Fuzzy Logic was introduced by Zadeh et al. [20] and the fundamental principle of Fuzzy logic is to determine to which degree an element belongs to a set. Fuzzy Logic is used to extend the use of Boolean (or crisp) values which can only be used to determine if an element is true (1) or false (0). Fuzzy variables, however, are not limited to be either 0 or 1 and can instead take on any value ranging between 0 and 1, where 0 means completely false and 1 means completely true.
In general, fuzzy logic systems are organized around three stages: Fuzzification, Fuzzing interference process, and Defuzzification [21]. The fuzzification stage is used to convert Boolean values to fuzzy variables. The Fuzzing interference process uses IF-Then rules to determine the output value based on the input value. Lastly in the defuzzification stage, the fuzzy variables are converted back into boolean variables.
In terms of security, Fuzzy Logic can be used as a method for mitigation since it can be used to detect whether nodes, for example in IoT or in MQTT, are malicious or not [21, 22].
4.3 Machine Learning
Another method that can be used to mitigate attacks that can emerge in environments such as MQTT protocols is Machine Learning algorithms. This method trains algorithms into identifying attacks that might occur. Machine Learning is commonly classified in three categories: supervised learning, unsupervised learning and reinforcement learning [23]. In Supervised learning, the desired output of the model is known even though the inner relationships of the data are unknown. The training with this learning technique is conducted by using a dataset to train the model, and a different dataset to check whether the developed model functions as it should, in for example, detecting attacks.
In unsupervised learning, the model and its desirable output are unknown which means that the model would instead try to find relationships in the form of shared similarities within the data and group them together.
In reinforcement learning, the main focus is trying to improve the used model by using trials-and-errors process and also rewards based on the acquired output. This model can improve itself by adapting with the help of the rewards. The rewards are used to determine a value which is used to identify how good the model was in its decision.
Machine Learning for mitigation in MQTT usually makes use of well-known supervised ML algorithms, for instance Decision Tree [24], Random Forest [25], Gradient Boost [26], Naive Bayes [27], Neural Networks [28], and Multilayer Perceptron [29].
5. Method and resources
In this section, I will explain why and how I chose the research papers I used to explore the purpose of this paper.
To find these papers, I first did a general search on MQTT to see whether there was enough research about MQTT and security breaches. I used Google scholar 3 and Primo4 for this
purpose, with keywords“MQTT and Vulnerabilities”. This general search revealed a lot of different papers with different security breaches and security measures. Based on the search, I decided to explore how Fuzzing is as a method for prevention and also explore how Fuzzy Logic and Machine Learning are methods for mitigation for the MQTT protocol.
In the second search on Primo the keywords that I used were “ MQTT and Fuzzy logic”, “MQTT and Fuzz”, MQTT and Fuzz testing” and “MQTT and Machine Learning”. Most of the papers that came up had nothing to do with the purpose of this paper, after going through their abstracts, which is why only five papers were selected. Two of these chosen papers consider prevention through Fuzzing whereas three papers consider mitigation through Fuzzy Logic and Machine Learning.
3 https://scholar.google.com/
6. Analysis and discussion
In this section, I will analyse and discuss the different papers I based my work on. In section 6.1 I will explore prevention through fuzzing, and in Section 6.2 I will explore mitigation through Fuzzy Logic and Machine Learning.
6.1 Prevention through Fuzzing
The word prevention, as mentioned above, refers to methods that are developed for finding vulnerabilities before an attack occurs. This means that the developer can implement the right measures to counteract the detected vulnerabilities before the system goes into production. The prevention method that will be analysed and discussed in this subsection is Fuzzing, also known as fuzz testing, which was described in Section 4.1.
In this section I will examine two papers, one written by Casteur et al. [4] which aimed to create a testing architecture for testing a generic blackbox fuzzer ability in finding vulnerabilities, and one by Zeng et al. [12] which proposed an extended and thereby improved version of already existing fuzzers, called Multi-Fuzz. Both of these papers will be analysed and discussed in terms of how good Fuzzing would be as a preventative method for finding vulnerabilities in the MQTT protocol.
In the paper written by Casteur et al. [4], the main objective was to test the ability of a blackbox fuzzing system in finding vulnerabilities within a MQTT protocol. To do this the authors created 600 different scenarios, where each of the scenarios contained at least two packets.
To create these scenarios, the authors decided to randomly assign a packet to either remain normal or to go through an alteration. For example, if a scenario contained 10 packets, 4 of them could randomly be assigned to be altered while the other 6 would remain normal. This means that normal packets contain the original message set that would have normally been delivered in a MQTT protocol while the altered packets however have messages sets that have been altered and might therefore make the system crash.
This alteration was set to happen in two different ways by the authors: an alteration could either happen by altering the header or by altering the payload of the packet. Whether the packet is altered at the header or the payload is also randomly assigned. In the previous example, 4 out of 10 packets were randomly decided to be altered. Thereafter, 3 out of 4 could randomly be assigned to be altered at the header and the last packet could randomly be assigned to be altered at the payload of the packet.
Each scenario would thus end up with different amounts of packets with a minimum of two packets that are randomly altered at either the header or the payload, furthermore, the ratio between normal to altered packets was set to also randomly vary from one scenario to another. This was done 600 times to create all of the 600 scenarios.
The reason why that many scenarios were used instead of one is that the authors aimed to gain as much feedback as possible by creating very easily different types of interactions with the broker. Castuer et al. explained that the more scenarios used, the more variation of interactions with the broker which in turn would yield more varied, but maybe also some unexpected, feedback.
Casteur et al. decided to calculate five different scores for each scenario used during the analysis: initial, log, time, delta, and final score, to determine how good their fuzzing method was in detecting vulnerabilities. The initial score was assigned prior to the attack and the score varied from scenario to scenario due to the different combinations of normal and malicious packets in each of the scenarios. The log score represents the score after an attack had happened, if an inconsistency was found the score would gain 200 points, if an error was found insead the score would gain 1000 points. The time score refers to the duration of the test. Delta score is the calculated difference between the two first scores, initial score and log score. Final score is the added total of delta score and time score.
As for time scores, the results indicated that more than 10 percent of the 600 scenarios, in total 68, were considered to have abnormal time scores. The authors further explained that the lower the time the more healthy the scenario was considered and longer time scores, especially above 185s were considered to be abnormal and thereby unhealthy. The time score is used to demonstrate the duration of the response time, this means that the unhealthy scenarios have longer response time than healthy scenarios.
As for the delta scores, which represent the difference in score between the initial score and the log score, the lower the score the better the scenario was. In total 40% of 600 scenarios had delta scores that were in the average range. As for the final scores, authors explained that a final score lower than 945 was to be considered healthy and a score above it to be unhealthy. Because of this, the amount of unhealthy scenarios was at around 35%.
In conclusion, the results showed that a higher score would mean that the attack had a greater impact on the MQTT broker. The authors state in their conclusion that there is a need for more improvement to this proposed approach but that the results can still be used to identify vulnerabilities that may occur because of malicious packets by looking at the final score.
Zeng et al. proposed a coverage-based fuzzing method called MultiFuzz, in order to detect vulnerabilities within the MQTT protocols [12]. This proposition stemmed from the fact that the authors felt that fuzzing is often only executed on two parties of the protocol and not all three parties, which are: subscribers, publishers and brokers. This is due to the fact that most fuzzers, according to Zeng et al. only enable single connections.
In single connection fuzzing, the fuzzer will pretend to be one of the three parties so that the fuzzing can be executed on the other two. For instance, if the fuzzer is pretending to be a publisher then the message sent through the publisher would be a fuzzed PUBLISH message however, the corresponding message that will be sent from the subscriber will not be a fuzzed SUBSCRIBE message. This is because the fuzzer can only control the party that it is pretending to be.
The issue with single connection fuzzing, is that some vulnerabilities might not be detected. A single connection fuzzer will be able to detect vulnerabilities in form fuzzed PUBLISH messages or fuzzed SUBSCRIBE messages separately, depending on which one it is pretending to be. However when it comes to a combination of fuzzed SUBSCRIBE and PUBLISH messages together they would not be detected by the fuzzer.
To improve Fuzzing, Zeng et al. developed Multifuzz, which has the main function to enable multiple connections whilst fuzzing. With multiple connections, the fuzzers, MultiFuzz; will be able to fuzz several parties at once, this means that it can be both the subscriber and the publisher simultaneously. This means that both a fuzzed SUBSCRIBE and a fuzzed PUBLISH can be delivered simultaneously.
MultiFuzz is a coverage-based fuzzer, which is a type of fuzzer often used to monitor and discover as many connection paths as possible within a protocol with the help of single inputs. This type of a fuzzer is categorized as a greybox fuzzing method.
The authors decided to implement Multifuzz in a MQTT broker to evaluate how good it is for preventing vulnerabilities. The ability of MultiFuzz was evaluated by looking into the amount of paths that it can discover in a program. The authors explained that the amount of paths discovered in a program is related to how good the fuzzer used is. Multifuzz was also compared to three other types of fuzzers called MOpt [30], AFL5 and AFLnet [31].
The results from the analysis showed that Multifuzz was able to find 44,6% more paths than AFLnet, 125.4% more than MOpt, and also 126.6% more than AFL. Overall, the authors found that MultiFuzz was faster in discovering paths for communication compared to other types of fuzzers and that it could also discover more types of paths. It is suggested that the different amount of paths discovered is because regular fuzzers are often only looking at two parties of the MQTT protocol rather than all three parties. Not being able to look at all three parties of the protocol means that some vulnerabilities within the protocol may not be
acquired which is why, according to the authors, the usage of MultiFuzz as a detection method is advantageous.
The results from both papers [4, 12] have demonstrated that implementation of fuzzing is very efficient for finding vulnerabilities within the MQTT protocol. Fuzzing can still be considered a novel technique and more research is still needed. However, as seen in these two papers, Fuzzing is efficient which in turn provides backing for further research in the future.
6.2 Mitigation through Fuzzy logic and Machine Learning
I explored how Fuzzing can be used as a prevention method for finding and dealing with vulnerabilities in the MQTT protocol while the system is in development in Section 6.1. In this section, I will explore methods for mitigating ongoing attacks in the protocol while it is in production, these mitigation methods are Fuzzy Logic and Machine learning.
6.2.1 Mitigation through Fuzzy logic
Fuzzy logic was examined in a paper written by Haripriya et al. [21], that wanted to suggest solutions that could aid in the detection of malicious behaviors in the communication, through the MQTT protocol. To do this, the authors proposed a lightweight intrusion detection system (IDS), labeled secure-MQTT, which uses a Fuzzy Logic approach to identify early malicious behaviors in the MQTT broker.
This paper focused on DoS attacks and how they disturb message requests by making the broker busy. The authors further explain that DoS attacks make it hard for the broker to differentiate CONNECT message packets (used by a client to request a connection to the broker) that are normal from those that are abnormal. These requests are then acknowledged by the broker through CONNECTACK messages. For this reason, the paper focuses on these two types of messages.
As described in Section 4.2, Fuzzy Logic in general is composed of three stages: Fuzzification, Fuzzy inference process and Defuzzification. In the paper written by Haripriya et al., the inputs were a Connection Message Ratio (CMR), which describes the proportion of CONNECT messages in all received messages, and Connection Acknowledgement Ratio (CAMR), which describes the proportion of clients actually connected. In order to execute the analysis, these two inputs were first converted to fuzzy variables through the fuzzification stage, then the outputs for CMR inputs and CMAR inputs were determined with IF-THEN rules, presented in Table 2, which they generated by calculating statistics from actual traffic. Finally in the defuzzification the outputs were converted back to crisp variables.
Table 2. Each column is representing a generated fuzzy rule (IF-THEN) which is used to predict the anomaly of the output. For instance, if the input CMR is Low and the input CMAR is also low then there is no anomaly in
the output. However, if CMR is high and CAMR is also high, the output will be considered as an attack. From [21].
Moreover, Haripriya et al. also used an additional stage before the defuzzification stage which was a Fuzzy rule interpolation stage. In this stage, if the inputs do not correspond to any of the IF-Then rules in the fuzzy interference stage, then the rules can be updated or new rules can be developed through a fuzzy rule interpolation method. The new rules are determined by the previous inputs and scenarios used.
In the analysis, Secure-MQTT was compared to MQTT-S [32] which uses SSL/TLS as a security measure for the MQTT protocol. In this comparison they looked into five different detection metrics: attack detection accuracy ratio, attack detection efficiency, precision rate, false positive ratio and the attack detection rate.
The attack detection accuracy was used to measure the total percentage of the malicious nodes that Secure-MQTT was able to successfully detect through the analysis. The attack detection efficiency was used to check how efficient Secure-MQTT was in detecting the malicious nodes, the total amount detected was then compared to the actual amount used in the network. The precision rate was used to check the total amount of correctly found nodes out of all the nodes that were detected through the analysis. False Positive ratio was used to determine the rate of how many of the nodes that Secure-MQTT found to be malicious were actually completely normal. Finally, the attack detection rate was used to measure how many of the supposedly detected malicious nodes were actually malicious, in other words it was used to check the amount of true positives.
Overall, based on these five detection metrics, Secure-MQTT was determined to outperform MQTT-S, according to the authors, this was due to the Fuzzy logic approach in Secure-MQTT. For instance, the detection rate efficiency demonstrated that Secure MQTT was found to be more than 80%, better on average at detecting malicious nodes in Secure-MQTT compared to MQTT-S.
Therefore, these results demonstrate that Fuzzy logic can be very useful in terms of mitigation on ongoing attacks on the MQTT protocol. This means implementation of Fuzzy Logic can be helpful in increasing the overall security in the MQTT protocol. However, the obtained result is only from one paper, and to my knowledge, this is the first paper considering Fuzzy Logic to mitigate ongoing attacks on MQTT protocols. Therefore more research is needed, but similarly to the Fuzzing results in Section 6.1, these results are promising.
CMR Low Low Low Medium Medium Medium High High High
CAMR Low Medium High Low Medium High Low Medium High
6.2.2 Mitigation through Machine Learning
In this section, I will analyse and discuss two papers on Machine Learning to see whether it can be considered an effective mitigation method for detecting and dealing with ongoing attacks within the MQTT protocol.
Implementation of Supervised Machine Learning algorithms, which was described in section 4.3, has been used in research for several years, particularly as a mitigation method to increase IoT security [33-38]. One problematic aspect within this past research is that many of the authors had to create their own datasets to train their chosen algorithms. One of the papers, written by Vaccari et al [13], that will be explored more thoroughly in subsection, aimed to create and release MQTTset, so that anyone that is interested in using it can do so in their research. By doing this, researchers do not have to create their own dataset anymore, and research on Machine learning for mitigation of attacks on MQTT systems can thereby become more attainable and make results comparable.
MQTTset is a dataset which was developed by Vaccari et al. [13] to include traffic that is considered to be legitimate and malicious. To develop MQTTset, the authors deployed eight different sensors that would be able to communicate through a MQTT protocol such as fan speed and temperature. The eight sensors were divided between two different rooms, shown in Figure 4. The authors decided to not put any security measures and started to record the traffic that occurred in the MQTT protocol. As expected, since no security measure was implemented, the MQTT protocol was attacked thus resulting in records of both legitimate and malicious traffic. The malicious traffic included variations of DoS attacks, SlowITe and Brute Force Authentication Attacks.
Before MQTTset could be released publicly, the authors needed to validate it, meaning ensuring that it was good enough to be released so that others can use it to train their own systems. If this validation reveals that MQTTset is good, then it can be used to train other similar algorithms for detecting these attacks. Six different algorithms, that has previously been used in terms of security, were used to test MQTTset: Decision Tree [39], Random Forest [40], Gradient Boost [41], Naive Bayes [42], Neural Network [43] and Multilayer Perceptron [44].
The authors found that all of the six algorithms had an overall detection accuracy of 94.6 % or above. Random forest had the highest detection accuracy at 99.4% and Multilayer perceptron had the lowest at 94.6 %. However, the authors decided to extend the test since they speculated that the results might have been affected by the unbalanced ratio of legitimate to malicious traffic in MQTTset. The ratio of malicious traffic was very low compared to the ratio of legitimate traffic. Thus, before remaking the test, the authors ensured that the ratio of malicious traffic was similar to the ratio of legitimate traffic in the dataset, thus making MQTTset more balanced in terms of traffic.
In the second testing, the analysis showed that all algorithms, except for Naive Bayes, had detection accuracies of 87% or above. The highest detection accuracy belonged to both Random Forest and Decision Tree at 91.5%. Naive Bayes had a much lower detection accuracy at 64.3%. The authors considered that the obtained results were enough to validate MQTTset and decided to release it publicly.
The authors also stated in their conclusion that a more balanced ratio between the legitimate to malicious traffic demonstrated more realistic results. This is because the confusion matrices presented in the paper implied that the unbalanced dataset test was found to be good at detecting attacks but that they were making mistakes when classifying the attacks. The more balanced dataset was better at classifying the types of attacks.
The other paper in this subsection that will be explored more thoroughly written by Syed et al. [15], and in this paper the purpose was to create a general detection framework for detecting DoS attacks and to also test it. They called the framework a “ machine
learning-based MQTT DoS attack detection framework”.
The authors created a framework that consisted of three major parts. First, a traffic generator, which creates traffic, which in this case was attack traffic. Second, a feature extraction tool which is used to extract data and compile it. This data could be from, for example, the destination port or the IP source. Lastly, a Machine Learning based detection system was incorporated into the framework.
In the analysis three different classifiers were used: Decision tree , Multilayer perceptron and Average One-Dependence Estimator (AODE) [45], a Naive bayes algorithm. The main function of these classifiers was to determine which inputs could be considered attacks and which of them were not. The main reason for using classifiers as a determiner is that once the type of input is known it can aid in applying the right type of measure to counteract the attacks that might occur.
To analyse the effectiveness of the ML based detection system with the three classifiers on DoS attacks, the authors used four different types of packets in their dataset. One of these packets was Normal without any attack whilst the three others contained attacks. The three attack packets were: MQTT-DoS, MQTT-FUZZ6 and TCP-DoS7
The attack packet called MQTT-DoS was developed by Syed et al. and according to the authors it could further be divided into four different subclasses based on the used control packet and the level of unauthorized access that the attack had. The four subclasses were: MQTT-DoS-BF1, MQTT-DoS-BF2, MQTT-DoS-BF3 and MQTT-DoS-IAUTHS.
These attack packets were used in two different datasets, one which is a 4-class MQTT dataset and one which is a 7-class MQTT data set. The four-class data set consisted of the normal packet and MQTT-FUZZER, MQTT-DoS and TCP-DoS whilst the seven-class data set consisted of the TCP-DoS, MQTT-Fuzz and Normal but with the addition of the four subclasses of MQTT-DoS.
After the analysis the authors calculated several detection metrics, these were: detection accuracy, training time, true positive rate, false positive rate. Detection accuracy was used to represent, in percentage, how accurate the classifiers were in detecting malicious attack traffic. Training time was used to represent the training time for each classifier. The true positive rate was used to determine the percentage of how many times the classifiers were able to correctly detect the attack traffic. The false positive rate was used to calculate the percentage how many times the classifiers were not able to correctly detect the attack traffic, this was used to check how many of the supposedly detected attack traffic was actually attack traffic and not normal traffic.
Overall, the results demonstrated that the classifier with the best detection accuracy for detecting malicious attack traffic was the AODE classifier. This result applied to both types of datasets, 99% for the 4-class data set and 88.2% for the seven-class data set. AODE also had the lowest training time. Moreover, all of the classifiers for the 4-class dataset had a detection accuracy of almost 96% or more. In the 7-class data set the detection accuracy varied more and the range was between 65.6 % to 99.8 %.
6
https://github.com/F-Secure/mqtt_fuzz.
The overall findings also suggested that the length and overall size of the packet had an effect on the detection accuracy. The authors concluded that this proposed framework, in its entirety, had a high capability of detecting attacks that could pose as threats in MQTT. In conclusion, both of the papers, written by Vaccari [13] and Syed [15], demonstrated that Machine Learning can be considered a very efficient mitigation method for the MQTT protocol.
7. Conclusion and Discussion
In this section, I will discuss the findings of my work. Section 7.1 will make a conclusion of the findings and relate it to the purpose of this paper. Section 7.2 will discuss social implications. Section 7.3 will discuss the project's potential for future development.
7.1 Conclusion
The purpose of this paper was to explore and understand how security breaches can be prevented and mitigated in the MQTT protocol with the help of Fuzzing, Fuzzy logic and Machine learning.
Fuzzing as a method for prevention was found, based on the two analysed papers in Section 6.1 [4, 12], to be very efficient. This means that if a developer wants to implement a method for preventing vulnerabilities by finding and correcting them while the MQTT protocol still is in development, Fuzzing would be an excellent choice.
The two fuzzing papers implemented fuzzing differently which means that the findings cannot be comparable since they have different purposes. Casteur et al. developed a testing architecture to explore how good a blackbox fuzzer is at finding vulnerabilities with the help of scenarios. Zeng et al. however, wanted to improve existing fuzzer thus developing Multifuzz which is a coverage-based fuzzer.
Casteur et al. found that implementing fuzzing was a very good way to find vulnerabilities and Zeng found that Multifuzz was indeed an improvement and could find more paths compared to previous fuzzers. The findings are not comparable but both methods can instead be used together to improve Fuzzing in terms of prevention.
Overall, the main advantage of fuzzing is that it can be used while the system is still in development, which means that any found vulnerabilities can be corrected prior to putting the system online.
As for mitigation of ongoing attacks on the MQTT protocol, implementations of both Fuzzy Logic and Machine learning were found to be efficient methods. This means that implementation of either Fuzzy Logic or Machine Learning would be good for detecting and dealing with attacks occurring on the MQTT protocol while it is in production.
Compared to the two Fuzzing papers, Machine learning was implemented quite similarly in both of the papers [13, 15] which means that the findings can be comparable. Both papers used datasets, with legitimate traffic and malicious traffic to train the algorithms in detecting attacks. There are some minor differences such as the type of attack traffic and which algorithms were used, but the overall general idea is the same. Both papers demonstrated that their chosen algorithms were very efficient in detecting attacks.
Moreover, MQTTset [13], was released so that anyone who wishes to use it can do it. This makes implementation of Machine Learning for mitigating attacks occurring on MQTT protocols more manageable.
Since there is only one paper on Fuzzy Logic the findings can not be analysed in terms of similarities and differences. Even though fuzzy Logic is also a mitigation method it cannot be related to the two Machine Learning papers either.
Based on the findings from all of the five papers that were analysed and discussed in Section 6, implementation of methods for both prevention and mitigation together would ensure a higher level of security for the users of the IoT devices that communicate through the MQTT protocol.
A suggestion would be to implement fuzzing as a method for prevention and then to also implement either Fuzzy Logic or Machine Learning for mitigation purposes. The idea is to first use fuzzing for finding vulnerabilities while the MQTT protocol is still in development and correct them. Thereafter, the developer can either use Fuzzy Logic or Machine learning to mitigate sudden attacks on the MQTT protocol while it is in production. Therefore, by combining methods of both prevention and mitigation the developer can ensure a higher level of security in the MQTT protocol.
7.2 Social and Economical Implications
MQTT protocols as a messaging protocol for IoT devices are widely used, for instance it is even implemented in Facebook. It is therefore very important to ensure that the privacy and security of the legitimate user is not going to be violated. A suggestion is that when a MQTT protocol is implemented, the developer should take methods for prevention, such as Fuzzing, and methods for mitigation, such as Fuzzy Logic or Machine Learning, into consideration in addition to authentication technologies. For example, Google has in the past years started to use Fuzzing to find vulnerabilities within Google Chrome.
Furthermore, in the paper written by Vaccari [13], the authors released their Machine Learning Dataset publically which means that anyone can use it and that a new dataset does not have to be developed every time someone wants to train their algorithms. This makes implementation of machine learning a bit easier and more accessible.
However, it is also important to discuss issues related to the ethical implications that might arise in relation to this thesis. Writing and publishing a thesis online, for anyone to see, that is dealing with security breaches and security solutions might lead to consequences that must be considered. For instance, a user with malicious intent might take the information and knowledge provided about the different security breaches that can be used to attack the MQTT protocol in Section 3 in this thesis and use it in a harmful way.
Unfortunately, not much can be done to reduce this kind of an ethical consequence since research on security breaches and security solutions are essential for improving the security level of the MQTT protocol. However, in this thesis, I only mentioned the different types of attacks briefly and described them generally to reduce the possibility for malicious use. I also did not mention how these attacks actually can be executed and therefore if a malicious user wants to conduct an attack similarly to the ones that were mentioned in Section 3, this thesis would not be much of a help.
Finally, the main contribution of this bachelor's thesis is to provide understanding and knowledge about different methods for prevention and mitigation that developers of MQTT protocols can take into consideration and implement to increase the overall security.
7.3 Project Development Potential
This project's potential for continued development is great. As for now, fuzzing as a prevention method was found to be very efficient for the MQTT protocol and this was also the case for both Fuzzy Logic and Machine Learning as methods for mitigation. In this paper, only five papers were analysed, if the time limit was longer a broader theoretical investigation on this research topic could for example yield a broader understanding and knowledge on how the security of MQTT protocol could be ensured.
As mentioned previously, the MQTT protocol is still considered new and is at the same time one of the most widely used protocols which means that most of the found research is still very recent. For instance, I examined only one paper about Fuzzy Logic for mitigation but I would have preferred to examine at least one more to, for example, find similarities and differences in implementation. There is a lot of research on Fuzzy Logic and how it can be used in general, but i could not find more papers where Fuzzy Logic was implemented in the MQTT protocol other than the one included in section 6.2.1.
Therefore, definitive conclusions cannot be drawn about how efficient Fuzzing is for prevention and how efficient Fuzzy Logic or Machine Learning are for mitigation only based on a few research papers. There is potential but more research should be done, and this is also a conclusion that is made by some of the authors from the analysed papers in Section 6.
8. Reflections on own Learning
In this section, I will reflect over my own learning and how I have improved during the writing process of this bachelor's thesis.
8.1 Knowledge and Understanding
Through the writing process of this paper I have gained a lot of knowledge and understanding about the MQTT protocol and the different security vulnerabilities that might pose threats for the users. I have also gained better knowledge and understanding about how methods for prevention such as Fuzzing, and methods for mitigation such as Fuzzy Logic and Machine learning can be used to prevent and mitigate attacks. Overall, writing this paper has been very educational.
I have also gained more knowledge and understanding for finding, analysing and discussing scientific papers. I have not used scientific papers similarly to this previously and because of this I had a difficult time comprehending the information and methods used in the papers at the beginning. However, by reading and rereading the chosen scientific papers several times, I learned how to better comprehend, analyse and discuss scientific papers.
At first, I did not know how I would “analyse and discuss” scientific papers and what the correct method was, but through the writing process of this paper I now know how an analysis and a discussion section should be written in a bachelor's thesis. I also learned how to link two or more papers together and both analyse and discuss them in relation to each other, for instance the two papers on mitigation in Section 6.2.2.
Furthermore, the majority of the “common knowledge” that the authors only referred to without any explanation, was not “common knowledge” for me which made it even more difficult. For example, I had no deep prior knowledge about the different supervised algorithms that were used for Machine Learning. Because of this I had to search for information in addition to what I found in the papers that were used.
However, now that I have bigger understanding and background knowledge about the different components in this paper, such as the MQTT protocol, Fuzzing, Fuzzy Logic and Machine Learning, the different scientific papers are easier to comprehend and understand.
8.2 Skills and abilities
I feel that I was able to formulate a very good and relevant purpose to write about in this paper. But at first, it was very difficult to formulate a purpose and decide on a topic that I wanted to explore through this paper. I knew that I wanted to write about IoT, but IoT is considered to be very broad and therefore not feasible. By reading more into this topic and looking into the components of IoT, I decided to focus my paper on the MQTT protocol. If I am going to write another paper in the future, I now know that i first need to first have some basic knowledge and understanding about the topic that I am interested in before formulating a purpose and not the other way around.
I have not written any bachelor's thesis previously, which means that i was at first very unsure about many components and sections of this paper. For instance, I have not used the Vancouver reference style previously, I have not read and analysed papers either, and I did not know that the overall layout was going to be organised this way. Although these small setbacks, I feel like I was able to do a very good job.
However, I would not have been able to do this myself. I consulted with my supervisors during the whole writing process, and through this consultation my supervisors were very helpful in explaining how a bachelor's thesis for computer engineering should be written correctly.
I tried to make this paper as comprehensible and coherent as possible for others who might be interested by explaining all components, for example, DoS attack, Fuzzing, Machine Learning, as thoroughly as possible and by giving the correct references. The words prevention and mitigation in this paper were defined and explained at the very beginning of the paper but also at other sections in the paper as reminders to lessen the confusion and make it easier to read without having to go back to the introduction.
I hope that I, through this thesis, was able to convey the purpose and findings of my paper in an understandable way and that I was able to inform about how prevention with Fuzzing and mitigation with either Fuzzy Logic or Machine learning, can be used to increase the level of security.
8.3 Evaluation ability and approach.
Knowledge and understanding of the vulnerabilities within the MQTT and what can be done to reduce them and insead increase the level of security is very important from a societal perspective. This is especially important in the MQTT protocol since it is so widely used. As mentioned previously, IoT devices are becoming a part of daily life and many of these devices utilize the MQTT protocol for communication.
A lot of reading was done the first few weeks of writing this thesis before any decisions were made about what I wanted to write about and how I wanted to specify my purpose. After choosing to write about the vulnerabilities within the MQTT protocol. When it comes to choosing methods for prevention or mitigation there were not a lot to read about which made it difficult. But based on the papers that I found and was able to read I decided to pick Fuzzing, Fuzzy Logic and Machine learning.
To increase the validity and reliability of this thesis, everything that has been written, apart from my own thought, have been correctly referred to using the vancouver style. Furthermore, I have tried to document everything while writing this thesis by saving all information I have had access to.
7. Reference list
[1] Soni, U. S. & Talwekar, R. H., ”Internet of Things in Smart Grid: An Overview”,
i-Manager's Journal on Communication Engineering and Systems, vol 8 (1), 2019, ss 28-36,.
DOI: 10.26634/jcs.8.2.16689
[2] Kashyap, M.; Sharma, V.; Gupta, N., “Taking MQTT and NodeMcu to IoT: Communication in Internet of Things”, Procedia Computer Science, vol 132, 2018, ss 1611-1618,.
DOI:10.1016/j.procs.2018.05.126
[3] Ali, B & Awad, I. A., “Cyber and Physical Security Vulnerability Assessment for IoT-Based Smart Homes”, Sensors (Basel), vol 18(3), 2018, ss 817- ,.
DOI: 10.3390/s18030817
[4] Casteur, G.; Aubaret, A.; Blondeau, B. ; Clouet. V.; Quemat, A.; Pical, V.; Zitouni, R., “Fuzzing attacks for vulnerability discovery within MQTT protocol”, 2020 International
Wireless Communications and Mobile Computing (IWCMC), 2020, ss 420-425,.
DOI: 10.1109/IWCMC48107.2020.9148320
[5] Ben-Daya. M.; Hassini. E.; Bahroun. Z., ”Internet of things and supply chain management: a literature review”, International Journal of Production Research, vol 57(3),
2017, ss 1-24
DOI: 10.1080/00207543.2017.1402140
[6] Gawali, S. K. & Deshmukh, M. K., “Energy Autonomy in IoT Technologies”, Energy
Procedia, vol 156, 2019, ss 222-226,.
DOI: https://doi.org/10.1016/j.egypro.2018.11.132
[7] Hussain, F.; Hussain, R.; Hassan, S. A.; Hossain, E., “Machine Learning in IoT Security: Current Solutions and Future Challenges”, IEEE Communications Service & Tutorials, vol
22(3), 2020, 1686-1721.,
DOI: 10.1109/COMST.2020.2986444
[8] Lee, S., Kim, G., Kim, S., “Sequence-order-independent network profiling for detecting application layer DDoS attacks”, J Wireless Com Network, vol 50, 2011,
DOI: 10.1186/1687-1499-2011-50
[9] Mishra, B. & Kertesz, A., “The Use of MQTT in M2M and IoT Systems: A Survey”
IEEE Access, vol 9, ss 201071-201086.,
[10] Dinculeană, D. & Cheng. X., “Vulnerabilities and Limitations of MQTT Protocol Used between IoT Devices”, Applied Sciences, vol 9(5), 2019, ss 848.,
DOI: 10.3390/app9050848
[11] Hwang, H. C.; Park, J.; Shon, J. G., “Design and Implementation of Reliable Message Transmission System Based on MQTT Protocol in IoT”, Wireless Pers Commun, vol 92(4), 2016, ss 1765-1777,.
DOI: 10.1007/s11277-016-3398-2
[12] Zeng, Y.; Lin, M.; Gue, S.; Shen, Y.; Cui, T.; Wu, T.; Zheng, Q.; Wang, Q., “MultiFuzz: A Coverage-Based Multiparty-Protocol Fuzzer for IoT Publish/Subscribe Protocols”, Sensors (Basel), vol 20(18), 2020, ss 5194.,
DOI: 10.3390/s20185194
[13] Vaccari, I.; Chiola, G.; Aiello, M.; Mongelli, M.; Cambiaso, E., “MQTTset, a New Dataset for Machine Learning Techniques on MQTT”, Sensors, vol 20(22), 2020, 6578., DOI: 10.3390/s20226578
[14] Stiawan, D., Idris, M. Y., Firsandaya, M., Nurmainim S., Alsharif, N., Budiarto., “Investigating Brute Force Attack Patterns in IoT Network”, Journal of Electrical and Computer Engineering, vol 2019, 2019, ss 1-13.,
DOI: 10.1155/2019/4568368
[15] Syed, N. F.; Baig, Z.; Ibrahim, A.; Calli, C., “ Denial of service attack detection through machine learning for the IoT” Journal of Information and Telecommunication, vol 4(4),
2020, ss 482-503.,
DOI: 10.1080/24751839.2020.1767484
[16] Douligeris, C. & Mitrokotsa, A., “ DDoS attacks and defense mechanisms: classification and state-of-the-art”, Computer Networks”, vol 44(5), 2004, ss 643-666.,
DOI: 10.1016/j.comnet.2003.10.003
[17] Liang, H., Pei, X., Jia, X., Shen, W., Zhang, J., “ Fuzzing: State of the Art," in IEEE Transactions on Reliability, vol. 67(3), 2018, ss 1199-1218,
DOI: 10.1109/TR.2018.2834476.
[18] Munea, T. L.; Luk-Kim, I.; Shon, T., “Design and Implementation of Fuzzing Framework Based on IoT Applications” Wireless Pers Commun, vol 93, 2017, ss 365-382., DOI: 10.1007/s11277-016-3322-9
[19] Boehme, M.; Cadar, C.; Roychoudhry, A., “Fuzzing: Challenges and Reflections”,
IEEE Software, 2020.,
[20] Zadeh, L.A., “Fuzzy sets”, Information and Control, vol 8(3), 1965, ss 338-353 DOI: 10.1016/S0019-9958(65)90241-X
[21] Haripriya, A. P. & Kulothungan, K., “Secure-MQTT: an efficient fuzzy logic-based approach to detect DoS attack in MQTT protocol for internet of things”, EURASIP Journal
on Wireless Communications and Networking, vol 90(2019), 2019.,
DOI: 10.1186/s13638-019-1402-8
[22] Alshehri, M. D. & Hussain, F. K., “ A fuzzy security protocol for trust management in the internet of things (Fuzzy-IoT)” Computing, vol 101, 2019, ss 791-818.,
DOI: 10.1007/s00607-018-0685-7
[23] Murphy, K. P., “ Machine learning: a probabilistic perspective.”, MIT press, ISBN: 9780262018029
[24] Breiman L, Friedman JH, Olshen RA, Stone CJ., “Classification and regression trees”, 1st edition, 1984 Wadsworth, Belmont
[25] Ho, T. K.,. “Random Decision Forests”, Proceedings of the 3rd International
Conference on Document Analysis and Recognition, Montreal, vol 1, 1995, ss 278–282,
DOI: 10.1109/ICDAR.1995.598994.
[26] Friedman, J.. “Greedy function approximation: A gradient boosting machine.”, Annals of Statistics, vol 29(5), 2001, ss 1189-1232.
DOI: 10.1214/aos/1013203451
[27] Domingos, P. M. & Pazzani, M., “On the Optimality of the Simple Bayesian Classifier under Zero-One Loss.” Machine Learning, vol 29, 2004 ss 103-130,
DOI: 10.1023/A:1007413511361
[28] Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities", Proc. Natl. Acad. Sci. U.S.A, vol 79(8), 1982, ss 2554–2558,
DOI: 10.1073/pnas.79.8.2554
[29] Cybenko, G., “Approximation by superpositions of a sigmoidal function”, Mathematics
of Control, Signals, and Systems, vol 2(4), 1989, ss 303–314,
DOI: 10.1007/BF02551274
[30] Lyu, C.; Ji, S.; Zhang, C.; Li, Y.; Lee, W. H.; Song, Y.; Beyah, R., “MOPT: Optimized Mutation Scheduling for Fuzzers”, In Proceedings of the 28th USENIX Security Symposium,
