A Comparison Between the Quality Characteristics of Two MicroserviceApplications

Faculty of Computing

Blekinge Institute of Technology

A Comparison Between the Quality Characteristics of Two Microservice

Applications

Filip Bahnan

18, 05, 2021

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the bachelor’s degree in software engineering. The thesis is equivalent to 10 weeks of full-time studies.

Contact Information:

Author(s):

Filip Bahnan

E-mail: filipbahnan@gmail.com

University advisor:

Mikael Svahnberg

Department of Software Engineering

Faculty of Computing

Blekinge Institute of Technology

Internet : www.bth.se Phone : +46 455 38 50 00

A

With the rise of cloud computing and the migration to web-based applications, scalable systems have become highly desirable. And while developing software is hard, designing a scalable system is even harder. The microservice architecture is an attempt to improve the scalability but may introduce additional challenges.

In order to correctly implement the microservice architecture, it is important to understand how the different mechanisms used in the architecture affect the quality of the application. The purpose of this research is to show how to evaluate

microservice applications and how much they can differentiate from each other.

A literature study and an architectural analysis are performed by reviewing research related to web applications and microservices. Subsequently, the empirical data is collected by evaluating and comparing two different microservice applications based on their quality characteristics.

The results of the literature study indicate that performance efficiency, compatibility, reliability, security, maintainability and portability are the most relevant quality characteristics of the microservice architecture. Furthermore, the architectural analysis describes how microservices affect these quality characteristics. Lastly, the evaluation showed that different approaches can significantly alter the strength of the different characteristics. For this specific comparison between the two selected applications, it was determined that the biggest differentiating factor is the asynchronous and synchronous messaging.

To conclude, the results show it is possible to evaluate a microservice application by its qualities. Additionally, while microservice applications may use completely different technologies, the fundamental concept behind them remains the same. What differs is the approaches used and how they affect the quality characteristics.

Keywords: Microservice, quality characteristic, and messaging.

C

Abstract ... i

Contents ... 2

1 Introduction ... 3

1.1 Background ... 3

1.2 Purpose ... 4

1.3 Scope ... 4

2 Research Questions ... 5

3 Research Method ... 6

3.1 Literature Study ... 6

3.2 Architectural Analysis ... 7

3.3 Evaluation ... 7

4 Literature Review ... 9

4.1 The Quality Characteristics ... 9

4.2 A Description of the Microservice Architecture ... 11

4.3 How to evaluate the quality of a system ... 13

4.4 Summary ... 14

5 Evaluation of Microservice Architectures ... 15

5.1 The Selected Quality Characteristics ... 15

5.2 Measuring the Microservice Architecture ... 15

5.3 Evaluation of Implementations ... 17

5.3.1 eShopOnContainers ... 17

5.3.2 Piggy Metrics ... 20

5.4 Comparison of Implementations ... 23

6 Conclusion ... 26

7 Validity Threats ... 27

7.1 Literature Study ... 27

7.2 Architectural Analysis ... 27

7.3 Evaluation and Comparison ... 27

8 Future Work ... 29

9 References ... 30

1 I

The monolithic architecture could be considered one of the most widely used while also being one of the simplest for designing web applications with. With this

architecture the logic of the entire application usually resides in only one process and therefore this process has many responsibilities. The main concern related to this is the limited scalability, since even if only certain functionality has a high user demand, the entire process needs to be scaled [1].

Since applications are becoming even more complex and scalability becomes increasingly important, a new trend in the software engineering world has emerged.

As the name suggests, applications using the microservice architecture are composed of small services, each running as its own process, with limited functionality and data responsibility [2]. Additionally, these services usually communicate through some form of messaging rather than in-memory calls [2]. However, microservices share the same complexities as other distributed systems and are therefore challenging to manage [3]. Furthermore, the many approaches available for the different

mechanisms make it difficult to decide which approach to use.

A definition of the microservice architecture:

In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and

communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. [2]

One of the most frequently discussed advantages of microservices is the scalability it enables [1]. Scalability can be expressed in a scale cube consisting of three axes representing the different forms of scalability an application can have. Older

architectures, such as the monolithic architecture, are limited to the x-axis and z-axis, which represent horizontal duplication and data partitioning. Microservices have additional scalability capabilities in the form of functional decomposition for the y- axis. This allows microservice applications to scale each service independently, rather than being forced to scale the entire application at the same time [1].

The importance of scalability relates to the need of having an adaptable system to the user demand. Especially for web applications, the demand can significantly differ during the different hours of a day [4]. This means that the ability to scale in and scale-out is a necessity and not only to improve cost efficiency but also to improve the availability of the system [5].

Microservices were first given some attention as far back as 2005 by Rodgers [6] but it was not until Netflix adapted the architecture and switched to cloud computing in 2012 that microservices attracted widespread attention by the industry [7].The reason behind this transition is that cloud computing has given microservices greater

possibilities in terms of cost and scalability, allowing it to overtake older

architectures as the need for scalability increases in the future [8]. With its simplicity, the monolith has been more than sufficient for most applications until now, the increased demand for more scalable systems has made cloud computing quite popular. This is where the microservices aim to achieve what the monolith could not [2].

Although the idea of having a distributed system is nothing new, the additional cost and complexity of splitting an application into many services has previously been too expensive. But as cloud providers such as AWS grow larger [9], cloud computing has finally become a more cost-efficient alternative over hosting everything on- premises. With cloud computing, some of the complexities of managing applications are reduced. This is because of the different service models eliminating some

responsibilities from the cloud consumer [5]. Furthermore, containerization technologies such as a docker can simplify the deployment of each microservice since it allows them to run in their own isolated virtual environment. This means that the microservices can be deployed either independently or at the same time using a container orchestration tool [3].

1.2 Purpose

The purpose of this study is to learn more about how different microservice

applications can differentiate from each other and how the quality of the application is affected by the different approaches. The many patterns necessary must be

implemented correctly for the application to be successful. An incorrect implementation of microservices might lead to the application becoming a

distributed monolith in disguise [10]. Each of these patterns can be approached in different ways which can severely affect certain quality characteristics of the application. Furthermore, as explained by Dragoni et al. [11], microservices bring many challenges that did not previously exist when using older architectures, such as the monolith. These issues include changes in one service affecting all other services and secure messaging over a network for the microservices. All these factors must be considered when choosing to develop a microservice application.

The aim of this thesis is thus to provide further insights into the microservice architecture and see how it affects different quality characteristics so that software developers can make more informed decisions during the design and building of microservice applications.

1.3 Scope

The scope is to study microservice architectures and see how the different approaches used can affect the quality characteristics. We select the quality

characteristics based on a literature study to understand which characteristics would be of high interest for us to focus on when evaluating a microservice application.

Moreover, we then conduct an architectural analysis to gain a deeper understanding of microservices to better understand how different approaches can affect the qualities. Finally, we evaluate the quality characteristics of different microservice implementations, and then compare each characteristic across the different implementation.

2 R

Q

RQ1: What are the most important quality characteristics to consider for a web application using the microservice architecture?

RQ2: How does the microservice architecture affect different quality characteristics in a web application?

RQ3: How do the quality characteristics of different microservice implementations compare to each other?

To conduct an evaluation of different microservice implementations, the different quality characteristics are be used as the metrics to measure them. Since there are different characteristics, the most important characteristics for web applications are prioritized, while the less important characteristics are ignored.

Before the evaluation of different implementations, the architecture must first be understood how it is intended to work. Seeing how the microservice architecture is in theory affecting the quality characteristics should make it easier to evaluate and compare the two different implementations.

An empirical study is conducted to better understand the advantages and disadvantages of the different approaches that can be used for the microservice architecture. By evaluating and comparing different microservice implementations, a conclusion can be drawn that describes what needs to be considered while building a microservice application to achieve strong quality characteristics.

3 R

M

The research carried out in this paper can be regarded as one empirical study, but each question, although dependent upon each other, is independent enough that it is

necessary to conduct independent research for each of them.

To answer RQ1, a literature study is conducted. The research by different researchers and authors is reviewed and compared to determine the appropriate quality

characteristics. We then use these characteristics as the metrics to measure the microservice implementations in RQ3. The literature study is an integral part of the empirical study as it gives a quantitative motivation behind the selection of the quality characteristics.

We perform an architectural analysis of the microservice architecture to answer RQ2.

This is done by doing a literature survey of different papers describing the

microservice architecture to understand how the architecture theoretically affects the quality characteristics selected in RQ1. This allows us to easier evaluate each of the different microservice implementations in RQ3.

Lastly, we select implementations of the microservice architecture for the evaluation and then compare each characteristic to see how the different approaches affect them.

The previous research for RQ1 and RQ2, with some additional research on evaluation and comparison techniques, is used to prepare for an evaluation with respect to the quality characteristics identified in answer to RQ1. Besides this, the documentation for each selected implementation is also thoroughly reviewed to better understand them.

3.1 Literature Study

Before performing a quantitative analysis of the different quality characteristics that are used for the measurement of the implementations, we first determine a set of characteristics which will be used in the evaluation and comparison.

While microservice architectures have been studied from the perspective of individual quality characteristics (see e.g. [11]), they have to the best of our

knowledge not been comprehensively studied using a set of quality characteristics.

Thus, in order to create a set of relevant quality characteristics, we start with an established industry standard for quality characteristics, the product quality model from ISO/IEC 25010 [12], and conduct a snowball sampling study on research done on web applications and microservices. This process is repeated until we feel that we have adequate coverage of the most cited papers that may be of interest. The

frequency with which each quality characteristic is used or discussed is measured and used as a basis to determine which quality characteristics are most commonly used in the context of microservice and web service architectures.

From this, we determine which quality characteristics from the product quality model are selected for RQ3 and which ones are disregarded. In such a way, only the most used or discussed characteristics are selected for the evaluation and comparison.

3.2 Architectural Analysis

The goal of this analysis is to evaluate the microservice architecture in the general case. We mainly focus on previous well-established research to see how the industry thinks the quality of an application is affected by the microservice architecture. The conclusion drawn for each quality characteristic is meant to assist with the evaluation in RQ3 by allowing us to better understand what aspects to focus on and how they are theoretically affected.

When evaluating different papers, it is important that we take the opinions of different researchers into consideration to ensure a varied description of the architecture. However, we will only focus on articles that use the same established definition of microservice architectures [2] as we do, which we presented in section 1.1.

We first perform the architectural analysis by reviewing and evaluating papers which describe the microservice architecture. The most cited papers are prioritized and can easily be sorted with the search tools provided by the research databases used for this study which include IEEE Xplore and Springer Link. However, any paper related to microservices can potentially be used for this architectural analysis and we are therefore not strictly limiting ourselves to well cited papers. Furthermore, after the initial review of these papers describing the architecture, the snowball sampling technique is used in the same manner as in the literature study for RQ1. We repeat the usage of the snowball sampling technique more than once but to a much more limited extent compared to the literature study in RQ1. By reason that the main priority of this architectural analysis is to analyze the architecture from the most well-established perspective and therefore less well-established papers are not as important in our analysis.

With this analysis concluded, we can give each selected quality characteristic a short but concise description of how it can be affected by the microservice architecture.

3.3 Evaluation

The goal of this evaluation is to better understand how the different design decisions made while building a microservice application can affect the quality characteristics.

The requirements for the selection of the microservice implementations are that they must be fully documented, developed by experienced developers, and implement microservices. To create a meaningful comparison, the implementations should be using different approaches for the most essential parts of the architecture identified in the architectural analysis. Smaller applications are preferred since they are easier to evaluate and compare. In the search for these applications, only the most well-known and publicly available applications are considered to ensure that the selected

applications are representative of the architecture.

When searching for our applications, we mainly focus on different github

repositories to find implementations that meet our requirements. Github allows us to easily search for repositories using keywords. By using the keyword microservice, we can find most of the microservice implementations hosted on the website.

Additionally, github allows users to bookmark repositories with stars which we can use to see which repositories have a high level of interest.

In order to better understand the selected applications and conduct a fair evaluation, the documentation available for each of them is extensively utilized. This means that we thoroughly review the documentation to ensure a high coverage for each

application.

When everything has been reviewed and analyzed, the evaluation of the selected microservice applications can begin. Each implementation is separately evaluated in terms of its quality characteristics one by one, since the comparison will focus on each characteristic individually. It is important that it is done systematically rather than randomly to ensure a high validity of our results. For this reason, we will find and try to use an already established method for evaluating the characteristics of an application. The most common method used by the industry to evaluate the quality characteristics of an application is selected for our study.

We evaluate the quality characteristics one by one in the order given in the product quality model, beginning with performance efficiency and ending with portability.

Since each quality characteristic has its own sub-characteristics, we separately evaluate each sub-characteristic of every selected quality characteristic and then aggregate them together. In the evaluation of the applications, a concise description of each selected characteristic is given to illustrate how it is affected by that specific implementation of the microservice architecture.

The next step after we have qualitatively evaluated each application is to compare the results. Although it is possible to make a comprehensive comparison of every single aspect of the applications, this comparison focuses only on how each quality

characteristic is affected. Thus, each quality characteristic is compared in the same order they were evaluated in, one by one, between the selected applications. The strength and weaknesses are put side by side and a short summary of every characteristic is given. Subsequently, we can answer RQ3 and draw a conclusion.

4 L

R

All papers in this literature review were found using publicly available research databases. These include IEEE Xplore and Springer Link, both of which contain scientific papers that have been peer-reviewed. The exception to this are are the ISO standards and the literature, which are not available in any of these databases. The advanced search in the databases allowed us to easily sort through the most cited papers and sort specific keywords in either the title or content of the paper. By combining keywords such as microservices and different quality characteristics, we can find potentially relevant papers.

In the search for papers, we first tried to select a starting point for the literature review. This was done by looking at the most well cited paper we could find using the search terms such as microservices, quality and web application.

By using a paper as the starting point, we could systematically find more papers that may be of interest through the use of snowball sampling. This process involved us looking at the references of that specific paper and the studies that have previously cited it.

This process was repeated multiple times throughout the entire literature review and allowed us to gather a large quantity of research papers. Papers mentioning the quality characteristics of microservices, web applications or anything closely related were considered for the literature review while everything else was disregarded. The following is a summary of the literature review for each research question. All literature described was found using the aforementioned method.

4.1 The Quality Characteristics

The product quality model described in the ISO standard [12] is an international standard for software quality characteristics. Therefore, we have chosen these

characteristics as the original pool to be selected from. Consequently, from this pool, we will now find the characteristics most suitable for use as a measurement of the quality for the empirical study.

Table 1: The product quality model.

Functional suitability

Performance Efficiency

Compati -bility

Usability Reliability Security Maintainability Portability Functional

complete- ness

Time-behavior Co- existence

Appropri -ateness recogniz- ability

Maturity Confiden -tiality

Modularity Adapt-

ability

Functional

correctness Resource

utilization Interoper

-ability Learna-

bility Availability Integrity Reusability Install- ability Functional

appropriate- ness

Capacity Opera-

bility

Fault tolerance

Non- repudiati on

Analyzability Replace- ability User

error protect- ion

Recover- ability

Account- ability

Modifiability

User interface aesthetics

Authent- icity

Testability

Accessi- bility

Laprie [13] writes in 1995 that the measure for the dependability of a system is reliability, availability, maintainability, and security (safety). In a research paper written by Offutt in 2002 [14], he conducted a survey in which he had asked web software development managers and practitioners what they thought were the most important quality criteria. They believed that reliability, usability, and security have the highest priority, while availability, scalability, and maintainability were

important but not as high of a priority.

Even if cloud computing and microservices are two different technologies, they are usually used in conjunction with each other and are both a result of the needs of today’s software industry. This makes it possible to indirectly find relevant quality characteristics for the microservice architecture by looking at the key attributes of cloud computing systems. According to Pan and Hu [15], the dependability of cloud computing systems includes five key attributes, namely, availability, reliability, performance efficiency (performability), security, and recoverability, all of which can be found in the product quality model as either a main quality characteristic or a sub characteristic.

In many books which address the microservice architecture [1] [16] [8], the

importance of simple deployability of today’s applications is strongly emphasized.

Simply deployability is what a microservice application should allow if it is implemented correctly. This seems sensible since one of the primary goals of the microservice architecture is to enable the services of the application to be deployed independently of each other. This characteristic is practically identical to the

portability characteristic that can be found in the ISO standard. Based on this, it can be assumed that portability is undoubtedly one of the more important quality

characteristics to focus at when measuring the quality of a microservice application.

Dragoni et al. [11] describe how microservices affect some quality characteristics, both positively and negatively. One of the most significant negative effects that it might have on an application is the performance efficiency and, more specifically, the time behavior. This is mainly because microservices rely on communication over a network which introduces latency.

In a paper written about microservices for The Internet of things [17], it is mentioned how the monolith is very troublesome to maintain and scale. This is in contrast to microservice applications which have very strong scalability, maintainability,

interoperability, and deployability. Especially interoperability and deployability since each service can use its own technology and have a fast deployment time.

In another paper written about microservices for a specific E-commerce website [18]

it is explained that, with the use of the microservice architecture, it is possible to achieve very strong scalability, agility, and reliability.

Although the coverage of these references may not be the largest, by primarily focusing on highly cited papers we are able to get a general view of what the industry might think. Thus, it is possible to draw a conclusion about the quality characteristics that can be interesting to use in the evaluation.

We focus the literature review on highly cited articles, and thereby assume that there is at least an implicit agreement that the used quality characteristics are relevant for the microservice architecture. By re-starting the search process with different search terms (as described in Section 3.1) we lessen the risk that we might only be finding articles from a specific sub-group or studied of a smaller sub-set of quality

characteristics.

Quality Characteristic Mentioned By:

Performance Efficiency [15] (performability), [11]

Compatibility [17]

Usability [14]

Reliability [13], [14], [15], [18]

Security [13] (safety), [14], [15]

Maintainability [13], [14], [17], [18] (agility) Portability [1] [16] [8] [17] (deployability) Table 2: Quality Characteristics discussed by the selected papers.

4.2 A Description of the Microservice Architecture

The microservice architecture follows the same principles as any other service- oriented architecture. It divides the application into multiple parts, each of which serve its own purpose. The definition introduced in the introduction will serve as the basis of this architectural analysis [2].

While scalability is not a quality listed in the product quality model, to understand the goal of microservices, it should first be understood. Scalability is not an

independent quality, rather it is the quality of how a system can continuously meet all its qualities as the user demand and the capacity of the application increases,

therefore it is not possible to measure the scalability by itself. The scale cube [4]

describes the different ways a system can scale with three different axes.

• The x-axis represents the distribution of the same work or mirroring of an application across multiple entities.

• The y-axis represents the distribution and separation of work responsibilities by “verb”

or action across multiple entities.

• The z-axis represents distribution and segmentation of work by customer, customer need, location, or value. [4]

The y-axis is the axis directly related to the microservice architecture and is often seen as one of the primary reasons for choosing it over other architectures. If

functional decomposition can affect the scalability positively to such a large extent, it is worth considering how it would affect the other quality characteristics of the application.

Figure 1: An example of scaling-out an application using two different architectures.

One of the most important aspects of the microservice architecture to consider is the messaging between services. Since each service is completely isolated, normal communication that would work in a monolith becomes impossible. Instead, a message mechanism is needed. A common method of handling messaging is to use RESTful APIs for the communication between the services, this means that the services only interact with each service’s interface. One of the many advantages of using this form of messaging is because each service can use completely different technologies, the interoperability of the entire application is greatly improved [19].

Although there are some older architectures such as the monolith, which have been used for decades, microservices provide several advantages over it. Some of which include scalability, maintainability, and reliability. These and the following

characteristics are affected by the microservice architecture in the following way [11]:

x In theory, the use of small services will have a positive effect on the availability but the complexity of having many services instantly available hurts it.

x Reliability may be greatly affected, depending on the simplicity of the integration mechanism and the reliability of the messaging between services.

x Maintainability is an enormous benefit of using microservices because the independent services are easier to change, fix, or even add new functionality for.

x Since microservices rely on communication over a network, the performance is especially negatively affected compared to a system that uses in-memory calls. The impact on performance will vary, depending on how loosely coupled the services are and how often messages are sent between them.

x Using messaging over a network negatively affects security. This means that microservices need additional security mechanisms.

x Since each service can be tested individually, testability is improved.

There are other qualities such as modularity [19], reusability, and deployability [8]

which are not only improved by the loose coupling of the microservices but also largely attributed to the use of virtualization techniques such as containerization which allow each service to run in complete isolation. This means that each service of the system can be modified, deployed, and even easily reused completely

independently. This also provides major advantages, such as improved scalability, rapid deployment of additional features, and greater flexibility. In this way, the system can be easily scaled through mechanisms such as auto-scaling with orchestration tools.

Notably, there are some qualities, such as recoverability and fault tolerance, that are very important and directly affected by the microservice architecture, but also by some external tools commonly used in conjunction with it which affect these

characteristics even more. As stated in [5], node shutdown and node recovery should not be the responsibility of the application, and instead, be the responsibility of the container orchestration tool, which manages all the individual microservices.

4.3 How to evaluate the quality of a system

The previously selected ISO quality model has a standard on how to evaluate the quality of an application [20]. Although this evaluation of different applications focuses on how the microservice architecture affects these characteristics, looking at ISO’s description of how to measure them will allow for a better understanding of how we can measure the selected quality characteristics.

It is worth mentioning that this empirical study only uses the ISO quality model as a guideline for the different quality characteristics an application might have. The evaluation performed is by no means a definitive way on how to evaluate all microservice applications using quality characteristics from the product quality model.

The standard includes a detailed description of every sub-characteristic of each quality characteristic. For each description, there is a table, which usually contains more than one method to measure that specific sub-quality and even some additional notes for things that might be unclear. For example, there are nine different methods to measure the operability sub-characteristic in conjunction with nine different measurement functions. The different measures for each sub-characteristic can be quite useful since they allow us to evaluate each characteristic according to an industry standard. However, only the measures that can be used theoretically will be selected since we are not performing a real time evaluation. Additionally, it is important that we ensure that each measure work in the context of a microservice application since this is no guarantee due to it being a general standard for all software. Moreover, it is important that each measure works in regard to the scalability quality previously mentioned in section 4.2.

Example of a useful measure and its description in the ISO standard:

Coupling of components - How strongly are the components independent and how many components are free from impacts from changes to other components in a system or computer program? [20]

4.4 Summary

We have been able to select a set of quality characteristics with the use of the product quality model [12] and a literature study. These characteristics have been deemed relevant for the usage as a measure of microservice applications and are used throughout the rest of this study. Furthermore, we have explained the scalability aspect of microservices since it is the primary reason for the architecture’s existence.

However, it will only be used as a guideline for how we measure the other quality characteristics, since it is merely a measure of them and therefore cannot be measured independently. Additionally, we have reviewed literature for how each selected quality characteristic is affected by the microservice architecture as

preparation for the evaluation of different microservice applications. Lastly, we have looked at an international standard for how to measure the quality characteristics and their sub-characteristics.

5 E

M

A

5.1 The Selected Quality Characteristics

Now, the information collected in the literature study can be used to select the appropriate quality characteristics through a quantitative analysis. But it is important to remember that different characteristics may have varying importance for different applications. For this evaluation, the selected quality characteristics will be those that should be relevant for most microservices applications.

Performance efficiency is an interesting characteristic because the literature review indicates that it is a highly relevant metric to use when measuring a microservice application [15] [11] . Unfortunately, it is difficult to compare the performance of different applications since the evaluation is done theoretically rather than in real time. Therefore, the performance efficiency measurement will strictly focus on how theoretically, the time behavior of the application is affected.

Five other qualities from the product quality model will also be used as the metrics for measuring the quality of different microservice implementations since they all have strong relevancy indicated by the literature review. These characteristics

include compatibility, reliability, security, maintainability, and portability, which are all strongly emphasized by different authors in studies related to web applications and microservice applications.

In the literature study, we were unable to find any substantial evidence that would suggest that functional suitability and usability described in the product quality model are relevant for this evaluation. Therefore, these two quality characteristics are disregarded from the evaluation and comparison of the microservice applications.

To summarize, from the product quality model, performance efficiency,

compatibility, reliability, security, maintainability and portability have been selected as the metrics to use for the evaluation of the microservice applications.

5.2 Measuring the Microservice Architecture

Now that the metrics for measuring the different implementations have been selected, it is possible to analyze how microservices affect these quality characteristics. The following analysis is based upon the previously conducted literature review in section 4.2.

Performance:

When adapting the microservices architecture, all services of the application will have to be loosely coupled. This means that communication can no longer be done through in-memory calls. The only way for the microservices to communicate with each other will be over a network, which will increase the latency of the

communication between services by a great amount compared to in-memory calls.

Therefore, the application must minimize the number of messages sent between each service to prevent the time behavior of the system from being severely affected. This

can be accomplished by reducing the responsibility of each microservice and avoiding tight coupling to ensure that communication between services is restricted to the lowest amount.

Compatibility:

Microservice implementations often rely on containerization to isolate each service of a system. This means that the co-existence of the different services on the same hardware works flawlessly because they all run as their own programs with their own technologies.

Since all services are completely isolated, a messaging mechanism is necessary to allow communication between services. Using technologies such as RESTful APIs for the messaging can allow for strong interoperability since all exchanges between each service are done using HTTP requests which are supported by most modern technologies.

Reliability:

Having small services should increase the availability, however, using many services will increase the complexity, which makes it difficult to have all of them instantly available. Additionally, there is no guarantee that a service will always receive a response when messaging another service.

The Recoverability and fault tolerance of a microservice application is mostly enabled by the loose coupling of the microservices. By using microservices, the system is divided into independent nodes, which can be used in conjunction with external node orchestration tools to manage each node. Additionally, since failure in one node will not cause a failure in the entire system, and because each microservice can be deployed independently, fault tolerance and recoverability are very strong for the application.

Security:

Choosing to use the microservice architecture will most likely negatively affect the security of the system due to the communication being over a network. Since the messaging mechanism is what allows the different services to communicate with each other, security becomes a necessity to implement for the messaging to ensure high confidentiality and integrity. Confidentiality is of the utmost importance to guarantee that the data sent can only be accessed by authenticated parties. A high degree of integrity is equally important to guarantee that all data sent is protected, using methods such as encryption. Security becomes even more important if the microservice application uses an open network for the communication between services since there is a greater risk of intrusion.

Maintainability:

Having the microservices be loosely coupled and running in their own virtual environment such as a container yields strong modularity and reusability for the application.

A microservice application will also have strong analyzability, testability, and modifiability since each service should be completely isolated. This means that each

service can be analyzed, tested, and changed individually and any issues will be easy to find since the issue is limited to only one service.

Portability:

The installability of microservices is very strong because it is very simple to deploy each service independently since it is usually loosely coupled from the other services and running in a container. For the same reason, adaptability and replaceability are also strong, because each service is completely isolated which means that an update in one service will not affect any other services. This allows developers to

independently roll out updates for each service.

5.3 Evaluation of Implementations

According to the architectural analysis, the most important factor that needs to be considered when selecting a microservice application is the approaches used for the messaging between microservices. Specifically, messaging can be done through synchronous communication or through asynchronous communication. Therefore, in order to make a meaningful comparison between applications, we have selected two applications which use these two different approaches for the messaging. Particularly in this case, one is using event-driven messaging for asynchronous communication while the other one is using REST APIs for synchronous communication.

This empirical study includes an evaluation of two microservice applications built using completely different technologies. The selected applications are

eShopOnContainers (asynchronous) and Piggy Metrics (synchronous). Below, a description of each application will be given to introduce its most important

components. Additionally, each quality characteristic will have a brief description of how the application affects it. Lastly, a table at the bottom summarizes all the

qualities of the application.

5.3.1 eShopOnContainers Information:

EShopOnContainers is currently still under development and will be so for the foreseeable future. The evaluated version of the application is the latest version at the time of this study, version 3.1.1 updated for .NET 5 and the edition used of the documentation is v5.0.

Repository: https://github.com/dotnet-architecture/eShopOnContainers Description:

EShopOnContainers is an open-source cross-platform .NET sample application, mainly focusing on microservices and their deployment with containers using

docker. The application is dedicated to achieving loose coupling and contains a large number of isolated microservices, each with its own dedicated database. The HTTP requests sent from the clients are handled by the API gateway, which forwards each request to the appropriate service. The API gateway is implemented using Ocelete, which is specifically designed for microservices and is also quite lightweight. The communication between services is handled by the event bus, which uses the

publish/subscribe method and is implemented with RabbitMQ. The event bus enables services to communicate asynchronously, which means that the microservices must rely on eventual consistency between each other. Additionally, each microservice of the system is continuously monitored for failures. This is to ensure that one service can always guarantee that the other service is in good health before utilizing it.

Moreover, eShopOnContainers integrate different patterns, such as the retry pattern to allow services to retry with the expectation that the service it is communicating with will eventually respond and the circuit breaker pattern to stop services from performing operations that are expected to fail. Lastly, all containers of the application can be deployed on the same docker host by simply using docker- compose or container orchestrators such as Kubernetes to deploy the containers on multiple clusters.

Performance:

The performance of this particular application is certainly not an afterthought.

EShopOnContainers did not choose to use REST APIs for the messaging but instead uses event-driven communication with an event bus. This allows for asynchronous messaging, which can greatly improve the performance compared to synchronous messaging with HTTP calls through REST APIs. The reason for this being that by using the publish/subscribe mechanism, messaging between services becomes non- blocking, which allows services to do other things while waiting for a response. This would have been especially noticeable when a fast service must wait on a response from a slower service.

The indirect messaging between microservices and the containerization of each service also increases the loose coupling, which helps improve the performance.

Furthermore, eShopOnContainers uses the retry pattern which can potentially seriously affect the performance as a result of large amounts of retries causing a slowdown in the service. The circuit breaker pattern tries to eliminate some of this performance loss by predicting when it is pointless to continue retrying.

Compatibility:

The co-existence in eShopOnContainers is undoubtedly very strong because it was specifically designed with docker containers in mind. By placing all microservices in their own separate virtual environments, sharing the resources of the docker host becomes simple since the processes are completely isolated from each other.

As expected from an application that uses microservices, strong interoperability is one of its major advantages. Messages cannot be directly exchanged between each microservice and instead services use the event bus to communicate indirectly. By having completely independent microservices, any technology used in conjunction with any pattern can be used to implement each individual service. All services within eShopOnContainers are built using .NET, but there is no limitation in this specific application that would not allow services to use a completely different technology.

Reliability:

Generally, considering the complexity of managing microservices, availability should be a major concern for most microservice applications. Fortunately, since the

entire application is built using only a limited amount of microservices,

eShopOnContainers is not very complex. This means that this application has the additional benefit of having an above-average availability for a microservice application, by not being too complex and having very small services with limited responsibility.

An interesting observation is that all four SQL databases are running in the same container. Although this may reduce some of the memory required for additional containers, it does reduce the overall availability of the system. If the one container fails, all SQL databases in the same container will become unavailable.

In addition, the event bus must be taken into consideration since it is potentially a single point of failure. If the event bus fails, then all microservices will be unable to communicate with each other.

The key to ensuring that eShopOnContainers have a high fault tolerance lies in the messaging, the eventual consistency, the circuit breaker, and the containerization. By making the application dependent on eventual consistency and using non-blocking communication through asynchronous calls rather than long chains of synchronous calls, a failure in one will not completely prevent all other microservices from operating. The circuit breaker mechanism also improves this because it prevents unnecessary retries to an unhealthy microservice.

The health checks used by eShopOnContainers for each microservice ensure that unhealthy services will never go unnoticed. This is very important to allow unhealthy services to recover. Although internal health checks are used, additional health checks can also be performed with an orchestrator such as Kubernetes which would stop forwarding requests to the unhealthy container while the container is being fixed or replaced.

Security:

The eShopOnContainers documentation mentions that in the future, authentication may be added to the API gateway. This would allow it to only forward authenticated clients to the microservices, but such a feature does not currently exist. Currently, the application relies on the OpenID package for authentication and JTW for

authentication tokens between microservices.

Additionally, eShopOnContainers uses the azure key vault to keep the secrets

between microservices secret. This ensures that keys are encrypted, and unauthorized access is not allowed which strengthens the integrity of the system.

Maintainability:

EShopOnContainers has a strong maintainability with all of its sub-characteristics positively affected. This can be seen by simply observing how loosely coupled the application is with its indirect messaging and the containerization of each service.

This is of course hardly surprising considering that one of the major goals of the microservice architecture is high maintainability.

Portability:

EShopOnContainers was built with containers in mind which make the installability strong since all microservices are containerized. Additionally, services being loosely coupled allow them to be deployed individually rather than everything at the same time. Thus, the portability of eShopOnContainers is excellent.

The containerization also makes the adaptability and replaceability very good since any environment supporting docker technologies will also support the deployment of eShopOnContainers.

Quality Aspect Affected Positively By Affected Negatively By Performance Event-driven

communication, loose coupling, circuit-breaker.

Retry-pattern.

Compatibility Containerization and loosely coupled microservices.

Nothing.

Reliability Architectural simplicity, messaging, health checks.

Four SQL databases in the same container and the event bus.

Security Authentication

mechanisms and secret integrity.

No authentication for API gateway.

Maintainability Loose coupling and

containerization. Nothing.

Portability Containerization, loose

coupling. Nothing.

Table 3: A summary of the quality characteristics of eShopOnContainers.

5.3.2 Piggy Metrics

Information:

The version evaluated is the latest release at the time of this study, which was updated for Spring Boot 2.0.3. All documentation can be found on its public GitHub page.

Repository: https://github.com/sqshq/piggymetrics Description:

Piggy Metrics is an open-source java application built using Spring Boot and Spring Cloud. Like most other microservice applications, it uses Docker to containerize each microservice to better isolate it. There are three core microservices, and each of them has its own private MongoDB database to persistently store data. These three

services can only be accessed through the API gateway, which forwards the request to the appropriate service. The gateway was built using the ZUUL technology developed by Netflix, which works nicely with Spring Cloud. Equally important is the service discovery built using Netflix Eureka. The Eureka server acts as a service registry which is necessary for communication to the microservices. Each

microservice must register itself before it can be interacted with through the use of Feign to send HTTP requests. Furthermore, by using Histrix, Piggy Metrics can implement the circuit breaker pattern, which helps reduce the load on services that

are under heavy load. Additionally, Histrix also allows the application to implement a monitor for the circuits of each service. The information from all microservices can then be easily displayed on a dashboard implemented using Turbine. Lastly, this application was developed with containerization in mind. The deployment of the entire application can easily be done by using docker-compose, which creates a container for each microservice.

Performance:

Piggy Metrics solely relies on REST APIs for the communication between each service. This means that the message delivery is synchronous, and although this is one of the least complex forms of messaging, it has a significant negative impact on performance. When a service, such as the notification service, wants to use the account service, it must send an HTTP request. The sender then expects to receive a response from the receiver of the request. This prevents the notification service from doing any work while waiting for the response, which means all microservices highly depend on each other. A slowdown in one microservice has the potential to cause a chain reaction, thereby slowing down the entire application. Since this messaging mechanism leads to tightly coupled services, the optimization of each microservice becomes challenging. However, Piggy Metrics tries to reduce some performance losses by using the Circuit Breaker pattern, which can help the recovery of unhealthy services.

Compatibility:

Piggy metrics use containerization for each microservice to put it in its own virtual environment. Not only does this isolate it from all other services but also other applications using the same hardware. As a result, the co-existence of this application is very strong.

At first glance, it may seem that the interoperability is rather weak because

everything is implemented using Java Spring and Java Spring compatible libraries.

However, Netflix technologies, such as the Eureka service discovery, allow the usage of non-JVM languages if used with Netflix sidecar. But this means that certain components such as the health monitor must instead be implemented separately for the non-JVM microservices. The communication between the microservices is accomplished by interacting with the REST API of each service, therefor if a service can handle the JSON format, there should be no issues sending requests to it.

Reliability:

The simplicity of Piggy Metrics can be considered one of its greatest advantages.

Although it is still far from the simplicity of a monolithic application, it is undoubtedly one of the least complex ways to design a microservice application.

Since the functional components of the application are only three microservices, one API gateway, and a service registry, the availability should not be negatively affected at the integration level. Not only is it simple, but each microservice also has limited responsibility and size, thereby reducing the fault proneness. However, this

simplicity also has a disadvantage. By making every microservice heavily dependent on the service registry, the availability suffers. Failure in the service registry might cause failure for the entire application.

Even if synchronous communication is easy to implement, it may introduce many problems when the system is under heavy load. A single failure in any microservice will cause a chain reaction that would affect other microservices. When using HTTP requests, a response is always expected, therefore a blocking situation can occur if no response is sent back. Eureka tries to prevent this by checking the health of the microservice before the request is sent to stop the request from being sent to an unhealthy microservice. In addition, circuit breakers are integrated with Histrix.

By using the monitoring integrated with Histrix, the system can easily recover from issues with the microservices. Once a microservice is deemed unhealthy, a new microservice can be deployed in order to replace it. Moreover, since each MongoDB instance is running in its own container, the data of a microservice can persist even when it is replaced.

Security:

The authentication for Piggy Metrics is handled by a microservice that has the authentication as its sole responsibility. This microservice grants OAuth 2.0 tokens so that only authorized users can access certain resources. The authorization microservice is also used for communication between microservices to add additional confidentiality to the application.

In the documentation, it is mentioned that using a more advanced configuration for security would be beyond the scope of this particular application. Additional security would most likely greatly increase the complexity of the entire application, so

considering this is just a sample application, it seems sensible that the security was not highly prioritized.

Maintainability:

Even if Piggy Metrics follows the microservice architecture, it is not as decoupled as one might expect from such an application. The microservices themselves are very simple and have limited responsibility. The problem lies in how tightly coupled they are together. By letting the microservices communicate directly with each other through REST APIs, they become too dependent on each other. This is also called an anti-pattern. In this scenario, it means the application is more similar to a distributed monolith rather than a microservice application in regard to its coupling.

Unfortunately, even with the use of containers the overall maintainability quality is very weak because of the tight coupling, a change in a microservice cannot guarantee that nothing else will be affected.

Portability:

Since each component of the application is in its own separate container, it is easy to either deploy all of them together with docker-compose or even individually one by one. Furthermore, these containers can be easily redeployed and replaced

independently because of the containerization.