Creating a method to evaluate frameworks used to build web applications

60  Download (0)

Full text


Creating a method to evaluate frameworks used to build web applications

Tony Eriksson

Tony Eriksson Spring 2020

Degree Project in Interaction Technology and Design, 30 credits Examiner: Ola Ringdahl

Master of Science Programme in Interaction Technology and Design, 300 credits


Single page applications are a new kind of applications, these run in almost any web-browser and can therefore use the same code-base in- dependent of the platform that they are used on. To create a Single Page Application a framework for development is often used. Today there ex- ists a wide variety of such frameworks, most with their own specialties and drawbacks. How well different development frameworks made to create Single Page Applications fits a project depends on the situation.

This work is creating a framework to evaluate how well a development framework fits in any given project. Blazor are a new framework that can be used to create Single Page Applications. Once the evaluation framework were defined it was used to evaluate how good Blazor would be if it would be used to develop a specific application. By doing this the framework are tested and at the same time knowledge about Blazor are obtained. The work is concluded by application of the developed framework and showing that Blazor are a good fit given the scenario.

This evaluation also demonstrates the power of the evaluation frame-

work and leads to the conclusion that it is a good tool that can help

strengthen decision on whatever or not to use a specific framework.


I would like to express my gratitude to my two supervisors.

Henrik Sch¨afer for his valuable help to get me setup and started at Vitec and for his contin- uous interest in my work.

Anders Broberg for his ongoing feedback throughout this work.


1 Introduction 2

1.1 Project Partner 2

1.2 Problem Description 3

1.3 Project Structure 4

2 Pre-Phase and History 5

2.1 A introduction to Single page frameworks 5

2.2 Blazor 6

2.3 Single page applications 6

3 Building a Testing Framework 7

3.1 Scalability, performance and legacy support 7

3.1.1 Summary 8

3.2 Testability and debugging 9

3.2.1 Modularity and loose coupling 10

3.2.2 Summary 10

3.3 Community and documentation resources 10

3.3.1 Summary 11

3.4 Code quality 11

3.4.1 Defining Quality 12

3.4.2 Filling the criteria for Quality 13

3.4.3 Summary 13

3.5 Chapter summary 14

4 The Testing Framework 15

4.1 Scalability, performance and legacy support 15

4.1.1 Scalability and Performance 15

4.1.2 Legacy support 16

4.1.3 Numerically evaluating results 16


4.3 Community and documentation support 17

4.4 Code quality 18

4.5 Summary 19

5 A Theoretical Evaluation of Blazor 20

5.1 Measuring values of the framework 20

5.1.1 Context of evaluation 20

5.2 Estimating the performance, scalability and legacy support of Blazor 21

5.2.1 Expansion possibilities 21

5.2.2 Scale capabilities with regards to change in system load 22

5.2.3 Supporting the intended target group 24

5.2.4 Support for long lasting continued development 25

5.2.5 Section Summary 26

5.3 Evaluating the testability and debugging options of Blazor 26

5.3.1 Tight coupled elements 26

5.3.2 Toolset of the framework 28

5.3.3 Section Summary 30

5.4 Estimating community and documentation resources available to Blazor. 30

5.4.1 Community size 30

5.4.2 Angular 30

5.4.3 Stars on github 31

5.4.4 Value of open source 32

5.4.5 Documentation Quality 33

5.4.6 Maturity and adoption 34

5.4.7 Section Summary 35

5.5 Evaluating code quality with Blazor 35

5.5.1 Development environment support 35

5.5.2 Design pattern support 36

5.5.3 Section Summary 37

5.6 Theoretical Summary 38

6 A Practical Evaluation and implementation with Blazor 39

6.1 The value and premises of a practical evaluation 39


6.3 Program flow 40

6.4 The Prototype 41

6.4.1 Prototype structure 41

6.4.2 Final prototype 41

6.5 Implementation 42

6.5.1 Components 43

6.5.2 Debugging and testing 45

6.5.3 Implementation time 46

6.5.4 Implementation Result 46

6.6 Summary 46

7 Evaluating Results and a final Conclusions 48

7.1 Results of evaluation 48

7.1.1 Summary of results from the theoretical evaluation 48 7.1.2 Verification and Contradictions found by the Practical Implementation 49

7.2 Limitations in the work 49

7.3 Final Conclusion 49

8 Summary 51

9 Future work 52


List of Figures

1 tight coupled class without dependency injection 27

2 loose coupled class with dependency injection 27

3 Diagnostics panel from visual studio 2019 29

4 Growth of stars over time of aspnetcore on github 31

5 Growth of stars over time of angular on github 32

6 Percentage of question tagged with Blazor on Stackoverflow 35 7 Percentage of question tagged with Angular on Stackoverflow 36

8 System specifications of development machine 40

9 flowchart visualizing interaction flow in the original application. Due to the sensitive nature of the data all names have been blurred out. 41 10 One part of the prototype displaying its structure. Due to the sensitive nature

of some data all property names have been blurred out. 42 11 data-flow from parent to child in Blazor components. names have been

blurred out. 44

12 A page of the final product implemented using Blazor Server. Due to the

sensitive nature of some data all names have been blurred out. 47


1 Introduction

The tech-world is rapidly changing with new techniques and methods constantly being de- veloped and launched. This makes it possible for companies to consistently provide better and more involved experiences for their users on their homepages. When looking at web development the expectations on the user experience on tech companies homepages are high. To really stand out it is fundamental that the correct technique is chosen when their pages or web applications are built. Making a incorrect choice of the most appropriate framework for development can come with huge costs in both development costs and time spent. There is also the possibility that an incorrect choice leads to problems with frame- work limitations. The purpose of this work is to narrow down the risk that companies make these kinds of mistakes by proposing a method, or perhaps better worded, a framework to help guide companies in their choice of a development framework. In the making of this framework different methods and important aspects of development are explored and identified. By highlighting these parameters, they can be used as input parameters for the evaluation framework. After defining the framework, it will be applied and used to evaluate Blazor. Blazor is a framework for development of Single Page Applications. The theoretical findings are then further verified by observations and knowledge gathered from an actual implementation. By comparing the output of the framework with some practical imple- mentations the promises made by the framework can be compared and verified by insights gathered for real implementations. By doing this, the reliability of the framework is further tested.

1.1 Project Partner

The project was performed at Vitec, in Ume˚a. Vitec is a market leading company for Vertical Market Software in the Nordic region. The group was founded in 1985 and as of today they have around 700 employees. Vitec has an interest to transfer some of their applications from native implementations onto the web by remaking them in form of Single page applications.

This could have a variety of different benefits. One is that maintenance of the applications over different platforms becomes easier to perform. Another is that some of the older ap- plications can benefit from newer design patterns and principles during re-implementation.

Before the company moves their apps, they need to choose which underlying framework to

use. Since Blazor looks like a promising framework, but is not yet a widely adopted one,

more evaluations of its capabilities are needed.


1.2 Problem Description

Vitec have an interest in exploring and evaluating the possibilities with Blazor. Evaluation of frameworks for development is currently a difficult task that requires a variety of audits and evaluations. Because Vitec, like many other software developers, develops a variety of different programs for different platforms and different target groups this task often must be repeated and redone for each new project. This work seeks to solve this recurring problem of constant, time-consuming reevaluation by creating a general testing framework for this type of evaluation. By providing this kind of framework evaluation and decisions that otherwise would be both time and energy -consuming can be optimized and performed on a much faster phase.

After the evaluation framework is defined the problem at hand is solved by putting the framework to use and an theoretical evaluation of Blazor is thereby obtained. To strengthen the theoretical results that the testing framework provides, one of Vitec’s applications gets re-implemented using Blazor. By remaking one of Vitec’s existing programs they get a chance to see and evaluate some actual results of implementations done in Blazor. At the same time theoretical evidence found using the framework can be evaluated with an actual implementation as a foundation and thereby ensure the validity of the frameworks theoreti- cal output.

The research questions this paper ultimately aims to answer can be divided in to two differ- ent but highly related questions the first one is:

• Are there any benefits or drawbacks of using Blazor over other frameworks when developing single page applications?

This question is especially of interest to Vitec. To answer this question a method of eval- uating frameworks of this kind are needed. Without a given context there is no value in measuring if a framework is good or bad for development. All major frameworks are likely to be good if the only thing measured are how they work in general cases. therefore, in or- der to estimate if the framework is good for development a context describing the intended scenario are needed. There are many different sides to development and therefore when evaluating a frameworks capability all different aspects of development need to be consid- ered. These facts are forming a second question whose propose are to provide a way of answering the first question. This question can be formulated as follows:

• How can an objective method for evaluation of development frameworks be designed to give a context aware and complete evaluation?

When looking at development it is important to differentiate between what is produced by a

framework and what it is like to produce something with the framework. What is produced

are the product of work done within the framework, this is the product that is created by

work within the framework. This product is a direct result of the amount of work done with

the framework. Without constraints on the amount of work it is possible that a wide variety

of frameworks can produce similar outputs, therefore the possible output of a framework is

of less importance than the efforts needed to achieve such a result. To answer the second

research, question a Framework for evaluation of the efforts needed to achieve a given goal

with a given framework are going to be created.


The development experience and efficiency of working with a framework are far more im- portant than what can be produced by the same framework as time consummation are a key factor in most development. Therefore, the evaluation framework that are produced should be aimed at the developer’s experience and potential to carry out effective work. If the de- veloper can be more effective, less time needs to be spent. In order to create a good testing framework all essential parts of development with the toolset that is to be evaluated must be covered. In order to ensure that the framework really covers the needs of the developer, some developers have given inputs about which areas they feel are most important. This in- put combined with knowledge obtained from previous studies will be making the core of the evaluation framework and have been compiled into four different main categories. These categories are as follows: “Performance, scalability and legacy support”, “Testability and debugging”, “Community and documentation resources” and “Code quality”. In chapter three, previous studies in these areas are reviewed and what the areas included is defined. In chapter 4 of this work a testing framework is defined, whose goal is to provide a toolset for evaluation of all the about mentioned areas. One goal of this work is for the new evaluation framework to be applicable to evaluate any arbitrary coding framework. Thereby this work will provide a toolset for Vitec and other companies to use when they need to decide on which framework to use during any arbitrary development.

1.3 Project Structure

The rest of this work is structured as follows:

• Chapter 1: Gives a introduction to the project and presents the project description

• Chapter 2: present the history behind single page applications.

• Chapter 3: Identifies what and why different parameters are of importance during development with a framework.

• Chapter 4: Create a testing framework using the findings from chapter 3 for evalua- tion of frameworks aimed at development of single page applications.

• Chapter 5: makes an theoretical evaluation of Blazor using the framework created in chapter 4.

• Chapter 6: make a practical evaluation of Blazor based on findings from the evalua- tion in chapter 5.

• Chapter 7: Verifies the validity of the framework created in chapter 4 by evaluating its resoults found in chapter 5 with findings from the practical evaluation done in chapter 6.

• Chapter 8: gives a short summary of the overall work

• Chapter 9: presents possible future work


2 Pre-Phase and History

During this section a brief history of Single page applications and their development frame- works are given.

2.1 A introduction to Single page frameworks

The network we know as the internet is a place that is constantly evolving and changing.

In 93’ the whole network contained around a million servers, ten years later this number

exceed 400 million servers [1]. This growth has pushed development of tools for commu-

nication and interaction on the internet forward with a rapid speed. Basic communication

techniques such as Email and Ftp where actuating tools that allowed the masses to connect

and interact from all over the world. Today we have come a long way from the common

web hosting models of the early days. The launch of GeoCities [11] can be seen as the

starting point of when individual users got the ability to publish their own content on the

web. Today this ability can be confederated given and not only are many publishing their

own content, but some are also operating their own servers. With the millions of webhosts

that now exists on the internet there is a constant need for tools and frameworks designed

to publish and operate webpages. In the last decade, several frameworks have been devel-

oped to help the user’s create content for the web that is good looking and also have the

capabilities of more classical desktop programs. This kind of webpages are often known as

Single Page Applications (SPA) [20] and are probably a big part of the future of the web as

they have a wide range of capabilities that common webpages lack. Single page applica-

tions can, and often are, designed and used as desktop applications. A big strength to these

applications is the capabilities they must run on different devices. A single page application

does not need any native code for different platforms, every platform that got a modern web

browser available are able to run them. This gives this kind of application a great benefit

when it comes to cost of development. Single page applications are often developed on top

of a framework, today there exists a few different frameworks that are open for the public

to use. Exactly how adopted and used these frameworks are is hard to tell, but by look-

ing at the star markings on Github[12] an approximation of the size of each project can be

obtained. The largest ones. as of today, according to their Githubs star rating are Angular

[2], React [22] and Vue.js [24]. All these frameworks are based on and programmed in

JavaScript [7] (or Typescript[4], that is a subset of Javascript). This is introducing a big

limitation as there exists many programmers that is specializing in backend languages other

than Java. When it comes to platform specific application development C# [18] is a com-

monly used language. As application development are moving more and more towards web

and SPA solutions there is a need for frameworks that is using other backend languages

than JavaScript in order to attract already skillful application developers. This is probably

one of the reasons why Blazor[8] was developed. Blazor is like Angular, Vue and React a


framework for building webpages and web-applications but differ in the backend language.

Instead of using JavaScript Blazor is using C#, allowing programmers already skilled in the language to move on to web development without changing development language. Bla- zor is a completely new framework created in 2018 and built as a feature of ASP.NET[8]

that extends the .NET development platform [8]. As Blazor is such a new technique little is known about its performance, usability, strength, and weakness. This missing area of knowledge is what this work is going to focused on. Getting a deeper understanding in this area not only needed to answer questions about how well the framework preforms but can also be crucial when new projects needs to decide which framework best suits their needs.

2.2 Blazor

Blazor is a completely new framework for web development focused on Single Page Ap- plications. The Blazor framework allows C# code to be used instead of JavaScript when rendering webpages. This allows for programmers that is already skilled in development of native programs in C# to move onto web-development without changing their development language and thereby preserving already existing knowledge and skills in these developers.

Being able to transfer knowledge and code from other development projects opens up for a great variety of new possibilities, especially when it comes to development of single page applications.

2.3 Single page applications

Single page applications or SPA:s are applications that is developed to run on the web. These

are often designed to have all their logic executed on the same web address, or a smaller

collection of subpages. Another thing that make SPA:s differ from classical webpages are

the more logical function execution that these often do, making them feel and look more like

classical desktop applications than webpages. A core advantage with SPA:s over classical

applications are the number of devices that they are able to support. As all that is needed to

run them are a modern web browser there is almost no extra work needed in making them

run on different systems or architectures. Classical applications on the other hand often

must be completely redone to be able to do the same thing, this requires countless hours of

extra work and effort.


3 Building a Testing Framework

In order to construct a well-functioning evaluation framework, critical and universal ele- ments must first be identified. Through discussion with well-established developers at Vitec and studies of common development patterns some important areas have been identified:

“Performance, scalability and legacy support”, “Testability and debugging”, “Community and documentation support” and finally “Code quality”. By building on these areas, a good framework for evaluation from a developer’s perspective can be created. For this framework to be properly creates it is necessary to specify each of these areas on a more scientific basis and look at conclusions and methods found and used by others work on each of these areas.

Only when the intentions of each area are well understood can a relevant and well-formed framework be created. The following sub-chapters intend to specify what defines each area, how they can be tested appropriately and of what importance to overall development they have. The actual Framework for evaluation is presented later in this work are derived from the knowledge obtained during this chapter.

3.1 Scalability, performance and legacy support

In the work made by Bondi A. B., -”Characteristics of scalability and their impact on per- formance” [5] scalability is described as a subject with two different aspects. The author defines one as ”Structural scalability”, a system’s property to be expanded without major modifications to its architecture. The other one is ”Load scalability”, how well a system preforms when the traffic is scaled up. When the assessment of whether a system has good properties or not is made, this classification of scalability suits well as it is divided into two distinct parts that each can be tested separately. in ”A framework for characterization and analysis of software system scalability” [10] the authors describe that scalability is a term that is often found in the context of computer literature but rarely is well defined, in this paper a framework for measuring scalability is presented. The authors framework is well defined but focused on a more advanced level than what is necessary for the overall per- spicuous evaluation framework that this work comprises. In their paper scalability testing is described as a multi-criteria optimization problem where this work limits it as a problem divided into the two parts previously mentioned.

When looking at ”Structural scalability” a variety of sub-attributes belonging to this def-

inition can be identified. The way in which code is written is one such attribute that also

holds a great importance for the overall project. This attribute is also overlapping with some

important qualities found in the ”Code Quality” part of this work. The ability to divide code

into reusable parts are of great importance as this improves the ease of which the system

can be scaled and the overall readability of the code. With insight that the different areas

chosen inevitably to have some overlapping attributes comes an understanding that certain

properties of a framework may be important in a variety of cases, and this also means that


some evaluated areas will have certain dependencies between each other.

Legacy support is another term that can be seen as an attribute of ”Structural scalability”.

Good Legacy support does not only mean that the framework can target and build for older architectures but also that upcoming future versions of the framework keep having good support for code designed for the current versions. The importance of good legacy support is especially valuable for long-lasting project and projects whose target group is one that is using older hardware or software. Bad legacy support might lead to a direct exclusion of a great part of the intended target group, which in turn also hinder the system’s ability to scale to its users.

”Load scalability” got a different set of attributes that is of equal importance. One of the most important attributes of this category are the ability to accept large amounts of users simultaneously. The rate of which the system load is changing depending on the current number of connected users is a good indicator of the performance of this attribute. In the paper ”The Future of Systems Research” from 1999 by John Hennessy [13] it is suggested that ”scalability is an aspect of research that should receive more emphasis in the future than performance”. Today, over 20 years after the article where published this is at least partly true, with processor performance expanding at a rate close to what is described by Moors law [21], the speed of operations are of less importance while the resource usage in terms of memory are more important. To put this in context with an example: it does not matter if a system can calculate the first million decimals of pi or calculate an optimal route from point A to point B in fractions of a second, if the same system cannot be scaled to more than one user without investing enormous resources. In most framework related cases a rule is that: if a system cannot be scaled up given a reasonable amount of resources it does not matter how good its performance is. This example illustrate how performance does not matter as much since scalability can have a more important impact on the overall performance of a system then the actual computationally performance of components in the system. Without possibilities to scale and acquire sufficient memory the system will not work or at least will not perform as good as expected.

3.1.1 Summary

Summed up, the attributes that are derived from the system’s structure and its backward compatibility and its scale factor are tightly bound to each other. A deviation in any of these areas can therefore have a large impact on all of them. To measure and evaluate these attributes it is of great importance to have a core understanding of some contextual attributes of the work. These attributes are:

• The projects expected life-cycle

• Comprehensive information about the target group and its size

• A good estimate of the potential simultaneously user-count and server load on the



3.2 Testability and debugging

The IEEE definition, given in the IEEE Standard Glossary of Software Engineering Ter- minology[15] of the word Debug are “To detect, locate, and correct faults in a computer program. Techniques include use of breakpoints, desk checking, dumps, inspection, re- versible execution, single-step operation, and traces”. Even though the IEEE definition in the context of computer science are quite old the core meaning remains the same. To debug a program, process, or anything else is to use the correct tools given the context to identify and correct existing problems. In the article ”Improving the Testability of Object-oriented Software during Testing and Debugging Processes” [23] by Sujata Khatri, R.S. Chhillar and V.B.Singh Testability is described as “Testability is the probability whether tests will detect a fault, given that a fault in the program exists”. This definition of the word clarifies the importance of good testability as it establishes the link between testability and debug- ging. With bad testability options, the tools for debugging are inadequate and as a result debugging will not be as effective. This in turn results in it being more likely that unwanted behaviours and bugs avoid getting detected or never gets properly solved at all. This high- lights the importance of having good testability and debugging options during development.

Both testability and debugging are critical parts in developing good and working products.

The above statements do only describe the importance of good testability and debugging but omits the problem of actually identifying parameters and presenting ways of measuring these parameters. Identifying which parameters that is of importance when measuring these values on a generic plan that fits all cases might be impossible as different methods and frameworks are using different ways and tools for debugging. Therefore, the context of the framework that is being evaluated must decide what values and tools are of importance and which are not.

In ”A preliminary testability model for object-oriented software” the authors B. W. N. Lo and Haifeng Shi try to capture what factors that affect testability in object-oriented soft- ware. Lo and Shi group these factors in to three classes, namely: structure factors, com- munication factors and inheritance factors. Citing the description of these factors from the authors: ”Structure factors reflect the testability of coding structure of the program segment; communication factors measure the testability of object-oriented software by the degree of coupling among classes and objects; and inheritance factors contribute to the testability via the inheritance features of object-oriented design.”[16] the similarities to fac- tors when developing a SPA are easy to detect. Applying this model to development of SPA Structure represent the testability of the components that most SPA are constructed with. These are often modular and have a high reusablity-factor similar to parts of classic object-oriented programs. Communication can be applied in the exact same way as during object-oriented development (in fact one could argue that development of SPAs often are done in a object-oriented fashion) where the degree of couplings among components are measured. Inheritance factors are the hardest to directly fit in the context of SPA develop- ment as the inheritance features of object-oriented design do not always have to be present in the context of SPA-development even if it often are, at least to some degree.

Assuming that this factors proposed by B. W. N. Lo and Haifeng Shi are a good way of

measuring testability of programs created by a specific framework there is still modifica-

tions that needs to be applied as the objective with this paper is to test a framework before

any actual products is produced rather than after. More specifically the goal is to test the

testability and debugging potentials of a given framework, and not the testability of any


already created specific output from such a framework. By the realisations given from the previously mentioned studies it is clear that the testability framework outputs (eg. programs or completed SPA’s) are depending on the tools provided by the framework, it also seems that better support for modality is of great help when doing debugging and testing as errors more easily can be narrowed down to smaller sections.

3.2.1 Modularity and loose coupling

Modality is the frameworks ability to construct its outputs of loose coupled or completely sterile objects. This way when a bug or error is encountered the object that is producing the error can be debugged instead of the whole system. If a systems classes and object are dependent on each other they are considered to be tight coupled. systems that have a lot of tight coupled elements are in general harder to debug and make changes to since one element may produce the output of another or depend on behaviors of others. For a system to have a high grade of Modality it should have none or very few tight coupled elements.

3.2.2 Summary

This section concludes that some important areas within frameworks for better debugging and testability support are: Modality, the debugging toolset, the ability to trace errors and the degree of dependencies between components needed to accomplish tasks. Less depen- dencies are considered as better.

3.3 Community and documentation resources

The size of the community does not directly affect the quality of the development frame-

work, a Framework can have almost zero adoption and still be perfectly good for complex

tasks. Effects of a large community are best observed when looking at the indirect affects

to developing with a framework. In the post titled ”The size of open-source communities

and its impact upon activity, licensing, and hosting” [6] by Donnie Berkholz the size of

development communities and their effects are studied: The author comes to the conclu-

sion that larger project with larger communities tend to have more active communities as a

whole. This statement are in line with what Dustin Marx explains in his post ”Software De-

velopment Communities: Size Matters” [17] where its described how products with larger

communities often are chosen by client as they tend to have a better community knowledge,

be more established and therefor also be seen as safe options. Frameworks that are of an

Open source nature get even more value from large communities as these project often can

be partly or completely community driven meaning that there is no founding company or no

paid developers that drive the work forward. In these cases, everything in terms of contin-

ued development, bug fixing and documentation are in the hands of the communities. If a

product is widely adapted and have a large amount of users there exists a variety of different

advantages. The product itself are more tested and the chance that there exists good guides

and templates are higher. When the user runs in to a problem there is a much higher possi-

bility that someone else have had a similar problem and have provided a solution for it. The

documentation is often better as its often improved by users. The list of benefits can be made

long and almost every aspect shows positive effects when a product has a large community.


There are of course some negative effects as well, especially within completely community driven projects. When for example there is a need for fast changes, disagreements and dif- ferent opinions within a large community can be a problem. Overall, the positive effects of a large community out weight the negative ones and a frameworks community size should have a part in its overall evaluation but that create a need for methods of measuring the size of a community. As it turns out community size is a hard parameter to measure where there exist no directly available values. In order to get a estimation of this parameter data from large internet forums dedicated to development in general can be used. One of the larger sites that fits this description are StackOverflow. StackOverflow are a site where users can ask questions and mark these questions to the area that they are related to. StackOverflow are also providing an API with functionality to count the number of occurrences of different parameters on the site. This functionality allows for counting of the number of questions asked on StackOverflow regarding a specific framework, the ratio of questions that got an accepted answer and the growth rate of questions for the framework. These values can all be used to get a representation on the size of the community. Assuming that the distribution of users coming from different frameworks are in direct correlation with the total numbers of users of the framework estimates of the frameworks total adoption can be made. With this assumption different frameworks can also be compared to each other to get a represen- tation of the overall popularity of the framework. When it comes to measuring the overall popularity of the framework another good indicator should be the star making and activity on Github. Github are a software development platform where source-code and projects under development can be kept. Using measurements from Github only works if the frame- work that is to be evaluated is present on the site. Since many frameworks are open-source nowadays chances are that they can be found on Github. Grow rate and popularity are prob- ably time dependent and when comparing different frameworks it is therefore important to measure them with data taken at the same time from launch and not just use snapshots of the current day to get proper estimates of growth rates and overall adoption. This data can also be used to project a potential future growth of the framework.

3.3.1 Summary

Summarized the important factors that should be considered to get the best possible results out of a framework can be narrowed down to: the total size of the community, weather the source-code are open source or not, the overall quality of the documentation and the maturity and adoption of the framework.

3.4 Code quality

The ability to write good code within a framework is essential for the developers that are to

understand and expand the framework. It is critical for the overall development that code

is of good quality, with bad code quality it is likely that a project encounter development

slowdowns and suffers throughout its entire course.


3.4.1 Defining Quality

To understand the meaning of ”Code quality” it is first essential to understand the meaning of the general term ”Quality”. Quality is a rather general term and can be applied to a wide variety of different scenarios. In ”Juran’s quality handbook” [3] it is explained how the term Quality can be defined in at least two different ways that somewhat contradict each other. Table 1 is presented in this work and explains this rather well. Here two definitions for the word Quality are given that both are valid but have widely different meanings. In the context of this specific project the second definition is probably more applicable since it focuses more on development and reduced rates of internal failure. Following this definition most of the parameters of the second definition that is desired when producing code can be concluded from the list given by Table 1 as follows:

• Reduce error rates and Reduce rework waste are both related to ability to produce code with low error

• Reduce failures can derives from the ability to identify problems on a early stage before the get critical.

• Reduce inspections,tests, and audits are translatable to reduced needs of debugging and code reviving.

• Shorten time to develop new products is directly applicable.

• Increase yields, capacity correspond to a higher amount of good code produced by a developer over time.

• Improve delivery performance and Reduce customer dissatisfaction both correspond to the ability to deliver a better product with less prominent problems.

Table 1 Juran’s two definitions of quality, Juran’s quality handbook, 2017, p.50 Juran’s two definitions of quality

Definition 1 Definition 2

Higher quality enables organizations to: Higher quality enables organizations to:

Increase customer satisfaction Reduce error rates

Meet societal needs Reduce rework waste

Make products and services salable Reduce failures, warranty charges exceed competition Reduce customer dissatisfaction Increase market share Reduce inspections,tests, and audits

Provide salesrevenue Shorten time to develop new products Secure premium prices Increase yields, capacity

Improve delivery performance

The major effect is on revenue the major effect is on costs

Usually higher quality costs more Usually higher quality costs less


3.4.2 Filling the criteria for Quality

If a framework got the ability to fill the previously listed criteria it can be considered to have good code quality. How can these criteria be measured then? The ability to produce code with low error can be dependent on a variety of different factors, one of these are the Testability of the code. From the previously part about testability and debugging it is stated that being able to break down code into smaller parts is an important factor to filling these criteria. Another important factor is the help that is provided by the development environment. If for example proper highlighting of code and spellchecking are provided it is much less likely that errors caused by typos appear in the code. Therefore, part of measuring code quality should be measuring the performance of the Integrated development environment (IDE) that support the framework.

A good tool when it comes to detecting problems as early as possible is the ability to test the code early on. Unit testing is a form of software testing where building blocks of an appli- cation are tested on a as small as possible level, this is often done on individual components or classes. Unit testing is commonly used during object-oriented development. If parts de- veloped within a framework can be unit tested in a good fashion the rate of failure on larger compositions of component will be reduced. Therefore, the ability to perform unit-tests or similar low-level testing should be seen as a significant criterion in any framework designed to handle larger projects.

Following common patterns allows written code to be more readable and developers not fa- miliar with the project but the pattern itself can more easily join, understand, and contribute to a project. The MVC (Model View Controller) design pattern [9] is one well defined and adopted design pattern. In MVC the code base is split into three different parts:

• Model - This is the pure data.

• View - This consists of the user interface.

• Controller - This is the application logic.

Some benefits of using this pattern are the re-usability of code, not only in the current appli- cation but Views and Models can often easy be refactored and reused over multiply project as well. Another strength of this pattern is that it facilitated the opportunity to work in groups. Since the code is split-up developers can work in different parts without interfering with each other. There are many other well established and good patterns available besides MVC that can be used with the same result. Since these patterns provide major benefits during development a criterion for a good framework should be to support, at least, one major and well adopted design pattern.

3.4.3 Summary

When looking into what is required to be able to produce high quality code some important criteria have been identified, these include:

• Support from a good IDE with tools designed to ease the overall development.

• The ability to test the smallest building blocks of the product often referred to as

components or classes.


• Support fort at least one major and well adopted design pattern.

If a Framework fills all these criteria probability that it supports code of good quality should be considered as good.

3.5 Chapter summary

By examining the different areas of interest using literature and previously made studies the most significant aspect of each area is identified and broken down to its basic properties.

Through this, the needs that must be fulfilled for each area to be considered as good is, is

made visible. In some cases, it is shown that multiply definitions and conclusions can be

drawn, here the context of this work is used as a reference to decide on which definition fits

this specific context best. The depth of which each area is investigated is limited to what is

seen as a reasonable level given the context of this work. One important finding during this

chapter is the value of the user context, a property that is completely irrelevant in one case

can thus be completely crucial in another.


4 The Testing Framework

The understanding of what defines the different areas of interest described under chapter 3 are used to define a framework for evaluation of SPA development frameworks. Evaluation done with this framework will naturally depend on the needs of the individuals. Their expectations of the product being evaluated are of high importance for the final evaluations and conclusions that this framework is intended to provide. Being based on evaluation of individual needs the framework should not be seen as a direct guide to finding the perfect product but rather as a set of tools to evaluate what pros and cons that exists given some expectations and criteria. Conclusions made are highly contextual. The framework itself is constructed of a set of evaluating questions where the answers must be provided by the user.

The framework is not designed or intended to directly provide the user answers but rather provide the correct questions given certain context. The answers are merely an indirect product provided to the user by conclusions drawn based upon the results of applying the framework. Evaluation the importance of different areas is to be done by the user with respect to their own context. To help the user facilitate this estimation, each part of the framework offers an estimated value of the area’s competence given its context. there is no requirement for this value to be used for a final evaluation, but rather it is provided as an extended tool and guideline for the importance of an area. This framework is designed to return evaluated subsets on the range from 1 to 10 in all the categories that previously have been analyzed. All results are based on individual assessments of the different areas and returns a fitting value for each context. There is no need to use this score-based system when using the structure presented in the framework, its sole purpose is to give an more comprehensible image of the evaluated objects strengths and weaknesses.

4.1 Scalability, performance and legacy support

To test how good the performance, scalability and legacy support of a system would per- form if it were implemented given a framework some core values have to be identified and evaluated. In chapter 3 these values are identified as attributes. The Importance of each attribute must be weighed against the expectations of every individual project. The sub chapters provide question are derived from the information gathered in chapter 3 and are designed to catch the core concepts of each area that will be evaluated.

4.1.1 Scalability and Performance

The following two questions are designed to cover the most important parts that needs to be considered when looking at the Scalability and Performance of a framework:

• How easily can a system developed with the given framework be expanded?


When answering this question, it is important to focus on the raw structure that the frame- work is using. Support within the framework for reuse of code, classes alongside with the ability to apply an object-oriented design pattern should for example be considered good attributes. If a system without much effort can be expanded upon its structural scalability it can be considered good. Since time is of great importance during development another factor that should be considered here is the amount of pre-made assets.

• How well does the framework that is to be evaluated handle an increase in load? Can a product created with the framework be expected to easily handle the expected load without the expenses for resources getting to high?

If the system can increase its load to a good value within the current context its load scala- bility can be considered as good.

4.1.2 Legacy support

When it comes to Legacy support, as proven in chapter 3, there are two parts that needs to be taken in consideration, one focuses on the hardware available to the intended target-group and the other focuses on how long products developed with the framework can be expected to work without major changes. The following questions takes this in consideration:

• How good are the legacy support of the out-puts of the framework and does it support the majority of the intended target-group?

• How good are the legacy support of framework itself and how well can code written against it today be expected to run in future updates?

4.1.3 Numerically evaluating results

To get a numeric evaluation of this part the questions presented can all be given an indi- vidual priority value that reflects the importance of the part given the context of the current framework and its expected goals. The following formula can then be used to get an overall score for this area of the framework.

S = (X


× R


+ X


× R


+ ... + X


× R



n10 (4.1)

The formula calculates a weighed mean value on the scale from 0.1 to 10. X is the evaluated

score of the current question, R is the importance of the current question and S is the final

evaluation score and is the total number of questions. This formula is evaluating the ques-

tions and taking their weighted importance in consideration. The formula assumes inputs

between 1 to 10 and returns a value between 0.1 to 10, 0.1 being the worst and 10 being the

best. The multiplication by ten done in the denominator adjusts the score to the zero-to-ten



4.2 Testability and debugging

In order to regard a framework to have good test- and debug-abilities it, or its develop- ment environment should provide at least one working tool of debugging for each well- established method or approach used within the framework. If it is applicable to the context this tool can be reused over multiply methods. The debugging capabilities should always include the ability to test individual functions and classes, follow data flows and variable values as well as easily preforming test executions separated from live environments. Even if the framework contains potential for good testability there is no guarantee that outputs of it will inherit this property as this, as shown in chapter 2 are very much dependent on the design pattern used during development. If the framework is bestowed with the above- mentioned properties it has a great foundation for potential good testability of the projects developed with the framework that is tested.

The following questions are based on the conclusions from Chapter 3 and are designed to include all essentials knowledge identified during that chapter. By answering these ques- tions, a general idea of the framework’s ability to perform in this area will be given. By giving all answers an value of 1 to 10 the evaluation function 4.1 can be used.

• Does the framework require the use of tight coupled elements to function properly?

If so to what extent are this type of couplings needed?

If non or very few tight couplings are needed the Modality grade when using the framework can be considered as good.

• Dose the framework, or it’s supported development environments provide a broad tool-set that includes tools for all development that is supposed to be done with the framework?

When answering this question, the quality of the frameworks built-in debugging tools and the development environments tools should be considered. A better-defined context will result in a more accurate answer for this question as developers previous knowledge of the existing debugging tool can be weighed in. If, for example, a tool that is commonly used by other frameworks or that is often used by the supposed developers is missing the frameworks score on this question should be decreased.

4.3 Community and documentation support

A good development community can have a large impact upon a development process.

When running in to problems there is a high probability that there exists knowledge within

a good community. The quality of the community does not entirely have to be dependent

on its size, but it is reasonable to assume that there is a better probability to find people

with similar problems within a larger community. As community size and quality are a

hard factor to measure it is suggested that development-oriented forums and source code

repositories can be used in size estimation of communities. To get any value out of an

estimation of a communities size it can be put in relation to its closest competitors. Through

this one can get a insight in the adoption of the framework. By answering the following


questions an approximation of the communities overall size and quality can be done. These questions are designed based on the findings from chapter 3.

• How does the frameworks community size compare to its closest competitors?

When answering this question, it is important to remember that a frameworks adoption can be related to the time it is been available. To avoid getting false impressions on newer frameworks the frameworks current rate of adoption can be looked at. By giving all answers an value of 1 to 10 the evaluation function 4.1 can be used.

• Is the documentation well written and can the community contribute to it?

This question needs to be partly answered by individual assessment as quality of the written word are a hard thing to measure. If the documentation however does allow for community suggestions or contributions it should be seen as a positive attribute,

• Is the framework open source and can users contribute to the development?

Projects that are made under open source are not as threatened by the releasing companies continued development. A largely adopted project under open source can be completely community driven.

• How mature and well adopted is the framework?

A well adopted and mature framework should have more internal resources available and does not tend to be changing internally as much as a newer and less adopted framework.

A few of the factors that can be taken in consideration when measuring these values are:

the age of a framework, its overall size, how often it is used by larger parties and its user activity.

4.4 Code quality

The term ”Code Quality” refers to the possibilities that a developer is given by the frame- work to write high quality code and not the quality of any actually written code. Hence, it is not what is already written but rather what is possibilities to write that is of interest when evaluating a framework. The word quality can be defined in multiply ways one that fits the content of an evaluation framework is chosen during chapter 3. This definition act as a foundation for the choice of evaluation areas that is of importance when evaluation the code quality of a framework. To establish if the framework enables the user to write code of high quality these following questions are design based on the conclusions drawn in chapter 3. By giving all answers a value of 1 to 10 function 4.1 can be used for evaluation.

• Are there any development environments that support the frameworks? If so are these

providing a good tool-set for development?


When answering this question, the tools needed can be contextual and therefor the supposed context should be taken in consideration.

• Is there any built-in support for any major and well adopted design pattern? If so, do this pattern fit well in to development of the supposed context?

Built-in support can be given both as a part of the framework itself or as a feature of the development environment that is used with the framework.

The final question is in many ways very similar to one of those given under the ”Testability and debugging” part of the framework, what separates them are the context of which they are asked. In some cases, the overlap of these two areas result in the questions having the exact same answers.

• Does the framework have the ability to test the smallest building blocks of the product created?

These are often the components or classes and the core of the foundation of everything created with the framework. Being able to perform tests on these makes more complex constructions easier to create as the foundation can be better trusted.

4.5 Summary

In chapter 3 the most important aspects of the following areas are identified: Scalability, performance and legacy support, Testability and debugging,Community and documentation support and Code quality. From these aspects a set off evaluation questions are presented.

These are designed to include all the important aspects of each area. These questions have a high variation in value depending on the context the framework that is being evaluated is supposed to be used. Therefore, a contextual assessment on the value of each area need to be made when using the questions of this framework. Under the first part of the framework an evaluation function is presented. By assigning a priority value and then giving each question an evaluative value, this function can be used to derive a score of each area. This is supposed to give the user of this evaluation framework a better overview if the evaluated framework is suitable to their context. Some parts of these questions require the user to do individual assessments, these are a result of different needs having different criteria.

For these assessments to be as good as possible it is important that the framework that

is evaluated have a well-defined context of what that is desired to be fulfilled before any

evaluation are done. If a framework has good results on the evaluation of these questions

probability for it to be suitable to the current context can be considered high.


5 A Theoretical Evaluation of Blazor

To properly evaluate Blazor an evaluation framework has been created. The included factors in this framework are identified in chapter 2 and with help from these the framework itself is defined in chapter 3. Previous chapters identify what is of importance with an evaluation framework and concludes that the most important factors are not what is possible to achieve within the framework but what is possible to achieve given a constraint on time. Therefore, this evaluation framework is created with the developer in mind intending that the impor- tance of the evaluation lies in the time and difficulty to achieve a certain goal rather than the possibility of achieving or not achieving such a goal. This chapter presents an evalu- ation done using this previously implemented framework. Its ultimate purpose is to give a complete insight into the Blazor framework and development process used with it. It is important to note that this evaluation is solely focused on the perspective of the developer, and the capabilities of Blazor during development. Therefore, all user testing on finished products developed with the Blazor framework is excluded. When evaluating Blazor there are two different results that are of particular interest, namely what the Blazor framework is capable of in theory and how well it performs in practice. A technical result is given by evaluation using the defined testing framework. To validate the theoretical results given by the testing framework a practical implementation of an application is made in chapter 5.

The experience of this implementation is then in chapter 6 used to make a validation of how well the result given by the testing framework in practice is consistent with actual imple- mentation. The practical nature for the implementation in chapter 5 act as an extension and verification of the results given by the framework in this chapter. This result is also used as a demonstration to Vitec of both the capabilities of the evaluation framework and what a practical implementation made with Blazor may look like. The results of this part provide an answer to the second research question presented in the problem description.

5.1 Measuring values of the framework

Several properties are to be evaluated within the framework. These properties have been identified in chapter 2 and all of them can be seen as central in successful development of single page applications. A value on a scale from 1-10 are to be given as a grade on each of these properties for the framework being evaluated, 1 being the worst and 10 being the best.

In the following sections the questions presented by our evaluation framework are answered with given context for development with Blazor in mind.

5.1.1 Context of evaluation

To better answer the initial question given by Vitec Blazor are to be evaluated with a context

that are provided by them and is likely to be applied in a real use scenario. Vitec are looking


to build an internal web application that is created to optimize in-house workflows. They need this application to be able to simultaneously handle a few hundred of connections, since the number of current connections is limited to the current number of employees the applications maximal load can easily be predicted. It is important that this application easily can be connected to a database and that working with it does not introduce unnecessary loading times. Since it is an in-house application the target group can be made to use newer software and thus the legacy support in terms of running on old software are of little importance. It is however important that this application can be used and upgraded during a long period of time as they do not want to invest in an application that get outdated within just a few years.

In order to better grade the property performance in some of the different areas a comparison with the native performance on c# will be used as a preference. As the areas that shall be evaluated are widely different from each other a comparison with c# will only to be used where it is fitting to use as a reference for good performance. A more detailed method for each of the property comparisons are explained in the subsections of this chapter.

5.2 Estimating the performance, scalability and legacy support of Blazor Here the questions given by the evaluation framework are answered with respect to the context that Vitec are looking to use Blazor in. The numeric evaluation provided from the evaluation framework is used in this evaluation and therefor the context is also used to decide a fitting priority value for each area.

5.2.1 Expansion possibilities

This subsection aims to give an answer to the first question given by the evaluation frame- work concerning the possibilities to expand upon a system that are created using the Blazor framework with regards to the current context.

How easily can a system developed with the given framework be expanded?

Vitec are in need of an in-house software that can be used to handle tasks needed within the company, as these kind of tasks often can vary over time the application that they are looking to build need to be able to be adapted according to the current needs. Therefor this question holds a high value in Vitecs case and is assigned the highest priority value possible, namely a priority of 10. Components

Blazor applications can be built using Components. Components are reusable objects which

when designed well does not have any hard couplings to other parts of the implementation

and therefore can be used anywhere within an application without depending on any other

element. Components got the ability to accept arguments and can even be stacked on top

of each other allowing for complex structures. The official documentation describe com-

ponents as ”A component in Blazor is an element of UI, such as a page, dialog, or data

entry form”[8]. Having this kind of structure should allow Blazor to be very adaptable and

expandable as adding, for example, a new form to a Web application should be as easy to


either reuse a already implemented Component for this purpose or simply create a new one.

As components can be designed to work completely independent of each other this should not introduce any dependency problems to the program. Library’s

Blazer is built upon ASP.NET Core with C# as implementation language, with this comes the ability to implement almost all library’s and extensions designed for this language and underlying framework. This gives Blazor the ability to use a huge variety of well- established and already implemented functions and libraries. If extensions that for example include using complex mathematical functions are needed there is a high chance that these functions can be provided by simply implementing an existing library instead of implement- ing them from the ground. This property of Blazor provides good possibilities of expansion if needed. Existing field knowledge

At the moment Blazor is a new framework that is not as well established as other frame- works with the same purpose, therefor expertise’s of development with the framework are harder to come by than for many competitive frameworks. This problem is however min- imized by the fact that Blazor are using C# as a development language. Even if Blazor specific knowledge is hard to come by C# knowledge is not. Taking the step over from classic C# development to Blazor development should not be harder than learning any other new extension library for C#. This puts Blazor in a good position where developers that can work with the framework should not be that much harder to acquire than for any other competing framework. Question summary

Blazor applications have good capabilities of expansion since these applications can be constructed by independent objects. Having this feature allows easy implementation of new features without affecting other parts of the application. There is also a large variety of ex- tensions and library’s available to help speed up development and ease the implementation of more complex functionality. Blazor itself do however have less available pure knowl- edge in terms of developers than many other frameworks since it is still new and not yet as adapted. The fact that C# knowledge easily are transfers onto Blazor helps stabilize this point of view. All this compiled puts Blazor made applications in a strong position where their capabilities to expand do not have many limits, the lack of existing knowledge do how- ever somewhat limit the score of this question and its final answer. Overall Blazor got good possibilities to expand and on a scale to ten it is given a score of 8.

5.2.2 Scale capabilities with regards to change in system load

This subsection aims to give an answer to the second question given by the evaluation

framework concerning the increase in system load with an increased number of users. Focus

are on a system that are created using the Blazor framework and with regards to the given



How well does the framework that are to be evaluated handle an increase in load? Can a product created with the framework be expected easily to handle the expected load without the expenses for resources getting to high?

Given the current context where the application will be used for a set number of users whose amount is not expected to have any drastic increase or decrease, extreme scalability is not that important since it is unlikely to happen. However, it is important that the set expected number of users can be handled without too much stress to the system. The priority value of this part is set to 4 as it is not a main concern for Vitec if the system cannot be scaled up multiply times without an impact on performance. Server side performance

Blazor apps can be created in two different ways, the first one are as a Blazor Server app, in this type of application all vital calculations are done on the server and there for each new connected client can be assumed to take up the same amount of processing power.

On this kind of Blazor app the systems base load can be assumed to be linear with each new connected client, if the application have functions that is of a more complex nature processing spikes can be expected when these are executed from the clients. If such spikes are expected the amount of extra processing power that always needs to be available given a certain percentile can be calculated using statistics formulas. This can for example be done by first calculating the average usage time of these function each day. That is done by multiplying the number of times the operation is expected to occur with the average processing time the operation takes. By then dividing that with the total time the system is in use during a day, the total probability that the function is executed at any given time is given. Given this value it is easy to estimate the probability of something happening within a given percentile. the following expression explains this and can be used to estimate how much extra processing power that needs to be available to the system at any time. The following expression can be adjusted to fit any given percentile.

( n(AT ∗ X )

T )


< ( P 100 ) AT is the functions average execution time

X is the average number of times the function is executed each day T is the total time the system is in use each day

P is the percentile that’s used

N is the number of simultaneously actions

By finding the largest possible value of N for this expression to be true the amount of extra processing power needed can be calculated as follows:

N ∗W = E

W is the processing power needed during execution of the function one time.

E is the total amount of extra processing power needed.

The total amount of computing power on the server side can, as seen above, easy be esti-

mated but applications that are using more resource intense functionality can also be pretty




Related subjects :