Progressive Web Apps and other mobile developing techniques: a comparison

Progressive Web Apps and other

mobile developing techniques: a

comparison

Progressiva Webbappar och andra

mobila utvecklingstekniker: en

jämförelse

Jonathan Kvist, Pontus Mathiasson

Final seminar: 24/5/2019 Examinator: José Font

Abstract

Developing an app for Android and iOS traditionally requires you to build two different versions. This is both costly and time consuming, which has led to the creation of many Cross-platform alternatives. One such alternative is to use a Progressive Web App (PWA). They are built like regular Web apps and include features previously unique to Native apps. Our focus is to compare PWAs to other Cross-platform solutions by measuring the performance of a PWA, Web app and Cordova app. Terms in the ecosystem are clarified due to inconsistencies, and the process of converting a Web app into a PWA is described. The measurements showed that the PWA and Cordova app had almost equal overall performance, and they were both faster than the Web app. This indicates that PWAs are a viable Cross-platform alternative, but more quantitative studies with performance in focus needs to be conducted due to the small scale of this work. Other aspects of PWAs also needs to be evaluated, such as app size and ease of development.

Table of Contents

1 Introduction ... 1

1.1 Problem ... 1

1.2 Research Questions ... 2

1.3 Research Framework ... 2

1.4 Overview of the app and the client ... 5

1.5 Contributions ... 7

1.6 Brief description of the remaining sections of this paper ... 8

2 Method ... 9

2.1 Research Strategy ... 9

2.2 Data Generation Methods ... 11

2.3 Performance Comparison ... 13

2.4 Implementation ... 14

2.5 Literature Study ... 15

3. Solution ... 16

3.1 Overview of the implementation ... 16

3.1.1 Files ... 17

3.1.2 Requirements ... 18

3.2 Description of testing ... 22

4 Results ... 23

4.1 RQ1 How does the literature define a Cross-Platform app, Web app and

Native app ... 23

4.2 RQ2 How can a Web app be converted into a Progressive Web App? ... 24

4.3 RQ3 How does a Cross-Platform app, a Web app and a Progressive Web

App differ in performance? ... 25

... 29

5 Analysis ... 29

5.1 RQ1 How does the literature define Cross-Platform apps, Web apps and

Native app ... 30

5.2 RQ2 How can a Web app be converted into a Progressive Web App? ... 30

5.3 RQ3 How does a Cross-Platform app, a Web app and a Progressive Web

App differ in performance? ... 30

6 Discussion ... 31

6.1 RQ1 How does the literature define Cross-Platform apps, Web apps and

Native apps ... 31

6.2 RQ2 How can a Web app be converted into a Progressive Web App? ... 32

6.3 RQ3 How does a Cross-Platform app, a Web app and a Progressive Web

App differ in performance? ... 33

7 Conclusion ... 34

8 Future research ... 37

1 Introduction

1.1 Problem

In 2017 the average mobile user would spend 16 times more time in mobile apps than they would on the web [1]. However, the web had 2.2 times more monthly unique visitors. This means that both Web applications (Web app) and mobile apps are important for a company that wishes to reach and keep engaging a lot of customers. In order to cover 97.3% of the mobile app market, a company must develop two unique applications [2]. One for iOS to be published in App

Store [2] and one for Android to be published in Google Play Store [3]. Developing

and maintaining these unique platform specific versions of the same application increases development costs almost linearly by the number of desired platforms [4].

Due to this, companies seek alternatives to minimize their development costs. One popular solution is to develop a single Web app. A Web app is an application that can be accessed through a browser, which is something that is available on most platforms [5]. This might be a desirable alternative since only one application needs to be developed, whilst still covering most platforms. However, native applications (Native app) are sometimes preferred over Web apps [6]–[8]. This is partly due to the increased number of hardware specific functionalities that a Native app can access [9]. On the other hand when developing a Web app a developer does not need to know each separate coding language for each desired platform [10].

Due to the growth of the mobile market [11], company employees and customers are in need of mobile versions of available websites. One major motivation behind this is the potential for offline access that these apps can offer. More than often users find themselves in an environment where a network connection is unavailable, such as when traveling or when being in countries with poor internet availability.

Companies have set out to solve the problem of having to develop a multitude of apps in order to reach a higher market coverage [1]. From these attempts, a new type of application has emerged which places itself in between Native apps and Web apps [12]. These are called Cross-platform applications (Cross-platform apps) and differs greatly between each other in the way that they are distributed to other platforms. Companies such as Adobe, Microsoft, Facebook and Google are all developing their own Cross-platform alternative [13]–[16]. The characteristic that all of these solutions have in common is that they are not required to be developed with a specific platform Software Development Kit (SDK) [7]. This means that they can be distributed to more than one platform, based on the same codebase.

In 2015, two developers at Google presented a technique that would make it possible to develop applications which combine desired features of Web apps and Native apps [17]. These applications were given the name Progressive Web Apps (PWA). A PWA is developed as a Web app but can offer more functionality

such as offline capabilities, push notifications, installation to the home screen and Cross-platform availability [9], [18], [19]. These features make PWAs look and feel more like Native apps. They can also be distributed to more than one platform from the same build.

1.2 Research Questions

Q1. How does existing literature define Cross-Platform apps, Web apps and Native apps?

Q2. How can a Web app be converted into a Progressive Web App?

Q3. How does a Cross-Platform app, Web app and Progressive Web App differ in performance?

1.3 Research Framework

In order to evaluate the performance of PWAs they have to be compared to the current mobile app development approaches. These approaches are:

- Web apps: Accessed via a Uniform Resource Locator (URL) and implemented using HyperText Markup Language (HTML), Cascading Style Sheet (CSS) and JavaScript (JS). These apps run via the browser which almost all devices have access to. However, a Web app has limited access to hardware functionalities such as the camera and the GPS etc [12]

- Cross-platform/Cordova: Cross-platform is a development approach that has the ability to distribute an application to a multitude of platforms based on the same codebase [6], [7], [12], [20]–[22]. Cordova is an example of such a framework, currently being open source and owned by Adobe [23]. It distributes a Web app as a Native app by wrapping the build generated from Cordova in a Native app, where it uses a webview which is an object that has the ability to display web content. With this combination of web content and a native wrapper, the result can look and feel like a Native app. For it to access native features such as the camera and the GPS, individual APIs for each functionality has to be implemented separately. Each such API is created by Cordova and are supported by multiple platforms, being referred to as Plugins [24]

- Native apps: Native apps are developed for a specific platform (e.g. Android, iOS, Windows) using the development environments, coding languages and APIs designed for that platform [6]–[8]. Android apps uses Android Studio as its development environment and Java/Kotlin/C++ as its coding language. These constraints entail that apps developed for one platform cannot be installed or used on another [6], [7]

In 2015, Alex Russell wrote a blog post in which he and Frances Berriman defined the characteristics of a PWA [17]. In the post, they discussed the Cross-platform advantages of the web and also summarized PWA as a concept for the first time. When researchers today try to define what a PWA is, it often results in listing the same characteristics as Russel and Berriman did in 2015 [9], [18], [19], [25]. In short, a PWA can be described as a responsive Web app which

progressively improves itself to provide an experience closer to that of a Native app [9], [18], [19], [25]–[27]. When a user has interacted with the application to a certain degree, 30 seconds at the time of writing, that app can then be added to the home screen without having to visit the devices dedicated application store.

Russell and Berriman defined the following attributes for PWAs in 2015 [17]: - Responsive: Should be built in order to work smoothly on a wide variety of

devices

- Connectivity independent: Should take advantage of Service Workers in order to function even without a network connection

- App-like interactions: Should use an application shell architecture and dynamically load content

- Fresh: Content should be up to date - Safe: Should use Transport Layer Security

- Discoverable: System should recognize them as applications

- Re-engageable: Should use system APIs such as push notifications to re-engage the user

- Installable: After a certain amount of interaction with the user, the app can be added to the home screen

- Linkable: Should be easy to install and share

Since the area of PWAs is still highly industry driven, mostly by Google, their documentation and blog posts are the most up to date source of information. The attributes defined by them today remain the same, but they have added the term Progressive which means that PWAs should work in any browser and on any platform [28].

Partly due to the fact that the term PWA was coined first in 2015 [17], the research field concerning them has not had time to grow. Older fields such as Cross-platform development and Mobile web development are more explored, and many of their findings are applicable to PWAs as well.

However, there exists some research in the field regarding PWAs. Two papers aimed at determining if PWAs has the potential to become a dominant way of developing apps in the future [9], [19]. Even if that question was deemed difficult to answer, both papers listed pros and cons with the current state of PWAs. Furthermore, they described what has to be improved in order for PWAs to increase their presence on the market. The authors discussed some existing advantages such as a decreased app size, increased conversion rates in places with poor internet connection and a satisfying user experience. Some of the existing limitations included access to hardware specific APIs, lack of research in the area and limited support by Apple's iOS standard browser Safari.

Another study investigated PWAs in terms of security and how the introduced features can be abused to leak information [25]. The study found three potential threats unique to PWAs:

1. Phishing via push notifications

2. The use of third-party push notification libraries with known security flaws 3. A new type of attack which is directed towards the cache and Service Worker Malavolta et al. presented the first empirical investigation of PWAs [29]. Their research aimed at determining what impact a Service Worker, the technique that stands at the core of PWAs, has on energy consumption. To generate a satisfactory result they compared seven pseudo randomly selected PWAs from the industry. These seven PWAs were compared on two different phones: one which represented a high-end flagship and one which represented an older device. The result indicated that the energy impact of Service Workers seemed to be close to nothing on both phones.

In 2018, Cardieri and Zaina performed a study where they compared a Native app, Web app and PWA from a user experience point of view [26]. They performed the study by looking at user experience through the perspective of both a user and a Human Computer Interaction specialist. The results could not identify any clear difference between the three development techniques. Performance analysis on apps has been done before. Willocx et al. looked at two different Cross-platform developed apps and compared those to apps developed for Android and iOS [30]. The study found that both Cross-platform solutions had worse performance compared to the Native apps. Heitkötter et al. composed a framework which describes how to evaluate Cross-platform apps. They looked at several characteristics, one of them being performance [12]. Since PWAs can be seen as a potential Cross-platform solution, this framework is relevant. The study also analysed three different Cross-platform methods using this framework, namely Web apps, PhoneGap (now called Cordova) and Titanium. According to the requirements presented by the framework, PhoneGap (Cordova) is the best alternative.

Performance has been proven important for how the user perceives a website. Several case studies have shown this:

● Pinterest lowered their waiting time by 40%, which lead to a 15% increase in conversion rate to sign up [31]

● A study showed that for every second the BBC site loads, they lose 10% of their customers [32]

● Google conducted a delay experiment in which they investigated how latency affected the amount of daily searches [33]. They found that increasing latency by 100ms reduced the number of daily searches by 0.2%, and increasing latency by 400ms reduced searches by 0.6%

just 1 second resulted in 5.8% more people leaving the site before the content finished loading [34]

From these examples we can see that users act differently depending on the performance of the application. This transforms into a lot of potentially lost revenue for companies which do not prioritize good performance [35], [36]. In 2018, Steiner made an analysis regarding the available PWA features when operating in different webviews [27]. This was tested using industry leading Web apps, some among them being Facebook and Twitter. By using the in app browsers he could see how broad the support for different PWA features were. The results concluded that iOS lacked support for many features required by a PWA. This was partly due to iOS not officially having a platform that supported PWAs at the time the analysis was performed. The result showed Android ended up reliably supporting 4 out of 15 features compared to iOS only supporting 1 out of 15.

1.4 Overview of the app and the client

Since we did not have access to a PWA at the start of this project, one had to be developed. Instead of building a PWA from scratch we had the opportunity to alter a Web app owned and developed by our client Axis. The app is called the Product Selector and is used by Axis resellers and customers, providing them with a platform to see big parts of the product catalog at once. To navigate among the products, users are provided with search and filtering features. When users click on a product, they can navigate to that specific product page displaying some of its functional details and unique features. The app also includes a comparison feature allowing users to make a side by side comparison of selected products.

The products are fetched from a Product Information API (PIA). PIA is a RESTful API containing information about the products to be displayed in the Product Selector. Requests for products are made to this API, and it responds in JSON. In order for our application to display products, they first need to be requested from PIA.

The Product Selector existed as a Web app and a Cross-platform developed Cordova app. The Cordova app is compiled into Android and iOS from the same codebase, only requiring a few alterations depending on what platform you wish to compile it to. An overview of the apps can be seen in Figure 3, also displaying the created PWA.

Both the Cordova app and the Web app were mostly based on the same HTML, CSS and JS, so they provided the same functionalities. Some small differences existed depending on what browser was used to view the Web app. Some browsers altered parts of the user interface elements such as sliders and buttons. An example of these differences can be seen in the spacing of the buttons when comparing the Cordova app and the Web app, shown in figures 1a-b and 2a-b.

mobile platform. This means that it may not be subject to the same appearance changes as the Web app. Since a Cordova app is served in a native shell, some additional files must be added to support this functionality compared to the Web app. These include Config.xml, additional JavaScript code to handle network requests and platform specific icons.

Figure 1a, 1b. The figures displays the application which we altered into a PWA. This is the Android version of the application, built with Cordova

Figure 2a, 2b. The figures displays the application which we aleted into a PWA. This is the Web app version of the application

Figure 3. Overview of the applications: Cordova, Web app and PWA

1.5 Contributions

Looking at the recent studies, our paper has a clear purpose since no earlier work has compared a Cross-platform app, Web app and PWA in terms of performance. Our research conducted the following in order to answer our research questions:

1. A literature review which scrutinized the research area. More specifically, we looked at the areas of PWA, Cross-platform app development, Native app development and Web app development

2. Remodeled an existing Web app to meet the requirements of being a PWA 3. Compared a PWA to a Web app and an Android app developed with Cordova One of the aims of this thesis was to assess whether PWAs are comparable to other Cross-platform apps in terms of performance. Other aspects could have been considered, e.g. one study assessed the user experience of PWAs [18]. We chose to look at performance, since it is an area in which traditional Web apps and Cross-platform apps usually get outperformed by Native apps [13], [22],

[30], [31]. Frequently cited papers also stated that a goal for Cross-platform development approaches is to achieve performance as close as possible to Native apps [22], [24]. In order for us to be able to compare the three versions of our app, measurements had to be conducted. Those measurements also became a contribution.

During the review of the literature and blog posts from the industry, an uncertainty among authors in how they used certain terms in the ecosystem became apparent. Some articles defined their own definitions through the creation of new taxonomies, many of which introduced completely new terms as in the case with Nunkesser [21]. Others simply branch out the current terms and provide new definitions of those already existing [30]–[33] [6], [7], [12], [18], [20]–[22], [30], [37]. The most commonly used terms in the industry as well as in the literature were:

● Native apps ● Web apps ● Hybrid apps ● Interpreted apps ● Cross-compiled apps

Hybrid, Interpreted and Cross-compiled are often considered subcategories of the term Cross-platform, and are sometimes accompanied with additional subcategories which vary throughout the literature [8], [13], [21]–[24]. Defining subcategories for the term Cross-platform was out of the scope of this thesis. This decision was taken due to the fact that we do not refer to more Cross-platform approaches other than Cordova and PWA. We instead chose to use the term Cross-platform when addressing techniques that cover a multitude of platforms from one code base. In order to stay consistent with other researchers vocabulary, this thesis first contributes by defining the most common terms used when addressing mobile app development.

The second contribution of this thesis was to measure and analyse performance data collected from the same application, built in three different ways (PWA, Cross-platform app and Web app). The PWA was also a contribution of itself since it had to be developed. The alteration process from Web app into PWA includes the finished artefact accompanied with a description of how a Web app is altered into a PWA. The data generated by the comparison has never been presented prior to our research. Since the PWA research area is new and not extensively explored, there existed a need for such data in order to motivate future research and to give an insight as to how the different platforms differ in terms of performance.

1.6 Brief description of the remaining sections of this

paper

After this introduction, a method section follows consisting of a detailed description of the chosen research strategy, where an explanation also exists stating how and why the PWA was created. The section also explains how the data for the comparison was generated and what metrics were used and why. It

also touches upon limitations of the performance tests. Furthermore, the section defines the features that were added to the Web app in order to turn it into a PWA. Finally, the method describes how the literature study was conducted and where we found our sources.

The solution section is solely focused on displaying what was performed in order to answer the research questions. It does so by first explaining the implementation of each PWA feature in a orderly fashion. It then describes the logging features of the performance testing. Finally, it contains a brief description of how the PWA was tested to make sure that the application features were functioning correctly.

Each research question results are presented in the result section. There they are presented separately from each other in the order that they are introduced. The result section is followed by an analysis section where the results of the research questions are interpreted and explained. The results from the performance test are also compared to each other to give further insights. After the analysis follows a discussion. Here, the results are evaluated in regards to the research questions. The performed literature study is also evaluated here, followed by a discussion about how we converted the Web app into a PWA. Finally, the performance measurements are evaluated and their limitations are discussed. The document ends with our conclusions,

acknowledgements and references.

2 Method

2.1 Research Strategy

In order to conduct the performance comparison, the same application had to exist as a PWA, Web app and Cordova app. We are trying to make our PWA a main contribution to knowledge while also comparing it to other alternatives. This scenario is perfectly captured in the Design and Creation model [38], and is the reason why we chose Design and Creation as our research strategy. A second reason why we chose Design and Creation was in order to make research that contributed with new knowledge and not just regular product development [38]. Through Design and Creation and the literature review we developed and contributed with two artefacts: a construct and an instantiation, as described by Oates [38].

- Construct: The assemblage of terms we identified during the literature review. This artefact defined the terms Cross-platform app, Web app and Native app

- Instantiation: Our PWA is a working system which we used as a tool in our performance comparison. It was produced with the purpose of comparing it to others, which motivated the Design and Creation strategy

In many ways we followed a traditional Design and Creation process, as defined by Oates [38]. This process, consisting of five iterative steps, is illustrated in Figure 4. The steps are:

- Awareness: Our recognition of the problem came from a mix of our clients’ interests and the literature review. When studying the literature, the advantages and disadvantages of both Web apps and Native apps presented themself. PWAs were identified as a potential way of combining desired features from both approaches

- Suggestion: A need to properly define the terms used in the current taxonomy was identified. In addition to that, we and other researchers in the field agree on the importance of quantitative measurements to assess how well PWAs actually do in practice. This entailed the idea that a contribution could be to measure and compare PWAs to other types of applications in terms of performance

- Development: Since the functionality we implemented mostly consisted of smaller changes which were independent of one another, a major system development methodology was not adopted. Our development team only consisted of two people, which meant that the communication paths were already kept at a minimum. Instead, pair programming was identified as a suitable strategy, in which both developers program together on the same computer [39]. We decided to adopt this strategy since neither of us had ever written a PWA before and pair programming is a proven way to increase software quality [39]. If our PWA would have been developed poorly with a lot of potential bottlenecks, the measurements would not have been useful in a comparison

- Evaluation: The evaluation of our system was performed by comparing it to other development techniques in terms of performance. This is described in more detail later

- Conclusion: Conclusions were drawn from both the literature review and the collected performance measurements

Figure 4. The iterative Design and Creation process

Our system also contributed to knowledge by being a “project where the IT

application is a vehicle for something else” [38]. The objective for the development

of the application was only to use the result as a “proof by concept”, even though the app itself also became a contribution. We wanted to develop the application in order to compare it to other implementations, and then see if it demonstrated an improvement or regression in performance [38].

2.2 Data Generation Methods

In the performance comparison, quantitative data was collected. The study measured the time taken before different events occurred on all three kinds of applications. Those measurements were then compared to see which performed better. Quantitative data was generated in two steps.

Step 1: In order for us to measure performance, we had to choose what parts

of the application to focus on and which methods to use when collecting data. Since the PWA and Web app are accessed through a browser and the Cordova app from a native shell, they operate in different ways. To measure application performance in PWAs and Web apps, popular tools such as built in features in Chrome Developer tools or WebLOAD can be used [40], [41]. The most popular way of measuring performance of Native android apps is to use the Android Profiler tool which is integrated into Android Studio [42].

Due to this fact, we could not perform measurements with the same automated tool across all applications. Instead, timestamps at different events were logged and the time elapsed was printed. This was done for two reasons. Firstly, using different kinds of tools and then comparing those measurement would reduce chances of correct reproduction. We could not know for certain if the tools would conduct the measurements in the exact same way, and therefore if the results would be the same if other tools were used instead. Secondly, logging comes with the advantage that one could easily add more kinds of apps to the comparison since almost all types of apps have access to a logging feature. The metrics we chose to study are inspired by metrics defined by Google. They focus on four aspects [43]:

- Is it happening? - Is it useful? - Is it usable? - Is it delightful?

The metrics associated with “is it useful” and “is it delightful” are metrics that are not directly correlated to performance. The metric “is it delightful” could instead be used to identify potential bottlenecks in the code. This is interesting for optimization purposes of how one could develop an overall faster app, but it does not provide an actual timestamp to include in our comparison. “Is it useful” is correlated to the first meaningful content metric. In order to measure this, developers have to define what they consider useful by targeting specific sections of the application. Such a metric is oriented to give insight as to what parts of the application loads the fastest. We feel that such information is again more suitable for optimization purposes. The metrics we decided upon using the performance test are:

● Estimated Input Latency (EIL) - time taken for the app to respond to user input [44]

● First Contentful Paint (FCP) - time taken for the first text, image, non-white canvas or SVG to be rendered [45]

● First Paint (FP) - the first key moment during page load. This is when the browser starts to render the page [46]

● Time to Interactive (TTI) - time taken for the app to become interactive, so that the user can do something with it [47]

All of these metrics were chosen since they directly correlate to the performance perceived by the user. Which is the type of performance metrics we are interested in. As seen in the case studies earlier, how the user perceives performance affects how they use the application [31]–[34]. Minimizing the time taken before these events occur is therefore of interest.

Step 2: The data generated from these measurements is always time elapsed,

measured in milliseconds. This is a form of ratio data since there always exists a “time zero”, which is when the clock starts for the measurement. This data is also discrete and every measurement were rounded off to one decimal, since more precise data is more or less not noticeable by the user.

We also performed some simple data analysis to our measurements. Bar charts were constructed so that the reader could see some apparent patterns [38]. A table which summarizes the results is also presented. More complex statistical techniques were not used since the generated data was rather limited with a lot of affecting factors. The result of such an analysis would be hard to validate and even not be to interesting in regards to our research questions, since we are mostly interested in how the different measurements relate to each other. In Figure 5, an illustration of the overall research methodology is displayed. It shows how the strategy chosen made use of different data generation methods and what system development methodology was adopted.

Figure 5. Overview of Research Methodology consisting of Strategy, Data Method and System Development Method

2.3 Performance Comparison

The carried out performance comparison used the logging feature in Chrome dev tools to gather the test data. Chrome dev tools is a built in tool in the web browser Google Chrome [48]. There is no officially supported way to distribute Chrome dev tool to other web browsers. This means that conducting the performance tests across multiple browsers would require using the tools in a non supported manner or the usage of different devtools across different browsers. We therefore decided to only use one browser for our tests, Google Chrome.

The performance testing was conducted on four different phones: ● Xiaomi mi mix 3: This phone represents a newer, high end device ● OnePlus One: This phone represents a lower end, older device ● Huawei P10 lite: This phone represents a mid range device ● Samsung S6 Edge: This phone represents a mid range device

that were available to us. Further efforts could be made to gather more phones for the tests. This was deemed too costly as would imply buying more phones, something that is not within our budget. With that being said the range of phones used for the performance test still covered a good portion of the different ranges of phones available at the time of writing.

IOS devices were left out of the tests for two reasons. The PWA running on iOS would have to be compared to the Cordova-app compiled to iOS as well. These would be different from the versions running on Android since they are two very different platforms, and that would result in their measurements not being directly comparable. The support for PWAs is also much greater on Android devices compared to iOS devices [49]. In order to be able to test PWAs as they are expected to work in the future, they had to be tested on an Android system.

2.4 Implementation

In order adopt the Product Selector into a PWA, we followed the Baseline Checklist provided by Google. It is a list of the core features required in a PWA, as shown in Table 1 [50]. To make sure these features were implemented correctly, a Chrome-plugin called Lighthouse was used. Lighthouse is an automated testing tool for assessing web pages [40], including tests suites that are specifically designed to measure to what extent a web page has implemented the PWA features. Each suite contains a series of sub-tests which are all required to pass in order for the suite to pass. Since the tests are automated and developed by a trusted third party, no additional time was spent on developing test suites.

Table 1. All features required for a PWA and a description of how to test them

Feature number Feature description How to test F1 Site is served

over HTTPS Use Lighthouse to verify Served over HTTPS

F2 Pages are

responsive on tablets & mobile devices

Use Lighthouse to verify Yes to all of Design is mobile-friendly , although manually checking can also be helpful.

Check the Mobile Friendly Test

F3 All app URLs

load while offline

Load various pages in the PWA with an airplane mode enabled. Ensure the app presents some content even when offline. Use Lighthouse to verify the start URL responds with a 200 when offline.

F4 Metadata

provided for Add to Home screen

Use Lighthouse to verify User can be prompted to Add to Home screen is all Yes.

F5 First load fast

even on 3G Use Lighthouse on a Nexus 5 (or similar) to verify time to interactive <10s for first visit on a simulated 3G network.

F6 Site works

cross-browser Test site in Chrome, Edge, Firefox and Safari

F7 Page transitions don't feel like they block on the network

Open the app on a simulated very slow network. Every time you tap a link/button in the app the page should respond immediately, either by:

Transitioning immediately to the next screen and showing a placeholder loading screen while waiting for content from the network. A loading indicator is shown while the app waits for a response from the network.

F8 Each page has a

URL

Ensure individual pages are deep linkable via the URLs and that URLs are unique for the purpose of

shareability on social media by testing with individual pages can be opened and directly accessed via new browser windows.

Source: Google [50]

As an addition to the core characteristics, push notifications were also implemented in the application, this is due to technique being closely associated with PWAs. The association origins from the attributes defined by Russell and Berriman who coined the term back in 2015 [17]. The push notifications functionality is relying on the Service Worker [51]. We defined the need for push notifications in the Product selector to re-engage and inform the user of the app when a product has been added or removed from the library of products in the app. With the push notification users can also be notified when for example a major bug has been fixed.

2.5 Literature Study

The literature study was conducted in a orderly fashion to ensure a consistent and full depiction of the current state of the literature. The literature involved the examination of journals, conference papers and one blogpost. The blogpost was included due to fact of it being the first recorded mention of the term Progressive Web App, which is the central concept of this thesis [17]. The key terms used for searching among the literature included: PWA, Progressive Web App, Progressive Web App Comparison, Cross-platform and Service Worker. The databases used were Researchgate, Google Scholar, Springer, ACM and IEEE.

To find additional articles and to make sure that we had covered the most cited research in the area, snowballing methods were also used [52]. In order to know what articles to use from other reference sections, the researchers summaries of previous research were reviewed. This further strengthened the validity in our found research gap, and also ensured that no significant research was overlooked. From the result of the literature review, 16 articles published from 2012 to 2018 were deemed relevant to this thesis.

Due to found inconsistencies regarding certain terms used, we decided to form our own definitions of the terms Native, Cross-platform and Web apps. 7 out of the 16 articles contained different definitions of these terms, described with other sources or with the authors’ own definitions. To further strengthen the reality of the issue we also researched how the terms were used in the industry. The motivation behind this was only to make sure that the same problem was present in the industry, we settled with three different blog posts. Even here, the same inconsistencies were apparent. This thesis therefore felt the need to define our own definitions of the terms for two reasons. Firstly, to avoid confusion for the reader when presented with the terms. This also gave us as writers freedom to use the terms as we pleased throughout the text, knowing that the reader will not feel any contradiction regarding the perception of them. Secondly, we wanted to contribute and show regard to the researchers before us who had spent time and effort to create their definitions of the terms. Our definitions are based upon the 5 out of 7 articles we found defining their own terms. This was done by comparing the different article definitions and selecting the parts of them most comparable to the current ecosystem.

3. Solution

During the implementation of the PWA, recommended guides provided by Google were followed [53]. These guides were highly detailed step by step instructions on how to implement the different core features of PWAs. During development, codes of best practices provided by Axis were followed in order to create an appropriate application structure while integrating our features with those of the existing app. Some of the PWA features were already fulfilled in the existing application, and these were left untouched since they already passed the test suites in Lighthouse.

3.1 Overview of the implementation

When adding the required features in Table 1 to the Product Selector, 3 files were created and 4 files had to be modified as displayed in Figure 6. A detailed description of the performed alterations follows below.

Figure 6. Overview of the development files and folders. Green means that the file was added, orange means that the file was modified and blue means that the file or folder remained unchanged

3.1.1 Files

serviceWorker.js - This is where network requests are interrupted. Routes are

registered for each file type and when a network request occurs the Service Worker decides if the request should go to the network, cache or both. The Service Worker was written using the library Workbox, which makes it easier to register routes and precache files [54]. A typical network request interruption looks like the one in Figure 7.

Figure 7. A typical network request interruption using Workbox. The first parameter specifies the route to be interrupted, and the second what to be performed when the request occurs. In this case, we wish to use the strategy StaleWhileRevalidate, which first checks if the content is available in the cache and otherwise falls back to the network [55]

more to use when the application is added to the home screen. This file is written in JSON and is necessary to include this functionality.

dev.html & app.html - In these files, code was added in order to do three

things:

1. Register the Service Worker. The Service Worker is registered using the navigator object and is able to intercept requests as soon as the installation is completed. In order to install and register the Service Worker, the application needed to serve content over HTTPS [56]

2. Make it possible to add the app to the homescreen. The requirements that has to be met in order to do this are [57]:

- The app cannot already be installed - Meet a user engagement heuristic

- Have a web manifest specifying information about the app for installation

- Serve content over HTTPS

- Have a currently registered Service Worker with an event handler responding to fetch events

3. Request permission to give the user push notifications. The notifications are sent by the Notifications API

images - This folder is where images needed for the application were placed.

The image smallAxisLogo.png was added, which is used as a logo for when the app is saved to the home screen.

piaCommunicator.js - This is the file which is responsible for fetching all the

products from the API. When the products are fetched for the first time, they are put in the cache for future use.

gulpfile.js - Code was added to the gulpfile to make sure that our created files

are included when building the application for development or production. All of the marked files were either modified or added in order to fulfill one or more of the 8 requirements specified in Table 1. By meeting all of these, a Web app can be called a PWA. This section is followed by a description of how each of these features were implemented in our environment.

3.1.2 Requirements

F1 Site is served over HTTPS. The existing Web app already fulfilled this

requirement through the use of proper certificates.

F2 Pages are responsive on tablets & mobile devices. The existing Web app

already fulfilled this requirement through Bootstrap and responsive CSS.

Service Worker is installed and all the necessary content for offline usage is precached. This includes all HTML, CSS, JS and some images. A default image is also cached since storing all the included images would take up too much space. As the app then fetches the products for the first time, they are also placed in the cache for future usage.

When a network request occurs, it is interrupted by the Service Worker. Depending on the request, different caching strategies are used. For the products, a pattern called Stale While Revalidate was used, illustrated in Figure 8. Every request first checks whether the content is available in the cache and otherwise falls back to the network [55]. This means that even when the network is not available, a request will succeed if the content is present in the cache.

Figure 8. Overview of the Stale While Revalidate strategy. Source: Workbox [55]

A benefit of this strategy is that the content is always kept up to date. Every request, no matter if the content was present in the cache or not, will attempt a fetch from the network. If that request is successful, the response is used to update the cache. This is necessary in our application since new products are added on a weekly basis, and these must be included in the app.

All the static content on the page, e.g. HTML, CSS, JS and some images use a different strategy called Cache First as displayed in Figure 9. This strategy first checks if the requested content is available in the cache. If it is, that content is returned. If it is not, an attempt to fetch the content from the network is initiated, and any potential response is placed in the cache [55]. This strategy was chosen since these static files are not updated as often as the products. Therefore, it is unnecessary to make a network request if the content was already present in the cache.

Figure 9. Overview of the Cache First strategy. Source: Workbox [55]

The last strategy used is called Network First. The purpose of this strategy is to first check if the content can be fetched from the network and if not see if it is available in the cache [55]. This strategy was implemented for the images of the products, so that they would be fetched from the network if the user has an available connection. If the request was unsuccessful, the request would instead fetch a default placeholder image from the cache. The motivation behind not caching all images was that at the time of writing, 321 product images existed. Caching all of these would require the app to reserve a lot of storage. Also, the main functionalities of the app are still available even if a default product image is displayed instead of the correct one.

F4 Metadata provided for add to home screen. This feature was implemented

by adding a file called manifest.json which was then linked together with the rest of the app. The file contained information necessary for adding the app to home screen, e.g. names, icons, colors and URLs all specified in JSON. When the user clicks to add the app to the home screen, the data specified in this file is used.

F5 First load fast even on 3G. To reduce the load time of the PWA, analytics

generated from Lighthouse were used. The test suite produced a list of possible optimizations that would potentially speed up the app. These suggestions were listed based on the estimated amount of time they would save if correctly implemented. Therefore, these suggestions were followed until the test passed. The test passed when the application became interactive in less than 10 seconds on a slow network connection. The following optimizations were performed:

1. Fetch properly sized images

As mentioned earlier, 321 product images are fetched at the time of writing. Even though these images are displayed at 100x100px at most, the Web app were fetching them at 200x200px. By instead fetching the images in the size that they will be displayed in, we were able to reduce the amount of bytes being sent over the network and therefore also make the application faster.

2. Minify Javascript and CSS

Minification is the process of reducing file sizes by removing comments, good variable names and whitespace without changing the functionality of the code [58]. Smaller files means faster load times and less data having to be transmitted over the network. The JS was minified using a library called Terser [59] and the CSS with a library called Clean-CSS [60].

3. Load all stylesheets asynchronously

The last optimization performed was to make all stylesheets load asynchronously instead of synchronously. This allowed the application to load other files or data instead of being idle while loading the stylesheets. The implementation consisted of adding “rel=preload” to all stylesheet link tags, which specifies that the stylesheet should load asynchronously.

F6 Site works cross-browser. The existing Web app already fulfilled this

requirement. It was achieved by only using libraries with functionalities supported by all major browsers. Even if some of the added PWA features were not supported across all browsers, the core page still loaded including filters and products.

F7 Page transitions don't feel like they block on the network. When the

user clicks a link or a button, he or she is immediately directed towards that page. The available static cached content is also displayed before any major network requests are initiated.

F8 Each page has a URL. The existing Web app already fulfilled this

requirement. It was fulfilled through the use of Angulars default routing service, Angular Router [61].

To implement the four logging features Estimated input latency, First contentful paint, First paint and Time to interactive, three different approaches were used. This was needed since the different logging features had to be implemented in different ways depending on what they were measuring and what they needed to access in the code.

Estimated input latency (EIL) was implemented using a PerformanceObserver,

which is a JavaScript interface supported by most major browsers [62]. EIL differs from the other three metrics in the sense that it is meaningful to log multiple times during the same test. The output produced by the metric varied a lot depending on what type of element it was observing, meaning that it became interesting to log it on more than one element. Therefore, EIL was placed on three different elements in the app. These included clicking on any of the products displayed (1), clicking the compare button (2) and checking the outdoor ready filtering checkbox (3). All placements can be seen in figure 10a and 10b. These elements were chosen since they are common elements to interact with during normal use. The logging itself was implemented through adding event listeners to the marked elements. When a registered element is clicked, the listener estimated the EIL and prints the result handled by a PerformanceObserver.

Figure 10a, 10b. Overview of what elements are observed to measure estimated input latency. The elements being observed are 1: Product 2: Compare 3: Filter checkbox

First contentful paint & First paint (FCP & FP) were also implemented using

the PerformanceObserver. The interface allowed for an observer to be set up on a PerformanceEntry, which is an encapsulation of a performance metric [63]. We hooked up an observing event on the PerformanceEntry called PerformancePaintTiming [64]. This entry provided information on two variables, first-paint and first-contentful-paint. The observing event triggers when either the first-paint or first-contentful-paint variables are updated. As the observer triggers one of the metrics, the result is printed for us to see.

Time to interactive (TTI) was implemented using a polyfill called Time to

Interactive Polyfill [47], since this functionality is not yet available in the PerformanceObserver API. It worked by invoking a method called getFirstConsistentlyInteractive(), which resolves when a TTI value has been found. Then the result could be printed for us to see.

FCP, FP and TTI were registered in the header of the index file, before any other scripts or stylesheets had executed. This was done so that they would not miss or interfere with any important event. The EIL was implemented in the methods which handles the events for each of the chosen functions, since that is how the interface is specified to function.

3.2 Description of testing

With each implemented feature, the Lighthouse test suite were ran in order to make sure that it was implemented correctly. Further testing was performed to assure that no features broke during the alteration process. Simple black box tests were conducted to make sure features which worked in the original Web app functioned the same in the PWA. The black box tests were formed based on the applications features, e.g. data fetching correctly from databases, functionality in input-windows, button-presses and more. Since most of the different features required for a PWA are independent of each other, the order in which they are implemented does not matter for the end result. Therefore we

simply implemented them in a pseudorandom fashion.

The testing relied heavily on a third parties test suites which the project had no control over in regards to what the tests covered. This introduced the problem of test suites not yet being implemented for three of the core features. Since these features still had to be tested, we added additional layers to our feature testing. These layers included specified testing scenarios for each of the features that were not included in Lighthouse. This was done to assure that the same quality control was kept throughout the testing of all the features. To stay consistent during the testing, steps provided by Google were followed describing how the three missing features could be tested [50]. The excluded features from Lighthouse test suites are the following: F6 Site works cross-browser, F7 Page transitions do not feel like they block on the network, F8 Each page has a url. The table in section 2.4 describes how the features were tested.

4 Results

4.1 RQ1 How does the literature define a Cross-Platform

app, Web app and Native app

In order to answer the research question regarding how the literature defined the environment terms, we presented three new definitions of current ones. These definitions were derived from the literature, compiling 5 articles’ definitions into a new one for each of the terms. It is important to note that the definitions are influenced by us as authors and other researchers active in the field:

- Native app: An application developed to function on one specific operating system using the programming languages, tools and development environments that the platform specifies. These techniques allow apps to provide a rich user experience and gain access to the underlying hardware. Each operating system has its own architecture and application development support, which makes an application developed towards one operating system only usable on that specific platform [8], [13], [23], [24] - Web app: A non installable application developed using HTML, CSS and JS.

A Web app uses the browser as runtime environment and can therefore be accessed by any operating system that supports web browsers. A Web app should also be optimized for mobile devices. Web apps have limited access to hardware, software or data on the device [8], [22]–[24]

- Cross-platform app: An application developed from one codebase but distributed across multiple platforms. A Cross-platform app has two goals. The first being to achieve the functionality and feel of a Native app. The second being to distribute itself to as many operating systems as possible. The term Cross-platform apps can be divided into a big number of categories to cover the different approaches that have emerged from the industry as how to implement such an application [8], [13], [22]–[24]

4.2 RQ2 How can a Web app be converted into a Progressive

Web

App?

In the method and solution section a detailed description as to how the Web app was converted to a PWA is given. The descriptions included all the steps that were taken when implementing the core features of a PWA. These included an overview of the implementation found in section 3.1 and a description of the testing found in section 3.2. In the solution section, the PWA is described mostly through explanation of how the features were implemented. This is due to fact that these changes are not affecting the content or the visuals of the app, when comparing it to the Web app. It is instead the wrapper that differs between the applications, and these differences can be seen. They include:

● A yellow notification bar ● No URL bar

● The app running in a separate window from the browser

In figures 11a and 11b, the yellow notification bar is displayed. Traditionally the URL bar would be placed between the notification bar and the content of the app. This is not the case for the PWA as it is hidden. Although the URL bar is also hidden in figure 2a and 2b which displays the Web app, this is simply a feature in Chrome and not something that is a feature of the Web app.

4.3 RQ3 How does a Cross-Platform app, a Web app and a

Progressive Web App differ in performance?

There were six metrics involved in the testing, and each metric was tested times on different devices. Every phone therefore produced 24 values (after averaging); 6 for each of the three implementations. They are presented in figures 12a-12g Each measurement was taken on the same location, using the same network. The phones had no other apps open during testing, and between every test the app was terminated. Every metric value was taken ten times for every solution on each phone, e.g. a value for TTI was taken ten times on the Xiaomi, OnePlus, Samsung and Huawei. An average of the ten measurements was then computed in order to get a comparable results to the other solutions. This average is presented in the graphs below. A total summation of all the averages is also presented in Figure 12g, showing which solution took the longest overall time to load on the different phones.

Figure 12a. Product Estimated Input Latency

Figure 12c. Filtering Checkbox Estimated Input Latency

Figure 12e. First Paint

Figure 12g. Total time taken

Table 2. Overview of the fastest to slowest app, measured in total time over all phones

First Second Third

First paint PWA WEB CORDOVA

First contentful paint

CORDOVA PWA WEB

Time to interactive

CORDOVA PWA WEB

Product EIL PWA WEB CORDOVA

Compare EIL CORDOVA WEB PWA

Filtering checkbox EIL

PWA WEB CORDOVA

Total time PWA CORDOVA WEB

5.1 RQ1 How does the literature define Cross-Platform

apps, Web apps and Native app

The first research question required the creation of three extensive definitions. These definitions had to be high level descriptions in order to cover applications with functionality and structures that are deviations of what is considered typical for the term. They also do so without overlapping with the other definitions. The definitions were constructed from two different types of articles, which were found during the literature review. The first type of article defined their own terms in one of two ways. Some created additional definitions to the already existing three terms: Native, Web and Cross-platform, while others instead created completely new definitions for the terms [6], [7], [12], [18], [20]–

[22], [30], [37]. The second type of article did not define the terms themself but

the ones used to create our definitions. The second type of article exists as an example of why our definitions were needed.

5.2 RQ2 How can a Web app be converted into a Progressive

Web App?

When converting a Web app into a PWA, the eight functional requirements listed in Table 1 had to be implemented. The implementation will vary depending on the original Web app, but the required core functionalities will be the same. One cannot create a general step by step process for converting any Web app to a PWA, which is why one must look at the outlined requirements instead. Many of these requirements is implemented by the use of specific techniques such as the Service Worker. One cannot deviate from these since the app would then not be recognized by the browser as a PWA, and could therefore e.g. not be added to the home screen.

5.3 RQ3 How does a Cross-Platform app, a Web app and a

Progressive Web App differ in performance?

The performance comparison can be summarized by comparing the total time taken on the different devices, as seen in Table 2. Note that total time taken is not a metric that we outlined, but exists rather as a summary of the above six metrics. What we can see in the table is the following:

1. The PWA and Cordova app came in first in three metrics each

2. The Web app usually came in second, with the exception of two metrics where it was last

3. The Cordova app came in last three times, whereas the PWA only did once Therefore, the PWA can be seen as the fastest app overall with the exception of when running on the Samsung device. Both the Cordova app and the Web app were faster there.

The Web app came in second place four times and third place two times. Based on our metrics, we can see that both the Cordova app and the PWA has better overall performance than the Web app. We can also see that the Cordova app and the PWA has performance that are often very close. However, the PWA was still faster overall and came in last place fewer times than the Cordova app did. A potential reason for the differences mentioned in the section above is that the PWA and Cordova app saves more information locally on the phone than the Web app does. The Cordova app does this upon installation, and the PWA utilizes the application cache. This means that they do not rely as much on network requests since they can fetch more information locally, which is faster. One could develop a caching strategy for the Web app as well which would improve its performance. However, since having an application shell architecture and caching as much static content as possible is one of the things that makes a PWA a PWA, we chose not to implement that in the Web app. This was because doing so would be taking the Web app a step in the direction of becoming a PWA. Instead we chose to compare the original Web app to a fully implemented PWA, which is the result that our client and potentially other clients find most relevant.

The results were in line with what could be expected. The Cordova app and the PWA has better opportunities for faster performance, mostly due to the reduced number of network requests. One should still note that there is not a clear pattern across all phones even within the same metrics. There are also variations in results between the different metrics, meaning that we cannot say that a single solution is always the best.

6 Discussion

6.1 RQ1 How does the literature define Cross-Platform

apps, Web apps and Native apps

When searching for articles we were aware of the scarce number of previous papers covering PWAs. To compensate for this we also looked at other research areas such as Cross-platform, which is an area more explored than that of PWAs. It is important to note that PWAs is a part of this bigger research area, since it can be seen as a Cross-platform solution. It should also be said that Cross-platform at the time of writing is a much broader term than both Native and Web. As more companies create more applications for different purposes, it is only natural that the gap between the applications within the terms starts to widen. If the ecosystem continues to grow, it is almost inevitable that the current definitions will have to be altered or that new terms will emerge. This might explain why we see researchers trying to define their own subcategories to Cross-platform [6], [7], [12], [20]–[22]. With this in mind, the goal of this thesis was never to create new terms but instead to summarize and redefine the current ones.

It was partly because of different authors defining their own subterms of Cross-platform that we felt the need to seperate us from any potential

misinterpretations regarding the usage of the same terms. This explains why the result is a assemblage of many sources. That was done to make sure that this thesis only contain parts of definitions that are widely agreed upon throughout the literature.

With the decision to create our own definitions, we mainly focused on doing so for the sole purpose of this thesis. We still acknowledge that the result of our definitions will add to the body of knowledge. We also argue that the presented result is generally applicable to similar works, hoping that it can be used by authors who feel the same need to define one, two, or all three of the terms. We acknowledge that these definitions might not be relevant in the future as the ecosystem grows and changes their meaning. This is especially apparent when looking at how Cross-platform has grown during recent years. Therefore, future research could include conducting a similar literature review in the close future. This would be relevant primarily for two reasons. Firstly to create new, relevant definitions that depict the current ecosystem. Secondly to perform a comparison of how the usage and definition of the terms have changed since this thesis was conducted.

6.2 RQ2 How can a Web app be converted into a Progressive

Web App?

From the result we can see the conversion from Web app to PWA was mostly successful. This is enforced by looking at the Lighthouse PWA testing suite. Since some PWA core functionalities had to be manually tested we can not be entirely sure that these were implemented correctly. This is why we cannot say at full certainty that it was fully successful. However, we are certain that the phones recognized the app as a PWA. It is still worth mentioning that the manually performed tests were conducted according to the descriptions of how those features should be tested written by the developers of Lighthouse, Google. The Web app was provided by our client Axis. This meant that none of us had any prior experience with the application, which ultimately lead to us having to set aside time in order to understand the functionality of the code. With this in mind, the conclusion was drawn that it still was beneficial to use an already existing Web app compared to the alternative of instead building a brand new one. Time was therefore saved on e.g. learning new coding practices, due to the current Web app already following the clients preferred ones. Another benefit was the added layer of realism and generalizability gained from the usage of a fully functional and deployable Web app. We felt that such a Web app created a good baseline for the PWA as it was a taken from the industry.

During the alteration of the Web app, we found that some of the core features of a PWA were already implemented. In these cases, the descriptions are a recollection of how these features were realised. We would like to acknowledge that these descriptions might not be as accurate and detailed as the descriptions of the features that were implemented by us. The recollections are a combination of our understanding of the code as well as input offered by the original developer of the Web app, which might result in them not being as rich and thorough as the others.