Comparison between Progressive Web App and Regular Web App

(1)

URI: urn:nbn:se:bth-18138

Comparison between Progressive Web App and Regular Web App

By Farid Said Tahirshah

june 2019

(2)

This thesis is submitted to the Faculty of Computing at Blekinge In- stitute of Technology in partial fulﬁllment of the requirements for the bachelor degree in Software Engineering. The thesis is equivalent to 10 weeks of full time studies.

Contact Information:

Author(s):

Farid Said Tahirshah

E-mail: ﬂatra20@yandex.ru

University advisor:

Michel Nass

Faculty of Computing

Blekinge Institute of Technology SE-371 79 Karlskrona Sweden

Internet : www.bth.se Phone : +46 455 38 50 00 Fax : +46 455 38 50 57

(3)

Abstract

In 2015 the term Progressive Web Application was coined to de- scribe applications that are getting the advantage of all Progres- sive App features. Some of the essential features are ofﬂine support, app-like interface, secure connection, etc. Since then, case studies from PWA’s implementation showed optimistic promises for improving web page performance, time spent on site, user engagement, etc. The goal of this report is to analyze some of the ef- fects of PWA. This work will investigate the browser compatibility of PWA’s features, compare and analyze performance and memory consumption effect of PWA’s features compared to Regular Web App. Results showed a lot of the features of PWA are still not supported by some major browsers. Performance benchmark showed that required https connection for PWA is slowing down all of the PWA’s performance metrics on the ﬁrst visit. On a repeat visit, some of the PWA features like speed index is outperforming the Regular Web App. Memory consumption on PWA increased more than 2 times the size of RWA. The conclusion is that even if some features are not directly supported by browsers, they still might have workaround solutions. PWA is slower than regular web app if https on your web server is not optimized. Different browsers have different memory limitations for PWA caches. You should implement https and PWA features only if you have HTTP/2 support on your web server, otherwise, performance can decrease.

Keywords: Progressive web application, comparison, performance, memory consumption.

(4)

1 Introduction

About twenty years ago a website looked like a page of a book, and the only use case was to read information on the page. Web sites were used mostly on a desktop, but with progress of mobile internet connections, internet became more available on mobile devices. From 2009 to 2018 the percentage of global web pages served to mobile phones increased from 0.7% to 52.2% [28] and global market share of mobile phones increased in similar manner from 0.67% in January 2009 to 47.96% in February 2019 [27]. With the spread of smartphones and mobile usage of internet, companies started to give more attentions to mobile users by developing native mobile applications and more mobile friendly websites. Now you can ﬁnd native mobile applications from pretty much any company. Even those that you never thought about, for instance

“Oral-B” [20] has an app that helps you control your electric tooth- brush, see the stats and much more. Even though native application have tons of possibilities and a lot more engaging than a regular web site, they also have some cons, the most important one is that they are not cross platform. You have to build new app for every Operating Sys- tem(OS) also for the most part people do not want to install your native application, because stats shows that majority of usage time is spent on top 3 application[22]. There is compromise like Hybrid app that are written in JavaScript, HTML, CSS and can be run on any mobile device, can simulate native app experience, but the code is run through middleware API. The problem with Hybrid apps can be bugs on middleware that you as a developer have no power to ﬁx, they are slower than native apps, they can be slow to update their API after On the other hand regular web sites don’t provide same interactive usability as native application. This made web developers think about development of alternatives for native apps, which lead to formation of Pro-

(8)

is supposed to have features from native applications which might con- cern web developers who are not used to building native applications.

This paper will explore unique features of PWA, their compatibility limitations and comparison between PWA and Regular web app(RWA) in terms of memory consumption and performance.

1.1 Background

1.1.1 Regular web application

These are regular web applications that you may ﬁnd on the internet.

They are hosted mostly through HTTP protocol, but HTTPS is also pop- ular, from top 1 million web sites 57%[30] are using HTTPS. There are mobile optimized websites. There is almost no difference between mobile-web apps and PWA, since both using interactive design, mobile friendly. The differences are PWA’s features, some of them RWA may have i.g. like https, but others are unique i.g. service workers, ofﬂine mode, push notiﬁcations etc...

1.1.2 Progressive Web Application

In 2015 designer Frances Berriman and Google Chrome-engineer Alex Russell come up with name Progressive Web Application to all web apps that have Progressive App features like Transport Layer Security, Ser- vice Workers, Manifest, Responsive Design, access to hardware, push notiﬁcation, ofﬂine support etc... [23] Some attributes might be new, some might be 4-5+ years old, but basically PWA is a web site that have best practices of web development for mobile that can simulate native-like experience.

(9)

1.1.3 Service Worker

Service worker is a script written in JavaScript and runs on the background. It doesn’t have access to a web pages, it only have access to data that goes in-between PWA and Internet. Service worker is an

"event driven" script, it needs an event to trigger a function. Meaning that whenever a user opens a page, the request and response first will go through the service worker and service worker will have all control over the requests/responses. Features like push notification and caching are run in service worker script [7] In the following figure 1 you may see how it interacts with other browser components.

Figure 1: Service Worker

1.2 Purpose

(10)

actively pushing idea of PWA and posting case studies on how adding PWA features will boost your web-apps performance, users average time spent on site, increasing user engagement etc[21][9]. Famoues companies like Twitter, Uber, Forbes, Alibaba etc already implementing PWA on their web apps[25], While I can’t test promises like time spent on site or user engagement, I can test how PWA features affects web apps performance and what are limitations of ofﬂine support. Hence this paper will cover unique features of PWA that a general web pro- grammer might be unfamiliar with and compare the result with regular web site to see if it worth it to build/rebuild your web site with PWA features and requirements.

1.3 Scope

In this work two web sites will be tested. One is PWA and the other is a regular web site(RWA). Focus of the implementation will be on PWA features. Hence, due to time constraint and for maximum similarity to real life web app a template for a single page web app will be used.

Only two parameters will be measured, one is performance in loading pages and the other is memory consumption. For the sake of comparison fairness, you could measure a lot of other parameters, but due to time constraint only these two will be tested. This experiment is only testing one type of web app, which is single page web app that will not be speciﬁcally optimized for either http caching or service worker caching. PWA features will be added the way they are in the guidelines. There won’t be speciﬁc performance optimization except server side compression on both of the web apps. Result of this work won’t be representative of every type of PWA nor RWA, since all web apps have different structure, this work can’t cover them all. This paper fo- cuses only on PWA and Regular web app, and won’t be covering Native mobile nor Hybrid app’s since they both need to be installed on a mobile device before they can be used, while PWA and Regular web apps are available on any device that have a browser and don’t have to be installed. Performance testing will be done on Chrome 74. since Light-

(11)

house is run on Chrome’s devtool and Chrome have most mobile market share(56%), next to it is Safari(21%)[26]. Available devices for benchmark are laptop with Windows 10 for performance testing and iPhone 8, iOS 12.2 for memory tests and screenshots of how the PWA app will look like.

(12)

2 Research Questions

RQ1 What are the limitations of requirement features of PWA in terms of cross-browser compatibility?

RQ2

How does Progressive Web Application features affect the performance of the web app compared to a regular web app?

RQ3

How does ofﬂine mode feature of Progressive Web Appli- cation affect memory consumption of the end-user device comparing to regular web-app and what are the limitations of cache storage for one PWA?

The goal of the thesis is to get to know what are the differences between PWA and a RWA. This RQ1 will contribute to the start of the work with clariﬁcations of compatibility limitations that you need to keep in mind in order to implement PWA features. The goal with this RQ is to help readers understand pros and cons of PWA features.

Performance is always one of the top priorities when it comes to web applications. One of the Google experiments showed that page load increasing from 0.5 sec to 0.9 sec made google searches decrease by 20% [13]. Meaning that people usually don’t like to wait too long for a web page to load. Hence this RQ2 will compare PWA and RWA to check how PWA features affect a web app’s performance.

Offline mode that PWA provides is exciting to think about because it allows you to control cache data even when offline while with default HTTP cache you can’t use on offline network. But it is unclear where you should store all the cached data. Is there a limit for usage per PWA and is there a difference between operating systems/devices/browsers in cache limit for PWA? How does it work on real apps? This is one of the concerns with PWA that this thesis will aim to clarify.

(13)

3 Research Method

To get the answer for RQ1 a qualitative study was made. Most references were collected from ofﬁcial PWA documentations/guidelines by Google, Mozilla and W3c. The answer for RQ1 contributed for better understanding of PWA’s limitations and lead to selecting appropriate testing strategy for RQ2 and RQ3. To answer RQ2 a quantitative experiment was made. PWA’s and RWA’s performance were tested and results were compared to determine how PWA features were affecting performance of the web app compared to RWA. To answer RQ3 both qualitative and quantitative approach were used. The reason for it was because i.g. Firefox have clear documentations on memory limitation while other browsers like Safari have poorly documented or not documented at all, so an quantitative testing was made.

3.1 Description of the method

To get the references for RQ1 following keywords were used: progressive web applications, features, compatibility, browser support, chrome, safari, mozilla. The results were collected and used as a basis for next research questions.

As a prerequisite to the RQ2, two web apps were used as a representa- tives for PWA and RWA. As a starting point for both apps I downloaded a template for single page restaurant. This template were used as an RWA and on its exact copy the PWA features were implemented, re- sulting in two identical web apps one with PWA features and the other without. Both web apps were launched to the same hosting provider with identical service packet. Both apps using compression, which were

(14)

web app for the ﬁrst time without cache or service workers and when the user would load our web app second time with installed service worker and cache storage. The web-hosting packet that was used for the web apps had limited bandwidth, so I had to carefully pick number of tests. 60 runs had great balance between accuracy of statistic result and bandwidth usage.

Results from RQ2 produced series of graphs and tables for each performance metrics available in Lighthouse tool. Reason for selecting Light- house as a benchmark tool is because it’s one of the few tools that of- fered repeat view testing, had clear metrics description and it could run

on local device. Another tool that could be used is https://webpagetest.orgwebpagetest.org, but it runs tests on their own servers and on certain ones queue times

were too big to perform multiple tests. Result on first run showed slow down for PWA. To find the reason for it, additional test were executed on RWA to test only affect of https and exclude other PWA features. 9 runs for each http://example.com(RWA) and https://example.com(RWA) to find out if https were slowing down the loading speed of the PWA.

To answer RQ3 following keywords were used: progressive web application, storage, cache, limitation, browser support. To get limitation for some browsers without clear documentation like Safari, PWA storage

limit testing were performed on this webtool https://demo.agektmr.com/storage/demo.agektm To get approximate storage quota for Chrome and Firefox a special

command (listed below) were executed in browser console.

1 if (’storage’ in navigator && ’estimate’ in navigator.

storage) {

2 navigator.storage.estimate().then(({usage, quota}) => {

3 console.log(‘Using ${usage} out of ${quota} bytes.‘);

4 });

5 }

(15)

3.2 Implementation of PWA

Google have a checklist for PWA following features are the main ones [8]

• Web App Manifest.

• HTTPS.

• Service Worker.

• Ofﬂine Mode

• Push notiﬁcation

• Installability

Push notiﬁcation is a feature that makes the PWA a native-like experience, but it’s not required to have on your web app to be called a PWA.

That’s why except Push notiﬁcation, all above features will be implemented in our PWA.

3.2.1 Web App Manifest

A Web App Manifest is a JSON ﬁle that contains metadata about web applications and how it should behave when installed. Appendix A on page 49 is manifest json ﬁle in our Progressive Web Application.

Web app manifest were created on https://app-manifest.ﬁrebaseapp.com/app- manifest.ﬁrebaseapp.com One of the advantages of this generator is

(16)

3.2.2 HTTPS

HTTPS (Hypertext Transfer Protocol Secure) is an extension of regular internet communication protocol HTTP(Hypertext Transfer Protocol).

My hosting providers by default using HTTP/1.1 protocol, so to add https feature I made use of https://www.sslforfree.com/"Let’s Encrypt"

Certiﬁcate, all the browsers support it, but it provides only Domain Validation.

3.2.3 Service Workers

To install a service worker on the PWA, an empty ﬁle named sw.js were created on the root folder of the web app, and in main JavaScript ﬁle named app.js bellow code were added.

1 if(’serviceWorker’ in navigator)

2 {

3 window.addEventListener("load", () =>{

4 navigator.serviceWorker.register("sw.js")

5 .then(console.log(’Service worker registered’)

6 .catch(console.error);

7 });

8 }

It doesn’t have to be in app.js file, it could be added directly to index.html file through "<script src=" attribute. Code above checks if your browser supports service worker API, if it is then it will register service worker when your page is loaded. That’s it, the service worker have been created, but it doesn’t do anything since sw.js file is empty.

Installing Service worker is required step before enabling other features like Ofﬂine mode and installability. [7]

(17)

3.2.4 Ofﬂine Mode

One of the main features of PWA is the offline mode. To implement offline mode a tool called Workbox [33] by Google were used, it generates precache code automatically for offline mode through cli commands.

If workbox cli was used for precache code in sw.js file, then additional src-sw.js file should be manually created to cache runtime data of API responses from backend or any other API service providers. Following code is workbox-config.js file:

1 module.exports = {

2 "globDirectory": ".",

3 "globPatterns": [

4 "**/*.{js,css,eot,svg,ttf,woff,woff2,otf,png,jpg,ico,gif ,html,json,scss}"

5 ],

6 "swDest": "sw.js",

7 "swSrc": "src-sw.js"

8 };

Reason for this structure is because sw.js generates through cli build command and if you put your manually written code to sw.js then after rebuilding of sw.js - all your manually written code in sw.js ﬁle would be deleted.

3.2.5 Installability

Installability makes it possible for the user to add the app to the Home screen. See appendix C on page 52 for code that was used for the PWA.

(18)

4 Related Work

This chapter will summarize and analyze related work about performance of PWA features.

Performance Analysis of TLS Web Servers (2006)[5] This paper analyses performance affect of adding TLS 1.0 protocol on HTTP connection, with 1024 bit RSA encryption key. The results showed that TLS does slowdown performance of web server, because encryption occu- pies server’s CPU time and this reduces throughput speed. More CPU power can decrease the difference, but will not remove it completely.

Performance Analysis of SSL/TLS(2014)[24] This paper researched how security layer affected response time of FTP, IMAP and POP3 servers when sending ﬁles/emails with size of 20,40,60 and up to 4000MB. The results showed that for secure connection response time slightly increased.

Measuring the Real-world Performance Impact of Service Work- ers(2016)[31] This experiment measured difference between service worker cache and default HTTP cache, they are using Google Analytics to collect performance metrics from their PWA. They divided users to 3 categories in relation to service worker support:

• supported controlled, returning visitor

• supported not controlled, returning visitor

• supported not controlled, new visitor

Two metrics were measured: "page load time" and "time to ﬁrst paint".

Results showed that "controlled, returning visitors" had fastest "page load time" and "time to ﬁrst print", second fastest were "not controlled, returning visitors" and slowest were "not controlled, new visitor". As- suming that controlled, returning visitors were using service worker’s

(19)

cache and not controlled, returning visitors were using default http cache. About 15% of users were from unsupported browsers, but their numbers weren’t counted. They were also saying that they were optimizing their web app for service worker, and didn’t spend much time optimizing for HTTP cache. Also affect of adding tls to the web app is not measured.

(20)

5 RESULT and ANALYSIS

5.1 Progressive Web App feature compatibility limi- tations

This section will be answering the Research Question 1. https://caniuse.com/feat=web- app-manifestcaniuse.com provides up to date tests for compatibility of

web features on different browsers.

5.1.1 Web app manifest

Figure 2: Web app manifest browser support

Figure 2 shows support for web app manifest in different browsers versions.

Red means no support, Green means full support, Olive color mean partial support. Browsers versions inside cells with black borders are current versions. Versions that are bellow these cells are "upcoming"

versions of the browsers.

(21)

All icons and some properties i.g. theme color and background color in the manifest doesn’t work on iOS Safari and older browsers. To ﬁx compatibility problem a script by Google library named "pwacompat"[10]

can be used.

5.1.2 HTTPS

HTTPS is providing secure data transfer between computer network by using Transport Layer Security protocol (TLS) or Secure Socket Layer (SSL) while HTTP using no security protocol. SSL is older version while TLS is newer one. Right now TLS is used everywhere, even though it is still referred as SSL[15].

Protocol Published Status SSL 1.0 Unpublished Unpublished

SSL 2.0 1995 Deprecated in 2011 [12]

SSL 3.0 1996 Deprecated in 2015 [11]

TLS 1.0 1999 Deprecation planned in

2020[32][4][19][29]

TLS 1.1 2006 Deprecation planned in

2020[32][4][19][29]

TLS 1.2 2008 TLS 1.3 2018

Since TLS 1.0 and TLS 1.1 is going to be deprecated soon enough, following ﬁgures 3 and 4 show browser support for TLS 1.2 and TLS 1.3 only.

(22)

Figure 3: Browser support for TLS 1.2

Figure 4: Browser support for TLS 1.3

5.1.3 Service Worker

Figure 5 on page 23 shows that all major browsers except Microsoft Explorer have support for service workers, last one to join the train was Apple Safari in April 2018.[2][6]

Service Worker Support means that you can at least register the Service worker on these browsers, it doesn’t mean all the feature of PWA like ofﬂine mode, push notiﬁcation that is supposed to run through Service Worker script will be supported.

(23)

Figure 5: Browser support for Service Worker

5.1.4 Ofﬂine mode

Ofﬂine mode is enabled through browser caches that is separate from default HTTP cache.

Figure 6: PWA cache browser support [2]

Figure 6 shows from which versions of the browsers the Ofﬂine mode is supported. For instance all Chrome Browsers from version 46 an above are supporting ofﬂine mode.

(24)

a PWA.

Prerequisite:

• Hypertext Transfer Protocol Secure (HTTPS)

• Manifest that includes name or short_name, start_url, display mode should be anything but browser mode, 192x192 and 512x512 sized icons.

• Service worker that has fetch event handler [14]

When all of the requirements are matched you will be able to offer the user to install your app, by adding a button that will trigger the install prompt. Figure 7 shows how Install Prompt looks like on Google Chrome.

(25)

Figure 7: Install prompt browser support

Figure 8 shows that most of the browsers doesn’t support PWA’s install prompt.

Figure 8: Install prompt browser support [16]

(26)

But some of the browsers have their own way of adding a web app to the home screen. For instance, iOS Safari have no limitations on what type of web app can be added to home screen, but PWA have manifest that makes the app have an icon and also display mode property that makes it different than regular web site when launched. Appendix B on page 51 shows how to add the web app to home screen without Install prompt on iOS 12.2, Safari, iPhone 8.

(27)

5.2 Performance comparison

This section will be answering the Research Question 2.

Tables in this chapter 5.2 representing results from Google’s Light- house performance benchmark. Both app were tested 60 times each.

30 with ﬁrst view (cleared storage) and 30 for repeat view (uncleared storage). Settings below were used for tests to get one of the slowest mobile networks:

• Device: Emulated Nexus 5X

• Network throttling: 562.5 ms HTTP RTT, 1,474.6 Kbps down, 675 Kbps up (DevTools)

• CPU throttling: 4x slowdown (DevTools)

• User agent (host): Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36

• User agent (network): Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5

Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3694.0 Mobile Safari/537.36 Chrome-Lighthouse

5.2.1 First Contentful Paint

This metrics measures the time from navigation to the time when the browser renders the ﬁrst bit of content from the DOM. This is an important milestone for users because it provides feedback that the page

(28)

App Mode Metric

Number of

sam- ples

Average Median

Standard Devia- tion

PWA First View

First Con- tentful Paint

30 2.777 s 2.7 s 0.442 s

RWA First View

30 2.653 s 2.5 s 0.37 s

PWA Second View

30 1.157 s 1.1 s 0.128 s

RWA Second View

30 1.207 s 1.15 s 0.202 s

First Contentful Paint benchmark

Table above shows that with ﬁrst view PWA and RWA have had almost the same performance, but on ﬁrst view RWA had better performance.

On average RWA was 4.4% faster than PWA. On second view PWA showed better results with averaging 4.1% faster performance and lower standard deviation of average which means PWA have more con- sistent numbers.

(29)

5.2.2 Speed Index

This is a page load performance metric that shows you how quickly the contents of a page are visibly populated. The lower the score, the better. Charts from this table are in appendix E on page 58

Number of

sam- ples

Standard Devia- tion PWA First

View

Speed

Index 30 18.907 s 15.65 s 6.857 s RWA First

View

Speed

Index 30 15.853 s 15.9 s 1.85 s

PWA Second View

Speed

Index 30 9.037 s 8.6 s 1.518 s

RWA Second View

Speed

Index 30 9.65 s 8.3 s 3.239 s

Speed Index benchmark

Speed index ﬁrst view PWA lost by a big margin 16.2%. But most of the test were more or less with the same performance except ﬁrst 5 runs when PWA got very bad results one of which went up to 43 seconds! I’m not sure why i’ve got this big spikes, but I decided to still include these test runs because other metrics on the same test runs were having normal values.

Second view went better, PWA still had better average performance by

(30)

5.2.3 The Time to Interactive

This metric measures how long it takes a page to become interactive.

The lower the score, the better. Charts from this table are in appendix F on page 60

Number of

sam- ples

PWA First View

Time to Inter- active

30 17.653 s 15.15 s 5.276 s

RWA First View

30 14.87 s 13.55 s 3.078 s

PWA Second View

30 7.9 s 7.8 s 1.315 s

RWA Second View

30 6.823 s 6.35 s 1.438 s

Time to Interactive benchmark

Time to interactive is one of the most important metrics which lets the user to control the page. First view PWA lost by 16% the same as speed index. Because of the ﬁrst 5 runs when PWA had very bad results.

Second view went better, PWA still had worst performance on average by 15.8% compared to RWA.

(31)

5.2.4 First Meaningful Paint

This metric identiﬁes the time at which the user feels that the primary content of the page is visible. The lower the score, the better. Charts from this table are in appendix G on page 62

Number of

sam- ples

PWA First View

First Mean- ingful Paint

30 2.777 s 2.7 s 0.442 s

RWA First View

30 2.653 s 2.5 s 0.37 s

PWA Second View

30 1.157 s 1.1 s 0.128 s

RWA Second View

30 1.207 s 1.15 s 0.202 s

First Meaningful Paint benchmark

(32)

5.2.5 First CPU Idle

This metric measures when a page is minimally interactive. The lower the score, the better. Charts from this table are in appendix H on page 64

Number of

sam- ples

PWA First View

First CPU Idle

30 14.223 s 14.45 s 2.396 s

RWA First View

First CPU Idle

30 13.303 s 13.15 s 0.966 s

PWA Second View

First CPU Idle

30 7.537 s 7.45 s 0.798 s

RWA Second View

First CPU Idle

30 6.457 s 6.05 s 0.977 s

First CPU Idle benchmark

With ﬁrst view numbers PWA lost on average by 6.5% and on second view the numbers became worst( 14.3%), one of the few metrics where PWA became even worst than RWA on second view compared to ﬁrst view.

5.2.6 Estimated Input Latency

This metric measures how responsive the input feature is on the web app. The less, the better. Charts from this table are in appendix I on

(33)

page 66

Number of

sam- ples

Average Median Standard Deviation

PWA First View

Estimated In-

put Latency 30 601.667 ms 605 ms 140.297 ms RWA First

View

Estimated In-

put Latency 30 725 ms 705 ms 131.037 ms

PWA Second View

Estimated In-

put Latency 30 1,075 ms 1,030 ms 297.173 ms RWA Second

View

Estimated In-

put Latency 30 1,061 ms 930 ms 390.255 ms Estimated Input Latency benchmark

On this metrics PWA had 20% better performance than RWA, but on second view PWA had slightly worst average number.

(34)

5.2.7 Performance Score

Lighthouse returns a Performance score between 0 and 100. 0 is the lowest possible score. 100 is the best possible score which represents the 98th percentile, a top-performing site. A score of 50 represents the 75th percentile. The more, the better. Charts from this table are in appendix J on page 68

Number of

sam- ples

Standard Devia- tion PWA First

View

Performance

score 30 24.533 25 3.037

RWA First View

Performance

score 30 26.6 28 3.286

PWA Second View

Performance

score 30 51.433 52.5 5.575

RWA Second View

Performance

score 30 57.067 58 6.878

Performance score benchmark

Performance score represent the overall score of the web apps performance. Table above shows that overall PWA had worst performance than RWA on both First and Second views.

5.2.8 HTTP vs HTTPS

Following table represents benchmark for RWA without any PWA code, executed with, and without https to test how Let’s Encrypt Certiﬁcation affects loading speed of the web app.

(35)

Number of

sam- ples

RWA HTTP

First

View Speed Index 20 15.666 s 15.826 s RWA

HTTPS First

HTTP

Second

HTTPS

Second

View Speed Index 20 3.852 s 3.842 s HTTP VS HTTPS benchmark

Benchmark showed that HTTPS is badly affecting both ﬁrst and second view of the web app. Related research showed that HTTPS can slow down request-response life-cycle if the service doesn’t have optimization for tls like better CPU’s or/and HTTP/2, which in my case it doesn’t.

5.2.9 Summary of the performance benchmark

All of PWA’s metrics showed worst average numbers on first view. First run is not affected by precaching, since service worker is registering when the page is already loaded and web app manifest is simple con- figuration file that runs when the PWA is installed on the device. Ad- ditional benchmark on HTTP vs HTTPS confirms that first view’s performance is affected by HTTPS. On second view PWA showed better results. With metrics like Speed Index, First Contentful Paint and First

(36)

5.3 Memory consumption of PWA comparing to regu- lar web app and limitation of PWA Memory usage

To get answer for RQ3 the memory consuption of PWA and Regular web app were tested on iPhone 8, iOS 12.2 All the data was cleared from Safari browser before test of each web app. Next two images are showing how much total storage was used by PWA on the left and RWA on the right.

Memory Usage of PWA Memory Usage of Regular Web App

(37)

PWA was storing in total 9 MB data from its server + third party library domains, while RWA storing 2,7 MB which in our case makes the PWA 3 times more memory consuming. All the third party libraries have the same size since both apps have the same core source. Except PWA has additional library from "storage.googleapis.com" which is CDN¹for the Workbox library. Uncompressed size of both PWA and RWA is almost the same 2.69MB. PWA is 20kb bigger. On the figure above Browser caches data from both web apps and from third party domains that our web app is getting those libraries. In case of PWA, it preaches all data configured in sw.js file. When service worker is installed and running, it will also cache data from runtime requests. PWA will cache their data separately from default HTTP cache. it’s done because browser doesn’t use default HTTP cache data if connection is offline. So you end up caching same data twice. Because on first view service worker is not installed yet, so browser will cache all data with default HTTP cache.

In case of Regular Web App it caches only compressed version of the app and third party libraries. But if the device is ofﬂine, then browser won’t use them.

Limitation for memory usage per web app is different between browsers.

It is not easy to find official specification on limit numbers for browsers from official sources. For Chrome last up to date numbers were from 2016 and it was <6% of the free space [3]. Since then Storage API changed and for both Firefox and Chrome storage limit can be checked for a origin by the browser console command:

1 if (’storage’ in navigator && ’estimate’ in navigator.

storage) {

2 navigator.storage.estimate().then(({usage, quota}) => {

3 console.log(‘Using ${usage} out of ${quota} bytes.‘);

(38)

When using code above on Chrome 74 with 72GB free space on hard drive results on 16.2GB(22.5%) quota per origin.

Mozzila on the other hand has up to date documentation which states that Firefox can use up to 50% of users hard drive free space, and this limit is called global limit. The other limit called group limit that can use from 10MB to 20% or 2GB (whichever is smaller) of global limit.

For example:

• mozilla.org — group1, origin1

• www.mozilla.org — group1, origin2

• joe.blogs.mozilla.org — group1, origin3

• ﬁrefox.com — group2, origin4

Mozilla.org, www.mozilla.org and joe.blogs.mozilla.org are the same group and collectively they can use up to 20% or 2GB of global limit.

ﬁrefox.com is separate group and it can alone use 20% or 2GB of global limit [17].

navigator.storage.estimate() on Firefox with 72GB free space results on 2GB quota which conﬁrms their rule.

Official offline storage quota for iOS Safari is hard to find. Last research about it was in 2014 and it showed 50MB[13] quota per domain and since then every article uses this number, even most recently ones, but testing on iOS 12.2 Safari showed that PWA could precache about 61MB data. Following figure 9 on page 39 shows precache storage for domain hitting limit on 61MB.

(39)

Figure 9: Precache iOS limit test

It is not including IndexedDB which you can also use for ofﬂine support. Test on https://demo.agektmr.com/storage/ from iOS 12.2 Safari for IndexedDB reached until 6.5GB, but then it refreshed the page and showed error "Not supported on this browser". On following ﬁgure 10 on page 40 captured 5GB av items in IndexedDB (last screenshot before page refreshed).

(40)

Figure 10: IndexedDB storage test

Figure 11 on page 41 Edge have quota depending on volume size.

(41)

Figure 11: Microsoft Edge storage limitation [1]

5.3.1 Cache eviction

Both Firefox and Chrome used LRU(Least Recently Used) when browser hits overall quota. If global limit passed, then browser doesn’t trim, it will delete all data from last accessed domain, if it wasn’t enough, browser will repeat this procedure with next domain until memory usage will go under quota. [17].

Safari and Edge have no eviction.[18].

5.4 Summary of Memory consumption and limitation RQ

In terms of data storage on users device PWA was more memory consuming than RWA. By cause of browsers was storing PWA’s data twice, once in default HTTP browsers cache and then on Service worker’s

(42)

6 Conclusion

Majority of internet users are no longer on desktops. Mobile surﬁng is now about equal to desktops, which makes mobile experience very important for companies providing services through internet. Thus web approaches for creating both desktop and mobile-friendly applications are very valuable. Usually this type of approaches have their cons, to ﬁnd them this comparison of Progressive web app and Regular web app was made.

Results showed that except Chrome that have full support of PWA, there are still a lot of PWA features that are not compatible with most of the browsers, but at the same time most of the browsers support service workers and PWA’s caches, which is enough for having ofﬂine support.

There is also some of the features like manifest and install prompt that are not directly supported by iOS Safari and some older version of browsers, but you still can have workaround solution to implement them. Generally most of the browsers are working on implementation of PWA features, some of them are already done it for years(Chrome) and some of them just started like (iOS Safari).

In terms of performance Progressive web app was slower that Regular web app on both ﬁrst view and repeat view. The slowdown is caused by https. There is solutions to minimize the problem by having better hardware on server or/and using HTTP/2 protocol that reduces over- head caused by https protocol. This could make PWA faster than RWA.

Memory tests showed that PWA’s caching doesn’t work on ﬁrst load, it leaves room for default HTTP cache to store all the cacheable data on browsers, this may result on storing the same data twice in the two different caches. This causes big memory consumption of PWA compared with RWA.

PWA limitations on storage size is depending on the browsers and on available free space on users hard drive, but usually iOS safari’s 60MB quota(not counting indexedDB) is the smallest PWA cache which seems

(43)

like enough for most of the web-apps. The biggest is Google Chrome’s that in case of the 72GB free space: showed 16,2GB domain quota including IndexedDB.

(44)

7 Future Work

Since https affected the performance of the PWA, one could redo the benchmark but on hosting that provides HTTP/2 for both PWA and RWA. To see if it makes any difference. More test could be done on other devices, such as any android smartphone or windows phone. Ad- ditional research could be done on hardware affect of PWA, for example cpu proﬁling and battery usage. Another approach for this experiment might be using real life data, to collect data from different devices and connections.

(45)

References

[1] Ali Alabbas. Service Worker: Going beyond the page. URL: https:

//channel9.msdn.com/Events/WebPlatformSummit/Microsoft- Edge-Web-Summit-2017/ES05 (visited on Apr. 24, 2019).

[2] Jake Archibald. Is Service Worker Ready? URL: https://jakearchibald.

github.io/isserviceworkerready/ (visited on Apr. 15, 2019).

[3] Joshua Bell. State of Ofﬂine Storage APIs. URL: https://docs.

google.com/presentation/d/11CJnf77N45qPFAhASwnfRNeEMJfR- E_x05v1Z6Rh5HA/edit#slide=id.p (visited on Apr. 24, 2019).

[4] David Benjamin. Modernizing Transport Security. URL: https://

security.googleblog.com/2018/10/modernizing-transport- security.html (visited on Apr. 24, 2019).

[5] PETER DRUSCHEL CRISTIAN COARFA and DAN S. WALLACH.

Performance Analysis of TLS Web Servers. URL: https : / / dl - acm-org.miman.bib.bth.se/citation.cfm?id=1124155 (visited on Apr. 24, 2019).

[6] Maximiliano Firtman. Progressive Web Apps on iOS are here.

URL: https : / / medium . com / @firt / progressive - web - apps - on-ios-are-here-d00430dee3a7 (visited on Apr. 15, 2019).

[7] Matt Gaunt. Service Workers: an Introduction. URL: https : / / developers.google.com/web/fundamentals/primers/service- workers/ (visited on Apr. 15, 2019).

[8] Google. Progressive Web App Checklist. URL: https://developers.

google.com/web/progressive-web-apps/checklist (visited on Mar. 12, 2019).

[9] google.com. Case Studies. URL: https://developers.google.

com/web/showcase/ (visited on May 10, 2019).

(46)

[11] IETF. Deprecating Secure Sockets Layer Version 3.0. URL: https:

//tools.ietf.org/html/rfc7568 (visited on Apr. 24, 2019).

[12] IETF. Prohibiting Secure Sockets Layer (SSL) Version 2.0. URL:

https : / / tools . ietf . org / html / rfc6176 (visited on Apr. 24, 2019).

[13] Eiji Kitamura. Working with quota on mobile browsers. URL: https:

//www.html5rocks.com/en/tutorials/offline/quota-research/

#toc-safari (visited on Apr. 24, 2019).

[14] Pete LePage. Add to Home Screen. URL: https://developers.

google . com / web / fundamentals / app - install - banners/ (visited on Apr. 21, 2019).

[15] Mike. Why people who know better still say ’SSL’. And ’hover- board’. URL: https://certsimple.com/blog/ssl-or-tls (vis- ited on Apr. 24, 2019).

[16] mozilla. BeforeInstallPromptEvent.prompt(). URL: https://developer.

mozilla.org/en-US/docs/Web/API/BeforeInstallPromptEvent/

prompt#Browser_compatibility (visited on Apr. 24, 2019).

[17] mozilla. Browser storage limits and eviction criteria. URL: https:

//developer.mozilla.org/en- US/docs/Web/API/IndexedDB_

API/Browser_storage_limits_and_eviction_criteria (visited on Apr. 24, 2019).

[18] Addy Osmani and Marc Cohen. Ofﬂine Storage for Progressive

Web Apps. URL: https://developers.google.com/web/fundamentals/

instant-and-offline/web-storage/offline-for-pwa (visited on Apr. 16, 2019).

[19] Kyle Pﬂug. Modernizing TLS connections in Microsoft Edge and Internet Explorer 11. URL: https://blogs.windows.com/msedgedev/

2018/10/15/modernizing-tls-edge-ie11/ (visited on Apr. 24, 2019).

[20] PG Productions. Oral-B. URL: https://itunes.apple.com/us/

app/oral-b/id698092608?mt=8 (visited on Mar. 12, 2019).

[21] pwastats.com. pwastats. URL: https://www.pwastats.com/ (vis- ited on May 10, 2019).

(47)

[22] Felix Richter. App Users Spend 77% of Their Time on Their Top 3 Apps. URL: https://www.statista.com/chart/3835/top-10- app-usage/ (visited on May 10, 2019).

[23] Alex Russell. Progressive Web Apps: Escaping Tabs Without Los- ing Our Soul. URL: https : / / infrequently . org / 2015 / 06 / progressive - apps - escaping - tabs - without - losing - our - soul/ (visited on Mar. 12, 2019).

[24] Asma Sajid. Performance Analysis of SSL/TLS. URL: https : / / www.researchgate.net/publication/321348984_Performance_

Analysis_of_SSLTLS (visited on Apr. 24, 2019).

[25] Kimberly de Silva. 5 Big Companies Doing Progressive Web Apps Right. URL: https : / / www . biznessapps . com / blog / 5 - big - companies-progressive-web-apps-right/ (visited on May 10, 2019).

[26] Statcounter. Mobile Browser Market Share Worldwide Feb 2018 - Feb 2019. URL: http://gs.statcounter.com/browser-market- share/mobile/worldwide/#monthly-201802-201902 (visited on Mar. 12, 2019).

[27] statcounter. Desktop vs Mobile vs Tablet Market Share World- wide Jan 2009 - Feb 2019. URL: http://gs.statcounter.com/

platform-market-share/desktop-mobile-tablet/worldwide/

#monthly-200901-201902 (visited on Mar. 12, 2019).

[28] Statista. Percentage of all global web pages served to mobile phones from 2009 to 2018. URL: https://www.statista.com/

statistics/241462/global-mobile-phone-website-traffic- share/ (visited on Mar. 12, 2019).

[29] Martin Thomson. Removing Old Versions of TLS. URL: https : //blog.mozilla.org/security/2018/10/15/removing- old- versions-of-tls/ (visited on Apr. 24, 2019).

(48)

[31] Philip Walton. Measuring the Real-world Performance Impact of Service Workers. URL: https://developers.google.com/web/

showcase/2016/service-worker-perf (visited on Apr. 24, 2019).

[32] Christopher Wood. Deprecation of Legacy TLS 1.0 and 1.1 Ver- sions. URL: https://webkit.org/blog/8462/deprecation-of- legacy-tls-1-0-and-1-1-versions/ (visited on Apr. 24, 2019).

[33] Workbox. JavaScript Libraries for adding ofﬂine support to web apps. URL: https : / / developers . google . com / web / tools / workbox/ (visited on Apr. 16, 2019).

(49)

8 Appendix

A APPENDIX:

1 {

2 "name": "India Palace Karlstad",

3 "short_name": "India Palace",

4 "theme_color": "#ffffff",

5 "background_color": "#ffffff",

6 "display": "standalone",

7 "start_url": ".",

8 "icons": [

9 {

10 "src": "assets/img/icons/icon-72x72.png",

11 "sizes": "72x72",

12 "type": "image/png"

13 },

14 {

16 "sizes": "96x96",

18 },

19 {

21 "sizes": "128x128",

23 },

24 {

"sizes": "144x144",

(50)

33 },

34 {

36 "sizes": "192x192",

38 },

39 {

41 "sizes": "384x384",

43 },

44 {

46 "sizes": "512x512",

48 }

49 ],

50 "splash_pages": null

51 }

52 \caption{Manifest}

(51)

B APPENDIX:

Step: 1 Step: 2

(52)

Step: 3 Step: 4

C APPENDIX:

Listing 1: style.css

1 .add-button {

2 position: absolute;

3 top: 1px;

4 left: 1px;

5 }

(53)

(54)

Listing 2: index.html

1 <button class="add-button">Add to home screen</button>

Listing 3: app.js

1 if(’serviceWorker’ in navigator)

2 {

3 window.addEventListener("load", () =>{

4 navigator.serviceWorker.register("sw.js")

5 .then(console.log)

6 .catch(console.error)});

7 }

8

9 let deferredPrompt;

10 const addBtn = document.querySelector(’.add-button’);

11 addBtn.style.display = ’none’;

12

13 window.addEventListener(’beforeinstallprompt’, (e) => {

14 // Prevent Chrome 67 and earlier from automatically showing the prompt

15 e.preventDefault();

16 // Stash the event so it can be triggered later.

17 deferredPrompt = e;

18 // Update UI to notify the user they can add to home screen

19 addBtn.style.display = ’block’;

20

21 addBtn.addEventListener(’click’, (e) => {

22 / hide our user interface that shows our A2HS button /

23 addBtn.style.display = ’none’;

24 // Show the prompt

25 deferredPrompt.prompt();

26 // Wait for the user to respond to the prompt

27 deferredPrompt.userChoice.then((choiceResult) => {

28 if (choiceResult.outcome === ’accepted’) {

29 console.log(’User accepted the A2HS prompt’);

30 } else {

(55)

31 console.log(’User dismissed the A2HS prompt’);

32 }

33 deferredPrompt = null;

34 });

35 });

36 });

The app is showing add button on chrome browser on desktop. When clicked install promt will pop up. If accepted it will save the app on the desktop.

Figure 13: Add to home button

Figure 14: Install promt

(56)

D APPENDIX:

First Contentful Paint, First View

(57)

First Contentful Paint, Second View

(58)

E APPENDIX:

Speed Index, First View

(59)

Speed Index, Second View

(60)

F APPENDIX:

Time to interactive, First View

(61)

Time to interactive, Second View

(62)

G APPENDIX:

First Meaningful Paint First View

(63)

First Meaningful Paint, Second View

(64)

H APPENDIX:

First CPU Idle, First View

(65)

First CPU Idle, Second View

(66)

I APPENDIX:

Estimated Input Latency, First View

(67)

Estimated Input Latency, Second View

(68)

J APPENDIX:

Performance score, First View

(69)

Performance score, Second View

Comparison between Progressive Web App and Regular Web App