Understanding Jamstack and its Perception in Web Development

(1)

Västerås, Sweden

Thesis for the Degree of Master of Science in Computer Science with

Specialization in Software Engineering, 15.0 credits

UNDERSTANDING JAMSTACK AND

ITS PERCEPTION IN WEB

DEVELOPMENT

Dragana Markovic

dmc20004@student.mdh.se

Milic Scekic

msc20005@student.mdh.se

Examiner: Antonio Cicchetti

Mälardalen University, Västerås, Sweden

Supervisor: Alessio Bucaioni

Mälardalen University, Västerås, Sweden

(2)

Acknowledgements

First and foremost, we would like to express deep and sincere gratitude to our supervisor Alessio Bucaioni for his patience, guidelines, and encouragement which helped us tremendously during this thesis.

Additionally, we wish to show our appreciation to all respondents that took the time to be part of our survey which enabled this research to be possible.

Finally, we must express gratitude to our families and friends for providing us with support and continuous encouragement throughout our years of study.

(3)

Abstract

In recent years, web development and technologies became pivotal for our lives and ubiquitous across business sectors. Web development is an inherently complex process, which involves different stakeholders, tasks, and technologies. Through the years, researchers and practitioners in web development created so-called web stacks as a way to alleviate the complexity associated to web development. Web stacks can be defined as sets of technologies coupled together for the purpose of designing and developing all the web infrastructure. As technologies and tools evolve, new stacks are designed and realized.

In this thesis, we investigate one such new stack: Jamstack. We investigate how Jamstack is perceived by practitioners in the web development domain focusing on its perceived benefits and limitations, its main application domains, and the supporting tools. We achieve this using an online survey involving 31 respondents. We analyze the collected data using both vertical and orthogonal analysis. The collected data indicate that Jamstack is perceived as a promising stack whose benefits overcome possible limitations. The data suggest that Jamstack will certainly play a role in modern and future web development although there is no strong evidence that it will take over and replace traditional stacks.

(4)

1. Introduction

With the widespread of fast and ultra fast connections and connecting devices, web development and technologies have become pivotal in almost all sectors of industry. Nowadays websites are expected to be secure, easily accessible, user-friendly and performative. Research by Google found that 53% of users would leave a web page if that does not load within 3 seconds [1]. The same research found that the average loading time for web pages is 22 seconds. Another research from Imperva confirmed the Google findings. The research from Imperva investigated how loading time can affect a website engagement, in the e-commerce domain. Using a survey on 4500 customers, Imperva found that 7% of those customers expected the website to load immediately, while 62% of the customers said they would wait up to 5 seconds before leaving the page [2]. Therefore, in order to follow the standards, one of the most important steps in development is the precise selection of technologies that would be combined to meet customer requirements. This set of combined technologies more known as web stacks are well-established and reliable technologies that can deliver a good product. One of the first designed web stacks was LAMP (Linux, Apache, MySQL, PHP) [3] followed by others like MEAN (MongoDB, Express, Angular, Node.js) [4] and MERN (MongoDB, Express, Angular, Node.js) [5]. Nowadays, these stacks are well-established and used by developers for creating high standard websites.

In 2015, Matt Biilmann introduced Jamstack (JavaScript, Application Programming Interface (API) and Markup), a new approach of building fast and scalable websites that do not require a traditional infrastructure [6] [7]. The main idea is that static websites do not need to be static any-more: with the growth of new tools and services they can have dynamic capabilities [8]. Jamstack, like every other stack, is composed of different technologies that work together to provide a good development workflow and solutions for making modern web applications. Jamstack front-end is pre-built into static pages and served directly from the Content Delivery Network (CDN) [6]. For the back-end, instead of a database and server-side programming, Jamstack uses JavaScript to communicate with the APIs provided from third-party components known as Headless Content Management Systems (CMSs). Since its introduction, Jamstack is still in its embryonic phase and the related research it is somehow scarce. However, in the last two years, Jamstack had signi-ficant growth in popularity among developers. With many new tools that support Jamstack, its limitations are now less marked while its benefits are becoming game-changing.

1.1. Thesis contribution

In this thesis, we investigate how Jamstack is perceived by practitioners from the web development domain. In particular, we investigate which are the perceived benefits and limitations of Jamstack, its main application domains, and the supporting tools. We achieve this using an online survey involving 31 respondents from the web development domain and consisting of 15 questions. We analyze the collected data using both vertical and orthogonal analysis. The collected data indicate that Jamstack is perceived as a promising stack whose benefits overcome possible limitations. Jamstack is currently used for the development of less complex and dynamic websites mainly. The data suggest that Jamstack will certainly play a role in modern and future web development although there are no strong evidence that it will take over and replace traditional stacks.

1.2. Thesis outline

The remainder of this thesis is as follows. Section 2. provides an overview of Jamstack and traditional stacks. Section 3. presents the research method we have adopted for this thesis, together with all its composing steps. Section 4. describes the results of our thesis. Section 5. discusses the findings of this research. Section6. presents some related works documented in the literature. Eventually, Section7. conclude the thesis with final remarks and future works.

(6)

2. Background

The steady introduction of the new IT technologies gives engineers the possibility of shaping the best working environments for the product they are working on. In the web domain, these environments are known as web stacks [9]. A web stack can be described as a set of technology coupled together for the same purpose [10]. Some of the most popular web stacks are MERN [5] (MongoDB, Express, React, Node.js), and LAMP [3] (Linux, Apache, MySql, PHP/Perl). Besides, there are new web stacks, such as Jamstack, that have been released since few years, only. In the remainder of this section, we describe some of these web stacks, which are related to this thesis.

2.1. Jamstack

The term Jamstack was created in 2016 by a co-founder and CEO of Netlify: Matt Biilmann [8]. Jamstack has been introduced as a way for coping with the tedious and insecure traditional de-velopment of dynamic websites [8]. There are other factors that have aided the rise of Jamstack among web developers including, the increasing importance of aspects such as performance and security, and the lack of bounds on the programming languages to be used. The infographic image presented in the article [11] shows the richness of Jamstack’s ecosystem. There are a number of different tools on offer, from headless CMS, utility APIs, functions, to static site generators. Com-bining these tools at the discretion of the developer makes their development experience satisfying, more flexible, and unique. Unlike any other traditional stack, the architecture of Jamstack is much simpler and comes down to just three components: JavaScript, API and pre-rendered Markup on the CDN. JavaScript is used for two things: making interactive web pages and communicating with the back-end through API. API is used in place of traditional databases, and it enables applications to connect with third-party services [12]. Markup is static pages delivered with CDN and served as HTML files using Static Site Generators (SSG), and not dynamic from the server [13] [14]. CDN provides fast transfer of loading assets for pages like HTML, JavaScript, images, etc [15].

2.1.1. JavaScript

JavaScript is a web browser language that has been introduced in 1995 by Brendan Eich [16] [17]. Today, it is mostly used for creating interactive web pages and it is considered among the most powerful and popular web browser languages. It can independently provide developers with everything they need for creating any type of web application. JavaScript is currently part of many popular front-end and back-end framework such as React [18], Angular [19], Vue.js [20], Node.js [21], and Express [22]. As a response to the increased popularity of SSGs, JavaScript has released its SSGs. Examples of these are: GatsbyJS, which was created in 2015 and is used for creating performative websites [23], and Next.js that supports server-side rendering [24]. The interested reader can find more details on SSGs in Section2.1.3. On the top of the JavaScript is the build API which is the second element of the Jamstack.

2.1.2. Application Programming Interface

The API is used for interaction between front-end and back-end and easy delivery of the content to any device. API is the main reason why static websites can also be dynamic and have functionality. For adding and modifying content Jamstack uses headless CMS which is a back-end only content management system. Unlike traditional CMS for example WordPress, with headless CMS the back-end and front-back-end are separated. [25]. Developers have a variety of options choosing headless CMS for their website. There are two types of headless CMS: API-driven and Git-based. API-driven CMS stores the content in the database while Git-based stores the content in files on Git [26]. Some of the most popular open source API-driven CMS are Strapi, Craft CMS, Directus, etc. There are also API-driven CMS that are only used as a cloud service meaning that all content data will be stored on their servers. The most popular kind of CMS are Contentful, DatoCMS, etc. There are also headless CMS like Sanity and Ghost that supports both open source and cloud service. For Git-based CMS there are a lot fewer options. Some of the great Git-based CMS are Netlify CMS, Forestry, Siteleaf. A full list of all headless CMS can be seen in [27], while comparison list

(7)

that a lot of above mentioned headless CMS use is GraphQL [29]. The main benefit of GraphQL is the way in which developers can gather data. In GraphQL there is no need for accessing multiple endpoints to get data, a developer can send just a single query and the server will respond with a JSON (JavaScript Object Notation) object with all data requested [30].

2.1.3. Markup

Jamstack website pages are generated using SSG which is served directly to the browser from the CDN [6]. SSGs like GatsbyJS combine templates and content into website pages [7]. Decoupling SSG and headless CMS allows developers to make functional dynamic websites that serve as static pages. This kind of websites will be faster and more secure than websites built using traditional technologies. Currently, there are several SSGs [31], where each of them has different benefits. For example, if a developer needs to create a simple static website he/she would use Hugo SSG [32] because of its speed. Hugo uses Go [33] programming language and, in contrast to other static site generators, it is compiled rather than interpreted. If a developer needs to make a more complex website then he/she would use an SSG with more features like Gatsby or Next.js React. They are both based on React, a JavaScript library that is used for creating user interfaces with reusable components that make the development of applications faster and easier.

2.2. Traditional stacks

Traditional stacks were first introduced in the late 1990s. LAMP was one of the first stack that was built from several free, open-source technologies [3]. One of the biggest reasons behind its success was the reliability of the composing technologies. For instance, Apache web server (used in LAMP) was released in 1995 [34] and to date is still the most used web internet server. Usually, Apache is used together with Linux. Starting from the mid-2000 and thanks to the rise of PHP programming language and MySQL, LAMP has been used for developing several popular applications [35]. One of these is the WordPress content management system, which is currently used by 33.5% of all websites on the internet [36].

MEAN and MERN are other examples of newer stacks. MEAN stack is composed of four elements: MongoDB [37], Express, Angular, and Node.js. Since all the four elements support the JavaScript language, applications in MEAN can be written in only one language for both front-end and back-end [4]. This is a great advantage in terms of developer experience. MEAN stack is used for building flexible and scalable dynamic web applications on cloud hosting [38]. This stack has its own web server with Node.js, which makes deployment much easier and faster. MEAN stores data using the MongoDB database. MongoDB is a No-SQL document database that stores the data in JSON format and is easily integrated into the Node.js environment. In the back-end, all server-side logic is handled by Express, which is a Node.js framework. On the client-side, the Angular JavaScript framework is responsible for the representation of the application and all the dynamic features.

MERN stack follows a well-established pattern called 3-tier architecture [39], which is separated into the following tiers: presentation tier (Angular), application tier (Express and Node.js), and data tier (MongoDB). As stated in [39] 3-tier architecture has benefits that include faster development of the application, improved scalability, and reliability which is tied to better security. MERN is similar to MEAN except that it uses the React library instead of Angular. MERN also follows the pattern of 3-tier architecture which can be seen in Figure1.

The main differences between web stacks are in the technologies they use, their architectures, and where their adoption. The LAMP stack uses Apache as a web server while the MEAN stack uses Node.js. Another difference is the database. LAMP uses Structured Query Language (SQL) database which is used for structured data that have a relation between each other while MEAN uses not only SQL (NoSQL) database which is a better option for storing non-relation data [40]. In terms of adoption, LAMP is mostly used for creating websites with CMS like WordPress, and for applications dealing with complex queries. While MEAN stack is a better option for creating dynamic and progressive web applications [41]. Jamstack is a “serverless" stack, which means that the back-end is decoupled from the front-end [42]. The main benefit is that developers have to focus only on the front-end side while the back-end one uses additional services from different

(8)

Figure 1: MERN stack architecture, Source: MongoDB [5]

vendors. In this way, the developer does not deal with web servers and databases. Regarding the architecture, MEAN and LAMP stacks have a traditional architecture that is composed of client, web server, and application server. When the client access the page, the request needs to go through a web server on the internet. Then, the application server sends a request for the data requested by the client [43]. Jamstack has a different approach. The main focus of Jamstack is to reduce the server-side needs by using third-party services that can provide functionality for the website [44]. In Jamstack the HTML is cached at a CDN so there is no need for every new request to the database [43]. Figure2 shows the differences between traditional stacks and the Jamstack.

(9)

3. Research method

We have designed and carried out this thesis following the guidelines defined by Petersen et al. [46]. Figure3presents the overall process, which composes of three main phases: planning, conducting and documenting.

Figure 3: Overview of the research methodology process

In the planning phase, we have clarified the motivations for conducting this research, specified the research questions, and defined the research protocol to be followed. The output of the planning phase is the research protocol. In the conducting phase, we have followed and performed the steps defined in the review protocol. Search and selection of relevant studies is done using one comprehensive scientific database. For designing and developing the survey we have used the set of identified studies. We have defined the questions composing the survey as evaluation type hence as close-ended questions to which respondents could agree or disagree. By doing so, we have ensured

(10)

to have relevant answers. After the definition, we have carried out an online survey using the Google Forms service [47]. In the documenting phase, we have analyzed the synthesized data to provide answers to the research questions. Both a quantitative and qualitative data analysis have been performed using vertical and orthogonal analyses, respectively. Eventually, we have analyzed possible threats to validity and written the following report.

A replication package containing the answers to the survey can be found at the followinglink. This research has been conducted according to the established ethical guidelines. All personal information about respondents has been excluded from the final report. The respondents were informed about the aim of the survey and how their data will be used. The research was not funded by any third party company or institution and respondents were not pressured to join the survey.

3.1. Research goal and questions

For this thesis, we have defined the following research goal: “investigate the maturity of Jamstack and compare it against traditional web stacks for understanding its actual benefits and limitations” We have split the above research goal into the following four RQs:

1. What are the main benefits of Jamstack? 2. What are the main limitations of Jamstack?

3. For which kind of web applications Jamstack is used? 4. What are the most popular tools that support Jamstack?

By answering RQ1 and RQ2 we provide a characterization of Jamstack in terms of its main benefits and drawbacks. By answering RQ3, we investigate the current adoption of Jamstack, while by answering RQ 4 we provide a catalogue of tools supporting Jamstack.

3.2. Search and selection

In the search and selection phase, we have selected the set of relevant papers that we have used for defining the survey. We have searched one scientific database being Google Scholar [48]. There are several reasons why we have chose Google Scholar including its reputation and its extensive index, which includes most peer-reviewed online academic books, journals, conference papers, theses and dissertations, pre-prints, abstracts, technical reports, court opinions, patents and other scholarly literature. We have searched the database using the following string: “Jamstack”. The search string was sought on title, abstract, and keywords of papers. The automatic search gave us 99 primary studies. From this initial set of papers, we have removed those not being research articles and merged duplicates. On this new set, we have applied the following exclusion criteria.

1. Studies not written in English. 2. Studies not available as full-text.

3. Studies whose main focus was not Jamstack.

The application of the selection criteria has led to a set of 5 studies. We have complemented the automatic search with a grey literature review [49], which has resulted in the selection of 4 additional studies of which 1 was not available as full-text. The final set of primary studies includes the 8 papers listed in Table1.

3.3. Survey definition and execution

In this phase, we have built and executed the survey following the guidelines by Kasunic et al. [50]. First, we have defined the target group for the survey, which is composed of web developers hav-ing experience in both Jamstack and at least one other traditional web stack. We have searched for

(11)

Title Authors Year Modern web development on the Jamstack: modern

techniques for ultra fast sites and web applications Mathias Biilmann, PhilHawksworth 2019 JAMStack Continuous Integration and Continuous

De-ployment with CircleCI and Netlify Tri Hoang 2020

Building Serverless Website on GitHub Pages Prayudi Utomo, Falahah 2020

Modern Web Development with Jamstack Edit Orosz 2020

Practical Jamstack Frank Zammetti 2020

The Jamstack Book Raymond Camden, Brian_Rinaldi 2021

Jamstack Handbook: Building Fast, dynamic apps with

Javascript and the static web Colby Fayock 2020

Hugo in Action: Static sites and dynamic Jamstack apps Atishay Jain 2021 Table 1: Selected papers for Literature Review

.

respondents on LinkedIn by manually picking respondents based on their experience and browsing through relevant groups1 2 3

Based on the findings from the literature review, we have created the survey using Google Forms. The survey composes of 15 questions among close- and open-ended questions grouped in four sections being: general information, Jamstack benefits, Jamstack limitations and a section with open-ended questions. In the general information section, we have put questions for gathering basic information about respondents such as their knowledge of and experience with web technologies, their role in web development, etc. In the Jamstack benefits and Jamstack limitations sections, there are close ended questions, which aim at assessing the opinion of respondents on specific claims related to Jamstack benefits and limitations that we have found in the literature. For each of these claims, we have provided examples for supporting the respondent’s understanding. Table 2 and Table3provide the lists of the claims for the Jamstack benefits and limitations, respectively.

The respondents could either agree or disagree with each statement. We have used the last section of the survey for gathering the opinions of the respondents on the future adoption of Jamstack, possible advantages of traditional stacks over Jamstack, and feedback on the survey. This section has used open-ended questions, only. After designing the first draft of the survey, we have performed a pilot survey to check its quality and collect feedback from the respondents. The pilot survey was sent to a target group of six people, chosen by years of experience in relevant technologies. We have used the feedback from the pilot survey for refining and building the final version of the survey. For instance, we have added one question aiming at identifying the tools supporting Jamstack, and have removed one question on a Jamstack limitation that was recently overcome. We have distributed the final version of the survey to the 31 respondents composing the target group. For this, we have used different copies of the survey so to minimize possible threats to validity (we could not assess the respondent’s prerequisites in these groups).

3.4. Data extraction

We have performed data extraction for close-ended questions using the Google Form services. Withing Google Form, the survey results are graphically represented using histograms, horizontal bars and pie charts. Besides, it allows automatic export of the results into a spreadsheet. For open-ended questions, we have used the affinity diagram technique suggested by Kasunic [50]. The affinity diagram technique is a tool that gathers text and organizes it into groups based on relationships between items [50]. During this process, we have interpreted answers to each open-ended question, categorized them into groups, and established relationships between them. Later,

1_{JAMstack group:} _{https://www.linkedin.com/groups/8722827/}_{on LinkedIn} 2_{Reactiflux group:} _{https://discord.com/invite/reactiflux}_{on Discord} 3_{MERN stack group:} _{https://www.reddit.com/r/MERN_Stack/}_{on Reddit.}

(12)

Name Claim

Performance

With static pages delivered through CDN the loading speed of the Jamstack website usually achieves the highest possible score on speed tests [7]. There are three aspects that are usually used as metrics of website speed and they are: LCP (largest contentful paint), FID (first input delay), CLS (cumulative layout shift).

Security

Thanks to the static storage or CDN that serves your application, APIs that are read-only and accessible exclusively for building environment, attacks can be reduced to minimum if not even removed completely. Since the layer has no database, sites are not vulnerable to server-side include injections. Besides, the use of external platforms such as Netlify can provide protection against DDOS and DNS attacks. [7]

Flexibility

Jamstack uses third-party services more known as headless CMS for managing content on the website. Since headless CMS are separated from the front-end of the website it means that the API that is created with headless CMS can be used and easily displayed across multiple platforms.

Pricing The whole Jamstack website requires a small amount of resources and can behosted on CDN using some free static hosting like Netlify or GitHub Pages which will reduce the cost.

SEO

Jamstack page content is static which means that the speed of the website will be fast and indexing the pages of the website will be a lot easier [51] in comparison with other web stacks which are the two most important factors for a good ranking on the internet.

Developer experience

Jamstack developers have more freedom in choosing the technologies they will use in accordance with their knowledge, wishes and needs unlike other web stacks. The fact that front-end is separated from the backend makes developer experience far more enjoyable and easier because they can focus on one thing. Since Jamstack is static, all files can be stored on GitHub, Netlify or Vercel which makes the deployment easy and what matters to many, all for free.

Table 2: Jamstack Benefits

Name Claim

Dynamic functionalit-ies

While Jamstack can be dynamic it is usually harder to implement some dy-namic functionality and it mostly requires relying on third-party services for adding more complex features (For example if you want to have search on your website you can’t do it on your own. You will need to use Algolia or other third-party service).

Reliance on third-party systems

In case the system goes down, your website will feel the consequences because there is no communication with the API. Whether a solution will be reached and if the system will function again, is out of your hands and depends entirely on the provider (Availability). Entrusting all content to a third-party is the risk that confidential content could be disclosed ( Confidentiality).

Delay in live preview

Before seeing the actual changes on the website, everything needs to be pushed on Github. This can cause frequent delays especially when it comes to very dynamic sites that need constant changing of the content. [52]

(13)

we have assigned a term to each group and re-analyzed the answers based on this categorization. The affinity diagram can be seen in Figure15.

3.5. Data analysis and synthesis

We have used the guidelines by Kasunic et al. for analyzing the synthesizing data to answer our RQs [50]. We have used both vertical and orthogonal analysis. We have used vertical analysis to present statistical details for each question. Besides, we have analyzed the final set of studies as a whole to find potential patterns. We have used orthogonal analysis for identifying relationships between different groups of data. We have represented the results of such analysis using cross-tabulations (banners) [53].

3.6. Threats to validity

We have conducted this research using a protocol, which we have built on well-established guidelines for empirical studies [46]. In this section, we discuss potential threats to validity and the measures we have taken to minimize them.

Threats to conclusion validity focus on the relation between the treatment and the outcomes [54]. In survey-based research, one of the main threats to conclusion validity is the quality of the survey, which can negatively affect its results. To minimize such a threat, we have performed a literature review and built the survey on the data extracted from the selected studies. In this way, we have minimized the authors’ biases and avoided the creation of a survey solely based on our experience. We have built the survey in a two-step process, which has involved a control group of respondents. This process has helped us in improving the quality of the survey and identifying potential validity threats. In order to reduce the risk of misinterpretation of the questions, we have supported each question with examples. Another potential threat to conclusion validity is that some results may have hidden meaning that the authors may have missed them out due to their lack of expertise. To migrate this threat and ensure accuracy, we have had one researcher extracting the data and the second one independently checking the extraction.

Threats to internal validity refer to a degree of confidentiality in a causal relationship between treatment and outcomes [54]. One of the most important threats to internal validity for survey-based research is subject selection. To be sure to have respondents with proven knowledge of Jamstack and traditional web technologies, we have manually selected some respondents after having checked their competencies through their LinkedIn profiles. In addition, we have shared the survey through relevant LinkedIn, Reddit, and Discord groups which has also served as an assurance to reach the desired number of respondents. To avoid respondents becoming demotivated, bored, or unfocused we have kept the survey to a reasonable length. Another threat to internal validity is finding respondents with the right experience. To minimize this threat, we have shared the survey through well-established LinkedIn groups related to both Jamstack and other web technologies.

Threats to construct validity focuses on the relation between the theory and the observa-tions [54]. One such threat is that respondents might only have limited experience with the technologies under investigation. To minimize this threat, we have excluded from the respondent group all developers with less than six months of experience in such technologies. Another threat that may occur is so-called hypothesis guessing, which happens when respondents try to figure out the main goal of the study and adjust their answers accordingly. We have made an effort to pose the questions objectively and referring to relevant sources.

Threats to external validity refer to the ability to generalize the study results [55]. One such threat might be that the studies we have used for building the survey are not representative of the subject. We have mitigated this threat by having a very inclusive search and selection strategy, which has excluded only studies not being in English or nor available as full-text.

(14)

4. Results

In this section, we present the results from our vertical analysis. The results are presented per research question and visualized using bar charts. The results from questions not related to any research question are discussed in Section5.

4.1. Vertical analysis

For closed-ended questions, we used extracted data done by Google Form services. While for the open-ended questions we categorized answers using the affinity diagram technique. To present results visually we have used Excel tool for creating bar charts. In the end of every RQ there is highlight part which gives a general overview.

4.1.1. Jamstack benefits (RQ1)

During the literature review, we have found six main benefits. For each of these benefits, we have created one close-ended question for which respondents could choose a score from 1 (“Strongly Disagree”) to 5 (“Strongly Agree”). Figure4 shows the results for the six benefits.

Figure 4: Jamstack benefits

All the benefits found in the literature were acknowledged by the respondents. The most acknowledged benefits are flexibility and SEO with 93% of answers being strongly agree or agree. This confirms that it is important to the respondents that the pages are static, that the content is reusable and it can be displayed on all possible devices using the API. Developer experience and performance are the third most acknowledged benefits with 87%. The respondents recognize crucial possibilities offered by Jamstack such as fast static pages, choosing technologies, decoupling front-end and back-front-end, and easy deployment. The fourth most acknowledged benefit is pricing with 84% of respondents strongly agreeing or agreeing with this benefit. This is linked to Jamstack requiring few resources and being deployable on free hosting platforms. Security is the last acknowledged benefit with 83% of positive responses.

The collected data confirm that all the Jamstack benefits reported in the literature are largely experienced in industrial practices, too. It is interesting to note that no respondents have reported additional benefits in the open-ended questions (besides those we have identified). While pricing and developer experience are among the most acknowledged benefits, they are also the benefits that collected more negative answers (together with performance). However, this can be explained by the use of multiple third-party services. In fact, the use of multiple (and less-suited) plugins can largely influence aspects such as developer experience, pricing, and performance.

Highlights – Jamstack benefits (RQ1)

(15)

• Flexibility is the most cited benefit from the respondents.

• The use of the plugins can slow down the performance of the websites, and add up to the cost, which can make the developer experience less satisfied.

4.1.2. Jamstack limitations (RQ2)

During the literature review, we have found three main limitations. Similar to the benefits, for each of the elicited limitations, we have created one close-ended question for which respondents could choose a score from 1 (“Strongly Disagree”) to 5 (“Strongly Agree”). In this case, the collected data depicts a more fragmented opinion of respondents, Figure5.

Figure 5: Results of Jamstack limitations

As stated above, we believe that this is mainly related to the plugins that have been used by a given respondent. The most mentioned limitation is confidentiality with 58% of votes confirming that the risk of disclosing confidential content is relevant. This finding also seems to be in line with security being among the least perceived benefit. Other highly mentioned limitations are availability and delay in live preview with 48% of votes. This confirms that a failure of third-party plugins may affect the availability of the website. Also, we came to the conclusion that a delay in the live view often occurs when updating the content. The limitation that was the least acknowledged by respondents is dynamic functionalities with 42% of votes. According to our results, all the limitations mentioned in the literature were confirmed. However, we have come to the conclusion that some limitations were not perceived as so important. For example, numerous respondents (35%) disagreed with the availability being one of the limitations. A possible reason is that APIs do not necessarily need to be hosted by a third-party provider as they can be built in-house and hosted on-premises. Similar to availability, respondents voted that other limitations such as delay in live preview (32%) and dynamic functionalities (29%) are not that frequent. It is almost certain that the reason for this is related to the release of a new version of Gatsby [56], the leading front-end React framework for Jamstack. The new version came out in March 2021 with a significantly improved build time. As for the dynamic functionalities, respondents explained that this limitation mostly depends on the experience of developers and that it can be resolved with the usage of the right tools.

Highlights – Jamstack limitations (RQ2)

(16)

• Delay in the live preview can be solved with the use of a new version of GatsbyJS [57] (front-end React framework).

• Dynamic functionalities is the least acknowledge limitation by the respondents.

4.1.3. Kind of web applications built using Jamstack (RQ3)

For this RQ we have asked respondents to choose which stack (between Jamstack and traditional web stacks) may be most suitable for building a particular type of website. The types of websites that we have chosen for this evaluation are: blogs and portfolios, news and magazine, e-commerce, social media and media streaming websites. These types of websites were selected because of their difference in complexity and requirements. Also, they are the most common types in web development. Figure6 shows the results of 5 most common types of websites.

Figure 6: Comparison between Jamstack and traditional stacks application type adoption According to the results, the type of website for which Jamstack is the most suitable option is blog and portfolio websites with 84% of votes. The reason is that this type of websites is usually static and does not require complicated queries. For the same reasons, news and magazine websites are the second most suitable option with 64% of the preferences, followed by e-commerce with 55% of the preferences. These results are in line with the elicited Jamstack benefits. Static pages that have good performance, pricing, SEO, and availability are great examples of which kind of websites Jamstack is more suited. For e-commerce websites, users do not want to wait for slow page load, which is the reason the performance benefit of Jamstack is such an important aspect for this type of website. Jamstack can use tools like Stripe [58] and Snipcart [59] to turn websites into a fully functional e-commerce platform. SportShoes did a study about the impact of speed on commerce [60]. They found that visits that were faster than average are 34% more likely to convert. On the other hand, websites that are much more dynamic and therefore more complex, have received more votes for traditional stacks than for Jamstack. Social media websites received the most votes (55%) for the traditional stack, followed by media streaming with 45%. This is related to the nature of the websites and their dynamism for which the traditional stacks may

(17)

Highlights – Kind of web applications built using Jamstack (RQ3)

• According to the respondents, websites that are static and do not require complex fea-tures are more suited to be realised using Jamstack.

• Respondents choose traditional stack for developing dynamic websites.

• For e-commerce websites, performance and security are the two most important factors, which is a reason why Jamstack is an good option.

4.1.4. Tools supporting Jamstack (RQ4)

In the RQ4 respondents were asked to indicate tools that they use together with Jamstack. Fur-thermore, respondents were able to add other tools not given as an option separately from the one listed in the survey. Figure7 shows the results about tools supporting Jamstack.

Figure 7: Results for tools

The most mentioned tool was GraphQL with 87% votes. The interested reader can find more information about GraphQL in Section2.1.2.. The second most mentioned tool that respondents find useful is Stripe, an online payment platform. The reason for Stripe getting many votes can be related to the e-commerce websites where Jamstack is a more opted solution. The third most mentioned tool is Auth0 [61], which provides authentication and authorization to the websites. The next tool that respondents find useful is headless CMS Strapi with 61% votes. Over half of respondents (58%) are familiar with the Algolia [62] tool that provides search functionality for websites. Figure8presents the results fro the tools suggested from the respondents and not listed in a survey.

Most mentioned tools were platforms that provide content management (Contentful) or hosting (Vercel, Netlify) services. Contentful [63] is an headless CMS like Strapi, which provides back-end functionalities. Vercel [64] is a cloud platform where developers can deploy the front-end of their websites. Similar to Vercel, Netlify [65] is a platform where developers can deploy and host their websites. Both options have a free tier, simple deployment, and a free Secure Sockets Layer (SSL).

(18)

Figure 8: Other tools

Highlights – Tools supporting Jamstack (RQ4)

• GraphQL is the most useful tool according to respondents with 87% of votes.

• Other suggested useful tools by respondents are content management and hosting plat-forms.

4.2. Orthogonal analysis

To uncover possible relations and patterns between different categories, we have cross-tabulated pairs of categories and visualized the results using the heat map data visualization technique within the RStudio [66].

4.2.1. Jamstack limitations vs. types of websites

When it comes to limitations and certain types of websites, we have noticed that opinions are quite divided. To investigate whether there is any relation between these two categories, we have compared all the limitations with different types of websites. Figure 9 shows the frequency of the relations between limitations and type of websites. For the limitations, we have considered only the answers that strongly agreed with the limitation, while for the types of the website only those answers that did not choose Jamstack. The heat map Figure9 clearly shows that most of the limitations are more frequent for more dynamic websites such as social media. In particular, confidentiality is the most voted limitation for the social media websites followed by availability and dynamic functionalities. These are also frequently encountered for e-commerce websites, although with a smaller frequency compared to social media. Oddly, the only frequent limitation for media streaming website is confidentiality despite most of the respondents thought that traditional stacks would be a better option for the development of this kind of websites.

We broke down the relations between the social media websites and the dynamic functionalities limitation, Figure10. The data confirmed our intuition. Hence, the respondents who agreed with the dynamic functionalities limitation chose traditional stacks as better option for developing social media websites. Respondents that did not agree with this limitation have fragmented opinions, but they still lean towards traditional stacks. Surprisingly, from nine respondents who disagreed with this limitation, none of them voted for Jamstack as a better option for the development of social media websites.

(19)

Figure 9: Jamstack limitations and type of website

(20)

4.2.2. Jamstack benefits vs. types of websites

When analyzing benefits and different types of websites, we have considered only those responses that strongly agreed with a particular benefit, and choose Jamstack as the most suitable option. The results, as shown in Figure 11, emphasizes the correlation between Jamstack benefits and less dynamic types of websites. Blog and portfolio websites stand out with a significant positive

Figure 11: Jamstack benefits and types of websites

correlation with almost all benefits. We can also see a relation between flexibility and SEO benefits with e-commerce and news and magazine websites. Pricing is one of the most pronounced benefits for static websites. No significant correlation is present between security and any type of website, in fact, security is the least perceived benefit for all types of websites. To go more deeply into the correlation between benefits and type of websites, we have analyzed the performance benefit separately with one static and one dynamic type of website. Figures12and13show the diversity of the results and emphasize the importance of the website type and the benefits associated with them. In the case of static websites (news and magazine), we have a gradual change of frequency from Jamstack to traditional stack, which confirms that a relation exists. For a more dynamic type of website (social media), there are no differences in ratios as respondents strongly choose traditional stack as a better option. The most surprising finding is that the data show no relation with respondents that slightly lean towards the traditional stacks (value 4 in the figure). This finding suggests that there is no strong correlation between performance and dynamic types of websites.

4.2.3. Respondents experience vs types of websites

Another interesting relation we want to check is whether respondent’s experience affects the choice of a stack with respect to different types of websites. The analyzed data suggest that all respondents are more likely to choose Jamstack for developing a more static type of websites like blog, portfolio, news, and magazine websites, regardless of their years of experience. For the more dynamic types

(21)

Figure 12: Performance and news and magazine Figure 13: Performance and social media relations. Respondent’s choice of a stack for any type of website cannot be traced to their experience as far as data of the survey shows.

(22)

5. Discussion

The main objective of this thesis is to identify Jamstack main benefits and limitations, especially when compared to other, more established web technologies. We have used a systematic process for eliciting and answering the thesis research questions. During the initial literature review, we have found 6 main benefits and 4 limitations. To confirm these findings, we have used an online survey where 31 respondents assessed whether the elicited benefits and limitations are perceived in practical applications, too. Respondents have confirmed all the six benefits with very similar judgments. Despite most of the limitations were confirmed, the opinions of the respondents were more divided. A possible explanation may be that Jamstack is constantly growing with new releases mitigating well-documented limitations.

The analyzed data remarked that Jamstack seems to be ideal for developing less dynamic sites. For this type of website, there is a strong possibility that Jamstack will become the reference stack. To gather the expert’s opinion on this matter, we created an open-ended question where we asked whether or not they thought Jamstack could become the reference stack. Figure14shows that 17 out of 27 respondents think that Jamstack can become the de facto stack. Data show that

Figure 14: Future of jamstack

Jamstack is already gaining a lot of ground on the market thanks to the all mentioned benefits, such as performance, security, cost, flexibility, and the launch of new versions of Gatsby and NextJS [67]. Only 5 respondents were not completely sure that Jamstack could replace traditional stacks. One respondent explained “Not in SSR (Server Side Rendering) heavy needed applications but yes in client-side, SPA’s, CMS’s, and with the trend of decoupling and containerization progression, in combination tooling such as microservices, Progressive Web Apps, Decentralized Apps (D’Apps), Serverless functions, etc.”. This means that there is a big possibility to overcome other stacks when it comes to websites like e-commerce, blog, portfolio, and news or magazine but not for other complex websites. Some respondents have not given judgment (7%), explaining that technologies are evolving at a rapid pace and that there is also a possibility that a newer stack will overrun Jamstack. The remaining 5 respondents thought that Jamstack is not the best option for every type of website mostly because, when it comes to server-side rendering applications, Jamstack can not achieve its goal without the help of other technologies.

Using another open-ended question, we have collected the opinions of respondents with re-spect to the advantages of traditional stacks over Jamstack. The answers of the respondents were clustered into five categories as can be seen in Figure15. We provided each category with a

(23)

suit-Figure 15: Adventages of traditional stacks over Jamstack

capabilities, no reliance on third-party services, more established, and more control. As Figure15 reports, the biggest advantage of traditional stacks over Jamstack is an higher number of capab-ilities. In contrast to Jamstack, developers do not need to rely on third-party services to make dynamic websites. This also ensures developers better control over the websites under development. The respondents also reported that traditional stacks are easier to learn and master as compared to a rather steep learning curve of Jamstack. In general, respondents seem to agree that there is no universal or preferred stack and stacks should be chosen based on needs and available expertise. The catalogue of tools supporting Jamstack may positively affect the developer’s experience by mitigating or removing some of the identified Jamstack limitations.

(24)

6. Related Work

Jamstack is a rather new technology and its related research is still in its embryonic stage. In 2020, Netlify [65] has conducted a survey among more than 3000 software development professionals and it has found that “44% of developers have been using Jamstack for a year, with 37% using it for 1-2 years and 8% for 4-5 years” [68]. Scrivito CMS in its 2019 white paper, has credited the Jamstack popularity to its ability in reducing the complexity, cost and vulnerability related to traditional server-based applications [69].

There are several works, which have compared Jamstack to traditional stacks and identified its main advantages. The thesis by Hoang [70] has provided a brief description of the main development differences between Jamstack and traditional stacks. The thesis has used a case study mimicking a real-life scenario for understanding these differences. The results have shown that the case study made using Jamstack is much faster, safer and scalable. Besides, Hoang found that the development process with Jamstack is much easier when supporting platforms, like Netlify, are used. Compared to the work by Hoang, in our thesis, we have used the survey as the main research method and have focused on several aspects of Jamstack (benefits, limitations, tools, etc.) rather than a subset of this only (deployment, hosting and performance). Similar to Hoang, we have come to the same conclusion that is, applications developed using Jamstack have better performance than they are developed using traditional stacks. However, we have collected slightly different results on its future adoption. While Hoang has claimed that “web development will shift from a monolithic architecture to JAMStack, which will vigorously continue to develop” [70], our data suggest that such a shift is not so straightforward due to the current limitation of Jamstack with respect to dynamic websites. The thesis [52] by Orosz has used a survey to validate four benefits found in a literature review. Similar to her work, we have also reviewed the literature to find the benefits and limitations of Jamstack. The difference between this thesis and the one by Orosz is that we have reviewed the literature in a more systematic way. Table2and Table3is extracted data from the literature review that was used to make specific and clear questions about the benefits and limitations of the Jamstack that respondents can agree or disagree with.

In his book [51], Frank Zammetti has described the advantages of the Jamstack and how they outweigh the disadvantages. He has listed some Jamstack properties that were also included in our survey as benefits (performance, SEO, security and pricing) or limitations (need for a developer). Numerous studies have been conducted to understand how much the success of Jamstack is related to the performance of the created websites [7].

The work by Karanjit has evaluated and compared the performances of MEAN and LAMP stacks by developing two sample applications, namely Chat System and Address Book System [71]. The author has used Apache Bench and Siege to examine the performance, data transmission, and database flexibility of both applications. Although these properties have been used for comparison of LAMP versus MEAN they can also be applied to Jamstack. In our thesis, we have investigated these properties through a survey in order to find out to what extent they are significant and whether Jamstack actually provides their improvement. We have done this research in hope that we will update, supplement and in some way improve and confirm all previous researches conducted on Jamstack.

(25)

7. Conclusion and Future Work

To create high-quality websites, practitioners in web development created web stacks, a set of technologies that are able to create optimal solutions for any kind of web application. This thesis investigates one such stack: Jamstack. The goal of this thesis was to assess for which kind of web applications Jamstack can be adopted and for which reasons. In addition, we investigated the limitations, which are preventing it to become one of the leading stacks. We have used an online survey that was build on the findings of a literature review. The survey was distributed to 31 practitioners in web development. The results indicate that all benefits and limitations of Jamstack found in the literature are experienced by practitioners, too. However, we discovered that some of the limitations are less pronounced today and that with the right usage of supporting tools they can be mitigated. The data showed that the most acknowledged benefits were flexibility and SEO, while the limitations were confidentiality and delay in live preview. Eventually, data showed that Jamstack is mostly adopted, for static websites such as blogs, portfolios, news and magazine. Although the current study was based on a small sample of respondents, the results suggest that Jamstack will certainly have a place in future web development even though there was no evidence that it will become the de facto standard. For more dynamic websites such as social media, media streaming, and e-commerce, the limitations that have prevented the use of Jamstack are confidentiality, availability and dynamic functionalities.

One line of future work may encompass developing dynamic websites using Jamstack so to uncover the specific issues related to those limitations together with possible ways for overcoming them. Another possible line of future direction would be to repeat this study with a larger number of respondents. Besides having a more significant statistical value, a larger study could help to uncover more interesting correlations such as those between benefits, limitations, and types of websites that could be further explored.

(26)

References

[1] D. An, ‘Find out how you stack up to new industry benchmarks for mobile page speed,’ "Think with Google-Mobile, Data & Measurement", 2018.

[2] M. Tim and B. Nicole, ‘Load time and mobile compatibility top online shopper demands,’ 2016.

[3] Wikipedia, Lamp, 2021. [Online]. Available: https : / / en . wikipedia . org / wiki / LAMP _ (software_bundle).

[4] MongoDB, What is the mean stack? 2021. [Online]. Available:https://www.mongodb.com/ mean-stack.

[5] Mongo, Mern stack, 2021. [Online]. Available:https://www.mongodb.com/mern-stack.

[6] Netlify, ‘Why jamstack,’ 2021.

[7] M. Biilmann and P. Hawksworth, "Modern Web Development on the JAMstack". O’Reilly Media, Inc., 2019.

[8] P. B. R. Raymond Camden, "The Jamstack Book". Manning Publications, 2021.

[9] I. Wigmore, What is Web stack? [Online]. Available: https://whatis.techtarget.com/ definition/Web-stack.

[10] Techopedia, Web stack, 2021. [Online]. Available:https://www.techopedia.com/definition/ 3309/web-stack.

[11] N. Radakovic, ‘Jamstack ecosystem,’ Retrieved from Bejamas: https://bejamas.io/blog/jamstack-ecosystem/, 2020.

[12] K. Meagher, ‘Jamstack explained in 5 minutes,’ 2021. [13] Divya, ‘What does the m in jamstack actually mean?,’ 2020.

[14] J. WTF, Wtf is jamstack, 2021. [Online]. Available:https://jamstack.wtf/.

[15] CloudfLare, ‘What is a cdn?’ Retrieved from Cloudflare: https://www.cloudflare.com/en-gb/learning/cdn/what-is-a-cdn/, 2021.

[16] M. Haverbeke, "Eloquent JavaScript: a modern introduction to programming". No Starch Press, 2018.

[17] C. Douglas, "JavaScript: The Good Parts". O’Reilly Media, Inc., 2008. [18] React, React, 2021. [Online]. Available:https://reactjs.org/.

[19] AngularJS, Angularjs, 2021. [Online]. Available:https://angularjs.org/.

[20] Vue.js, Vue.js, 2021. [Online]. Available:https://vuejs.org/.

[21] Node.js, Node.js, 2021. [Online]. Available:https://nodejs.org/.

[22] Express, Express, 2021. [Online]. Available:https://expressjs.com/.

[23] Gatsby, Our story, 2021. [Online]. Available:https://www.gatsbyjs.com/about/.

[24] J. Bedford, ‘Gatsby vs next.js - what, why and when?,’ 2019.

[25] WordPress, Wordpress, 2021. [Online]. Available:https://wordpress.com/.

[26] S. C. Davis, ‘Api-driven or git-based? figuring out the right cms for you.,’ 2020.

[27] Jamstack, Headless cms, 2021. [Online]. Available:https://jamstack.org/headless-cms/.

[28] C. Comparison, Headless cms comparison, 2021. [Online]. Available:https://cms-comparison. io.

[29] GraphQL, Graphql, 2021. [Online]. Available:https://graphql.org/.

[30] H. to GraphQL, ‘Graphql vs rest - a comparison,’ 2021.

[31] Jamstack, Static site generators, 2021. [Online]. Available:https://jamstack.org/generators/.

(27)

[34] Wikipedia, Apache http server, 2021. [Online]. Available: https : / / en . wikipedia . org / wiki/Apache_HTTP_Server.

[35] E. Smith, ‘Lamp stack history,’ 2019.

[36] Isitwp, ‘Top 10 popular cms by market share,’ 2020.

[37] MongoDB, Mongodb, 2021. [Online]. Available:https://www.mongodb.com/.

[38] I. C. Education, ‘Mean stack,’ 2019.

[39] I. Education, ‘Three-tier architecture,’ 2020.

[40] M. Smallcombe, ‘Sql vs nosql: 5 critical differences,’ 2020. [41] B. Marijan, ‘Mean vs. lamp: Which is better?,’ 2021.

[42] R. MacManus, ‘Up the stack: A year-end review of serverless, jamstack and javascript,’ Re-trieved from TheNewStack: https://thenewstack.io/up-the-stack-a-year-end-review-of-serverless-jamstack-and-javascript/, 2020.

[43] A. Jain, "Hugo in Action: Static sites and dynamic Jamstack apps". Manning Publications, 2021.

[44] S. Jallapuram, ‘What is jam stack,’ 2019.

[45] Jamstack, Jamstack, 2021. [Online]. Available:https://jamstack.org/.

[46] J. S. Molléri, K. Petersen and E. Mendes, ‘Survey guidelines in software engineering: An an-notated review,’ in Proceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement, 2016, pp. 1–6.

[47] Google Forms: Free Online surveys for Personal Use. [Online]. Available: https : / / www . google.com/forms/about/.

[48] G. Scholar, Google Scholar. [Online]. Available:https://scholar.google.com/.

[49] Wikipedia, ‘Grey literature,’ 2021.

[50] M. Kasunic, ‘Designing an effective survey,’ Carnegie-Mellon Univ Pittsburgh PA Software Engineering Inst, Tech. Rep., 2005.

[51] Z. Frank, "Practical JAMstack". Springer, 2020.

[52] E. Orosz, ‘Modern web development with jamstack,’ 2020.

[53] L. Bourque and E. P. Fielder, "How to conduct self-administered and mail surveys". Sage, 2003, vol. 3.

[54] P. W. M. Trochim, ‘Research methods knowledge base,’ 2020.

[55] D. S. Cruzes and L. ben Othmane, ‘Threats to validity in empirical software security research,’ Empirical Research for Software Security: Foundations and Experience, 2017.

[56] H. Warren, Gatsby v3 Incremental Builds in OSS, new Gatsby Image, and more, Mar. 2021. [Online]. Available:https://www.gatsbyjs.com/blog/gatsby-v3/.

[57] MediaCurrent, ‘What is gatsbyjs?,’ 2018.

[58] Stripe, Stripe, 2021. [Online]. Available:https://stripe.com/en-gb-se.

[59] Snipcart, Snipcart, 2021. [Online]. Available:https://snipcart.com/.

[60] D. Peiris, ‘The impact of speed on ecommerce,’ 2020.

[61] Auth0, Auth0, 2021. [Online]. Available:https://auth0.com/.

[62] Algolia, Algolia, 2021. [Online]. Available:https://www.algolia.com/.

[63] Contentful, Contentful, 2021. [Online]. Available:https://www.contentful.com/.

[64] Vercel, Vercel, 2021. [Online]. Available:https://vercel.com/.

[65] Netlify, Netlify. [Online]. Available:https://www.netlify.com/.

[66] RStudio, Rstudio, 2021. [Online]. Available:https://www.rstudio.com/.

(28)

[68] C. Christina, ‘The state of the jamstack in 2020,’ 2020.

[69] Scrivito, ‘Jamstack for web developers,’ Faster, more secure and easier to administer, 2020. [70] T. Hoang, ‘Jamstack continuous integration and continuous deployment with circleci and

netlify,’ 2020.