Users perceptions about the usability of a LCDP mobile application

(1)

Bachelor of Science in Software Engineering May 2021

Users perceptions about the usability of

a LCDP mobile application

Morris Andersson

Oscar Lang

(2)

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfilment of the requirements for the degree of Bachelor of Science in Software Engineering. The thesis is equivalent to 20 weeks of full time studies.

The authors declare that they are the sole authors of this thesis and that they have not used any sources other than those listed in the bibliography and identified as references. They further declare that they have not submitted this thesis at any other institution to obtain a degree.

Contact Information: Author(s): Morris Andersson E-mail: moan16@student.bth.se Oscar Lang E-mail: osln17@student.bth.se University advisor: Usman Nasir

Department of Institution of Software Engineering

Faculty of Computing Internet : www.bth.se

Blekinge Institute of Technology Phone : +46 455 38 50 00 SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

(3)

Abstract

In the last two decades, software designed to solve specific tasks for its user have seen a big rise and not least of these are mobile applications. Low-code development platforms has over the last few years been introduced as an alternative which pro-vides lower development times and costs. These low-code development platforms do however trade features and functionality to reach these lower costs. This could aﬀect the usability for the platforms end products’.

The goal of this thesis is to identify usability issues in a mobile application developed in a LCDP, analyse existing usability guidelines for native mobile applications and propose new or adapted usability guidelines for low-coded mobile applications. To validate the gap in low-code development platform usability literature that led us to our initial problem statement we performed a literature review. Making use of individual interviews and focus groups we could produce the empirical data needed to identify potential user experienced usability issues within a low-code developed mobile application.

We reviewed current native usability literature and found that most models, frame-works and usability guidelines are in some way tied to the ISO 9241-11 standard. Individual interviews were held where we found that the overall experienced usability was positive. We then conducted seminars with focus groups which led to a thematic analysis. We summarized the quantitative data from the interviews and the quali-tative data of the thematic analysis on the focus group seminars and then tied it to what we found in the literature review.

Through our research we found multiple areas of improvement in the application but the most discussed of these were that the users expected similar functionality to be presented in a similar fashion. With both the data from the literary analysis and empirical study we propose a new and adapted set of low-code usability guidelines with strong ties to the ISO 9241-11 standard and its core attributes.

Keywords: Usability, Low code development platform, User-oriented.

(4)

Acknowledgments

Morris: Firstly i want to express gratitude to my research partner, Oscar Lang, who through his dedication, humor and disposition have made this into a learning and fun experience. I would also like to express my appreciation to Lena Sellberg who through her advice helped me plan my studies.

Special appreciation goes to my partner, Sofia Starck, whom through her enthusiastic encouragement and assistance enabled me to spend the time needed with this thesis. Finally, I wish to thank my mother for her support and encouragement throughout my study.

Oscar: I would like to direct my gratitude to Morris Andersson for his strong per-formance and for pushing me and himself through the rough times we have faced. I would also like to thank my family, girlfriend, cat, and friends for helping and encouraging me through this thesis.

We would also like to express our deep gratitude to our supervisor Usman Nasir for his willingness to give his time and expertise so generously.

(5)

List of Figures

2.1 Flow chart of our research process . . . 6

3.1 Attributes of usability. Adapted [29]. . . 12

4.1 Example views of the Task Manager mobile application . . . 14

4.2 Participants age and smartphone brand distribution . . . 16

4.3 Interviewees Likert scale answers . . . 17

4.4 Percentage of interviewees that had to redo a step . . . 17

4.5 Thematic analysis process . . . 20

List of Tables

1.1 Usability attributes in ISO 9241-11 . . . 3

4.1 All questions and their subsequent category . . . 16

4.2 Codes and themes derived from the thematic analysis . . . 21

4.3 Themes organized into its corresponding ISO 9241-11 attribute . . . . 22

(8)

Terminology

Drag & Drop – When a user first select a object/component and then drags it to the desired location. The object is "dropped" when the user releases the used input method.

Guideline(s) – Instructions that is intended to be used as advice for specific users when doing specific tasks.

IDE – Integrated development environment, a software that can be used to write code and develop other applications.

LCDP – Is short for Low-Code Development Platform and refers to a develop-ment environdevelop-ment utilized to create software or applications through more graphical means. A LCDP can either produce complete software or applications ready to be deployed or products in need of more fine tuning through traditional coding.

Low-Code– In this context this refers to "low amount of code" and should not be confused with low-level programming languages(E.g. Assembly).

Native – A mobile application developed for a specific target operating system in mind.

Perception – Refers to the identification, interpretation and mental classification of information gathered through a persons different senses to be able to understand the information or environment. A persons ability to perceive information is affected by their expectations of an interaction that is either planned or coming. An object, abstract or not, can be perceived differently by different people.

(9)

Chapter 1 Introduction

The usage of mobile applications has seen an exponential rise in the last ten years, and the number of yearly downloads of mobile applications has reached 218 billion [1]. New mobile applications are oﬀered to customers every day on the smartphones application stores. Besides that companies are seeking development of mobile appli-cations targeted for their workforce and internal usage, creating a need and demand for rapidly developing mobile applications.

In the last few years, the software development industry has seen an increasing num-ber of new mobile application development environments referred to as "Low code development platforms"(LCDPs). These development environments or platforms of-fer a new way of developing applications, mainly by removing the repetitiveness of traditional hand-coded development, decreasing the amount of code required to im-plement key functionality and supplying the developer with a visual development interface. To develop a mobile application in LCDPs, small amount of coding ef-fort is required thus it becomes considerably rapid in comparison to native mobile application development environments.[2]

Mobile applications developed in LCDPs are created by a set of predefined modules or drag & drop feature sets [3]. However, the limitations in customisation restrain developers from implementing their ideas fully, leading to poor user experience and usability issues[4]. This thesis will address these potential problems.

1.1 Background

1.1.1 Native Mobile applications

A mobile application (or App) is a program that runs inside the operating system of a mobile device. Today many diﬀerent mobile operating systems exist however, the most common two are: Google’s Android and Apple Inc’s IOS [5]. The mar-ket for mobile apps is still expanding and the usage today is higher than personal computers[6].

(10)

Chapter 1. Introduction 2 Apps can be developed by using multiple technologies and development platforms. The “traditional” way of developing a native mobile application is with Java or Kotlin for Android, and Swift for IOS. Apps developed this way, with a specific OS as tar-get, are often called “native” in the area of software development [7]. And according to Suyesh [8] native development compared to other methods entails a slower devel-opment with less adaptability to diﬀerent systems.

1.1.2 Low code development platforms

The term "low-code" was first introduced in 2014 by Richardson and Rymer[9] to categorize the rising number of platforms for rapid development and delivery of software applications. The term was subsequently accepted in the area of software development, and platforms such as Microsoft PowerApps, OutSystems, and Mendix started to use the term to categorize their products.

Traditional development platforms usually consists of a graphical IDE where the developer can write code in one ore more specified languages. This IDE usually has a built in compiler, debugger and a code editor where the developer can write their code [10]. In an LCDP, the IDE usually has an automated compiler and debugger, and instead of a code editor there is usually a graphical interface where the developer can insert pre-built components such as text input. Adding logic to the application is usually done with a abstracted graphical, logical language. This commonly consists of a set of logical pre-defined instructions. Some LCDPs also oﬀer the functionality to automate parts outside the front-end of an app. A common example is automatically generated SQL-queries for the back end.[3]

The end product can diﬀer between LCDPs. Some LCDPs generate native mobile application installer/packages that are installed and run directly on a targeted de-vice[11]. Whereas, many LCDPs generate an application that requires a sandbox application provided by the platform for the targeted operating system [12][13]. The second approach eliminates the need for an LCDPs app to be distributed using an app store, but it has the disadvantage of lock-in to the LCDPs ecosystem.

1.1.3 Usability Testing

Usability is one of the fields in software engineering that is the closest to the user and is generally the step that is the most important for the end user. ISO 9241-11[14] defines usability as "the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use". Effectiveness can best be described as to how accurately a specific task can be executed. The term efficiency can best be described as to how much effort and time that need to be used to execute a specific task. The last criterion, satisfaction, is how much the actual response from the system or service differs from the expected outcome. We have summarized each attribute in table 1.1 and it is these definitions that we use for these attributes.

(11)

Chapter 1. Introduction 3 Table 1.1: Usability attributes in ISO 9241-11

Attribute Example

Eﬀectiveness How accurate a user can perfom a task

Eﬃciency How much resources(time, energy) a user spends on a task Satisfaction How the user responds to using the system/application

Nielsen [15] has three more dimensions in his definition of usability, learnability, memorability, and error handling. Learnability, how easy it is for a new user to learn the system or product. Memorability, how easy is it to return to the system after a time away. Error handling, how easy is it for the user to cope with errors within the application and how much these aﬀect the user.

According to Nielsen, no user will want to use a manual for an application or a website [16]. Usability testing is a field within the area of human–computer interaction (HCI) that tries to asses how easy to use a specific system is. The user should be able to learn how to use the website or application fast, else they might not use it for long. The best way of testing the usability of a software, application or system is to use input from either an expert or from a real user [15][16][17].

In the context of mobile applications, which usually have a lot of content on small screens, the testing of usability must be more user-centric from the start. The end user should be included in the usability testing at an early stage so that satisfaction, eﬃciency, and eﬀectiveness can be perfected before major extended functionality is implemented, at which point a new usability evaluation with the end customer should be conducted[18].

1.2 Purpose

Good usability may be one of the key factors between the success and failure of a mobile application[19]. While developing a mobile application focus on the usability becomes extra important due to a mobile phones’ smaller screen that limits the layout choices[20].

Many usability guidelines and tools are available in literature that oﬀers evaluation of native mobile applications however, there is a gap in the literature as no existing native guidelines can be directly adapted for use on usability evaluation of mobile applications developed in LCDPs.

Therefore, we want to evaluate the usability of an mobile application created in an LCDP and combine this evaluation with existing guidelines and models of native mobile applications to create a set of guidelines for future evaluation of low-code developed mobile applications. As an increasing number of companies, small firms, and freelancers start adopting LCDPs, they will need a clear set of guidelines to enhance low-code developed mobile applications’ usability, which in term should create more usable mobile applications.

(12)

Chapter 1. Introduction 4

1.3 Research questions

1. What are the diﬀerent models and frameworks for measuring users’ perception about usability in native mobile applications?

The aim of answering this question is getting a better understanding of how the field of usability is worked with when targeting mobile applications. This will help us get a better understanding of the area and will be the groundwork for creating the guide-lines for working with usability in a low-code developed mobile application.

2. What are the users’ perceptions about the usability of a mobile application developed in a LCDP based on using a low-code developed mobile application? Answering this question with the data that we collected through our empirical study allowed us to discover if our participants experienced any usability issues they were unaccustomed to and to identify if the application had apparent and inherent us-ability flaws. We should also be able to start drafting the aforementioned set of guidelines based on the results from the empirical study.

1.3.1 Scope

The scope of this thesis is limited to the user�s perception of the usability of a mobile application developed in a LCDP. The usability of an application can be tested and measured by many diﬀerent methods but this thesis aims to define it by the perception of the end user. The two biggest operating systems for mobile phones, Android and IOS, have diﬀerent guidelines for design and usability. Including both operating systems is not within the time restraints for this thesis, and we will not include sources that target usability in IOS.

1.4 Thesis Layout

Methodology Here we present our initial planned methods to answer our research questions as well as the actual methods we used.

Literature Review We review the existing literature for measuring and defining the usability of native mobile applications, as well as the documentation of market-leading LCDPs.

ResultsIn this chapter, we present the results gathered from our empirical study. Analysis For our analysis, we discussed the results that were produced for the previous chapters and put them into perspective.

Conclusion Wrapping up our thesis we conclude the thesis as a whole while also providing potential validity threats and possible future work paths.

(13)

Chapter 2 Methodology

As described in the previous chapter our focus lies with the users’ perceived usability of as well as their experienced usability with a low-code developed mobile application. With the existing gap in usability research based around LCDPs we need to start by understanding where usability stands from a technical perspective in current native development and where it stands in the area of low-code development platforms. User testing and input from users is the most common method for measuring both the perceived and experienced usability of an application or software [15] [17]. While heuristic evaluation, for example automated testing through software, of the usability of an application is good for getting an understanding of potential usability problems, user input usually results in more detailed and exploratory problems that can be solved quicker. [17]

Therefore we deem the users’ expectations and experiences of interacting with a mobile application as a good starting point to look for improvements in the perceived usability of said mobile application. Eg. what can be improved to increase the users eﬀectiveness and eﬃciency with the application as well as the users overall satisfaction with using the application.

2.1 Research Process

As seen in figure 2.1, we have used four steps for answering our research questions and for developing our guidelines. First, we identified our problem and research question formulation, both of which are described in detail in chapter 1, section 1.2 & 1.3.

To answer the research questions, we collected data through means of a literature review, interviews and a focus group. We chose to conduct a literature review as we had to get a better understanding on the current state of usability development in native mobile applications and to answer RQ1. Information gathered from the review would later be used for comparison with data gathered through our empirical study.

(14)

Chapter 2. Methodology 6 Problem Identification Research Question Formulation Defining Research Methods Data Collection Interviews Literature

Review Focus Group Guideline

Development

Figure 2.1: Flow chart of our research process

We then decided to run both individual interviews and two focus group seminars as we wanted to extract both quantitative and qualitative data to get a broader data set. We also wanted to use the same participants for both methods as this meant that we could look for changes of opinion between a single participant and the consensual answers of the groups.

The goal for utilising this research process is ultimately to develop guidelines for measuring the usability of a mobile application developed in a LCDP.

2.1.1 Limitations & alternative methods

Inherent limitations of a literature review such as issues with selection of literature and lack of literature are factors we have tried to mitigate through utilizing search terms and only selecting first-hand sources. However, we do realize that our mit-igation tactic could also put us in a precarious position, as a large portion of our selected literature is not based on the exact same area that we have studied.

We first considered using surveys as an alternative empirical collection method of the users‘ expectations and experiences with the usability of a mobile application. This would result in a larger sample pool however, we would still have needed our participants to use an application and to complete specific tasks within it. This would be hard to control in a survey and we see interviews and focus groups as favorable as the amount of resources, time, and participants needed for a survey would be outside the scope of this thesis.

There are however, a few limitations with our selected empirical study methods. Interviews potentially suﬀers from being inconsistent and therefore aﬀecting the data however, we mitigated this through basing the interviews on a manuscript, which we discuss in more detail in section 2.2.2.

(15)

Chapter 2. Methodology 7 Data gathered through a focus group faces the potential limitation of its participants not getting to voice their opinion freely or at all, to prevent this we made sure the active moderator allowed all participants to speak freely and not be suppressed.

2.2 Data Collection

In this section we delve deeper into each of our three selected data collection methods as well as describe in more detail how we conducted each of them. Through doing this we were able to get a good understanding on the current status of native usability research and our participants opinions.

2.2.1 Literature review

We found surprisingly low amount of literature regarding usability for low-code de-veloped mobile applications, we wanted to conduct the literature review on existing native Android usability research, and similar research. We gathered sources from the digital library SUMMON@BTH, IEEE Xplore, and ACM Digital Library. We only used scholarly and peer-reviewed sources from SUMMON@BTH. The following search terms where used in all digital libraries:

• (Usability) AND (Mobile Application)

• (Usability) AND (Model) AND (Mobile Application) • (Challenges) AND (Usability) AND (Mobile Phones)

Lastly, we also went through the development documentation of the market leading [21] [22] LCDPs, OutSystems, Mendix and Microsoft PowerApps. We added this to understand how our research subject of usability is inherently incorporated in market leading low-code development platform systems.

2.2.2 Interview

We decided to use structured interviews, with the interviewees answering the ques-tions in a specific order. We based most quesques-tions on the System Usability Scale [23], as we feel it granted the easiest to measure metrics. However, our data should not be directly compared to the results of other research using the same method. This is partly because we have changed some questions and not included all of the questions provided with the System Usability Scale. Furthermore, our test mobile applica-tion is not based on the same technology as the majority of the mobile applicaapplica-tions previous research has tested.

The participants of the interviews were individuals between the ages of 18 and 30. We used convenient sampling to find our participants, meaning that we asked people that we knew if they would be interested in taking part in our study. This was partly due to the COVID-19 pandemic, which meant that people where generally less willing to participate in interviews. This also meant that all interviews and focus group sessions

(16)

Chapter 2. Methodology 8 took place online on Zoom1_{. Therefore all participants also needed to use their own}

mobile phones for testing the mobile application. Since usability is aﬀected by the environment and the context of use, the usage of the mobile application should have preferably taken place in an controlled environment. But this was not possible at this time. which in turn may have aﬀected our results.

We created an Action Plan (see Appendix A) to keep everything consistent through all interviews. This action-plan consists of an introductions stage for getting the participants up to speed, the steps that the participants should complete and the questions which should be answered. We also prepared a download portal for an easy installation of the application on the participants mobile phones. Since the application is not signed and ready for download from the Google Play Store, we also prepared instructions on how to install unsigned applications on the Android OS.

The actual process of conducting the interviews and the aforementioned questions are discussed further in chapter 4.

2.2.3 Focus group

We created focus groups with them attending seminars guided by us ,the authors, and at the occasion provide the groups with discussion facilitating questions that we could gather qualitative data from. The participants of this method consisted of individuals that had previously participated in the interviews and we split these into two diﬀerent groups.

We, the authors, participated during both sessions. One of us acted as the moderator of the group, and had the task to make sure that the discussion moved on smoothly and on track in the area of discussion. The second author had the task of observing the discussion, and we had an open communication channel where the observer could add follow-up questions that they thought were interesting and wanted the moderator to ask the group. We had a total of eight questions that we wanted each group to discuss. However, we had no specific order for these questions, and we skipped some questions if we felt like we had already got a good understanding of the opinion of the group in that area.

After completing both focus group sessions, we listened to the recordings and created a transcribed summary of what was said by each group. We tried to note what the groups agreed on and where they disagreed. We also took note when a participant had an opinion which diﬀered from the rest of the group. We then extracted the qualitative data of the focus group by performing a thematic analysis. We used a inductive approach, i.e we based our themes on codes we found in the summary. Lastly, we analysed the codes and themes in a semantic approach, meaning that we explicitly tried to analyze what the groups arrived at, and tried to not make assumptions of what was presented.

(17)

Chapter 2. Methodology 9

2.3 Guideline development

One of our goals for conducting this research was to develop basic evaluating guide-lines, similar to those available for native mobile application development, adapted to LCDPs.

By combining the existing frameworks, models and guidelines based on the literature review, and the data extracted from the interviews & focus groups, we could present our own guidelines for approaching future evaluations of the usability of a low-code developed mobile application.

We believe that a set of guidelines are to be regarded as the most optimal solution to our initial problem statement, restrained only by the resources available to us.

2.4 Chapter summary

In section 2.1 we presented the research process we have used for the thesis as a whole. We then continued by further explaining each method of our data collection. The empirical study was conducted from a users’ point of view and with user input in focus, which in turn provided us with dependable usability experience feedback. Lastly, we present the process of our guideline development which came as a result of our selected data collection methods.

(18)

Chapter 3 Literature Review

In this chapter we present our reviewed literature in greater detail, explain some literary choices and bring forth what we have learned from the review.

OutSystems has been named the de-facto market leading LCDP by both Gartner Re-search[22] and Forrester[21]. Even though OutSystems documentation and whitepa-per [24] for their product originally is a misfit source to include in a Literature Review we found that the documents in question revealed a significant absence of usability guidelines and user interaction guidelines. To validate or discard this fact, we dou-ble checked other market leading LCDPs [22] documentation [25] [26] and found the same usability guideline absence, these were brought up in section 2.2.1 and was subsequently further reviewed here.

We therefore want to gather and review pre-existing sources discussing diﬀerent def-initions, attributes, concepts and terminologies that can be used to measure and improve usability in Android mobile applications. This literature review will be used to answer RQ1, we then want to see if these attributes, guidelines, and instrumenta-tion’s can be applied to mobile applications developed in a LCDP.

3.1 The limitations of mobile applications

The field of usability was first introduced into computer science as a way of improv-ing the user experience of computers and applications within computer operatimprov-ing systems. But the usability of a application is highly dependent of the context that it is being used in [14]. A mobile application diﬀers from a desktop application and usually has a smaller screen size to operate within and a diﬀerent input method. Zhang and Adipat [27] introduced six features exclusive to mobile applications:

1. Mobile Context: The usage of mobile phones can be very varied in both location and environment, which can be distracting for the user of the appli-cation.

2. Connectivity: Mobile phones requires a wireless connection to be able to communicate with the internet and other services. This is aﬀected by the location of the user and diﬀerent connection ranges has to be taken into con-sideration when designing the application.

(19)

Chapter 3. Literature Review 11 3. Small screen size: Mobile phones have much less screen area and the design and navigation of mobile applications have to be thoroughly well designed to be usable.

4. Different display resolutions: Different phones have different resolutions so the application should be tested on many different brands and models for the best results.

5. Limited processing capability and power: Mobile phones may suﬀer from a limited performance and all apps can not be run on all models. Some functions may have to be disabled for some users, and this has to be thoroughly explained to the user.

6. Data entry methods: The size of the mobile phone means that the input methods of the application has to be clear and minimize the physical restraints of the user.

Gong and Tarasewich [28] also presents a set of guidelines that are unique in the context of mobile applications and emphasises the importance of the context that the mobile applications are used in. The applications should be designed for diﬀerent levels of attention from the user. Furthermore, the authors highlight that the inter-face should be designed for a single user, in contrast to desktop applications that usually has multiple users, and should therefore be highly customizable to suit the users’ preferences.

3.2 Evaluation of usability attributes, models and

measurement frameworks in mobile applications

Lewis [29] prompts the question "Are the Dimensions of Classical Usability Corre-lated or Independent?". The three parts that make up classical usability that Lewis refers to here were presented in chapter 1, section 1.1.3 and can again be seen below in Figure 3.1. Figure 3.1 also presents a rough answer to his question. Satisfac-tion differs from the other two core parts, effectiveness and efficiency, as it operates in the subjective dimension whereas the other two are objective based parameters. Lewis argues that efficiency and effectiveness are objective parameters measured by task completion rate and times, whereas satisfaction is measured through the users perceived usability.

Apart from the previously discussed attributes effectiveness, efficiency, and satisfac-tion, further research has presented new models and attributes that can be applied to mobile applications. Hussain et al. [30] presented an usability metrics model which used the Goal Metric Question measurement approach to measure the usabil-ity of mobile applications. The authors divided the attributes from ISO 9241-11 into different goals, namely Simplicity, Accuracy, Time Taken, Features, Safety and Attractiveness. The authors then derived questions from these goals and tested them with two different applications. The resulting metrics aided the authors when eval-uating the applications and they also highlighted the benefit of easily being able to add new goals and to adapt the model for other applications.

(20)

Chapter 3. Literature Review 12 Eﬃciency Eﬀectiveness Satisfaction Objective Subjective

Figure 3.1: Attributes of usability. Adapted [29].

Harrison, Flood and Duce et al. [20] introduces the People At the Centre of Mobile Application Development(PACMAD) usability model which uses existing usability attributes such as efficiency, effectiveness and satisfaction but also adds the attributes of learnability, memorability, errors and cognitive load. The authors also categorises three different factors that are important for mobile application usability: User, Task and Context of use. The authors describe the factor of the User as how the application is handled by different end users and the authors highlight that the application must be designed so that user from different backgrounds and experience can use the application within the attributes defined earlier. A task refers to specific task that the user is trying to achieve within the application. An application with multiple features may backfire on the user as a specific task may be blocked by other features and making sure that each feature can be done with ease is key for good usability. The last factor, Context of use, refers to the environment in which the application is used. Similar to how Zhang and Adipat [27] defines Mobile context, the context of use is dependent of different factors such as the user being in a public transport, only being able to use one hand and so on. All of which can be distracting for the user. The authors describe the factor of Context of use and the attribute of cognitive load as the biggest contribution of the PACMAD model.

Most research focuses on on identifying different attributes and factors to measure the usability of mobile applications. Kortum and Sorber [31] presents an alternative way of measuring the usability of mobile applications. Instead of focusing on identifying different attributes and factors, the authors implemented their own version of a System Usability Scale (SUS), which was first introduced by Brooke [23] in 1996. The SUS scale is a set of 10 questions which, after the completion of the survey, results in a number between 0 and 100. This number can then be used to compare the application with other similar applications, where "68" is the average of any given application. A SUS score below 68 means further improvement in the usability of the application is needed. Kortum and Sorber [31] used their SUS scale to evaluate 3,575 different participants, out of which 778 were Android users, of their experience with ten different mobile and tablet applications.

(21)

Chapter 3. Literature Review 13 Similarly, Hoehle, Aljafari and Venkatesh [32] introduced an instrument to measure and score diﬀerent applications based on characteristics derived from the old Mi-crosoft Usability Guidelines 1_{. The authors divided these guidelines into "codes",}

such as font size, and then created a questionnaire based on these codes which they tested with five diﬀerent social media applications with two samples(n=404; n=501) of participants. They found that based on the usability score of their instruments, they could predict the continued intention to use and the brand loyalty of these applications.

Moumane, Idri and Abran [33] evaluated two different mobile applications by their own framework which consists of usability experiments and a questionnaire called QUIS 7.0. They evaluated the effectiveness and efficiency of the applications by measuring the time taken of each task and by using an eye tracker to track how the users used the mobile application. They then measured the level of satisfaction of all users by using the questionnaire.

JongWook, NeungHoe and Hoh Peter [34] presented a method of detecting usability issues by comparing the user behaviour of participants using an application. The proposed algorithm looks for diﬀerences in the behavior of all participants and can therefore easily find which part of an application where multiple users experience usability issues. The authors do however, acknowledge that their algorithm has some problems with finding usability issues which only one or a few participants encounter.

3.3 Chapter Summary

As we have mentioned briefly before this chapter we have now been able to vali-date our impression of the lack of internal usability guidelines for several market leading low-code development platforms. This portrays a great divide in what us-ability literature is available from the start for native versus low-code development environments.

Limitations of mobile applications include smaller screens, diﬀerent data entry meth-ods and diﬀerent contexts of use. One important finding was that designing for dif-ferent use cases is important while working with mobile applications, as these can be used in a greater range of contexts. These limitations still apply while evaluating the usability of a low-code developed mobile application.

We have found a large sample of existing research regarding the evaluation of usability in native mobile applications. The attributes of the ISO 9241-11 were used by almost all existing usability literature. The most common method is user-centric evaluation, similar to how we conduct our research, but the approaches diﬀer between interviews, questionnaires and automatic comparison of user behavior. We believe that some of these models and frameworks can be adapted for measuring usability in mobile applications developed in LCDPs.

(22)

Chapter 4 Results

This section aims to display the results of our chosen empirical methods, as described in chapter 2. We begin with introducing the mobile application which we used during the interviews, and our motivation for choosing that specific application.

We then move on to the results of the interviews and discuss the process of this method and what the results where. Lastly, we present the results of our focus group sessions and subsequent thematic analysis of the data extracted from these.

4.1 The mobile application

Task Manager is a low-code developed mobile application which is available as a sample/demo application in OutSystems. We have chosen this application as it contains most of the diﬀerent actions that a user executes while using a typical smart phone application. Other sample apps that could have been used did either not oﬀer the range of possible interactions found in Task Manager and no other low-code sample app was easier to distribute than Task Manager.

(a) Home view (b) Adding a new task (c) Projects view

Figure 4.1: Example views of the Task Manager mobile application 14

(23)

Chapter 4. Results 15 When we chose our application, we made sure to choose one from a market leading platform as we wanted some an application that would be representative of a good low-coded application. OutSystem, which the application has been developed in, has been designated as a market leader by both Gartner research [22] and Forrester [21].

Figure 4.1 shows some example views of our chosen application, Task Manager. The main purpose of the application is to list upcoming tasks and to-do�s. The user has the ability to organize tasks in diﬀerent projects, and there is also functionality for searching and deleting tasks. The mobile application synchronizes the tasks via cloud-storage but also has local storage functionality, it is also not dependent on any external service or ecosystem and works directly after it has been installed on the participants mobile phone.

4.2 Interviews

As mentioned earlier in chapter 2, section 2.2.2, we created an Action plan (See appendix A). After a pilot test of the steps and questions, defined in the Action Plan, we decided to update and divide the steps to complete into diﬀerent stages. The following stages and steps are required to be completed by all participants: Stage 1

1. Use guest account “Patricia”

2. Survey the application, what can you find and see, use it for a couple minutes Stage 2

3. Create a new task(Task #1)

4. Create a new task and arrange it in a new project (Task #2) Stage 3

5. Mark Task #1 as completed 6. Delete Task #2

We then grouped the questions into each stage, for example, the question Was it easy to create a new task in the application? was asked after stage 2 had been completed. Some questions could be answered by simply stating Yes or No but most questions were answered by responding according to the following likert scale:

• 1 = I strongly disagree • 2 = I disagree

• 3 = Neutral • 4 = I agree • 5 = I fully agree

The questions that the interviewees where asked are grouped into diﬀerent categories, based on which attribute of the ISO 9241-11 [14] it fits the most (See Table 4.1 ).

(24)

Chapter 4. Results 16 Table 4.1: All questions and their subsequent category

Sr. Question Measured_attribute Type

1 Was it easy to create a new task in the application? Effectiveness Likert 2 Do you feel that the amount of time it took you to_{create a task felt reasonable?} Efficiency Likert 3 Did you have to redo any steps in order to successfully_{create a task in the application?} Effectiveness Yes/no 4 Did you feel that the actions you took to create a task_{in the application was natural?} Satisfaction Likert 5 Did you feel that organizing tasks into projects felt_natural? Efficiency Likert 6 Was it easy to find the button for marking a task as_complete? Efficiency Likert 7 Was the feedback from the application good when_{marking a task as complete?} Satisfaction Likert 8 Was it easy to find the button for deleting a task? Efficiency Likert 9 Was the feedback from the application good when_{deleting a task?} Satisfaction Likert 10 Was navigating the application complex? Effectiveness Likert 11 Did you experience the application as easy to learn? Effectiveness Likert

4.2.1 Interview responses

We interviewed a total of 13 participants during the course of one week. All par-ticipants were within our targeted age group of 18-30. Our subjects had diﬀerent backgrounds and the most common type of occupation where "Studying". Other types of employments of our participants included warehouse workers, nurses, and software developers. The most common brands of smartphones amongst the partic-ipants were Samsung(54%) and One Plus(31%).

20yo 15.4% 21yo 7.7% 23yo 15.4% 24yo 30.8% 25yo 23.1% 29yo 7.7% Samsung 54% OnePlus 31% Huawei 15%

(25)

Chapter 4. Results 17 0 % 10 % 20 % 30 % 40 % 50 % 60 % 70 % 80 % 90 % 100 % 1 2 4 5 6 7 8 9 10 11 69.2% 92.3% 61.5% 23.1% 84.6% 46.1% 46.1% 15.4% 69.2% 23.1% 30.8% 61.5% 7.7% 15.4% 15.3% 30.8% 30.8% 7.7% 7.7% 30.8% 23.1% 23.1% 15.4% 7.7% 15.4% 7.7% 7.7% 30.8% 15.4% 30.8% 53.8% Percentage Qu es tion

Strongly disagree Disagree Neutral Agree Strongly agree Figure 4.3: Interviewees Likert scale answers

Figure 4.3 shows the results of the likert scale data that was the result of all questions except for question 3. In general, the Strongly agree is to be considered as positive in the sense that the perceived usability of the participant is positive. Only question 10 diﬀers from this as the question was asked in a reverse order.

In general, most participants individually perceived the usability of the mobile ap-plication as good. We describe each question and the responses in more detail later on in this section as well as categorise them into the 3 core usability attributes as mentioned in table 4.1.

No 69.2%

Yes 30.8%

(26)

Chapter 4. Results 18 Eﬀectiveness

The majority of the interviewees responded that it was easy to create a new task in the application. No subject responded that they disagreed that it was easy to create a task. As seen in Figure 4.4, 30.8% of the interviewees responded that they had to redo a step to successfully create a new task.

The question regarding the navigation of the mobile application is a reversed ques-tion, so strongly disagree in this sense is considered a positive result. A majority of the interviewees responded that they disagreed that the application was hard to navigate. A clear majority also responded that the mobile application was easy to learn.

Eﬃciency

Regarding question 2 (See Table 4.1), a clear majority of the interviewees responded that they thought that the time spent on creating a new task was in line with what they expected, and that they eﬃciently could create a new task.

Only 15.4% of all participants disagreed that organizing the tasks in projects felt natural, which could be because they had a hard time finding the projects and how to add tasks into them.

As for question 6 & 8, regarding finding buttons for marking a task as complete & deleting a task, the results where split. Most responded that they thought it was easy to find the button to mark a task as completed. However, most participants had trouble with finding the button for deleting a task and the majority responded that they did not agree that the delete button was easy to find. The user has to navigate two steps before finding the delete button and we discuss the problems that this implies further in the Analysis chapter.

Satisfaction

Most interviewees responded that they were mostly satisfied with the actions that they completed in the mobile application and how the mobile application provided feedback for the user. The majority responded that they fully agreed with the state-ment that the actions needed for creating a task felt natural. Most also responded that the mobile application provided good feedback while marking a task as complete and after the deletion of a task.

4.3 Focus group

We ran two focus group seminars with six participants in each group. We introduced all participants to the background and reasoning for using focus groups and made sure to get everyone comfortable to the situation. As discussed in chapter 2, section 2.2.3 we decided to record the focus group seminars. We made sure to have permission to both record the sessions and use the discussions in our thesis. We informed them that everything will be analyzed anonymously, and that we are more interested in the opinion of the group instead of individual opinions.

(27)

Chapter 4. Results 19

4.3.1 Focus group discussion summary

These three references will be used frequently in this section: • Group 1: The group of the first seminar

• Group 2: The group of the second seminar

• Mobile application: Task manager (see section 4.1)

Both groups began discussing what is important in terms of usability for a mobile application to be part of their daily use. Both groups arrived at the conclusion that a mobile application should be easy to navigate and that the most important functionality should only be a few clicks away or be shown directly at the home screen. Group 1 also added that there should be a basic need for the mobile application and its functionality. Group 2 expanded on the importance of not experiencing any crashes or major errors while using the mobile application. One participant in Group 2 also added that if the mobile application uses push notifications, these should arrive instantly and with close to 100% concurrency as this can greatly aﬀect the satisfaction of the mobile application.

The groups then discussed if the mobile application that they had tested before, Task Manager, would fit according to the collectively discussed template of a good mobile application. Both groups came to the conclusion that the mobile application was easy to learn and relatively easy to navigate. Group 2 added that it could be part of their daily use but that there probably exist better alternatives. Group 1 also added that the navigation to some of the functionalities were very complex and there were some inconsistencies in how similar functionality worked. For example, marking a task as complete required one step, while deleting a task required two steps. The participants of the group came to the conclusion that a delete button at the home screen would be favourable for increasing the eﬃciency of the mobile application. Group 2 had an similar opinion and believed that the home screen felt useless and that the functionality of that view was limited, even though it took up much space and attention of the user.

Regarding the amount of time required to learn any given mobile application, both groups agreed on that the fundamentals of any application should be very easy and straightforward, never taking more than a couple of minutes to learn. Regarding ad-vanced functionality, the groups were in agreement that more time would be required to fully learn it, upwards of a week were the baseline in both groups.

The groups had mostly similar opinions when asked how usability could be improved within the tested mobile application. Group 1 had the most nuanced standpoint, they specifically addressed the applications information output to the user, stating that color coded messages and more dialogue-like messages from the system would greatly improve the received information. Both groups were in consensus that while creating a sub task the user should be allowed to immediately create a new project and group the aforementioned task into it, instead of having to create them separately and then group them together. Group 1 expanded on this, mentioning that visually enhancing folders/projects with color coding would make navigation easier. Group

(28)

Chapter 4. Results 20 1 provided further feedback that can easily be adopted onto other apps, with that the main functionality of a mobile application should be presented first and always be in focus. They also discussed how only the most important and frequently used secondary features should be accessible with one step.

When prompted with the question "did the test mobile application require more en-ergy (physical enen-ergy / cognitive load) to make use of all its intended functionality?" both groups were quick to answer that it did not.

Both groups agreed that they received good feedback from the mobile application when they had marked a task as completed or deleted a task. Both groups did how-ever acknowledge that they were easy to miss. Group 1 added that they believed that each information bar should appear in the middle of the screen with an confirmation button. One participant also liked the idea of a "undo" button for each action. The participants of Group 1 agreed on that they believe that the date and time should be automatically assigned to the current time and date. Group 2 also believes that the input of time and date could be improved, as it is currently a bit frustrating. Both groups ended at a similar point with the last question tasking them to put a score on the application between 1-10 and with a motivation for the chosen score. For Group 1, the consensus was 5 points and for Group 2 it was 6 points. Both groups felt that better confirmation when executing functionality within the mobile application was necessary. Group 1 heavily implied that customizing the application to each end-user would be crucial to receive a higher score. Group 2 did not provide a solution to receive a higher score, however they did emphasize the importance that the main functionality should always be reachable.

4.3.2 Thematic analysis

We analyzed the seminar sessions using a Thematic Analysis, the process and meth-ods of this are shown in more detail in Figure 4.5. We began with summarizing and extracting the raw data from both seminars, as seen in section 4.3.1.

Data Extraction • Summarising focus group seminars • Extraction of key feedback points

Data Codification

• Identification and codification using line by line read-through

• Comparison of key points for labeling consistency

Translation into Themes

• Translated codes into themes • Ensured themes are consistent and

distinctive

Attribute assignemnt

• Compared codes and themes with existing research

• Assigned themes to ISO-9241-11 attributes

(29)

Chapter 4. Results 21 We continued with extracting important phrases and answers from the sessions and their subsequent transcribed summaries. These phrases and answers were then grouped into various codes. To determine if a phrase or answer was important we filtered them by occurrence and importance (based on the participants experiences). As seen in table 4.2, we then assigned these codes to various themes. To wrap it up, we categorized each theme into its corresponding ISO 9241-11[14] attribute, see table 4.3.

Table 4.2: Codes and themes derived from the thematic analysis

Code Theme

action confirmation Accuracy

revert action element / button

preferred mobile application Attractiveness

better alternatives

trust in the system Concurrency

dissimilarity between functions straightforward use cases

low cognitive load Energy consumption

crash prevention Error handling

wasted space / focus Facilitation

functionality grouping end user personalizing

easy to navigate Findability

show the most important feature color coded navigation

information bar placement

easy to learn Learnability

more time to learn advanced functionality

color coded messages Received information

dialogue like messages received good feedback system messages easy to miss

template information in forms Time spent

(30)

Table 4.3: Themes organized into its corresponding ISO 9241-11 attribute

Attribute Theme Summary

Eﬃciency Time Spent Refers to the amount of time a user needs to spend to complete a given task. It can also refer to what can be changed to lessen this time.

Facilitation Refers to how much help the sys-tem provides the user actively or pro-actively in their quest to complete a task.

Energy Consumption Refers to the amount of energy a user needs to expend to reach a goal / com-plete a task.

Eﬀectiveness Findability Refers to how easy a functionality is to find or how hard it is to find and how said functionality is presented to the user.

Received Information Refers to how the system presents in-formation to the user.

Accuracy Refers to how accurate a user can per-form a task, but can also mean how of-ten the user inputs wrong information.

Learnability Refers to how easy the specified func-tionality or the application as a whole is to learn.

Satisfaction Error handling Refers to how the system presents and handles potential errors.

Concurrency Refers to the systems dependability when it comes to specific features Attractiveness Refers to the systems capability to

draw and convert users, primarily ei-ther through visuals or functionality

(31)

4.4 Chapter Summary

We first introduced the chosen low-code developed mobile application that all partici-pants used during the interviews. We then discussed how we performed the interviews as well as the data & results of this method. We then grouped each answer with its corresponding ISO 9241-11 standard[14] attributes.

As for the focus group, we summarized the discussion of both seminar groups and on their consensual discussion answers we performed a thematic analysis. The contents of the tables seen in this chapter are not ordered in any way other than alphabetical and no distinction between the groups was taken into account when we created the tables. The thematic analysis was constructed with the use of extensive codes and subsequent cohesive themes derived from the seminar discussions. Some of these can be connected to the ISO standard 9241-11 [14] which in turn provides good backing for our found themes.

(32)

Chapter 5 Analysis

In this chapter, we analyse the results our research process and methods, described in chapters 2, 3 & 4, have produced in order to begin answering our research questions. We summarize the chapter and analysis with a proposed solution for our initial problem statement through the development of usability measurement guidelines to be used for low-code developed mobile applications.

5.1 Usability evaluation of native mobile apps

Based on the results from the literature study conducted in chapter 3, we could clearly see that most usability models are based on the ISO 9241-11 standard. Some studies and models add more attributes such as learnability, memorability, error handling and cognitive load, but it can be argued that these can be fitted into the attributes of ISO 9241-11. For example, learnability is strongly correlated to Effectiveness, and cognitive load is correlated to Efficiency. Still, most models and frameworks seem to benefit greatly by dividing the ISO 9241-11 attributes into more characteristically attributes, or at least by creating different goals or codes for each attribute.

While mobile applications may be limited by physical phone features such as small screen size, limited processing capacity and a constant need for a wireless connection, they offer a context of use that is unavailable for desktop users. One recurring pattern that we found during our literature study was the importance of designing for different contexts of use. A mobile phone is designed for being used in different contexts, such as while walking. The mobile application has to be designed for these different contexts and specific use cases such. This in turn demands greater attention towards usability while developing the mobile application.

The most common method for measuring usability we found was using questionnaires and interviews with direct input from users. Most questions origin was derived from attributes of the ISO 9241-11 standard or based on characteristics found in related guidelines. Some research have introduced methods in which they can compare the usability of one mobile application with other mobile applications. For example, the System Usability Scale results in a metric between 0 and 100 and based on this metric the usability of one mobile application in comparison to others can be measured.

(33)

Chapter 5. Analysis 25 As seen in the second last paragraph in section 3.2, some research also used eye-tracking software to evaluate the eﬀectiveness of a mobile application. Eye-eye-tracking allows the researchers to more clearly see where the test people focus their eyes while using mobile test applications, this in turn can increase the usability of the application by revealing where the users spend more than the speculated allotted time to complete tasks. There also exist methods for detecting usability issues by using algorithms and looking for diﬀerences in the behaviour of users who experienced issues. It works best for a large sample group, where multiple users experience the same issues. Both of these methods include extensive time for setting up the experiments and then analyzing the data. The data can be more precise about what issues the users experience, but one would have to factor in the time needed and then look at what is more important for their research.

5.2 Usability evaluation of low-code developed

mobile application

Both the interviews and the focus group resulted in similar results in terms of the users’ perception of the usability of the mobile application developed by OutSystems, which should help us answer RQ2. The interviews revealed similar answers and opinions to some questions and radically diﬀerent to others. The focus group seminars provided cohesive answers from which we extracted codes and themes using semantic thematic analysis. We then connected these themes to the core attributes of the ISO 9241-11 standard.

From data gathered during our empirical study related to the participants usability perception of the applications eﬀectiveness we can clearly see that the majority ex-perienced the application as positive. For reference see figure 4.3 and section 4.3.2. Users responded that it was easy to create new tasks, and a majority could do so without having to redo any steps. An improvement here would be that the user should be presented with a confirmation button before performing an action such as marking a task as complete.

All of the interview participants agreed or fully agreed (see fig 4.3) that the applica-tion was easy to learn, and that it was in line with their expectaapplica-tions of never taking more than a couple of minutes to learn the basic functionality. The users expressed that the use case of this application was straight forward enough to not confuse a regular user, and we feel that there is no need for improving the learnability of the mobile application at its current stage.

Most participants agreed that the mobile application was easy to navigate, but that it required some improvements. Based on input from the users, such improvements could include color coding headers and navigation buttons for more precise actions. Some users also reported some inconsistencies in how the animations were displayed while navigating, and while this is not strictly tied to usability, we have found that it does aﬀect a users perception on the mobile applications’ usability.

(34)

Chapter 5. Analysis 26 Removing tasks proved itself a more difficult task than most others we created for the participants. This was discussed during the focus group seminars (see section 4.3.1) and both groups agreed that the functionality to remove tasks was not presented in a similar fashion as similar functionality, which by itself creates usability issues as users’ expectations is important to factor in when developing mobile applications. The groups would have liked to see this functionality available to reach from the base view of the application, similar to how the functionality for completing tasks is presented, which a majority of users liked. The functionality of removing tasks is, in this case, important as it bridges the gap between effectiveness and efficiency in regards to usability for our selected application.

The participants made it clear that both the amount of time and the energy con-sumption, as in cognitive load, needed was low while creating a new task. The participants expressed that it was straight forward, although they did bring up that the date and time fields where not filled in at the start of creating a new task. For faster turn-around time of creating a task, and for improving the overall eﬃciency, the date and time should be prefilled to the current time and date.

During the interviews, most participants agreed that organizing tasks into projects felt natural, and that they enjoyed the functionality. However both groups of the focus group seminars brought forward that it was implemented poorly by having the project displayed in a separate view. For improving the eﬃciency of creating a new task in projects, we feel that there should be an option to create a new project from the same view where you create new tasks.

We also see the potential for improving the eﬃciency of the application by creating the functionality of customizing the home screen to fit a specific user�s need, as users often find the fastest way of doing tasks by themselves. This can also be validated through literature as seen in chapter 3, section 3.1

Regarding the feedback that the mobile application displays for the user, after mark-ing a task as complete or removmark-ing a task, the participants agreed that this feedback was acceptable. They did, however express that the information bar was easy to miss and that it would have been better if it was displayed in the middle of the screen instead. We also believe that the satisfaction and the eﬀectiveness of the user could be improved if there was a "undo" button for each feedback system message, as one might mark a task as complete by accident.

The users did not experience any major crashes during their testing of the mobile application, but both focus groups made it clear that they expect a mobile appli-cation to handle errors without making it unnecessarily painful for the user. They also expect the mobile application to be consistent in displaying Android system notifications as the unexpected absence of a notification can inflict distrust in the mobile application, which in turn aﬀects the satisfaction of the user.

We mentioned earlier that similar functionality should be presented in a similar fashion, both focus groups also landed on this conclusion and saw it as a problem needed fixing. The eﬀect such a problem has on satisfaction is important to take note of as two similar or identical functions that are not presented in the same way could sway the user into feeling confused what the functionality actually provides.

(35)

Chapter 5. Analysis 27 Straightforward and easy to understand use cases also has an impact on the users satisfaction when using an application since unnecessarily convolution of use cases will only confuse users and may prohibit them from fully utilising the functionality provided by an application. The focus groups both agreed on this, they also came to the conclusion that this is closely related to if an application would be selected by them as their preferred mobile application for solving a problem they have. If the potential preferred mobile application have hard to understand use cases it wont be recognised as a candidate and will only be discarded. This is further expanded upon from both groups when they both mention "better alternatives", selecting an app to solve a specific problem is hard since there is much competition and much of it boils down to how clear an application’s use case is.

5.3 Guidelines: Usability Measurement of a mobile

application developed in a LCDP

After reading our chosen low-code development platform OutSystems documenta-tion and the other market leading low-code development platforms documentadocumenta-tion we have found a common denominator among them all which we believe pose a fun-damental flaw in terms of how low-code development platforms handle usability. In very few parts of the OutSystems documentation, as well as the other market leading platforms documentation, is the user either in focus or not mentioned at all. This in turn limits the base capability for developers using the LCDPs to work towards usability since no guidelines is provided. The lack of an internal source for how to handle usability and user interaction could be interpreted as a sign that usability for LCDPs is an under explored area. The areas that are instead intently focused on are scalability and stability which might help with enabling many potential end users however, these will not help with the mobile applications work towards improved usability as much as an internal set of usability guidelines.

Based on the literature on the usability of mobile applications, as seen in section 5.1, and what we have gathered on users� perception of usability in the mobile application created in a LCDP, we propose a new set of low-code usability guidelines adapted from Hussain et al. [30] Goal Metric Questions (as seen in section 3.2). The thematic analysis we performed on our empirical study�s data presents specifically three themes as extra important for low-code development:

• Facilitation • Accuracy • Attractiveness

As presented in table 4.3 these three themes can be explained as: Facilitation, how much help the system provides the user to complete a task. Accuracy, how accurate a user can perform a task and can be measured by number of mistakes. Attractiveness, the applications� capability to draw and convert users, primarily either through visuals or functionality.

(36)

Chapter 5. Analysis 28 Selecting these three themes to turn into goals was done through surveying the empirical and literary gathered data. The literary data revealed that commonly found native usability issues were related to these three and our empirical data validated that the same was experienced by our participants.

Hussain et al.’s already presents two of these, Accuracy and Attractiveness, as goals in their usability framework, which in turn provides us with enough groundwork to create the third goal of Facilitation tied to the attribute of Eﬃciency.

Choosing to turn facilitation into it’s own goal for our guidelines mainly came from our empirically gathered data within which many participants expressed their discon-tent with the mobile applications current facilitated usability features. As a theme in our analysis, facilitation presented itself as how much the system helps the users eﬃciently use the systems intended functionality and how much the system allows to user to customize the system for their own needs.

The goal of facilitation is best described by breaking it down into parts. Since a mobile application has to conform to the dimensions of a smartphone important functionality should be presented and focused in a way the enables the user to utilize this unhindered without needlessly using screen space. To increase the users eﬃciency with a systems intended use, functionality that likens each other is to be grouped and presented together. All in a manner that diﬀerentiates grouped functionality from individual. The user of the application should also be able to customize their application to their need, as users usually create workflows that fit them and their needs themselves.

Therefore we propose a goal-oriented guidelines consisting of the ten selected themes presented in our thematic analysis(see table 4.3) whereas three have been focused on and turned into standalone goals. These guidelines will pose as a good usability resource for low-code developers which they can use in order to produce more usable mobile applications.

These guidelines will be a contribution to the low-code developer community as a whole since usability exists in every interactive system and object. The end users should also be positively aﬀected by our proposed guidelines as future mobile appli-cations might incorporate them, thus leading to a better understanding of usability amongst the developers.

Users perceptions about the usability of a LCDP mobile application