A Comparative Evaluation of Usability for the iPhone and iPad

(1)

Master Thesis Computer Science

Thesis no: MCS:2011:21 Sept. 2011

A Comparative Evaluation of Usability

for the iPhone and iPad

Muhammad Azam

Luqman Ahmad

School of Computing

Blekinge Institute of Technology SE-371 79 Karlskrona

(2)

This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulllment of the requirements for the degree of Master of Science in Computer Science. The thesis is equivalent to 20 weeks of full time studies.

Contact Information: Author(s):

Muhammad Azam

Address: Gyllenstjärnas Väg 18, 20 Post code 37140 Karlskrona,

Sweden

Email: azambth72@gmail.com

Luqman Ahmad

Address: Gyllenstjärnas Väg 18, 20 Post code 37140 Karlskrona,

Sweden E-mail: luqman.khan83@gmail.com University advisor: Dr. Veronica Sundstedt School of Computing School of Computing

Blekinge Institute of Technology Internet : www.bth.se/com

SE-371 79 Karlskrona Phone : +46 455 38 50 00

(3)

ACKNOWLEDGMENTS

In the name of Allah, the Most Benecent, the Most Merciful.

First and foremost we would like to thank Allah the Almighty, Who blessed us courage and devotion for this study.

We heartily thank our supervisor Dr. Veronica Sundstedt, who encouraged us and gave guidance throughout this research work. Her constructive suggestions, directions and invaluable advice made us capable to complete this thesis. We would also like to express our appreciation to those who participated and dedicated their valuable time for these studies, without their participation these studies would not have been feasible.

Last but not least, we express our gratitude to our parents and family members, for providing us untiring support and prayers during this research work.

(4)

Abstract

Many everyday systems and products seem to be designed with little regard to usability. This leads to the frustration, wasted time and errors. So the usability of the product is important for its survival in the market.

In many previous studies the usability evaluation of the iPhone and iPad carried out individually and very little work has been done on the comparative usability evaluation. However, there was not any study conducted on the comparative us-ability evaluation and measuring the performance of the iPhone versus iPad in a controlled environment.

In this research work, the authors performed the comparative usability evaluation and measured the performances of the iPhone and iPad on the selected applications by considering the young users as well as the elderly users. Another objective of this study is to identify the usability issues in performances of the iPhone and iPad.

A survey and experiment techniques were used to achieve the dened objectives. The survey questionnaire consisted of 42 statements that presented the dierent usability aspects. The objectives of the survey study were to validate the identied issues from the literature study, identify new issues and measure the signicant dierence in user opinions for the iPhone and iPad. However, the experiment studies helped to measure the performance signicances between the devices against the three user groups (novice user, experienced user, elderly user) and among the groups over the devices. Further, objective was to measure the satisfaction level of the participated users against the iPhone and iPad.

The experiment was performed in a controlled environment. Total six tasks (two tasks per application) were dened and each participant performed the same task on both devices. Generally the authors found that the participants performed better on the iPad with lower error rates as compare to the iPhone.

Keywords: Usability, Usability Evaluation, Touch Screen, Smart-phone, iPhone, iPad, performance.

(5)

3 RESEARCH METHODOLOGY 23 3.1 Literature Review . . . 24 3.2 Survey Design . . . 24 3.3 Experimental Design . . . 27 3.3.1 Participant Selection . . . 27 3.3.2 Variables Selection . . . 28 3.3.3 Hypotheses . . . 28 3.3.4 Task Denition . . . 28 3.3.5 Procedure . . . 32 3.3.6 Experimental Material . . . 32

3.3.7 Post Test Questionnaire . . . 33

3.3.8 Pilot Studies . . . 33

3.4 Evaluation Methods . . . 33

(6)

3.4.2 Normality Test . . . 34

3.4.3 Analysis of Variances (ANOVA) . . . 34

3.4.4 Unpaired T-test . . . 34

3.4.5 Paired T-test . . . 34

3.4.6 Wilcoxon Matched Paired Signed Ranks Test . . . 35

3.4.7 Error Bars . . . 35

3.5 Validity Threats . . . 35

3.5.1 Internal Validity Threats . . . 36

3.5.2 External Validity Threats . . . 36

3.5.3 Constructive Validity Threats . . . 36

3.5.4 Statistical Conclusion Validity Threats . . . 36

4

STATISTICAL ANALYSIS, RESULTS AND DISCUSSIONS

37 4.1 Survey . . . 37

4.2 Demographics Results . . . 37

4.3 Usability and Preference Testing . . . 37

4.3.1 System Results . . . 37

4.3.2 Touch Screen Gestures Results . . . 41

4.3.3 Keypad Results . . . 43

4.3.4 Applications Results . . . 46

4.3.5 Participant Comments . . . 50

4.4 Measuring Usability Attributes . . . 51

4.4.1 Task Time . . . 51

4.4.2 Accuracy Measurement . . . 61

4.4.3 Satisfaction Results . . . 64

4.4.4 Participant Comments . . . 68

5 CONCLUSIONS AND FUTURE WORK 70 5.1 Conclusions . . . 70

5.2 Future Work . . . 71

6 APPENDICES 73 6.1 Appendix A Survey Questionnaire . . . 73

6.2 Appendix B Survey Questionnaire Results . . . 76

6.3 Appendix C Survey Questionnaire p-values . . . 78

6.4 Appendix D Post Test Questionnaire . . . 80

(7)

List of Figures

1 The Overall Study Process. . . 12

2 Shackel's Usability Denition [21]. . . 14

3 Nielsen's Usability Denition [21]. . . 15

4 ISO 9241-11 Usability Denition [21]. . . 16

5 Dierent Consistency Issues in SMS and Mobile Voip Applications. . . 20

6 Aordance Issues in the Weather Applications of the iPhone and iPad. . . 21

7 Small Search Box Issues in the Applications of the iPhone and iPad. . . 21

8 Invisible Control and Missing Instructions Issues in the Twitter and Face-book Application on the iPhone and iPad. . . 22

9 Presentation. . . 25

10 System. . . 25

11 Touch Screen. . . 26

12 Keypad. . . 26

13 Applications. . . 26

14 Participants Selection for the Experiment. . . 27

15 Laboratory Setup for User Trials. . . 32

16 Participant Group Levels Comparison Across the Device. . . 34

17 Device Levels Comparison Across the Participant. . . 35

18 Percentage Graph Results of the Degree of Agreement Across Each System Statement on the iPhone. St. Represents Strongly. . . 41

19 Percentage Results of the Degree of Agreement Across Each System State-ment of the iPad. St. Represents Strongly. . . 41

20 Graphs Show Percentage Results of the Degree of Agreement Across the Each Statement of the Touch Screen Gestures on the iPhone and iPad Respectively. . . 43

21 Graph Shows Percentage Results of the Degree of Agreement Across the Each Statement of the Keypad on the iPhone. . . 45

22 Graph Shows Percentage Results of the Degree of Agreement Across the Each Statement of the Keypad on the iPad. . . 46

23 Percentage Graph Results of the Degree of Agreement Across Each Appli-cations Statement on the iPhone. . . 49

24 Percentage Graph Results of the Degree of Agreement Across the Each Statement of the Applications on the iPad. . . 49

25 Device Levels Comparison Graph of the Performance Measurement of the Participant Group Levels for the Task Facebook Login. . . 54

26 Device Levels Comparison Graph of the Performance Measurement of the Participant Group Levels for the Task Send a Message. . . 55

(8)

27 Device Levels Comparison Graph of the Performance Measurement of the

Participant Group Levels for the Task Location Identication. . . 56

28 Device Levels Comparison Graph of the Performance Measurement of the Participant Group Levels for the Task Location Close View. . . 58

29 Device Levels Comparison Graph of the Performance Measurement of the Participant Group Levels for the Task New Note. . . 59

30 Device Levels Comparison Graph of the Performance Measurement of the Participant Group Levels for the Task Evernote Logout. . . 60

31 Novice User Mistakes Results Graph on the iPhone and iPad. . . 62

32 Experienced User Mistakes Results Graph on the iPhone and iPad. . . 63

33 Elderly User Mistakes Results Graph on the iPhone and iPad. . . 63

34 The Graph Presents the Median Comparison of the Degree of Agreement Against Each Question by Three User Groups on the iPhone. . . 67

35 The Graph Presents the Median Comparison of the Degree of Agreement Against Each Question by Three User Groups on the iPad. . . 68

(9)

List of Tables

1 iPhone and iPad Specication Comparison. The Bold Values Show the Dierences in Specication. . . 18 2 Usability Issues on the iPhone and iPad. . . 22 3 Advantages and Disadvantages of Qualitative Approach [52] . . . 23 4 Results of Surveyed Participants Comments Against the Dierent Features

of the iPhone and iPad. . . 50 5 Results from the ANOVA Tests for Estimating the Signicance Among the

Group Levels and Between the Device Levels Across Each Task. The p-value Shown the Signicance Among the Mean Values of Each Participant Group Against the Each Task. However, the Bold (p-values) Values Shown the Signicant Dierences Among the Groups and Between the Devices. . . 51 6 Participant Group Levels Comparison Results. The p-values Showed the

Dierences Between the Participant Group Levels on the iPhone and iPad. However, the Bold p-values Shown that there was a Signicant Dierence Between the Group Levels. . . 52 7 Device Levels Means Comparisons Results. The p-value Showed the

Dif-ferences Between the Means of the iPhone versus iPad Against each User Group. However, the Bold p-values Shown the Signicant Dierence. . . . 52 8 Means Results and Dierences of Each Type of Mistake Made by Each

User Group on the iPhone and iPad. However, the Bold p-values Shown the Signicant Dierence. . . 61 9 Post Test Questionnaires Results in the Form of the Median, the Standard

Deviation and the Dierences Between the Users Opinions Against Each Question. However, the Bold p-values Shown the Signicant Dierence. Q, iP and id Represented the Question, iPhone and iPad Respectively. . . 64 10 Frequency and Percentage Results of the User Comments on the Dierent

(10)

Chapter 1

1 INTRODUCTION

Human Computer Interaction (HCI) is concerned with the joint performance of tasks by the human and machines [1]. It has become valuable due to the fact that it involves the design, implementation and evaluation of computer interactive systems that human beings use in dierent contexts to accomplish their tasks. The main goal of HCI is to produce computer systems which are easy to use, well functional, ecient, eective, and that are in accordance with the requirements of the user [2]. The HCI goal would be ensured with the usability of the product. According to the Webster dictionary, the usability originate from the word usable which means capable of being used or convenient and practicable for use.

We are living in a mobile society [3]. New information and communication technologies (ICTs) have become a part of people's everyday lives. The mobile technologies are devel-oping very quickly [5] and the trend is changing from conventional desktop computing to the mobile computing. It facilitates the users to access and use their services anywhere anytime [4] e.g. doing trading, shopping, banking and bill payment etc [6]. So in the last few years, the developers redirect their design eort from a desktop perspective to a small screen approach [7]. Under the constraint of small size, the usability of the mobiles devices and their applications is a key factor for the success of mobile computing [8].

The mobile phone technologies have an impact on all age groups in our society. Studies illustrate that young people are more active users of technology [9, 10]. They are the rst users who adopt the new technology and develop an actual mobile communication culture [10]. On the other hand, the elderly people are also very keen to use the new mobile technology but because of the small size, invisibility of text and complexity of the functions, they avoid to use it [11]. The elderly population is growing rapidly in almost all the economically developed countries [12]. It is noticed that elderly people take more interest in games as compared to younger but because of the complexities and diculties of the user interfaces they cannot take part more eectively [13]. Earlier studies presented that most of the services are designed by considering the young population and mistakenly the elderly people are also considered as a part of the same age group [14].

In the mobile phone multi-touch technology, the iPhone, iPod Touch, and iPad are sophisticated devices, which provide advanced and sensitive user interactions based on gestures [15]. The iPhone and iPad support millions of the applications that designed and developed by the Apple and the third party. It has been considered that the Evernote is used by millions of the people in the world to remember their important notes and

(11)

synchronize them with their PCs [38]. Meanwhile, in the social network sites the Facebook is one of the leading applications having more than 500 million active users in the world [69]. Only in the Sweden more than 4 million people are using the facebook [71]. Similarly, the Google web application (search engine) is also a famous site having the large number of users [70]. The simplicity and consistency of the devices motivated the authors to conduct the comparative usability research studies of the iPhone and iPad.

In order to nd out whether the iPhone and iPad are performing according to user expectations or not, the authors performed the survey and empirical usability evaluation studies. The authors selected three frequently used applications (Facebook, Google Mobile Application, Evernote) and measured the usability performances of three user groups (novice user, experienced user, elderly user) as well as the iPhone and iPad. The age range for the young people is 22-34 and for the elderly people is 60-75.

1.1 Aim and Objectives

The aim of this research work is to evaluate and compare usability performance of the iPhone and iPad on three selected applications considering the elderly people as well as the young people. To achieve this goal the following objectives are dened:

Identifying usability issues regarding the iPhone and iPad.

Investigating usability performance dierences of the same applications on the iPhone and iPad.

Investigating the performance dierences between dierent user groups using the same applications and same devices.

1.2 Research Questions

1. What usability issues exist in the iPhone and iPad?

2. How does the performance dier while using the same application on the iPhone and iPad?

3. How does the performance dier using the same application on the same device by dierent user groups?

(12)

1.3 Study Process

The study process presented a hierarchical ow that was carried out during the the whole research work.

Figure 1: The Overall Study Process.

1.4 Thesis Structure

Chapter 1 introduces the thesis and denes the research domain. In Section 1.1 aim and objectives are given. In Section 1.2 research questions are given. In Section 1.3 the overall research process is presented in the form of diagram .

Chapter 2 provides background knowledge about the usability. In Section 2.1 usability and its dierent attributes are discussed that dened by dierent authors. Section 2.2 describes how to measure the usability. Section 2.3 describes the performance measuring. Section 2.4 describes the usability evaluation. Section 2.5 describes the touch screen and smart phones. Section 2.6 describes the iPhone and iPad. Section 2.7 is about the main features of the iPhone and iPad. Section 2.8 presents the related work and Section 2.9 presents the usability issues concerning the iPhone and iPad.

(13)

a literature review. Section 3.2 presents the survey design . Section 3.3 consists of the experiment design . Section 3.4 presents the evaluation methods and Section 3.5 presents the validity threats in research.

Chapter 4 presents the survey and experimental analysis results and discussion. Sec-tion 4.1 presents the survey. SecSec-tion 4.2 describes the demographics results of the survey. Section 4.3 presents the usability and preference testing of the survey data. Section 4.4 describes the participants comments regarding the usability of the iPhone and iPad in the survey questionnaire. Section 4.5 presents the measurement of usability attributes.

Chapter 5 contains conclusions and future work of the thesis. Section 5.1 presents the conclusions of the research study and Section 5.2 presents the future work.

Chapter 6 contains the Appendices. In Section 6.1 Appendix A represents the survey questionnaire. In the Section 6.2 Appendix B survey Questionnaire results are given. In Section 6.3 Appendix C contains the p-values of the survey results. In the Section 6.4 Appendix D presents the post test questionnaire. Section 6.5 contains the Appendix E that presents the experimental data i.e task time on both devices.

(14)

Chapter 2

2 BACKGROUND

2.1 What is Usability?

Usability in itself is not a new topic [17]. It is considered as a core term HCI [18] and has become an important part in the design of products, software applications and devices. However the user centered design and goal directed interaction design has increased us-ability of products [17]. The role of usus-ability is to make the system eective and easy to use [19]. There are many denitions of the term usability. According to ISO 9241-11 the term usability can be dened as The extent to which a product can be used by specied users to achieve specied goals with eectiveness, eciency and satisfaction in a specied context of use [20].

Usability is the composition of the entities [21] e.g. eectiveness, eciency, learnability, memorability etc. Some times these entities are dened as dimensions, components, scales or factors of usability. All these forms have the same meaning, so the commonly used term for them is usability attributes [21].

Shackel [19] is one of the famous authors who realized the importance of usability and dened it as a relative property of a system. This property makes the relationship between users and the systems; i.e. the subjective usability measure. The other side relates to objective measures of interaction i.e. how easily and eciently users could achieve their goals . Shackel dened the following attributes to measure the usability.

Eectiveness: how eciently and with minimum errors the user completes his goal.

Learnability: how much time the user required learning and how long this could remember it.

Flexibility: adaptation to change in tasks. Attitude: user satisfaction with the system.

(15)

Nielsen [22] is another usability expert, who dened usability by considering the following attributes.

Learnability: the system should be simple and easy to learn.

Eciency: the system should be capable to perform the users work within specied time.

Memorability: it should be easy to remember, so that the user can easily use it after some period of not using the system, without having to learn all again. Errors: the system should have low error rate and provide help to the user while

he/she make errors.

Satisfaction: the system should be attractive, so that the user feels comfort and satisfaction while using it.

Figure 3: Nielsen's Usability Denition [21].

Since from last 15 years, the International Standard Organization (ISO) has developed dierent standards for HCI and usability. The functionality of these standards is to make the system simple and consistent. The ISO 9241-11 decompose usability into three attributes [23] that are as discussed below.

Eectiveness: the accuracy and time for specied goal achievement by specied users.

Eciency: utilization of resources for completion of specied tasks. Satisfaction: the user must be feeling comfort while using the system.

The software engineering community has also associated usability with interface design. The ISO 9126-1 dened usability as quality in use. This denition has the same meaning as the ISO 9241-11 dene usability but it is used in specic context. For the measurement of usability, ISO 9126-1 specied the following usability attributes [24].

Understandability: the software product must have the capability that the user can easily understand and utilize it for the specied task.

(16)

Figure 4: ISO 9241-11 Usability Denition [21].

Learnability: the software system should have the capability that the user should learn it easily.

Operability: the software system must have the capability that user could easily operate and control it.

Attractiveness: the software system must have capability to attract the user. The existing usability studies on mobile applications dene the nine usability attributes such as: learnability, eciency, memorability, errors, user satisfaction, eectiveness, sim-plicity, comprehensibility and learning performance. Selection of suitable attributes for evaluation of mobile applications depends on the nature of the applications and the ob-jectives of the usability study [25].

2.2 Usability Measurement

Many past studies presented that the usability is dependent on the context of use [18]. This context of use is formed by the user, task, equipment and environment [26]. Most of the studies are conducted in controlled environment, where the user performed some predened tasks using the specied system. The usability can be assessed by consider-ing several measurable parameters. These parameters can be divided into two groups: subjective preference measures and objective performance measures.

Subjective preference measures are being assessed by considering what the user likes and dislikes about the system. It can be measured by using for example a questionnaires [27]. An advantage of a questionnaires is that they can provide the evaluator with feedback from the user's point of view. Another benet of the questionnaires is to provide the comparable measures across the systems that being evaluated [28].

The objective performance measures relate to the eectiveness of system (e.g. time, errors, and number of activities). Objective performance measures are being assessed by giving the specied tasks under a specied time limit. Sometimes it can be measured by counting the number of tasks per unit time [27]. The eciency measurement of the system is also related to the total time on task, usages of the resources and mental resources used to manipulate the system interfaces.

(17)

2.3 Performance Measuring

Performance is the degree to which the system accomplishes its designated function under certain constraints [29]. In this study, the authors followed the usability attributes that dened by ISO 9241-11 i.e. eectiveness, eciency and satisfaction. To test the usability attributes following metrics were measured.

Task Time: How long it took a participant to complete a task in seconds [30].

Errors: Errors are an unintentional action, mistakes, slips and omissions, a user makes while performing a task [30].

Satisfaction: It is the reaction of the participants about the overall performance of the system.

2.4 Usability Evaluation

Usability evaluation is the systematic process of collecting data, in ordered to have a better understanding of the users and how the user groups use the product to perform a specic task under specied conditions [31]. It has been required in dierent stages of the system development process. Its goal is to provide feedback that helps to improve the quality and functionality of the system [32]. Zhang [33] dened three approaches of usability evaluation methods such as: testing (e.g. Coaching method, performance measurement, thinking aloud protocol), inspection (e.g. heuristic evaluation) and inquiry (e.g. interviews, eld observation, questionnaires). In the usability testing approach, specied users perform specic tasks using the system or prototype [21]. The usability evaluator examines, how easily and eciently the users perform their tasks. The usability inspection approach involves the usability experts or professional to examine whether the system or each part of the system follows usability principles. In the usability inquiry approach the evaluator collect information about user perceptions and understanding of the system through interviews, surveys and verbal discussions [21].

2.5 Touch Screen and Smart Phones

Touch screen user interfaces have become more attractive in electronic devices [34]. There are two types of touch screens: capacitive and resistive. Resistive touch screens are made of a number of metallic and electronic conductive layers, separated with a small gap. When a user tap on the touching surface, both layers make a connection and cause the electric current that activate the touching event. Capacitive touch screens work by sensing conductive object e.g. nger. They allow the multi-touch functionality that can be performed by using multiple bare ngers [35]. The touch screen input method is becoming

(18)

popular in the smart-phones and other mobiles devices such as PDAs and tablet laptops [36]. In 1993, IBM presented the rst smart-phone and its name was Simon. The main features of Simon were a calendar, a calculator, a world clock, and an email client. It had a touch screen interface that let the users write text by using an on-screen keyboard or a stylus (i.e. stick like a pen). There is no proper denition of the term smart phone. However, it is considered as a mobile phone that has a greater computing power than the normal cell phones and having the features like the PCs. After launching the iPhone, the user interface design had great impact on the smart-phones with multi-touch technology [37].

2.6 The iPhone and the iPad

Apple designs, manufactures and markets a range of digital products. These products contain personal computers, mobiles devices, music players, related software and third-party applications. In the past few years, the company has launched products such as the iPhone, iPod, and iPad [38]. The iPhone and iPad are considered as revolutionary Apple products [39] and both have some similar features and look like the smaller and larger version of each other. However in some perspectives they have some dierences such as RAM, size, resolution and weight [40]. These similarities and dierences are presented in the Table 1.

Feature iPhone 4 iPad

Chip A4 A4

OS iOS 4 iOS 4 Processor 1 GHz ARM Cortex-A8 1 GHz ARM Cortex-A8 RAM 512 MB 256 MB Connection Wi-Fi Wi-Fi

Size 4.5 x 2.31 x0.37 inch 9.56 X 7.47 X 0.52 inch Resolution 960 X 640 pixel 1024 X 768 Pixel Weight 137g 730g

Table 1: iPhone and iPad Specication Comparison. The Bold Values Show the Dier-ences in Specication.

2.7 Features of the iPhone and iPad

The main features of the iPhone and iPad are described below: 2.7.1 Multi-touching

The multi-touch technology is used as an input method for the iPhone and iPad. It is dened as the technology where the user can interact with system with multiple inputs at a time e.g. two or more ngers for a single task. The users can use their ngers for

(19)

typing email, enlarging text, swapping through photos and for using zooming functions. All input functions are performed by using the ngers at touch screen interfaces.

2.7.2 Operating System (iOS 4)

The iOS 4 is an operating system that is being used by Apple for the iPhone 4 and iPad. It is considered as a simple operating system. The user can perform various tasks (e.g. writing email and zooming text) simply using the ngers on the touch screen interface. The user can use millions of dierent built-in applications with a few nger gestures such as tap, drag, swipe and pinch [38].

2.7.3 Applications (Apps) Store

Apps Store is a database that is especially designed for Apple's products. It contains business applications, education applications, entertainment applications, social network-ing applications, news applications and so on. The applications are designed by Apple's designers and developers as well as by a third party. Some applications are free but for some applications you will have to pay [38].

2.8 Related Work

Tsung [41] performed an age based usability study on the iPhone. Three age groups i.e. college-level youth, middle-aged professionals and senior citizens were taken. The conclusion of this study was that the system should not only consider the glamorous animation and fancy utilities but all human physical conditions e.g. hearing and eye sight that will make the system more senior friendly.

Chaparro [42] evaluated the keyboard performance of the Netbook and iPad. They measured the performance on three dierent style keyboards. The iPad soft touch key-board (portrait as well as in landscape) and the Acer Netbook physical keykey-board. The attributes and their metrics are given below.

Performance (task time and errors) Perceived Mental Workload

Satisfaction

The study showed that there was a slight dierence in the performance of the Netbook physical keyboard and the iPad landscape and no dierence in the performance of the iPad portrait and landscape keyboards. The satisfaction level of the iPad landscape was slightly higher than the physical keyboard and portrait was slightly lower than the physical keyboard. The preference level of the landscape iPad keyboard and physical keyboard was same the while portrait keyboard was less preferred by the participants .

(20)

In another study, six sta members at Curtin University were supplied with the iPad to test usability for 100 days. The iPad was handed over to each sta member and at the end of the third month these participants were interviewed. The research showed that there was no problem for inexperienced users using the iPad. The consistency issues were noted like the other studies and also some common functionalities were identied [43].

The Nielsen and Norm group (NNGroup) [44] conducted a usability study of the iPad applications and contents, a few weeks after Apple launched the device. They tested seven users; all had at least three months iPhone experience and only one of them was an iPad user. In this study they assessed the interface of applications, websites and specic iPad applications, touch screen, changing orientation, and dierent gestures. They criticized and pointed out that the applications are designed without considering the importance of usability e.g. read tab asymmetry, zoom the map or zoom the page, use of proper mental models, small target area, accidental tapping and the back button etc. They highlighted many usability issues e.g. inconsistencies and accidental actions etc.

2.9 Issues in the iPhone and iPad

From the literature [44], the authors found some usability issues that are discussed below: 2.9.1 Consistency

The consistency issue remain almost all the time with the iPhone and iPad applications. Figure 5 shows the inconsistency in the button, keyboard appearance and writing envi-ronment of the messages of SMS and MobileVoip application of the iPhone.

(21)

2.9.2 Aordance

The other issue in the iPhone as well as the iPad application was aordance. Users don't know that something is touchable unless it looks so [44]. In the weather application of the iPhone it is very dicult to add new a location as the option lacks the aordance. Figure 6 shows this aordance problem in the weather application of the iPhone. The i at the right bottom side of the application is used for adding the new location but the option is not very clear as well as not seems to be tappable.

Figure 6: Aordance Issues in the Weather Applications of the iPhone and iPad.

2.9.3 Small Search Box and Missing Information

Figure 7 shows the small search box and missing search button at the bottom of the Apps Store application of iPad. As there is a big screen on the iPad the search box needs to be more clear and also users expect the search button at the bottom of the application.

(22)

2.9.4 Invisible Control and Missing Instructions

Figure 8 shows the invisible control and missing instructions in the iPhone and iPad applications. In Figure 8 at the right side when the message is too long it create a scroll bar but it remain invisible until a someone tap on it. Similarly in twitter there are four pages that can swipe forward and backward but there are no instructions to move them. Another miss leading button at the top of the twitter application that seems like a setting button but in fact it is used for reply option.

Figure 8: Invisible Control and Missing Instructions Issues in the Twitter and Facebook Application on the iPhone and iPad.

Table given bellows describe the list of other usability issues in the iPhone and iPad.

Usability Issues Detail

Inconsistency in interaction Nothing happens Enlarging the picture

Hyper-linking to a more detail page about that item Replace the picture with new picture

Popping up a set of navigation choices Accidental touch Touch something that one did not mean to

False back button Button that look like back button, but when pressed it lead to new page

Lack of aordance Tappable elements look like it not supposed to touch and untappable look like tappable

Hyperlinks Missing of important hyperlinks like wired magazine using back button instead of hyperlinks

Changing orientation Application looks dierent in portrait and landscape Gestures Dicult to remember gestures

Multiple panels Small font size and crowded contents Auto-correct Should turn into autocomplete

(23)

Chapter 3

3 RESEARCH METHODOLOGY

The research methodology is the systematic way to solve the research problem. Commonly two main approaches are used for the research studies such as quantitative and qualitative [45]. However, Creswell [46] has also introduced a third type of the research approach called a mixed method approach i.e. the combination of quantitative and qualitative approaches.

The quantitative research approach is based on the measurement of quantity or amount. It is used when one begins with a hypothesis and tests for conrmation and dis-conrmation of that hypothesis [47]. It is applicable to phenomena that can be expressed in terms of quantity [48]. Generally, the survey and experiment are used in the quantitative approach to inquiry the hypothetical issues and participants of the studies [46]. Some advantages of the quantitative approach [49] are listed below:

1. Data can be gathered from large user groups in a short period of time 2. Large sample data can be generalized to the entire population

3. Quantitative research is repeatable

The qualitative research approach is concerned with subjective assessment of attitudes, opinions and behaviors [48]. It is an interpretative approach which uses the meaning of phenomena as understood by the participants of the studies [50]. Generally, open ended questions are used in qualitative research [51]. Some of the advantages and disadvantages are listed in Table 3.

Advantages of Qualitative Approach Disadvantages of Qualitative Approach It goes deep and takes into account all details in Finding cannot be generalized because they the environment are particular environment dependent No need to recruit a large number of participants The nding can be biased according to the Flexibility in time and place understanding of the researcher

No need to conduct interviews of all participants at same time

Table 3: Advantages and Disadvantages of Qualitative Approach [52] .

In this research, the authors adopted the mixed research approach. The quantitative research approach was used to perform the experiment and the qualitative research ap-proach was used to conduct the literature review and analyze the open ended questions and participants comments. These research studies were carried out in multiple steps such as a literature review, survey design, experiment design, pilot tests studies, experiment

(24)

execution, selection of suitable evaluation methods, analysis of gathered data, results, conclusions and future work.

3.1 Literature Review

In the rst step a literature review was done. The objectives of a literature review was to understand the usability, usability attributes, usability issues in smart phone, and the usability issues in the iPhone and iPad. Another objective of a literature review was acquiring the understandability of the dierent statistical usability evaluation methods.

The authors utilized the all available information resources to carried out this re-search work. The mentioned keywords are used to nd out the related data from all available databases. However some of the authentic knowledge resources were listed as BTH Library's books, IEEE, ACM digital library, Engineering Village, Google Scholar, Springer-link, Scopus, and Ebrary.

3.2 Survey Design

To nd usability issues the authors used the survey technique. The objectives of the surveys were:

1. To validate the issues regarding the usability of the iPhone and iPad that authors nd out during the literature review.

2. To get the input for the experimental design.

3. To nd out the users point of views about the usability of the iPhone and iPad. A survey is an eective way to gather information from a large number of the population in a short period of time at relatively low cost. It is considered that the respondents are less likely try to please the researcher and provide the social acceptable responses. The survey technique gives time to the respondents to think about the question or statement before answering and do not bind them to complete it in one setting [53].

The seven point Likert scale was used in a survey questionnaire that helps the users to express their degree of agreement with a statement. The numbers (1-7) were used for strongly disagree, disagree, near to disagree, neutral, near to agree, agree and strongly agree respectively for each statement.

In order to collect feedback from the users about the iPhone and the iPad, the authors divided the survey into demographics, systems (iPhone, iPad) information, applications information, touch screen features and keypad functionality. Each part contained several related statements, as well as a space for own comments. The detailed survey questionnaire is given in Appendix A. However, here the small piece of each section of questionnaire was presented and structured as follows.

(25)

1. Presentation This section presented the users demographics (age, sex, qualication, using experience), purpose of the survey study, explain the questionnaire and provide the general information. Following is the presentation section of the survey questionnaire.

Survey Questionnaire for Evaluating the Usability of the systems (iPhone/iPad)

Participant Number: ______________ Sex: ___________________ Home Country: _____________________ Age: ________________ Education level: 2 BSc 2 MSc 2 PhD 2 Other __________________

This questionnaire gives you an opportunity to describe your experiences of the system. Your responses will help us to evaluate the usability of the system.

To as great degree as possible, consider all the tasks that you have been performing with the system while you answer these questions. Please read each statement and indicate how strongly you disagree or agree with the statement by marking (X) in the circle. If a statement does not apply to you, leave it empty or use the word (N/A) at the comments line.

Please write comments to elaborate on your answers (if you have any).

As you complete the questionnaire, please do not hesitate to ask any questions. Thank you.

Figure 9: Presentation.

2. System (iPhone/ iPad) This section presented the overall impression of the de-vices. The objective of this section was to obtain the feedback about the system design, graphics, capabilities and somehow about the ease of use. A piece of the system section of a questionnaire is presented in Figure 9 below.

Figure 10: System.

3. Touch Screen This section presented the usability of the touch screen gestures. Figure 10 shows a few statements of such section.

(26)

Figure 11: Touch Screen.

4. Keypad This section presented to gain the user opinions about the performance of the keypad. The statements contained the dierent keypad aspects such small size of keys, landscape style and single character keys. The Figure 10 shows a few statements regarding the keypad section.

Figure 12: Keypad.

5. Applications This section presented the dierent usability features related to the applications simplicity, ease to learn, consistency, visibility of the text and recoverability of the mistakes. A small piece of this section presented in the Figure 12.

(27)

3.3 Experimental Design

Experimental design is the process of planning a study to meet specied objectives. Proper planning of an experiment is important in order to ensure that the right type of data, a sucient sample size and resources are available to answer the research questions of interest as clearly and eciently as possible [54, 55]. The following aspects provide further insight into the experimental design.

3.3.1 Participant Selection

The authors selected the three user groups such as young novice users (20), young expe-rience users (20) and elderly users (20). All the young users were bth students and well known with the English language. In the elderly users 12 were bth employees; ve were the Karlskrona public library employees and three were Ronneby public library employees. In the beginning, the authors were interested to divide the elderly people into two groups like the young people (novice, experienced) and also visited the old people apartments in Karlskrona and Ronneby. However, after meeting with the management of the old people apartments the authors realized that it is dicult to collect reliable data from the elderly people because of language communication problems. So the authors decided to make a single group of the elderly people and involved all those people who would speak, write, read and understand the English language. Figure 13 shows the participants selection for the experiment.

Figure 14: Participants Selection for the Experiment.

Novice Participant The person who was never used the smart phone, touch screen systems or the internet on small devices.

(28)

Experienced Participant The authors considered the person as an experienced par-ticipant who was using the smart phone or the iPhone more than three months.

Elderly Participant The person who was at least using a computer system or smart phone or having the experience about the touch screen system.

3.3.2 Variables Selection

A variable is any character or attribute that can vary across people or situation or thing [56]. There are two basic types of variables; independent variable and dependent variable. The independent variable is one which the experimenter controls and manipulates to see the eect on the treatments (dependent variable). On the other hand, the dependent variable is the responses of the independent variable [56].

In the experiment, the authors used two types of the variables, independent variable (factor) i.e. Participant group and dependent variable i.e. Device. The Participant group variable has three levels such as novice user, experienced user and elderly user. Similarly, the Device variable also has two levels such as the iPhone and iPad.

3.3.3 Hypotheses

The rst step in the research process is the formulation of hypothesis about a specic issue [57]. It is a precise problem statement that can be directly tested through empirical investigation [58].

In comparative usability study of the iPhone and iPad across the three groups of par-ticipants, the authors were interested to measure the performance dierences between the device levels as well as the dierences among the participant group levels. The hypotheses statements were dened as as follows:

H01: There is no dierence in group levels over the device variable. Ha1: There is a dierence in group levels over the device variable. H02: There is no dierence in device levels across the group variable Ha2: There is a dierence in device levels across the group variable.

Here H0and Ha represent the null hypotheses and alternative hypotheses respectively. 3.3.4 Task Denition

For the execution of usability tests, six tasks were dened. Each task and its description were designed on a separate paper. The presentation of the tasks was as follows:

(29)

Facebook

Task 1: Login to Facebook.

Email address: usbtest2011@gmail.com Password: usbtest2k11

Description:

Tap on Facebook

Tap on the First Text box Email and enter the email address Tap on the password text box and enter the password

Tap on Go at the keypad To use capital text tap on To use numbers tap on

(30)

Task 2: Compose a new message on Facebook and send it. To: azambth72@yahoo.com

TEXT: This is a usability test message for the iPhone Send

Description:

Tap on the Facebook Home button Tap on messages

Tap on create message at the top right corner of the Facebook application Enter email address to whom you want to send a text , i.e. Given above Tap just below the line and Write a short message

Tap on Send Button at the top of application Google Map

Task 3: using the Google mobile application nd the location of Blekinge Tekniska Högskola and tap on the sign to conrm the location.

Description:

Tap on Google mobile application Tap on Apps at middle of bottom Tap on Maps

Write Blekinge Tekniska Högskola Tap on search button

(31)

Task 4: Make a close view of Blekinge Tekniska Högskola building with zoom in function.

Description:

Tap on Layers at the bottom left corner Select the satellite from list

Tap on search box

Write Blekinge Tekniska Högskola Tap on sign for zoom in

Evernote

Task 5: Create a new note in the Evernote application User name: azamjarral

Password: aS5AK9 Title: First Note

Text: My rst note in the Evernote application Save

Description:

Open Evernote application Tap on Sign in button

Tap on text box username and enter username Tap on the text box password and enter password Tap on Sign in

Tap on New Note at the bottom left corner. (For iPhone at the middle of Facebook application)

(32)

Task 6: Logout from Evernote Description:

Open Evernote application Tap on the setting icon Find the Logout option Tap on Logout

3.3.5 Procedure

The authors booked a silent room in BTH that was used as a usability laboratory. To provide the same environment to all the participants, the authors xed the devices and chairs. Before starting the experiment, all the tasks were arranged at a separate table. it was conducted with a single user at a time. The experimental setup is shown in Figure 14.

Figure 15: Laboratory Setup for User Trials.

3.3.6 Experimental Material

The experimental material contained the iPhone, iPad, documents for the tasks descrip-tion, demonstration and post test questionnaire.

(33)

3.3.7 Post Test Questionnaire

To know the participants usability experience abut the systems, a post-test questionnaire was designed. It contained close ended as well as open ended questions. The close ended questions were designed by using seven points Likert Scale and open ended questions were also part of the same questionnaire but they were in the form of participants comments. The close ended questions limit the respondent to the set of alternatives being oered, however the respondent could easily respond the maximum questions within the limited time. On the other hand the open ended questions help to explore the respondent and thus avoiding the bias which may occur in the case of close ended questions [59]. The detailed questionnaire is given in Appendix D.

3.3.8 Pilot Studies

Three pilot studies were conducted i.e. an experiment, a survey and a post test question-naire. Before starting the experiment pilot test were conducted to assure that the tasks were properly dened and the participants would have no diculty in understanding every step in each task. Three tests were conducted, one for each group and improvements were made on the basis of their feedback. At the start the authors decided to just mention the task e.g. login to Facebook, however after the pilot test, the authors realized that it was not easy for the novice users as well as for elderly user to successfully complete the given tasks. Then the authors decided to write down the step by step description of each task with the symbolic representation wherever required. In this way the ambiguities were removed in the experimental tasks.

Similarly the pilot studies were also conducted for the survey questionnaire and post test questionnaire. After those studies, the changes were made in both questionnaires according to the requirement of the users.

3.4 Evaluation Methods

In order to evaluate the survey and experimental data, the authors use multiple statistical test methods. The detail of each method is given below.

3.4.1 Mann Whitney Test

The Mann Whitney test is a nonparametric test that can be used for comparing two unequal sample size and unpaired data [60, 61]. The key result is the p value that shows the signicant dierence between both samples [62]. In this research, the survey data was unpaired, unequal samples size and consist on rating scale that is why the Mann Whitney test used to measure the signicance between the iPhone and iPad.

(34)

3.4.2 Normality Test

After obtaining the data from the experiment, it was important to test the normality of the data. The authors read three test techniques: Shapiro Wilk Test, Kolmogorov-Smirnov (KS) Test, and Histogram. The KS test is used for more than 50 samples, so the authors did not use this method. On the experimental data, the Shapiro Wilk Test and Histogram techniques could be applied to check the normality. however the authors used Shapiro Wilk test online software that directly show whether the data is normally distributed or not [63].

3.4.3 Analysis of Variances (ANOVA)

ANOVA is the most powerful and robust parametric method used to measure the signif-icance. It has some assumptions such as the homogeneity of variance or normally distri-bution of the data. The ANOVA is used to measure the signicant dierence between two or more than two groups [64]. In the experiment, there were three user groups and two devices. The purpose experiment was to measure the dierences among the groups and between the devices. So to analyze the data, the authors used two way ANOVA with replication. Further to see where the signicant dierences between the devices and among the groups were, the unpaired t-test and paired t-test were used respectively. 3.4.4 Unpaired T-test

The unpaired t-test is used to see where the signicant dierences among groups across the each device were. The comparison of the groups was made as follows.

Figure 16: Participant Group Levels Comparison Across the Device.

3.4.5 Paired T-test

The paired t-test used to see where the signicant dierences between the devices across the each group were. The comparison between devices across groups was as follows.

(35)

Figure 17: Device Levels Comparison Across the Participant.

3.4.6 Wilcoxon Matched Paired Signed Ranks Test

The Wilcoxon Matched Paired Signed Ranks test is a nonparametric test that is an alternative to the paired t-test. It should be used if the distribution of dierences between pairs may be non-normally distributed [65]. The authors applied this method to nd out the signicance between the post-test questionnaires of the iPhone and iPad.

3.4.7 Error Bars

The error bars are used to estimate the variation or distribution of a statistical data in the graphs. They can represent the standard deviation, the standard error, the condence interval, or some other measure of error or uncertainty. Here the authors only discussed the standard deviation and standard error. The standard deviation is a measure of variation in data about their mean and standard error is a standard deviation of the estimated statistical data such as mean, and regression coecient etc. The error bars do not let you decide whether the dierence between the mean is statistically signicant or not when the two samples error bars do overlap or not [66]. It indicates how closely the population mean is likely to be estimated by the sample mean.

3.5 Validity Threats

Validity refers to the conceptual and scientic soundness of a research study and investi-gation [67]. There are many potential issues that may aect the reliability of the research. Four types of validity threats discussed in the literature [46] are as follows:

Internal validity threats External validity threats Constructive validity threats

(36)

3.5.1 Internal Validity Threats

Internal validity threats refer to the research design that causes the interferences of the researcher's ability to draw the correct inference from the gathered data [46]. They might be the inadequate procedures, wrong participant's selection, and technical skill of the researchers, ambiguities in the experiment, post-test questionnaire and survey design and improper instrumentation of the experiment [67, 68]. To overcome such types of issues the authors studied the core concepts of survey design, experiment design and ongoing research studies about usability of the iPhone and iPad. In the survey and post test questionnaire, the authors obtained the demographics that help to minimize the wrong selection of the participants. Pilot tests were conducted for the survey questionnaire, experiments and post-test questionnaire which helped to nd out the ambiguities in all of them. Furthermore the authors also used the counterbalancing technique to overcome the bias.

3.5.2 External Validity Threats

External validity threats relate to the generalizability of the results of a research study [67]. They might be the sample characteristics, stimulus characteristics and settings, reactivity of experimental arrangements, multiple treatments interference, novelty eect, reactivity of assessment and timing of measurement [67]. To minimize the external validity threats, the authors requested all of the participants to report their own experiences because the purpose of the study is to evaluate the performances of devices not to the participants. However there might be external validity risks, as people really do not report their personal reality because they want to see themselves in a good light [68].

3.5.3 Constructive Validity Threats

Constructive validity threats relate to the selection of denitions and the measures of the variables in a study [46]. Authentic data sources such as IEEE, ACM Digital Library, ScienceDirect provide the correct denitions and measures of the variables. The authors try to use the authentic data sources to minimize such types of validity threats.

3.5.4 Statistical Conclusion Validity Threats

Statistical conclusion validity threats relate to the statistical conclusion of a study. These threats might be the cause of the inadequate selection of a statistical power such as sampling and suitable methods [67, 68]. To reduce such type of validity threats, the authors studies dierent statistical methods in detail.

(37)

Chapter 4

4 STATISTICAL ANALYSIS, RESULTS AND DISCUSSIONS

This chapter presents the analysis, results and discussions of the results about the collected data of the studies. There are two main sections of this chapter; the rst one presents the results and discussions on the results of the survey data and the second one presents the results and discussions on the results of the experimental data.

4.1 Survey

In order to evaluate the survey data, the authors clustered the 42 survey statements into subsections such as system results, touch screen results, keypad results and applications results. The results of the survey responses are given in the Appendix B and the p values are given in the Appendix C.

4.2 Demographics Results

This section of the survey was presented the demographics of the respondents. The authors collected feedback from 14 (8M, 6F, age = 21-35 years) iPhone and 10 (7M, 3F, age = 23-47 years) iPad users through the survey. 69% of the respondents had been using the iPhone for more than six months. 8% of the respondents had been using the iPhone for four to six months and the remaining 23% participants had been using it for the last one to three months. On the other hand, the iPad responses showed that 20% of the respondents were using the iPad for more than six months. 30% of the respondents had been using the iPad for four to six months. 30% of the respondents had been using it for the last one to three months and 20% of the respondents had been using the iPad for less than one month.

4.3 Usability and Preference Testing

4.3.1 System Results

The rst section of the usability testing questionnaire named system contained 15 state-ments. Each statement was related to the overall satisfaction about the iPhone and iPad. The results and signicances between the users opinions against each statement regarding the iPhone and iPad are:

The system is easy to use for opening applications.7% (1) of the iPhone participants disagreed with a statement and they considered it dicult for opening applications. 21% (3) were near to agree, 14% (2) were agree and 57% (8) were strongly agree that the

(38)

iPhone was easy to use in opening application. On the other hand, the iPad participant's responses results presented that 10% (1) was near to agree, 70% (7) were agree and 20% (2) were strongly agree with the above statement. However the comparative study regarding this statement presented that there was no signicant dierence (p = 0.183 > 0.05) between the user views for the iPhone and iPad.

The system is easy to use in closing applications. The iPhone results regarding this statement showed that 8% (1) of the respondent was disagree and he thought it is not easy to use in closing applications; 25% (3) participants were neutral; 8% (1) participant was near to agree, 25% (3) were agree and 33% (4) were strongly agree. The iPad participants responses results presented as: 11% (1) was disagree, 11% (1) was near to disagree; 11% (1) was neutral; 44% (4) were agree and 22% (2) were strongly agree. The comparative study of the user's opinions concerning the iPhone and iPad presented that there was no signicance (p = 0.404 > 0.05).

The system is easy to use in nding help. The iPhone participant's responses results concerning this statement were presented as: 14% (2) were disagree, 7% (1) was near to disagree; 43% (6) were neutral; 14% (2) were near to agree, 7% (1) was agree and 14% (2) were strongly agree. Similarly, the iPad participants responses results presented as: 10% (1) was disagree, 20% (2) were near to disagree; 40% (4) were neutral; 20% (2) were near to agree and 10% (1) was agree. The comparative study of the user's views showed that there was no signicant dierence (p = 0.083 > 0.05) between the iPhone and iPad. I can easily use the system in the sun light. The iPhone participants responses results regarding this statement were presented as: 7% (1) was near to disagree; 21% (3) were neutral; 36 % (5) were near to agree, 21% (3) were agree and 14% (2) were strongly agree. The iPad responses results presented as: 20% (2) were near to disagree; 40% (4) were neutral; 30% (3) were near to agree and 10% (1) was agree. The comparative study concerning this statement showed that there is a signicant dierence (p = 0.011 < 0.05) in user's views regarding the iPhone and iPad. The statistical result showed that the users preferred the use of iPhone in sun light than the iPad.

It is easy to nd a new application in the Apple store. The iPhone responses results showed that 7% (1) was near to disagree; 14% (2) were neutral; 14% (2) were near to agree, 7% (1) was agree and 57% (8) were strong agree. Similarly the iPad responses results regarding this statement showed that 10% (1) was disagree, 10% (1) was near to disagree; 30% (3) were neutral; 30% (3) were near to agree, 10% (1) was agree and 10% (1) was strongly agree. Comparative statistical result showed that there was a signicant dierence (p = 0.018 < 0.05). The users responses result showed that they would easily use the iPad for nding the new applications on the Apple store comparing the iPhone.

The layout of the applications on the system screen is clear. The results of the iPhone respondents showed that 7% (1) was disagree; 21% (3) were neutral; 14% (2) were near to agree, 21% (3) were agree and 36% (5) were strongly agree. Similarly, the iPad participants

(39)

responses results were presented as: 10% (1) was neutral; 60% (6) were agree and 30% (3) were strongly agree across above statement. The comparison result showed that there was no signicance (p = 0.322 > 0.05) in users views for the iPhone and iPad.

The graphics of the system are appealing for all age groups. Responses results regarding the iPhone presented that 7% (1) was disagree; 7% (1) was neutral; 21% (3) were near to agree, 29% (4) were agree and 36% (5) were strongly agree. Similarly the iPad results presented that 11% (1) was near to agree, 56% (5) agree and 33% (3) were strongly agree. The comparison result showed that there was no signicant dierence (p = 0.341 > 0.05) between the participants opinions about the iPhone and iPad.

I can easily change the background colour of the interface. The iPhone participant's responses results presented as: 14% (2) were disagree, 14% (2) were near to disagree; 21% (3) were neutral; 14% (2) were near to agree and 36% (5) were strongly agree. On the other side, the iPad participant's responses results showed that 10% (1) was near to disagree; 30% (3) were neutral; 20% (2) were near to agree, 30% (3) were agree and 10% (1) was strongly agree. However the comparative study showed that there was no signicance (p = 0.489 > 0.05) between the participants opinions regarding the iPhone and iPad.

I think the system is designed for all age groups. The iPhone results showed that 7% (1) was strongly disagree, 7% (1) was disagree; 21% (3) were neutral; 14% (2) were near to agree, 21% (3) were agree and 29% (4) were strongly agree. Alike, the iPad results presented that 20% (2) were near to disagree; 10% (1) was neutral; 10% (1) was near to agree, 30% (3) were agree and 30% (3) were strongly agree. There was no signicant dierence(p = 0.467 > 0.05) in the user's opinions for the iPhone and iPad.

I like the nishing of the interface. The iPhone responses results presented that 7% (1) was disagree; 7 % (1) was neutral; 21% (3) were near to agree, 21% (3) were agree and 43% (6) were strongly agree. Similarly the iPad results presented that 20% (2) were neutral; 20% (2) were near to agree, 20% (2) were agree and 30% (3) were strongly agree. The comparative study showed that there was no signicant dierence (p = 0.382 > 0.05) in the participants' views for the iPhone and iPad.

The interface of the system is pleasant. The results of the iPhone responses: 7% (1) was disagree; 14% (2) were neutral; 21% (3) were near to agree, 29% (4) were agree and 29% (4) were strongly agree. Similarly, the results of the iPad responses: 10% (1) was neutral; 40% (4) were near to agree, 20% (2) were agree and 30% (3) were strongly agree. The comparative study showed that there was no signicant dierence(p = 0.467 > 0.05) between the users outlook for the iPhone and iPad.

It is easy to install the required application. The iPhone responses results showed that 7% (1) was disagree; 7% (1) was neutral; 14% (2) were near to agree, 29% (4) were agree and 43% (6) were strongly agree. Similarly, the iPad responses results showed that 11% (1) was near to disagree; 11% (1) was neutral; 33% (3) were near to agree, 22% (2)

(40)

were agree and 22% (2) were strongly agree. The comparison result showed there was no signicant dierence (p = 0.174 > 0.05).

It is simple to uninstall the application. The iPhone responses results showed that 7% (1) was disagree, 7% (1) was near to disagree; 7% (1) was neutral; 7% (1) was near to agree, 36% (5) were agree and 36% (5) were strongly agree. On the other side, the iPad responses results showed that 11% (1) was near to disagree; 11% (1) was neutral; 11% (1) was near to agree, 44% (4) were agree and 22% (2) were strongly agree. The comparative study showed that there was no signicance(p = 0.279 > 0.05) in the iPhone and iPad users views.

The system has all the functions and capabilities I expect it to have. The iPhone responses results showed that 29% (4) were disagree, 7% (1) was near to disagree; 7% (1) was neutral; 29% (4) were near to agree, 21% (3) were agree and 7% (1) was strongly agree. Similarly, the iPad responses results showed that 30% (3) were disagree, 10% (1) was near to disagree; 20% (2) were neutral; 20% (2) were near to agree and 20% (2) were agree. The comparison of the users point of views about the iPhone and iPad was showed no signicance(p = 0.265 > 0.05).

I like using the interface of the system. For such statement, the iPhone responses results showed that 7% (1) was strongly disagree, 7% (1) was near to disagree; 14% (2) were neutral; 7% (1) was near to agree, 43% (6) were agree and 21% (3) were strongly agree. The iPad responses results showed that 13% (1) was near to disagree; 13% (1) was neutral; 50% (4) were near to agree and 25% (2) was strongly agree. Similarly the comparative study about the user´s point of views regarding the iPhone and iPad showed no signicance (p = 0.233 > 0.05). Figures 17 and 18 showed the results graphs of the users opinions across the each statement regarding the iPhone and iPad respectively.

(41)

Figure 18: Percentage Graph Results of the Degree of Agreement Across Each System Statement on the iPhone. St. Represents Strongly.

Figure 19: Percentage Results of the Degree of Agreement Across Each System Statement of the iPad. St. Represents Strongly.

4.3.2 Touch Screen Gestures Results

The second section of the usability testing questionnaire named Touch Screen Gestures contained 16-21 statements. Each statement presented dierent usability features such as

(42)

zooming functionality, gestures memorability and learnability. The results obtained from the user responses for each statement presented as follows.

I can easily use the text zoom in and zoom out signs (gestures). The results of the iPad responses presented that 7% (1) was strongly disagree; 7% (1) was neutral; 7% (1) was near to agree, 14% (2) was agree and 64% (9) were strongly agree. The iPad results showed that 10% (1) was near to disagree; 10% (1) was neutral; 40% (4) were near to agree, 20% (2) were agree and 20% (2) were strongly agree. The analysis of the relationship showed that there was signicant dierence (p = 0.027 < 0.05). For this statement the users views result showed that they could easily performed the zoom in/out gestures using the iPad than the iPhone.

I like the gestures for selecting and zooming the text. The results of the iPhone re-sponses showed that 14% (2) were near to disagree; 7% (1) was near to agree, 50% (7) were agree and 29% (4) were strongly agree. Similarly, the results of the iPad responses showed that 20% (2) was near to disagree; 20% (2) were neutral; 10% (1) was near to agree, 40% (4) were agree and 10% (1) was strongly agree. The comparative study showed that there was no signicance(p = 0.108 > 0.05) in users opinions for the iPhone and iPad.

I think the gestures are easy to learn. The iPhone results showed that 7% (1) was disagree, 7% (1) was near to disagree; 7% (1) was neutral; 43% (6) were near to agree, 7% (1) was agree and 29% (4) were strongly agree. Similarly, the iPad results showed that 20% (2) were near to disagree; 10% (1) was neutral; 20% (2) were near to agree, 50% (5) were agree. There was no signicance (p = 0.424 > 0.05) in users opinions for the iPhone and iPad.

I think the gestures are easy to remember. The iPhone results showed that 7% (1) was near to disagree; 14% (2) were neutral; 29% (4) were near to agree, 21% (3) were agree and 29% (4) were strongly agree. On the other side, the iPad results showed that 40% (4) were neutral; 20% (2) were near to agree, 30% (3) were agree and 10% (1) was strongly agree. The comparative study showed that there was no signicance(p = 0.214 > 0.05) between users views for the iPhone and iPad.

I can easily change the location of the contents (icons) on the interface. The iPhone results showed that 7% (1) was disagree, 7% (1) was near to disagree; 7% (1) was neutral; 7% (1) was near to agree, 29% (4) were agree and 43% (6) were strongly agree. On the other hand, the iPad results showed that 10% (1) was near to disagree; 20% (2) were neutral; 50% (5) were near to agree and 20% (2) were agree. There was a signicance(p = 0.015 < 0.05)in users opinions for the iPhone and iPad. The results showed that the user could easily change the contents location using the iPad as compared to iPhone.

I can easily select the written text for deletion. The iPhone results presented that 7% (1) was strongly disagree, 14% (2) were disagree, 14% (2) were near to disagree; 14% (2) neutral; 21% (3) were near to agree, 14% (2) were agree and 14% (2) were strongly agree. Similarly, the iPad results showed that 30% (3) were disagree, 20% (2) were near

(43)

to disagree; 10% (1) was neutral; 30% (3) were near to agree and 10% (1) was agree. The analysis relationship result showed that there was a no signicance (p = 0.198 > 0.05). In Figure 19 presented the graphs results of the users opinions against each statement on both the devices.

Figure 20: Graphs Show Percentage Results of the Degree of Agreement Across the Each Statement of the Touch Screen Gestures on the iPhone and iPad Respectively.

4.3.3 Keypad Results

The third section of the survey questionnaire named keypad contained eight statements. Each statement assessed the performance of the keypad of both devices. The results obtained from the user's responses for the each statement are presented as follows.

I enjoy using the keypad while writing Emails. The results of the iPhone responses showed that 7% (1) was strongly disagree, 21% (3) were near to disagree; 14% (2) were neutral; 14% (2) were near to agree, 7% (1) was agree and 36% (5) were strongly agree. The results of the iPad responses showed that 10% (1) was strongly disagree, 10% (1) was disagree; 10% (1) was neutral; 20% (2) were near to agree, 40% (4) were agree and 10% (1) was strongly agree. The comparative study of the users opinion showed that there was no signicance(p = 0.446 > 0.05) for the iPhone and iPad.

I like the keypad in landscape style while writing Emails. The results of the iPhone responses showed that 14% (2) were disagree; 21% (3) were neutral; 14% (2) were near to agree, 29% (4) were agree and 21% (3) were strongly agree. Similarly, the iPad results

A Comparative Evaluation of Usability for the iPhone and iPad