IN
DEGREE PROJECT
COMPUTER SCIENCE AND ENGINEERING,
SECOND CYCLE, 30 CREDITS
,
STOCKHOLM SWEDEN 2017
Usability factors for event
platform users
IDA VAINIONPÄÄ
KTH ROYAL INSTITUTE OF TECHNOLOGY
Usability factors for event platform users
Ida Vainionpää
idava@kth.se
Master’s Thesis at the School of Computer Science
and Communication, KTH
Human Computer Interaction and
Master’s Program in Computer Science
Supervisor: Helena Tobiasson
Examiner: Jan Gulliksen
Principal: Confetti Events
Provided by Confetti Technology AB
Usability factors for event platform users
ABSTRACT
Användbarhetsfaktorer för användare av
eventplattformar
SAMMANFATTNING
Usability factors for event platform users
Ida Vainionpää
KTH Royal Institute of Technology Stockholm, Sweden
idava@kth.se
ABSTRACT
This thesis examines which functions motivate experienced event marketing platform users to continue to use a specific event platform with regards to usability.
The event marketing platform service Confetti was chosen as principal. Qualitative information was gathered through usability tests with existing users of Confetti and quantita-tive information was collected by sending out a questionnaire to existing Confetti users. A contextual analysis was then performed on the compiled material.
It was found during the study that the most highly re-garded feature was customizability and that inconsistent feedback was the main source of friction experienced by the users. The main entry point for users of Confetti was via recommendations, and users with a free subscription would consider a payment model if they arranged more events in general. Lastly, the simplicity and the ease of use of the Confetti platform was highly appreciated by the majority of the contacted users.
A suggestion for future research in this area would be to conduct similar studies with event marketing platforms similar to Confetti in order to get a broader perspective on the usability demands as a whole.
Keywords
Event platform, event site, questionnaire, usability test, us-ability
1.
INTRODUCTION
In recent years the market for event marketing platforms has increased. These platforms serve the purpose of creating and managing custom event web sites, where the adminis-trator may (among other things) provide event information, spread the information to a wide or selected group of people and manage the sale of tickets. Many such platforms exist today and in this fiercely competitive market it is key to stay ahead of your competitors if your platform is to succeed.
.
The well trodden strategy of heavy marketing is of course an ever present option to quickly (and possibly expensively) get a large user base. However, marketing can only go so far and because users are presented with multiple platform options there exists a need to extinguish one’s platform by providing better functionality sets, ease of use, customer support etc. This is why the usability of one’s platform is such an important factor for success. This matter is dis-cussed by author S.Smith-Atakan, in his book on Human-Computer Interaction, HCI, where he argues that a product is considered good if it consists of good technology, usability and marketing [23].
1.1
Usability
D.A.Norman [19] discusses in his book, The Design of Ev-eryday Things the many challenges that occur when it comes to designing the most obvious items, for example a kitchen stove, and how these designs effect the people using them, every single day. This indicates that design needs careful thought and consideration, not only for the devices in our homes but also the designs for the web. From the perspec-tive of a Marketing and Sales team, designing for the web could arguably be considered to be of even greater impor-tance because of the users’ always present ability of easily leaving a service and choosing another service. Thus, keep-ing users pleased durkeep-ing their whole experience on the web is desirable, which is also touched upon by author S.Smith-Atakan [23].
It is easier for users today to have negative opinions about just about any product on the web due to the vast amount of different available products. This makes it harder to sat-isfy a user in all different manners, which is why, according to the author to an article in the Magazine Interactions, it is important to provide the user with pleasure from their experience with a product, for their return in the future [18]. When users return to a service it is probable for them to spread the word, knowingly or unknowingly, by word of mouth, thus promoting an organization or company to oth-ers. When it comes to user expansion this is an excellent way to increase the number of consumers, which again reminds us to create enjoyable products, for the word of mouth to work as intended.
1.2
Evaluating usability
fleet-ing concept by nature in a way that it isn’t always trivial to evaluate it, seen from a resource perspective consider-ing the time spent on the testconsider-ing and afterwards evaluatconsider-ing the users’ feedback, behaviour and thoughts. There exists a multitude of evaluation strategies but two common ones that were implemented in this thesis are usability testing and common questionnaires.
A questionnaire is a simple tool for making a quantita-tive analysis targeting a broad range of users, with easily quantified measure points but with the drawback of lacking human-to-human interaction and the fact that respondents may not always be correct in their responses [22].
Usability tests is a complementing technique for making a qualitative analysis targeting a narrower range of users, with the ability to observe and interact with users directly and draw conclusions from the results [17].
Naturally, a mix of both strategies can be used to make the one strategy cover up for the other’s weak spots if one or the other isn’t sufficient by itself.
2.
RELATED RESEARCH/BACKGROUND
In order for a proper exploration of the related research considering usability in an event marketing platform to be reached, it is of importance to take part of research regarding designing for the web, user recurrence and user experience.2.1
Designing for the Web
Keeping the user satisfied is key when creating an en-joyable product, which in the case of an event service in-cludes the selection of functions offered, pricing, required skills, the learning curve and customer support, to name a few. Costa discusses the matter of keeping users satisfied and concludes that web page aesthetics attract users the most, more precisely the aesthetics include simplicity, con-sistency and clarity of design [5], which translates to the above mentioned requirements for keeping a user satisfied in an event service. D.A.Norman [19] claims on page 3 in the The Design of Everyday Things that “Two of the most important characteristics of good design are discoverability and understanding.”, which translates to, firstly, the user needs to work out which actions can be performed and how they are to be performed. Secondly, the user needs to try to understand what the design means and why it is available. For this to work affordances and signifiers are needed [19], explained on page 14 by D.A.Norman as: “Affordances de-termine what actions are possible. Signifiers communicate where the action should take place. We need both.” The af-fordances and signifiers thus help by directing the users to find the answer to their desired question or action.
S.Krug [12] introduces the concept of Do-It-Yourself Us-ability Testing in chapter 1 in his non-research book Rocket Surgery Made Easy, which described in short, is a method of getting insights to a web service. This, consisting of per-forming ”do-it-yourself”-tests on users while, from another room, being observed by the product’s stakeholders, devel-opers and other interested parties, whom later decide on which issues to lay focus on. S.Krug discusses the fact that these tests are important because of the improvements that can be made in a short amount of time, and how significant the effects are for a working project. The usability improved can strikingly be compared to projects that only make a sin-gle testing session by the end of the project. Taking part of these continuous test sessions give the creators a lot of
feed-back during the course of the project. D.Tasse, A.Ankolekar and J.Hailpern show in their research study that an impor-tant aspect of getting users enjoying their experience is good feedback based on updates on the web page [24].
2.2
User Recurrence
Although the research on recurrent users is limited it is arguable to say that since design has a considerable effect on the users of a web service, the same reason is just as important considering getting recurrent users. If the user gets an enjoyable first experience they are more likely to return [16], the probability of this increases if the episodic memory holds a positive recollection of the experience [10]. A study conducted by G.Lindgaard has concluded on page 115 that: “... visual appeal can be assessed within 50 ms, suggesting that web designers have about 50 ms to make a good first impression” thus expressing the importance of a pleasing first impression [15].
Studies done on user revisitation show that users return to previously visited web pages from 50% [7], [14] up to 80% [4] of the time spent surfing online. A study done on user’s browser behaviour also show that it is common for people to return to previously visited web pages [20, 21]. It could therefore be considered quite common to have onetime users of a service returning to the same service. Another indicator of interest from the user’s point of view is the time spent on the web page, and, the longer time spent on a certain service, the more likely the user is to enjoy it [25].
Other than keeping users satisfied with the product, it is important to get in touch with first-time visitors in the be-ginning of their experience to offer possibilities for feedback and help, which increases their retention and engagement [3].
A viewpoint presented by H.Banati, P.Bedi and P.S.Grover [1] is the connection between good usability and trust and reputation. These factors of trust and reputation, initially serve the purpose of keeping the user engaged in her first use of the site, while later providing a reason to revisit and reuse the site as an effective motivator for retention. They further argue that a satisfied user is more likely to spread their personal trust of the site to others, in effect creating a positive feedback loop of increasing trust.
2.3
Principal & Similar Services
The current market offering event services is wide. The event marketing platform Confetti Events1 was chosen to be the principal during the production of this thesis. The choice was made mainly because Confetti is a minor player on the event marketing platform market and thus have a need to differentiate themselves through excellent usability, making them a good target for this thesis’ usability studies. Confetti offers a platform to create event sites, where users may provide event information, manage attendees, sell tick-ets, without any prior design or programming skills needed. They offer services for free, with a subscription or a pay per event purchase model. Because of the wide selection the users can be selective in their choice of service, it is there-fore interesting to research how users got in contact with Confetti from the start. Similar services to the one Confetti offers are:
1
• Lyyti2, • Splashthat3, • Eventbrite4, • Picatic5 • and Billetto6.
2.4
UX
User Experience (UX) is defined as: “Person’s perceptions and responses resulting from the use and/or anticipated use of a product, system or service.” According to [9]. This stands in contrast to the concept of usability in the sense that UX focuses on the emotional and perceptive side of a service’s usage, while usability focuses on the ease of which a user may use and understand a service.
According to a broad, longitudinal study of UX profession-als globally by C.Lallemanda, G.Groniera and V.Koenigb [13] no real consensus was found on whether UX should be approached qualitatively or quantitatively, which is inter-preted here as there being a point in performing a mix of both usability testing and questionnaires to get a broader understanding of a product’s usability and UX.
2.5
Questionnaire reliability
Comparing mail and telephone surveys to web surveys it was found that web surveys receive a response rate approx-imately 10 % lower than that of mail and telephone surveys [6]. High response rates are preferable and as Baruch and Holtom [2] discusses they believe that email surveys will prevail in the future which hopefully will generate higher response rates for web surveys and therefore increase the probability to draw general conclusions from these types of surveys.
2.6
Purpose
The purpose of this thesis is to explore perspectives to the question: what are the main functions motivating ex-perienced event platform users to continue using a specific event platform?
3.
METHOD
In order to gain material to explore the possibilities to answering the research question, which includes finding as-pects to understanding how users of Confetti experience the different functions of the event site, a questionnaire was sent out to 250 existing users of Confetti of which 34 replied and usability testing has been performed with 10 existing users of Confetti.
3.1
Questionnaire
A questionnaire was compiled and before it was sent out it was reviewed together with Confetti. Attention was given to ensure that responses to the questionnaire were likely to answer the general questions of: how the user is using Confetti today, how pleased they are with the product and what features they value most.
2 lyyti.com 3splashthat.com 4 eventbrite.com 5picatic.com 6 billetto.se
The questionnaire was sent out by Confetti in two different batches, one to users of Confetti with a free subscription and one to users with a paying subscription and answers were collected in both English and Swedish. One section of the form involved a description regarding the upcoming usability testing and one could hand in an email address if they were interested in taking part of the testing. The questionnaire may be found in its full form in Appendix B.1 and previous to the questions the respondents landed on a title page explaining the purpose of the questionnaire.
Some replies were found in the near days and later on Confetti chose to send out a push for users to answer the questionnaire, where they could get a 20% discount on a yearly subscription if they handed in the form no later than 6th of April 2017.
3.2
Usability testing
Some participants had showed interest in taking part of the usability testing through the questionnaire, by leaving their email address. They were contacted via email, from a Confetti domain email, together with other users of Con-fetti. To get the target group as mixed as possible the goal was to find participants from many different industries. The participants were given an option of time slots and as com-pensation Confetti offered to buy lunch of coffee for the par-ticipants. To make the testing as easy as possible for the participants, they each chose their own venue.
Before any usability testing started a pilot study was con-ducted, which showed that some questions needed clarifica-tions. In the end the choice to remove two questions was made due to their structure and the fact that they did not fit the context of the rest of the test. Another benefit of this was that it shortened the length of the test, which took over one hour to complete and it was desirable to complete it under the hour.
All in all 10 user tests have been conducted with 10 partic-ipants who all use Confetti in different ways. The user tests were recorded with combined audio and screen-capture and were later on transcribed since the participants were using the think aloud protocol during the testing.
The layout of the test was that only two people were present, the participant and the test-leader. The test was performed on a laptop and of the 10 tests conducted, 9 were in Swedish, one in English, with 9 in person and one over Skype. Some of the 10 participants had previous to the test also answered the questionnaire. The test started with the participant getting an explanation of the agenda of the test, which consisted of five parts:
1. introduction
2. thoughts on the Confetti landing page
3. a practical part where the participant got assignments to perform
4. a task built on working with paper where the partici-pant got to rank possible functions to add to the Con-fetti event site
5. and lastly a closing and summing up part
methodological point of view the ranking assignment is con-sidered an independent method but was during this thesis included in the usability testing to receive even more guid-ance of the users’ opinions.
The results were derived through content analysis, which in short is a way to analyze and process collected text ma-terial [11].
4.
RESULTS
The results from the questionnaire and the usability test-ing will be presented separately to make it easier to review.
4.1
Questionnaire
The questionnaire got 34 responses, from a broad spec-trum of people, ranging from the self-employed and start-up entrepreneurs to employees at larger companies with more than 200 employees. The majority of them work in the IT sector, with table 1 showing their previous experience in us-ing the Confetti platform and table 2 showus-ing the type of events they organize mainly. 18 of the respondents, 53%, had used Confetti between two to five times, 8 have used it once, one of the respondents had never used Confetti, but had registered an account and the rest have used Con-fetti five times and above. This selection of respondents was chosen to provide a broad range of previous experiences in Confetti as well as a broad range in ages, based on the ques-tionnaire being sent to people of various age, even though the respondents did not specify their age. By the respon-dents, the most popular events to arrange were seminars, conferences and workshops.
Table 1. How many times have you used Confetti Times 1 2-5 5-9 10 or more never used Confetti
Responses 8 18 3 4 1
Table 2. What type of events are you organizing? Options Responses Seminars 16 Conferences 15 Workshops 15 Marketing events 9 Parties 8 Courses 6 Internal meetings 6 Festivals 2 Weddings 2 Clubs 1 Other 2
As mentioned in the method section the questionnaire was sent out both to users with a free subscription and to users with a paying subscription. Users with a free subscription were asked what would make them upgrade to a yearly sub-scription, and the response showed that 38% would consider upgrading if they planned events more regularly.
The focus of the questionnaire was to find answers to the questions: what got you using Confetti from the start, the most important feature, if there is something missing in the tool and something to ease the user to create event sites. The respondents had choices to include free text of thoughts, and
some of them will be included. The respondents were asked to rate certain topics that they find key in their choice of event platform: how easy it is to create an event site, how pleased they are with the general experience of Confetti, over-all how satisfied they are with Confetti, how likely they are to use Confetti again and, lastly, asking how likely they are to recommend Confetti to others. Since the questions were not mandatory, some of the respondents did not reply to all questions asked. The questions were marked as optional to increase the probability of receiving a higher answer rate, in the sense that more replies could be received if not every question is marked as required. There could in some cases be difficulty or other issues hindering a respondent from an-swering a certain question and by leaving all questions op-tional this prevents this matter from occurring. Of the 25 questions asked there were 10 questions that did not receive a full response rate. The majority of these 10 questions were missing 1 to 3 replies, some were missing up to 8 replies but the most notable part was that two questions were missing 17 replies and these questions were How would you compare the quality of our products to that of our competitors? and Is there anything that would make it easier to create your event site?.
4.1.1
Questions
The respondents were asked in multiple-choice questions what initially lead them to start using Confetti. Table 3 shows that 53% were recommended to use Confetti by a friend or colleague and 18% started using Confetti after sign-ing up for an event.
Table 3. What got you using Confetti from the start? Options Responses Recommendation by friend/coworker 18 Heard about Confetti and wanted to check it out 11 Previously used Confetti to sign-up for an event 6
Searched online 3 Advertisement online 1
Other 1
It was found that the most valued factors of using Confetti were the intuitive design and ease of use, with 13 and 11 responses respectively. Some quotes from the respondents were:
”The whole experience of making something that looks good.” - Respondent A
”Most important feature is the sign up button.” -Respondent G
”An easy way of communicating with my guests.” - Respondent I
”A reminder function for people who have not opened the email.” - Respondent B
”Loading up images can lead to bugs like it up-loads multiple times or just crashes all together.” - Respondent D
”I’d like to add my own font.” - Respondent H
When the respondents gave their feedback to the question Is there anything that would make it easier to create your event site? many answers were left empty, a number of respondents had typed in ”No” and a few found creating an event site very straight forward. Some quotes to include ideas from the respondents:
”It is kind of hard to find where to set up different things.” - Respondent C
”I have had some trouble understanding the sign up form and there are a few too many sub cate-gories maybe.” - Respondent E
”More drag and drop.” - Respondent F
4.1.2
Ratings
Results represented in a table correspond to questions the respondents were asked to rate from one to five, and results represented in diagrams answer to an opinion scale, with the same scale. The rating included visual feedback for the user in the shape of a progress bar and the opinion scale had an explanatory text for the steps and also emphasized the chosen number on the scale.
50% of the respondents rated How easy is it to create an event site? as a 5, followed by 29% rating it as a 4. No respondent rated the function as a 1, and the results deviate somewhat as one respondent rated it as a 2. All together 79% had rated 4 or 5, so a conclusion of it being fairly easy to create event sites, can be made and the ratings can be seen in table 4.
Table 4. How easy is it to create an event site? Ratings 1 2 3 4 5
Answers 0 1 5 10 17
Asking about the general experience of Confetti most of the respondents agreed by rating it as a 4, more exactly 47% did and 26% rated it as a 5. Again over 70% of the respondents chose the number 4 or 5 and the results deviate with two respondents rating 1 respective 2, as seen in table 5.
Table 5. How pleased are you with the general experi-ence of Confetti?
Ratings 1 2 3 4 5 Answers 1 1 7 16 9
Questions Overall, how satisfied are you with Confetti? and And how likely are you to use Confetti again? received a lot of votes for 4 and 5, namely 17 and 11 respectively. Ratings 1 and 2 received one vote each, and as the results shown in figure 1 it is clear that one respondent failed to answer. Figure 2 shows 21 respondents, 62%, voting 5, thus
saying they are very likely to use Confetti again, followed by 24% voting 4. 86% of the respondents indicate that they are very likely to use Confetti again. The average for how satisfied people were overall is 4,10 and 4,36 for how likely they are to use Confetti again.
1 2 3 4 5 0
5 10 15
from not satisfied to very satisfied
n u m b er o f ra ti n g s
Figure 1. Overall, how satisfied are you with Confetti?
1 2 3 4 5 0 5 10 15 20
from not likely to very likely
n u m b er o f ra ti n g s
Figure 2. And how likely are you to use Confetti again? Lastly the question How likely are you to recommend Con-fetti to others? received an average of 4,18. One respondent rated this feature as a 1, 6 respondents as a 3, 12 respon-dents as a 4 and 15 responrespon-dents rated it as a 5. 79% rated 4 or higher which can be thought of as a good review, the ratings are seen in figure 3.
1 2 3 4 5 0
5 10 15
from not likely to very likely
n u m b er o f ra ti n g ss
Figure 3. How likely are you to recommend Confetti to others?
The usability study had 10 participants, 6 of them female and 4 of them male, whom all had used the Confetti service at some point. They came from various backgrounds con-cerning their working industry, which also was the intended target group. Some of these 10 had also answered the ques-tionnaire. Because the questionnaire was anonymous if the respondent chose not to leave their email address, it is un-known if all participants of the usability test had previously answered the questionnaire.
The participants of the usability test were in the introduc-tory part asked what got you using Confetti from the start?, 80% of them gave the answer that Confetti is an easy tool to use and many also pointed out that with Confetti you are able to create beautiful event sites. Other received com-ments were: quick and intuitive tool, visually pleasing and a simple way to create professionally looking event sites. Some participants got in touch with Confetti by signing up for an event as a visitor and cared for the setup. Others were drawn to Confetti via word of mouth.
The introductory part was followed by a part where the participant got to describe the Confetti landing page, which quickly followed by the substantial third phase, the practical part, which without exception gave a lot of useful feedback. Reviewing the material of the transcriptions it is rather safe to say that the users all managed to perform the test. The users had different backgrounds based on their work of industry which is why they use it for different occasions, thus having varied experience using the different functions and therefore finding some of them easy to use and others more difficult, while another user found them just the opposite.
Below the results from the practical phase and the ranking assignment will be presented. The material considering the landing page were not included as well as some of the assign-ments from the practical part, due to similar questions being presented and the results from the left out questions would not yield any further information to draw the conclusions.
4.2.1
Practical phase
During the practical phase the users received 16 assign-ments, one at a time. They were encouraged to think aloud and comment on missing features or clarifying parts and im-provements throughout the whole test. Besides logging onto the service the users were asked to create an event site for an upcoming happening at their work and were able to freely write and design the site as they preferred. After the initial stage of creating their site they automatically landed on the content page, where they were asked to either explain their next steps on that page or actually perform them by writing and filling in content to their site. Some users, more exactly 40% of the users experienced that the ”register now” button was left empty. By reviewing the recorded screen material it has been clarified that this occurred if the user had clicked on the placeholder text under ”Button title” but not added any text. Figure 4 shows the placeholder text for the tagline and button title. The users were able to continue adding text to the tagline without the placeholder text disappearing, but if the users entered a button title but decided to remove it, the feedback to the user was misleading. The placeholder text with ”Register now” appeared in the button title and when the user saved, the placeholder text did not follow.
A user had pressed the button title field but not added any text and then saved. They continued on to the actual event site where they searched for the register now button
Figure 4. Showing the placeholder text from the content page
but could not find it.
”I can’t see where the text for the button would be linked to the event site, so I’ll try to type in ”Hello” to see if it will make a change. It’s in-teresting that it now appears, I don’t really un-derstand why but at least now it’s there.” - User A
The user did not find any feedback to why it in their opinion suddenly started to work, which is why the creating of event sites and more specifically the greyed areas and placeholders can appear somewhat inconsistent.
When the users were asked to upload images to their gallery for their created event 10/10 participants searched for the gallery under the heading content. Below the reader finds two quotes from two participants.
”The gallery. I’d say that’s part of content.” -User G
”I always look for the design option in content. I understand that it’s found in settings but if you’re not into coding you feel that design is part of the content on your site. At least that’s how I feel.” - User C
All the users eventually found the design option in the set-tings heading, for some it took a while and the more events they had created the easier it was for them to find it un-der settings. Another user explains their thoughts in the following quote.
”It feels like content and design are quite con-nected so I got a bit confused at first when it was time to upload images, because I couldn’t find this under content. For me it feels intuitive to find image upload in content, because it’s an image on my site. But content here is more just text. On the other hand you can also upload images here, for organizers and sponsors so it’s a bit confus-ing.” - User E
The user explained that some of the confusion for them lied in the possibility to upload images for organizers and sponsors to the event in the content section, which is why their intuition was to go to content to upload images to their event.
”This was a bit odd. Now the same image appears 4 times although I only added one image.” - User F
The user added one image and something failed to make it appear four times instead of once. Other users experienced similar difficulties, for example, removing an image from the gallery which appeared in the editor again after saving. Or, adding two images and removing one followed by adding one more and receiving a notification about the gallery being full even though it only had five images with the maximum being six. Below is a quote from an experienced user with their solution to the problem.
”I’ve experienced this before. Maybe you’re just too quick when uploading or maybe it just takes a long time for the upload to complete. Usually I solve this by refreshing the site and by trying again and that usually solves it.” - User C
60% of the participants had some sort of issues when up-loading the images, all of them relating to images either appearing too often or in the wrong places or not appearing at all or not in the expected position. There were no differ-ences seen from a gender perspective since 50% of the 60% were male and the other half female.
From an observational point of view the results for the following assignments such as adding organizers, creating a sign-up form, changing location and time and exploring the email functions went all together as planned.
4.2.2
Improvements
The users were throughout the whole test able to com-ment on features they felt were missing, features that in their mind needed improving or if they felt that something was out of place. Again, some users brought up the flexi-bility of the event site, and more specifically asked to: edit the amount of register buttons, change the placement of the logo, edit the placement of the map function, broader for-matting functionality, for example, using underline in the descriptions or removing signing up through Facebook.
Users answering if there is anything else they would like to add:
”Yes, probably the possibility to have a question-naire or a poll on the site where people could vote. Or even not here on the site. But lets say when you register, a pop up opens and you’re able to vote. Sometimes we have issues when organiz-ing a conference, because we don’t know when the best date is. So we could propose it here and peo-ple could vote for a preferred date when they sign up. And then you have the results as well. That’s something.” - User D
”That the organized by images are round. We often encounter squared images and it would be practical for us if we could choose to have round or squared images.” - User B
”Sometimes the save button doesn’t follow when you’re scrolling the page, so it’s easy to forget to save. But when it is seen in the upper corner there’s no problem remembering to save.” - User C
4.2.3
Ranking assignment
The results of the ranking of the different functions during the usability test can be found in Appendix A.1 where the reader can find the rankings of each of the 12 different not already existing functions. To create an overview of these functions a list has been created, found in list 4.2.3. The list is based on a counting system where the position a certain function had in the ranking is used as a representation of its position, and these positions were summarized by adding all the digits to one. The list was created by putting the func-tion with the lowest digit on posifunc-tion one, followed by the function with the next to lowest digit etc. Because the par-ticipants had the possibility to remove unwanted functions from their ranked list, these functions were given a number 12 in the summarizing part, to make all the rejected func-tions equally valued, since the rejected funcfunc-tions were not ordered by the user. This list is found in list 4.2.3 and the summarized digit is found in the parenthesis of each row and in figure 5 the users are seen working on this assignment.
List 4.2.3: Results from the ranking assignment regarding the different features
1. Drag and drop sections on the event page with a pos-sibility to add your own blocks (32)
2. Add your own fonts (36)
3. Preview changes to the event page before publishing them (43)
4. Add Schedule (57)
5. Print name tags directly from the web page (61)
6. Track how people found your event page, example 30% via Facebook, 50% via Google etc. (67)
7. Send text messages to all registered participants of a certain event (78)
8. Widget on your web page (79)
9. Add tabs (84)
10. Photo editing (86)
11. Buy a domain to your event directly from the Confetti web page (91)
12. Register a +1 to a free event without having to specify their name (94)
Figure 5. Users performing the ranking assignment 1 3 5 7 9 11 rejected 0 1 2 3
Figure 6. Add your own fonts
1 3 5 7 9 11 rejected 0
1 2 3
Figure 7. Drag and drop sections on the event page with a possibility to add your own blocks
Functions such as registering a +1 to a free event without needing to hand in their name to the event accommodator was in many cases chosen at the bottom of the ranked list, and also rejected in 3 cases, which is seen in figure 8. In addition to this, the possibility to buy a domain to an event site from the Confetti web page was also ranked low by most of the participants, as seen in figure 9.
1 3 5 7 9 11 rejected 0
1 2 3
Figure 8. Register a +1 to a free event without having to specify their name
1 3 5 7 9 11 rejected 0 1 2 3 4
Figure 9. Buy a domain to your event directly from the Confetti web page
The function for previewing changes of the event site be-fore publishing them, was chosen in the top three in the ranking list 60% of the times and, of these 60%, 40% cor-responded for the function being chosen in first place. The function was despite its high rankings put in third place because the participants of the usability test clearly had dif-ferent opinions where some ranked it in eight place and even ninth, and one user rejected it, which is shown in figure 10.
1 3 5 7 9 11 rejected 0 1 2 3 4
Figure 10. Preview changes to the event page before publishing them
Considering the gender aspect there were no indications that a certain function was ranked higher or lower by a cer-tain gender.
5.
DISCUSSION
able to find which factors are motivating them the most. After reviewing the results there were three main aspects to consider improving from the usability tests. These are consistency, user feedback and adding more flexibility. The results from the questionnaire were consistent in telling that the respondents were in general very pleased with the cur-rent appearance and functionality of the Confetti web page.
5.1
Differences questionnaire vs. usability test
It is worth noting that participants of the user test of-ten gave more specific answers than the respondents of the questionnaire, presumably due to the participants using the event platform to create an event site while answering ques-tions, compared to the respondents who were filling out a form. And maybe doing so without looking at the web page at all.Users from the usability test experienced some inconsis-tency issues while creating their event site, which was not mentioned by the respondents of the questionnaire.
Questions regarding factors to upgrade to a yearly sub-scription were only addressed in the questionnaire.
5.2
Similarities questionnaire vs. usability test
Many contacted users had come in contact with Con-fetti via word of mouth, which is by H.Banati, P.Bedi and P.S.Grover [1] related to trust and understanding, due to users being more likely to spread the word if they are pleased with a service. Understanding in a service could, in the case of the questionnaire, be linked to the question of how easy it was to create an event site, which received ratings four or higher in 79% of the cases. The respondents gave an av-erage 4,18 rating, on a scale from one to five, when asked how likely they were to recommend Confetti to others, and an average of 4,10 when asked how pleased they were with Confetti. These ratings strengthen the likelihood of users spreading the word when pleased, which as mentioned was, today, the most common way to becoming a user of Confetti. Some confusion between which categories to find under the heading content and the heading settings were found in both the questionnaire and during the usability tests. The design category was found under settings and 9/10 users searched for the category under content before realizing it was found under settings. Many users commented on this as something happening most of the times using Confetti and while trying to find the image upload, which is found under the category design. Many explained that it is logical to find this under content, since it is something that clearly appears on their event site and other parts such as event information and heading are edited under content.Besides this confusion of finding the design category, the discoverability of the site has proven to be satisfactory through-out the usability tests, and replies from the questionnaire have had overwhelming responses of the intuitive design and ease of use. As D.A.Norman [19] discusses, discoverability is key when it comes to which characteristics to focus on in a good design. The other key factor is understanding, which is also applicable on respondents describing the creating of an event site as very straight forward. According to [19] these characteristics work as intended in the care of affor-dances and signifiers. Thus helping the user understanding what functionality is available and afterwards applying this to understanding the design and its purpose.
Another similarity was that many respondents and users
were interested in finding it possible to have more flexibility when creating the event site. This feature would make it possible to have even more personalized events, and when creating the same type of events on a regular basis the users found it difficult to create varied event sites. An example of what kind of flexibility to include was to be able to drag and drop sections on the event site, this feature alone could enhance the feeling of two similar events looking different. The ranking assignment from the user test had the Drag and drop sections on the event page with a possibility to add your own blocks function in first place in the ranking list and was also in top three on the users’ lists in 60% of the times.
Overall all participants have showed a clear liking to the design of Confetti.
5.3
Method
Two research methods were used in order to gather the material necessary to draw the conclusions made in this thesis. This was to ensure both a longitudinal perspec-tive through the questionnaire and a qualitaperspec-tive perspecperspec-tive through the user tests.
One obvious drawback of only studying Confetti’s existing user base is that information on the users of similar tools, as well as users that have yet not chosen which platform to use, was not included in this thesis. One must therefore be more speculative in the attempt to draw general conclusions outside of the scope of Confetti’s platform.
From a research perspective the number of participants and respondents in the usability tests and the questionnaire (10 and 34 resp.) are too few to draw general conclusions and this must be kept in mind when reviewing the results. However, given the time constraint for this thesis project, 10 participants is a realistic amount to study within a rea-sonable time frame. Regarding the number of questionnaire respondents, this was the number of who offered to volun-teer and here as well the time constraints limited the ability to find more respondents.
On a similar note, it is worth noting that roughly 14% of the contacted users for the questionnaire volunteered. It is unknown if there may be some similarities between these users that do not apply to the remaining 86% who did not volunteer. This opens up the possibility for a certain bias between the respondents. Regarding questionnaire reliabil-ity the low answer rate must again be considered if further evaluation on the results would be made. Another factor is the optional questions, if they had any impact on the an-swer rate or not, if required questions could have increased the answer rate and not left any questions unanswered or if the response rate would have decreased if the questions were required.
have required an opinion from the respondents.
Lastly, as always, the quality of the information attained through both usability testing and questionnaires might ar-guably not give the full picture of users’ thoughts and opin-ions but in hindsight the chosen methods still gave valuable input to explore the research question.
5.4
Results
The results show that users of Confetti were all in all very pleased with the service, which was due to the site being easy to use and easy to understand. By the definition of usability [8] and the users being satisfied it arguable to say that the service holds great usability factors and therefore the users achieve their specified goals in time and in an efficient way, which is a reason for the users continuing to use the service with a pleasing tone. The word of mouth was the main entry point for users of Confetti and because of the good usability conditions it is probable that existing users because of this reason feel the urge to spread the word to others, as well as wanting people in their vicinity to experience the same easy to use service.
Another aspect to consider is the UX of the site and users have described the site as a site with a good design, which is an indicator for the service also focusing on the UX and doing so in a successful way.
That users have expressed the need and want to customize their event sites even more is understandable. If a user ar-ranges many events with the same type of character it is desirable to make the event sites deviate from the previous ones. This could generate more attendees in a sense that the design of the site could reach more people because of curiosity but still keep the regular users. An example of a highly appreciated function in the customizability depart-ment is the drag and drop sections on the event site, which is a simple yet effective way to create a new feel and touch to an event site and is again an example of where the UX of the site generates appreciative functions and therefore increases the users’ satisfaction.
A factor to consider when signing up to use Confetti is that an account is needed, which includes sharing personal information such as name and email and could in some cases reject potential users. On the other hand nowadays most on-line services include sharing personal information and in the case a user willingly shares more information e.g. bank in-formation, they are likely doing so to receive even greater functionalities e.g. in this case being able to receive pay-ments from attendees.
The results presented and gathered from this thesis are specific for one event platform but it is arguable that the conclusions can be used by other services, perhaps quite dif-ficult in a direct way, but valuable when considering new functionality, upgrades or design and layout. An indication toward the service having good UX and usability factors has been made, which is desirable for other services as well in order to reach a higher customer satisfaction rate.
This thesis has shown how important usability is for web services, which is widely discussed by S.Krug [12]. It has also shown that first impressions could determine whether a user becomes a recurring user or not, also argued in the article by [15], which is a matter that might not be obvious from a start but actually is quite crucial for success. It is thus interesting for people making design decisions to take part of these conclusions in order to get insight into what
aspects and functions are highly valuable for users.
5.5
Sustainability & societal aspects
Due to the nature of event marketing platforms and their low impact on the environment it has been difficult to con-nect this thesis with sustainability. Therefore great care has been taken to ensure that the test subjects and re-spondents have been diversified with different ages, genders, backgrounds etc. By doing this the goal has been to re-ceive input from people of many parts of society, which is important if one wants to promote diversity.
Sending invitations to events has been improved from an ecological point of view because no physical invitation is needed when using event platforms, since they are sent elec-tronically. All attendee information can be gathered online and no paper or prints are needed. Instead servers are used to store information, which has an impact on the environ-ment but as a whole the servers’ effect on the environenviron-ment is lesser than e.g. printing invitations.
Considering an ethical perspective and the social sustain-ability, some people could have issues using the event plat-form due to accessibility limitations, no such candidate took part of this study because nobody was found and therefore this possible limitation has not been analyzed. It is possible to use the platform with more than one person which could be a solution for being able to take part of the service even with some limitations.
No negative societal economical effects could be identified in connection to this thesis due to the company efficiently using their assets and therefore functioning in a profitable way.
5.6
Future research
Future studies can be carried out by users using a com-peting tool for getting insights to why they chose another platform over Confetti. This could also generate valuable information for Confetti, to find out if there are some signif-icant improvement or additional functionality they are cur-rently missing in their service.
Another similar approach is to conduct the same study on non-users, that is, people who could benefit from using an event platform but have currently chosen not to. An out-come of that could be to better understand how, for example a first-time user would become a recurrent user of an event platform.
A follow-up study could also be made in the event that added customizability has been implemented, which can lead to valuable feedback from the start of these new features.
Yet another approach for the future could be to conduct the same study with a target group that mostly work with weddings and clubs etc. As seen in table 2 most of the respondents use Confetti for work related events and a com-parison of the results from a new study with the results from this study could yield input on if the same conclusions regarding usability and UX would stand.
6.
CONCLUSION
The following conclusions are derived from the study of Confetti’s users, but it is reasonable to believe that they can be applied to similar event platforms as well.
Customizability of an event site is the most sought after feature by far. Whether this is something that should actu-ally be implemented is a decision that needs to be made on a case-by-case basis. It is arguably safe to assume though, that it is something appreciated by users.
Adding to this, it is also desirable for the customization to have a high degree of usability. Especially when the number of customization options grow increasingly complex.
6.2
Feedback consistency
It was found that inconsistent feedback was the main source of friction experienced by the users during the user tests.
6.3
Free vs. Paying users
Most non-paying users claimed that they would upgrade to the subscription model, if, and only if, they had more events per year. If Confetti wants to acquire more paying users and thus generating greater revenue, one method of doing this would be to try to make users create more events per user.
6.4
Entry point
When asked how users first started using Confetti the overwhelming majority claimed that it was due to recom-mendations from friends and colleagues. Since this seems to be the main entry point for Confetti’s user base its reason-able to argue that the recommendation method is something to pursue in the future in new user acquisition.
6.5
Simplicity
It is also worth mentioning that the simplicity and ease of use was something appreciated by virtually all contacted users.
7.
ACKNOWLEDGMENTS
This paper was conducted as a master thesis project, su-pervised by Helena Tobiasson and examined by Professor Jan Gulliksen. Thank you Helena for your guidance, it has been very helpful. I would like to thank Johannes Edel-stam for giving me the opportunity to write my master the-sis at Confetti and introducing me to the rest of the team. Thank you to you all Martina Elm, Jonny Str¨omberg and Johannes Edelstam and Confetti for all the great support and exchange of ideas during this whole project.
8.
REFERENCES
[1] H. Banati, P. Bedi, and P. S. Grover. Evaluating web usability from the user’s perspective. Journal of Computer Science, 2(4):314–317, April 2006. [2] Y. Baruch and B. C. Holtom. Survey response rate
levels and trends in organizational research. Human Relations, 61(8):1139–1160, August 2008.
[3] G. L. Ciampaglia and D. Taraborelli. Moodbar: Increasing new user retention in wikipedia through lightweight socialization. Proceeding CSCW ’15 Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work Social Computing, pages 734–742, March 2015.
[4] A. Cockburn and B. McKenzie. What do web users do? an empirical analysis of web use. International Journal of Human-Computer Studies, 54(6):903–922, June 2001.
[5] P. Costa. Evaluating web site design. ACM Proceedings SIGDOC ’08, 26:265–266, September 2008.
[6] W. Fan and Z. Yan. Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior, 26(2):132–139, March 2010. [7] E. Herder. Characterizations of user web revisit
behavior. In Workshop on Adaptivity and User Modeling in Interactive Systems ABIS05, pages 32–37, 2005.
[8] Ergonomic requirements for office work with visual display terminals (vdts) - part 11: Guidance on usability. Standard, 1998.
[9] Ergonomics of human-system interaction - part 210: Human-centred design for interactive systems. Standard, 2010.
[10] L. Jin, G. Liu, C. Wang, and L. Feng. Personal web revisitation by context and content keywords with relevance feedback. IEEE Transactions on Knowledge and Data Engineering, (99), February 2017.
[11] K. Krippendorff. Content Analysis: An Introduction to Its Methodology, Second Edition. Sage Publications, Thousand Oaks, London, New Delhi, 2004.
[12] S. Krug. Rocket Surgery Made Easy: The
Do-It-Yourself Guide to Finding and Fixing Usability Problems. New Riders, California, chapter 1, 2010. [13] C. Lallemanda, G. Guillaume, and K. Vincent. User
experience: A concept without consensus? exploring practitioners’ perspectives through an international survey. Computers in Human Behavior, 43:35–48, February 2015.
[14] T. Linda and G. Saul. How people revisit web pages: empirical findings and implications for the design of history systems. International Journal of
Human-Computer Studies, 47(1):97–137, July 1997. [15] G. Lindgaard, G. Fernandes, C. Dudek, and J. Brown.
Attention web designers: You have 50 milliseconds to make a good first impression! Behaviour
InformationTechnology, 25(2):115–126, 2006. [16] J. Nielsen. User interface design for the www.
Proceeding CHI EA ’97 CHI ’97 Extended Abstracts on Human Factors in Computing Systems, pages 140–141, March 1997.
[17] J. Nielsen. Usability Engineering. Academic Press Inc, California, page 165, 1993.
[18] D. A. Norman. The way i see it: The transmedia design challenge: technology that is pleasurable and satisfying. ACM Interactions, 17(1):12–15, January + February 2010.
[19] D. A. Norman. The Design of Everyday Things. Basic Book, New York, pages 3 & 14, 2013.
[20] H. Obendorf, H. Weinreich, E. Herder, and M. Mayer. Web page revisitation revisited: Implications of a long-term click-stream study of browser usage. Proceeding CHI ’07 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 597–606, April + May 2007.
[22] W. E. Saris and I. N. Gallhofer. Design, Evaluation, and Analysis of Questionnaires for Survey Research, Second Edition. John Wiley Sons, Inc., Hoboken, NJ, page 4, 2014.
[23] S. Smith-Atakan. Human-computer Interaction. Thomson Learning, London, pages 159-160, 2006. [24] D. Tasse, A. Ankolekar, and J. Hailpern. Getting
users’ attention in web apps in likable, minimally annoying ways. Proceeding CHI ’16 Proceedings of the
2016 CHI Conference on Human Factors in Computing Systems, pages 3324–3334, May 2016. [25] S. Zahoor, D. M. Bedekar, and P. K. Kosamkar. User
APPENDIX
A.
APPENDIX
A.1
User tests
Results from the ranking assignment divided into each function not yet presented, seen in tables 11, 12, 13, 14, 15, 16 and 17. 1 3 5 7 9 11 rejected 0 1 2 3
Figure 11. Add tabs
1 3 5 7 9 11 rejected 0 0.5 1 1.5 2
Figure 12. Send text messages to all registered partici-pants of a certain event
1 3 5 7 9 11 rejected 0
1 2 3
Figure 13. Track how people found your event page, example 30% via Facebook, 50% via Google etc.
1 3 5 7 9 11 rejected 0 0.5 1 1.5 2
Figure 14. Add Schedule
1 3 5 7 9 11 rejected 0 0.5 1 1.5 2
Figure 15. Print name tags directly from the web page
1 3 5 7 9 11 rejected 0
1 2 3
Figure 16. Photo editing
1 3 5 7 9 11 rejected 0
1 2 3
a.
How many times have you used Confetti?
1 time
A2-5 times
B5-9 times
C10 or more
DI have never used
Confetti
E
b.
Do you use Confetti on a regular basis?
Yes
Y
No
N
c.
Have you at some point used a service similar to Confetti?
Yes
Y
No
d.
What got you using Confetti from the start?
Choose as many as you like
Recommendation by friend/coworker
ASearched online
BAdvertisement online
CPreviously used Confetti to sign-up for an event
D
Heard about Confetti and wanted to check it out
E
Other
F
e.
How likely are you to recommend Confetti to others?
Not likely Very likely
f.
How pleased are you with the general experience of Confetti?
a
1a
2a
3a
4a
5g.
How would you rate the design of Confetti?
g.
How would you rate the design of Confetti?
a
1a
2a
3a
4a
5h.
Is the pricing plan reasonable?
Not reasonable Very reasonable
i.
Something you would like to comment about the pricing.
j.
What do you find is the most important feature with Confetti?
k.
Is there anything you are missing in the service today?
l.
How easy is it to create an event site?
a
1a
2a
3a
4a
5m.
Was it easier or harder than you thought to create your first
event site?
Harder than I thought As expected Easier than I thought
n.
Is there anything that would make it easier to create your event
site?
o.
Do you often need help when you are using Confetti?
Yes
Y
No
N
p.
Have you used our FAQ?
Yes
Y
No
N
q.
Do you find our FAQ useful?
Not useful Very useful
r.
Anything you would like us to add, in terms of customer service?
Choose as many as you like
Phone number
AVideo tutorials
BFaster chat
CMore extensive FAQ
D
Other
E
s.
Overall, how satisfied are you with Confetti?
1
2
3
4
5
Not satisfied Very satisfied
t.
And how likely are you to use Confetti again?
Not likely Very likely
u.
Is there anything you would like to add?
2
Section 2 | About you
a.
How many people are working at your company?
I use Confetti as a
private person
G
b.
What industry are you working in?
Music
AIT/Technology
BStartup
CEducation
DSports/Athletics
EEvent agency
FWedding planner
GPR/Marketing
HI don't use Confetti for work
I
Other
J
