IN
DEGREE PROJECT MEDIA TECHNOLOGY, SECOND CYCLE, 30 CREDITS
STOCKHOLM SWEDEN 2018,
Incorporating aspects of flow theory to design an
achievement-oriented interface
DESIRÉE FREDRIKSSON
KTH ROYAL INSTITUTE OF TECHNOLOGY
SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
ENGLISH TITLE
Incorporating aspects of flow theory to design an achievement-oriented interface
SWEDISH TITLE
Design av ett prestationsinriktat gränssnitt med hjälp av aspekter ur flow teorin
Desirée Fredriksson desireef@kth.se
Degree project subject: Media Technology
Program: Master of Science in Engineering in Media Technology Supervisor: Vygandas Simbelis
Examiner: Roberto Bresin
Date of submission: 2018-06-26
ABSTRACT
The aim of this study was to explore how flow theory could be applied to improve the process for completing an application process to a creative education program. This was done by first formulating an approach inspired by flow theory and characteristics that promotes creativity in digital tools. The question was then explored using research through design with the goal of formulating the aspects of the flow theory inspired approach as generative intermediate level knowledge in the form of guidelines. User centered design was applied during the design process, starting with an empathizing phase where fundamental information about the application process and the users was attained. It went on to explore design implementations based on the flow theory inspired approach during a workshop and then on to creating an interactive prototype which was evaluated by six participants, iterated and tested in a second user evaluation.
The findings of the study implicate that incorporating the guidelines in the flow theory inspired approach could be helpful when designing supportive achievement-oriented environments.
SAMMANFATTNING
Syftet med studien var att undersöka hur flow teorin kan implementeras i syfte att förbättra ansökningsprocessen till ett kreativt utbildningsprogram. Detta undersöktes genom att först formulera ett antal riktlinjer inspirerade av aspekter ur flow teorin samt faktorer som främjar kreativitet i digitala verktyg. Forskningsfrågan utvärderades genom en användarcentrerad designprocess, processen började med att förstå användarnas upplevelse och skolans mål med ansökningsprocessen. Under en workshop undersöktes det hur de framtagna riktlinjerna kunde appliceras i ett gränssnitt. Efter denna workshop påbörjades arbetet med att generera en prototyp som sedan utvärderades i ett användartest, itererades och utvärderades i ett andra användartest. Målet med designprocessen var att undersöka huruvida riktlinjerna som formulerats hade potential att användas som generella riktlinjer vid gränssnittsdesign.
Resultaten från studien tyder på att riktlinjerna som formulerades har potential att skapa bättre, mer stöttande, upplevelser i prestationsinriktade miljöer.
Incorporating aspects of flow theory to design an achievement-oriented interface
Desirée Fredriksson KTH Royal Institute of
Technology Stockholm, Sweden
desireef@kth.se
ABSTRACT
The aim of this study was to explore how flow theory could be applied to improve the process for completing an application process to a creative education program. This was done by first formulating an approach inspired by flow theory and characteristics that promotes creativity in digital tools. The question was then explored using research through design with the goal of formulating the aspects of the flow theory inspired approach as generative intermediate level knowledge in the form of guidelines.
User centered design was applied during the design process, starting with an empathizing phase where fundamental information about the application process and the users was attained. It went on to explore design implementations based on the flow theory inspired approach during a workshop and then on to creating an interactive prototype which was evaluated by six participants, iterated and tested in a second user evaluation.
The findings of the study implicate that incorporating the guidelines in the flow theory inspired approach could be helpful when designing supportive achievement-oriented environments.
Author Keywords
Flow theory, Creativity, Motivation, Usability.
INTRODUCTION
Creativity is a multifaceted construct. It has many advantages, it facilitates problem solving, adaptability, and allows people to express themselves [21]. One thing that can inhibit creativity however is fear. This can be the fear of not being skilled enough, the fear of failure and of being judged by others. This fear can be overwhelming and cause insecurities and self-doubt which in turn inhibits creativity [13].
When an individual is put in an achievement-oriented situation they might need help to reach the creative state and avoid getting caught in these fears. Flow is one way of helping individuals to reach a more creative state [23]. In Csikszentmihalyi’s[5]flow theory the experience of flow is described as the optimal enjoyable experience when someone is fully immersed in an activity. A person who is
experiencing the state of flow disregards all thoughts that are not relevant to the present task. Flow can therefore facilitate creativity since it prevents these fears from emerging.
The aim of this study is to explore if a more motivating and supportive environment can be designed by incorporating qualifying factors of flow and characteristics of promoting creativity in digital tools. A motivating interface could increase users’ positive affect and might even enhance performance [3]. In this study the achievement-oriented situation being examined is the application process to a creative business school which offers education programs in the digital area. The application process (see Appendix 1) includes a task which requires the applicants to be creative in producing and communicating an innovative idea.
Factors that facilitates the flow state[5] and characteristics that promote creativity in digital tools [7, 23, 26] were formulated into a flow theory inspired approach. The approach was formulated with the intent of providing guidelines for designing an interface which motivates users and help them enter the creative state without getting caught in insecurities and fears. This without including elements of gamification which has been the focus of previous studies [27, 28].
The research question being explored in this study is the following:
How can a flow theory inspired approach be used to design an interface which supports the applicants needs and motivation while completing an application to a creative education program?
THEORY & RELATED WORK Creativity and flow
Looking up “creativity” in a dictionary will provide you with definitions such as “producing or using original and unusual ideas” and synonyms like “originality, innovativeness and imagination”
. It can be seen both as the
attribute of being creative as well as the power to create,
which Krathwohl [12] describes as “putting elements together to form a novel, coherent whole or make an original product”
. It has been shown that this process of
creating novel and original concepts benefits from an environment free from stressors, such as evaluation and pressure which often lead to anxiety that could hinder creative thinking [13, 21].
One way of fostering creativity that has been explored is promoting the state of flow [23]. Flow is not directly tied to creativity, however many individuals engaged in creative tasks do experience being in the state of flow. Flow, a term coined by Csikszentmihalyi [5], is described as an optimal, enjoyable experience when someone is fully immersed in an activity. When a person is in a state of flow they are experiencing a total concentration on the present task and a perceived agency over the situation. Distractions and thoughts are blocked out and insecurities such as self-doubt disappears. During the activity the person is fully engaged and it is not until after the flow experience that they will realize the positive experience of the activity. These dimensions of the flow experience can foster the creative process. For instance, a research [31] examining the presence of flow in the use of instant messaging, IM, found that the experience of flow, through facilitating exploratory behavior and positive affect, had a positive impact on the users' perceived creativity. The same study also established that perceived control over the situation can be both an antecedent as well as a dimension of flow.
To enter a state of flow there are some qualifying factors;
the task at hand needs to have clear goals, clear and immediate feedback on the progress being made and the challenge must match the perceived skills [5, 17].
Creativity is a highly collaborative process and often emerges from sharing ideas with others [16]. Information technologies that facilitate collaboration can therefore have a positive impact on the expected creativity among its users [7, 22, 31]. Many studies have recognized the fact that the use of different information technologies can result in improved creativity [2, 11, 18]. According to a study by Perkins[18] technology can shape an individual’s thinking by promoting combinatorial work and play with information and ideas. Brainstorming is a well-known method used in creative processes and is a way of combining things that comes to mind in order to create novel ideas. The study on IM and flow experience [31]
showed that computer mediated communication can have a positive impact on the perceived creativity. There are also many other studies [4, 19] supporting the notion that flow experience occurs in information technology use.
Promoting creativity in digital tools
There are some characteristics that have been shown successful in tools designed to support creativity. One of these characteristics is that it should be easy to explore and
try out different ideas and concepts. This goes hand in hand with supporting iteration, meaning that the tool should support the ability to delete or save as desired [7, 23, 26].
Another attribute for supporting creativity is to facilitate collaboration with others. In the early stages of a creative process the collaboration could concern problem definition and setting goals. In this stage the collaboration must be handled carefully because individuals can fear the rejection, ridicule, or rip-off on their idea or concept. It is therefore important for this collaboration to be supported in an environment where individuals will feel safe to share their ideas [25].
A tool for promoting creativity should also be designed with a low threshold, meaning that it should be easy to learn so that novices can start using it. As important as keeping a low threshold is the high ceiling, making it possible for experts to work with the tool on more sophisticated projects [26]. And as already mentioned a tool for promoting creativity should provide wide walls, meaning that it should support the possibility to explore different ideas.
Motivation
Motivation is the external or internal factors that prompts a person to engage in something. Motivation is therefore an important aspect in understanding users’ behaviour and performance.
Dweck and Leggett[6] have presented a model where goals are used to explain how individuals react when faced with challenges. In their model, two categories of goals are identified. These categories are performance and learning goals. Performance goals are categorized by the aim to demonstrating competence and the strive to gain positive judgement from others based on one's abilities. Learning goals are oriented towards improving or acquiring new skills. This means that an individual with learning goals seek out challenging tasks where they can attain new knowledge while someone with performance goals tend to look for tasks where the risk of failure is minimized. In the performance context, high effort would be considered a personal deficiency while challenges in the learning context would be considered an opportunity to improve one's knowledge rather than something that would imply less ability. This lead to the notion that performance and self-cognition are likely to benefit from a focus on learning rather than performance goals.
Online achievement-oriented environments can be challenging for users, because they require them to set motivating goals and practice self-regulatory skills. A study by Bittner and Zondervan [3] explored whether performance related pop-ups could be implemented to elicit motivation goals and increase the user experience of an achievement-related website. The human need for achievement have been shown to be affected by
performance-related pictures. In one study [24], for instance, people that were shown a picture of a woman winning a race on their way to work performed better on a brainstorming task than others.
Bittner and Zondervan’s study[3] on exploring the effect of pop-ups on motivation used achievement-related pictures together with an achievement-related word. The results showed that the pop-ups improved the user experience of the website. For achievement-oriented environments, it could even be anticipated that pop-ups consisting of a word and picture related to the task could influence the performance of the task by eliciting motivational goals.
User experience and usability
According to ISO 9241-110:2010 clause 2.15 [29] user experience is a: “person's perceptions and responses resulting from the use and/or anticipated use of a product, system or service
”. User experience can be regarded as the
consideration of information technology from the human perspective. It is the notion of understanding the role of emotions and desires as antecedents, consequences and mediators of technology use. UX focuses on positive affect, not only preventing usability problems but also allowing technology to evoke positive emotions and creating satisfying user experience [8].
In a study by Pilke [19] the concluding hypothesis was that designing interfaces which promotes the experience of flow is to design for good usability and vice versa. Usability is about considering the enjoyment and ease of use of a design. The definition of usability according to ISO 9241-11:2010 clause 2.13 [29] is the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use
”. Meaning that for an interface to be
usable the user should be supplied with the right tools to perform the given task and to be able to do this in an optimal way and with positive feeling of the experience.
Flow theory and games
Flow theory has been used extensively in assessing and designing games and gamification[4, 28, 30]. One instance is GameFlow[30] which is a model for evaluating players’
enjoyment in games. GameFlow is grounded in the dimensions of flow and consists of eight different elements – concentration, challenge, skills, control, clear goals, feedback, immersion, and social interaction. Using the model for evaluating two real-time strategy games, they found that: “
clear goals are presented through an
introduction that provides background, motivation, and overriding goals, in-game cut-scenes that present goals and further the story, as well as clear and specific mission objectives;
feedback involves notifying the player of completion or
failure of missions, keeping a log of mission goals, objectives, and status, providing a score and summary at the end of the mission, as well as visual and auditory feedback on actions, tasks, and events;”
- [30]
According to Chen [4] the components of flow can be mixed and matched in order to provide an enjoyable experience for the widest variety and number of players.
METHOD
The research question was explored using Research through Design (RtD)[10, 32]. This is a method where knowledge is gained through the design process. The findings of RtD can be articulated through intermediate-level knowledge such as annotated portfolios [14,15], strong concepts [10]
and guidelines [15]. Generative intermediate-level knowledge is insights gained from design research that is abstracted to fit more than one particular design instance, but lacks the generalization that characterize a theory [9,10].
The aim of this study was to explore if a flow theory inspired approach could be used to design an interface that motivates and supports users in an online achievement-oriented environment. If it was found to be useful, then the aspect of the flow theory inspired approach could be used as guidelines for other design instances.
DESIGN PROCESS
During the design process a user-centered design approach based on the recommendations of ISO 9241-110:2010 clause 6.1 [29] was applied. The process included an empathizing phase, workshop, two prototyping phases and user evaluations. After the empathizing phase the flow theory inspired approach was formulated.
Empathizing Phase
The process started with an empathizing phase, where two semi-structured interviews with current applicants were conducted. The goal with the semi-structured interviews was to understand the applicants needs and what motivated or discouraged them during the process.
Apart from the semi-structured interview, user insight was gained by attending the school’s webinar, support event and co-working session for applicants. These events are organized to help the applicants getting started on their application task. The webinar and support event was divided into an idea generation session and a session where they could ask all the questions they might have had about the process. The idea generation consisted of a brainstorming session where the participants brainstormed around three different areas related to the task. After the brainstorming, the participants randomly combined
elements from the three different areas to create as many concepts as possible.
The co-working session was an opportunity for the applicants to work on their respective task together with other applicants in the school’s facilities. During these different events, informal discussions about how the applicants perceived the application process and some observations were made. There is also a Facebook group initiated to support the applicants throughout the process. In this group applicants can ask questions to previous students and other applicants as well as ask for feedback on their task. This group was analyzed to see which questions occur more often than others.
Apart from exploring the process from the perspective of the applicants it was also important to learn which aspects the student recruiters are looking for when assessing the application task. This was important to make sure that the proposed changes to the process still allowed the applicants to show the competences that the recruiters are evaluating.
To understand this perspective a discussion with a student recruiter was carried out. In addition to the interview the evaluation criteria were explained while going through previous application task presentations.
Flow theory inspired approach
Based on the literature study on flow theory [5, 17] and supporting creativity in digital tools [7, 23, 26] as well as the findings from the empathizing phase a flow theory inspired approach was designed.
Provide clear goals Facilitate collaboration
Provide clear and immediate feedback
Promote exploration of different ideas
Design to elicit a feeling of progress
Design for high ceiling with low threshold
Make the task match users’ competences
Table 1. A list of the guidelines incorporated in the flow theory inspired approach.
Clear goals, clear and immediate feedback on the progress and task matching skills are antecedents of flow which were incorporated in the approach. Clear and immediate feedback on the progress was divided into two separate guidelines, the reason for this was that if there is no feeling of progress it is not possible to provide feedback on this. So, to be able to provide feedback it is important to also evoke a sense of progress if it is not already an apparent part of the interface.
These antecedents were chosen since they are the qualifying
factors for reaching the flow state and the positive effects of it [5]. The antecedents of flow were combined with a selected group of characteristics that have been shown to promote creativity in digital tools [7, 23, 26]. The full list of guidelines incorporated in this flow theory inspired approach are listed in table 1.
These guidelines were considered when designing the prototype, where examples of how they could be implemented are described, together with the aspects of designing for a good overall user experience and usability.
Workshop
Once the empathizing phase and literature study was completed, the outcomes were discussed during a workshop with a group of five interaction designers. The workshop started with a presentation of the most important findings from the empathizing phase, it went on to explain the flow theory inspired approach and then an initial sketch was presented. The sketch of a possible interface allowing applicants to create their presentation of the task was constructed as a simple WYSIWYG, “What you see is what you get”, editor. The proposed functionalities included the possibility to add images, videos, links, documents and audio files, see figure 1. The idea of keeping the interface simple came from the a study by Rosson [20] which showed that a simple text editor resulted in longer and better stories than editors which allowed people to format the text.
Figure 1. A zoomed in view on the initial sketch of the WYSIWYG editor and the different functionalities; adding images, videos, links, documents and audio files.
The goal of the workshop was to gather ideas on how the different aspects of the flow theory inspired approach could be incorporated in the prototype, while considering the needs of the users.
User evaluations
Once the prototype was completed a user evaluation was conducted to see the effects of the proposed changes.
Participants for the evaluation were recruited through the support group on Facebook. In total 6 participants were recruited for the user evaluations, out of these participants, five had gone through the application process within the last months and one applied in the spring of 2016. Three participants were female and the other three were male. For the time of the initial user evaluation the five participants who had just applied had not received the invitation to the
next step. In time for the second evaluation they had all received the invitation to the next step, meaning that they were all successful in the first step. The one participant who had applied in 2016 had also been successful since he later attended the school.
Initial user evaluation
The evaluation started with a short introduction of the purpose of the research. After the introduction, a couple of questions about their own application task were asked to trigger the recollection of the process.
A collection of 28 words was then presented to the participants. They were asked to choose the words that described how they perceived the application process or the feelings that it provoked. This method was inspired by the reaction cards developed in the desirability toolkit by Microsoft. The desirability toolkit and the reaction cards is a method that prompts users to tell a more detailed description of their experience and explores the desirability aspect of UX [1]. The words were of both negative and positive character, with a distribution of 43% negative and 57% positive words. The recommended distribution being, 40% negative and 60% positive words meant that the distribution of words in the reaction cards presented to the participants was very close to this recommendation. The collection of words was based on words that could be used to explain the experience of being in a state of flow as well as words used to describe the process by the applicants interviewed in the empathizing phase, see the full list in table 2.
Clear Flexible Overwhelming
Complex Focused Playful
Comprehensive Helpful Rewarding Confusing In control Safe Difficult Ineffective Self-doubt Distracting Insecure Supporting
Easy Inspiring Time consuming
Empowering Intimidating Unapproachable Enjoyable Motivating Unclear
Table 2. A list of the 28 words used for the product reaction cards.
After choosing the words the participants were asked to explain the reason for choosing these. The reason for having the participants try to recollect the experience of the application process was twofold. The first reason was that it would make the participants more able to evaluate the
prototype with the process in mind. The second reason was to verify the findings from the empathizing phase.
They were then introduced to the prototype and asked to think aloud while walking through it. After exploring the prototype the word session was repeated.
Second user evaluation
After the initial user evaluation, the prototype was iterated and made more interactive so that the participants could test the different features. The prototype was then tested in a second user evaluation. The participants from the first evaluation had gotten a clear understanding of how the new system was intended to work during the first evaluation, because of this they were asked to take part in the second evaluation as well. This decision was made so that the iteration and the second evaluation could build on the most relevant parts of the interface. All participants from the first evaluation took part in the second evaluation.
The users had the prototype sent to them and were given instructions on how to test it. The second prototype was limited to the application task and the editing interface where they could create the presentation of the task. This decision was made to obtain a better insight on how such a system would influence the applicants in terms of perceived requirements of the presentation of the task. After testing the prototype the participants were asked to answer a questionnaire about their experience and if they would have changed the way they presented the task and if they would have created the presentation using the interface of the prototype.
RESULTS
Empathizing phase
Applicants found the lack of direction of the application task to be discouraging. The brief is very broad which led the applicants to have doubts about where to start and what was expected from them, especially when it came to the final deliverable.
“I felt that the brief was a bit tricky and difficult to understand. I didn't understand what was expected of me, especially when it came to deliverables.”
- P2
Being a creative school the perceived expectation was that there is a high claim on the aesthetics of the final presentation of the task. Especially after looking at projects that students of the school have developed the applicants mentioned feeling that the presentation of their application task needed to be visually pleasing.
“You look at examples from students’ projects and they’ve done these super amazing things, then you start feeling some pressure.”
- P1
In fact, there is no requirement on the appearance of the presentation, the focus should be on effective rather than attractive communication. However, this perceived requirement on the appearance of the presentation was mentioned as a stress factor and something that added to the insecurities concerning the task. Especially for applicants who had no prior experience of design. One participant mentioned having put several hours looking at learning materials on graphic design to be able to create a visually pleasing presentation.
The events such as the webinar and support events was explained as the step when a lot of the pieces fell together for the task and some insecurities were reduced.
“I signed up for the workshop and that’s where it kind of fell into place, and I understood how to formulate my ideas
”
- P1 Workshop
To incorporate the factors that facilitate flow the suggestion was made to start the process narrow with the simple questions and build up to the application task. This would allow for the implementation of a progress bar that could establish the sense of progress being made.
Another way to evoke a sense of progress that was mentioned in the workshop was to create a digital version of the webinar and support events. The intention being that applicants could reach the state that they mentioned experiencing after the events, when they felt that the pieces fell into place, at an earlier state in the process. This would also make the progress on the task more apparent since they would step through a process before starting on the presentation of the task.
Making the task match the competences of the applicants is complex since all applicants have different backgrounds and skills. So, to incorporate this flow antecedent the aspect of designing for high ceiling with low threshold was considered. To be able to design for high ceiling with low threshold “Hidden features” in a text editor interface was discussed. These features could be detected when exploring the interface, making the interface more playful while at the same time allowing more advanced actions.
The format and the functions of the WYSIWYG editor was another feature that was discussed. The sketch that was presented had the format of an article, an alternative to this could be to have the interface as a presentation format, such as Keynote or PowerPoint presentations. The final decision was to keep the article format in the future prototype.
First prototype
After the workshop the work on creating a prototype started (see Appendix 2). All guidelines of the flow theory inspired approach, listed in table 1, were considered during the process of prototyping.
To incorporate the aspect of providing clear goals the first page that applicants would see when commencing the application was developed into a step by step breakdown of the process. The deadline of the application, as well as how many days before deadline was left was stated at the top right corner throughout the whole process.
Once the application process had commenced a progress bar would appear on the left side of the interface, indicating which step the applicants were on. As mentioned during the workshop this could be a way of evoking a feeling of progress as well as providing clear feedback. When moving on to the next step the applicants would get a visual feedback that they had completed the step by a check icon replacing the number of that step in the progress bar, see figure 2. This is similar to the findings of the study on GameFlow[30]where feedback was found in “ notifying the player of completion or failure of missions, keeping a log of mission goals...”
Figure 2. The image to the left shows part of the progress bar being displayed on all pages, here the second step is the active one, step one is completed and step three has not been initiated. The image to the right shows the deadline and days left.
A digital version of the idea generation used during the webinar and support events was incorporated in the prototype. The applicants were presented with an instruction on how the brainstorming session would work and a demonstration of the brainstorming view. The brainstorming consisted of elements from three different areas which were randomized for the applicants, see figure 3. The task for the applicants in this step would be to combine at least two of these elements into an idea. A time limit of 30 minutes was set, this was to indicate to the applicants that they should not put too much time on each idea.
Figure 3. The brainstorming page, the cards represents the three different areas.
The aim with this feature was to help the applicants
generate multiple ideas in the beginning of the task. In this way, eliciting a sense of progress as the applicants in the next step will be presented with all the ideas that they have generated, see figure 4. By visualizing all the ideas in this way the aim was that they would have the progress of the task clearly visualized in the same space, which they would not have had if they were only thinking of different ideas.
Breaking down the task in different parts was also done with the aim of eliciting a feeling of progress. This was done by substantiating a step that they would have done in another way otherwise.
By saving and visualizing all ideas from the brainstorming session the applicants are provided with a tool for exploring different ideas. Apart from exploring the ideas generated in the previous step they were also enabled to make copies of these ideas. By making a copy of an idea they would be able to explore different ways of presenting that idea as well.
Figure 4. The page where all the ideas from the brainstorming are saved and presented.
Discrete pop-ups were added to the bottom right corner of the interface. The pop-ups contained different information about the task and the assessment of it. The intention was to add reminders that would encourage the users to focus on the right things. In the instructions for the task the participants were told that they could work on their task even after they had submitted it. The participants were reminded of this opportunity in one of the pop-ups, the other pop-up reminded them about the evaluation criteria.
Incorporating a share function was a way of implementing the findings from the study on IM and flow [31] which showed that communicating with others could facilitate flow. The share function would allow applicants to easily share a link to their task and receive comments directly in the interface.
Making the task match the competences of the applicants was incorporated in the editing page, the interface was designed to elicit the feeling that a simple presentation, with emphasize on the content, of the task would be sufficient.
The interface consisted of the simple text editor presented during the workshop together with the same functionalities.
The “Hidden feature” suggestion from the workshop was discarded due to implementation difficulties, and the high ceiling was instead implemented in the possibility of adding a document created in other, more advanced, programs.
Initial user evaluation Current application system
The word session that was conducted to evaluate the current application process resulted in the distribution of negative words being 53% of the total chosen words while positive words were 47%. The strongest negative dimensions were overwhelming and unclear,
chosen by three participants
each. Playful, rewarding
, motivating and enjoyable, were
the strongest positive dimensions chosen by two participants each. Figure 5 shows the distribution of positive and negative words for the first word session, the size of the words is determined by the frequency of which the word was chosen.
Figure 5. Words chosen for current approach to apply, positive words to the left and negative to the right.
The part of the process that the participants mentioned as most unclear was what was expected from them in regards to the task. They were not sure on what the task entailed, two participants even started on a project and then realized that it was not what was expected and had to come up with a new idea. They also expressed that they were not sure about what was expected of them when it came to the presentation of the task.
“What is it that I have to do exactly? It was unclear what they wanted to see.”
- P3
These comments were consistent with the findings from the empathizing phase where participants brought up the same insecurities. However, the freedom of the task was also mentioned as a positive dimension by one participant:
“There was no list of criteria that I had to tick of, and that made it more creative and interesting, so it was an enjoyable experience.”
- P2
Insecure and self-doubt
are two similar words which were
chosen by four out of six participants in total. These words were mentioned when the participants were talking about
the task being overwhelming and they pointed out that they were not sure if they “had what it took” to successfully complete the task:
“It was overwhelming in the beginning, when I started reading the task I felt stressed and didn’t think I was good enough to do this.”
- P4
Two participants mentioned that this feeling declined after having participated in the webinar and workshop.
They
mentioned that the reason that these insecurities declined after the events was that they had gained a better understanding of the goal of the task.
Prototype with suggested improvement
The second word session that was conducted to evaluate the prototype with suggested improvement resulted in negative word choices being 14% of the total while positive words choices were 86%. A significant difference compared to the results of the reaction cards chosen for the current system.
Unclear was the strongest negative dimensions,
chosen by
two participants, however clear was also selected by two participants. The strongest positive dimensions, chosen by the majority of the participants were playful
and helpful,
these were both chosen by 4 out of 6 participants. Figure 6 shows the distribution of positive and negative words from the second word session, evaluating the prototype.
Figure 6. Words chosen for the prototype, positive words to the left and negative to the right.
The result from the word sessions shows that the overall experience of the application process was improved by the implementations in the prototype.
“I would never choose easy, because you still need to deliver something, but this experience made it easier than before.”
- P3
During the walk through of the prototype several participants expressed that they were positive towards the brainstorming feature which was inspired by the webinar and support event. They mentioned that it was nice to have your ideas saved in one place and that you could continue to explore several ideas further.
“It’s a good start, it helps you to come up with ideas. And since there’s a time limit you won’t leave it blank
” - P4
Some of the participants mentioned that they did not fully understand the process of the brainstorming session, and that this was the reason for choosing unclear
during the
word session. They got an explanation and after that they understood the process and said that they were positive to the idea.
A couple participants said that it was motivating to have a tool to work with, that there is a clear process compared to simply being presented with the brief of the task and having the next step being to submit it. They expressed that it felt like a huge final step to go in to the application and submit everything along with the task. Having a process to follow as well as the opportunity to continue working on the task after having submitted it made it less daunting.
Second prototype
When the initial user evaluation had been conducted an iteration of the prototype was carried out. This prototype was created using Framer and made more interactive than the previous one (see Appendix 3). In the iteration of the prototype the participants were able to try the brainstorming session and write down ideas, to get a clearer insight to how it would feel to use this feature. They were also able to input text, add a placeholder video and image in an imaginary task presentation, the features of adding documents and audio were still visible but not functioning.
None of the participants noticed the pop-ups with information in the first prototype. To make these more apparent strong colors and animations were added to the elements. These changes were made so that the effectiveness of the pop-ups could be explored during the second evaluation. Instead of only including text as in the first prototype, icons were added to the pop-ups. As suggested in the study exploring pop-ups as motivators [3]
one of the icons chosen was achievement-related. The icon showed hands applauding accompanied with a text saying
“You’re doing great”. The second icon was a heart together with the text “Keep up the good work”. The idea was that these would be presented while the applicants are working on their presentations of the task. The third pop-up only included informative text about the assessment of the task.
A preview function was added on the editing page that allowed the participants to see how their task would look like when viewed by those assessing it. In this view an estimation of the time that it would take to review their tasks was visualized as feedback.
On the editing page the submit button was changed to
“Check and submit” to clarify that there would be a chance to check the whole application and the application task
before submitting. The deadline indicator that in the first evaluation had been a random date was changed to the deadline for the second user evaluation and the interface would show the number of days until deadline for the participants.
Second user evaluation
When the participants were asked if they would have used the new system to create their task if they would have been given the possibility, three answered that they would and the other three said that they would have used the brainstorming functionality but then done the presentation in another program and used the document attachment possibility. The reason for this was that they would have liked more control over the design of the presentation.
Overall the comments on the experience of using this system to do the application task were positive:
“The steps are much clearer”
- P1
“It would have made it easier to get started with an idea, as well as decreased the pressure to make a pretty, more selling presentation”
- P6
It was articulated that the information was given when needed and that there was no need to leave the process to get the information that was needed. The estimation time function was mentioned as a positive addition:
“I can definitely feel like there is advantage to see the time it takes to review, so that you can prioritize and focus on the right things”
- P2
The modified pop-ups were received positive by all participants. When asked to rate the the statement “The pop-ups felt motivating” from 1 to 6 where 1 was “Do not agree at all” and 6 was “Totally agree” the mean value of the answers was 5.2. It was mentioned by one participant that he wished that the informative pop up, would be more subtle while he said that the emotional ones as he called them were good as they were. Another participant mentioned that he would have liked to have some pop-ups with answers to frequently asked questions.
DISCUSSION
The aim of this study was to explore how a flow theory inspired approach could be applied to improve the process for completing a creative application task. The guidelines of the flow theory inspired approach were considered when designing the interface and are explained in more detail in the design process section. Looking at the results from the study it can be concluded that considering the guidelines in the design of the interface was helpful at improving the process.
Flow theory inspired guidelines
Some of the participants said that they had felt self-doubt and insecurities during the application process but that these feelings had declined after having attended the support event. They mentioned that the reason that these insecurities declined after the events was that they had gained a better understanding of the goals of the task and how to tackle it.
This indicates that providing clear goals, without changing much of the description of the task itself, was an effective way of supporting the applicants in the current system for the application process. This also suggested that the addition of clear goals in the prototype would have the possibility of having the same effect. In the user evaluation, the design implementation of clear goals was something that were mentioned by several participants as a positive addition which helped them focus on the right thing.
Another possible reason that the participants mentioned feeling better after attending the support event could have been because of the collaborative nature of the workshop, which could have allowed them to get perspectives from other applicants that helped them in the process. This is however just a speculation based on the observations from the events and nothing that was stated during the interviews.
The aspect of providing clear goals was incorporated in the prototype by adding a breakdown of the process step by step on the first page, adding the progress bar and the fact that each view of the interface only contained one step of the process. That the interface only showed one step at the time was mentioned as something which allowed the participants to be more focused. It was also mentioned that it was easier to know what they were supposed to do at all times. Since the participant did not think about the other steps it also made them feel less stressed. Adding clear goals by defining the steps can therefore be considered a successful way of supporting the applicants during the process. By adding an extra step to the application task the brief could remain the same while the participants mentioned that the task was made more clear.
The pop-ups added in the first prototype was not mentioned to have any significant effect on the experience of the process. When adapted to be more in line with the design recommendation from the study on pop-ups as motivators [3] in the second prototype they were mentioned by participants as a nice addition and as a motivating factor.
These comments from the participants indicates that the findings in regards to the pop-ups correlates to the findings of Bittner and Zondervan’s study [3]. In a sense these pop-ups, the pop-ups from Bittner and Zondervan’s study [3] and the picture of the winning woman in Shantz and Latham’s study [24] can be seen as another way of implementing clear goals even though the intention was that they, in this study would provide feedback. In these studies
the elements are presented in a way that is supposed to elicit achievement. And by doing so they also elicit the unconscious goal of being successful.
The interface itself was mentioned as an improvement - allowing a clearer sense of progress and not only being the huge final step of submitting the task as before. As mentioned by P6 it also decreased the pressure to make a visually pleasing presentation which was the intention and would allow applicants to focus on the important parts. This while still allowing those who are interested in design an outlet for this by allowing document attachment, meaning that the challenge of task can be adjusted and in this way match different competences.
Possible implications
The findings can have implications on, for example, learning environments or in other talent acquisition situation where users’ needs to be motivated to achieve something. Either the design instances used in this study could be applied or other ways of implementing the aspects of the flow theory inspired approach could be explored and implemented in new ways more suitable for that particular situation.
Adjusting the guidelines based on users’ goals
The ways of implementing the aspects of the flow theory inspired approach should possibly be adjusted based on the goals of the users. Considering the different types of goals defined by Dweck and Leggett [6], users with performance related goals compared to users with learning related goals might need different types of support. This was indicated by P2 who seemed to have a more learning related goal when it came to the task. P2 mentioned that he liked that the task was so free:
“There was no list of criterias that I had to tick of, and that made it more creative and interesting, so it was an enjoyable experience.”
While other participants had mentioned the lack of clear criteria for the task as something which had a negative effect on the experience. This might suggest that P2 had more learning related goals than the other participants had.
Since individuals with learning goals are more focused on improving or acquiring new skills they don’t view challenging tasks as negative in the same way as someone with a performance oriented goals would [6]. If P2 had learning oriented goals this could explain why he was not so worried about reaching certain criteria, since his goal would be more focused on improving his skills, and his goal of learning more is already clear. One possible insight that this could provide us with is that clear goals is not as important when implementing the flow theory inspired approach in an environment where users have learning related goals as it would be in a performance environment.
This indicates that the implementation of the guidelines
should be adjusted based on users’ different goals.
Methodology criticism & future research
The participants saw and evaluated the interface in an experimental situation which is highly different from the real situation where they mentioned feeling overwhelmed and insecure. This feeling is more likely to appear in a situation where participants really feel like they need to accomplish something. In the situation of applying to a school this situation is also extra stressful, because if the applicants are not successful it means that do not get to study something that they really want to study. So, to properly test the effect of this flow theory inspired approach it would need to be tested in an environment that better simulated that situation.
Another aspect that could have had an impact on the result is the selection of participants. All participants in the study were successful in completing their online application in the sense that they all got invited to the next step of the process.
If participants who did not succeed would have been part of the study this could have provided other insights since those applicants might have needed even more, or other kinds of support compared to those who were successful.
The current study does not explore whether the new system and guidelines would elicit applicants to enter a state of flow when working on their application. To test this a study exploring the topic on a wider scale and in a more realistic situation rather than the experimental setting used in this study is needed and the findings should therefore be considered with this in mind.
CONCLUSION
The aim of this study was to explore how a flow theory inspired approach could be applied to improve the process for completing a creative application task. The approach included the list of guidelines provided in table 1.
All participants in the study were positive to the changes that had been made and expressed that the new systems would be more enjoyable to work with. Therefore, the findings of the study implicate that following the guidelines - and incorporating antecedents of flow and characteristics of promoting creativity could be an appropriate way of supporting and motivating applicants in an achievement-oriented environment. This could have implications on e-learning and other achievement-oriented environments where it is desirable to support the users throughout the process. The current study has only explored some ways of how this flow theory inspired approach could be incorporated in an online achievement environment.
There are certainly many other potential ways of incorporating these aspect which could be explored in future research.
REFERENCES
1. Carol M. Barnum and Laura A. Palmer. 2010.
More than a feeling: understanding the desirability factor in user experience. In Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems - CHI EA ’10
, 4703.
2. Sarit Barzilai and Anat Zohar. 2006. How does information technology shape thinking? Thinking Skills and Creativity
1, 2: 130–145.
3. Jenny V. Bittner and Robin Zondervan. 2015.
Motivating and achievement-eliciting pop-ups in online environments: A user experience
perspective. Computers in human behavior
50:
449–455.
4. Jenova Chen. 2007. Flow in games (and everything else). Communications of the ACM 50, 4: 31.
5. Mihaly Csikszentmihalyi. 1996. Creativity: Flow and the Psychology of Discovery and Invention
.
Harpercollins.
6. Carol S. Dweck and Ellen L. Leggett. 1988. A social^cognitive approach to motivation and personality. Psychological review
95, 2:
256–273.
7. Sharon L. Greene. 2002. Characteristics of applications that support creativity.
Communications of the ACM
45, 10.
https://doi.org/ 10.1145/570907.570941 8. Marc Hassenzahl and Noam Tractinsky. 2006.
User experience - a research agenda. Behaviour
& information technology
25, 2: 91–97.
9. Kristina Höök, Martin P. Jonsson, Anna Ståhl, and Johanna Mercurio. 2016. Somaesthetic Appreciation Design. In Proceedings of the 2016 CHI Conference on Human Factors in
Computing Systems - CHI ’16
.
https://doi.org/ 10.1145/2858036.2858583 10. Kristina Höök and Jonas Löwgren. 2012. Strong
concepts: Intermediate-level knowledge in interaction design research. ACM Transactions on Computer-Human Interaction
19, 3: 1–18.
11. Linda A. Jackson, Edward A. Witt, Alexander Ivan Games, Hiram E. Fitzgerald, Alexander von Eye, and Yong Zhao. 2012. Information
technology use and creativity: Findings from the Children and Technology Project. Computers in human behavior
28, 2: 370–376.
12. David R. Krathwohl. 2002. A Revision of Bloom’s Taxonomy: An Overview. Theory into practice
41, 4: 212–218.
13. Chris Lewis. 2016. Too Fast to Think: How to Reclaim Your Creativity in a Hyper-connected Work Culture
. Kogan Page Publishers.
14. William Lidwell, Kritina Holden, and Jill Butler.
2010. Universal Principles of Design, Revised and Updated: 125 Ways to Enhance Usability, Influence Perception, Increase Appeal, Make Better Design Decisions,
. Rockport Publishers.
15. Jonas Löwgren. 2013. Annotated portfolios and other forms of intermediate-level knowledge.
Interactions
20, 1: 30.
16. Fiona Maciver and Julian Malins. 2016. Two Heads Are Better Than One: Principles for Collaborative Design Practice. In Collaboration in Creative Design
. 13–31.
17. Jeanne Nakamura and Mihaly Csikszentmihalyi.
2014. The Concept of Flow. In Flow and the Foundations of Positive Psychology
, Mihaly
Csikszentmihalyi (ed.). Springer Netherlands, Dordrecht, 239–263.
18. D. N. Perkins. 1985. The Fingertip Effect: How Information-Processing Technology Shapes Thinking. Educational researcher
14, 7: 11.
19. E. M. Pilke. 2004. Flow experiences in
information technology use. International journal of human-computer studies
61, 3: 347–357.
20. Mary Beth Rosson. 1983. Patterns of experience in text editing. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems - CHI ’83
.
https://doi.org/ 10.1145/800045.801604
21. Mark A. Runco. 2004. Creativity. Annual review of psychology
55: 657–687.
22. R. Keith Sawyer and R. Keith Sawyer. 2006.
Educating for innovation. Thinking Skills and Creativity
1, 1: 41–48.
23. Ted Selker. 2005. Fostering motivation and creativity for computer users. International journal of human-computer studies
63, 4-5:
410–421.
24. Amanda Shantz and Gary P. Latham. 2009. An exploratory field experiment of the effect of subconscious and conscious goals on employee performance. Organizational behavior and human decision processes
109, 1: 9–17.
25. Ben Shneiderman. 2007. Creativity support tools:
accelerating discovery and innovation.
Communications of the ACM
50, 12: 20–32.
26. Ben Shneiderman, Gerhard Fischer, Mary Czerwinski, Mitch Resnick, Brad Myers, Linda Candy, Ernest Edmonds, Mike Eisenberg, Elisa Giaccardi, Tom Hewett, Pamela Jennings, Bill Kules, Kumiyo Nakakoji, Jay Nunamaker, Randy Pausch, Ted Selker, Elisabeth Sylvan, and Michael Terry. 2006. Creativity Support Tools:
Report From a U.S. National Science Foundation Sponsored Workshop. International journal of human-computer interaction
20, 2: 61–77.
27. Chaklam Silpasuwanchai, Xiaojuan Ma, Hiroaki Shigemasu, and Xiangshi Ren. 2016. Developing a Comprehensive Engagement Framework of Gamification for Reflective Learning. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems - DIS ’16
.
https://doi.org/ 10.1145/2901790.2901836 28. Graziela Sombrio, Leonardo Enrico
Schimmelpfeng, Vânia Ribas Ulbricht, and Vilma Villarouco. 2016. Gamification in Education Through Design Thinking. In Lecture Notes in Computer Science
. 311–321.
29. Swedish Standards Institute. 2012. Ergonomics of human-system interaction – Part 210:
Human-centred design for interactive systems (ISO 9241-210:2010).
30. Penelope Sweetser and Peta Wyeth. 2005.
GameFlow: a model for evaluating player
enjoyment in games. Computers in Entertainment 3, 3: 3.
31. Maliha Zaman, Murugan Anandarajan, and Qizhi Dai. 2010. Experiencing flow with instant messaging and its facilitating role on creative behaviors. Computers in human behavior
26, 5:
1009–1018.
32. John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’07
.
https://doi.org/ 10.1145/1240624.1240704
APPENDIX
On the following pages the appendices listed below can be found:
1. EXPLANATION OF THE CURRENT ONLINE APPLICATION PROCESS AND THE SETUP OF THE APPLICATION TASK
2. LINK TO INITIAL PROTOTYPE
3. LINK TO VIDEO OF SECOND PROTOTYPE
1. EXPLANATION OF THE CURRENT ONLINE APPLICATION PROCESS AND THE SETUP OF THE APPLICATION TASK
All the following steps are presented in the same view:
1. Selecting the programs to apply to
a. In this step the applicants are presented with drop down lists where they can choose the programs they want to apply to.
2. Selecting country of residence
a. In this step the applicants choose their nationality and country of residence and if they need a visa to attend the school.
3. Personal info and contact details
a. In this step the applicants fill out personal information and contact details.
4. Uploading documents
a. In this step the applicants upload documents such as CV and photos.
5. Submit the Application Task
a. In this step the applicants are presented with a file attachment where they can upload their application task. Under the file attachment there is a link to the description of the application task for the year.
b. When clicking the link the applicants will get to a page where the task is presented.
The brief states the area in which the applicants are suppose to come up with an idea or solution. Other than that the brief is very open and free and allows the student to present their task in any way they like, the only requirement is that the presentation should not take more than five minutes to review.
2. LINK TO INITIAL PROTOTYPE
Link to figma
3. VIDEO OF THE SECOND PROTOTYPE
Link to Vimeo video
Password to see the video: MasterThesis
www.kth.se
