Department of Science and Technology Institutionen för teknik och naturvetenskap
Linköpings Universitet Linköpings Universitet
SE-601 74 Norrköping, Sweden 601 74 Norrköping
LITH-ITN-MT-EX--06/014--SE
The construction of a Haptic
application in a Virtual
Environment as a post Stroke arm
Rehabilitation exercise
Ulrika Dreifaldt
Erik Lövquist
The construction of a Haptic
application in a Virtual
Environment as a post Stroke arm
Rehabilitation exercise
Examensarbete utfört i medieteknik
vid Linköpings Tekniska Högskola, Campus
Norrköping
Ulrika Dreifaldt
Erik Lövquist
Handledare Matt Cooper
Examinator Matt Cooper
Norrköping 2006-03-10
Rapporttyp Report category Examensarbete B-uppsats C-uppsats D-uppsats _ ________________ Språk Language Svenska/Swedish Engelska/English _ ________________ Titel Title Författare Author Sammanfattning Abstract ISBN _____________________________________________________ ISRN _________________________________________________________________
Serietitel och serienummer ISSN
Title of series, numbering ___________________________________
Nyckelord
Keyword
URL för elektronisk version
Division, Department
Institutionen för teknik och naturvetenskap Department of Science and Technology
2006-03-10
x
x
LITH-ITN-MT-EX--06/014--SE
The construction of a Haptic application in a Virtual Environment as a post Stroke arm Rehabilitation exercise
Ulrika Dreifaldt, Erik Lövquist
This thesis describes a six-month project based on stroke rehabilitation and involves designing with medical doctors, a physiotherapist and an occupational therapist, prototyping and evaluating with both stroke patients and other users. Our project involves the construction of a rehabilitation exercise system, based on virtual environments (VE) and haptics, designed for stroke patients. Our system uses a commercially available haptic device called the PHANTOM Omni, which has the possibility of being used as a rehabilitation tool to interact with virtual environments. The PHANTOM Omni is used in combination with our own developed software based on the platform H3D API. Our goal is to construct an application which will motivate the stroke patient to start using their arm again.
We give a review of the different aspects of stroke, rehabilitation, VE and haptics and how these have previously been combined. We describe our findings from our literature studies and from informal interviews with medical personnel. From these conclusions we attempt to take the research area further by suggesting and evaluating designs of different games/genres that can be used with the PHANTOM Omni as possible haptic exercises for post-stroke arm rehabilitation. We then present two different implementations to show how haptic games can be constructed. We mainly focus on an application we built, a game, using an iterative design process based on studies conducted during the project, called "The Labyrinth". The game is used to show many of the different aspects that have to be taken into account when designing haptic games for stroke patients. From a study with three stroke patients we have seen that "The Labyrinth" has the potential of being a stimulating, encouraging and fun exercise complement to the traditional rehabilitation. Through the design process and knowledge we acquired during this thesis we have created a set of general design guidelines that we believe can help in the future software development of haptic games for post-stroke arm rehabilitation.
Haptics, stroke, rehabilitation, Virtual reality, Mixed reality, Virtual Environments, 3D, Game development, design, implementation, testing, evaluation
Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare –
under en längre tid från publiceringsdatum under förutsättning att inga
extra-ordinära omständigheter uppstår.
Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner,
skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för
ickekommersiell forskning och för undervisning. Överföring av upphovsrätten
vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av
dokumentet kräver upphovsmannens medgivande. För att garantera äktheten,
säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ
art.
Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i
den omfattning som god sed kräver vid användning av dokumentet på ovan
beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan
form eller i sådant sammanhang som är kränkande för upphovsmannens litterära
eller konstnärliga anseende eller egenart.
För ytterligare information om Linköping University Electronic Press se
förlagets hemsida
http://www.ep.liu.se/Copyright
The publishers will keep this document online on the Internet - or its possible
replacement - for a considerable time from the date of publication barring
exceptional circumstances.
The online availability of the document implies a permanent permission for
anyone to read, to download, to print out single copies for your own use and to
use it unchanged for any non-commercial research and educational purpose.
Subsequent transfers of copyright cannot revoke this permission. All other uses
of the document are conditional on the consent of the copyright owner. The
publisher has taken technical and administrative measures to assure authenticity,
security and accessibility.
According to intellectual property law the author has the right to be
mentioned when his/her work is accessed as described above and to be protected
against infringement.
For additional information about the Linköping University Electronic Press
and its procedures for publication and for assurance of document integrity,
please refer to its WWW home page:
http://www.ep.liu.se/Environment as a post-Stroke arm Rehabilitation exercise.
Ulrika Dreifaldt
Erik Lövquist
1. Introduction . . . 1
1.1 The scope of the project . . . 2
1.2 Project Goal . . . 2
1.3 Methods . . . 3
1.4 Thesis outline . . . 4
2. Literature review . . . 5
2.1 Stroke . . . 5
2.1.1 Rehabilitation exercises today . . . 8
2.2 Haptics and Virtual Environments . . . 12
2.2.1 Mixed Reality versus Virtual Reality . . . 14
2.2.2 Haptics and Reality - Virtuality boundaries . . . 16
2.3 Rehabilitation and haptics . . . 20
2.3.1 Previous work . . . 20
2.3.2 Previous haptic games . . . 22
3. Game Designs . . . 23
3.1 Design guidelines for haptic exercises . . . 23
3.1.1 Game design . . . 23
3.1.2 Interface design (Input/Output) . . . 25
3.2 Game suggestions . . . 26
3.3 Two more detailed game ideas . . . 29
3.3.1 MugMasterMind . . . 29
3.3.2 The Ballroom design . . . 30
4. Implementation . . . 33
4.1 Implementation tools and methods . . . 33
4.1.1 Toolkit of choice, H3D API . . . 34
4.2 Implemented games . . . 35
4.2.1 Hidden Objects . . . 35
4.2.2 The Labyrinth . . . 36
4.2.3 Example of possible mazes . . . 41
4.3 Implementation of "Ecological Approach to Human Perception" . . . 43
5. Testing and Evaluation . . . 45
5.1 Pre study - Single Person case . . . 45
5.2 Science Fair - UL 051113 . . . 46
5.2.2 Observations . . . 47
5.2.3 User observations . . . 49
5.3 Young Scientist - Dublin 12/01/06-14/01/06 . . . 50
5.3.1 Introduction . . . 50
5.3.2 Observations . . . 51
5.3.3 User observations . . . 52
5.4 Cooperative Evaluation Session of "The Labyrinth" with stroke patients . 52 5.4.1 Topics of interest . . . 53
5.4.2 Subjects . . . 53
5.4.3 Method - Cooperative Evaluation Session . . . 53
5.4.4 Experiment . . . 53
5.4.5 Observations . . . 54
6. Conclusions and Future work . . . 57
6.1 Conclusions . . . 57
6.2 Future work . . . 59
6.3 Summary . . . 59
Appendix 63 A. Project time plan . . . 64
B. Game Design Requirements . . . 65
B.1 Hidden Objects Design Requirements . . . 65
B.1.1 Must have . . . 65
B.1.2 Preferable . . . 65
B.1.3 Excellent . . . 65
1.1 The PHANTOM Omni . . . 1
2.1 The Box and Block Test . . . 8
2.2 The Purdue PegBoard . . . 9
2.3 The Grip Test . . . 9
2.4 The Hidden Objects Test . . . 9
2.5 Stick Othello . . . 10
2.6 Eye Toy Playstation setup . . . 10
2.7 The Mirror table . . . 11
2.8 Guiding blocks through a mace . . . 11
2.9 Phantom Omni . . . 13
2.10 The Novint Falcon, (http://www.novint.com/falcon.htm) . . . 13
2.11 The Virtuality Continuum (Milgram et al. 1994, (c)IEICE) . . . 14
2.12 The boundaries between virtual and real environments (Benford et al. 98) 15 2.13 The classification of some haptic systems . . . 16
2.14 Immersion Haptic Workstation, (http://www.immersion.com, permission requested) . . . 17
2.15 Immersive Workbench, (http://www.sensegraphics.se/products.html) . . . 17
2.16 Screen, PHANTOM Omni and headtracking . . . 18
2.17 Screen and PHANTOM Omni . . . 18
2.18 A joy pad with in-built force-feedback . . . 19
2.19 Cyber Glove, http://www.immersion.com, permission requested . . . 21
3.1 The card game Solitaire, ( (c) 1981-1999 Microsoft CORP., permission requested) . . . 27
3.2 Puzzle game, ( (c) 2002 ROBOT Inc, permission requested) . . . 27
3.3 3D Pong . . . 28
3.4 Design idea of MugMasterMind . . . 29
3.5 Design of two different levels . . . 31
3.6 Design of two more levels . . . 32
4.1 Screenshot of Hidden Objects . . . 36
4.2 Showing a difficult level of the Labyrinth and explaining a possible path. 37 4.3 The Labyrinth with dynamic objects . . . 39
4.4 The grid of how to create walls . . . 40
4.5 Text-file example . . . 40
4.6 Example of "normal" difficulty . . . 41
4.7 Example of harder difficulty . . . 41
5.1 Axes in 3D space . . . 47
5.2 Ulrika explaining the concept of depth . . . 48
5.3 Erik and a boy enjoying the exercise . . . 49
There are many people we would like to gratefully acknowledge for their support. First of all we would like to thank the team at the Interaction Design Centre, University of Limerick, Ireland for all their feedback, support and advice. Especially our supervisor Mr Mikael Fernström for the support and making us see how to approach things in new ways! Another thanks to Prof. Liam Bannon and Dr. Annette Aboulafia for keeping us on track and helping out with the report and Mr. Michael Cook for valuable advice about our studies. Thank you Dr. Micheal O’Haodha for your time and effort helping us with the language of the report. A special thanks to Mr. Paul Gallagher for all the encouraging feedback, support, lifts to meeting etc.
We also would like to thank our examiner Dr. Matthew Cooper, University of Linköping, Sweden for guiding us through the project. Mr. Karl-Johan Lundin, University of Linköping, Sweden for all the great advise regarding haptics and programming. Mr. Mark Dixon, Mr. Johan Mattson Beskow and Mr. Daniel Evestedt at SenseGraphics, Stockholm, Sweden for all support, advice and ideas regarding haptics and programming. Dr Martin Rydmark (MD), Mr George Pope (MD), Ms Susan Coote (PT) and Mr Jurgen Broeren (OT) for all valuable information about stroke patients and their symptoms.
A very special thanks to the three stroke patients at Högsbo, Gothenburg, Mr G and all other participants of our observations.
Ulrika would like to send big thank you hugs to her family and friends especially her parents Sören and Carina and her sisters Therese and Lindha. "Thanks a million for all the love and encouragement, which made me who I am, you’re the best!" She would also like to send another special thanks to her boyfriend Paul, for the supportive hugs he gives after a long day of work!
Erik would like to thank all of his family and friends, especially my parents Tommy and Kerstin, for all the love and support you give me. I would not have come this far without you! And a very special thanks to Grace for making my world shiny!
This thesis describes a six-month project based on stroke rehabilitation and involves de-signing with medical doctors, a physiotherapist and an occupational therapist, prototyp-ing and evaluatprototyp-ing with both stroke patients and other users. Our project involves the construction of a rehabilitation exercise system, based on virtual environments (VE) and haptics, designed for stroke patients. Our system uses a commercially available haptic device called the PHANTOM Omni, which has the possibility of being used as a reha-bilitation tool to interact with virtual environments. The PHANTOM Omni is used in combination with our own developed software based on the platform H3D API. Our goal is to construct an application which will motivate the stroke patient to start using their arm again.
We give a review of the different aspects of stroke, rehabilitation, VE and haptics and how these have previously been combined. We describe our findings from our literature studies and from informal interviews with medical personnel. From these conclusions we attempt to take the research area further by suggesting and evaluating designs of different games/genres that can be used with the PHANTOM Omni as possible haptic exercises for post-stroke arm rehabilitation. We then present two different implementations to show how haptic games can be constructed. We mainly focus on an application we built, a game, using an iterative design process based on studies conducted during the project, called "The Labyrinth". The game is used to show many of the different aspects that have to be taken into account when designing haptic games for stroke patients. From a study with three stroke patients we have seen that "The Labyrinth" has the potential of being a stimulating, encouraging and fun exercise complement to the traditional rehabilitation. Through the design process and knowledge we acquired during this thesis we have cre-ated a set of general design guidelines that we believe can help in the future software development of haptic games for post-stroke arm rehabilitation.
The World Health Organization estimates that in 2001 there were over 20.5 million strokes worldwide. A stroke occurs when the blood flow in the brain gets interrupted and this causes a lack of oxygen in the brain. The consequences of a stroke can vary a lot depending on what part of the brain that is damaged. Rehabilitation is essential for the patient to be able to return to everyday life.
The new innovation rehabilitation has seen is with the introduction of computer en-hanced environments. These environments, in combination with haptic devices for post-stroke rehabilitation have become a popular area of research (Adamovich et al. 2005, Boian et al. 2003, Broeren et al 2002, Rydmark et al 2005). Haptics refers to the modality of touch and associated sensory feedback that allows users to sense (’touch’) and manip-ulate three-dimensional virtual objects with respect to such features as shape, weight and surface textures. One of the main reasons for introducing the use of virtual environments in combination with rehabilitation is the possibility of making a number of alternative exercises that are engaging, stimulating and fun. To fulfill these properties for a stroke patient, the exercises require careful design.
Fig. 1.1: The PHANTOM Omni
A commercially available haptic device called PHANTOM Omni 1 has been used as an input and output device to control an exercise we have designed and built. The PHANTOM is a mechanical arm that has five joints, and enables the user to interact with objects in the virtual environment. There is a ’stylus’ attached to the end of the mechanical arm and the user holds and manipulates it with their hand to interact with
virtual objects. The PHANTOM Omni is connected to a computer and the movement of the physical pen corresponds to the movement of a virtual stylus on screen.
For a stroke patient the main concern is improvement. Rehabilitation is crucial for recovery and has to be performed on a daily basis (Carr and Shephard 2003). Every individual has different needs and preferences hence it is important to have a wide range of exercises to choose from. One of the most common symptoms after a stroke is impaired arm movement and the PHANTOM Omni can potentially be used in the rehabilitation process as an alternative to ordinary rehabilitation exercises.
1.1 The scope of the project
Researchers and medical personnel in the area believe that haptics can be used as a very powerful therapeutic rehabilitation tool for people with several different kinds of t.b.i. (traumatic brain injury) e.g. car crash or a stroke. In this thesis we will focus on stroke patients. We have made this choice because the symptoms of a stroke and the recovery afterwards are similar for all patients. If, instead, we had focused on people that have acquired their brain damage from a car crash we then had to take in to account several other injuries that might occur to other parts of the body, for example a broken arm. However, we do think that a person that is undergoing the same type of rehabilitation as a stroke patient can also gain from this.
Doctors and nursing staff in the area of stroke rehabilitation believe that a haptic sys-tem can be used to help ease the rehabilitation of several different symptoms due to brain damage such as perception, coordination, muscular capacity etc.
We believe that this small, portable and relatively in expensive system can be used both as an efficient home-based rehabilitation tool and for clinical-based rehabilitation. Before this tool can be used in the patient’s home it has to be proved to be effective. Such a study has to be done in a clinic with the presence of a doctor and was not possible within the time constrains of this project. We have therefore focused our thesis on designing, creating and prototyping an exercise that can be used by a clinic for future studies and design guidelines that can be used by designers and programmers. The guidelines are designed as an aid for the development of haptic rehabilitation exercises.
1.2 Project Goal
The main goal of this thesis is to design and develop a haptic application as an exercise for post-stroke arm rehabilitation that is rewarding, engaging and fun. Our goal is also that our application will motivate the stroke patient to start using their arm again. This
application can be used for both home-based and clinical-based post-stroke arm rehabil-itation. We hope that our work can be used as a stepping-stone and guideline for other researchers in this area.
1.3 Methods
Throughout this project we have used an iterative design process. We have given a num-ber of presentations and demonstrations during this time and all of them have contributed to the iterative design process. See Appendix A for more information about the time plan for our project.
One of the methods we have used is informal interviews with doctors, a physiothera-pist, and an occupational therapist. The interviews have involved an open-ended discus-sion approach with a few already prepared questions. We have also made observations of stroke patients, at a voluntary stroke patient meeting in Limerick, Ireland and of rehabil-itation exercises in the rehabilrehabil-itation centre in Högsbo, Gothenburg, Sweden.
Throughout the project we have worked closely with a user that has cerebral palsy from birth asphyxia, who we, from now on, call Mr. G, see section 5.1. He heard about our project and volunteered at an early stage to help us out, his right arm has similar symptoms to a stroke patient’s arm. He has helped us to understand the different con-straints imposed by an impaired arm. He has also been regularly involved in our iterative design process.
In the early stages of the design phase, in September 2005, we demonstrated the PHANTOM at the Open Days, University of Limerick in Ireland. The demonstrations gave us a better understanding of how people with no experience of Mixed Reality, Vir-tual Environments and Haptics uses and understands the system.
In early November 2005 we presented our work at the fortnightly research meeting in the Regional Hospital, Limerick. About 20 people attended the meeting including doctors, nurses, physiotherapists and occupational therapists. They all liked the idea of using advanced computer technology for rehabilitation and provided great feedback.
In November 2005 we demonstrated our application, "The Labyrinth", at the Science Fair, University of Limerick, (see section 5.2) Ireland and at the VR-Forum 2005 in Gothenburg, Sweden. These two demonstrations provided a lot of feedback and a deeper understanding of how people understand and interact with our system.
In mid January 2006 we demonstrated our application at the Young Scientist exhibi-tion in Dublin, see secexhibi-tion 5.3. This was a good pre-study and preparaexhibi-tion for the im-portant cooperative evaluation of our application which we finally conducted with three stroke patients at the end of January 2006 in Högsbo, Gothenburg, see section 5.4.
1.4 Thesis outline
In chapter 2 we introduce the subject of stroke and provide an explanation of stroke and how this condition affects the person. We also review the rehabilitation methods currently in use. We then introduce haptics and virtual environments and discuss how this can be used to create a motivating rehabilitation exercise.
In chapter 3 we introduce design guidelines that have evolved from design sessions and studies carried out during the project. These guidelines can be used for design of a haptic exercise for stroke rehabilitation. We also present some exercise suggestions and some evaluated exercise designs.
In chapter 4 we describe some available implementation tools and also which ones we chose and why. We also present the two different applications which we have im-plemented, - "The Hidden Objects" and "The Labyrinth". We then finish the chapter describing the way in which we implemented ecological approach to human perception, which we did in the absence of a stereoscopical display at the beginning of the project.
In chapter 5 we describe the testing and evaluation phase of the project. As discussed previously we have used an iterative design process and this chapter explains the different presentations and demonstrations we have done throughout the project.
To understand the different aspects that have to be considered when creating a haptic exercise for stroke patients in a virtual environment, an extensive literature review were undertaken. In this chapter we analyze the questions of stroke, rehabilitation, haptics and virtual environments.
2.1 Stroke
According to the National Stroke Register in Sweden (2005) 30,000 people per annum suffer a stroke (population of Sweden approximately 9 million). Every 17th minute some-body in Sweden will have a stroke. Eighty per cent of these people are over 65 years of age (the age for retirement in Sweden). The number of people having strokes is projected to increase by 30% in Sweden before 2010, due to the ageing population of Sweden. Ev-ery year 8,000 people in Sweden die as a result of a stroke. This makes stroke the third largest cause of death. Stroke is estimated to cost 14 billion Swedish kronor per year to the Swedish exchequer
Sweden is not unique. According to the Heart disease and Stroke statistics from the American Heart Association (2005) 700,000 Americans are projected to have a stroke in 2005 (population approximately 297 million). Stroke is also the third most common killer and a leading cause of severe long-term disability in the United States. On average, someone dies every third minute in the United States due to stroke. The estimated cost of stroke in the US in 2005 was US$56.8 billion.
Stroke is the collective name for several conditions that can affect the brain. The most common one (85% of all cases in Sweden according to the National Stroke Register in Sweden, 2005) is a ischaemic stroke. It is caused by a blood clot blocking a blood vessel. The second most common cause is a cerebral haemorrhage, which happens when a blood vessel either inside or on the surface of the brain breaks. These conditions interrupt the blood flow and cause lack of oxygen in the brain. The brain can only cope for a short time without oxygen, before the brain cells die. This usually happens within the first few minutes and can continue for a few hours. When brain cells die they also start a chain reaction that can damage a large surrounding area of brain tissue where the blood flow is compromised but not entirely cut off. Without immediate and effective medical treatment
the affected brain cells in this area also die, causing even more damage. When brain cells die, the facility that this particular part of the brain control is lost.
Some of the most common symptoms which indicating that a stroke is occurring are paralysis, problems with speech, impaired sight and touch, dizziness and headache. Due to the stress and high pressure on people in today’s society, strokes are also becoming more common in younger age groups. High blood pressure, for instance, is a major cause of stroke.
The consequences of a stroke can vary considerably depending on what part of the brain that has been damaged. Some of the most common consequences are depression, memory problems, speech problems and half-side paralysis. There can also be hidden handicaps such as tiredness, problems with concentration, sensitivity to stress and sudden shifts in mood.
The difference between death and survival is strongly dependent up on how soon the patient gets medical assistant. The consequences of stroke can be treated in several different ways depending on how it has affected the patient. Surgery, medication, hospital care and rehabilitation are all common treatments. Frequently the patient needs one of these treatments or a contribution of a number of them in tandem. Surgery can be needed to remove blockages from the brain. Different medications can be used depending on the individual patient’s needs. Botox injections can also be used to relax one specific tensed muscle, or sedatives can be used to relax the whole body. Analgesics are also commonly used by stroke patients as a form of treatment. Hospital care for varying lengths of time is common after a stroke. Rehabilitation is often necessary for the patient so that they can function within their everyday lives.
Most of the patients’ recovery happens within the first 30 days following a stroke. This initial recovery is generally a consequence of spontaneous recovery. As previously discussed, when brain cells die that part of the brain is irrecoverably damaged. To com-pensate for this, however, many patients can learn and adapt new and different neural pathways. This process can be improved through rehabilitation. The goal of the reha-bilitation is to make the patient as independent as possible. This could mean learning new skills or relearning old ones that have been lost. It is also important to improve the person’s physical condition so as to prevent stiff joints, falls, bedsores and future strokes. Rehabilitation can make the difference between returning home or being subject to a long stay in an institution.
According to the American Heart Association, the success of stroke rehabilitation is dependent on five things;
• The extent of the brain injury • How early the rehabilitation starts • The attitude of the patient
• The skills of the rehabilitation team • The cooperation of family and friends
According to the American Heart Association it doesn’t matter how experienced the re-habilitation team is, the patient will not progress adequately without the right attitude and support from both family and friends. According to current research sources e.g. Andersson (2003) the patient normally makes good progress at the beginning of the reha-bilitation program, when there are regular meetings and practice sessions with the phys-iotherapist/occupational therapist (PT/OT). The problems often begin when the patient is supposed to continue their rehabilitation at home and on their own. Lack of motivation due to boredom and lack of support from family and friends can make the rehabilitation progress slow down. According to Broeren (2002) the use of meaningful and rewarding activities has been shown to improve the patient’s motivation to practice.
According to Andersson et al. (2003), a new system needs to be implemented to re-habilitate stroke. This is particularly urgent as a consequence of the reduction in hospital funding and the increasing queues for healthcare. Andersson et al. (2003) suggest mov-ing more of the rehabilitation to the patient’s home. To do this, the home rehabilitation exercises need to be introduced at the rehabilitation clinic, so that the patient can get used to and understand the exercises. The problem in this case is that rehabilitation at home is not as efficient as rehabilitation in a clinic working under the expertise of nursing staff. This is mainly because the rehabilitation at home is irregular and not undertaken fre-quently enough. This is mainly because the patients find the exercises boring and not stimulating, and also because the feedback from the clinic is unsatisfactory and not fre-quent enough, Andersson et al, (2003). If it was easier for patients to see their progress immediately then they would be more motivated to keep on doing their exercises.
Carr and Shephard (2003) states that it is important to involve the patient’s family and close friends in the rehabilitation process, so as to motivate the patient. If a patient has been spending a long time in hospital and has just arrived back home to his/her family, their biggest priority will probably be to spend time with their family.
It is obvious that for a stroke patient immediate survival lays in the hands of the med-ical care personnel and there is consequently a strong emphasis on the constant
develop-ment of more efficient care. According to the National Board of Health and Welfare in Sweden more special stroke clinics will soon open in Sweden where the staff are special-ized in treating stroke. Rehabilitation after a stroke also plays a major role in the patient’s ability to deal with the activity of daily living (ADL). According to Carr and Shephard (2003) fun and engaging rehabilitation exercises can make a difference to the patient whether staying in an institution or moving back home. It is clear that more needs to be done to make the rehabilitation exercises more engaging, more varied, more rewarding and fun enough to motivate the patient to do them regularly.
Another factor that needs to change, according to both Carr et al (2003) and Ander-sson et al (2003), is the patient’s attitude to rehabilitation. We suggest that an increase in involvement of the patient’s family and friends can do this. To achieve this it is also clear that the family and friends have to understand the patient’s illness and needs to support and motivate the patient to do the exercises. It would be more rewarding if the attitude of the family and the patient towards the rehabilitation incorporated an element of fun so as to avoid boredom. According to the interviews Andersson et al (2003) under-took with stroke patients most of the rehabilitation equipment currently available is very unattractive looking and is designed for repeated and boring exercises.
2.1.1 Rehabilitation exercises today
The authors of this thesis visited the rehabilitation clinic in Högsbo, Gothenburg Sweden where an OT presented different test and rehabilitation exercises. The following infor-mation and pictures date from that visit. There are several ways to test the mobility of a patient, including both the physical and the cognitive. Some of these tests are presented here:
• The Box and Block Test (BBT) is used to evaluate gross movements of the hand/arm. The test requires moving the maximum number of blocks, one by one, over the middle wall from one compartment of a box to another of equal size, all within the timeframe of one minute.
• Purdue pegboard: Place small pegs into holes on a board as quickly as possible within a 30-second period. There are several different versions of this test, all of which measure fine manual dexterity.
Fig. 2.2: The Purdue PegBoard
• The dynamometer hand grip strength called Grippit (AB Detector, Gothenburg, Sweden). This device involves gripping the black cylinder and squeezing as hard as possible for 10 seconds so as to measure the strength in the grip.
Fig. 2.3: The Grip Test
• Hidden objects: This tests the sense of touch. Different objects are hidden behind a board and the OT gives different objects to the patient. The patients has to use their tactile sense to determine what the object is.
• 3D - Tic tac toe: : Five in a row. This is a 3D practice incorporating both the cognitive and fine manipulative functions of the patients.
• Jigsaw: This combines perceptual, cognitive training and manipulation exercises. • Stick Othello: This is the traditional game Othello played with sticks. The board can
be set at different angles to accomodate different levels of difficulties. The board is initially positioned horizontally and can then be placed vertically for more difficult training. The exercise practices the mobility, manipulation and cognitive skills of the patient.
Fig. 2.5: Stick Othello
• Video and computer games: These offer many possibilities for rehabilitation. The best example is Eye Toy for Playstation 2 where a camera is used to track the move-ment of the user so as to complete different tasks, for example punching and kicking. This exercise improves the coordination and the mobility of the patient. Initially the therapist stands behind the patient and help them with their balance.
A central concept in occupational therapy is the importance of activity (Meyer 2001, Kielhofner 1995). Therefore different kinds of sports are also commonly used in rehabil-itation.
• Swimming: The sense of weightlessness in water at body temperature supports the patient in practicing their mobility and also in preventing stiffness of the joints.
• Badminton: Helps the patients to practice eye-body coordination and mobility. • Darts: Helps the patients to practice eye-body coordination and mobility, mainly in
the arm.
There are also specially designed systems to help with the rehabilitation and to practice the activities of daily living.
• The mirrored table: This specially-designed table has lots of different exercises of which all practice the fine-manipulation skills of the patient.
Fig. 2.7: The Mirror table
• 3D Blocks: Guiding blocks through a maze enhances the fine-manipulation skills of the patient.
• Living: Patients living in special apartments. The patient can practice things like cooking, baking, going to the toilet etc.
• Wood craft: making stools, nesting boxes etc. Practices fine manipulation, mobility and cognitive skills.
• Office exercises: These exercises enables better computing, document handling etc. Practices fine-manipulation and cognitive skills.
• Gardening: Gardening can involve planting seeds, etc. Practices fine manipulation, mobility and cognitive skills.
According to Carr and Shephard (2003) computer games are likely to be increasingly used in training for various aspects of upper-limb movement. Carr and Shephard (2003) state that the use of a game focuses attention on the outcome of the movement as opposed to the movement itself. The motivating effects of being an active participant on an inter-esting task may be powerful facilitators in the rehabilitative process. In this project we are trying to enhance our knowledge of the rehabilitative process through the exercises available today and creating a novel way of exercising using virtual environments and haptics. It is not our intention to provide a substitute for those exercises already in use. We hope our application can complement them. Once again we return to the main reasons for introducing virtual environments in combination with the rehabilitation process; the possibility of providing a number of alternative exercises that are engaging, stimulating and fun.
2.2 Haptics and Virtual Environments
Haptics refers to the modality of touch and associated sensory feedback. Researchers working in this area are concerned with the development, testing, and refinement of tactile and force-feedback devices and supporting software that permits users to sense ("feel") and manipulate virtual objects with respect to such features as shape, weight, surface textures and other physical properties.
The simplest devices which have force feedback and the possibility of interacting with virtual environments include such devices as game pads and joysticks. For example, var-ious sorts of steering wheels exist which give the user a more realistic sensation through using force-feedback.
A desktop stylus is a desk-mounted mechanical arm incorporating several degrees of freedom (DOF). Usually a stylus is mounted at the end of the arm but, in the case of more specific solutions a surgical drill can be used to train medical students, for example.
In this project a desktop stylus called the PHANTOM Omni has been used. It is a mechanical arm with 5 joints, that enables the user to interact with objects within a Virtual Environment.
Fig. 2.9: Phantom Omni
The Omni is tracked through six DOF (x,y,z and pitch, roll, yaw) but only gives force-feedback in three dimensions (x,y,z). Compared to some other high-end solutions, i.e. PHANTOM Desktop 1, PHANTOM Premium 2 the Omni generates a lower force and lower resolution, which decreases the accuracy when "feeling an object". In the case of rehabilitation, however, the force generated from the PHANTOM Omni is likely to be sufficient to engage with the level of the stroke patient’s weakness. Some high-end solu-tions can also give force feedback in six DOF, which may increase the tactile engagement with the objects.
A possible alternative to the PHANTOM Omni is the, soon to be released, Novint Falcon3. The interaction capabilities of this devices are the same as for the PHANTOM Omni. It uses a stylus to interact with and feel objects in a 3D environment.
Fig. 2.10: The Novint Falcon, (http://www.novint.com/falcon.htm)
The difference is that it is targeting the home market so as to be used primarily for games as opposed to within research, medical and industrial applications. At present, the
1SensAble Technologies, http://www.sensable.com/ 2SensAble Technologies, http://www.sensable.com/ 3Novint Technologies, http://www.novint.com
information about the Novint Falcon is not extensive enough to determine if it can be used with more complex applications. In Novint’s product overview the manufacturers have, as of yet, only compared the Novint Falcon with other gaming devices like joysticks and steering wheels.
There are also on-body devices available, i.e. a piece of equipment used on a specific part of the body, for example a glove. An example of a haptic glove is the CyberGrasp
4. It has force feedback applied to the fingers giving the illusion of being able to grasp
virtual objects. The tips of the fingers receive no force feedback but the device can be used in tandem with a tactile glove. The tactile feedback of this device consists of specific vibrations that send stimuli to the fingers when touching a virtual object. Another on-body device is the CyberForce 5 explained in section 2.3.1. There are also on-body
devices that can be used with feet, for example the Rutgers Ankle 6 also described in
section 2.3.1.
2.2.1 Mixed Reality versus Virtual Reality
From our research and reading it appears that Virtual Reality (VR) is sometimes misused as a collective name for all things involving 3D computer graphics. To describe the system used in this project, definitions by Milgram et al 1994, are useful. He stated that a virtual reality environment is a synthetic world in which the user is totally immersed and has access to interaction abilities. The computer-generated environments used in this project only have partial immersion and therefore merge the real world with the virtual one. Milgram called this process Mixed Reality (MR) and uses a scale for defining how virtual and/or real an environment is. The scale is called the Virtuality Continuum (VC) and can be used to describe the degree of computer-generated stimuli within the particular environment, see figure 2.11, below.
Fig. 2.11: The Virtuality Continuum (Milgram et al. 1994, (c)IEICE)
4Immersion Corporation, http://www.immersion.com/3d/ 5Immersion Corporation, http://www.immersion.com/3d/ 6http://www.caip.rutgers.edu/vrlab/projects/ankle/ankle.html
Reality is illustrated to the left of the Virtuality Continuum, Virtual Reality to the right and Mixed Reality is situated in the middle. In the area of MR, Milgram gives two different definitions: Augmented Reality involves enhancing a real-time environment with computer graphics whereas Augmented Virtuality is a synthetic world that uses real textures or video streams as a part of the environment. These various definitions, as illustrated in figure 2.11, can be used to describe the degree of immersion.
The VC is based on different display systems. For some systems it can be more ef-fective to describe the boundaries between virtual and real environments and how trans-parent they are. These issues are discussed by Benford et al. (1998) and are based on the VC. They introduce a classification of environments according to the dimensions of artificiality (computer-generated stimuli) and transportation (immersion). The classifica-tion is used for shared spaces but can be applied to mixed and virtual realities generally. Figure 2.12 is a graphical representation of these two dimensions. All these definitions can be used as an aid to understanding the different properties when using haptic devices. Haptic systems with different boundaries are described in the following section.
2.2.2 Haptics and Reality - Virtuality boundaries
As described in previous section the degree of computer-generated stimuli determines where to place a certain environment along the Virtuality Continuum. When using a haptic device within a particular system other stimuli from the computer-generated en-vironment can also be received and felt: i.e. the sense of touch. This aspect of the haptic device increases the degree of immersion. Using the definitions of Benford et al. (1998), it is also possible to classify different haptic systems according to their degrees of immersion. In Picture 2.13 below, the following haptic systems are classified.
Fig. 2.13: The classification of some haptic systems
In the Immersion Haptic Workstation, two CyberGrasps are connected to mechanical arms giving the lower arm and hand force-feedback ability. The display-system used is a head mounted display (HMD). This system fulfils the definition of virtual reality with the HMD and the haptic device creating an immersive environment incorporating realistic interaction.
Fig. 2.14: Immersion Haptic Workstation, (http://www.immersion.com, permission requested)
With the Immersive Workbench a mirrored glass reflects the screen in front of the user which makes the physical haptic device and the virtual environment co-located. Be-cause the users hand is being used within an intuitive environment in combination with a stereoscopic display, makes the environment highly immersive.
Fig. 2.15: Immersive Workbench, (http://www.sensegraphics.se/products.html)
Example of a third possible setup is the use of a normal screen, a PHANTOM Omni placed beside it and a web camera tracking the head-movement. When the user’s head moves, the scenery in the virtual environment changes according to the head movements. We have attempted to prototype the Gibson approach, explained in the following section 2.2.2 - "Display systems to decrease boundaries". The implementation of it is described in section 4.3- Implementation of "Ecological Approach to Human Perception".
Fig. 2.16: Screen, PHANTOM Omni and headtracking
Another possible setup is the use of a normal screen and a PHANTOM Omni placed beside it. The movement of the Omni corresponds to the movement of the virtual stylus but is separated in space making the degree of immersion decreasing in relation to the immersive workbench. The lack of stereoscopic representation affects the sense of reality and also affects the navigation in space.
Finally a very simple possibility is the use of a screen and a joypad. To enhance the interactive experience when playing video- and computer games most of the joypads today have in-built vibrations. For example when the user drives a car into a wall or is hit by a monster, the joypad vibrates. Even if the feedback itself is some distance from the "real" experience, the degree of immersion is nevertheless increased in most cases.
Fig. 2.18: A joy pad with in-built force-feedback
Display systems to decrease boundaries
To get a more realistic feeling and increase the immersion a stereoscopic display can be used to create a sense of depth within a virtual environment. Two images of the scenery/environment can be displayed slightly apart. Then shutter glasses can be used to make the user believe he/she is seeing the scenery in stereo. This resembles how the human perception works in relation to short distances, and allows us to get a sense of depth in space and "know" how far away objects are.
In haptic systems using a desktop stylus the screen is generally displaying a 2D image describing a 3D environment. Without a stereoscopic display it can be very difficult for a user to determine the depth relations between different objects. As part of a haptic system it is necessary for some sort of depth information to be provided to enable the user to navigate with the stylus. Stereoscopic display systems are often used to solve this problem.
Another way to determine how far away objects really are is described by Gibson (1977). When a human moves their head the distances between visible objects are de-termined by how much the objects move in relation to one another and their overlapping edges. Also the texture of an object is perceived differently depending on the distance affecting the understanding of a particular 3D spatial distribution. For example, the fol-lowing scenario occurs if you happen to be looking at a plant that is two metres in front of you and behind the plant there is a lamp which is positioned ten metres away. When you
move your head the plant will appear to move more as compared with the lamp because the plant is nearer to the observer. The lamp will also appear to be more blurred whereas the plant will appear in a sharper detail. The method described here is quite unusual in computer graphics but has the potential to be used in applications which allow the user to see almost 3D-spatial distribution. We have attempted to prototype this method which is described in section 4.3- Implementation of "Ecological Approach to Human Perception".
2.3 Rehabilitation and haptics
With the new technology presented in previous section, there are a number of new pos-sibilities available in the field for rehabilitation of stroke patients. This technology has been used already for a number of years in various ways to rehabilitate different forms of physical weaknesses and paralysis. Numerous forms of haptic devices have been used as an input for a range of computer applications of rehabilitative technology.
The motivation for research in this area is often one or more of the following:
• To make the training more fun and engaging • To monitor progress more easily
• To permit the use of guided pathways
• To allow more effective training with a range of advanced equipment • To practice everyday skills with simulated objects
Today, haptics and rehabilitation is still at the research stage for the most part but the technology is becoming less expensive and as more patients are in need of effective training systems, this is a subject area which is increasing in importance.
2.3.1 Previous work
An advanced home rehabilitation systems incorporating both virtual reality and haptics has been proposed by Rydmark et al. (2005). In this system the patient’s home-based workstation is connected to a rehabilitation unit consisting of a game server, a patient database and a patient management system. Through the database system a PT/OT can access information concerning motion capture data (movement of the haptic device) and using both data visualization and text output it is possible to analyze the work done by the patient while playing the game. The PT/OT can then evaluate the progress of the patient’s rehabilitation and adjust the parameters of the game using the relevant patient
data. System updates (game updates, new games, etc.) are carried out by the system developers when necessary. The benefit of this system is that new games may frequently be added and the PT/OT receives instant feedback on the patient’s progress. The haptic device Rydmark et al use is the PHANTOM Omni. It is used for rehabilitation of hemi-plega (paralysis) in both the left and right upper-body. It also includes examples of games that are as yet very basic in nature, see next section.
In McLaughlin et al. (2005), some new and different aspects are introduced to the process of rehabilitation. They present an algorithm which anticipates incorrect body movements. This algorithm also enables the recording of activities and the use of a spoken dialog system, a "snap-to"-system and various Internet-based mutual systems. A game is also presented entitled the "Space Tube" and judging from the report and pictures it functions as a reasonable simple but engaging rehabilitation application.
Another method used for post-stroke rehabilitation is "The Rutgers ankle" (Boian et al. (2002)).This involves controlling a computer game with a haptic device strapped to the foot. By moving the ankle an airplane or boat is guided on-screen and the built-in-resistance helps rehabilitate the control mechanism of both the foot and the lower leg muscles. It is also worth noting that efforts have been made to create more engaging games by using force-feedback. For example, when the airplane bumps into something, as part of the game previously mentioned, the device gives the user feedback. No cost information is provided in the relation to this game but it appears to be an expensive and complicated system as compared with the PHANTOM.
An example of another device that is used for rehabilitation is the CyberGlove (Adamovich et al (2005)). It exercises various finger movements and increases strength. This system is complex and specially designed for the sole purpose of hand rehabilitation.
2.3.2 Previous haptic games
SenseGraphics is a haptic software development and hardware provider and reseller com-pany based in Stockholm, Sweden7. They have previously made some game prototypes
for stroke rehabilitation which also use haptic devices. These games are still in the de-velopment stage, yet exhibit significant potential for the future dede-velopment of haptics exercises. Some of these are:
• Simon: The scenario in this game consists of a number of separated planes. Each of the planes corresponds to a musical note and a musical sequence is initially played. Then the user is required to replay that same sequence by touching the correspond-ing panel.
• Sling: The user grabs a ball with the haptic device and is supposed to throw it towards a target. The closer to the centre of the target the ball hits the higher score the user’s. Between the ball and the virtual stylus there is a spring force, which is applied to the haptic device.
• Memory: This is the classic memory game where the goal is to find two correspond-ing figures hidden behind a number of planes. The haptic device is used to choose between the different cards.
In this chapter present a set of general guidelines that we have created based on our work during this thesis. Then we present what we believe are some possible games/genres that can be used for haptic games. After that we describe a couple of our game designs and their possibilities created during the process but not yet implemented.
To meet a patient’s needs, Rydmark et al. (2005) suggest that several games ought to be developed and made available. As an aid to the future development of haptic ex-ercises for stroke patients we have developed a set of potential design guidelines. These guidelines address several factors that need to be taken into account when designing an exercise for stroke patients using a force feedback-based interaction tool. These guide-lines are user-driven as regards their design focus. The most important outcome for the patients is improvement of their physical and mental health. This means that the en-tire application design and implementation must be informed by the expertise of doctors, physiotherapists and occupational therapists. A good start is to motivate the patient to begin to use their impaired arm as much as possible again, and that is our aim with these game designs. A close collaboration between medical experts, designers and developers is crucial so as to achieve an efficient and effective exercise.
3.1 Design guidelines for haptic exercises
The set of guidelines presented in this section is a product of six months of literature stud-ies, informal interviews with doctors, a PT and an OT and several studies with different users. This is our final outcome that we have created as an aid for future development of haptic games for stroke patients based on our experiences during the project. Our guide-lines provide information on how to potentially create a successful exercise in terms of both game design and interface design. Examples of how to implement these guidelines are provided with our application The Labyrinth, as described in section 4.2.2
3.1.1 Game design
• Rewarding: When working with stroke patients it is very important to remember that their primary concern is the improvement of their health. The rate of
improve-ment will inevitably affect the rest of their lives. The exercise therefore needs to be designed so as to encourage the user to keep trying and improve the speed of their recovery. A system which includes an efficient reward system has a better chance of keeping the patients interest and motivation at a high level. There are a number of ways to enhance the patient’s motivation:
– Use a generous score system: Always end up with a good score, with only a few minus points and many plus points. The user feels a sense of accomplishment. They feel that they are making progress.
– Provide the patient with instant feedback of higher improvement (ex. high score list, score bar). Also provide the exercise with regular status-feedback during the game play, so as to highlight the score when it changes.
– Design the exercise so that the patients can compete against themselves (points, time etc.). The possibility of competing against others should be very carefully considerated before such a function is implemented. It might be advised to avoid unwanted comparison between individuals as there can result in feelings of inadequacy on the part of some patients.
• Possibilities for control: As the needs of each patient vary and change in tandem with their progression it is therefore important to ensure that the exercise has ad-justable parameters (for PT/OT and in later stages for the patient, himself/herself)
– System difficulty needs to be adjustable so as to keep the patients interest and to match the abilities of each individual user. An adjustable system allows the game to be very simple in the beginning and then gradually increase in difficulty (OT/patient).
– Control of movement (PT/OT). If possible allow the OT to fine-tune the pa-tient’s movement with special exercise parameters.
– Control of the type of training (PT/OT). Parameter to be discussed with PT/OT. – Control of intensiveness of training (PT/OT). Parameter to be discussed with
PT/OT.
– Provide the exercise with a data-storage facility for PT/OT; so that the move-ment, the score, the number of times that the exercise was used etc. can be monitored.
• Engaging:
– Vary the game play (especially if the movement is repetitious!). (changes in level design, bonuses, different environments etc.)
– If creating exercises that do not comprise of a specific game, use meaningful ADL-exercises and combine them with suitable design guidelines. For example cutting bread within a certain time and receiving a certain score.
• "Easy to use":
– The task should be fairly easy to understand; intuitive game play. – Provide as few explanations (text) as possible.
– Crucial to avoid all bugs which can frustrate the user. If the patients get frus-trated with the exercise they are likely to give up.
3.1.2 Interface design (Input/Output)
• Haptics:
– Design exercises to enhance the potential of the haptic device.
– Make use of particular physical properties, so as to ensure efficient training and so as to make the exercise more interesting. For example, objects can be smaller, heavier (i.e. lifting boxes with different weights).
– Control movement with haptic obstacles placed in the direction of movement so as to force the patient to do larger or different movements (for example the use of grids).
– Avoid all possibilities of cheating!
– Make the patient aware of when he/she is doing "wrong".
– Consider all possibilities for a range of different arm movements. Reaching (backwards, forwards, sideways) and different types of manipulation. Large and small movements.
– Avoid using the buttons except for advanced patients. The buttons have a par-ticular potential when used for training that incorporates finer manipulation. – Consider the possibilities for using guided pathways. Using volume haptics
(Lundin et al 2005), invisible paths are possible, path which enable the patient to feel their way of movement through the haptic device. This works as a re-pelling magnet on which the stylus floats. This can be set to different strengths and at different parameters.
• Audio:
– Use disapproving sounds when doing "wrong" and encouraging sounds when doing right.
– Consider the possibility of using dynamic sounds; doing better leads to more pleasant sounds.
– Use background music to increase the overall experience. Try to make it as soft and harmonic as possible so as to avoid unnecessary stress.
• Visual:
– Try to create interesting and "pleasant" environments. As this changes from person to person a useful guideline is to use textures that will help the user for-get about the game as an exercise and be less aware of the surrounding clinical environment.
– Bright neutral colours.
– Highlight with colour when touching an object and highlight also when inter-acting with objects and events. Try to give regular information to the patient so that it is a natural part of the exercise.
– If using a stereoscopic display the navigation in 3D should come fairly easily to the user. If the game is designed for a normal screen a way to facilitate space orientation is the use of reference points. The orientation in two dimensions (x, y) is fairly simple for the user, so it is only necessary to have the markers which describe position-in-depth (z) are needed. An example of the use of reference points is the projection of guidelines from the tip of the pen in both (x)-directions thereby enabling the user to see where the pen is in reference to the surrounding objects.
3.2 Game suggestions
Currently there are relatively few games that have been developed for haptic desktop devices and it is nearly always in the context of research that a haptic system’s abilities are tested. When creating a game for stroke patients different considerations need to be taken into account compared with the way games are developed in the traditional game industry (video and computer games). Yet there are many similarities between the ways a haptic exercise and a game attempts to grab the attention of the user, i.e. fun, stimulating and by being visually soothing. The way these games are constructed can be used as an inspiration as long as no copyrights are violated. Another challenge again is to design a game that fully uses the physical possibilities of the haptic device. In this section we give some examples of genres/games that could be stimulating for a stroke patient.
A game that would be understandable to almost anyone is a traditional card game -for example Solitaire. The idea is very simple and yet can intrigue a player -for many
hours. One version of implementing this game would require the patient to move the cards to their correct location by picking a card with the haptic pen and then moving it to a specified location (according to Solitaire rules). Different weights can be applied to the cards thereby giving the patient varied types of training. A 3D-grid with different heights between the cards can be used to enforce the user performs different movements. Though, haptic elements that may feel awkward in a gaming situation, a grid, for example, needs to be added to ensure adequate levels of exercise.
Fig. 3.1: The card game Solitaire, ( (c) 1981-1999 Microsoft CORP., permission requested)
As for the card games which exist, there are many simple puzzle games that can be implemented for the use of rehabilitation. An example is a game consisting of many different symbols divided by a 2D-grid, see picture 3.2 below. A symbol can change place with one of its neighbours as symbols and the idea of the game is to move symbols so that they make up three in a row or more. These selections can be made with the haptic pen and haptics can be used to create a force opposite to the moving direction. The grid can then be set to a certain height (making it a 3D-grid instead) forcing the patient to move above it. This is just one example of the thousands of many small games that can be implemented for the purpose of rehabilitation.
Another easy and suitable game which can me used for a haptics device is a version of the classic video game Pong, entitled 3D Pong. It is a game that can be described as a form of tennis in space without a net. The user controls a racket to hit a ball and plays against a player/computer who controls the other racket. If the ball passes a racket (a player) and goes off the court the other player wins a point, similar to tennis. This game is easy to understand and although it is a simple game idea it has a long lifetime, especially if you are playing against a friend or another patient. It’s possible to design this game with relatively simple primitives. Some people can have problems getting a sense of the 3D environment, however.
Fig. 3.3: 3D Pong
Another possible approach is to create exercises based on activities of daily living(ADL). These exercises are constructed to resemble activities that are necessary to cope within ordinary everyday situations. Of course real-life scenarios are best done in real lives but sometimes it can be convenient and even more fun to practice them as an exercise in a mixed reality environment. For example, buttering bread as quickly as possible without wasting any bread and making it possible to monitor the progress of the action. Another consideration is that it sometimes is safer for the patients to do the exercise in a virtual world, where for example the cup will not really break and hurt them if they drop it or they can practice cutting bread without a real sharp knife. They can also get help through the guided pathways mentioned already.
There are also a number of traditional rehabilitation exercises that can be adapted to a haptic application. For example, the practise of moving the hand following a spiral line in an outgoing circle. The benefit with haptics- approach is the possibility of adding gaming elements and making the exercise more engaging. It is also very easy to record the movement of the Phantom and observe the patient’s progress.
The haptic arm can also be used to test the arm abilities of a particular patient. Many of the tests in section 2.1.1 can be implemented within a haptic system and the movements, strength, times etc. can be stored for both the patient’s own observation use and for general evaluations.
3.3 Two more detailed game ideas
The designs presented in this section are our own work based on the knowledge we ob-tained during the initial stages of the project. We have also discussed these game designs together with Mr. G (see section 5.1) and a PT.
3.3.1 MugMasterMind
The user of MugMasterMind must be able to move the arm and use the hand to grip things, e.g. hold the pen, but does not need to have fine manipulation skills. The game idea is built on the classic game known as Master Mind. Here the user moves cups of different colours from a counter to a shelf and places them in the order that the user believe the computer has chosen. The computer then gives the user feedback on how many cups are correctly coloured and how many are correctly placed or not.
Fig. 3.4: Design idea of MugMasterMind
Initially there are only few colours to choose from. Each time you proceed to a new level the level of difficulty increases. For example, the number of different coloured mugs might increase or the number of mugs ought to be sorted. The shelves can also become taller.
When the PHANTOM stylus is close to a mug the mug will "magnetically" attach to the pen. The mug will then stay attached until it touches the shelf. A variation on this designed for the user with better manipulation skills, who can use the button on the pen to pick up and drop the mug. Another variation on picking up the mug that demands slightly more manipulation involves having a hole in the mug where the user sticks the pen to so as lift the mug. When the pen is in the hole the mug is attached to the pen by either a magnetic force or by pressing the button. The weight of the mug can be set depending on the strength of the patient. Initially the mugs can be weightless. Guided pathways can also be used to guide the user from the counter to the closest shelf.
3.3.2 The Ballroom design
This exercise requires a more advanced user. The patient has to have fairly good coordi-nation skills and be tolerant to a certain amount of stress because of the moving obstacles. It is an exercise suitable for users who have less impairment of their arm.
The ballroom exercise consists of a playing field and different coloured goal areas which are separated by walls. Through hatches in the walls or the ground balls with different colours are released into the playing field. The balls are released with different frequency in accordance with the difficulty-level, there by giving different amounts of balls for the user to handle. A ball of certain colour is supposed to be picked up using the button or the power of the magnetic forces depending on user abilities and it can then be moved to the goal area of the corresponding colour and released. If the ball lands on the goal area a certain score is achieved and then the user continues on with the rest of the balls. If a ball stays too long on the playing field without being picked up minus points are allocated to the user and it continues like this until that ball is scored.
The force feedback effect is used very efficiently in the ballroom game design. The walls separating the goal areas and the play field can be set to different heights. It is impossible to move through the walls so the user is enforced to go above them. With the height of the walls the movement of the user can be controlled; i.e. a higher wall forces the patient to undertake larger movements. The balls are dynamic objects and can be set to different weights and sizes. By increasing the weight of the balls they get heavier and thereby provide for more training for the user who moves them from the playing field to the goal area. As the sizes of the balls and playing area can be changed, the game’s level of difficulty can be set to give a more difficult or intriguing game.
The playing field can be designed by the OT/PT to create an exercise that is custom-made for each individual user. This also makes it possible to create endless variations of the playfield. The OT/PT does not have to know any particular coding because the playfield can be created from coordinates specified within a text file and similar to the technique described in section 4.2.2.
By setting the position of the goal areas in relation to the playing field the various movements can be controlled. The playing field can be set low and the goal areas high in the air to get one type of movement. Another example is to have the playing field near the user and then the goals further back to get a different type of movement. This gives the patient different types of arm rehabilitation using the same game-idea. As described in our guidelines these designs should be done in advice in conjunction with or by an OT/PT.
One of the main goals of this project was to design and develop a haptic exercise, which would motivate the user to begin to use their impaired arm again. In this chapter we describe the tools used and the result of our implementations. The two games presented here have been in constant development during the whole process and particularly "The Labyrinth" has been the core of our studies and functioned as an essential aid in the creation of our design guidelines.
4.1 Implementation tools and methods
The development and programming of an application is generally a time-consuming pro-cess and as the time-constraints of this project were limited it was nepro-cessary to allocate extra time to explore product-usability. To be able to create test applications in a fast, easy and intuitive way an efficient software toolkit had to be employed. Today there are several toolkits available to do this. Very brief descriptions of some of these products include:
• Open Haptics toolkit, Sensable Technologies:
The Open Haptics toolkit is used to develop low level applications in C++. It is patterned after the OpenGL API and can be integrated with already existing OpenGL applications. This makes it very useful for graphics programmers with a knowledge of OpenGL.
• GHOST SDK, Sensable Technologies:
GHOST SDK is based on Open Haptics but is easier to use during the development process because it is based on a scene graph structure. It has predefined objects and touch effects but the programmer has to do all the graphical rendering manually.
• Reachin API, Reachin Technologies:
The Reachin API is a scene graph just like GHOST SDK but it takes care of all re-quired graphical rendering. It uses the language VRML for the scene graph structure and the scripts which handle events are written in C++ or Python.
