Department of Science and Technology Institutionen för teknik och naturvetenskap
Linköping University Linköpings universitet
g n i p ö k r r o N 4 7 1 0 6 n e d e w S , g n i p ö k r r o N 4 7 1 0 6 -E S
LIU-ITN-TEK-A--15/020--SE
The Impact of Augmented Reality
Support in Warehouse Trucks
Mikael Pettersson
Martin Stengård
LIU-ITN-TEK-A--15/020--SE
The Impact of Augmented Reality
Support in Warehouse Trucks
Examensarbete utfört i Medieteknik
vid Tekniska högskolan vid
Linköpings universitet
Mikael Pettersson
Martin Stengård
Handledare Ali Samini
Examinator Karljohan Lundin Palmerius
Upphovsrätt
Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare –
under en längre tid från publiceringsdatum under förutsättning att inga
extra-ordinära omständigheter uppstår.
Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner,
skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för
ickekommersiell forskning och för undervisning. Överföring av upphovsrätten
vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av
dokumentet kräver upphovsmannens medgivande. För att garantera äktheten,
säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ
art.
Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i
den omfattning som god sed kräver vid användning av dokumentet på ovan
beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan
form eller i sådant sammanhang som är kränkande för upphovsmannens litterära
eller konstnärliga anseende eller egenart.
För ytterligare information om Linköping University Electronic Press se
förlagets hemsida
http://www.ep.liu.se/
Copyright
The publishers will keep this document online on the Internet - or its possible
replacement - for a considerable time from the date of publication barring
exceptional circumstances.
The online availability of the document implies a permanent permission for
anyone to read, to download, to print out single copies for your own use and to
use it unchanged for any non-commercial research and educational purpose.
Subsequent transfers of copyright cannot revoke this permission. All other uses
of the document are conditional on the consent of the copyright owner. The
publisher has taken technical and administrative measures to assure authenticity,
security and accessibility.
According to intellectual property law the author has the right to be
mentioned when his/her work is accessed as described above and to be protected
against infringement.
For additional information about the Linköping University Electronic Press
and its procedures for publication and for assurance of document integrity,
please refer to its WWW home page:
http://www.ep.liu.se/
Master Thesis
TruckAR - The Impact of Augmented
Reality Support in Warehouse Trucks
Authors:
MikaelPettersson and Martin Steng˚ard
Supervisors: Ali Samini Link¨oping University BorisAhnberg Toyota Material Handling Europe
Examiner:
Karl-Johan Lundin Palmerius
in the
Department of Science and Technology
LINK ¨OPING UNIVERSITY
Abstract
Institute of Technology
Department of Science and Technology Master of Science in Media Technology
TruckAR - The Impact of Augmented Reality Support in Warehouse Trucks by Mikael Pettersson and Martin Steng˚ard
This Master Thesis analyzes how augmented reality (AR) can be used as assistance for a warehouse truck driver in order to make the driver more safe and efficient. The work-ing environment is observed in an everyday warehouse and solutions are developed for situations based on observations. The AR solutions for the situations will be developed and evaluated. The software will be made for a pair of android see-through AR-glasses. Based on user tests and internal evaluation, the solutions will be evaluated for usage in the industry today or in the future.
Contents
Abstract i
List of Figures v List of Tables vii Abbreviations viii 1 Introduction 1 1.1 Background . . . 1 1.2 Problem description . . . 2 1.3 Purpose . . . 3 1.4 Research questions . . . 3 1.5 Method . . . 4 1.6 Limitations . . . 4 1.7 Hardware requirements . . . 5 2 Background 6 2.1 Augmented reality . . . 6 2.2 Kansei engineering . . . 7 2.3 Software . . . 7 2.4 TruckAR overview . . . 8 2.4.1 Iterative approach . . . 8
2.5 Related works and technology . . . 9
3 Situation analysis 11 3.1 Decision of situations to solve . . . 12
3.1.1 First situation filtering process . . . 12
3.1.2 Second situation filtering process . . . 13
3.2 Warehouse management . . . 14
3.3 Result from situation analysis . . . 15
3.3.1 Loading and unloading . . . 15
3.3.2 Transportation . . . 16
3.3.3 Intersection . . . 16
3.3.4 Displays . . . 17
4 Concept generation 18 4.1 Concept generation methods . . . 18
Contents iii
4.2 Concepts . . . 19
4.2.1 Concepts for loading and unloading . . . 19
4.2.2 Concepts for transportation . . . 21
4.2.3 Concepts for intersections . . . 22
4.2.4 Concepts for displays . . . 23
5 Methods 25 5.1 Tracking . . . 25
5.1.1 Camera tracking . . . 25
5.1.2 Marker tracking . . . 26
5.2 Stereo rendering and eye separation . . . 27
5.3 Catmull-rom splines . . . 29
5.4 Distance independent size of objects . . . 30
5.5 Billboard . . . 30
5.6 Turning radius . . . 31
5.7 A* search algorithm . . . 33
6 User testing 35 6.1 First user test . . . 35
6.1.1 User test for loading and unloading . . . 36
6.1.2 User test for intersections . . . 39
6.1.3 User test for displays . . . 40
6.1.4 User test for transportation . . . 41
6.1.5 Summary of first user test . . . 43
6.2 Second user test . . . 44
6.2.1 User test for loading and unloading . . . 44
6.2.2 User test for intersections . . . 46
6.2.3 User test for transportation . . . 47
6.2.4 User test for displays . . . 49
6.2.5 Summary of second user test . . . 49
7 Results 50 7.1 Loading and unloading . . . 50
7.1.1 Follow path . . . 50 7.1.2 Hologram of truck . . . 51 7.1.3 Tooltip . . . 51 7.1.4 Stripped tooltip . . . 53 7.1.5 Death star . . . 53 7.1.6 Safe zone . . . 54 7.2 Transportation . . . 55
7.2.1 Hansel & Gretel . . . 55
7.2.2 Extended pallet tooltip & pallet tooltip . . . 56
7.2.3 Hologram map . . . 56
7.3 Intersections . . . 57
7.3.1 X-Ray tooltip . . . 57
7.3.2 X-Ray . . . 58
Contents iv
7.4 Displays . . . 60
7.4.1 Virtual HUD . . . 60
7.4.2 Marker based displays . . . 60
7.5 Discarded concepts . . . 61
Spirit Level . . . 61
Eagle eye . . . 61
Environmental overlay . . . 61
Red riding hood . . . 62
Terminator . . . 62 8 Discussion 63 8.1 Design decisions . . . 63 8.2 Limitations . . . 63 8.3 Epson BT-200 . . . 64 8.3.1 Hardware . . . 64 8.3.2 Viewing angle . . . 64
8.3.3 Future of mobile hardware . . . 65
8.4 Use of Augmented Reality . . . 65
8.5 Use of Unity3D and Metaio SDK . . . 66
8.6 User testing . . . 66
8.7 Result discussion . . . 67
8.8 Considered techniques . . . 67
8.8.1 SLAM . . . 67
9 Conclusion and future work 69 9.1 Conclusion . . . 69
9.2 Can Augmented Reality be used to assist in truck navigation tasks in a typical Warehouse without distracting the user from normal activities during such tasks, and in that case how? . . . 70
9.3 Some drivers have a hard time aligning the truck against the load, es-pecially in confined spaces. How can this process of picking pallets be supported by using Augmented Reality? . . . 71
9.4 How can Augmented Reality combined with an all knowing system be used in order to make each driver more aware of potential upcoming traffic situations? . . . 72
9.5 How can warehouse truck information be shown to the driver using AR? How should this be presented in a way that does not distract the driver and can replace the current displays and their features in the trucks? . . . 72
9.6 Future work . . . 73
List of Figures
1.1 An EUR-pallet and its dimensions. . . 2
1.2 Epson Moverio BT-200 and its accompanied external control unit. . . 5
3.1 The affinity diagram that was created. The top row of words is a container for that column of words. . . 13
3.2 Bt-reflex, the truck model used in this thesis . . . 15
4.1 Concept sketches for loading and unloading . . . 20
4.2 Concept sketches for Transportation . . . 21
4.3 Concept sketches for Intersections . . . 22
4.4 Concept sketches for Displays . . . 23
5.1 An ID-marker. X and Y are depicted, whereas Z is straight out from the marker. . . 26
5.2 View through glasses. . . 28
5.3 Marker-to-Glasses transformation. T1 is the transformation between the tracker and the camera, T2, T3are the right and left screen transformations relative to the camera position. . . 28
5.4 Image of a curved line made using catmull rom algorithm. p0 is the starting point, p1the control point and p2the end point. D is the distance between marker and user. . . 29
5.5 Illustration of how the distance independent size of objects works. Alter-ing the distance of the camera to the object has no impact of the size of the object on the screen. . . 30
5.6 Illustration of how billboarding works. . . 31
5.7 Overview of how the rotation radius is calculated as explained in 5.6 . . . 33
5.8 The representation of the testing environment from above. The leftmost red circle represents the start position of the test driver. . . 34
6.1 Set-up for evaluation of displays . . . 40
7.1 Result of follow path concept . . . 51
7.2 Result of hologram of truck concept . . . 52
7.3 Result of hologram of truck concept when zoomed in . . . 52
7.4 Result of Tooltip concept . . . 53
7.5 Result of Stripped tooltip concept . . . 53
7.6 Result of Death star concept . . . 54
7.7 Result of Safe zone concept . . . 54
7.8 Result of Safe zone concept. Showing how the color of the zone change when the user are closer to the truck . . . 55
List of Figures vi
7.9 Result of the Hansel & Gretel concept . . . 55
7.10 Result of the Pallet tooltip concept . . . 56
7.11 Result of the Hologram Map concept . . . 57
7.12 Result of the X-ray tooltip concept . . . 58
7.13 Result of the X-Ray concept . . . 59
7.14 Result of the Green Wave concept . . . 59
7.15 Result of the virtual HUD concept . . . 60
List of Tables
6.1 Answer from question about how often drivers drive close to each other. 1 - never, 5 - very often. . . 37 6.2 Answer from question about how often drivers drive close to each other . 37 6.3 Answer from the question if the drivers thought Follow path would help
to navigate in confined spaces. * - No, I prefer to drive by my senses . . . 38 6.4 Answer from the question if the drivers thought Hologram of truck would
help to align and collect pallets. . . 38 6.5 Answer from the question how easy it is to pick the correct pallet today
compared to with the concepts. 1 hard, always pick wrong pallet, 5 -easy, never pick the wrong pallet . . . 38 6.6 Answer from question about if the environmental overlay is enough. . . . 40 6.7 Answer from a question about the Marker based displays concept. 1
-completely irrelevant, 5 - -completely relevant . . . 41 6.8 Answer from a question about the Pallet tooltip concept. 1 - much worse,
5 -much better . . . 42 6.9 Answer from a question about navigation . . . 42 6.10 Answer from a question about Hansel & Gretel and Hologram Map 1
-not useful, 5 - very useful . . . 43 6.11 Answer from personal information. Driving experience in years . . . 44 6.12 Answers from how usable a concept is. . . 46 6.13 Answer from question if the conccept will make the driver work more
efficient. . . 46 6.14 Answer from the question if the driver think that a concept will help to
prevent accidents. . . 46 6.15 Answer from usefulness of Intersection concepts. 1 - not useful, 5 - very
useful . . . 47 6.16 Answer from accident prevention for the Intersection concepts. . . 47 6.17 Answer from usefulness of Transportation concepts. 1 not useful, 5
-very useful . . . 48 6.18 Answer about efficiency of Transportation concepts. . . 48 6.19 Answer from usefulness of Display concepts. 1 - not useful, 5 - very useful 49
Abbreviations
AR Augmented Reality VR Virtual Reality
HMD Head Mounted Display FOV Field Of View
GPS Global Positioning System WMS Warehouse Management System TMHE Toyota Material Handling Europe SLAM Simultaneous Localization And Mapping HUD Heads-Up Display
DOF Degrees Of Freedom
Chapter 1
Introduction
In this chapter, the background of the thesis and TMHE (Toyota Material Handling Europe) are described. This master thesis tries to answer four questions which can be read more about in this chapter. Hardware and software requirements are also brought up.
1.1
Background
There are often many warehouse trucks present in any major warehouse or industry. Traffic rules vary between warehouses, but are usually non-existent because they disrupt the work-flow of the drivers too much. Instead drivers rely on eye contact and a general understanding of the environment. While this necessarily does not create any incidents, improvements can still be made in order to make the drivers feel more safe and work more efficiently.
This is a struggle, as an efficient driver is a fast driver. The driver of a warehouse truck needs to be quick at lifting, transporting and lowering the pallets, see Figure 1.1. There are different types of warehouse management systems that aids the driver with handling the administrative logistics of the pallets. The information from the warehouse management system is typically taken from computer systems or printed out on paper. TMHE in Mj¨olby, Sweden, develop and produce almost 60 000 warehouse truck units per year. The factory in Mj¨olby has a workspace of 76000 m2
and 1850 employees which 1
Chapter 1. Introduction 2 makes it one of the largest production of material handling products in the world. They are investing a lot of money in research and development and want to use new technology in their products to make a better experience.
Today there are around 190 registered drivers of warehouse trucks at TMHE. They receive and put pallets into storage, move pallets around and finally transport functional warehouse trucks out of the factory. This makes an intense environment for the drivers, and creates a lot of different scenarios each day.
This master thesis will look into how the effectiveness, safety and productivity of the drivers can be increased with the support from Augmented Reality and how this tech-nique can be implemented.
Figure 1.1: An EUR-pallet and its dimensions.
1.2
Problem description
The field of Augmented Reality is wide and growing. The concept of AR has been around for long, see Chapter 2, but the technology has not been mature for simultaneous tracking and rendering up until recently. This thesis aims to tell in what way AR can be utilized in order to help the warehouse truck driver be safer and more efficient. The thesis aims to answer what kind of situations of normal work-flow AR can be implemented in. It should also answer how solutions should be implemented for these situations.
Chapter 1. Introduction 3 The thesis is based on existing technology and research, made by researchers in the field. The theory behind state-of-the-art techniques will be fit with the current generation of drivers. One interesting aspect is in what the truck drivers will think of AR in their workday. It is also interesting to see what this will lead to. Is the technology mature enough and ready to be implemented in everyday warehouse work-flow? If not, when can it be estimated to be mature enough?
1.3
Purpose
The purpose of this thesis is to see how AR can be utilized to make a truck driver work more effective and to be safer. The situations a truck driver faces and how AR can be applied in order to solve them is an aim of this thesis.
The current technology for AR will be studied and evaluated by how likely it is that it can be used in everyday work in a warehouse. What happens in the future is also interesting, and will be taken into account while evaluating the technology.
1.4
Research questions
The project is divided into four main question that will be answered.
• Can Augmented Reality be used to assist in truck navigation tasks in a typical Warehouse without distracting the user from normal activities during such tasks, and in that case how?
• Some drivers have a hard time aligning the truck against the load, especially in confined spaces. How can this process of picking pallets be supported by using Augmented Reality?
• How can Augmented Reality combined with an all knowing system (see Section1.6) be used in order to make each driver more aware of potential upcoming traffic situations?
Chapter 1. Introduction 4 • How can warehouse truck information be shown to the driver using AR? How should this be presented in a way that does not distract the driver and can replace the current displays and their features in the trucks?
1.5
Method
A pre-study will be made to gain experience of how truck drivers operate and to un-derstand how their work-flow can be improved with support from AR. This pre-study should cover observations of situations from warehouse trucks and a study of similarly made research, mainly in the automotive field (see Section 2.5). The situations will be further examined to find which states that can be improved the most using AR. A product development method called kansei will be used. Kansei and AR are described in Chapter 2.
The proposed solution will consist of working prototype software for the Moverio glasses (see Section 1.7) that will solve some of the analyzed situations, where improvement can be made. Drivers and the design department will evaluate the functionality and usability. This leads to a product validated by a user case study.
The user case studies will be used as feedback of how good the solution is. Since this product should be for the end user, this will be a measurement of usability.
1.6
Limitations
For this master thesis, an all-knowing system is required and because of technical dif-ficulties and the time limit, it will be simulated. This all-knowing system knows the speed, location and direction of every object in the warehouse, including pallets, trucks and people. An estimation that the floor of a warehouse is flat will be made and some distances can therefore be measured in 2 dimensions.
This master thesis focuses on picking and putting down pallets. For this, it is assumed that the truck driver handles the standard EUR-pallets which he picks and put down in pallet racking. In reality, forklifts can handle other logistic solutions than EUR-pallets, but it is overlooked for this thesis.
Chapter 1. Introduction 5
1.7
Hardware requirements
For this master thesis, the AR glasses Epson Moverio BT-200 [1] were used. They run on Android 4.0.4, Ice Cream Sandwich, and are controlled and powered via an external control unit that comes along. The Moverio glasses features a See-Through display at 960x540 pixels with a horizontal FOV of ∼ 23◦, a front-facing 640x480 VGA camera with
up to 30 frames per seconds of filming, GPS, compass, gyroscope and an accelerometer. BT-200 is not a device that is made to the regular user, it is made for developers.
Chapter 2
Background
To get some background knowledge of this thesis, this chapter presents some of the ba-sic theory behind it and provides information about the theory behind AR and kansei engineering. The tools used to develop the application are discussed as well as related work and technology.
2.1
Augmented reality
In the words of Alnabhan et al. [2], Augmented Reality is when virtual objects is superimposed over a scene of the real world that is captured in real-time, so that they together seems like one environment. The computer generated objects could be 2D or 3D graphics, images, text, audio or video which interacts with the real world in a way that enhances the users experience and is useful. It is important that the information provided through AR is valuable so that unnecessary information does not obstruct the view of the user. It is easy to bloat the users vision by augmenting too much. In relation to virtual reality, AR does not have to create a virtual environment but can instead use the real world as the environment.
The earliest mention of AR was probably in the book ”The Master Key”[3] from 1901. The thought of AR is old, but the implementations are recent, because of hardware limitations.
Chapter 2. Background 7
2.2
Kansei engineering
Kansei Engineering is a product development method that will be used in this thesis. It is used in many Japanese and Chinese companies, but is not widely known in the western countries. Kansei engineering is often mentioned as ”Affectionate Engineering”, this is because emotions is widely used as a mindset throughout the development of the product. By knowing who the end-user of the product is, you can determine what kind of emotions you want to bring up when the product is used.
The goal is to make the end user connect certain emotions to the product. For instance, low lifting forklifts often have a removable platform for the driver to stand on. TMHE made use of kansei engineering in order to understand what types of coil springs and dampeners to use for this platform in a study [4]. Professional drivers were used for testing, with several different platform types. The result was an optimal spring dampener independent of user weight.
By associating a product with the feelings that the product provides, kansei states that the user feels more connected with the product. This applies to both hardware and software. Kansei is a stepwise method and begins by expanding words connected with feelings, and then reduce these to a graspable amount of words, which later describe product parameters. Kansei is all about narrowing down the product parameters to the most fitting product for the end user. Kansei Engineering was used in this thesis as a tool to specify the end product.
2.3
Software
The thesis application, called TruckAR, is developed for the Epson Moverio BT-200 glasses which runs on the Android platform. These Augmented Reality glasses support development in Android SDK, Wikitude Augmented Reality SDK, Metaio SDK and Unity3D.
Metaio SDK is a cross-platform framework for many parts of AR. Metaio can be used as an extension for an Android application, taking control of the camera and using its tracking configuration in order to achieve the correct transformation.
Chapter 2. Background 8 Metaio SDK was used as a plug-in for Unity3D. Unity3D is a cross-platform game engine. The usage of a game engine in 3D creation is a preferred method when much content needs to be tested and under certain controllable conditions. Scripting and class creation allows for control and extensibility, while 3D game tools creates a fast way of controlled content creation in 3D. Unity3D also helped the achieve the goal of this thesis in the limited time frame.
Using a pre-made AR toolkit combined with a game engine provides all the necessities for a rapid development of an AR product, without getting too technical which suited this thesis’ time frame.
2.4
TruckAR overview
The TruckAR application was made as a test suite for drivers. It contains a basic navigation interface for each of the situations. The user can click to change solution and change some different values of the solution.
Each solution is shown independently in order to evaluate them one-by-one. While the end solution is intended to work altogether, evaluating them one-by-one provides better test result for the individual solution. This makes each part of a solution open for customization.
2.4.1 Iterative approach
TruckAR was developed using an iterative approach. When the concepts had been decided, the development was done in two major iterations. Two milestones, set as two user tests, were the goal for each major iteration, spread by a month. During the months of development, minor weekly sprint-like iterations were utilized.
At the end of each week, some of the concepts were at a working-prototype state, allowing for input and feedback from Toyota. Goals were set up, as what concepts were most important to develop until next week and how much of them could be done. This was done until the user tests and kept the TruckAR in an ever changeable manner.
Chapter 2. Background 9
2.5
Related works and technology
Currently, there is a push in the vehicle industry to establish a standard for vehicle-to-vehicle communication. The U.S. Department of Transportation has published a recommendation of this, for car manufacturers to follow[5]. The research done can be seen as a guideline for how vehicle communication should work, and it covers much of how the future cars will interact with each other. By knowing the speed, position and much more of each car, the roads could be made safer and more efficient.
Much work with AR has been done in the vehicle industry, e.g. Kim et al. [6] and Rao et al.[7], which focuses on the use of AR in cars regarding navigating and crash warnings through Augmented Reality information displays. While these articles cover some of the thought-of implementations of AR in cars, it does not cover the full spectrum of AR in warehouse trucks. The fundamentals of navigation and crash warnings, however, is something that will aid this thesis.
Tran et al.[8] suggests a projected field in front of oncoming vehicles to visualize a danger zone while doing left turns. Drivers often misjudge the speed of oncoming vehicles, and the time it takes for them to reach their position. They suggest different ground projected fields in front of the oncoming vehicles and proceed with user testing the AR technology. As safety is an aspect of this thesis, their report is used as a base for some of the concepts.
A study made by Baumann et al. [9] shows the potential of head mounted displays (HMD) with graphical interfaces in warehouse environments and how they can stream-line the process. They show that color coded labeling lowers the rate of error when picking by hand. This study did not choose to implement an AR solution, but show that pickers prefer color codes and feedback while picking load.
Order picking by visual tunnel by Biocca et al. [10] implements an attention funnel to direct the user’s attention towards the correct racking position while picking. They implement the solution for people in warehouse picking, by tracking and projecting a tunnel that spans from the eye position to the location of the load.
The German logistics company DHL discusses the use of AR in logistics and provides examples in their 2014 research report [11]. Some best practices in different fields are
Chapter 2. Background 10 brought up and concepts about AR in logistics are discussed. These concepts include pick-by-vision, which removes the need of bar code scanners, warehouse planning and freight loading among others. The concepts provided in the DHL report can be seen as a guideline for AR in logistics.
Reif et al. [12] observes the use of AR pick-by-vision and compares it to standard paper lists. They develop the attention funnel [10] further and shows that pick-by-vision users are faster and make fewer errors. They mention that ”The use of HMDs on fork lifts is
problematic due to reasons of labor safety. Most of the purchasable HMDs limit the field of view (FOV) increasing the risk of accidents.”, suggesting that HMDs are not useful
as they limit the FOV of the user. Reif et al. [12] also mentions that their work did not cover forklifts, which makes their statement open for testing.
The authors did not find any previous work done on AR with forklift drivers. There has been AR research done in the logistics and vehicle industry, but not much combined. Many of the suggestions will be based on the observations done in the beginning of this thesis.
Chapter 3
Situation analysis
The master thesis’s foundation is reoccurring events, or situations, that happens to a truck driver when he performs his task to pick up a pallet at position A and leave it at position B.
In order to create an assisting application for drivers, a warehouse analysis was made. The authors filmed 94 different situations in the warehouse of TMHE Mj¨olby. The reason why the observation process was made as third person viewer instead of interviewing drivers was to get a glimpse of the nature of warehouse truck driving, without interfering. Each film was then analyzed according to [13], where an unstructured observation was made. This means that no certain properties were specifically looked for. Instead each film was analyzed and every interesting aspect of that situation was described. This gives a more flexible analyze of the situation and does not overlook any aspect of the situation.
From an observer’s viewpoint, situations that occur on routine are especially interesting. Because of the frequency of these situations, they are generally overlooked by personnel. How and why these situations were decided is described in this chapter.
Chapter 3. Situation analysis 12
3.1
Decision of situations to solve
In order to know whether a film was important or not, they were divided into sub-situations, each sub-situation was weighted after importance. The sub-situations is events that occur in the film. An important situation was a situation that, for instance, an unsafe sub-situation was more interesting than a sub-situation where the driver was exiting the truck, and would have a higher weight. Each situation could potentially contain several sub-situations.
• Bad sight
• Exiting the truck • Aligning • Slow • Unsafe • Stop • External signals • Non-ergonomic • Use of computer • U-turn • Pedestrians
Films containing any of the events above were taken further into the filtering process.
3.1.1 First situation filtering process
The weight for each sub-situation was chosen as of how important the sub situation was. To get a more credible value, the weights were taken to a reference group at TMHE where they were evaluated to see if the values reflected the real world.
Chapter 3. Situation analysis 13 Adding the weights up for each situation gave that unique situation a score. This score was used as a value of how interesting that situation was. This was done in order to reduce the amount of situations to filter out the less important ones.
The situation will be considered interesting and further examined in the second filtering process in Section 3.1.2. A total of 61 films were interesting enough to be taken to the next step.
3.1.2 Second situation filtering process
Following kansei engineering, all of the interesting films, from the first filtering, plus Toyota’s value words Productivity, Safety, and Quality were described using adjectives words with emotion. A total of 101 words were found using this method.
This many words was hard to work with, so an affinity diagram was created and can be seen in Figure 3.1. Based on the meaning and their relationship they were organized into groups and duplicates or synonyms were removed.
Safe
Safety Feeling
Communi-cation Acting Productivity Desirable
Driver perception Reliable Foreseeing Steady Confident Stimulated Attentive Toyota-way Informative Assisting Responsive Alignability Straight forward Calmly Alert Efficient Professional Innovative Risk Free User Friendly Enjoyable Physically Comfortable Effortless Concentrated
Figure 3.1: The affinity diagram that was created. The top row of words is a container for that column of words.
The films that went through the first filtering process was analyzed with these words in mind. Each situation in each film had the second level words graded from 1-5, determined by how much the word fulfilled the situation. Following kansei engineering, all words are positive. An interesting situation would be a situation with a low total score because that would mean that something about the situation is bad and can be improved.
Chapter 3. Situation analysis 14 Weights were used for each top level and second level word to calculate each situations total score. When a solution has been generated to solve the problems in a situation -the score is supposed to go up.
These weights were decided by a reference group at TMHE. This reference group con-sisted of members from different branches in TMHE, providing a spread knowledge base in the driver and truck field. The top level words in the affinity diagram were graded 1-3, where 1 was a word of low importance and 3 a word of high importance. This was graded from the perspective of a warehouse manager and with AR in mind.
The following is how the top level word grades ended up: Grade 3 Safety, Productivity, Communication Grade 2 Feeling, Acting, Desirable
Grade 1 Driver perception
Safety and Productivity are two leading words at Toyota, which is why their grades are high. Communication was thought to be a main aspect of AR, to communicate information to the driver. Driver perception was graded low because the impact of AR was thought to not have much impact on Physical comfort and such.
3.2
Warehouse management
There are many aspects of warehouse management. Most warehouses put their items on EU-pallets in racking. These racking range from hand height up to as high as 18 meters. The racking are used for different types of work. Different truck types are used for different tasks.
The truck type this thesis will focus on is the model called BT-Reflex that can be seen in Figure 3.2, and is of the type Reach truck. There are several models of this truck and the most high-end can lift up to 2.5tn up to 12.5m [14]. What makes this truck different from others is that it can automatically tilt the cab when lifting on high heights. The Reflex was chosen because it is flexible and has many of the typical difficulties in a forklift. This includes occluding chassis, eye-to-fork distance that causes depth perception problems
Chapter 3. Situation analysis 15 and bad sight around the truck because of a fixed driver position. The driver typically sits facing one of the sides of the truck, meaning he has to turn his head in order to look in the driving direction. The direction of the forks is to the right of the driver.
Figure 3.2: Bt-reflex, the truck model used in this thesis
3.3
Result from situation analysis
By weighting the value words and knowing the truck type, the situations were filtered down to a remaining total of 22 situations. These situations ranged a mean value of 1.65 to 4.08 in the scale 1 − 5. A value low that meant that the situation did not fill the criteria for a ”good enough” situation, and will thus be prioritized.
Key words were associated with each situation, in order to know if two situations were alike. These key words would be something that described the situation, such as if the situation contained pedestrians, other trucks or intersections. Similar situations were looked into and evaluated against every other similar situation. This was made in order to reveal a pattern between bad values and their situations.
After evaluating these situations, four situations proved to be the most interesting in terms of improvability. Section 3.3.1 to 3.3.4 explains these situations.
3.3.1 Loading and unloading
When approaching pallet in pallet racking, the drivers in the situations need to focus a lot on the pallet, their surroundings and themselves. This creates a risky moment,
Chapter 3. Situation analysis 16 where their attention simply cannot focus on everything at once. They either have to:
A Focus their attention on the pallet, then slowly back out and divide their attention as they slowly make their way from the pallet racking.
B Focus their attention on the pallet, knowing the usual traffic around the area, and quickly move away from the pallet racking.
C Focus on the surroundings, not caring much about the pallet and quickly move away from the pallet racking.
AR should be implemented in a way that the driver knows that the pallet is secured and that it is the correct pallet. The driver should also be aware that the surroundings are clear. This contributes to a safer and more efficient loading and unloading environment for the driver.
3.3.2 Transportation
Driving a truck is generally to load pallet at A, then unload it at B. The road between these locations is full of obstacles such as pedestrians and other trucks. There is generally bad sight at industries, and putting up stop signs will make the traffic too slow. To start a truck from a complete stop takes more time and requires more power from the battery and therefore warehouses would save money if the trucks doesn’t have to stop.
Bad sight can cause the driver to feel unsafe, and thus drive slower. If the driver drives too fast, the driver instead causes a risk. If AR can be implemented to make the driver aware of the surroundings and to provide foresight, a safer and more efficient way of transportation can be achieved.
3.3.3 Intersection
During the transport there might be many intersections. Some warehouses have rules for intersections, which generally drags down the tempo in order to produce a safe working environment. Intersections creates a situation where the driver needs to know about oncoming objects in order to plan the driving into and away from the intersection. The driver needs to be aware of:
Chapter 3. Situation analysis 17 A Oncoming trucks and their path
B The driving order (generally first come first served) C Pedestrians
If AR can be implemented to inform the driver about any upcoming situations in the intersections, the driver can plan his driving accordingly. A system can give recommen-dations for the driver to follow, which further increases the flow in the warehouse. Doing this will increase the efficiency of the drivers and make them feel safer.
3.3.4 Displays
The current displays in the forklifts are constructed as an extra aid for the driver. They give sensor information back to the driver per display. It is up to the driver to make use of the information given. Some drivers use this during their daily tasks.
Some of the displays are: • Height indicator
• Wheel-rotation indicator • Weight indicator
• Dashboard symbols
These displays could probably be removed if they were integrated into AR. This could make them highly customizable, communicate information easier to the driver and be more user friendly and enjoyable to the driver. If this can be fulfilled, it can create a better working environment for the driver.
Chapter 4
Concept generation
Concepts were generated for the four situations that were discussed in Chapter 3. How these concepts were generated is described in this chapter. Some concepts were chosen to proceed with and to not proceed with and why are brought up. This chapter also gives an explanation of every concept that were decided to be further developed.
4.1
Concept generation methods
Based on the four situations found in Section 3.3, concepts were generated. The situ-ations were divided into parts of situsitu-ations, to easier generate concepts for the smaller parts. This was made to see a situation as smaller steps that were integrated with each other.
Generating concepts can be made in numerous ways. According to Tae-Hong [15], a lot of concepts should be generated, and there are methods that help with the creativity behind concepts. This thesis used two methods of concept generation, brainstorming and analogies. Each method was used in two different sessions.
Brainstorming is the most common way of generating concepts. The basics behind it, is to think wild, write down any idea that is thought of and visualize them. No evaluation of the ideas should be made during the brainstorming session in order to allow ideas that are seldom thought of.
Chapter 4. Concept generation 19 The ideas should fit the demands that were made clear from the situation analysis. This would mean an increase in the value words generated. The concepts should also fulfill the purpose of the situations, and be visualized within the frame of AR.
The second session was an analogy-session. The thought is to think of similar situations that are solved in other fields. The same criteria was used as in the brainstorming session regarding demands and AR. This often led to similar situations in car traffic, air traffic but even situations solved by phones and computer games.
These two concept generation methods were chosen because of the time pressure and the scale of the project. There are more comprehensive methods of concept generation, but they were not used.
4.2
Concepts
The situations were divided into a total of 9 part-situations and concepts were generated for each of these 9. A minimum of three concepts were generated per part-situation, which landed in a total of 39 concepts.
The situations are sometimes similar, intersections can, for example, be viewed as a part of transportation. This created similar solution for many situations. This is necessarily not bad, since it could potentially create a feeling of recognition between the situations. The kansei graded situations were used as a reference value to the concepts, and each concept had an estimated kansei grade. This estimated kansei grade was estimated from the reference situation, but with the AR concept applied.
4.2.1 Concepts for loading and unloading
Follow path Based on the wheel rotation and the truck pivot point, the predicted path can be calculated. The user will be presented with an augmented path on the ground that will help him align the truck to the rackets. This concept is related to parking assist systems that can be seen in modern cars such as the parking assist system presented in [16]. According to the study, the assist parking system makes the parking time slower, but it increases the feeling for safety. The predicted wheel
Chapter 4. Concept generation 20
(a) Follow Path (b) Spirit level (c) Hologram Truck (d) Eagle eye
A71
(e) Tooltip (f) Stripped Tooltip (g) Death Star (h) Safe zone
Figure 4.1: Concept sketches for loading and unloading
position path will try to improve that by not only showing the predicted path but also the desired path. When the predicted path is aligned with the desired, the driver can drive with the steering wheel in the current position and he will be aligned correctly.
Spirit level A virtual spirit level is shown to the driver, which reminds the driver to keep the forks well adjusted for the lift. This is augmented by the forks and shows the desired position of the forks’ tilt and side-shift relative to the pallet.
Hologram of truck A 3D model that mirrors every part of the truck such as the forks and how it is rotated relative to the racket. This help the user get a view from another perspective and in that way be able to see his truck in one view. Desired positions of forks and such will be show to help the driver with his next move. The pallet that the driver should pick up should pop out from the rest of the pallets in the virtual racket.
Eagle eye By putting one wide-angled camera on each side of the truck, a view of its sur-rounding can be stitched together using the video feed from those. This will make the user able to see a birds eye view of his truck an surroundings.
Tooltip This concept will help the driver to find the correct pallet in the rackets. An over-lay with useful information about the pallet is created. The information contain information such as fragility and racket position, but can vary between pallets and
Chapter 4. Concept generation 21 users, so that everyone gets the information they desire. The overlay will float in the air near the pallet. If the user looks away so the pallet is not in the glasses’ FOV, the overlay will have a rubber band effect and still be viewable. When the rubber band ”break”, the user will be presented with an arrow, showing in which direction the pallet is.
Stripped tooltip What distinguishes this concept from the Tooltip concept is that it doesn’t contain any information. It simply highlights the pallet to only show its location.
Death star The solution for AR order picking presented in [10], shows how order picking by hand can be made more efficient using AR. The concept will be adjusted to fit the needs of truck loading. A tunnel consisting of squares that follows a path to the pallet will be presented to the user to show the location of the pallet. A similar concept can be seen in action in the movie Star Wars Episode IV: A New Hope [17] when Luke Skywalker is blowing up the death star.
Safe zone When the driver is focusing on picking a pallet, he can miss a potential danger nearby. The surrounding drivers will see a projected Safe zone around the truck in thus the driver can be focusing on his pallet. This safe zone is an area that is not allowed for those trucks to enter. If one or more truck enter this zone, the driver in the picking truck will be notified.
4.2.2 Concepts for transportation
(a) Hansel & Gretel
A71
(b) Pallet Tooltip A71 (c) Extended Tooltip A71 (d) Hologram Map Figure 4.2: Concept sketches for TransportationHansel & Gretel A path is laid out in front of the truck, showing the recommended path the driver should take in order to get to the end location. This is similar to the way Hansel and Gretel follows a trail of candy in the well-known fairy tale by the Brothers Grimm.
Chapter 4. Concept generation 22 Pallet tooltip A sign is augmented on top of the pallet on the forks. This sign has information about what is on the pallet, where the end location is and eventually other related information.
Extended Tooltip The pallet racking location is highlighted on distance. A sign telling the driver which aisle and location is augmented next to this highlight.
Hologram Map An overview 3D map is shown to the driver showing the driver position, the end location and the closest path to it.
4.2.3 Concepts for intersections
(a) X-ray
John Doe ~~~~~~~~
(b) X-ray tooltip (c) Environmental
Overlay
(d) Red riding hood
(e) Green Wave
Figure 4.3: Concept sketches for Intersections
X-ray Hidden objects have an overlay augmented on top of them. At bad sight this augments an ’X-ray’ overlay on top of the occluding walls in intersections, making the user aware of nearby trucks or other objects. This concept presents information about the hidden object.
X-ray tooltip Similar to X-ray, this concept, instead of information overlays, puts labels con-nected to each hidden object that does not contain any significant information other than the objects’ position and what object it is.
Chapter 4. Concept generation 23 Environmental overlay An overlay over the passage an incoming truck is coming from will be shown to the user. This allows him to know from which direction a potential danger is coming from.
Red riding hood Similar to the Concept Hansel & Gretel , this concept will show the user his as well as other truck’s path on the ground. The difference from Hansel & Gretel is that in this concept, the drivers will see everyone else paths and not only his own. If a prioritized truck is incoming, the paths that does not belong to the prioritized truck will be cut off to allow them to know that they should slow down.
Green Wave The ’Green wave’ is an expression in traffic regulation where if a car drives at a certain speed, it will only face green lights. This is used to reduce the down-time of traffic overall and create a better flow. This concept adapts the traffic light to a linear representation, where the driver instead should drive with a green wave representation. Slowing down when needed and sometimes speeding up before intersections will create a better flow. This should be recognized by an all-knowing WMS. The concept is partly based on a bicycle project made in Copenhagen [18], where lights on the ground turn green to represent the green wave.
4.2.4 Concepts for displays
(a) Terminator (b) Virtual HUD (c) Marker based
dis-plays Figure 4.4: Concept sketches for Displays
Terminator Similar to a famous scene from the 1984 movie ”The Terminator”[19], informa-tion is projected based on where the driver is looking. For example, informainforma-tion concerning the forks are projected at the forks.
Chapter 4. Concept generation 24 Virtual HUD Similar to notification panels of smartphones, this virtual Heads-Up-Display is projected in front of the warehouse truck when the driver decides. All the infor-mation is put on this panel and if important inforinfor-mation needs to be shown, the panel should indicate this.
Marker based displays Markers are put up in the truck, representing a usual display of the truck. These markers are thought to make the placement of the displays more dynamic for each driver. The display is then augmented on top of these markers.
Chapter 5
Methods
This chapter covers the various technical methods used in order to achieve the result. Marker tracking and the technical difficulties that were had with several of the concepts are described. The theory explained will be used in order to create the different concepts of this thesis.
5.1
Tracking
In Virtual and Augmented Reality, tracking is a crucial feature. In VR, tracking relative to a starting position is often used. VR does not necessarily need any tracking of the surroundings, whereas AR needs an initial position and one or many real world measurement references.
Tracking can be done in many ways. Some ways of tracking include camera tracking, ultrasound tracking, electromagnetic tracking and inertial tracking[20]. Each type of tracking has its own pros and cons. This thesis will be focusing on camera tracking, as it is robust and easy to implement.
5.1.1 Camera tracking
Camera tracking can be done with one or more cameras and can be adjusted to work in many situations. One can track the head mounted display by using markers on the
Chapter 5. Methods 26 users head, or track the environment from a head mounted camera. Since the BT-200 has a built-in camera, it comes natural to track the world from the head position. One major drawback of cameras is that they are light-sensitive, and computationally expensive compared to other tracking methods. Camera tracking methods are also frame rate dependent, meaning that tracking can not be faster than the camera can gather frames.
The camera resolution of the BT-200 glasses is up to 640 by 480 pixels. Even at this resolution, the camera tracking suffers due to the performance of the hand-held device. To compromise, the resolution of the camera feed in TruckAR was lowered to 320x240 pixels. To find a marker, the application goes through all the pixels of the input image from the camera. A lower resolution image will mean fewer computations and a faster process, which leads to a higher frame rate. The camera downsampling improved the frame rate from around 8 fps up to around 23 fps.
5.1.2 Marker tracking
In TruckAR, marker based tracking is used to track the environment and to decide where in the world to put the augmented objects. An ID-marker, as shown in Figure 5.1, is a black and white matrix image that is recognized by the camera. The advantages of marker tracking compared to image tracking or environmental mapping methods, like simultaneous localization and mapping (described in Section 8.8.1), is that it is precise and does not require much computation when using a lower resolution - thus the method has acceptable performance, even on the moderate hardware of the BT-200.
Figure 5.1: An ID-marker. X and Y are depicted, whereas Z is straight out from the marker.
Chapter 5. Methods 27 By knowing the physical size of the marker, a relationship between the marker and the camera can be calculated, see Figure 5.3, as following:
CX CY CZ 1 = R11 R12 R13 T1 R21 R22 R23 T2 R31 R32 R33 T3 0 0 0 1 MX MY MZ 1 = TCM MX MY MZ 1 (5.1)
where Cx,y,z is the position of the camera, Mx,y,z the marker position. TCM is the
4x4 camera-marker transformation matrix with R11, ..., R33 is the rotation matrix and
T1, ..., T3 is the translation vector. By knowing the size of the marker, the objects can
be scaled accordingly, so that they fit to the real world.
The marker-to-camera transformation matrix can be used to position objects in relation to the marker in the real world. The tracking is implemented in Unity in such a way that a metaio marker object is created and the objects that should have their position and rotation relative to this marker is set to this marker object’s children.
The marker object in Unity contains a tracking configuration file that states which marker that specific tracking object belongs to, which marker tracking algorithm to use and optimization configurations.
When an object is attached as a children to a marker object it will inherit the mark-ers transformation and will be augmented in the real world on top of it, as shown in Figure 5.2
5.2
Stereo rendering and eye separation
See-Through stereo vision is achieved through the Moverio glasses by capturing marker position through the camera and by knowing the eye separation of the glasses. Each pair of glasses are assumed different and are in need of calibration before use. The calibration is done manually by adjusting an augmented plane on top of a marker of different sizes. Known marker sizes gives the camera position through Equation 5.1. In a similar way this gives the transform for each eye ER and EL relative to the camera CM as seen in
Chapter 5. Methods 28
Figure 5.2: View through glasses.
Figure 5.3. The hand-held device saves these values in an XML-file used for per-unit calibration when running the application.
z y x z y x z y x z y x
T
1T
2T
3Figure 5.3: Marker-to-Glasses transformation. T1is the transformation between the
tracker and the camera, T2, T3 are the right and left screen transformations relative to
the camera position.
By knowing the transformation matrix between camera and each eye, a frame can be rendered for each eye. Stereo vision is achieved by rendering the left and right frame with the offset from the camera, as seen in Figure 5.3. The left and right frame is projected per eye, as the BT-200 projects two screens. The background of the 3D space is drawn in black, as it gives the most transparent look on the semi-transparent screens.
Chapter 5. Methods 29
5.3
Catmull-rom splines
The visual tunnel concept, described in chapter 4 uses the method catmull-rom splines to create the curved path between the user and the marker, on which the outline of the tunnel is placed on. Catmull-rom splines are a family of cubic interpolating splines [21]. Unlike a natural cubic spline that produces C2
-continuous interpolation, a catmull-rom spline exhibit C1 continuity and have a simple piecewise construction [22].
The tangent at each control point, pi, is calculated using the previous and next point
τ (pi+1− pi−1) of the spline. The tangent at the start point p0 is set to τ (pi+1− pi) and
the end point, pn is assumed to not be interpolated [21]. τ is the tension of the curves,
in other words, how sharply the curve bends. The geometry matrix is given by
p(s) =h1 u u2 u3i 0 1 0 0 −τ 0 τ 0 2τ τ − 3 3 − 2τ −τ −τ 2 − τ τ − 2 τ pi−2 pi−1 pi pi+1 (5.2)
where p(s) is a single Catmull-Rom segment.
In the implementation, only one control point p1 is used to give a slight curve. The
distance between the user and the marker is D. p1 is defined to be in the the direction
the user is looking at a distance of D/2 from the user, as can be seen in Figure 5.4.
P
0D
1=D/
2
P
1τ(P2-P0)D
P
2Figure 5.4: Image of a curved line made using catmull rom algorithm. p0 is the
starting point, p1 the control point and p2 the end point. D is the distance between
Chapter 5. Methods 30
5.4
Distance independent size of objects
The glasses’s horizontal field of view of 23◦ is a problem. When approaching an object,
it might be too large and take up too much of the user’s vision. This is compensated by calculating the distance between the marker and the camera and scale the object according to it.
The script is attached to the objects that is supposed to have this property. Since the cameras are not moving they will always be in origin. When this information of the camera and the tracking object position is known, the distance can be calculated. The result will be a scale factor S = [0...n], which is the value used to scale the object as can be seen in Figure 5.5. The precision won’t be high, but good enough for this specific purpose.
Figure 5.5: Illustration of how the distance independent size of objects works. Al-tering the distance of the camera to the object has no impact of the size of the object
on the screen.
5.5
Billboard
The user should always be able to read a sign, therefore the sign should always be rotated towards the user. This is accomplished by calculating the direction from the object to the camera and rotating the object, by subtracting the two position vectors, and then using this direction as the forward direction of the object. This is depicted in Figure 5.6. To give a more natural feel, the rotation between the initial rotation of the object and the rotation that makes it look in the correct direction, is interpolated with a time delay.
Chapter 5. Methods 31 This time delay makes the object appear more in contact with the real world as the user turns his head.
Figure 5.6: Illustration of how billboarding works.
5.6
Turning radius
In the Concept Follow path in chapter 4 it is all about the turning radius of the truck. Each wheel Wb (back wheel), Wf r (front right) and Wf l (front left), in Figure 5.7, will
have its own turning radius that depends on the rotation of the back wheel Wb and the
angle αwb, which is the turning angle of the back wheel. The rotation of the back wheel
also gives the direction the truck will be heading. The truck’s direction will then be given by the angle αp = 180 − αwb. Depending on if the angle αp
The turning radius R is given by
R = D1 tan(αwb)
(5.3) where D1 is the distance between the back wheel and the pivot point P of the truck.
Given the radius R, the pivot point P of the truck as a whole will be translated horizon-tally with the distance R. The two front wheel Wf r and Wf l will have its independent
turning radius with the center as the pivot point. The turning radius for the front wheels is given by
Chapter 5. Methods 32 Rf wr= 1 sin(αwb) ∗ 1 D3 , (5.4) Rf wl= 1 sin(αwb) ∗ 1 D2 (5.5) where D3 is the distance between the front right wheel and the pivot point and D2 the
distance between the front left wheel and the pivot point. Rf wrand Rf wlis the resulting
radius for the front right wheel and the front left wheel. Using these radii the predictive path can be rendered as lines along the circumference of the circles with the radii Rf wr
Chapter 5. Methods 33
α
wbW
bW
frW
flR
wfrR
wflD
1α
pR
D
2D
3P
Figure 5.7: Overview of how the rotation radius is calculated as explained in 5.6
5.7
A* search algorithm
A* search algorithm [23] (pronounced A-star) is a heuristic graph traversal algorithm commonly used in Artificial intelligence applications. In a connected network of weighted nodes, A* finds the optimal path from one node to any other node. By traversing each node’s least cost path towards the goal, the optimal path is found.
A* works by having an open and a closed list, storing each potential node from the current node in the open list and each passed node in the closed. By dividing the floor of the warehouse into a set of eight-connected nodes, the floor can be viewed as a 2D graph of interconnected nodes. From each node, a distance to the end node can be calculated and stored. Traversing the 2D-graph is done by picking the next node as the node with the shortest distance to the end node, storing the cost from the start node. The distance to the end node is made as a cost estimate, not counting for obstacles or such.
While traversing, if at any time a lower cost is found in any of the adjacent open nodes, the current node will be dropped from the closed list and the node with the lowest cost will be chosen as current. Doing this assures that one of the closest distances are found, and the end node will be found fast.
Chapter 5. Methods 34 By assuming a colliding radius of the forklift, estimations done on a map of the warehouse ensures that none of these nodes should lie closer than the radius of the truck to any obstacle.
For this to work properly, a 3D representation of the testing environment had to be done. This was done in a 1:1 scale for the testing room at TMHE. For truck positions, similar objects were represented using cubes.
Figure 5.8: The representation of the testing environment from above. The leftmost red circle represents the start position of the test driver.
In Figure 5.8, the blue inner area represents the node network in 2D at ground level. The nodes are no closer to any of the obstacles than the estimated radius of the driver’s test truck. Each node, except for edge nodes, are connected with the eight closest neighbors, as can be seen in the top-left enhancement of the Figure. The right rectangles in Figure 5.8 are stacked pallets, the yellow cube is another truck.
The node network can then be combined with the A* algorithm in order to find the optimal path from the position of the truck to the pallet. By following this path, and storing the position on some of the intermediate nodes, the Hansel & Gretel concept can be implemented.
Chapter 6
User testing
The solutions were tested on users in two user tests. The two user tests were done in a demonstration facility at TMHE. The first test was done with a conceptual model of the solution, and the other with more fine-tuned solutions. Due to the small size of the driver test group, only trends and general observations are discussed.
This chapter goes more into into detail of the two user tests made for this thesis. How the tests were carried out and what information that was gathered from the user tests are brought up and summarized in this chapter. The questions asked were asked using a questionnaire.
6.1
First user test
The first test was conducted in order to get an idea of how good the concepts were and what was desired from the standpoint of a truck driver. The test was done with six drivers at TMHE, who were instructed to follow a trail of thought of how the concepts were thought to be used. The test participants were told that the purpose of the test was to evaluate how good the concepts were.
None of the test subjects had previously tried AR before. Feedback was taken from a set of questions for each concepts and thoughts from the drivers during the test were noted. No hardware limitation feedback were taken into account for the test, i.e. tracker lag or input-related issues.
Chapter 6. User testing 36 The test drivers were to try each different solution in the thought situations.
No driving was done during this test. Instead, the participants walked or stood still during the test, while evaluating the concepts using the glasses. The test participants were told to focus on evaluating the concepts, solutions and how the AR technique was used. Two stations were set up to showcase the different concepts, questions about the concepts and usability were asked.
The drivers ranged from 4 to 30 years of working experience with warehouse truck driv-ing. These were experienced drivers, who drives approximately 4-6 hours each working day.
6.1.1 User test for loading and unloading
The situation was described to them as they were close to a pallet racket and was looking for the correct pallet. They were then placed in front of a marker that had been put up top of a pallet. The truck to the left was used to demonstrate the concepts safe zone,
predictive path and hologram truck.
First, the Tooltip and Stripped tooltip concept were shown. In the prototype shown in the evaluation, the stripped tooltip was a red crosshair with some opacity and the tooltip (with information) contained information about what was in the pallet and where its destination was. All the drivers thought that green would be a better color to use and that the information was good but could be complemented with information about the pallets weight and the number of pallet collars currently attached. One driver wanted to be able to see the date of arrival of the pallet and where the center of gravity is located. The drivers was asked about which tooltip they prefered and the answers resulted in a tie between the two.
With the the same set-up, the Death star concept was evaluated. All of the driver agreed that it was easy enough to understand. However, four of six drivers thought that the concept took up too much space of the view and that it was therefore not usable because they wouldn’t see where they drove.
A marker was placed on top of a pallet in a pallet racking. The drivers was asked to stand in front of marker 2 and look at it. As can be seen in the Table 6.1, they drive
Chapter 6. User testing 37 dangerously close to each other often and therefore the safe zone concept is a good one. During the first evaluation the radius of the safe zone was set to one meter. According to four of the drivers, it was a good size and gave them enough space too feel safe. Two test subjects wanted the safe zone to be a little bit bigger.
How often do drivers drive dangerously close to each other?
Driver 1 Driver 2 Driver 3 Driver 4 Driver 5 Driver 6
5 5 5 4 4 3
Table 6.1: Answer from question about how often drivers drive close to each other. 1 - never, 5 - very often.
The drivers had different views on the Follow path concept. When a driver is mature enough and has experience, he knows exactly how the truck behaves and how every adjustment moves the truck in a specific way. Four of six of the drivers felt like it wouldn’t help them to align the truck to the rackets because they felt it would be faster to just drive by using their senses. This made the concept unusable for experienced drivers but it would be useful for inexperienced drivers or in educational purposes.The two drivers who answered yes on the question in Table 6.2 thought the concept would be useful for inexperienced drivers.
Does Follow path help with aligning the truck and makes it easier to know how the steering wheel is positioned
Driver 1 Driver 2 Driver 3 Driver 4 Driver 5 Driver 6 Yes No No No Yes No
Table 6.2: Answer from question about how often drivers drive close to each other
Except for aligning, one of the purposes was to help drivers when driving in confined spaces. The drivers were consensual that the concept Follow path would not allow for any easier navigation in these situations. Instead they prefer to drive by using their senses. The result can be seen in Table 6.3.
Even though the Hologram of truck concept was not fully developed for the evaluation it was well received even though it lacked the ”zoom to the forks” feature. The drivers thought that it would help to find the correct pallet as well as aligning the truck. Positive feedback about the mirroring of the real truck was received. One driver thought it was unnecessary to use the hologram truck concept when driving to the racking but that it
