Department of Computer and Information Science
Final thesis
Statistical Analysis of Team Tr aining in
Emer gency Management Simulator System
by
Muhammad Nasir J ahangir
Muhammad Fahadullah
LIU-IDA/LITH-EX-A--09/028--SE
2009-04-06
Linköpings universitet SE-581 83 Linköping, Sweden
Linköpings universitet 581 83 Linköping
Rapporttyp Report category Licentiatavhandling Examensarbete C-uppsats D-uppsats Övrig rapport Språk Language Svenska/Swedish Engelska/English Titel Title Författare Author Sammanfattning Abstract ISBN ISRN LIU-IDA/LITH-EX-A--09/028--SE
Serietitel och serienummer ISSN Title of series, numbering
Nyckelord Keywords
Date
URL för elektronisk version
X
Avdelning, institution Division, department
Institutionen för datavetenskap Department of Computer and Information Science
Statistical Analysis of Team Training in Emergency Management Simulator System
Muhammad Nasir Jahangir & Muhammad Fahadullah
In this thesis work, we compare the results obtained from two kinds of teams forming a hierarchical organization participating in a fire fighting simulation environment called as C3Fire. First kind of teams used paper-based maps for spatial reasoning of the command tool while the other kind of teams has GIS based maps with full access to positioning data of the fire fighting units as well as sensor information about fire break.
The collected data was from 11 teams of each kind having 6 members in each team making a total of 132 participants belonging to different parts of the world. We made a statistical analysis on the data and then a comparison is done to analyze the performance and efficiency of both kinds of teams.
Emergency management, GIS (Geographical information systems), T-Test
2009-04-06 Linköpings universitet
Final Thesis
Statistical Analysis of Team Training in
Emergency Management Simulator System
by
Muhammad Nasir Jahangir
Muhammad Fahadullah
LIU-IDA/LITH-EX-A--09/028--SE
2009-04-06
Supervisor: Dr. Rego Granlund Examiner: Prof. Dr. Arne Jönsson
Department of Computer and Information Science, Linköping University, Sweden.
ii
Dedication
In the name of Allah, the Most Gracious, the Most Merciful.
Dedicated to our beloved parents, who have always been most caring, stood by us for all the times, lifted us when we could not reach, gave us the faith and inspiration, strengthen us when we were weak, and never letting us fall.
iv
Acknowledgement
This study was conducted at Linköping University (LiU) at the Department of Computer and Information Sciences (IDA).
We are grateful to our Supervisor Rego Granlund for all the guidance throughout the project. The facilities and equipments provided were really helpful during the project. We also thank Helena Granlund for her assistance for the project.
vi
In this thesis work, we compare the results obtained from two kinds of teams forming a hierarchical organization participating in a fire fighting simulation environment called as C3Fire. First kind of teams used paper-based maps for spatial reasoning of the command tool while the other kind of teams has GIS based maps with full access to positioning data of the fire fighting units as well as sensor information about fire break.
The collected data was from 11 teams of each kind having 6 members in each team making a total of 132 participants belonging to different parts of the world.
We made a statistical analysis on the data with help of T-Test statistical medhod and a tool is designed by using Java as programming language and PostgreSQL database for importing data from log files and then applying statistical T-Test method on the fetching data from log files.The results are stored in database as well as excel files. Then a comparison is done to analyze the unit performance, communication and efficiency of both kinds of teams.
Keywords:
viii 1. Introduction ……… 1 1.1 Project Aim ….………. 1.2 Outline ……….………. 2. Research Background ……… 2.1 Problem area ………. 2.2 Research Aim ……… 2.3 Research Method ..……… 2 2 3 3 4 4 2.4 Statistical Analysis ………..……….. 4 3. Research Project ……… 5
3.1 The C3Fire Environment ..……… 3.1.1 Overview ………. 5 6 3.1.2 Simulation ……… 7
3.1.3 Map Layer ……….. 7
3.1.4 Geographical Object Layer ……….. 7
3.1.5 Fire Simulation Layer ……… 8
3.1.6 Fire User Interface ……… 8
3.1.7 Logging ……….. 9 3.2 Experiment design ……… 9 4. Stat Theory ………... 4.1 Arithmetic Mean ……… 4.2 Standard Deviation …...……… 4.2.1 Basic example 11 11 12 13 14 15 15 16 18 18 18 18 20 21 ……… 4.3 T-Tests ……… 4.3.1 Are two sets of data really different? ……… 4.3.2 Paired Data ……… 4.3.3 Procedure ……… 4.4 Analysis of Variance (ANOVA) ……… 4.4.1 The Purpose of Analysis of Variance ………. 4.4.2 Why the name analysis of variance? ………. 4.4.3 The Partioning of Sums of Squares ……… 4.4.4 One-way ANOVA ………... 4.4.4.1 One-way ANOVA calculations ………. 5. Implementation ………... 24 6. Results ……….
6.1 Unburned Area ……… 6.2 Analysis for Fire Module ………
26 27 28
ix
6.5 Command and Control ……… 6.6 Analysis for Command Module ………
32 33 7. Discussion ………... 7.1 Performance ……….. 7.2 Communication ………. 7.3 Command Module ………....………. 7.4 Summary of Main Finding ………. 7.4 Conclusion ……….………. 7.4 Future Work ……… 36 36 36 37 37 37 38 8. References ………... 39 9. Appendix ………... 41
x
Fig. 2.0: The GIS productivity curve according to Buckley (1998) ……… 3 Fig. 3.1: Example on a C3Fire environment setting ……… 5 Figure 3.7: The monitoring process in C3Fire ……….. 6 Figure 3.8: Team composition and communication channels between the team
members ……… 7
Figure 5.1: Program flowchart ……… 24 Figure 6.1: Condition 1, the commanders have access to various forms of maps,
the exact position of the fire brigades and located fires in real time via the GIS
system ……… 26
Figure 6.2: Condition 2, the commanders only have access to the state of the simulation through the e-mails they exchange with the ground chiefs and the notes they make on a traditional paper map ………... 27
xii
Table 4.1: Average and Standard Deviation ………... 15 Table 4.2: Partitioning of variance ………... 18 Table 4.3: ANOVA results ………... 19 Table 6.1: Performance in terms of burned down area of both teams over trials …. 28 Table 6.2: Number of send e-mails (mean) in each trial for both conditions ………. 31 Table 6.3: Analysis of Analog and Digital Map during Command Sessions ……….. 33
1
1. Introduction
A geographic information system (GIS) captures, stores, analyzes, manages, and presents data that refers to or is linked to a location. In the strictest sense, the term describes any information system that integrates, stores, edits, analyzes, shares, and displays geographic information. In a more generic sense, GIS applications are tools that allow users to create interactive queries (user created searches), analyze spatial information, edit data, maps, and present the results of all these operations. Geographic information science is the science underlying the geographic concepts, applications and systems, taught in degree and GIS Certificate programs at many universities (wikipedia.org).
Geographic information system technology can be used for scientific investigations, resource management, asset management, archaeology, environmental impact assessment, urban planning, cartography, criminology, geographic history, marketing, logistics, and other purposes. For example, GIS might allow emergency planners to easily calculate emergency response times in the event of a natural disaster, GIS might be used to find wetlands that need protection from pollution, or GIS can be used by a company to site a new business location to take advantage of a previously under-served market (wikipedia.org).
Currently, many organizations working with emergency management (EM) invest in information and communication technologies (ICT), such as advanced (non-conventional) databases, global information sharing systems, geographical information systems (GIS), expert systems, etc., with the goal to increase performance and control in their everyday work, i.e., managing everyday accidents and emergencies, as well as in the time of crisis. A development of powerful, easy to use – and understandable – computer based decision support and information management systems is proposed as means to assist EM participants in achieving a higher level of sophistication in information assessment, integration and manipulation in emergency response (Iakovou and Douligeris, 2001; Fedra, 1998).
At the same time, the possible impacts of modern ICT on real-life work practice have already been debated. Fields like computer supported cooperative work (Schmidt & Bannon, 1992), distributed cognition (Hutchins, 1995) and cognitive systems engineering (Hollnagel and Woods, 2005) have all emphasized the importance of studying the actual use of new systems in practice, rather than to draw conclusions from hypothetical gains of new technology.
A similar sort of emergency management study was carried out in a firefighting simulation system, called as C3Fire. This comprises of two sorts of teams: one using GIS based maps having the capability to directly updating of unit positions and fire
2
outbreak sensor capacity, and the other team had paper-based maps providing them only learn to handle the forest fire.
1.1 Project Aim
The aim of the project is to analysis data, which is taken from GIS studies. For the analysis of the data the a statistical method T-Test is used for analysing performance, communication and unit performance. The statistical analysis results are taken during implementation phase by importing data into PostgreSQL database from log files and appling statistical T-Test method through Java routines and results are stored in database and excel files as well.
1.2 Outline
Chapter 1 Introdution
In this chapter a brief introduction about GIS field and its beneficial during emergency management (EM).
Chapter 2 Research Background
This chapter elaborates about research background of GIS in emergancy management.
Chapter 3 C3Fire Envirnoment
This chapter describes C3Fire Envirnoment that how it is designed, how it is works and how it can be used for GIS study.
Chapter 4 Stat Theory
This chapter provides overall study about statatistical methods for this project to understand the appropriate method.
Chapter 5 Implementation
This chapter contains implementation process that how program works to perform statistical analysis through java routines.
Chapter 6 Results
This chapter keeps all of the results conducted during the implementation phase and provide all statistical analysis about performance, communication and unit management.
Chapter 7 Discussion
3
2. Research Background
An event – emergency, crisis, or disaster – represents a complex and dynamic situation. In EM, such events often involve descriptive information with a geographical & spatial dimension (de Silva, 2001). This information is identified as geographical, alternatively spatial, information (GI), i.e., information about objects and phenomena that are associated with a two- or three-dimensional position in space (Walker, 1993). GIS are computer-based systems integrating different technologies for spatial visualization, data management, decision modeling, planning, etc., providing a common organizational framework for heterogeneous multidimensional data (Longley et al., 1999; Laurini and Thompson, 1996; Bernhardsen, 1999; Maguire et al., 2001).
2.1 Problem area
Training the use of GIS in emergency situations is thus a central question if we want the system to be an enabler rather than just an administrative control tool. It is already known that the implementation of a GIS needs to be done carefully, and that it is unrealistic to expect large productivity gains on a short perspective. Oppositely, from an organizational point of view, it may take years before the investment in a GIS pays off in terms of increased performance (see Fig. 1) (Buckley, 1998).
Fig. 2.0: The GIS productivity curve according to Buckley (1998).
This productivity curve shows that there is a downfall in productivity in initial stages, meaning that an organization without a GIS actually has an advantage in the early phase compared to an organization that has implemented GIS, although they will be less productive in the long run. However, it has to be noticed that Buckley (1998) focused on
4
productivity of day-to-day use by business organizations and not on GIS usage in EM. Furthermore, research on productivity/learning of individual decision makers or teams of decision makers, in EM concerning GIS usage has not been made to the knowledge of the authors.
In many civilian organizations that are involved in EM, responsible decision makers do not work with GIS on a daily basis. These decision makers have often problems to fully exploit the functionality as wells as to understand and interpret the presented information in modern GIS systems. Consequently, the GIS could, as suggested above, be a bottleneck for performance rather than an improvement. This will also be investigated in terms of performance over trials.
2.2 Research Aim
This research aims at increasing understanding about the mechanisms behind the usage of GIS in EM, as well as its effect on the overall command and control task. The research utilizes traditional hypothesis testing.
2.3 Research Method
The method used is traditional hypothesis testing by controlled experiments using simulations (microworlds). Microworlds can be seen as a bridge between traditional laboratory research and field studies, since it allows the researcher to confront the participants with the dynamics of real-world situations at the same time as the participant is in control of the environment.
To be able to obtain productivity curves we carried out experiments where teams of students were solving an emergency-like task (forest fire) in the C3Fire environment.
2.4 Statistical Analysis
Our task in the research project is to analyse the data from the project. We apply statistical analysis to the available data. We studied and take into consideration many statistical methods and then find out the best appropriate to be implemented. Please see chapter 4 Stat Theory for more details.
5
3. Research Project
3.1 The C3Fire Environment
C3Fire is a microworld that provides an environment that allows controlled studies of collaborative decision-making in a dynamic environment (Granlund, 2002; Granlund & Johansson, 2004). In C3Fire participants can perform team tasks such as co-operation and coordination of actions and plans. The researchers can select some important characteristics of the real world and create a small and well-controlled simulation system that retains these characteristics.
The participants' organisation and communication structures can be set up as wanted depending on the research goal. The user interfaces and communication tools can also be individual set-up for all participants.
The domain, which is forest fire-fighting, is of subsidiary interest and has been chosen because it creates a good dynamic environment for the participants. It is possible to view the generated session as a simplified version of the work tasks and the division of labour conducted in an emergency task (Johansson, Trnka, Granlund 2005). See Fig. 3.1 for an example of a hierarchical organisation.
Figure 3.1: Example on a C3Fire environment setting
For this experiment series they have used a GIS module with GIS capabilities corresponding to present GIS in EM. This new GIS module includes modifications in three areas: GI visualization, GIS functionality and map input.
In this experiment, the participants were provided with two different map windows: control window and overview window. The control window is in principle a traditional C3Fire work
6
view. In this window, the traditional GIS functionality, such as panning, is added. Besides, the participants can also command the simulated units and modify the operational information, for example defining directions for unit movements, in this window. In the overview window, the participants can see the overall representation of the concerned geographical area in the microworld. However, they cannot modify any type of information.
3.1.1 Overview
C3Fire provides environmental task for group of people for improving management abilities within a team. In a typical setting of organization environment, it consist of four layers such as; emergency alarm, the command and control staff, the fire fighting unit chef and the ground units.
Figure 3.2: Organization Example
In organization configuration, an organization is mostly named by configuration of the human players and communication arrangement. The size of the organization can vary from one to twenty players depending on the speed of the used computer systems.
Figure 3.3(a): Hierarchic Organization Figure 3.3(b): Flat (network based) Organization
In this setup name of the player is used to identify a player as well as diary tools and the mail that a player can use. By configuration of the mail, diary and distributed map tools communication become possible between the players. Each player is configured individually and they have their own session. During the whole session all important activities regarding to communication, event handling etc are recorded.
7
3.1.2 Simulation
In C3Fire the simulation environment setup becomes possible by interacting simulation layers. There are three major layers such as; a unit layer, geographical layer and a fire layer. Two dimensional matrix is the base of all of the layers. The geographical object layers is consist of populated area, forest, landscape and other types of the static objects. The fire layer is a cellular automata interacting with other layers. Different sorts of the moveable objects are included in unit simulation layer such as; emergency units as fire-fighting units and reconnaissance persons. Weather simulation also comprise in simulation part which interact with the fire simulation.
Figure 3.4: C3Fire on a Map Matrix
3.1.3 Map Layer
The matrix of C3Fire is the surface on which the other layers can interact. Nothing exists outside the matrix. The fire will not spread outside this area. Weather, units and geographical objects cannot be configured to have any meaning outside it.
3.1.4 Geographical Object Layer
The populated area, vegetation and other static object are included in geographical object layer. In vegetation area different types of the vegetables are used regarding to their burning quality such as slow, fast burning. Building objects also vary according to their burning features. For the trained people it provides complex decision task.
8
Figure 3.5: Geographical Objects Example
3.1.5 Fire Simulation Layer
The fire simulation is a cellular automaton. Each position on the map has its own fire simulation, together they represents the fire. The fire's start positions are determined in the session configuration, but how the simulation develops is depending on each positions fire simulation. The geographical object layer, the unit layer and the wind simulation affect the fire simulation.
The states of fire are represented on the map by four images, no fire, fire, fire closed out, fire burned out or fire break. Example of standard images:
Figure 3.6: Different States of Fire
3.1.6 C3Fire User Interface
C3Fire user interface consists of three basic components such as; a map system, a diary and email system. The major functionality of map system is to give geographical information to end users. An email system provides communication facility between players. A diary gives a set of instruction to individual player about how to proceed with a specific situation.
Whole communication process within players is controlled and viewed by commanders and staff. Primary task of the staff is to take care of whole situation. During this process
9
fire-fitting unit chiefs communicate with staff and computer simulated fire fighting units and reconnaissance persons.
3.1.7 Logging
To be able to analyze the collaborative work in the C3Fire system, computer-based monitoring is used (Granlund, 2002). During a session the C3Fire system creates a log with all events in the simulation and all computer mediated activities (see Fig. 3.7).
Figure 3.7: The monitoring process in C3Fire
The log process receives information from the simulation about the current activities in the simulated world. It also receives information about individual work, in terms of marks in the personal map (GIS), and on the collaborative work in terms of information about the e-mail communication and the use of the distributed GIS and diary. The log information is stored in structured log files and in a SQL database which make it possible for the researchers to process the session information in an advanced manner (Johansson, Trnka, Granlund 2005).
3.2 Experiment design
The study is a between groups design with one factor: (a) teams using GIS combined with direct update of unit positions and fire outbreak sensor capacity, and (b) teams using paper-based maps. The difference lays in that the participants in teams using GIS faces two learning tasks, the GIS and the handling of the forest fire problem, while the participants in the teams using paper-based maps has to only learn to handle the forest fire, but has less sophisticated decision support since they lack the direct update of positions (Johansson, Trnka, Granlund 2005).
10
We have tested 11 teams in each condition, meaning a total of 22 teams consisting of 6 persons, summing up to 132 participants. Each team consists of six participants; three working as commanders and three as ground chiefs controlling three fire-brigades each (see Fig. 3.3). The reason is to increase the workload on the team members.
Figure 3.8: Team composition and communication channels between the team members
Every team performs five trials, lasting for 20 min each. They are also given an introduction to the system for about 20 minutes. An earlier study by Svenmarck & Brehmer (1991), using a preceding forest-fire simulation called D3Fire, has shown that the performance curve stabilizes between the third and the fifth trial. Therefore it was chosen to limit the study to five trials, partly for practical reasons, but also because the above mentioned indication that performance stabilizes at that level. To engage the participants in the experiments for more than three-four hours may also lead to undesired fatigue, which could have unwanted effects on the outcome.
11
4. Stat Theory
Based on the data, we calculate and analyze the efficiency of the teams both individually and collectively. For this task, we take the use of statistical methods.
Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data. Also with prediction and forecasting based on data. It is applicable to a wide variety of academic disciplines, from the natural and social sciences to the humanities, government and business.
There are many statistical methods and procedures for analyzing data. We study the following methods and then select one method which we found to be the best and appropriate one:
• Arithmetic Mean • Standard Deviation • Student’s t-test
• Analysis of variance (ANOVA)
4.1 Arithmetic Mean
The arithmetic mean is the “standard” average, often simply called the “mean”.
The mean is the arithmetic average of a set of values, or distribution; For example, the arithmetic mean of six values: 34, 27, 45, 55, 22, 34 is:
According to the “St. Kliment Ohridski”, The mean of a data set is simply the arithmetic average of the values in the set, obtained by summing the values and dividing by the number of values. Recall that when we summarize a data set in a frequency distribution, we are approximating the data set by “rounding” each value in a given class to the class mark. With this in mind, it is natural to define the mean of a frequency distribution by
12
The mean is a measure of the center of the distribution. As you can see from the algebraic formula, the mean is a weighted average of the class marks, with the relative frequencies as the weight factors. We can compare the distribution to a mass distribution, by thinking of the class marks as point masses on a wire (the x-axis) and the relative frequencies as the masses of these points. In this analogy, the mean is literally the center of mass—the balance point of the wire.
Recall also that we can think of the relative frequency distribution as the probability distribution of a random variable X that gives the mark of the class containing a randomly chosen value from the data set. With this interpretation, the mean of the frequency distribution is the same as the mean (or expected value) of X
In addition to expressing the variability of a population, standard deviation is commonly used to measure confidence in statistical conclusions. For example, the margin of error in polling data is determined by calculating the expected standard deviation in the results if the same poll were to be conducted multiple times. (Typically the reported margin of error is about twice the standard deviation, the radius of a 95% confidence interval.) In science, researchers commonly report the standard deviation of experimental data, and only effects that fall far outside the range of standard deviation are considered statistically significant. Standard deviation is also important in finance,
.
4.2 Standard Deviation
In statistics, standard deviation is a simple measure of the variability or dispersion of a data set. A low standard deviation indicates that all of the data points are very close to the same value (the mean), while high standard deviation indicates that the data are “spread out” over a large range of values.
For example, the average height for adult men in the United States is about 70 inches, with a standard deviation of around 3 inches. This means that most men (about 68%, assuming a normal distribution) have a height within 3 inches of the mean (67 inches – 73 inches), while almost all men (about 95%) have a height within 6 inches of the mean (64 inches – 76 inches). If the standard deviation were zero, then all men would be exactly 70 inches high. If the standard deviation were 20 inches, then men would have much more variable heights, with a typical range of about 50 to 90 inches.
13
where the standard deviation on the rate of return on an investment is a measure of the risk (wikipedia.org).
Formulated by Francis Galton in the late 1860s, the standard deviation remains the most common measure of statistical dispersion. A useful property of standard deviation is that, unlike variance, it is expressed in the same units as the data.
4.2.1 Basic example
Consider the following data:
There are eight data points total, with mean (or average) value of 5:
To calculate the standard deviation, we compute the difference of each data point from the mean, and square the result:
Next we average these values and take the square root, which gives the standard deviation:
Therefore, the data set above has a standard deviation of 2.
The variance of a data set is the arithmetic average of the squared differences between the values and the mean. Again, when we summarize a data set in a frequency distribution, we are approximating the data set by “rounding” each value in a given class to the class mark. Thus, the variance of a frequency distribution is given by
14
The standard deviation is the square root of the variance:
The variance and the standard deviation are both measures of the spread of the distribution about the mean. The variance is the nicer of the two measures of spread from a mathematical point of view, but as you can see from the algebraic formula, the physical unit of the variance is the square of the physical unit of the data. For example, if our variable represents the weight of a person in pounds, the variance measures spread about the mean in squared pounds. On the other hand, standard deviation measures spread in the same physical unit as the original data, but because of the square root, is not as nice mathematically. Both measures of spread are useful.
Again we can think of the relative frequency distribution as the probability distribution of a random variable X that gives the mark of the class containing a randomly chosen value from the data set. With this interpretation, the variance and standard deviation of the frequency distribution are the same as the variance and standard deviation of X(fmi.uni-sofia.bg).
4.3 T-Tests
“Student” (real name: W. S. Gossett [1876-1937]) developed statistical methods to solve problems stemming from his employment in a brewery. Student’s t-test deals with the problems associated with inference based on “small” samples: the calculated mean (Xavg) and standard deviation ( ) may by chance deviate from the “real” mean and standard deviation (i.e., what you’d measure if you had many more data items: a “large” sample). For example, it is likely that the true mean size of maple leaves is “close” to the mean calculated from a sample of N randomly collected leaves. If N=5, 95% of the time the actual mean would be in the range: Xavg± 2.776 /N1/2 ; if N=10: Xavg± 2.262 /N1/2 ; if N=20: Xavg± 2.093 /N1/2 ; if N=40; Xavg± 2.023 /N1/2 ; and for “large” N: Xavg± 1.960 /N1/2 . (These “small-sample” corrections are included in the descriptive statics report of the 95% confidence interval.)
15
4.3.1 Are two sets of data really different?
If we have two collections of maple leaves (i.e., two samples), it is quite likely that in detail the collections are different: different highs, lows, and average leaf sizes. Is the measured difference in average leaf size large enough that we should reject the null hypothesis that in fact such differences are due to “chance”? Given the above sort of information on the likely range for the actual mean of each sample, the question basically reduces to whether the likely ranges overlap (in which case the means could be the same: in the overlap of the intervals, and we may not reject the null hypothesis) or if they do not overlap (in which case we must reject the null hypothesis: the difference is most likely not due to chance). To report the variety of possible outcomes: from means not “significantly” different to means in fact “significantly” different, the probability that the difference is due to chance is reported. Reject the null hypothesis if P
4.3.2 Paired Data
is “small”.
Very often the two samples to be compared are not randomly selected: the second sample is the same as the first after some treatment has been applied.
Cedar-apple rust is a (non-fatal) disease that affects apple trees. Its most obvious symptom is rust-colored spots on apple leaves. Red cedar trees are the immediate source of the fungus that infects the apple trees. If you could remove all red cedar trees within a few miles of the orchard, you should eliminate the problem. In the first year of this experiment the number of affected leaves on 8 trees was counted; the following winter all red cedar trees within 100 yards of the orchard were removed and the following year the same trees were examined for affected leaves. The results are recorded below:
tree number of rusted leaves year 1
number of rusted leaves year 2 difference: year 1 – year 2 1 2 3 4 5 6 7 8 38 10 84 36 50 35 73 48 32 16 57 28 55 12 61 29 6 -6 27 8 -5 23 12 19 average 46.8 36.2 10.5 standard dev 23 19 12
16
As you can see there is substantial natural variation in the number of affected leaves; in fact, a unpaired t
i.
-test comparing the results in year 1 and year 2 would find no significant difference. However, if we focus on the difference we find that the average difference is significantly different from zero.
We use this test for comparing the means of two samples (or treatments), even if they have different numbers of replicates. In simple terms, the t-test compares the actual difference between two means in relation to the variation in the data (expressed as the standard deviation of the difference between the means).
4.3.3 Procedure
First, we will see how to do this test using "pencil and paper" (with a calculator to help with the calculations). Then we can see how the same test can be done in a spreadsheet package (e.g. Microsoft Excel)
We need to construct a null hypothesis - an expectation - which the experiment was designed to test. For example, if we are analyzing the heights of pine trees growing in two different locations, a suitable null hypothesis would be that there is no difference in height between the two locations. The student's t-test will tell us if the data are consistent with this or depart significantly from this expectation. [NB: the null hypothesis is simply something to test against. We might well expect a difference between trees growing in a cold, windy location and those in a warm, protected location, but it would be difficult to predict
ii. List the data for sample (or treatment) 1.
the scale of that difference - twice as high? Three times as high? So it is sensible to have a null hypothesis of "no difference" and then to see if the data depart from this.
iii. List the data for sample (or treatment) 2.
iv. Record the number (n) of replicates for each sample (the number of replicates for sample 1 being termed n1 and the number for sample 2 being termed n2
v. Calculate mean of each sample (
) 1 and 2
vi. Calculate
). 2
for each sample; call these 12 and 22 [Note that actually we are using S2 as an estimate of 2
vii. Calculate the variance of the difference between the two means ( in each case]
d2) as follows
viii. Calculate d(the square root of d2 ix. Calculate the t value as follows:
17
(when doing this, transpose 1 and 2 if 2 > 1
x. Enter the t-table at (n
so that you always get a positive value)
1 + n2
xi. If the calculated t value exceeds the tabulated value we say that the means are significantly different at that level of probability.
-2) degrees of freedom; choose the level of significance required (normally p = 0.05) and read the tabulated t value.
xii. A significant difference at p = 0.05 means that if the null hypothesis were correct (i.e. the samples or treatments do not differ) then we would expect to get a t
Now compare your calculated t value with tabulated values for higher levels of significance (e.g. p = 0.01). These levels tell us the probability of our conclusion being correct. For example, if our calculated t value exceeds the tabulated value for p = 0.01, then there is a 99% chance of the means being significantly different (and a 99.9% chance if the calculated t value exceeds the tabulated value for p = 0.001). By convention, we say that a difference between means at the 95% level is "significant", a difference at 99% level is "highly significant" and a difference at 99.9% level is "very highly significant".
What does this mean in "real" terms? Statistical tests allow us to make statements with a degree of precision, but
value as great as this on less than 5% of occasions. So we can be reasonably confident that the samples/treatments do differ from one another, but we still have nearly a 5% chance of being wrong in reaching this conclusion.
cannot actually prove or disprove anything
When there are three or more levels for the nominal variable, a simple approach is to run a series of t tests between all the pairs of levels. For example, we might be interested in the heights of athletes in three sports, so we could run t test for each pair of sports. (Note that this approach is not the same as a paired t-test. That comes later.) A more powerful approach is to analyze all the data in one go. The model is the same, but it is now called a one-way analysis of variance (ANOVA), and the test statistic is the F ratio. So t tests are just a special case of ANOVA: if you analyze the means of two groups by ANOVA, you get the same results as doing it with a t-test (Will G Hopkins 2008).
. A significant result at the 95% probability level tells us that our data are good enough to support a conclusion with 95% confidence (but there is a 1 in 20 chance of being wrong). In biological work we accept this level of significance as being reasonable.
18
4.4 Analysis of Variance (ANOVA)
4.4.1 The Purpose of Analysis of Variance
In general, the purpose of analysis of variance (ANOVA) is to test for significant differences between means. Elementary Concepts provides a brief introduction into the basics of statistical significance testing.
4.4.2 Why the name analysis of variance?
It may seem odd to you that a procedure that compares means is called analysis of variance. However, this name is derived from the fact that in order to test for statistical significance between means, we are actually comparing (i.e., analyzing) variances.
4.4.3 The Partioning of Sums of Squares
At the heart of ANOVA is the fact that variances can be divided up, that is, partitioned. Remember that the variance is computed as the sum of squared deviations from the overall mean, divided by n-1 (sample size minus one). Thus, given a certain n, the variance is a function of the sums of (deviation) squares, or SS for short. Partitioning of variance works as follows. Consider the following data set:
Group 1 Group 2 Observation 1 Observation 2 Observation 3 2 3 1 6 7 5 Mean Sums of Squares (SS) 2 2 6 2 Overall Mean Total Sums of Squares 4 28
Table 4.2: Partitioning of variance
The means for the two groups are quite different (2 and 6, respectively). The sums of squares within each group are equal to 2. Adding them together, we get 4. If we now repeat these computations, ignoring group membership, that is, if we compute the total
19
SS based on the overall mean, we get the number 28. In other words, computing the variance (sums of squares) based on the within-group variability yields a much smaller estimate of variance than computing it based on the total variability (the overall mean). The reason for this in the above example is of course that there is a large difference between means, and it is this difference that accounts for the difference in the SS. In fact, if we were to perform an ANOVA on the above data, we would get the following result: MAIN EFFECT SS df df F p Effect Error 24.0 4.0 1 4 24.0 1.0 24.0 .008
Table 4.3: ANOVA results
As you can see, in the above table the total SS (28) was partitioned into the SS due to within-group variability (2+2=4) and variability due to differences between means (28-(2+2)=24).
SS Error and SS Effect: The within-group variability (SS) is usually referred to as Error variance. This term denotes the fact that we cannot readily explain or account for it in the current design. However, the SS Effect we can explain. Namely, it is due to the differences in means between the groups. Put another way, group membership explains this variability because we know that it is due to the differences in means.
Significance testing: The basic idea of statistical significance testing is discussed in Elementary Concepts. Elementary Concepts also explains why very many statistical test represent ratios of explained to unexplained variability. ANOVA is a good example of this. Here, we base this test on a comparison of the variance due to the between- groups variability (called Mean Square Effect, or MSeffect) with the within- group variability (called Mean Square Error, or Mserror; this term was first used by Edgeworth, 1885). Under the null hypothesis (that there are no mean differences between groups in the population), we would still expect some minor random fluctuation in the means for the two groups when taking small samples (as in our example). Therefore, under the null hypothesis, the variance estimated based on within-group variability should be
20
about the same as the variance due to between-groups variability. We can compare those two estimates of variance via the F test (see also F Distribution), which tests whether the ratio of the two variance estimates is significantly greater than 1. In our example above, that test is highly significant, and we would in fact conclude that the means for the two groups are significantly different from each other.
Summary of the basic logic of ANOVA: To summarize the discussion up to this point, the purpose of analysis of variance is to test differences in means (for groups or variables) for statistical significance. This is accomplished by analyzing the variance, that is, by partitioning the total variance into the component that is due to true random error (i.e., within group SS) and the components that are due to differences between means. These latter variance components are then tested for statistical significance, and, if significant, we reject the null hypothesis of no differences between means, and accept the alternative hypothesis that the means (in the population) are different from each other.
Dependent and independent variables: The variables that are measured (e.g., a test score) are called dependent variables. The variables that are manipulated or controlled (e.g., a teaching method or some other criterion used to divide observations into groups that are compared) are called factors or independent variables. For more information on this important distinction, refer to Elementary Concepts (StatSoft, Inc).
4.4.4 One-way ANOVA
In statistics, analysis of variance (ANOVA) is a collection of statistical models, and their associated procedures, in which the observed variance is partitioned into components due to different explanatory variables.
In general, the purpose of analysis of variance (ANOVA) is to test for significant differences between means. There are two main types of the analysis of variance (ANOVA):
• One-way ANOVA is used to test for differences among two or more independent groups. Typically, however, the One-way ANOVA is used to test for differences among at least three groups, since the two-group case can be covered by a T-test (Gossett, 1908). When there are only two means to compare, the T-T-test and the F-test are equivalent; the relation between ANOVA and t is given by F = t2. • Factorial ANOVA is used when the experimenter wants to study the effects of
21
ANOVA is the 2×2 (read: two by two) design, where there are two independent variables and each variable has two levels or distinct values. Factorial ANOVA can also be multi-level such as 3×3, etc. or higher order such as 2×2×2, etc. but analyses with higher numbers of factors are rarely done by hand because the calculations are lengthy and the results are hard to interpret. However, since the introduction of data analytic software, the utilization of higher order designs and analyses has become quite common (wikipedia.org).
4.4.4.1 One-way ANOVA calculations
Step 1: Compute CM, the correction for the mean.
Step 2: Compute the total SS.
The total SS = sum of squares of all observations - CM
The 829.390 SS is called the "raw" or "uncorrected" sum of squares. Step 3: Compute SST, the treatment sum of squares.
22
T1 = (6.9) + (5.4) + ... + (4.0) = 26.7 T2 = (8.3) + (6.8) + ... + (6.5) = 38.6 T1 = (8.0) + (10.5) + ... + (9.3) = 42.8 Then
Step 4: Compute SSE, the error sum of squares.
Here we utilize the property that the treatment sum of squares plus the error sum of squares equals the total sum of squares.
Hence, SSE = SS Total - SST = 45.349 - 27.897 = 17.45. Step 5: Compute MST, MSE and their ratio, F.
MST is the mean square of treatments; MSE is the mean square of error (MSE is also frequently denoted by ).
MST = SST / (k-1) = 27.897 / 2 = 13.949 MSE = SSE / (N-k) = 17.452/ 12 = 1.454
where N is the total number of observations and k is the number of treatments. Finally, compute F as
23
According to study the T-Test is suitable, when there are two groups and ANOVA provides facility to get analysis for more than two groups.
We apply T-Test after importing the data from log files into the PostgreSQL database and then computing T-Test through java routines.
24
5. Implementation
In implementation, we selected Java as a programming language, Netbeans and Postgre-SQL as a data storage during the log-files analysis.
We implemented the following model using the above mentioned data to calculate the statistical analysis.
Figure 5.1: Program flowchart
Start Calculation type selection Open DB Read data Calculate Insert result in DB
Write result in file
25
According to the flowchart, Initially program will start and it will ask for analysis type. After this database connection will be established. Then program will read the data from database and perform the calculation methods for analysis of the data. Regarding to the analysis method, we have used T-Test as statistical method. After doing this kind of work, it will store the result in the database and write the file. In the end database connection will be closed by the program.
26
6. Results
As in previous study T-Test and ANOVA are suitable for statistical analysis. During the analysis phase the T-Test method is used for statistical analysis of collected data.
The results presented in this section are on the basis of log-file data concerning fire fighting, e-mail traffic and command controller. The results in this report are the data from a total of 22 teams (11 in each condition), consisting of six participants in each team (N=132). A total of 26 teams participated in the study, but the first four teams were used for necessary adjustments of the scenarios and had to be excluded from the analysis.
Figure 6.1: Condition 1, the commanders have access to various forms of maps, the exact position of the fire brigades and located fires in real time via the GIS system
Every team participates in one simulation session. Each session consists of one training and five full trails. Each trail lasts for twenty minutes. The session is concluded with a debriefing. The participants are members of the Swedish municipal crisis management committees (Johansson, Trnka, Granlund 2005).
27
Figure 6.2: Condition 2, the commanders only have access to the state of the simulation through the e-mails they exchange with the ground chiefs and the notes they make on a traditional paper map.
6.1 Unburned Area
A common measure in all fire-fighting simulations is the amount of non-burned down area. In the C3Fire microworld, the area is divided into cells that take a certain state depending on: (a) if they are unharmed, (b) on fire, (c) closed out or (d) even burned out. At the end of each trial, the percentage of non-burned out cells is counted and used as a performance measure (a high score is desired). This is motivated by the fact that a successful team should be able to respond quickly to incoming alarms while at the same time use their resources effectively in order to minimize the consequences of the forest fires. The design of the scenarios is made in such a way that a team that uses all of their resources on only one fire has problems in limiting the other two fires. An inattentive commander will also face issues of missed incoming messages concerning new fires. Consequently this inattention leads to a slower response and greater loss of terrain, i.e., greater number of burned down cells (Johansson, Trnka, Granlund 2005).
The first overall measure of burned out area between the two conditions clearly show that the teams using GIS system perform better in terms of percentage of saved area.
28
6.2 Analysis for Fire Module
ConditionName SessionNumber GroupNumber FireStateBurnedOut FireStateObjectBurnedOut
Analog 5 2 121 0 Analog 5 4 108 0 Analog 5 6 38 0 Analog 5 8 152 0 Analog 5 10 376 0 Analog 5 12 72 0 Analog 5 14 258 0 Analog 5 16 332 1 Analog 5 18 457 0 Analog 5 20 209 0 Analog 5 22 165 1 Digital 5 1 206 0 Digital 5 3 56 0 Digital 5 5 123 0 Digital 5 7 106 0 Digital 5 9 152 0 Digital 5 11 131 0 Digital 5 13 53 0 Digital 5 15 113 0 Digital 5 17 322 1 Digital 5 19 92 0 Digital 5 21 224 0
Table 6.1: Performance in terms of burned down area of both teams over trials
t-Test: Paired Two Sample for Means
Analog Digital
Mean 208 143.4545
Variance 17805.2 6373.273
Observations 11 11
Pearson Correlation 0.419166
Hypothesized Mean Difference 0
df 10 t Stat 1.733622 P(T<=t) one-tail 0.056824 t Critical one-tail 1.812461 P(T<=t) two-tail 0.113649 t Critical two-tail 2.228139
From the achieved result, we find that the analog team was able to refrain more area from fire in the beginning as compared to the GIS based team. But as the GIS team takes time to learn the coordinates, it proved to become more efficient resulting in securing more area from burning down.
29
Figure 6.3: Overall Graph for comparison of paper based and GIS based teams for overall analysis for fire module
Figure 6.4: Graph for comparison of paper based and GIS based teams for overall analysis for fire module in session 5 (last session)
Mainly due to statistical analysis there is a distinction between the performance of the two conditions, if we take into account all the trials and team. The results show that geographic information systems law, the performance even better. These two conditions are also improving their performance in the number of trials. However, between groups in individual
30
studies (repeated measures), there is no significant effect.
In this session there is clear average difference between both Digital and Analog map. Fire fighters are working more professionally by using digital map in term of saved area.
See Appendix for all sessions analysis
6.3 E-mail Density
The number of e-mails sent between the commanders and the ground chiefs suggests that the GIS condition reduce the need for communication between the commanders and the ground chiefs.
6.4 Analysis for Mail Module
ConditionName SessionNumber GroupNumber Sum
Analog 5 2 87 Analog 5 4 267 Analog 5 6 181 Analog 5 8 167 Analog 5 10 128 Analog 5 12 147 Analog 5 14 164 Analog 5 16 182 Analog 5 18 104 Analog 5 20 197 Analog 5 22 135 Digital 5 1 138 Digital 5 3 54 Digital 5 5 277 Digital 5 7 121 Digital 5 9 59 Digital 5 11 34 Digital 5 13 98 Digital 5 15 151 Digital 5 17 84 Digital 5 19 96 Digital 5 21 127
31
t-Test: Paired Two Sample for Means
Analog Digital
Mean 159.9091 112.6364
Variance 2409.091 4321.655
Observations 11 11
Pearson Correlation -0.01433 Hypothesized Mean Difference 0
df 10 t Stat 1.898071 P(T<=t) one-tail 0.043449 t Critical one-tail 1.812461 P(T<=t) two-tail 0.086898 t Critical two-tail 2.228139
The analysis report shows that there is a significant difference between the conditions in terms of the number of messages sent (P<0.05).
32
Figure 6.6: Chart for analysis for mail module for session 5
The result is not unexpected considering that participants in the command module in the paper-based map condition only can receive information about what is happening at the field level by exchanging e-mails with the ground chiefs. In the GIS condition, the commanders can see everything that the ground chiefs can see, and thus have less need to ask for or receive information concerning positions of units or the fire outbreak.
In comparison of geographical representation and paper map; GIS (digital map) is more efficient in terms of exact data available about simulation state and faster communication. Therefore, as we can see in the graph, the message exchange for the GIS team is comparatively lower as compared to the regular paper-based team.
See Appendix for all sessions Analysis
6.5 Command and Control
Commander team using command and control system with GIS functionality exchanged significantly fewer messages that command teams using paper maps. Just like in many real-world situations, these are not systems that the commanders/participants work with on an everyday basis (Johansson, Trnka, Granlund 2005). In second condition commanders are not supported with geographical representation. They communicate with their subordinates and get data about simulation state through mails.
33
6.6 Analysis for Command Module
ConditionName SessionNumber GroupNumber Sum
Analog 5 2 310 Analog 5 4 417 Analog 5 6 301 Analog 5 8 551 Analog 5 10 387 Analog 5 12 351 Analog 5 14 484 Analog 5 16 356 Analog 5 18 509 Analog 5 20 448 Analog 5 22 473 Digital 5 1 438 Digital 5 3 498 Digital 5 5 621 Digital 5 7 362 Digital 5 9 577 Digital 5 11 454 Digital 5 13 633 Digital 5 15 570 Digital 5 17 555 Digital 5 19 659 Digital 5 21 524
Table 6.3: Analysis of Analog and Digital Map during Command Sessions t-Test: Paired Two Sample for Means
Analog Digital
Mean 417 535.5455
Variance 6888.8 8315.073
Observations 11 11
Pearson Correlation -0.12148 Hypothesized Mean Difference 0
df 10 t Stat -3.0117 P(T<=t) one-tail 0.00654 t Critical one-tail 1.812461 P(T<=t) two-tail 0.013079 t Critical two-tail 2.228139
According to the analysis GIS/Digital map is more effective in emergency management. In two conditions there is difference in commander’s level. They are facilitated in terms of data source (paper map) and geographical representation (digital map). In geographical representation more accurate real time data is available for commanders and communication is also fast between commanders and their subordinates. But in paper map there is slow communication and less precise state data is available. The difference lays in that the participants in condition one faces two learning tasks, the GIS and the handling of the forest fire problem, while the participants in condition two only has to learn to handle the forest fire, but has less sophisticated decision support since they lack the
34 GIS (Johansson, Trnka, Granlund 2005).
Figure 6.7: Overall graph for comparison of paper based and GIS team’s analysis for command module for mean
Figure 6.8: Chart for comparison of paper based and GIS team’s analysis for command module in session 5 (last session)
35
From the graph, it is depicted that the GIS team contains more data in comparison with the paper-based analog team.
36
7. Discussion
The result of this study indicates the geographical representation is more effective in Emergency Management System (EMS) as compare to paper map. Buckley suggests that there is a rather unusual productivity curve related to GIS implementation, something that indicates that GIS even may potentially hamper performance in EM, at least if the involved personnel does not work with GIS on a daily basis. An earlier study by Johansson et al. (2003) has also suggested that the gain of introducing GIS systems is far from self-evident (Johansson, Trnka, Granlund 2005).
7.1 Performance
Concerning the log-files generated in this study, the data shows that novice participants in the experiment performed better - as a team - than participants equipped with only a paper-based map. Thus, the hypothesis no.1 – different performance will be observed when the emergency organization uses GIS or paper-based map – is confirmed. On the other hand, the hypothesis no.2 – GIS teams will perform worse than the teams using traditional paper-based maps initially, but later outperform the paper-based map teams – is rejected in this study. This hypothesis suggested that the GIS condition, due to the complexity of the technology, should perform worse than the paper-based map teams initially. This does not seem to be true.
However, this interpretation must be understood in connection to the fact that there also was a difference in terms of access to real-time data. The participants in the GIS condition did not only have access to a richer set of data concerning the terrain, they also had access to real-time data concerning the position of the fire-brigades and fire outbreak (Johansson, Trnka, Granlund 2005).
7.2 Communication
The hypothesis no.3 – type of orders sent from the commanders (mission-type orders or direct control) will correlate with the performance of the teams – and no.4 – GIS teams to use more direct control tactics in the first trials than the teams using paper-based maps – suggest changes in communication structure and content between teams using GIS and teams equipped with paper-based maps. In other words, the commanders using GIS, which, by providing a very detailed view of the “world” also gives the impression that the world can be controlled in detail, may give highly detailed orders to their ground chiefs, at least in the first trials (Johansson, Trnka, Granlund 2005).
The statistical analysis of the data shows a significant difference in terms of number of sent messages in the two conditions. In the GIS condition, there is an average of 92 messages in each trial and in the paper-based map condition, there is an average of 140 messages (P<0.1, N=132).Theoretically, this means that communication (questions and answers) regarding position and activity of the fire-brigades between the commanders and
37 the ground-chiefs is less necessary.
7.3 Command Module
Looking into the figures and graphs for the command and control module illustrate that the command system aided by paper-based map was not much efficient because it has support from a comparatively slow email communication system and therefore having slower response time to get sufficient information to make decisions and direct commands. On the other side, the command system aided by GIS-based map was far better and efficient comparatively as it processes less number of messages with more accurate data. Therefore, unlike the other two modules, it is observable that the command and control module accompanied by GIS-based map is certainly faster, proficient, and accurate.
7.4 Summary of Main Finding
The findings related to the four hypotheses are as follows:
• Hypothesis no.1 – different performance will be observed when the emergency organization uses GIS or paper-based map – confirmed. The GIS condition performs significantly better in the performance measure non-burned down area. • Hypothesis no.2 – GIS teams will perform worse than the teams using traditional
paper-based maps initially, but later outperform the paper-based map teams – rejected. No significant differences were found.
• Hypothesis no.3 – type of orders sent from the commanders (mission-type orders or direct control) will correlate with the performance of the teams –further data analysis necessary.
• Hypothesis no.4 – GIS teams to use more direct control tactics in the first trials than the teams using paper-based maps – further data analysis necessary.
7.5 Conclusion
With the C3Fire environment (as described in chapter 3) and the data collected with the 11 teams comprising of 132 participants in total, we had the goal to illustrate our findings to compare the teams using the traditional paper-based map and the other kind of team using the GIS-based digital maps. Looking into the logs, we determined that there is a need of some statistical methods to have a better view and understanding for the analysis. Therefore, we took into consideration various statistical methods (as described in chapter
38
4) among which we found the T-Test as the most appropriate one because it assesses whether the means of two groups are statistically different from each other. This analysis is appropriate whenever you want to compare the means of two groups, and especially appropriate as the analysis for the posttest-only two-group randomized experimental design.
To accomplish this task, we selected Java as a programming language, Netbeans and Postgre-SQL as data storage during the log-files analysis, and implemented the model (as described in chapter 5).
With the results calculated through the model, we established our findings that the paper-based map team was able to refrain from more area with fire in the beginning as compared to the GIS based team. But as the GIS team takes time to learn the coordinates, it proved to become more efficient resulting in securing more area from burning down (as described in chapter 6). The achieved results and graphs reveal that the teams equipped with GIS-based digital maps took some time in the beginning to grasp the technique, but gradually appear to become more efficient in saving more area from burning with fire, takes less exchanges of messages resulting in more proficient communication, and more accurate real time data is available for commanders and communication is also fast between commanders and their subordinates.
7.6 Future Work
With this study, we were able to compare the two kind of teams, i.e. paper-based maps and the GIS-based digital maps. As it was concluded that GIS-based teams are more efficient and effective according to our findings with the T-Test as the selected statistical method. This can also be further evaluated with another statistical method called Analysis of Variance (ANOVA) because the T-Test tells us if the variation between two groups is "significant". In general, the purpose of analysis of variance (ANOVA) is to test for significant differences between means. Elementary Concepts provides a brief introduction into the basics of statistical significance testing.
39
8. References
Bernhardsen, T. (1999). Geographic Information Systems (Second edition), John Wiley & Sons, New York.
Buckley, D. J. (1998). The GIS Primer: An Introduction to Geographic Information Systems, Pacific Meridian Resources, Inc., USA.
de Silva, N.F. (2001). Providing spatial decision support for evacuation planning: a challenge in integrating technologies, Disaster Prevention and Management, Vol. 10, No. 1, pp. 11-20.
Granlund, R. (2002). Monitoring Distributed Teamwork Training Ph.D. Thesis at
Department of Computer and Information Science, Linköping University, Sweden, 2002. ISBN 91-7373-312-1.
Granlund, R., Johansson, B. (2004). Monitoring Distributed Collaboration in the C3Fire Microworld, in Samuel, G., Schiflett, L.R., Elliott, Salas, E. and Coovert, M.D. (Eds.). Scaled Worlds: Development, Validation and Applications" Eds. Aldershot, Ashgate. Hollnagel, E., Woods, D.D. (2005). Joint Cognitive Systems. Foundations of Cognitive
Systems Engineering. Taylor & Francis, Boca Raton, FL.
Hutchins, E. (1995). Cognition in the Wild. MIT Press, Cambridge, MASS.
Iakovou, E., Douligeris, C. (2001). An information management system for the emergency management of hurricane disasters, Int. J. Risk Assessment and Management, Vol. 2, No. 3/4, pp. 243-262.
Johansson, B., Granlund, R. & Waern, Y. (2000). The Communicative Aspects of Distributed Dynamic Decision Making in the ROLF-Environment. In Friman, H. (Ed.), Proceedings to the 5th International Conference on Naturalistic Decision Making, Tammsvik.
Johansson, B., Granlund, R. & Waern, Y. (2005). Research on Decision Making and New Technology - Methodological Issues. In Brehmer, B., Lipshitz, R. & Montgomery, H. (Eds.), How Professionals Make Expert Decisions, Lawrence Erlbaum Associates, Mahaw, New Jersey.
Laurini, R., Thompson, D. (1996). Fundamentals of Spatial Information Systems, The APIC Series, Academic Press Limited.
Longley, P.A., Goodchild, M.F., Maguire, D.J., Rhind, D.W., et al (1999). Geographical Information System – Volume 2: Management Issues and Applications (Second edition), John Wiley & Sons, Inc.
Maguire, D.J., Goodchild, M.F., Rhind, D.W. (1991). Geographical Information Systems: Principles and Applications, Longman Scientific and Technical, UK.
Svenmarck, P., & Brehmer, B. (1991). D3Fire, an experimental paradigm for the study of distributed decision making. In Brehmer, B. (Ed.). Distributed Decision Making. Proceedings of the Third MOHAWC Workshop, Belgirate, Italy. Roskilde: Risö National Laboratory.
40
Walker, R. ed. (1993). AGI Standards Committee GIS Dictionary, Association for Geographic Information, UK.
Statistics, Wikipedia , http://en.wikipedia.org/wiki/Statistical_analysis, accessed on November 9, 2008
Analysis of Variance, Wikipedia, http://en.wikipedia.org/wiki/Analysis_of_variance accessed on November 9, 2008
1-Way ANOVA calculations, Engineering Statistics Handbook, National Institute of Standards and Technology,
http://www.itl.nist.gov/div898/handbook/prc/section4/prc434.htm, accessed on November 9, 2008
Student t test, Wikipedia, http://en.wikipedia.org/wiki/Analysis_of_variance, accessed on Jan. 10 2009
Student’s t-test, School of Biological Sciences, The University of Edinburgh,
http://www.biology.ed.ac.uk/research/groups/jdeacon/statistics/tress4a.html, accessed on November 9, 2008
Mean and Standard Deviation, Faculty of Mathematics and Informatics. University of Sofia "St. Kliment Ohridski", http://www.fmi.uni-sofia.bg/vesta/virtual_labs/freq/freq2.html,
accessed on November 9, 2008 Standard deviation, Wikipedia,
http://en.wikipedia.org/wiki/Standard_deviation#Basic_example, accessed on November 9, 2008
Student's t-Test s, College of Saint Benedict and Saint John's University,
http://www.physics.csbsju.edu/stats/t-test.html, accessed on November 9, 2008 Will G Hopkins, 2008, T Test and One-Way ANOVA, A New View of Statistics, http://www.sportsci.org/resource/stats/ttest.html, accessed on November 9, 2008 StatSoft, Inc. (2007), Electronic Statistics Textbook, ANOVA/MANOVA,
http://www.statsoft.com/textbook/stathome.html, accessed on November 9, 2008 ANOVA: ANalysis Of VAriance between groups, College of Saint Benedict and Saint
John's University, http://www.physics.csbsju.edu/stats/anova.html, accessed on November 9, 2008
StatSoft, Inc., 1984-2008, ANOVA/MANOVA,
41
9. Appendix
Analysis Report for Fire module
Collected Data During 1st Session of Fire Module
Condition Name
Session Number
Group
Number FireStateBurnedOut FireStateObjectBurnedOut
Analog 1 2 294 0 Analog 1 4 676 4 Analog 1 6 358 1 Analog 1 8 92 0 Analog 1 10 421 1 Analog 1 12 350 0 Analog 1 14 72 0 Analog 1 16 349 1 Analog 1 18 410 1 Analog 1 20 630 6 Analog 1 22 440 5 Digital 1 1 315 1 Digital 1 3 154 0 Digital 1 5 387 0 Digital 1 7 343 4 Digital 1 9 315 1 Digital 1 11 496 3 Digital 1 13 106 0 Digital 1 15 425 3 Digital 1 17 402 1 Digital 1 19 147 0 Digital 1 21 620 5
Analysis Report for 1st Session
T-Test: Paired Two Sample for Means
Analog Digital
Mean 372 337.2727273
Variance 34314.2 24379.21818
Observations 11 11
Pearson Correlation -0.072267165
Hypothesized Mean Difference 0
df 10 t Stat 0.459337712 P(T<=t) one-tail 0.327907415 t Critical one-tail 1.812461102 P(T<=t) two-tail 0.65581483 t Critical two-tail 2.228138842
