Degree project in Computer Science Second cycle
Stockholm, Sweden 2013
Describing scenes by qualitative spatial relations
MARIA DEL CARMEN MOLLA GARCIA
Describing scenes by qualitative spatial relations
MARIA DEL CARMEN MOLLA GARCIA
Master’s Thesis at NADA
Supervisor: Patric Jensfelt / John Folkesson Examiner: Stefan Carlsson
TRITA xxx 2013-03
Abstract
This thesis focuses on describing scenes by using qualitative spatial relations. Not all the spatial relations are suitable for describing the environment. For this reason a selection will first be made of the most appropriate relations for this task. Given these relations, descriptions for a number of the scenes will be generated. Based on these descrip- tions, patterns will be extracted. These patterns guide the design of a model which is able to generate new scenes with similar qualitative description. Finally some experiments will show the results of the model for different inputs.
Referat
Denna avhandling fokuserar på att beskriva scener med hjälp av kvalitativa spatiala relationer, som t.ex. på och nära. Inte alla spatiala relationer är lämpliga för att beskriva omgivningen. Därför görs först ett urval av de mest lämpliga relationerna för denna uppgift. Genom att använda dessa relationer för att generera beskrivningar för ett antal olika scener, kan vissa mönster extraheras. Baserat pådessa mönster, kommer en generativ modell att skapas som kan generera nya metriska konfigurationer för scener med samma kvalitativa konfiguration. Slut- ligen presenteras några experiment som visar vad modellen genererar med olika indata.
Contents
1 Introduction 1
1.1 Problem statement . . . . 2
1.1.1 Delimitations . . . . 2
1.1.2 Desired Properties . . . . 3
1.2 Contributions of the thesis . . . . 3
1.3 Outline . . . . 4
2 Analysis of Related Work 5 3 Modeling and Design 9 3.1 Methodology . . . . 9
3.2 Qualitative spatial relations . . . . 10
3.2.1 Selection of spatial relations . . . . 11
3.3 Analysis . . . . 15
3.4 The hierarchy . . . . 16
3.4.1 The importance of a hierarchy . . . . 16
3.4.2 Building the hierarchy . . . . 16
4 The Method 21 4.1 Introduction to the model . . . . 21
4.2 Data collection . . . . 22
4.2.1 Photo shot . . . . 22
4.2.2 Data extraction . . . . 23
4.3 The method . . . . 24
4.3.1 Generating descriptions for scenes . . . . 24
4.3.2 Creating a model that can generate configurations of the objects 26 4.3.3 Taking a new scene and generating a description with relations 28 4.3.4 Sample example scenes . . . . 30
4.4 Discussion of the method . . . . 30
5 Experimental evaluation 33 5.1 Results by varying the threshold value . . . . 33
5.1.1 Scene configuration for threshold=50% . . . . 33
5.1.2 Scene configuration for threshold=30% . . . . 34
5.1.3 Scene configuration for threshold=10% . . . . 35
5.1.4 Scene configuration for threshold=100% . . . . 36
5.1.5 Conclusions . . . . 36
5.2 Results by varying the hierarchy of the objects . . . . 37
5.2.1 Scene configuration for a hierarchy in which all the objects are at the same hierarchical level . . . . 37
5.2.2 Scene configuration for a hierarchy in which every objects depends on all the previous objects in the hierarchy . . . . . 38
5.2.3 Conclusions . . . . 41
6 Summary and Conclusions 43 6.1 Future Work . . . . 44
Bibliography 45
Appendices 46
A Implementation of the hierarchy 47
A.1 Matrix of the hierarchy . . . . 47
Chapter 1
Introduction
From the beginning, Robotics’ purpose has been to help humans, facilitating their work, both in industry and in services. Different from the huge and fast industrial robots development, the development of service robots is progressing more slowly.
One of the reasons for this slow progress is their necessity of reasoning about the world, unlike the industrial ones. They need to deal with unstructured environments governed by human decisions and preferences. The problem comes with the necessity of representing these environments. And that is the point of years of research about the perception and representation of the semantic meaning of the world.
Robots are expected to understand scenes and space at a very high level in a near future. They have to be able to identify every single object in a scene and their function, what they are expected to be used for and why. One example of what is expected could be to imagine a robot working in a hospital taking care of the patients and checking that everything is working correctly in the building. The robot comes into a room to do the routine inspection and to start the checking it begins, for example, with the bed. It will need to know first where the bed is usually placed in the room and then that the bed is the place where patients, the
"objective" which it has to take care of, usually lays. Moreover, the bed is also the object around which other people (doctors, visitors,..) and objects (droppers, small tables for medicines,..) may be placed. This is a possible example of an analysis of the scene that robots could do.
The previous example is an illustration of a long term goal for Robotics. First
it will be necessary to find a suitable way to represent all the information of the
environment, with every single detail. Starting with basic paths of representation,
like metric or numerical data which are simple and accurate, but they do not suit
well when it regards to reasoning. Humans rarely use in their language the complete
and unequivocal descriptions of the objects’ position or relations. Robots need to
convert the human expressions into accurate data, and vice versa, and this is not an
easy task. Imagining an hypothetical case where the robot is looking for one object
by its exact position on the table but, due to small perturbations, the object has
changed its position insignificantly, it still holds the previous relations with the rest
CHAPTER 1. INTRODUCTION
of objects. The robot will have problems to find it in that exact location, indicating that something is wrong when actually it is a correct configuration. It is better if the robot look for the object "on" the table again than only at that exact location as before. The idea is to go further, to achieve a human commonsense with which the robot realizes that in the last example nothing was wrong, with which it is possible to guess object’s behavior in specific scenes such as the most likely locations and movements of the objects or the typical relations among them. In this context the key is to endow robots with this commonsense capability.
The purpose is to describe the environment in some way which takes into account high level semantic components of space and ground them in lower level spatial concepts, which human usually employ. Since the main aim is to achieve this way of reasoning, all research that are focused on the same purpose ([1], [2], [3], [4], [5], [6] and [7]) propose qualitative spatial relations as solution to describe the space.
The spatial relations are the relations which specify how some object is located in space in relation to some reference object. They allow robots to reason in a similar way to humans since they are an important part of the human descriptive language and they also provide qualitative information of the space that makes the learning and reasoning easier. They represent aspects of the space with functional relevance and if some trivial changes happen over time, the description of these aspects will remain stable. Thereby, the idea of encoding the space by using qualitative spatial relations seems the suitable solution for the problem of representation.
1.1 Problem statement
To achieve the long term goals mentioned previously, first they need to be applied at a lower level. It is necessary to establish the problem assumptions and delimitations.
The general idea is to be able to make a description of a specific scenario by using qualitative spatial relations so that in cases of small perturbations in the scene, the description of the scene does not present significant changes.
1.1.1 Delimitations
To this end, a specific scenario will be defined (e.g: a typical tabletop like a desk).
The point is to define patterns in that scenario (e.g: relations between objects) and then transfer these patterns to another general scenario. Ultimately, the robot, with the information of these patterns, will have to be able to make predictions about the likely positions of the different objects in a specific tabletop.
But to achieve this point it is important to make first some assumptions about the relations that are going to be used. It is necessary to establish some properties that the relations need to satisfy to be suitable for being used to describe and encode a scene.
2
1.2. CONTRIBUTIONS OF THE THESIS
1.1.2 Desired Properties
Imagining an example a robot entering into a room to proceed with the data ac- quisition. One of its goals is to be able to measure the locations and geometries of the objects as starting point. The adequate data acquisition is only possible if the robot is endowed with suitable equipment such as sensors, cameras, etc. Thus, one of the desired properties will be:
Perceivable It is desired that the relations can be perceived by sensors, i.e. it has to be possible to use sensors to measure the relations.
After being set the suitable equipment, the robot moves around the room to acquire the data from different points of view. The position of the objects does not change while the robot is moving, but the relation acquired by the robot between the objects may change depending on the view point of the robot. This does not mean that the actual relation between the objects is changing, this is the reason why another desired property is:
View point independent Some relations change when the view point changes.
This means that, for instance, to-the-right-of might turn into in-front-of. It is preferred that the value of each relation does not change with the view point.
Some days after the last data acquisition, the robot comes back to same room to perform a new data acquisition. Probably the positions of some objects have changed, but it does not necessarily mean that the relation between them has also changed. The robot has to be able to understand this. Thus, the stability of the relation will be also a desired property:
Stable Ideally the description of the world should not change as a result of small perturbations of the world (e.g. position of objects). The relations must be stable to small changes in the position of the objects.
It is also important that the robot understands the difference between when a relation is significant or not. That is the reason why it is desired to establish a property which grades the level in which a relation is fulfilled, to go beyond a logical output such as the relation fulfills or not. This is the point of the next desired property:
Continuous output In many cases it is beneficial if the spatial relations is calculated as a continuous number that expresses how well a certain configuration matches that spatial relation rather than only true/false.
1.2 Contributions of the thesis
This thesis makes some contributions towards the vision of being able to repre-
sent the world using qualitative spatial relations. From analyzing the most suitable
CHAPTER 1. INTRODUCTION
qualitative spatial relations for describing a particular space to endow robots with a way to reason about space closer to humans that allow them to make good pre- dictions about how the space is expected to look like.
1.3 Outline
The thesis is structured as following:
• Chapter 2 describes how the related work fits with the desired properties of the current work.
• In Chapter 3 the main hypotheses made during the process and the theoretical details of the method performed is explained.
• Chapter 4 describes the steps performed for developing the method and achiev- ing the desired execution.
• Chapter 5 shows the results of applying the theoretical knowledge in some experiments.
• Chapter 6 summarizes all the process and presents the conclusions and some ideas for future work.
4
Chapter 2
Analysis of Related Work
A lot of research has been focused on finding a way of representation based on the capability of reasoning in a way closer to humans. The research support that adopting human-like cognitive patterns is likely to help robots approach human-like performance in the context of environments that are products of human preferences, [8] and [4]. They justify the use of spatial concepts as the way of representation of the space upholding it provides qualitative abstractions facilitating the reasoning since the spatial concepts are an important part of linguistic interaction with human beings [5]. In this chapter it will be also analyzed how much the relations’ properties in the different research fit with the established desired properties of this thesis.
The purpose of all the research of Sjöö et al. has been to study the best modes to express the semantic spatial knowledge of the real world and, also, to use this new form of expression as a way of guiding visual object search. Their research in [1] and [2] is focused on the study of different ways for introducing spatial relations that can be perceived with robot’s sensors. The most of these studies are working only with the topological spatial relations on and in which, by definition, are view point independent. Moreover, the values obtained for the topological relations fall within the continuous range [0,1].
There is some research that also work with the topological relations in/on, trying to succeed on the searching of the target object. It is more focused on the theoretical part of the spatial relations’ study, specifically on improving the efficiency of the visual search. One example of these theoretical research is the one that Aydemir et al. implement [6]. Their aim is to provide a decision theoretic strategy selection method to obtain a near-optimal search behavior.
Sjöö et al. also focus on the improvement of the efficiency of the search in [5].
Their whole point is to introduce general perceptual measures for the topological relations and to show how they can be used as a way of guiding visual object search.
Also focusing on the topological part of the RCC, Cohn et al. [9] aim to design
logical calculi for qualitative spatial reasoning. With what they call continuity
network, they establish continuous transitions among all the RCC relations, making
the stability of the relations hold.
CHAPTER 2. ANALYSIS OF RELATED WORK
Combining exploration and active visual search in the real world are some of the principal goals for some of the research. At [7] Gobelbecker et al. present a principled planner based approach to the active visual object search problem in unknown environments. The same at [3], where Sjöö et al. use a planner that performs an indirect search with a hierarchical model for representing and finding object locations.
Also the topological relations and their in-depth study are the aim of some re- search like [10]. Assuming the topological spatial relations as the way to describe the environment, Egenhofer et al. aim to reduce the computational effort for topo- logical relations between two point-sets and demonstrate that it exists a framework within which any topological relation falls. Although the work does not specify any- thing about the output of the relations, all the research about topological relations assumed a continuous output of the relations, between 0 and 1.
Other research of Skubic et al. continue exploring different ways of generat- ing multi-level spatial relations [11]. In an attempt to expand their possibilities of making a more accurate description, Moratz et al. and Stopp et al., who are more focused on demonstrating how natural language can be a mode of access to the autonomous systems [4] and a way of human-robot interaction [8], go beyond topological relations like on, in, at, near, surronded. They make use of view point dependent, directional and distance relations that enable a richer description of the environment, and they also have a continuous output: right, left, beside, above, in front of, behind, below,etc.
In all the research the perception of the relations is implicit inside the method, because these relations are going to be perceived by the sensors and cameras imple- mented on the robot. With respect to the stability of the relations, it depends on the research. In most of them the relations between the objects are stable as long as the objects are visible, detectable by the sensors of the robot. Some research specify the stability.
Table 2.1 summarizes the information above, in which it is shown how the re- lations used in each paper relate to each one of the desired properties as it has been explained. When the spatial relations used in the paper fit with one of the properties, in the respective box of the table will appear an ’x’, otherwise it will be a ’-’.
Paper Perceivable View point independent Stable Continuous output
[1][2][3][5][9][10][6][7] x x x x
[11] [8][4] x - x x
Table 2.1. Summarize of how each research adjusts to the desired properties
After observing the previous table it can be noticed that the work by Sjöö et al. [1],[2], [3]and[5] and the rest of the papers in the first row of the table fulfill
6
all the properties defined as desirable in the first chapter. Although the desired properties are some of the properties that are wished for the relations, it does not mean that the rest of properties are not important. For example, one interesting relation would be the relation with the ability to describe all significant aspects of a scene. Sjöö only works with the topological relations on and in, which are not suitable for such a task. The point is to describe the scene by using relations for representing as much detail as possible. According to this point, the information provided by the related work suit with the purpose of the thesis, but in the case of Sjöö, they just limit the research to improve the spatial representation based on topological relations.
All the papers of the table believe also in qualitative spatial relations as a level of spatial instruction knowledge representation. Their basic goals are to investigate the best methods and ways to describe and work with spatial relations and with them to provide a natural language of representation and reasoning to access to the autonomous systems. This is also the point of this thesis, but instead of focusing on the most efficient way to describe a relation or to express it, the ultimate goal is to go beyond the exhaustive search of the perfect spatial relations and to focus on making a good description of the space with which afterwards the robot will be able to make successful predictions.
To get this ultimate goal all the information provided by the related work is going
to be taken as basis, even if it is not fitting perfectly with the desired properties. For
example, some research like [11], [8] and [4] support the use of viewpoint dependent
relations such as right, left, behind,etc. which are expected to enable a richer
description of the environment, which is one of the main aims of the thesis.
Chapter 3
Modeling and Design
Describing scenes by using qualitative spatial relations is the principal aim of the thesis. A scene is the place where an action or event occurs. Every scene is charac- terized by the presence of particular types of objects in it, which help to carry out the specific actions in the scenes. These particular objects are generally tied to each other, which means that their positions in the scene are correlated. Furthermore, the configurations of the objects are not random, the positions and movements of the objects fulfill specific patterns. To achieve a satisfactory description of the scenes it will be necessary to express these patterns and relations between the ob- jects somehow, and this is the function of the spatial relations. To find out which is the best way to proceed and which are the best spatial relations to achieve this purpose is the base of the following sections.
3.1 Methodology
Since the aim of describing scenes by using spatial relations is quite extensive and ambitious, some limitations and assumptions need to be established to start. One of the first assumptions will be to define the workspace, limiting the study area to a tabletop, specifically a typical desk. It will be also required to define the main concepts that are going to be used to identify the objects when referring to them while they are being described, depending on their role in the description.
Providing coherence with some previous works, like in the work of Sjöö et al., the same concepts are going to be used to define the same objects. Thus, it is going to be defined as:
Trajector: the focus object, i.e. the object which is being described referred to another by the relation.
Landmark: the object to which the trajector is referred.
To clarify, in the example ’The mouse is to the right of the laptop’, the mouse would
be the trajector and the laptop would be the landmark.
CHAPTER 3. MODELING AND DESIGN
Subsequently, and following with the methodology, it will be defined a proper set of relations which enable to make a stable description of the scene. With this proper description of the scene, the analysis of the scene may be performed and with it, it will be possible to pass judgment on the configuration of the scenes, such as the existence of patterns in this configurations and the importance of the hierarchy of objects. This points will be analyzed in the following sections.
3.2 Qualitative spatial relations
The procedure for getting a first list of possible relations will be executed by analyzing many different scenes and describing them using the necessary amount of relations to describe every little detail of the scene. This step will be implemented by hand. After this step, it will be collected a list with all these relations and their definitions to, later, select the ones that best suit the final purpose. After having analyzed the different scenes, one possible list of relations is the following:
List of spatial relations:
on: One object supports another, against gravity.
to the left: The trajector is physically located to the left side (from the left lateral edge) of the landmark.
to the right: The trajector is physically located to the right side (from the right lateral edge) of the landmark.
beside: The trajector is located closer to one of the lateral parts of the landmark.
behind: The described object is physically located on the back side of the land- mark.
in front of: Physically located on the front side of the landmark.
close/near: The shortest distance between the objects is not significant with re- spect to the size of the table.
surrounded: The trajector has the landmark around it. The distance between them is short.
surrounding: The trajector is placed around the landmark and the distance be- tween them is short.
connected: The objects are joined, in contact. In some cases one part of one object is inside the other object.
in the center of: The trajector is located in the central part of the landmark that supports it.
10
3.2. QUALITATIVE SPATIAL RELATIONS
at the back: Physically located on the back side or edge of the landmark.
in the front: The trajector is placed on the frontal part or edge of the landmark.
on the corner: The trajector is placed on one of the landmark’s corners. The landmark will be bigger in size than the trajector.
on the right/left side: The trajector is located on the right or left side of the landmark regarding to the landmark’s frame of reference.
on the boundary: The trajector is placed on the boundary of the landmark (bound- ary includes the four borders of the landmark (table): front, back, right, left).
3.2.1 Selection of spatial relations
When the previous list of possible spatial relations is analyzed it can be noticed that some relations are redundant as they are a combination of several other relations, like the following examples:
surrounded/surrounding= (to right OR to left OR in front OR behind) AND near.
on the center/at the back/on the corner/in the front/on the boundary=
on + near to the center/edges/front/etc.
For this reason, is obtained removing all these redundancies, a final selection of relations which, besides being non-redundant, better fit with the purpose of describ- ing the scenes in detail, and also fitting with the most of the desired properties for the relations such as stability, continuous output, etc. The final selected relations are:
on, to right, to left, in front of, behind, near
Before describing each one of the relations it is interesting to explain some assumptions made about the objects to facilitate the comprehension of the definition of these relations that will define the objects’ configuration. It is assumed that the objects are boxes with the size such that the object would fit perfectly inside the box. The objects are considered as point clouds, in order to facilitate the description by using the relations. The table is supposed to be stationary, without changing its position, and the center of the global system of reference will be placed in the left-frontal corner of the table. To simplify, it will be assumed that the objects can only rotate about the z axis.
Formalizing the relations
The relations have a continuous output, which means that the output of the relation
will be a percentage evaluating how much each relation is fulfilled. The final list of
relations is the following:
CHAPTER 3. MODELING AND DESIGN
on: The points of the trajector are inside the boundaries delimited by the external points or edges of the landmark in the xy-plane. Also the height coordinate of the bottom surface of the trajector has to be equal or higher than the coordinate of the superior surface of the landmark. There will be a counter which counts the points of the trajector that fulfill the conditions of being on the landmark and the ones that do not fulfill. Figure 3.1 shows a configuration of some objects on the table formed by points. The result of how much on the trajector is over the landmark will be:
On = pointsOV ER
pointsOV ER + pointsN OT over ∗ 100 (3.1)
Figure 3.1. Frontal view of the schema of some configuration with the objects formed by points
to the right: The points of the trajector whose x coordinate values are greater than the point of the landmark with the greatest value of the x coordinate (xmax), as it is shown in the figure 3.2, where the laptop is the landmark.
There will be a counter of the number of points that are to the right and to the left of this point (xmax). The expression for the calculus of the right relation will be:
T oright = pointsRIGHT
pointsRIGHT + pointsLEF T ∗ 100 (3.2)
123.2. QUALITATIVE SPATIAL RELATIONS
Figure 3.2. Top view of the schema of some configuration with the objects formed by points
to the left: The points of the trajector whose x coordinate values are less than the point of the landmark with the lowest value of the x coordinate (xmin). There will be a counter of the number of points that are to the right and to the left of this point (xmin). The expression for the calculus of the left relation will be:
T olef t = pointsLEF T
pointsRIGHT + pointsLEF T ∗ 100 (3.3)
in front of: The points of the trajector whose y coordinate values are less than the point of the landmark with the lowest value of the y coordinate (ymin).
There will be a counter of the number of points that are in front and behind of this point (ymin). Figure 3.3 shows how this relations is measured being the laptop the landmark. The expression for the calculus of the left relation will be:
Inf rontof = pointsIN F RON T
pointsIN F RON T + pointsBEHIN D ∗ 100 (3.4)
CHAPTER 3. MODELING AND DESIGN
Figure 3.3. Top view of the schema of some configuration with the objects formed by points
behind: The points of the trajector whose y coordinate values are greater than the point of the landmark with the greatest value of the y coordinate (ymax).
There will be a counter of the number of points that are in front and behind of this point (ymax). The expression for the calculus of the left relation will be:
Behind = pointsBEHIN D
pointsIN F RON T + pointsBEHIN D ∗ 100 (3.5) near: It is assumed that if the trajector and landmark are in contact they are 100%
near. The near relation will take into account the size of both objects. It con- sidered that the trajector starts to be near of the landmark if the distance between them is five times the size of the surface of the smaller object (trajec- tor or landmark), otherwise the value of near will be null. Thus, first it will be made a comparison between them to test which one is smaller. The value for the relation near will be the quotient of dividing the minimum distance between the closest points of landmark and trajector (dmin) and the width of the table (widthtable). The table will be the reference to compare if the distance between them is short or long. The expression for near will be:
N ear = (1 − dmin
widthtable ) ∗ 100 (3.6) It is important to notice that at this point, the selected relations for describing scenes which were view point dependent such as left, right, in front or behind become view point independent which is one of the desired properties. Now, with the formalized descriptions of these relations, the relation, for example, "to the left of" will be to the left of with respect to the table, i.e. with the table as reference frame.
14
3.3. ANALYSIS
3.3 Analysis
After defining the relations that are going to be used to describe the scene, the analysis of data will be performed. The analysis consists of observing different scenes of desks and collecting the information about the typical position of the objects and the relations between them in all of them. This step will also be performed by hand. During the analysis it was noticed that some important features in objects’
behavior can be distinguished:
• Patterns in the frequency of their presence in the scenes.
• Patterns in objects’ movements.
• Patterns in the relations between them.
These characteristics are relevant details. Finding a structured way of organiza- tion to collect them and to make them easier to be classified, will help to understand more easily the environment’s knowledge, more specifically the scene’s behavior, and to be able to represent it.
Patterns in the frequency of their presence in the scenes
Since one of the established assumptions of the thesis is to start analyzing a tabletop as a setting, specifically a desk, the amount of possible objects on the scenario will be also limited to this particular scene. Comparing different scenes it is possible to recognize two types of objects in our example scenes: the typical objects that are usually placed on most of the desks, and the secondary objects, that only appear in some of the scenes and not in most of them. Some examples of objects inside each group are:
Typical objects: laptop, screen, keyboard, mouse, lamp, etc.
Secondary objects: books, notebooks, pens, bags, stapler, post-it, etc.
It is necessary to note that this information is very dependent on the scenes.
The collected dataset is very small, therefore these conclusions are not general, they only refer to the specific analyzed scenes.
Patterns in objects movements
Another aspect easy to notice when analyzing the different scenes is that the moves
that each kind of object makes on the table happen according to a pattern. In the
following chapter, it will be possible to notice these patterns by observing the scenes
in Figures 4.1 and 4.4. They are two different images of the same scenario in different
moments in time. The objects that have been classified above in different groups
usually have the same model of movement depending on the group to which they
belong. The objects defined as typical are usually more statics, which means that
CHAPTER 3. MODELING AND DESIGN
their positions on the table do not make significant changes over time. Otherwise, the objects classified as secondary do not follow a pattern of movement and position and they are usually changing their position on the table over time.
Patterns in the relations between them
The patterns referred to the relations between the objects are also dependent on the different kinds of object described above. The typical objects usually show patterns in the relations between them unlike the secondary objects that do not show any significant relations with the rest of the objects. Furthermore, inside the typical objects it is possible to distinguish some special objects that can be defined like key objects. The key object is the object to which an important part of the other typical objects are usually related.
3.4 The hierarchy
By definition a hierarchy is an arrangement of items (objects, categories, etc.) in a graded order. An important hypothesis of the thesis is the classification of the objects by a hierarchy. In the following sections the reasons will be explained why this decision is important and which is the procedure to implement this hierarchy by using the information collected in the analysis.
3.4.1 The importance of a hierarchy
The importance of the hierarchy sustains on the fact of collecting and gathering all the information obtained in the analysis about the scene behavior in an ordered way that facilitate the comprehension of the objects configuration. Having a hier- archy helps the simplification of the process. Knowing the typical positions, moves and relations between the objects makes it simple to build a model that simplifies reasoning about relations, that simplifies the process of describing the scenes and that also simplifies the computational calculi. A hierarchy is a pattern itself of the scene that helps the robot to build a sample schema about how the scene is going to look like. The hierarchy provides all the necessary information about the configura- tion and behavior of the scene and it defines which the significant and insignificant relations of the configuration are.
3.4.2 Building the hierarchy
After analyzing a number of examples and having extracted the main patterns it is possible to define a hierarchy with the main objects that represent the scene in a proper way and the main relations that are obtained from the previous studies.
First of all, it will be necessary to define which the main objects are that form the scene. Studying the example scenes it is possible to obtain a set of objects which defines the scene’s behavior. The following is one possible list with the typical
16
3.4. THE HIERARCHY
objects and some of the secondary ones in the scene from which the process will be built:
Table, Laptop, Screen, Keyboard, Lamps, Mouse, Paper, Notebook, Mouse, Mouse pad, Pen.
In like manner, by studying the example scenes, a prototypical configuration of the scene can be obtained, i.e. the main relations between the objects. To justify these decisions, the next tables show the analysis of six different scenes with six different desks in which all of them the relations between two couples of objects of the final list got measured by hand in reality. The values in the tables represent the value of each relation between the trajector(mouse) and the landmark(laptop or pen) in every scene. Table 3.1 shows the relations between the mouse and the laptop, and Table 3.2 the relation between the mouse and the pen.
Relation(mouse/ Scene 1 Scene 2 Scene 3 Scene 4 Scene 5 Scene 6 laptop)
right 100 100 100 100 100 96
left 0 0 0 0 0 0
on 0 0 0 0 0 0
near 69 88 26 74 62 89
infront 100 100 0 0 61 100
behind 0 0 0 0 0 0
Table 3.1. Percentages for relation between mouse (trajector) and laptop (landmark) for different scenes
Relation(mouse/ Scene 1 Scene 2 Scene 3 Scene 4 Scene 5 Scene 6 pen)
right 100 100 87 100 0 0
left 0 0 0 0 100 100
on 0 0 0 0 0 0
near 0 0 83 0 65 21
infront 100 0 0 96 12 0
behind 0 100 69 0 7 89
Table 3.2. Percentages for relation between mouse (trajector) and pen (landmark) for different scenes
The previous tables show the first clues of an existing hierarchy between the
objects. Analyzing Table 3.1 it is shown how in every scene the relations between
CHAPTER 3. MODELING AND DESIGN
the objects remains more or less stable. This can be seen by observing the values of one of the rows, for example, the row for the relation right. In this row the values for the relation right between the mouse and the laptop are 100% in five of the scenes and 96% in one of them. These values prove the stability of this relation along the different scenes. Like the relation right, the relation infront and near are also quite stable, although their values are not as constant as in the case of the relation right. For the row of near in five of the scenes the mouse is more than 60%
close to the laptop, and just in one of them, scene 3, the value is lower than 50%.
With this values to estimate that the mouse is close to the laptop is reasonable and the same for the relation infront. Since the values of the relations left and behind are opposite, in this case, to the values of right and infront the value of zero does not mean that the objects are not related. With all this information it is acceptable to conclude that the mouse and laptop are hierarchically related.
On the other hand, Table 3.2 with the results between the mouse and the pen shows different results. For example, the row of the relation near shows null values for three of the scenes, two values greater than 60% and one of them around 20%.
With this values is not possible to establish which is a reasonable relation near between this couple of objects. The same for the relation behind, in the case of the scenes 2, 3 or 6 the biggest part of the mouse is placed behind the pen. On the contrary, scenes 1, 4 and 5 show null or almost null values for the same relation.
These discrepancies in the results are evidence of a probable absence of the relation between both objects. The same kind of differences in the values of the rest of relations back up the idea of a nonexistent relation between both objects. This is just an example for justifying when it is reasonable to state that two objects are not related. In this case, this nonexistent relation was expected due to the pen is an odd choice for being the landmark.
Analyzing all the relations between all the objects, as it has been made with the previous tables for the relation between the mouse and the laptop and pen, it is possible to obtain the results which will facilitate the construction of a hierarchy, in which the top of the tree will be the table, and the rest of the tree is shown in Figure 3.4. It is important to emphasize that while this method works for more data, the conclusions are drawn from very little data.
18
3.4. THE HIERARCHY
Table
Laptop Lamp
Screen Keyboard Mouse pad Paper Notebook
Mouse Pen
Figure 3.4. Tree defining the hierarchy of the objects
The hierarchy has by several levels. Each object placed in one of the lowest
levels will depend on the objects in the previous levels with which it is related. This
dependency means that the position and movements of the objects that are related
to others at higher levels will always depend on the positions and movements of the
last ones. For instance, in this case, the position and movements of the keyboard
will be dependent on the position of the laptop and also of the table because they
are hierarchically related. Thereby, it means that in the higher levels will contain
key objects, i.e. the ones on which most of the rest of typical objects depends.
Chapter 4
The Method
This chapter focuses on describing the selected method to proceed and achieve the purpose. It will be explained which the main steps of this method are and the reason why each one of them has been selected for taking part in the achievement of the final purpose.
4.1 Introduction to the model
The basic purpose to be achieved is to describe scenes by using qualitative spatial relations and also to implement the capacity of predicting possible configurations of the objects on the scenes. These configurations are expected to remain stable for small perturbations. This means that even if the positions of the objects are not exactly the same in every new scene, the pattern that the objects follow will keep the same. There are many possible ways to achieve this purpose, the one followed in this thesis will be described along this chapter. Its basic ideas are collecting data from real scenes, describing different scenes with the formalized relations and using the information from these descriptions to create new scenes which follow the same patterns.
The reason why this method has been chosen is due to the desired aims are achieved in a simple and effective way. Once the definitions of the spatial relations are formalized, it is possible to describe the scenes in a qualitative way. Extracting the patterns from the descriptions, the program will be able to predict possible configurations for the objects. The number of these possible configurations is infinite but all of them have something in common, the pattern of the relations between the objects. This means that while matching this pattern, the configurations do not need to fulfill fixed positions. The established purpose will be achieved while the pattern of the relations remains constant.
Therefore, for developing the method, one of the first steps will be the acquisition of all the necessary data to be able to get an appropriate base for starting working.
It will be taken some pictures of different scenes of a desk, the specific tabletop.
After collecting different images, they will be analyzed to acquire the most important
CHAPTER 4. THE METHOD
patterns and similarities between them. Once the patterns between them have been identified, a list of the most common objects on the specific tabletop will be defined, their typical geometries and positions. The importance of this is to have real data with which the results of the process can be compared. Thus, assuming as known the geometry of the objects and extracting the positions from the data, it is possible to proceed deeper into the method.
4.2 Data collection
4.2.1 Photo shot
To obtain the data, the first step of the process is to collect some images of real desks to extract all the possible information from them. Figure 4.1 represent an example of one of the real desks that have been used for the data collection. In the image it can be appreciate some of the objects defined as typical in the previous chapters: laptop, lamp, mouse, mouse pad, papers, notebooks, etc.
Figure 4.1. One of the real examples of a desk that has been taken to make the analysis and the data collection
22
4.2. DATA COLLECTION
4.2.2 Data extraction
After analyzing many scenes and establishing some assumptions it is possible to build an example scene with typical data obtained from all of the scenes. Table 4.1 is an example to justify why this prototype configuration is taken for the defined objects in the scene. In this table the values of all the relations are shown between two objects of the desk, the mouse and the laptop, for different example scenes.
After analyzing all the scenes the mean and the standard deviation of all the values for each relation has been calculated. The results show the stability in the values of the relations along the scenes. For example, the right relation has a mean of 99.33%
and a standard deviation of 1.49%. In this case it is reasonable to state that in the prototype scene the mouse will be placed to the right of the laptop. By studying all of them it is justified that it makes sense to consider the mouse placed to the right, in front and near to the laptop in the example scene.
Relation(mouse/ Scene 1 Scene 2 Scene 3 Scene 4 Scene 5 Scene 6 Mean Std. Dev laptop)
right 100 100 100 100 100 96 99.33 1.49
left 0 0 0 0 0 0 0 0
on 0 0 0 0 0 0 0 0
near 69 88 26 74 62 89 68 21.12
infront 100 100 0 0 61 100 60.17 16.26
behind 0 0 0 0 0 0 0 0
Table 4.1. Percentages for relation between mouse (trajector) and laptop (landmark) for different scenes
After analyzing the rest of the values for the relations between all the objects
selected for the example scene, it is possible to define the most likely positions and
geometries for all the objects in the scene. The configuration of the objects showed
in Table 4.2 is just one possible example of the configuration that a typical desk
could present and it will be the base of the following experiments. This example
configuration taken as a prototype scene is represented in Figure 4.2.
CHAPTER 4. THE METHOD
Objects x (m.m.) y (m.m.) z (m.m) length width height
Table 600 375 15 1200 750 30
Laptop 520 400 50 340 230 40
Mouse 802 233 55 50 90 40
Mouse pad 855 285 32.5 210 210 5
Screen 520 550 180 360 40 300
Keyboard 540 220 40 375 120 20
Lamp 920 590 205 125 125 350
Sheet 390 136 52 210 297 15
Notebook 595 136 40 210 297 20
Pen 60 400 37 15 150 15
Table 4.2. Positions (coordinates x, y and z of the center of mass) and geometries of the objects on the table)
In Table 4.2 the first three columns represent the position where the center of mass of each object will be placed, in the axis x, y and z of the system of reference. The last three columns contain the values of the length, width and height which determinate the geometry of the objects. This configuration will be the configuration used as a prototype scene to work with in the rest of process.
4.3 The method
In this section it will be described step by step the method and one way to test it. It will be focused on the theoretical explanation of the process. Starting with the formalization of the selected relations already done in the previous chapter, the method will explain the steps to achieve the established goals. From describing different scenes to sample example scenes which follow the patterns in the configu- ration of the objects that present the real scenes.
4.3.1 Generating descriptions for scenes
Descriptions of the different scenes will be generated with the relations that were previously described. The aim of this step is to describe the scenes in a way that the computer is able to understand. The way to proceed will be: for each one of the relations right, on, infront, etc., one matrix will be generated with all the values for the relations between all the objects of the scene. At the end of this step the description of the scenes will be generated and it will be represented by a collection of matrices, one for each relation, with the values of all the relations between all the objects. One example of the mentioned matrices is the following table. The table represents the values for the relation infrontof describing the real scene showed in Figure 4.1. Since the size of the matrix is too big to be shown completely, it will be
24
4.3. THE METHOD
shown the relations between some of the objects in the scene, but not between all of them.
Objects Table Laptop Mouse Mouse pad Lamp Box Cup Case Notebook Pencil
Table 0 38.16 25 25 67.11 78.95 48.68 68.42 0 50
Laptop 0 0 0 0 95.83 100 37.50 95.83 0 37.50
Mouse 0 100 0 0 100 100 100 100 0 100
Mouse pad 0 50 4.55 0 100 100 86.36 100 0 90.91
Lamp 0 0 0 0 0 49.70 0 0.59 0 0
Box 0 0 0 0 0 0 0 0 0 0
Cup 0 0 0 0 100 100 0 100 0 12.50
Case 0 0 0 0 0 100 0 0 0 0
Notebook 6.67 100 70 66.67 100 100 100 100 0 100
Pencil 0 0 0 0 100 100 0 100 0 0
Table 4.3. Matrix with the values for the relation in front of between all the objects of the real scene in Figure 4.1
The values of the table describe the scene according to the definition of each relation made in the previous chapter. In this case, each value indicates how much in front is the object of the corresponding row with respect to the object of the column. Thereby, looking, for instance, the row of the notebook and its relation with the mouse pad, the value showed in the table is 66.67%. This value means that 66.67% of the notebook is in front of the mouse pad or, mathematically, the values of the y coordinates of 66.67% of the notebook points are less than the value of the point of the mouse that has the minimum value for the y coordinate.
Subsequently, after the collection of all the descriptions of the different scenes, the main values of the relations between each object will be synthesize to obtain one typical description of a prototypical scene by summarizing all of them. This example scene will be used in the following sections for proceeding with the method.
With the typical positions of the objects that have been mentioned in Table 4.2,
one representation of a typical scene with the selected object and relations is shown
in Figure 4.2.
CHAPTER 4. THE METHOD
SCREEN
LAMP
MOUSE PAD
MOUSE
KEYBOARD LAPTOP
PAPER
NOTEBOOK PEN
Figure 4.2. Model of the table with the defined positions and geometries of the objects
4.3.2 Creating a model that can generate configurations of the objects The aim of this step is to create a model which generates reasonable configurations of the objects in the scene. This model will take a labeled scene, which contains the geometry and position of the objects, and the hierarchy of these objects as input.
With this input the relations between the objects will be evaluated as it is explained in the previous section. The procedure followed by the model is the one showed in Figure 4.3.
Figure 4.3. Scheme of the used model
26
4.3. THE METHOD
Labeled scene and Hierarchy
The input of the model will be the collected information about the objects’ geometry and position and their hierarchy, for this example it will be used the hierarchy defined in 3.4. With this information, it will be possible to establish the priority of some objects over the others to be placed on the tabletop and to known which relations are relevant and which ones are not.
Generating a qualitative description
With the descriptions of the relations defined previously, the relations between the objects on the tabletop will be evaluated. These values will be stored in the matrices to work later with them. Table 4.4 presents an example of description for the labeled scene in Figure 4.2, specifically for the relation to the right. The values of the table indicate how much to the right is the trajector (object of each row) situated with respect to the landmark (object of each column).
Objects Table Laptop Mouse Mouse pad Screen Keyboard Lamp Sheet Notebook Pen
Table 0 42.97 20.66 31.40 42.15 39.67 17.35 58.67 42.15 88.43
Laptop 0 0 0 0 0 0 0 57.14 0 100
Mouse 0 100 4.54 63.63 100 100 0 100 100 100
Mouse pad 0 100 0 0 100 100 0 100 100 100
Screen 0 5.40 0 0 0 0 0 56.76 2.7 100
Keyboard 0 10.52 0 0 7.89 0 0 60.52 7.89 100
Lamp 0 100 13.61 100 100 100 0 100 100 100
Sheet 0 0 0 0 0 0 0 0 0 100
Notebook 0 9.09 0 0 4.54 0 0 95.45 0 100
Pen 0 0 0 0 0 0 0 0 0 0
Table 4.4. Matrix with the values for the relation to the right between all the objects
Adding an object on the table
The objects will be added on the table with respect to their position on the hierarchy.
The first object to be added will be the object with the highest level in the hierarchy
shown in Figure 3.4. The next objects will be added according to the hierarchical
order, completing every branch before starting a new one. Each object will be added
in a random position on the table, but this new position has some restrictions which
will be tested in the following step.
CHAPTER 4. THE METHOD
Testing if the relation holds
Once the new position of the added object is known, the relation with respect to the other objects on the table will be evaluated. The idea is to compare these new relations with the previous relations that are stored in the matrices for the same objects, but bearing in mind that it will be only necessary to check the relations between the objects that are related to the new object hierarchically. That is, only the objects with which are directly dependent according to the hierarchy in Figure 3.4 need to be checked.
The comparison will be bounded within a threshold which will determinate if the new position holds or not with respect to the original one. The function of the tester is to check this comparison. If the new position holds, the next object in the hierarchy will be added; if not, the same object will be added in a different position checking the relations until one of the positions fulfills the requisites. This process will be repeated a limited number of times
1. If the new position never fits inside this limited number of times, the program will change the position of the previous object in the hierarchy related to the last one and it will proceed again in like manner.
4.3.3 Taking a new scene and generating a description with relations
The idea is to generate the description of a new scene proceeding like in the previous sections. The aim is to test if the description generated for this new scene makes sense compared to the descriptions generated for all the previous scenes, from which the patterns about the positions and geometries of the objects have been extracted.
Figure 4.4 shows an example of a new scene. This scene is for the same scenario showed in Figure 4.1 but in a different moment in time. By describing this new scene it will be tested if the configuration between the objects holds over time.
To be able to compare the new description with the description made previously for Figure 4.1 in Table 4.3, it will be shown the description for the same relation, infront, used in the table 4.3 and between the same objects. Table 4.5 shows the results for the description of the new scene.
150 in our tests
28
4.3. THE METHOD
Figure 4.4. Real example of a new scene to be described
Objects Table Laptop Mouse Mouse pad Lamp Box Cup Case Notebook Pencil
Table 0 37.92 32.89 25 67.11 78.95 84.21 55.26 0 5.26
Laptop 0 0 0 0 95.83 100 100 54.17 0 0
Mouse 0 4.57 0 0 100 100 100 100 0 0
Mouse pad 0 57.32 48.96 0 100 100 100 100 0 0
Lamp 0 0 0 0 0 53.85 87.57 0 0 0
Box 0 0 0 0 0 0 29.41 0 0 0
Cup 0 0 0 0 0 0 0 0 0 0
Case 0 0 0 0 100 100 100 0 0 0
Notebook 26.67 100 100 86.67 100 100 100 100 0 36.67
Pencil 0 100 100 100 100 100 100 100 0 0
Table 4.5. Matrix with the values for the relation in front of between all the objects of the new real scene in Figure 4.4
CHAPTER 4. THE METHOD
Comparing the description of the new scene with the description of the scene in Figure 4.1, it is noticeable the fact that the analyzed objects keep their configura- tion even when the values for the relations are different between both scenes. For example, looking the row of the mouse pad in both tables 4.3 and 4.5, its relation with the laptop is 50% infront in Table 4.3 and 57.32% in Table 4.5. And the values for the mouse with respect to the laptop are 100% and 4.55% in each one of the scenes. This results justify that even for different values in the descriptions, like between the mouse and the laptop, the configuration of the objects holds. This is possible because, although the mouse is not in front of the laptop in the scene in Figure 4.4, it is still placed on the mouse pad, the value between the mouse and the mouse pad is 0% in both scenes. This means that only one value does not establish if the configuration of the scene is holding or not, but it is a set of the values and relations between the objects which establish the real configuration. These patterns are what is expressed in the hierarchy.
It is also noticeable that this stable configuration is only holding when the objects defined as typical are involved. When it is about secondary objects the relations more seldom hold from one scene to another, as it is the case of the relations of, for instance, the cup and the case, which are secondary objects, with the lamp, a typical one. For the scene in Figure 4.1, the cup is 0% in front of the lamp and the case 100%. In the scene in Figure 4.4, the values are the opposite, being the cup 100% in front of the lamp and the case 0%. The values are different but they also are different for the relations with the rest of objects, typical and secondary ones, like with the laptop for example. Thereby, analyzing the whole description of the scenes it is possible to determine that the relations for the defined as typical objects (mouse, laptop, lamp) remain stable over the time unlike the relations of the objects defined as secondary in the section 3.5 (pencil, case, cup), which change considerably over time.
4.3.4 Sample example scenes
With the generated descriptions, the model that can generate proper configurations and the gathered information from describing different scenes and the scenes over time, the final step will be to sample some example scenes by introducing different inputs to the process. These example scenes will be compared with the original ones to observe if they maintain the same patterns in the distribution of the objects. The purpose is to demonstrate the importance of the hierarchy of the objects regarding with these patterns. The next chapter will show some results of example scenes with different inputs.
4.4 Discussion of the method
By analyzing different scenes we extracted the typical patterns in the spatial con- figuration of objects. Describing scenes using qualitative spatial relations has been proofed to be an efficient method to also predict possible new scene configurations
30
4.4. DISCUSSION OF THE METHOD
as well as keeping the original descriptions stable under the influence of small per-
turbations of the setup. These patterns can successfully be applied to generate new
scenes with reasonable collections of objects which will still follow our originally es-
tablished patterns. The generation of descriptions of the different scenes will provide
the necessary information for establishing the typical patterns in the configuration
of the objects. Once the patterns have been set, the model will be able to generate
new scenes with reasonable configurations of the objects following the established
patterns. Thereby, the purpose of being able to generate descriptions of the scenes
will be achieved as well as the one about predicting possible configurations which
will hold even in case of small perturbations in the scene over time.
Chapter 5
Experimental evaluation
In this chapter some results of the process explained in the previous chapter will be presented for different combinations in the input. The inputs of the process are the matrix with the objects, the hierarchy of the objects in its matrix form, and the threshold as a percentage that restricts the accuracy of the tester step mentioned in the section 4.3.2.
5.1 Results by varying the threshold value
The threshold will introduce a restriction when the program for generating a new scene, explained in the section 4.3.2, is adding a new object to the scene. The threshold will determine if the comparison between the new relation and the typical one holds, i.e the difference between them is within the threshold. Thereby, it is expected for high and less restrictive values of the threshold, a more disordered configuration of the objects without respecting the reasonable relations. Otherwise, an ordered structure in which the relations make sense is expected for lower values in the threshold. In the following, some examples for different threshold values, with the list of objects and hierarchy defined in previous chapters are presented.
5.1.1 Scene configuration for threshold=50%
With a threshold of 50% it is expected that the relations between the objects that are hierarchically related fulfill the expected configurations, but without much accuracy.
This means that the new relations between the objects may differ from the value of the as defined typical relations in at least 50% of the value of the last one. According to the hierarchy in Figure 3.4, it is possible to distinguish in Figure 5.1 relations in the positions of the objects of each branch. The laptop, the screen, the keyboard, the mouse pad and the mouse have a typical configuration except for the position of the keyboard, which is due to the high threshold. Also the lamp and the paper and notebook are fulfilling the common configurations, staying close to each other.
The function of the hierarchy can be noticed in the overlapped positions of the
CHAPTER 5. EXPERIMENTAL EVALUATION
laptop and the pen. Since they are not related trough the hierarchy, they don’t need to fulfill any relation. A future more complete model would have to take this information into account.
Figure 5.1. Result for all the objects with a threshold=50%
5.1.2 Scene configuration for threshold=30%
Since the value of the threshold is lower than in the previous example, the relations are expected to be fulfilled more precisely, but still with some inaccuracies. Although the objects are placed on the table in a not very common location (more to the left side of the table than usual), the relations between them are fulfilled with more accuracy. For example, in this case the keyboard is not completely over the mouse pad and the pen is over the notebook. It is also perceptible the influence of the hierarchy observing the pen and the notebook placed on the keyboard, due to they are not hierarchically related and this configuration is allowed.
34
5.1. RESULTS BY VARYING THE THRESHOLD VALUE
Figure 5.2. Result for all the objects with a threshold=30%
5.1.3 Scene configuration for threshold=10%
Figure 5.3. Result for all the objects with a threshold=10%
With 10% of threshold it is expected to have a configuration of the objects similar
to the one taken as typical example. The result presented in Figure 5.3 shows now a
more centered position of the objects. The screen, the keyboard and the laptop have
now a desirable configuration as well as the mouse pad and the mouse respect to
them. The other branch of the hierarchy in Figure 3.4 is also fulfilling an acceptable
CHAPTER 5. EXPERIMENTAL EVALUATION
configuration, since the lamp is at the back side of the table to illuminate the paper and notebook, and the pen is placed also close to the notebook.
5.1.4 Scene configuration for threshold=100%
Since the values of the relations are rated between 0 and 100, a threshold of 100%
means that the difference between the new relation and the typical one may be up to one hundred, ergo the new relation may not fulfill any pattern, as it is the case of this example. The most of the objects are placed in random positions without keeping any relation: the keyboard behind the laptop, the mouse pad and mouse to the left of the laptop, the paper and the screen almost out of the table. This example proves with clarity the effect of the threshold in the process.
Figure 5.4. Result for all the objects with a threshold=100%
5.1.5 Conclusions
The effect of the threshold in the process has been demonstrated in the previous examples. The lower the value of the threshold is, the more accurate is the fulfillment of the typical configuration. The threshold is a restriction for the process to achieve a satisfying configuration of the objects on the table. This is a reason why the value of the threshold will be connected to the property of stability of the relations. As it has been mentioned, with lower values of the threshold the final configurations of the objects are more accurate, which implies a higher stability of the relations.
A high stability of the relations means that the objects will fulfill the reasonable configurations with higher accuracy. By studying the trade-off between threshold and stability it is noted how the stability improves when the value of the threshold decreases. In general terms this means that the configuration of the scene will be
36
5.2. RESULTS BY VARYING THE HIERARCHY OF THE OBJECTS
more stable when the restrictions imposed to the process will be tighter, which is when the value of the threshold is lower.
5.2 Results by varying the hierarchy of the objects
The hierarchy is another input of the process and the most important one. It estab- lishes the relations between the different objects, setting which ones are significant and which ones are not. The hierarchy gives the configuration of the scene its be- havior. The results of the previous section have been generated with the hierarchy in Figure 3.4 as constant input. The following examples will show the results for different configurations of the hierarchy, the hierarchy will be the variable input of the process and the threshold the constant one.
5.2.1 Scene configuration for a hierarchy in which all the objects are at the same hierarchical level
In this example all the objects on the table are at the same level in the hierarchy, which means that none of the objects is related to the others, all of them are independent. Since they are only dependent on the table the result expected may be whatsoever, from all the objects clustered at nearby positions to each object in a different random and not significant position on the table. Figure 5.5 shows the schema of the new hierarchy.
Table
Laptop Screen Keyboard Mouse Mouse pad Lamp Paper Notebook Pen
Figure 5.5. Schema of a hierarchy in which all the objects are at the same level
