Communicating relatedness
The development and implementation of a
metaphor in a web interface context
Hans Bergstrand,Thor BrinkThanks to
First we would like to thank Professor Srinath Srinivasa, Aditya Rachakonda and Mandar Mu-talikdesai for helping us to evolve our thoughts and think in new directions with our prototypes. A big thank you to the entire Batch of 08 from IIIT-B for helping us with our long user tests. We would also like to especially thank Sandeep, Tarun, Sushant and Swapnil for aiding us with technical problems as well as giving us feedback in further solutions of our prototypes.
Ricardo, Murthy and Harsha we would like to give a special place of appreciation for their help with matters within school as well as outside school.
Finally we would like to thank professor Dinesha and professor Sadogapan for helping us with contacts with companies as well as practical issues at campus.
Table of Contents
Introduction ...6 Research Question ...7 Context...7 ... 3.1 Web Interface 8 ... 3.2 Environmental Necessities 9 ... 3.3 Web 2.0 10 ... 3.4 Stakeholders 11 Definitions...11 ... 4.1 Communication 11 ... 4.2 Keyword Extraction 12 ... 4.3 Metaphors 13 Methods ...16 ... 5.1 Brainstorming 16 5.2 Online Questionnaires...16 ... 5.3 Paper Questionnaires 17 ... 5.4 Pilot Tests 17 ... 5.5 User Tests 17 Limitations ...17 Design Process ...18 ... 7.1 Prototype 1 187.2 User test 1 Prototype 1 ...21 ...
7.3 Re-design Prototype 1 according to user test 1 24
...
7.4 User test 2 Prototype 1 26
... 7.5 Re-design of Prototype 1 according to user test 2 27 Design Process Prototype 2 ...28
...
8.1 User test 1 prototype 2 31
... 8.2 Re-design of prototype 2 acccording to user test 1 32
...
8.3 User test 2 prototype 2, Sweden 33
...
8.4 Paper survey prototype 2 34
Discussion ...35
Conclusion ...38
Works cited...41
Abstract
This report is an evaluation of if it is good or bad to use a metaphor in order to display the results of an academic search engine in a web interface. In order to evaluate this we are describing our work with developing two different web interfaces for an academic search engine by the name Silverfish. This project has been a co-operation between Indian Institute of Information Technol-ogy in Bangalore, India and Malmˆ University K3, Sweden.
To start our report we describe how we see our context we are to work within. We define our stakeholders as being academics worldwide and also define that we are working within a web 2.0 context. To strengthen our choices regarding the design process of the two different interfaces as well as in order to give more validity to our discussion surrounding metaphors we continue with presenting different studies and facts that give more weight to the above mentioned parts. To make it possible to create the interfaces we have made use of several methods. We give a short definition of how these methods are to be used and later describe in the design process how we have made use of them.
To describe how we have made use of the methods as well as to describe how we have developed our prototypes we continue our report with describing the design process, regarding which deci-sions we have made and why we have made them. To summarize our report we come to a con-clusion regarding our thesis question; communicating related key phrases through web interface metaphors; good or bad?
Regarding our question we have found that the orientational metaphor we are using does not work as it is supposed to. We believe that further studies are required in order to get a deeper un-derstanding of how the user understands the orientational metaphor we are using. This informa-tion could help us come to an understanding of how we could make better use of our tional metaphor, or help us find out of a metaphor that would be better to use than our orienta-tional metaphor.
1 Introduction
We first came in contact with the Silverfish project through Professor Srinath Srinivasa and Adi-tya Rachakonda from the Indian Institute of Information Technology in Bangalore (IIIT-B) who proposed the collaboration between us and the Silverfish team. We were given the opportunity to join the team to develop a web user interface for the Silverfish project. The project started in 2005, under the name Vidiyavahini and was supported by a grant from the HP Adaptive Enter-prise Group.
The objective of Silverfish is to automatically extract certain entities from uploaded academic papers, uploaded Call for Papers, course websites and Institutional web pages. These entities are such as titles, authors, abstracts, institutions and keywords. These are all extracted in a pattern of certain algorithms, which run along a co-occurrence graph, which are part of a set of algorithms called Knowledge Explicator. This means that the extracted entities are cross-checked with enti-ties from other documents and uploads to establish if there might be some kind of relationship between them (Srinath Srinivasa, 2008).
Keywords that occur in such a way that there can be an assumption made of an existing relation-ship are known as co-occurring keywords. The systems knowledge of these relationrelation-ships is meant to aid the user by presenting information that is of relevance in more ways than e.g. that a title matches the words put in a search field. The way it is supposed to aid the user is by drawing conclusions about which kind of information the user is interested in. The system will draw these conclusions based on keywords with different weights that are connected to the user and key-words that are connected to different entities in the system. The system will over time learn more about the specific interests of the user to become more precise when suggesting information. Silverfish can be compared to other academic search engines like ACM, Microsoft Libra and CiteSeerX. The main difference between these and Silverfish is that Silverfish lets its users up-load documents or insert links to make the content grow while the others rely on crawling the web on its own for new content (Srinath Srinivasa, 2008). By our opinion this is a feature that crosses Silverfish into more being a community, web 2.0, web site than just as to act as a pure search engine to the user. We will give an explanation of web 2.0 further down in this report.
One of the main issues for the Silverfish team was how to best present the information fetched by the algorithms that run along the co-occurrence graph to make this into as much of an asset as it could become to a user.
In the process of creating an interface for the Silverfish we will focus on researching the best way of communicating the co-occurrence relations between keywords, and at the same time fo-cus on delivering two functional interface prototypes for the Silverfish project. We wish to find a way of communicating the results that can be the outcome of this process in such a way that will make this into an as effective and efficient experience as possible to the user.
We have chosen to explore the possibility and the outcome of us trying to communicate these results by using one or a number of metaphors in a web interface context. The reason that meta-phors were brought to mind is their ability to communicate something that we are not familiar with by using terms that we find familiar. We will work towards finding a metaphor that will communicate the related keywords while exploring the benefits and limitations of doing so. In order to be able to work as effective as possible we have divided the work into certain areas, where one of us was in charge to make certain that that area would be finished in time. We di-vided the work into a programming part, design part and a written part. Thor has been in charge of the programming part and Hans has been in charge of the written part. The design part of the work has been an ongoing process were we both have been equally involved, in developing the prototypes.
2 Research Question
The question that we will answer in this report is: Communicating related key phrases through web interface metaphors; good or bad?
3 Context
When designing for Silverfish we are working within the web interface context. Our stakeholders are defined, by the Silverfish team, as academics whether it is students or faculty. We are
work-ing towards creatwork-ing a web interface that can be defined as a web 2.0 interface. As mentioned earlier web 2.0 will be explained later in this report.
Some of the first web pages were created in the 1990s. There were no actual standards set yet, and all the pages were mainly text based with next to no design or thought of approaching the user. The user’s expectations were at the same time very limited, so the developers did not have so much to live up to. Today the situation is completely different. This leads to that there is a great competition within web development for trying to create a page that makes the user willing to spend more time at the specific page (Dave Lawrence, 2007). The user needs have also grown and people are quick to leave a certain page if they do not find it sufficient (Dave Lawrence, 2007).
We have been studying different sites that are similar to the Silverfish site. These pages are Cite-SeerX, Google Scholar, ACM, DBWorld and Libra. These sites all have similarities with Silver-fish and are aimed at the same stakeholders (Srinath Srinivasa, 2008). One thing that we have found is that we consider all these pages to be in a state where they give us a feeling of being in a state of development. The information is there, but without any thought of how to present it to the users. We are well aware that the main thing that the user wants to accomplish when entering one, of the above mentioned sites, is to retrieve information quickly. We still believe that to give the user a feeling that there is put some thought into the page can create an environment that is more inspiring.
3.1 Web Interface
When designing within a Web interface context there are several factors that you have to con-sider (Dave Lawrence, 2007). We will briefly give an explanation for some of the key factors to consider.
The layout is important for creating an easily understandable page. To be able to accomplish this you need to be familiar with some basic design standards. These standards are balance, propor-tion, rhythm and unity. The goal is to place items on the web page in such a way that the user easily can find his or her way around the page and accomplish the tasks. When filling a page with information there is one way that has shown positive results. The method used, is to place the different texts and objects within a grid format, this creates a pleasant feeling for the user
(Dave Lawrence, 2007). Although the use of grid systems is a good way to create order in our page you should not let it overrule your design (Dave Lawrence, 2007).
Type and Typography are a useful way of displaying messages or uniqueness. When using type within computers they are named fonts. There are various ways in which you can use fonts in order to create a unique web interface context. When altering the appearance of the letters or their size you can create a sense of meaning for the user. The different appearances of the letters can help the user to get a sense of what the page is supposed to display, and this is only with al-tering the fonts (Dave Lawrence, 2007).
3.2 Environmental Necessities
There are other aspects that have to be taken in consideration when designing for a certain con-text, these are called environmental necessities. There are four environmental necessities that need to be thought of when establishing the requirements. The first requirement is the physical environment this includes the extent of light and noise that will be in the area where the system is to be used. When taking the physical environment in consideration there can be factors to think of like for instance if the users will have to wear protective suits, if the area where it is used is a crowded one or not and so on (Jennifer Preece, 2002).
In our case when working with Silverfish we have some problem to associate to this factor due to that Silverfish is meant to be worldwide. We have assumed that all Universities around the world are of a standard that does not create any problems of the above mentioned factors to the users. Secondly you have to consider the social environment. The issues faced are if the users are col-laborating or need to coordinate their work in any way. This means that perhaps the users are working together in a project and need to be able to access the information at different stations at the same time, or are the users working by taking turns in editing the information. If the users are working together in a project perhaps they are spread around the world and need ways of com-municating (Jennifer Preece, 2002).
In Silverfish there was initially a messaging system that enabled the users to communicate. In the case with Silverfish we find it necessary to have some kind of messaging system because of the
fact that students often live some distance away from University or from their co working stu-dents. We have added a way for students to also submit their not yet finished papers, in order for fellow students and professors to critique the texts. This can also be good for students working together in projects to be able to upload the part that they have written for the other group mem-bers to read. The user only needs to upload a paper and then choose to whom, or which group, it should be sent and the draft or paper will be sent.
Thirdly is the aspect that handles the organizational environments, meaning if and to which ex-tent the provider supplies user support. Also what needs to be considered is how the communica-tions infrastructure is working. The ways in which the hierarchical administration is built can also have an effect (Jennifer Preece, 2002). This aspect has been hard to take in consideration when working with Silverfish.
The fourth and last aspect is the technical domain, meaning issues with for instance what techni-cal limitations that needs to be considered and which platforms that the system should be com-patible with (Jennifer Preece, 2002). Regarding this aspect we do need to have in consideration to make a highly adaptive product so that it can work on platforms worldwide.
3.3 Web 2.0
As we mentioned earlier we are working within a web interface context. We are also working with creating and enhancing the web 2.0 part of the web interface. In order to understand what the web 2.0 context is we will give a brief explanation of it.
Web 2.0 is a network that is used as a platform. Web 2.0 applications are the ones that take ad-vantage of the full capabilities of the platform. It delivers software that enhances in quality and usability the more the users use it. Also the administrators of the page post data, which can be used and altered in any way by anyone. All in order to create a page where everyone is partici-pating in a network, to enhance the user experience (O'Reilly, 2007). We are aware that there are several definitions, of the web 2.0 but we believe the one mentioned above to be the one that de-fines it in the most simple and straight forward manner.
3.4 Stakeholders
As we mentioned earlier our stakeholders are academics, these are also a part of our context. Stakeholders are defined as “people or organizations who will be affected by the system and who have a direct or indirect influence on the system requirements” by Kotonya and Sommerville (Jennifer Preece, 2002). Kotonya and Sommerville also mention that the meaning in general is that when designing a system, the number of stakeholders for your system will be wider spread than you believe and also people that you do not initially consider as stakeholders could become a part of the system (Jennifer Preece, 2002).
4 Definitions
4.1 Communication
When talking about communication you can mean several things. It can be communication through speech, gestures, clothing, television and communication of different activities on a computer screen (Fiske, 1990). Communication is vital in our everyday life, without it children and parents could not communicate each other’s needs or computer programmers could not de-sign computer software to communicate with the hardware and so on (Hauser, 1997).
Communicating face to face is not accomplished only by words; it also involves a frequent use of facial expressions, the use of short words that acknowledge that you are listening as well as ges-tures (Jennifer Preece, 2002).
What we mean by “communicating” in the question on which this report is based on is that we need to give the user an understanding of a particular idea. This idea is the relation between dif-ferent entities in a system. A choice that we have based on the context, in which this idea is pre-sented, is to present it graphically, either by text or by images.
In everyday conversations we often experience something called breakdowns. Breakdowns are usually experienced when two, or more, people are having a conversation and one person for in-stance changes the subject without the others acknowledging the change of subject. In every day
communication face to face this problem is often easily solved by our ability to repair these situations.
When using electronic communication the ability to repair is often greatly decreased, and there-fore the problem is not mainly breakdowns in communication but rather the lack of ways in which we can recover from the breakdowns (Alan Dix, 2004). This is something that we will have to take in consideration throughout the whole process.
4.2 Keyword extraction
Keywords are known to be either one word or a couple of words that grammatically give an ex-planation of a document (K Frantzi, 2000). They are meant to contain enough information to cre-ate an understanding for the user what the main topic and content of the full document is (I. Witten, 1999).
We will here describe how the extraction of key words and key phrases is accomplished in Sil-verfish. We will start by giving an explanation of three components used in SilSil-verfish. The com-ponents are Knowledge Explication Algorithms (KEA), Lucene and MySQL (Srinath Srinivasa, 2008).
KEA is an adequate and functional algorithm used to extract key phrases, either single keywords or multi-word key terms. KEA works in two stages, training and extracting. During the training stage KEA creates a learning model, taking use of foundation documents which have human authored key phrases. More clearly KEA selects two candidate key phrases from the recom-mended document. For each candidate two feature attributes are estimated, the attributes are Term Frequency–Inverse Document Frequency (TFIDF) (Appendix 1) and first occurrence (Yongzheng Zhang, 2005). To estimate the first occurrence you take the amount of words that exist before the first display of the candidate and divide that with the complete amount of words in the document. The candidates that are found that also are human approved key phrases are de-cided as samples in the KEA construction model.
When extracting from new documents, KEA determines a couple of candidate key phrases and estimates their two feature values as explained earlier. After this stage the candidates are given a
weight, this determines how likely it is that the candidate is a key phrase (Yongzheng Zhang, 2005).
Lucene is a text search engine library that is created in Java and it is convenient for most applica-tions that need full text search. The basic approaches in Lucene are index, document, field and term. The index consists of an array of documents. The document is an array of fields. The field is a named array of terms. The term is a text string. An equal text string in two various fields are acknowledged as an unrelated term. This enables that the terms are illustrated as two combined strings where the first string names the field, and the second string names the text within the field. The index saves data regarding terms in order to simplify term based search productivity (Uwe Schindler, 2007).
MySQL is a highly popular database, mainly because of its high performance, easy usage and compatibility with most the popular platforms (Sun Microsystems).
Silverfish takes advantage of KEA in the way that it extracts key phrases from the entire text document, and then takes advantage of these key phrases to require more knowledge about the paper (Srinath Srinivasa, 2008).
Lucene is used within Silverfish in the way that it stores all the text, Lucene also has a good tracking built in it, which enables it to find nearby duplicates. Because of Lucenes capability of finding duplicates all the different entities that Silverfish extracts using KEA also are stored in Lucene (Srinath Srinivasa, 2008).
MySQL is used to store all the entities that are extracted, as mentioned earlier, by KEA. All these entities are classified into different entity types and are stored as entity instances. The entity in-stances are stored in MySQL as nodes which are represented in a considerable graph. The edges in the graph express the Co-occurrence information (Srinath Srinivasa, 2008).
4.3 Metaphors
We are exploring the possibility and the outcome of using a metaphor to visualise a particular part of the user interface of Silverfish, the relations between different entities. To defend our choice of metaphor we will explain the meaning of metaphors, to the extent we feel is necessary.
The New Oxford Dictionary of English defines metaphor as “a thing regarded as representative or symbolic of something else, especially something abstract” (The New Oxford Dictionary of English, 1998). This means that we can use a “thing” to explain “something else”; we will elabo-rate this definition later on.
The reason for us to look into what metaphors can do for us is that we need to find a way to visu-alise the entity relations to the user.
Metaphors are most commonly used to explain something unfamiliar by terms of something fa-miliar (Jennifer Preece, 2002). It is argued by George Lakoff and Mark Johnson that metaphors are not only in use when communicating something to someone but in constant use in a person’s conceptual system. They argue that we use metaphors as references to most things that are thought and acted out. Lakoff & Johnson also argue that since metaphors are used to reference our thinking to past experiences they shape the way that we will perceive and think of things that are new to us. This is surely an efficient way of understanding something unfamiliar, but there is a downside to this efficient system for learning new things and grasping more abstract concepts. What we see as a downside is also something that is brought up by Lakoff and Johnson, the limi-tations of metaphors. Because metaphors we use to grasp an unfamiliar concept only has certain qualities that are comparable to the concept, the metaphor tends to hide the other qualities of the concept and thus making us not take them into aspect when thinking the concept (Georg Lakoff, 1980). Of course this is something that can be taken advantage of, and used when applying a metaphor to a concept, where you intentionally want to leave certain aspects out.
Metaphors in the context of computer interfaces have proven to be successful to users when learning how a system works and how to navigate within the certain system (Jennifer Preece, 2002). However, there is opposition to using metaphors in computer interfaces due to some dis-cussed issues. One of these issues is regarding whether the interface metaphor impairs the user’s ability to understand all the different parts of systems functions (Jennifer Preece, 2002). This is what we mentioned earlier about metaphors only showing certain qualities in the object or con-cept where it is applied, and thus hiding the other qualities.
Another issue is that metaphors are contextually dependant. A metaphor which may be perfectly logical to one person might not be so to another, depending on differences in for instance cul-tures (Alan Cooper, 2003).
Another issue with metaphors is that they can become crippling by becoming outdated and not being scalable to more than a certain extent. Mark Stefik brings this problem to light when treat-ing the metaphor of the “information superhighway”. The problem that he faces is that this par-ticular metaphor, which has been with us since Al Gore invented it in 1979, is that it has become exactly what we mentioned before, outdated and grown out of proportion (Mark Stefik, 2001). As we mentioned before we will further define our metaphor by breaking down the meaning of a metaphor in our context into pieces. This leaves us to define what “thing” we will use when try-ing to explain “somethtry-ing else”.
In this case the “thing” is the visual representation of the relations between entities. An entity is related to a number of other entities, and all of these relations have a specific weight. These weights tell us how strong an entity’s relation is to another entity. This is how the system repre-sents the relations in numbers.
We have been looking at what the relations could mean to us when using them as a way to find further information. We quickly came to the conclusion that the closer a relationship is between two entities, the more relevant the information should be when the user is making his or her way through the entities by going from one to another. This of course means that entities with weaker relations should be less relevant.
What inspired us to use the metaphor which we are using were the discussions of orientational metaphors (Georg Lakoff, 1980). Orientational metaphors are metaphors that are founded on concepts of spatial orientation. Examples brought up by Lakoff & Johnson are such as: happy is up, sad is down, positive is up, negative is down, health and life are up, sickness and death are down. They point out that the orientational part of these concepts each derive from a physical basis. An example of one of these is that life is up and death is down due to that the natural posi-tion when someone is dead is to lie down, and when alive, the representative posiposi-tion is to stand up. This is right if the case is not that the subject is ill or sleeping, where the orientation of the
subject would also be a lying down position. This explains why these two are also represented as down in orientational metaphors.
With orientational metaphors as a starting point we have developed a visualisation of the rela-tions between different entities. We discussed how we talk about relarela-tions as spatially existing concepts. When a person for instance says “I have a close relationship to him”, there is a concept of a spatial factor to the relationship. We have extracted the literal meaning out of this concept and are using it to graphically represent the distances of the relations. We are doing this by posi-tioning entities closer to or farther away from the entity to which they are related to.
5 Methods
Here we will give an explanation regarding the different methods, we have used during our proc-ess. Further down we will describe how and why we used them.
5.1 Brainstorming
This method is used as a way of getting inspiration and creativity started within group projects. There are some guidelines that should be followed when conducting a brainstorming session, no one within the group should ever criticize each other, the objective with a brainstorming session is to create as many ideas as possible and not only one good idea, try to combine different ideas and see if improvement can be achieved and finally express all ideas that comes to your mind no matter how strange they might sound (Robert I. Sutton, 1996).
5.2 Online Questionnaires
This is becoming a popular method to use because of its ability to reach out to a great variety and quantity of people. Online Questionnaires can be conducted in two different ways either by e-mail or web-based questionnaires. The advantage of using e-e-mail is the ability to aim your ques-tionnaire to specific users. The disadvantage with using an e-mail quesques-tionnaire is that they are limited to text. When using web- based questionnaires you have a greater variety of choices such as the ability to add check boxes, help screens and graphics. Often they also give you the ability to get your answers summarized and presented in an excel sheet or a form (Jennifer Preece, 2002).
5.3 Paper questionnaires
This method is known to most people. Questionnaires are a series of questions that are created to extract precise answers from the test persons. Usually the answers require a Yes or No. The ques-tions can also require the users to make a longer response (Jennifer Preece, 2002).
5.4 Pilot tests
Pilot tests are used to test the plans for your evaluation before actually performing the evaluation. It is meant to confirm that your plan is a valid way of going through with your evaluation. It is also an opportunity to confirm that the different tools used in the test are working and for you as a supervisor of the evaluation to practice your interview skills. This enables you to correct any eventual mistakes made (Jennifer Preece, 2002).
5.5 User tests
Often used to see if the product you have created is functioning in a way that makes it useful for its intended users. When performing a user test it is possible to measure the time it takes the us-ers to go through different tasks and also to view if and where they experience difficulties. The target with user testing is in most cases to answer a specific question or to acquire new knowl-edge (Jennifer Preece, 2002).
6 Limitations
In order for us to be able to keep our focus at the main parts of our thesis work we have been forced to limit our research of certain areas. Because of this we only mention certain areas and give a brief overview of how we have thought about the subject.
Regarding our choices of colors we are forced into only doing the readings necessary and have not actually user tested the impact the colors of our choice have on the users. This is mainly be-cause this is not our main focus in our thesis work, and it would require too much valuable time from our studies regarding our usage of metaphors.
As we also mention our stakeholders are academics worldwide, this leads us to our second limi-tation that we have only user tested our second prototype in Sweden and India. We have not been able to launch the interface to the web, due to technical problems with the servers, which could have enabled us to try the interface further in a lot of different countries.
7 Design process
7.1 Prototype 1Our progress with the re-design of Silverfish has been presented on a weekly basis to the Silver-fish team. The SilverSilver-fish team consists of Professor Srinath Srinivasa and Aditya Rachakonda that are the heads of the project, there are also three students that are responsible of working on improving the performance of Silverfish.
Former students that have been working on Silverfish are also welcome to attend the meetings. This has enabled us to ask the team questions and also to get feedback from them regarding our ideas and design proposals. To us it has been valuable to be able to get feedback regarding the possibilities of creating our prototypes, code wise.
In the initial part of our redesign of Silverfish we started by creating an understanding of how Silverfish was working and tried to extract what we thought could be the core values of Silver-fish (Appendix 2). This part of the design process is according to Human-Computer Interaction (HCI) called “Requirements- what is wanted” (Alan Dix, 2004). This turned out to be a very challenging part because of the complexity of the Silverfish engine. When discussing with the Silverfish team they told us that they wanted to show the difference between related key phrases and co-occurring tags. We needed to have an understanding of the way these two functions were working before we could start to design a solution for how to show the difference. Our initial phase of working with the design took us two weeks in order to grasp all the functions and also creating our own understanding of what we found to be the core values of Silverfish. During these two weeks, our initial interest for metaphors and how they could be a way for us to show the difference between the related key phrases and co-occurrence started to evolve. Our problem with trying to grasp the Silverfish engine was mainly due to that the Silverfish team is very
tech-nical and we are not in the same extent. We had to make it clear that we did not possess the same high technical skills that they do, in order to get more thorough explanations regarding the struc-ture of Silverfish.
Our next step was to get an understanding of our stakeholders. We needed an understanding re-garding if they frequently used pages equal to Silverfish and also to see if there was something that the stakeholders thought was missing. In order for us to get a better understanding regarding our stakeholders we created an online survey, (Appendix 3) the survey was also a way for us to get inspiration regarding functions that the stakeholders were missing at the currently available search engines. The reason for us to choose online survey as a way of retrieving the information needed was that we wanted to conduct the survey in Sweden and India. Also as mentioned by Preece, Rogers and Sharp in Interaction Design an online survey often support the administrator to get instant feedback regarding the statistics of the answers. This saved us a lot of time, when not having to put the statistics together by ourselves.
We were aware that there could be problems with getting a sufficient number of answers when sending out an online survey, but because of the number of people we were sending it to we es-timated to at least getting enough answers for us to be able to make a broad estimation. The sur-vey was sent to K3 of Malmö University and IIIT- B. We received eight answers from K3 and forty from IIIT-B. About six of the surveys were not answered in a serious manner but the re-maining forty-two contained valuable information. The main information we extracted from the survey, was information regarding functions that the survey participants think is missing in exist-ing search engines. We also had two questions that were asked in order for us to see what the test participants thought about presenting the title of the different users of Silverfish, and also what they would think of having the possibility to create critique groups and being able to upload drafts of papers that only get sent as drafts to the members of the group. According to the user test fifty four percent thought that the title of the person giving critique mattered. Thirty percent found this to not be important, due to that as long as the critique was valid it should not matter which title the person had. The remaining sixteen percent did not have an opinion regarding the matter.
Our idea with creating an opportunity for the users to create online critique groups was to enable everyone to participate with critique. When presenting critique in a classroom a lot of valuable critique might get lost because of some people not being willing to speak in public. We believe that people do tend to be more willing to speak freely when writing on the Internet. According to our online survey eighty nine percent found online critique groups to be of value. Two percent did not think that it would be a good idea because of the fact that the online critique could easily be corrupted and could easily turn into a forum instead of a critique group. The way we intend to enable the online critique, is that you create your own group of critique members. There is al-ways a risk of getting non valid critique, but it is alal-ways up to the user to evaluate what is valid critique or not.
At this stage we had ideas of what could be done with the Silverfish, meaning placement of ob-jects, choices of color and so on. We also had an understanding of what the users thought was wrong with the existing pages. From the material and ideas we had gathered we started brain-storming about how to best present these ideas in a web interface. The brainbrain-storming session made us come up with the idea that we wanted to create a page that contained two different spaces; from here on they will be referred to as the personal and academic space.
On the left side we placed the personal space, which would handle all communication between users. This page would also contain all the user’s bookmarked papers as well as the user’s up-loaded papers.
When a user first enters a web page they initially view the upper left corner in the shape of a Z, and then continue to the right and down to finish viewing the page straight down on the right corner of the page (Alan Cooper, 2003). We made the choice to place the personal space to the left side due to that we wanted the academic search to be in focus, because first and foremost our task, as stated from the Silverfish team, was to create an easy to use academic search engine. The right side we made into a strict search result page with some of the basic functions that the user could need, this was inspired by the online surveys as well as our own thoughts.
As mentioned earlier we got interested in the possibilities of using metaphors for displaying the related key phrases. We had a brainstorming session about different ways of showing relatedness.
During the brainstorming session we came up with different solutions of how to, according to us, show relatedness (Appendix 4).
For the first prototype we chose to evaluate the possibilities of a tag cloud (Appendix 5), to show the related key phrases. Instead of the tag cloud used in the original Silverfish (Appendix 6), consisting of words, our tag cloud would consist of pictures representing the different entities. The different pictures would appear in different sizes depending on how strongly related they were to the users performed search.
To continue to the next phase we had to put all the different requirements and functions that we wanted to apply to the Silverfish in order. We accomplished this by creating a semi functional prototype.
We combined the two phases called Analysis and Design, according to HCI (Alan Dix, 2004), into one in order to speed up our process. We were also aware of that until now the design of the prototype had been created according to our thoughts of the stakeholder’s needs as well as the online survey. In order for the user to achieve their goal in a better way, they might need guid-ance (Jennifer Preece, 2002). They might find the system that they are using for the moment is fine, but that could be because they do not know any other possibilities. This is what we aimed at creating for our stakeholders, an academic search engine interface that does the work for the user and not the other way around.
7.2 User test 1 prototype 1
The basic guidelines for conducting a user test we got from Interaction Design- beyond human computer interaction (Jennifer Preece, 2002).
Before we performed the actual user test we wanted to evaluate the user test on a test pilot. We got one person to perform the user test, and this enabled us to have an understanding of some parts we had not made so clear for the test persons. We also found some problems with the func-tionality of the user page that we were able to fix before conducting the test. The information found in the pilot test was not used in our continued work (Jennifer Preece, 2002). From the be-ginning we had not thought of having a pilot test to try our actual user test. It was when we were
visiting the Hewlett Packard User Interface lab in Bangalore, and were discussing our progress with them that they mentioned the importance of having a pilot test.
When conducting our first user test, we mainly wanted to find out if the design decisions we had made regarding placement of objects (Appendix 7), usage of space, usage of guidelines and icons made the page easily understandable to the users. Our stakeholders are, as mentioned earlier, de-fined as academics worldwide. For the first user test we wanted people that were used to search-ing through academic search engines like for instance DBWorld, Scholar and so on. The reason for this choice was that many of these academics know what they find wrong with the different pages and therefore want an improved search engine, thus hopefully they are willing to give a lot of feedback in order to get a better search page.
The test was conducted as follows; we had set up our two computers in our office at IIIT-B. The first computer showed a number of different icons; beside it was a notebook with a pencil. The reason for us to use pen and paper as answering sheet was that we thought this would encourage people to write more seriously, than if they just would have been sent an online survey to fill out. Also as mentioned in Interaction Design by Preece, Rogers and Sharp the number of answers might get lower using an online questionnaire. Both of us were in the room to be able to answer any questions that could occur, and to make observations regarding possible problems that could occur. The icon test took an average of five minutes per test person. The icons were a wide vari-ety of twenty eight icons (Appendix 8) that we had chosen for eventually using in Silverfish. Our main priority was to see if the icons we had chosen meant the same thing to other users as they did to us. When the test persons were finished with the icon test, we guided them to conduct the user test of the interface. We gave a short explanation of that the information of the site was static and some minor problems that could occur.
We used a software called CamStudio, that enabled us to record all the activity on the screen. This software also recorded sound so that we could record any comments made or suggestions proposed. The users were not informed about the fact that we were filming their on screen movements. They were also not informed about that the sound was recorded. The reason for us to not mentioning this was that people tend to act differently when being filmed and recorded; this could lead to that they do not actively look around at the interface with the cursor, afraid of
showing their possible mistakes. Also when knowing that the speech is being recorded people have a tendency to not speak freely (Jennifer Preece, 2002). The audio was strictly for our pur-pose as a reminder of the problems mentioned and solutions propur-posed. Because of the user test being anonymous we found it valid not to mention the film and audio recording. After the brief explanation as mentioned earlier the users were presented three basic tasks (Appendix 9). The tasks we wanted the test persons to conduct were aimed at getting an understanding of whether the users easily could find their way around the interface or if there were any problems regarding placement of the different objects and links?
The main issue that we found regarding our design of the interface was one of the icons that we had chosen; the icon was the one on the top left corner that was supposed to display “my docu-ments” (figure 1). Almost all of the users found this icon confusing. Another
problem that some users found disturbing was the fact that they found the search field and search button to be out of focus and not placed in a way that they easily could find them. Many of the users were mentioning Google as a good example of how to display the search field.
Some users also found it confusing regarding saved texts. In our prototype we are calling them bookmarked and the test persons found it hard to grasp that
they save a text but it is situated under the bookmarked subject. This problem was due to a miss spelling from us in the written task. In the task we had written saved texts and in the interface it was called bookmarked. Our mistake lead to that some of the test persons could not find their saved texts.
Regarding the use of space in dividing the personal area from the academic area there were some comments made that the personal space, even if smaller than the academic space, takes up too much space. Some of the test persons also thought that they felt something was missing at the left side of the page when the menu bars were not active. The personal menu consists of a drop down menu; in the mail box menu many of the users found it disturbing that the menu continued so far down that they had to scroll on the sidebar to be able to view the content. This was a prob-lem that we were aware of from the beginning but could not find a good solution for. We got a good suggestion from one of the test persons, that perhaps instead of showing all the messages in
the inbox we could make the choice of only showing a limited amount of mails and having a plus sign, or something similar, that could enable the user to view more messages. Also users wanted to have more of the basic mail functions in the message function. The basic functions they thought were missing were the ability to categorize different messages in different folders, as for instance spam, trash and deleted messages.
The second part of the user test varied a lot regarding how long time the test persons spent on it. The range was somewhere in between three to fifteen minutes. Totally in our first user test we had sixteen people who performed the user test, this is more than sufficient for an initial user test (Alan Dix, 2004).
We found a lot of the test persons being very interested in the work we had done and many of the persons found our basic mockup to be a great improvement of the initial interface. We also got a lot more feedback than we initially thought that we would get, which was very satisfying for our continuing development.
7.3 Re-design of Prototype 1 according to user test 1
During the user test we had obtained valuable information that we needed to analyze. Once again we found ourselves in our combined analysis and design phase. According to the results in the user test we started to consider an alternative design that could simplify the usability of the Sil-verfish interface. When viewing the recorded movements on screen (Appendix 10) we could clearly see that finding “My documents” and trying to conduct a search were the two main issues we needed to consider. We analyzed the results of our icon test in order to find which icon, ac-cording to the users, would be most likely to communicate “My documents” (figure 2). To con-tinue developing the prototype we started evaluating which colors
to use.
In the original Silverfish blue was used in a variety of different nu-ances. The blue nuances were used for displaying objects, which were in the foreground of the page for instance the links, white was
main topics, as well as for information messages given by the system. Blue and white is a good color combination, as you will read further down, when creating a web page, hence there are no problems with the colors being used. The problem is instead the way the colors are used.
Because of the human eyes shortage of blue photoreceptors it is hard for humans to acknowledge thin blue lines and small blue items. When colors only contrast in the amount of blue, the edges easily disappear and different content is emerged into one (Peggy Wright, 1997). Blue is a color that is seen as being calming, gentle, formal and peaceful. Blue, white and green are the three colors that have the most consistent meaning throughout a variety of countries and cultures (Thomas J. Madden, 1999).
When designing Silverfish we are designing an academic search engine that is meant to be used worldwide. This involves a lot of different cultures and religions; therefore we find that using blue and white as our two main colors will help us prevent creating an environment that can af-fect people in a negative way.
As mentioned by Murch in “Physiological Principles for the Effective Use of Color” blue is a color that the human eye best acknowledges in the periphery (Peggy Wright, 1997). This is the reason for us to use blue as a background color. We find the blue background to help enhance the readability of the interface. We find the background to enhance the readability, because it helps the user attention be more focused at the white text fields.
Experiments have showed that the traditional black text on white background still offers, if not the best, at least good readability to the users; this does not mean that there are not alternatives, which could be better (Caldera, 1996). With Silverfish we have chosen to display all body copies on a white background. We have set the color of the body copies to black.
As we mentioned earlier a nuance of blue was used for presenting clickable links in the original Silverfish. When one link was clicked in the original Silverfish it was not differentiated from the links that were unvisited.
When creating clickable links you should try to differentiate unvisited links from visited links by enhancing the focus of the unvisited link. This can be accomplished by giving them a more vivid, bright or saturated color. The visited links should give the user a feeling of being “worn”. The two colors used should be alternatives of the same color in order to inform the user that they are
related. Using two radically different colors will make it hard for the user to associate the two link types and also to understand which the visited link contra the unvisited link is. Blue is the color that is most commonly used for displaying links, and is also the color that presents the user with the most solid feeling of links (Nielsen, 2009). We do believe that different shades of blue is the best way to show links, however we do not find the clear blue links that are mostly used are so good. In our opinion we find the bright blue link color that is often used, to be disturbing for the eyes to focus on for reading.
According to our findings regarding colors,we created a color scheme by using a website called Adobe Kuler (Incorporated, 2008). Adobe Kuler lets you upload a color and from that color it creates a scheme of colors that matches the color you uploaded. We took the original blue color from the Silverfish logotype and created a color scheme from that (Appendix 11).
Our goal from the start has been to create a fully functional site, in order to accomplish this we managed to get a copy of the Silverfish engine. Because of the lack of documentation and ob-vious structure of the code the process to implement our design on to the Silverfish engine became a highly time consuming project. We managed to apply our design to the actual Silver-fish engine and now had a next to fully functional academic search engine.
7.4 User test 2 prototype 1
As mentioned in our previous user test, you should always conduct a pilot test before conducting the actual user test. We got one person to conduct a pilot test for our second user test. We found out that we had made a mistake regarding one of the questions as well as finding a new problem regarding a message informing the user that the message has been sent. We also found a greater mistake that we all ready had found in the first user test of the prototype but forgotten to change. The pilot test informed us that the test would take approximately six minutes for the user to per-form.
For our user test we got six persons to participate. We conducted the user test using our two computers. One was a Macintosh (Mac) computer were the actual tasks were conducted (Appen-dix 12). The other computer was only there for the users to read the task that should be
per-formed. To prevent any eventual problems with users not being used to a Mac touchpad we were using an external mouse. The reason for us to use a Mac, was that in order for us to get Silverfish working in as great extent as possible we had to install Ubuntu (Appendix 13) on our machines. The Ubuntu did not work fully on the PC, therefore we installed it on the Mac were it worked. We had a set of nine questions that guided the user through the parts of Silverfish that we wanted to evaluate (Appendix 14). Our main changes made from the previous user test were placement of the search field, placement of the academic menu and also some functions regarding the per-sonal menu. The intentions of the second user test were to evaluate the performed changes, in order to see if they made the prototype more usable. We found that the test persons did not have any problems with finding the search field in the new prototype. The entire group of test persons went directly to the search field that was more emphasized in our second version. Two users mentioned that they found the readability of the search results to be bad.
They thought it was hard to distinguish the different search results from each other. Four users found one or two of the icons we had chosen to be confusing, they told us that when seeing them for the first time they did not associate them with the meaning that we had given them. Regard-ing the personal menu they found it to fill its function well. On problem they complained about was the fact that when wanting to view the next results in your mailbox the box was closed and the users had to open it again. The test persons wanted to view the results without having to open the inbox again. This part was not in the user test but two subjects found this problem when try-ing all the different functions. The test persons that were trytry-ing the whole page also found a cou-ple of bugs regarding functions that had stopped working and also a page that had not been con-nected to our style sheet, meaning that it had the old Silverfish design.
7.5 Re-design of Prototype 1 according to user test 2
In our second user test of our first prototype we found that the main issue according to user test one, regarding the search field being out of focus, had been solved. Even if we break the rule of not using blue as a color to display an object in the foreground, we still do believe that it is filling its purpose when used at the search field. The blue background color we are using, as mentioning before, in order to enhance the focus of the white text fields. In the same way we believe that the
white background enhances the users focus upon the blue search field. According to the user test we made a change regarding the icon for papers (figure 3). The new icon we
used in order to show papers was a more well know icon for a pdf file (figure 4). Two users found the icon for bookmarks to be a hard to understand. According to our previously mentioned icon test the icon that we are using for the moment was the one that the most test persons found to be equal to bookmarking saving something. We also believe that users perhaps will hesitate the first time viewing the icon, but once they have viewed the tooltip the icon used will be obvious for them. As mentioned earlier one page was found that was not connected to our design so in order to create a fully functional site we implemented our design onto the page. Once we had created the design and structured the last page the first prototype was finished (Appendix 15).
8 Design process Prototype 2
The first prototype we created was aimed at trying to create a page that was well organized and that used visual metaphors as well as text to guide the user into easily finding his or her way. When developing the second prototype we were able to go directly to the analyzing and design phase because we already had the initial data regarding requirements and design, from creating the first prototype. With the second design prototype we aimed at creating a page that used the knowledge we got from our first prototype, but presenting the search results in a completely dif-ferent way. In order for us to get some inspiration, regarding difdif-ferent ways of conducting and presenting searches and search results, we started by searching the internet for visual search en-gines. We viewed several different pages (Appendix 16) to get an idea of how we could present the searches in Silverfish. We were aiming to create a search interface where the results got visu-alized through a metaphor. Our goal was to use a metaphor that would create an understanding for the user that the different entities presented are related. Our initial thought of how to conduct the search and present the results was to create an interface that consisted of an earth globe. The idea emerged when we were talking about different ways that people and artifacts around us can
figure 3
be related. We discussed that the globe we live on is something that we all have in common; therefore the earth globe could be something that we all could relate to in being a way of display-ing relatedness. Our idea was to show that although the different entities occur in different coun-tries on the map they are all related to each other through being a part of the same earth globe. In the same way all the information and users in Silverfish are related through being a part of the Silverfish.
In the initial phase we created a paper drawing of our idea (Appendix 17). The following step we took was to create a Photoshop mockup to be able to present our idea for the Silverfish team (Appendix 18, 19). When presenting the idea for the Silverfish team we got positive response regarding the idea of creating a completely new way for presenting search results.
Though they liked the idea they could not see the functionality with presenting all of the different entities on an earth globe. They could see a usage of displaying different authors on the map to get an understanding of their placement in the world, but they could not see the same strength with presenting the papers on a map. Also there could be issues regarding if one country con-sisted of multiple entities. This could create a situation where there were multiple entities on top of each other. The issues with displaying entities was according to us not the big problem, the problem was instead that we agreed with the team that the users would not gain anything by viewing the searched paper results according to which country they belong to. Presenting the dif-ferent entities on an earth globe created a situation where the search results were not in focus; instead the earth globe itself was the focus.
The meeting made us re consider our second design solution. We considered what our idea would be without the earth globe as a way of showing search results. We looked for inspiration regarding different meanings of metaphors in Metaphors We Live By written by George Lakoff and Mark Johnson.If we were to remove the earth globe visualization we would have the differ-ent differ-entities displayed on the same page in differdiffer-ent distances from each other.
During our first user test of prototype 1 many of the test persons mentioned Google as a very good search engine because it´s simplicity. We started to consider if this could be a solution for prototype number two. The idea that emerged was a combination of orientational metaphors and
the Google search field. We started to create sketches of how it could be displayed (Appendix 20).
In order to create a prototype to present to the Silverfish team we created a complete Photoshop prototype (Appendix 21). The prototype consisted of all the functions that we found necessary in prototype number two. When presenting the idea to the Silverfish team they found it interesting. They mentioned that instead of having to click at one of the different entities to enable the user to see which entities were related to the chosen entity; perhaps they could be presented directly in some way. We were suggested to view several visual search engine pages, for inspiration of a different way of presenting related entities. After viewing different solutions of how to visualize related entities we decided to create a tag cloud solution. We made some changes to our initial Photoshop prototype and presented our new prototype to the Silverfish team (Appendix 22). The feedback we got was that they found the prototype to be good and would like us to continue with evaluating it. The group mentioned that perhaps it could be a problem with displaying many different entities at the same page once again, but our idea was to set a limit to ten search results. We base this upon that usually when people conduct a search at Alta Vista they are presented the ten most related results. They have the choice to continue viewing more results, but only 15% of the people using Alta Vista actually continue to page two, containing results eleven to twenty, of the search results (Craig Silverstein, 1999).
We started looking at different ways of how to be able to create a functional prototype. We did not see Java as a possible solution because of the required amount of code and the limited time. We decided to build our second prototype using the Adobe Flex framework. This framework is based on the Adobe Flash platform (open source software: Adobe Flex, 2009). The reason for us to choose to work with Flex was that we saw it as an easier solution than to work with Java. Also it would let us use the existing queries used to communicate with the MYSQL database via PHP. Because we had never used Flex before the initial development phase moved along slow. After a week we were able to get the different entities to display when you wrote a search word. The problem we were facing was that the entities did not stop appearing; also they were not appear-ing in the way we wanted them to. We approached the Silverfish team with our problem to see if they could help us with a solution. The group told us that they had no background in working
with Flex but helped us to get in contact with a student named Swapnil Mishra who was working on a similar project. He helped us to create an algorithm that gave us an idea of how to place the entities in different distances from the search being conducted according to how strongly related they were to the typed search word.
Another student by the name of Ricardo Lage helped us create the actual algorithm to display the entities in a good way, and within a week we had a first prototype working. At this stage we had the search part of the prototype working. We then created a way for the users to see the abstract when they hovered a paper (Appendix 23). In the second prototype we are using drag and drop to enable the user to interact with the entities. The user can bookmark texts, institutions, authors and add other users to his or her network, by simply dragging the entities over the icons used for displaying the functions.
8.1 User test 1 prototype 2
As with earlier user tests we started with having a test pilot evaluate the user test. During the user test we found a couple of bugs, which enabled us to inform the test persons about them. The rea-son for us not to fix the bugs before the actual user test was that they would require at least one day for us to correct, and we did not have that much time. We also found out that the user test would require five minutes for each test person to perform.
The actual user evaluation was carried out by six people. The user evaluation was conducted on a Macintosh (Mac), which was displaying the second interface. In order to run the prototype we needed Flex and this program was only installed on the Mac. In order to remove any eventual problems with users not being used to the Mac touchpad we used a wireless mouse. We had a second computer, which displayed the tasks of the evaluation to the participants (Appendix 24). In our user test of prototype two we wanted to see if the users understood the way that the enti-ties were displayed, meaning if they conducted a search would they understand that the entity displayed the closest to the search field is the entity that is the most related to the search? As mentioned earlier we are using drag and drop for the interaction with the entities. The user evaluation was also a way for us to see if the users found it usable with this kind of interaction or
if they thought it slowed down the process? The user evaluation was off course also a way for us to see if icons, functions, placement of objects and understanding of the history browser were clear to the users.
We noticed in the user evaluation that the test persons, when doing the user evaluation, all clicked at the entity that was the closest to the conducted search. The users did not seem to find this to be confusing, they did not even seem to reflect upon the way the entities were displayed in different distances from the conducted search. When asking the evaluation persons if they had thought about why the entities were displayed the way they were, four of them mentioned the distance of a way of showing how close they were to the search or different entities.
Regarding drag and drop, the evaluation group found it to be working well, three of them men-tioned the possibility of instead of using drag and drop perhaps the menu could be displayed to the user when hovering an entity. The users also mentioned that when dragging an entity to one of the menu icons and releasing it they were waiting for feedback regarding what was happening with it, meaning if it had been added to your bookmarks if it failed and so on.
The main issues we found during the user test were that the evaluation group found the history browser to be out of focus and also that it was hard to tell in the history browser which was the first entity to be clicked. The feedback within the page was in general not good either, the users said that feedback during a download or when adding a bookmark would prevent them from do-ing the same action twice.
8.2 Evaluation and re-design of Prototype 2 according to user test 1 The main changes we made to prototype two after the user test was that we wanted to enhance the focus and placement of the history browser. As the previous picture of interface two shows the placing of the history browser was on the top left corner. The history was ordered in the way that the first entity the user clicked was placed furthest to the left, and then the following clicked entities were placed after that entity to the right. In our re- design we still have the history browser in the left side but instead of going from left to right it is shown in a falling order from up to down. We have also added the behavior of fading down the older history entities more,
based on how far away the user advances from them. We believe that this enables the user to both get the feeling that he or she is getting further away from the initial search as well as an un-derstanding of the order in which the entities have been clicked.
We worked with supplying the user with more feedback in order to prevent the user from not knowing if for instance he or she had added a text or not. We also created tooltips for the menu icons, in order to inform first time users of the meaning of the icons. In order to create a sense of completeness to the page we implemented the final design. Regarding the design of the second prototype we chose to use similar colors and fonts as we used in prototype one. This was mostly to save some time, in order to be able to concentrate more on the displaying of the metaphor as well as the functions of the history browser and so on. We created a final version of the second prototype in order to show the results of the user test as well as our own thoughts of how to cre-ate a visual academic search engine (Appendix 25). This interface differed greatly from the original interface and how the search results were originally presented (Appendix 26).
8.3 User test 2 prototype 2, Sweden
In order for us to be able to get a greater understanding regarding our usage of the orientational metaphor we wanted to conduct a user test in Sweden as well. This would help us to obtain fur-ther understandings in the differences between the Swedish and Indian context, which would give us a broader perspective regarding our stakeholders worldwide.
The user test was conducted according to our previous user test. We did not conduct a pilot test because we had already done this before the first user test in India.
The user test was conducted as follows; we had one PC set up with the tasks that were to be con-ducted. We had a Mac computer for the users to conduct the actual tasks on. The user test was conducted with six persons with varied computer knowledge.
During the user test we found that almost all the test persons did not acknowledge that the search results were presented in different distances from each other. The user started reading the text beneath the icon based search results in order to find the one that was most related to their search results. It was first in the discussion that the test persons mentioned that they saw that there was a
difference in distance between the objects and that this could display the importance of the re-sults presented. They mentioned that when first seeing the search rere-sults being presented visually spread out on the interface they thought this was merely for presenting them in a nice way. The test persons also thought that it would require something more in order for them to get an easy understanding of how related the presented results were to their conducted search. An interesting fact that we found out of the test was that we were informed by users that they ordered their search results from the top to the bottom.
Another issue that occurred during the user test was that all the users had a hard time finding the history bar at the left side of the page. All the users mentioned that it perhaps could be better to place the history bar above the search results.
8.4 Paper survey prototype 2
As mentioned earlier the user evaluation of the second prototype, did not give us so much infor-mation regarding if the users understood the meaning of why the objects were displayed as they were.
In order for us to get a greater understanding regarding this matter we created a paper survey (Appendix 27). When printing the paper survey we acknowledged that the printout was not clear, but because this was our only option for printing at the moment and that we were running out of time we decided that we would hand the surveys out. The survey was handed out to twenty per-sons. The reason why we only handed it out to twenty persons was that at IIIT-B the final exams had just been and it was hard to get a hold of people.
We received only eight answers due to that many of the evaluation persons said that they were not able to differentiate the icons from each other. We did not want to explain the icons for the evaluation persons because this would create a situation where the evaluation persons were not objective any more.
Regarding the eight answers we did receive, six of the answers informed us that they did under-stand that there was a relationship between the searched word and the entities surrounding it, but none of the evaluation participants mentioned that the way they were presented could be a way of displaying how closely they were related.
