An interactive interface for multiple-resolution analysis of large images

IT 14 034

Examensarbete 15 hp Juni 2014

An interactive interface for multiple-resolution analysis of large images

Nguyen-Anh-Thu Tran

Institutionen för informationsteknologi

Teknisk- naturvetenskaplig fakultet UTH-enheten

Besöksadress:

Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0

Postadress:

Box 536 751 21 Uppsala

Telefon:

018 – 471 30 03

Telefax:

018 – 471 30 00

Hemsida:

http://www.teknat.uu.se/student

Abstract

An interactive interface for multiple-resolution analysis of large images

Nguyen-Anh-Thu Tran

Digital image analysis has contributed greatly to medical sciences. Modern microscope slide scanning systems are capable of producing large images which can be more than one giga-pixel. It is useful for researchers to be able to view these images at multiple resolutions. For instance, to implement image-based sequencing of messenger ribonucleic acid (mRNA), high resolution images are required in detailed analysis while those at low resolution offer better overall visualization. Taking that as the motivation, a map-viewer-like user interface with zooming and panning options has been developed to support detailed analysis in high resolution and at the same time be able to get a full overview in lower resolution. This thesis describes the context in which the interface is used as well as its design process.

Tryckt av: Reprocentralen ITC IT 14 034

Examinator: Olle Gällmo

Ämnesgranskare: Carolina Wählby

Handledare: Petter Ranefall

Contents

Contents 3

List of Figures 4

1 Introduction 6

2 Background 7

3 Methods 8

3.1 Prototyping . . . . 8

3.1.1 What is prototyping? . . . . 8

3.1.2 From low to high fidelity prototypes . . . . 9

3.2 Collecting feedback . . . . 10

3.2.1 Consult experts - Usability inspection . . . . 10

3.2.2 Interview users - Usability testing . . . . 10

4 Results 12 4.1 Version 1 . . . . 12

4.1.1 Lo-fi prototype . . . . 12

4.1.2 Resulted web interface . . . . 16

4.2 Version 2 . . . . 19

5 Discussion and conclusions 37

A Modules in the system 38

B Workflow 39

Bibliography 40

List of Figures

3.1 Prototyping iteration . . . . 10

4.1 Version 1 - Design of ”Home” page . . . . 12

4.2 Version 1 - Design of ”Pre-process” page . . . . 13

4.3 Version 1 - Design of ”Manual registration” page . . . . 14

4.4 Version 1 - Design of ”Show image” page . . . . 15

4.5 Version 1 - ”Home” page . . . . 16

4.6 Version 1 - ”Pre-process” page . . . . 17

4.7 Version 1 - ”Manual registration” page . . . . 18

4.8 Version 1 - ”Show image” page . . . . 19

4.9 Version 2 - Design of ”Home” page . . . . 20

4.10 Version 2 - Design of ”Open project” page . . . . 21

4.11 Version 2 - Design of ”Project information” page . . . . 22

4.12 Version 2 - Design of ”Pre-process” page . . . . 24

4.13 Version 2 - Design of ”Manual registration” page . . . . 27

4.14 Version 2 - Design of ”Analysis test-drive” page . . . . 31

4.15 Version 2 - Design of ”Full analysis” page before consultation . . . . 33

4.16 Version 2 - Design of ”Show image” page . . . . 34

Acknowledgements

I would like to thank Carolina W¨ ahlby and Petter Ranefall for letting me join the team and write my thesis in the context of this challenging yet interesting project. Thank you, Carolina, for your positive attitude and wonderful ideas. Thank you, Petter, for guiding me through the project, for your willingness to listen and clarify my doubts.

I would also like to express my gratitude to Mats Lind, who introduced my to the project.

Thank you for your patience and great input to help me finish this thesis work.

1. Introduction

Gene expression is the process in which genetic constitution of a cell, an organism or an indi- vidual leads to the formation of functional gene products, most common of which is protein. It is important to examine activities in this process as any disruption or change that occurs may result in diseases. Gene expression profiling is a technique being used in molecular biology to collect the activity of thousands of genes simultaneously. It is useful for the studies of cells, the process of detection and treatment of diseases. Since messenger ribonucleic acid (mRNA) car- ries generic information needed for the formation of protein, one way of doing gene expression profiling is to identity the primary structure of mRNA. Such method is also known as mRNA sequencing. The project in which this thesis took place was to develop a web-based tool to apply the newly proposed image-based mRNA sequencing method [11]. My main responsibility covered designing and implementing a graphical user interface for the web application.

The rest of this report will present in more detail what has been done in the thesis work.

Chapter 2 introduces briefly the context in which the thesis work was carried out. Methods being

used are explained in chapter 3. Chapter 4 demonstrates user interface of the web application as

the final product. Finally, strengths, weaknesses and possible future extensions to the interface

are discussed in chapter 5.

2. Background

The project coordinating this thesis work aimed at implementing image-based approach [11] to in situ sequencing of mRNA [7] which would benefit gene expression profiling. The approach in question offers a possibility to examine cells directly in their natural context, thus, to relate primary structure of mRNAs to spatial information of the surrounding environment. There are, however, two primary challenges of applying this image-based sequencing technique.

Firstly, slide scans need to be handled at different levels of resolution. A highly detailed version of the image gives a more accurate analysis while its low resolution counterpart offers better visualization over the whole image. Also, it is not straightforward to analyse such large images due to limitations in hardware (e.g. memory constraint) and software (e.g. image analysis tool). This first challenge was addressed in [12]. The idea was to convert the large image into a number of smaller tiles. Small tiles would act as input to the image analysis software, for instance, CellProfiler ¹ [3]. For viewing the image and visualising analysing results, a javascript tool called OpenSeadragon ² was used.

The second problem that needs to be solved when building the web application in question relates to usability issues. So as for the image analysis software to produce accurate results, the list of slide scans should be arranged carefully. The order of those image files is predefined and fixed. In other words, users do not have the flexibility to input files randomly, they must adapt to the system’s rule instead. Furthermore, structure of the application being developed is fairly complex. Appendix A illustrates relations among modules within the system. When being showed to the users, the application, however, must be simple and easy to use. It should appear clear and comprehensible to the users. Ideally, the workflow in Appendix B should be carried out by the users without them noticing which underlying modules are being run. The objective of this thesis work is to meet the usability challenge. In order to do that, a graphical user interface for the system needs to be carefully designed and implemented.

1

CellProfiler - a free and open-source software designed for cell images analysis. http://www.cellprofiler.org/

2

OpenSeadragon - An open-source, web-based viewer for zoomable images, implemented in pure JavaScript.

http://openseadragon.github.io/

3. Methods

The main method being used in this thesis work is prototyping together with feedback collection.

Firstly, paper prototypes of the system’s interface are made. I then consult experts about the design before going on with building an interactive web interface. The resulting product is evaluated in collaboration with the project’s stakeholders, including the end-users of the system.

The process of prototyping-evaluating-improving is carried out iteratively.

3.1. Prototyping

3.1.1. What is prototyping?

A prototype is an early version of the whole or a part of the system under development. It is helpful in demonstrating concepts and design ideas, especially those relating to the graphical user interface. It can also be used as a means of communication among the project’s stakehold- ers, i.e. those who are affected by the outcomes of the project [5]. In other words, prototypes make it easier for project manager, developers, users and other involved entities to discuss the design of the system with each other. In the context of this thesis work, prototypes are used as an input to meetings within the Quantitative Microscopy Group at Uppsala University and between the team and its collaborators at the Karolinska Institute.

There are several reasons why prototyping, i.e. the method of making use of prototypes in different levels of detail to model and develop the system iteratively, is suitable for the project in question. Firstly, it allows end-users to experiment with the system early [13]. Users will be able to see how helpful the expected final product can be for them. They also have chances to give instant feedback to the development team based on what the team presents to them.

The feedback may concern what is good and/or not good about the currently modelled part of the system, errors originated from misinterpretation of requirements and possible improvements for later prototypes. By taking these feedback in consideration and iteratively developing the system, the final product will be more likely to have high usability.

Secondly, prototyping is a way to cope with changes that may arise and affect the original development plan of the system. Changes in requirements may occur due to the nature of agility in environments where the system will be used [2]. In this thesis work, however, I find adjust- ment to the initial design are more likely to take place owing to the complexity of the system.

The web application in question is cross-disciplinary, thus, requires the development team to integrate knowledge of different fields such as biotechnology and computer science, including image analysis, human-computer interaction, software engineering and programming. It was quite difficult to understand the system requirements correctly at first. Hence, prototyping is a reasonable method to embrace potential changes and ensure the right problems are being tackled in the development process. As a result, the method helps shorten the time needed to delivery of a usable product.

Last but not least, prototyping plays an important role in designing a user interface,

which is the main focus of this thesis work. According to Sommerville [13], the only practical

approach to build a graphical interface of a system is doing rapid prototyping and having its users

participate in this process. It is usually through the graphical interface that users interact with

the core of the system. Therefore, making the interface intuitive is of paramount importance. A

fast and direct way to test whether the design has achieved its goal is to prototype the system, then let the users experience and evaluate it. Feedbacks from users will be a valuable source for improvements in the graphical interface as well as the underlying modules of the system.

3.1.2. From low to high fidelity prototypes

There are two main kinds of prototyping, distinguished by the level of detail in their prototypes.

Low fidelity (lo-fi) prototypes usually come in the form of manual drawings. These sketches can be either paper-based or created using simple drawing tools. They do not allow live interaction and are designed-to-be-thrown-away [1]. It is because lo-fi prototypes are meant to be simple.

They do not aim to model the complete system but its permanent content, layout and basic functionality. This is a time-efficient and cost-effective way to explore the problem space as well as getting input to later activities in usability design [10].

High fidelity (hi-fi) prototypes are more detailed models of the system. They may look identical and function almost in the same manner as the final product. Hi-fi prototypes are produced using specialised software or by programming directly in the system’s real development environment. I would prefer the latter since it helps pointing out technical limitations which cannot be identified in either lo-fi prototypes or hi-fi prototypes in designated software. In contrast to its primitive counterpart, hi-fi prototypes are interactive. For instance, users can click a button to toggle visibility of an item, send a request and receive an actual response. Hi-fi prototypes are quite powerful in getting detailed feedback from users, especially those on visual effects, functional features and learnability ¹ .

As discussed above, both lo-fi and hi-fi prototypes are useful in the system modelling process. The former helps getting started with the project, exploring problems and giving ways for ideas on solutions. The latter brings the project closer to its final product, addresses problems in a greater depth and possibly takes the development team to concrete solutions.

Therefore, both kinds of prototyping were carried out in this thesis work. I first drew main

”screenshots” of the website using Pencil ² . Besides Pencil, I did make use of real paper and pen to create sketches since it is simple and fast. The lo-fi prototype were refined after consultations with experts. I then coded hi-fi prototypes in the real environment of the system, using PHP ³ , HTML ⁴ and Javascript ⁵ . Later on, the process from lo-fi to hi-fi prototypes (Figure 3.1) was repeated for each change being introduced to the system.

1

Learnability - a criterion concerning how much a graphical interface can support novices in quickly learning to use the system

2

An open-source GUI prototyping tool. http://pencil.evolus.vn/

3

PHP - a popular general-purpose scripting language. http://www.php.net/

4

HTML or HyperText Markup Language - standard language for create web pages

5

Javascript - a popular dynamic programming language used in web environment

Figure 3.1.: Prototyping iteration. First, a lo-fi protoype is created and refined after a series of consultations with experts. Hi-fi implementation of the interface is made based on the resulted lo-fi prototype. The hi-fi prototype is evaluated by users during a usability testing session. Taking users’ feedbacks into consideration, another iteration will be carried out. For the sake of time-efficiency, it should start with a lo-fi prototype rather than a hi-fi one. Finally, when users are satisfied with the interface, the latest hi-fi prototype will become the official product.

3.2. Collecting feedback

Prototyping immensely aids in designing graphical user interface. However, it is simply a tool to collect feedback. Prototypes will remain ”fake” and useless if they are not open to comments and improvement suggestions. In this thesis work, feedback were collected from two sources, experts and end-users.

3.2.1. Consult experts - Usability inspection

One way to evaluate an user interface is to consult experts, which can be referred to as usability inspection. Grounds for this method are verified design principles and experts’ own experience, thus, it can be deployed as soon as the design process is started [9]. If getting end-users’

feedback on the interface is difficult due to time and budget limitations of the project, expert evaluation is certainly an appropriate alternative [4]. It is simple, yet rather efficient [1]. In the context of the mentioned project, end-users are already specified and are available for the test. However, this 10-week thesis work took place in Uppsala while the group of users resided in Stockholm. It was the matter of time and distance which made usability inspection more favourable. Prototypes of the web application were evaluated by three experts. Their specialised fields covered human-computer interaction, biotechnology, bioinformatics and image analysis.

Consultations were conducted before and after each refinement of a lo-fi prototype. The final lo-fi prototype was made into a hi-fi one in order to demonstrate in front of other project’s stakeholders, including the end-users, and to move on with integrating other modules of the system.

3.2.2. Interview users - Usability testing

The most common method to collect feedback on the graphical interface of a system is to let

the real users experience it. It reflects precisely how users interact with the system and which

problems they face while they are trying out the interface [8]. In this thesis work, an interview

with five end-users was organised after the formation of the first hi-fi prototype. The model, with

graphical user interface and underlying modules, was demonstrated in form of a walkthrough.

After that, the users were asked for feedback. There was no specific questionnaire, so users

were free to give comments on everything related to the interface. Based on these feedback, the

prototype was improved in order for the design to proceed to its final version.

4. Results

The thesis work resulted in two versions of graphical user interface for the web application. They were corresponding to two prototyping iterations with a usability testing session in between.

As mentioned in chapter 3, in each iteration, a simple lo-fi prototype of the interface was first created. A sequence of consultations with experts were arranged and changes were introduced to the lo-fi prototype. The outcome of each iteration was an hi-fi prototype, which was actually used as part of the real web interface, of the system.

4.1. Version 1

4.1.1. Lo-fi prototype

Figure 4.1.: Version 1 - Design of ”Home” page

Figure 4.1 is the first sketch of the web application’s ”Home” page. Given that users have

already logged in, they can choose whether to view an already analysed image (”Show an

existing image”), perform pre-process (”Make tiles and create .csv file”) or run analysis on an

image (”Run CellProfiler”). These three choices corresponds to three basic steps users need to

go through in the system. At the top right corner, there is an icon leading users to settings

regarding their accounts. Options to account settings like privacy policy, password, as well as

a possibility to log out of the system will be showed when the icon is mouse-hovered.

Figure 4.2.: Version 1 - Design of ”Pre-process” page

Figure 4.2 sketches the screen for ”Make tiles and create .csv file”, which is later on

called ”Pre-process” in version 2. This is an important step that must be done properly before

running the analysis process. On the left side of the screen is a folder tree which illustrates the

file structure of the server’s database. For the time being, the web application uses only image

files which are already stored on the server. Users can drag and drop slide scans from the folder

tree to their corresponding place-holders in the submission list. Also, users have options to

perform auto alignment of images, create a comma-separated values (CSV) file referring to slide

scans’ location and make tiles of a specific size for each big image. Initially all three options are

set to on and the tile size is 2048 × 2048 pixel.

(a) Before consultation

(b) After consultation

Figure 4.3.: Version 1 - Design of ”Manual registration” page

Figure 4.3 shows layouts of ”Manual registration” page before (Figure 4.3a) and after

(Figure 4.3b) consultations with experts. ”Manual registration” is an optional step which

can be performed if auto alignment is incorrect. Initially, I dedicated fixed areas for lists

(”Overlays”,”Control points”, ”Regions”) and properties of list items. Experts pointed out

that this took up too much space. Images were given only the lower half of the screen while

they should be the main focus. One way to save space is presented in Figure 4.3b. Lists were made into accordion mode and showed side-by-side with properties. Hence, three-fourth of the screen could be used for displaying images.

(a) Before consultation

(b) After consultation

Figure 4.4.: Version 1 - Design of ”Show image” page

Figure 4.4 is the screen sketch, before (Figure 4.4a) and after (Figure 4.4b) consultations,

for the last state of the web application, where results from the analysis process will be showed.

In the beginning, the image and panels (”Overlays”, ”Regions”, ”Annotations”, ”Statistic”) were respectively allocated to the left and right side of the screen. It seemed fine since the image and panels were treated as two independent blocks. However, the content of ”Overlays”

and ”Annotations” (which is later on called ”Markers”) were outputs from the analysis process while ”Regions” showed a list of contours drawn by the user and ”Statistics” gave information about gene expressions found within a specific region. This difference led to a suggestion that

”Overlays” and ”Annotations” should be separated from ”Regions” and ”Statistics”, which was illustrated in Figure 4.3b. Also presented in the sketch, a toolbox to support drawing of contours was placed next to the ”Region” panel.

4.1.2. Resulted web interface

Figure 4.5.: Version 1 - ”Home” page

Figure 4.5 presents the hi-fi implementation of the system’s ”Home” page. The layout of the

web-page is identical to that being showed in Figure 4.1. A few decorations, i.e. and ,

were added to each option in the main panel. The logo of Uppsala University was also put in

its place.

Figure 4.6.: Version 1 - ”Pre-process” page

There are quite many differences between the HTML version of ”Pre-process” page (Fig- ure 4.6) and its lo-fi sketch (Figure 4.2). Firstly, a menu and a navigation bar was added to the top of the web-page. This change was also implemented ”Manual registration” and ”Show image” pages. There are two fields in the menu bar, ”Main steps”, which offers short-cuts to all pages (steps), and ”Help”, which, as its label suggests, gives users access to the system’s guide and documentation. Below the menu bar is the navigation bar to tell users where they are in the system at the moment. The navigation bar was my effort to support ”visibility of system status” [8]. However, it led to confusion which will be further discussed and addressed in version 2 of the interface. Regarding the directory tree to the left side of the screen, an option to hide folders whose names ended with ” files” was added. These folders do not contain data owned by or uploaded by users but created by the system. Slide scanner images which users need to drag and drop into the main list are very likely to be outside of these folders.

Thus, hiding them helps filter the directory tree for files which users are possibly looking for.

Thirdly, in the main file list, an option to set the reference base was added to fulfil a functional

requirement while checkboxes for creating CSV file and making tiles from large images were

removed. Creating CSV file and making tiles are considered to be the primary tasks needed

to be done in pre-process, so they should always be automatically executed once users submit

the list of images. Furthermore, for the later image analysis process to run successfully, the file

list must include images corresponding to A, C, T, G, DO and nuclei of each base. To ensure

satisfying this constraint, the ”Submit” button was initially disabled and would be enabled only

when the list was filled completely. Lastly, this HTML prototype sees an insertion of a files

checklist to the right side of the screen. The checklist was intended to help users keep track of

which list item was filled or missing.

Figure 4.7.: Version 1 - ”Manual registration” page

The hi-fi prototype of ”Manual registration” page (Figure 4.7) is quite similar to its lo-fi

counterpart (Figure 4.3b). A small change was that icons for visibility and removal were bound

to each item in ”Overlays”, ”Control points” and ”Regions” instead of standing alone on top

of the group of these lists. The closeness to an item helps users learn what a particular icon

would do.

Figure 4.8.: Version 1 - ”Show image” page

Web interface of ”Show image” page (Figure 4.8) and its sketch (Figure 4.4b) have the same layout. However, in the hi-fi prototype, most of the screen was dedicated for displaying the image while in the paper prototype, nearly half of its width was occupied by panels (”Overlays”,

”Annotation”, ”Regions” and ”statistics”). To achieve this, panels were made into sliding mode.

Instead of being anchored to a specific position with a fixed width, a panel on the web-page would be showed (i.e. sliding into viewing area) only when users clicked on its title.

4.2. Version 2

Another version of the user interface was needed for some reasons. Firstly, the first version did not integrate CellProfiler to the web application. In other words, it missed a page for calling and running the image analysis process. Secondly, there were changes in the system requirements.

On the 8 ^th of May, 2014, a meeting among the development team and other stakeholders,

including the system’s end-users, was organised. The first hi-fi prototype version of the web

interface was presented in order to call for feedback. This resulted in a new requirement from

users, an option for adding extra files to the list in the pre-process step. Additionally, from

the side of the development team, we thought a project-like structure could be used in this

web-based tool. More specifically, when users get started with the system, they need to create

a new project to store all necessary information such as a CellProfiler pipeline to direct the

image analysis process, a CSV file telling the system which symbol should be used to represent

which gene expression, file paths to slide scanner images and the status of users in the system,

i.e. which steps they have and have not performed. Last but not least, navigation in the first

version was too confusing. The navigation bar appeared like a breadcrumb telling users which

page they were at and how they got there from the ”Home” page. However, a breadcrumb is not

appropriate for this web-based tool. A progress tracker should be used instead. A breadcrumb

is useful for the website whose content is rich and organised [6]. It briefly reveals the website’s

hierarchical structure and enables users to jump among pages. The website for the application

in question is not great in term of the amount of information, yet it has a specific flow which users need to follow (see Appendix B). Except the ”Home” page, one cannot go any further without having done the pre-process. ”Show image” page cannot be reached if the step of image analysis has not been carried out. In other words, users cannot switch freely among the web-pages. Taking these facts into account, the menu and navigation bar presented in version 1 should be removed and replaced with another solution for guiding users through the system.

Considering these issues, I started another ”From low to high fidelity prototypes” iteration.

It resulted in a different and probably nicer graphical user interface for the web application.

Unfortunately, the hi-fi prototype has not been created yet due to time limitations. The finalised lo-fi prototype in this iteration is presented below.

Figure 4.9.: Version 2 - Design of ”Home” page

Figure 4.9 shows changes to the ”Home” page in order to adapt to the project-like structure

which will be deployed in future work. Given that users have already logged in, they can choose

to either create a new project or open an existing project to continue working on it. Besides,

I also used the real name of an end-user in this prototype. This is a small but necessary

adjustment.

(a) Before consultation

(b) After consultation

Figure 4.10.: Version 2 - Design of ”Open project” page

For opening a project, my first thought was using a dialog to prompt users to choose a

name from the list of their projects (Figure 4.10a). Later, this step was given an entire web-page

(Figure 4.10b). This not only supports consistency of the interface but also gives more room to

display the project list together with the date on which each project was created.

(a) Before consultation

(b) After consultation

Figure 4.11.: Version 2 - Design of ”Project information” page

Figure 4.11 sketches web-page where users will provide information for a new project. All

three fields, ”Project name”, ”CellProfiler file” and ”Taglist file” are compulsory. Moreover, as

mentioned at the beginning of this section, the menu and navigation bar in version 1 should be

removed to avoid confusion. Version 2, therefore, introduces a solution which uses a progress

tracker to guide users through the system (Figure 4.11b). The project tracker is placed at the

top of the screen. It shows four basic steps that users need to go through - providing project

information, doing pre-process, running image analysis and viewing the result. Besides the

tracker, navigational buttons, i.e. going backward and proceed forward, were also added to the

bottom right corner of the screen.

(a) Before consultation

(b) After consultation - substep 1 - initial

Figure 4.12.: Version 2 - Design of ”Pre-process” page

(c) After consultation - substep 1 - file dropped

(d) After consultation - substep 2

Figure 4.12: Version 2 - Design of ”Pre-process” page (continued)

Figure 4.12 presents the lo-fi prototype for ”Pre-process” which is probably the most

important step in the workflow. This step was broken down into two sub-steps. The first sub-

step (Figure 4.12b and Figure4.12c) would give a place for users to drag and drop files from the

directory tree into their correct positions in the main list. The second sub-step (Figure 4.12d)

was meant to confirm the file list and ask them what they would like to do next, letting the

system run auto image analysis for several hours then viewing the result (Figure 4.16) or doing

manual registration (Figure 4.13) or getting quick testing analysis output corresponding to some

certain regions on an image (Figure 4.14). Regarding the web-page for users to build the image

list, the file checklist was removed to make the interface simple and to save more space for full

filepaths. As a replacement for the checklist, small lines under tab names were used to keep

track the list’s status (Figure 4.12c). A base with no, some and all required image files would

be represented by no, one and two lines respectively under its tab name. Furthermore, in the

meeting organised in May, users mentioned that they would like to have an option to add extra

files besides those six files for A, C, T, G, DO and nuclei. Hence, plus and minus icons were

added right under the field for nuclei file in order to satisfy this new requirement.

(a) Before consultation

(b) After consultation - substep 1 - initial

Figure 4.13.: Version 2 - Design of ”Manual registration” page

(c) After consultation - substep 1 - new point

(d) After consultation - substep 1 - set point

Figure 4.13: Version 2 - Design of ”Manual registration” page (continued)

(e) After consultation - substep 1 - correct point

(f) After consultation - substep 2

Figure 4.13: Version 2 - Design of ”Manual registration” page (continued)

In version 2, the page for ”Manual registration” (Figure 4.13) sees removals of ”Overlays”

and ”Regions” panels. This change is reasonable since overlays are only available in the final test

(after the full image analysis process completes) and drawing regions do not help in aligning two

images. Figure 4.13b-f also proposes a change in OpenSeadragon interface. Experts suggested

moving the small top-right navigator to the left which was a more standardised position for

elements in this category. Figure 4.12c-e details the procedure of adding a pair of control points

to the images. When the adding icon next to the label of ”Control points” panel is clicked, this

procedure starts and the cursor is changed into a pin (Figure 4.13c). Users must first click on

the primary image to set the reference point (Figure 4.13d). A temporary pin will be added

automatically on the secondary image. Users then need to specify the correct corresponding

point on the secondary image to make a pair of control points (one on the primary and another

one on the secondary image) (Figure 4.13e). It is necessary to have at least three pairs of

control points to run the transformation process. Once the transformation process finishes,

the two aligned images will be showed and users are asked for the choice of the next step

(Figure 4.13f).

(a) Before consultation

(b) After consultation - substep 1

Figure 4.14.: Version 2 - Design of ”Analysis test-drive” page

(c) After consultation - substep 2

Figure 4.14: Version 2 - Design of ”Analysis test-drive” page (continued)

”Analysis test-drive” (Figure 4.14) is a new step in version 2. Like ”Manual registration”,

it is optional and can be skipped to the full image analysis. The full image analysis process

may take several hours to finish, so the test-drive offers a faster way to detect problems with

slide scanner images, e.g. improper alignment. The idea behind this is to run image analysis

not on the entire tile set (which is equivalent to analysing the whole big image) but on some of

them only. The tiles to be tested will be determined by the regions drawn by users. Test result

(Figure 4.14c) will be showed in a similar way as the full analysis result (Figure 4.16).

Figure 4.15.: Version 2 - Design of ”Full analysis” page before consultation

Initially, I though it would be useful to show again CellProfiler and Taglist files and ask

users whether they would like to change them before commanding the system to run the full

image analysis process (Figure 4.15). However, since it is very likely that users are sure about

these files and they can always go back to ”Project information” page in case of modification,

this ”Full analysis” page may be eliminated from the website’s structure.

(a) Before consultation

(b) After consultation - initial

Figure 4.16.: Version 2 - Design of ”Show image” page

(c) After consultation - new region

(d) After consultation - set region

Figure 4.13: Version 2 - Design of ”Show image” page (continued)

After the time-consuming image analysis process, users will end up at a page showing

the final result (Figure 4.16). Besides the chosen slide scanner image, main elements displayed

in this page are panels for the list of overlays, markers representing gene expressions, regions

drawn by users and the corresponding statistics. Sliding mode is a nice way to present those

panels. Nevertheless, they were not correctly grouped and arranged in version 1. In version

2, ”Overlays” and ”Markers” were put in one group called ”Analysis output” and positioned on the left side of the screen. ”Regions” panel stayed at its old place while ”Statistics” was moved to the bottom of the screen. I still took advantage of the sliding mode to dedicate more space for the main image. Figure 4.13c-d details the procedure of drawing a region of interest on the image. The procedure starts when the adding icon next to ”Regions” label is clicked.

At this point, a brief tip saying ”Draw a polygon to mark a region” will be showed together

with a finishing button for the drawing procedure. Also, the cursor will turn in to a large white

cross to tell users that they can now use the mouse to draw a region. Final result will be then

displayed as in Figure 4.13d.

5. Discussion and conclusions

In situ RNA sequencing [7] benefits significantly gene expression profiling process. By taking advantage of high-quality images produced by modern slide scanning systems, image-based approach [11] to in situ RNA sequencing facilitates the profiling process even more. One of the problems occurred in the project of making this approach widely and easily accessible via a web application was usability. This thesis work has presented a way to address the mentioned usability issue by building a graphical user interface for the web-based tool in question. The development process of the interface was carried out using prototyping method together with collecting feedback from fields’ experts and real end-users.

What the thesis work gained was a graphical user interface for the web-based tool whose goal was to apply image-based approach to in situ RNA sequencing [11]. In order to design and implement the interface, the thesis work also suggested a structure for the website where the tool would be reside. Given that structure, the work-flow to perform RNA analysis on images will be more comprehensible to end-users.

There are, however, issues that need to be improved in this thesis work. First and foremost, usability testing should have been done more properly. The application in question is a complex system that requires the development team to integrate knowledge of different domains. It was quite difficult to get the system requirements right at first. Hence, to verify that the graphical interface gives ways for underlying modules to operate as well as to validate that it meets the users’ needs, usability testing must be conducted carefully. However, due to time limitation and travelling distance between development team’s office and users’ workplace, only one usability testing session was organised within ten weeks of the thesis work. In addition, more efforts should be put in documentation. Even though the system may be easy to follow, help and documentation are always necessarily be included so that users can refer to them whenever in need.

Having discussed the thesis’s weaknesses, I would like to propose possible future work for the development of the web application’s user interface. First of all, the second version of hi-fi prototype of the interface should be implemented before the next meeting with users.

Also, a design for the log-in page is needed in order to complete the interface for the web application. Furthermore, usability testing on real end-users should be conducted more seriously.

Because meetings with users cannot be held regularly, a good preparation before each meeting will help running the test session in the most efficient way. Preparation includes setting up necessary hardware and software beforehand so as for users to try out the system once the test session is started. Furthermore, during the design process, some issues which need users’ input may arise. These can also be written down as part of the preparation for usability testing.

Apart from getting users’ evaluation, future work on the project may consider paying more

attention to documentation. Documentation not only helps users in using the system but also

greatly assists development team in future maintenance. It often takes time to write good help

and documentation for a system. Hence, the early this task is started, the more detailed the

documents can be. A starting point for help and documentation can be a description of the

system, what it does, and basic steps in using the system.

A. Mo dules in the system

B. W orkflo w

Bibliography

[1] David Benyon, Phil Turner, and Susan Turner. Designing Interactive Systems. Pearson Education, 2 edition, 2010.

[2] Andrew Branson, Richard McClatchey, Jean-Marie Le Goff, and Jetendr Shamdasani.

Cristal : A practical study in designing systems to cope with change. CoRR, abs/1402.5753, 2014.

[3] Anne E Carpenter, Thouis R Jones, Michael R Lamprecht, Colin Clarke, In H Kang, Ola Friman, David A Guertin, Joo H Chang, Robert A Lindquist, Jason Moffat, et al. Cell- profiler: image analysis software for identifying and quantifying cell phenotypes. Genome biology, 7(10):R100, 2006.

[4] Heather W. Desurvive. Faster Cheaper!! Are Usability Inspection Methods as Effective as Empirical Testing? In Jakob Nielsen and Robert L. Mack, editors, Usability Inspection Methods, chapter 7, pages 173–202. John Wiley and Sons, Inc., New York, NY, 1994.

[5] R Edward Freeman. Strategic management: a stakeholder approach. Pitman series in business and public policy. Pitman, Boston, 1984.

[6] Jacob Gube. Breadcrumbs in web design: Examples and best practices, March 2009.

[7] Rongqin Ke, Marco Mignardi, Alexandra Pacureanu, Jessica Svedlund, Johan Botling, Carolina W¨ ahlby, and Mats Nilsson. In situ sequencing for RNA analysis in preserved tissue and cells. Nature methods, 10(9):857–860, 2013.

[8] Jakob Nielsen. Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993.

[9] Jakob Nielsen. Usability inspection methods. In Conference Companion on Human Factors in Computing Systems, CHI ’94, pages 413–414, New York, NY, USA, 1994. ACM.

[10] Jakob Nielsen. Paper prototyping: Getting user data before you code, April 2003.

[11] Alexandra Pacureanu, Rongqin Ke, Marco Mignardi, Mats Nilsson, and Carolina W¨ ahlby.

Image based in situ sequencing for rna analysis in tissue. In ISBI, 2014.

[12] Petter Ranefall, Alexandra Pacureanu, Christophe Avenel, Anne E. Carpenter, and Car- olina W¨ ahlby. The giga-pixel challenge: Full resolution image analysis – without losing the big picture. In SSBA. Lule˚ a University of Technology, March 2014.

[13] Ian Sommerville. Software Engineering. Addison-Wesley, Harlow, England, 9 edition, 2010.