Thesis no: MSE-2013:141 09 2013
Image-based Microscale Particle
Velocimtery in Live Cell Microscopy
Edyta Tomalik
School of Computing
Blekinge Institute of Technology SE-371 79 Karlskrona
Edyta Tomalik 860916-P107 E-mail: edta10@student.bth.se University advisor(s): Dr. Ludwik Ku¹niarz School of Computing School of Computing
Blekinge Institute of Technology Internet : www.bth.se/com SE-371 79 Karlskrona Phone : (+46)0455-385853 Sweden
Dr. Eng. ukasz Mirosªaw
Wrocªaw University of Technology Wrocªaw University of Technology AI Department
Wrocªaw University of Technology Wybrze»e Wyspia«skiego 27
Celem niniejszej pracy dyplomowej jest zbadanie techniki zwanej Particle Image Velocimetry do analizy problemu biologicznego, zwi¡zanego z adhezj¡ komórek na obrazach kontrastowo-fazowych.
Adhezja komórek to zªo»ony proces odpowiedzialny za przyª¡czenie si¦ komórek do odpowiednich organów. Analiza tego problemu zwi¡zana jest ze zªo»onymi zale»no±ciami chemicznymi, ograniczon¡ wiedz¡ dotycz¡c¡ zale»no±ci zycznych wyst¦puj¡c¡ podczas adhezji oraz na trudne w analizie ±rodowisko bada«. Pierwszym podej±ciem do rozwi¡zania problemu jest propozycja modelu mate-matycznego, który opiera si¦ na analizie ruchu mi¦dzy dwoma komórkami. Drugi model zakªada natomiast, »e wa»na w adhezji jest pr¦dko±¢ danej komórki w cza-sie t.
Sama ewaluacja algorytmów, które mogªyby zosta¢ wykorzystane do badania pro-blemu, opiera si¦ na przeprowadzeniu dwóch eksperymentów.
Pierwszy eksperyment wi¡»e si¦ z ekstrakcj¡ komórki z obrazu. Algorytmy pod-dane analizie zostaªy wybrane poprzez przegl¡d literatury oraz ich wst¦pn¡ ocen¦. Drugi eksperyment porównuje Particle Image Velocimetry oraz jedn¡ z metod przepªywu optycznego - algorytmu zwanego Lucas Kanade. W obu eksperymen-tach gªównym problemem jest wybór miary, na podstawie której algorytmy zo-staj¡ poddane ewaluacji.
Rezultatem pracy jest wybór i ewaluacja algorytmów dotycz¡cych analizy obrazu. Na podstawie otrzymanych wniosków stwierdzono, »e efektywnym podej±ciem do badania adhezji komórek wydaje si¦ by¢ poª¡czenie algorytmu Lucas Kanade oraz idei wykorzystanej w Particle Image Velocimetry. Ponadto podj¦to dyskusj¦ na temat wykorzystania szeregów czasowych w ocenie algorytmu ±ledz¡cego komórki na obrazie.
Sªowa kluczowe: Particle Image Velocimetry, adhezja komórek
Background: Nowadays, one of the medical problem is rolling cell adhe-sion. Rolling cell adhesion is a complex process that requires the analysis of the challenging environment such as body uid and is the process responsi-ble for recruiting the cell to speci c organs. In order to explore the rolling cell adhesion, mathematical model is proposed. Dierent image processing methods are created, such as optical ow - Lucas Kanade algorithm, and other type of methods related to mechanical uid, namely PIV (Particle Image Velocimetry).
Aim: The aim of this master thesis is the identi cation of challenges while using PIV in live cell images and propose the algorithm, which may ana-lyze the rolling cell adhesion problem.
Methods: In order to understand properly the rolling cell adhesion problem from biological site, literature review combined with the expert consultation is performed. According to gather information, mathemati-cal model is proposed. Particle Image Velocimetry is explained according to literature review, where at the beginning the expert recommends some books as a primary research. As a result of this research, PIV challenges are identi ed and generally PIV idea is explained. Then two experiments are performed. The rst experiment evaluates detection algorithms and the second one, analyses track algorithm vs. PIV. In order to evaluate the mentioned algorithms, some evaluation method are selected and some criteria are de ned. Unfortunately the found methods are not perfect, therefore a new method related to performance evaluation using time se-ries is proposed.
Thesis result: The result of this thesis is a proposition of the algorithm, which can be used in the rolling cell adhesion. The algorithm is formed according to the detailed exploration of the rolling cell adhesion and anal-ysis of the selected algorithms related to the image analanal-ysis during the theoretical research and experiments.
In this place I would like to express my sincerest apprecia-tion to all those who provided and motivated me to com-plete this research.
In particular, I wish to express my gratitude for the help, technical support, consultation and forbearance from my both supervisors, Dr. Eng. ukasz Mirosªaw and Dr. Lu-dwik Ku¹niarz.
Furthermore I would like to acknowledge with much appre-ciation the support of my Parents.
1.1 Result of using Particle Image Velocimetry algorithm . . . 2
2.1 Illustration of cross-correlation . . . 8
2.2 Illustration of particle displacement in two consecutive images . . 10
2.3 Immersion process of watershed . . . 12
3.1 Illustration of segmentation . . . 14
3.2 Idea of Lucas Kanade algorithm . . . 18
4.1 Comparison of cell detection algorithms results . . . 27
5.1 Distance calculations . . . 34
5.2 Distance between two independent trajectories . . . 35
5.3 Marked cell moves slowly (the image is taken in 0 minut) . . . 38
5.4 Marked cell moves quickly (the image is taken after 24 minutes) . 38 5.5 Results obtained from PIV algorithm . . . 39
5.6 Paths calculated by PIV algorithm, Lucas Kanade Method and manually . . . 39
5.7 Paths calculated by PIV algorithm with centroids . . . 39
6.1 Densities of α - stable distribution for many dierent coe cients . 44 6.2 Densities of skewed α - stable distribution for many dierent coef- cients . . . 45
6.3 Cumulative distribution functions of α - stable distribution for many dierent coe cients . . . 45
6.4 Cumulative distribution functions of α - stable distribution for many dierent coe cients . . . 46
6.5 Empirical distribution of manual marked data . . . 47
6.6 Empirical distribution of results from Lucas Kanade algorithm . . 47
6.7 Empirical distribution of manual marked data . . . 48
6.8 Empirical distribution of results from Lucas Kanade algorithm . . 49
7.1 Rolling cell adhesion algorithm design ow . . . 52
4.1 Structure of image preprocessing . . . 24
4.2 Average standard deviation . . . 26
4.3 Precision-recall average . . . 26
4.4 Confusion matrix . . . 29
5.1 Metric for all results presented in Figure 5.2 . . . 36
5.2 Metric for the rst ten results presented in Figure 5.2 . . . 36
5.3 Metric for the last fty results presented in Figure 5.2 . . . 36
5.4 Comparison of a cell with slow motion vs. cell with quick motion 37 5.5 Comparison of cell with slow motion vs. cell with quick motion . . 37
7.1 State Transition Table . . . 51
Abstract ii
1 Introduction 1
1.1 Background . . . 1
1.2 Aims and objectives . . . 3
1.3 Research questions . . . 3
1.4 Expected outcomes . . . 4
1.5 Research methodology . . . 4
1.6 Validity threats . . . 5
2 Research problem 6 2.1 Rolling cell adhesion problem . . . 6
2.1.1 Phase Contrast Images . . . 7
2.1.2 Rolling cell adhesion model . . . 7
2.2 Particle Image Velocimetry . . . 7
2.2.1 PIV challenges . . . 10
2.2.2 Cell detection algorithms . . . 11
2.3 Related work . . . 12
3 Theory 14 3.1 Pre- and post- processing methods . . . 14
3.1.1 Segmentation . . . 14
3.1.2 Interrogation window . . . 15
3.1.3 Mathematical morphology . . . 15
3.1.4 Threshold . . . 16
3.1.5 First and second derivatives . . . 16
3.1.6 Local standard deviation . . . 16
3.2 Detection and tracking algorithms description . . . 16
3.3 Performance evaluation of cell segmentation . . . 19
3.3.1 Supervised evaluation method . . . 19
3.3.2 System-level evaluation . . . 20
3.4 Performance evaluation of cell motion tracking . . . 21
4.2 Conducting the experiment . . . 25 4.3 Analysing the experiment result . . . 28 4.4 Discussion . . . 29
5 Experiment 2 31
5.1 Planning the experiment . . . 31 5.2 Conducting the experiment . . . 32 5.2.1 Manually marked data preparation and observation . . . . 33 5.2.2 The tracked data preparation and observation . . . 35 5.3 Analysing the experiment results . . . 40 5.4 Discussion . . . 41 6 Performance evaluation of cell motion using time series 43 6.1 Theory . . . 43 6.2 Real data analysis . . . 46 6.3 Discussion . . . 49 7 Rolling cell adhesion algorithm proposition 50 7.1 Problem de nition . . . 50 7.2 The overview of the proposed method . . . 50 7.3 Algorithm design . . . 51
8 Conclusions 53
8.1 RQ1: Rolling cell adhesion problem . . . 53 8.2 RQ2: Investigation biological image analysis by adopting PIV
al-gorithm . . . 54 8.3 RQ3: Rolling cell adhesion algorithm proposition . . . 55 8.4 Future work . . . 55
References 57
Introduction
This chapter presents generally the overview of the research problem. Moreover it de nes the aim, objectives, research questions and expected outcomes of this thesis. The end of this chapter describes methods used in this thesis and discuses validity threats.
1.1 Background
Qualitative evaluation of rolling cell adhesion seems to play a crucial role in studying infections and injuries [19]. Study of cell kinematics and dynamics can be performed in vitro with a method called micro-PIV, where the cells are ob-served under the microscope [28][31]. Information about a relative position of analyzed cells on a microscopic slide is indispensable for proper assessments of cell rolling.
Dierent image processing methods are evaluated to track the cells in time by means of microscopic images. A special emphasis has been put on a proper usage of the PIV method [31]. A quantitative comparison of the detected trajectories with one trajectory generated by an expert is performed. Finally, it is elaborated that the PIV method seems to be reliable and robust for a selection of time-lapse movies acquired with dierent microscopes.
Biological process, called cell rolling, is responsible for slowing down the cells ow in the blood vessels. Furthermore, the process causes the cell adhesion to the vessel wall [24] [7] [20].
In order to elucidate the complex cell adhesion process, two mathematical mod-els are proposed, namely kinetics model of the cell adhesion and an equilibrium model of cell adhesion. The kinetics approach takes on boards the dynamic of cell adhesion process but the equilibrium model applies poor chemical and biological mediation between cell-cell or cell-matrix adhesion [13].
The kinetics model of adhesion, from computer scientists point of view, seems to be much more reliable, owing to the fact that it provides information related to
cell dynamic [14]. However, this model does not seem to consider cell deformation during the adhesion process.
In order to obtain quantitative measurement of rolling and adhering cells, Particle Image Velocimetry (PIV) [9][18] combined with image analysis methods are used to obtain the quantitative cell ow. The image analysis method, such as detection algorithm, prepares medical images to PIV calculations. For this reason, the aim of this master thesis is to adopt PIV, which comes from mechanical uid, to solve a biological problem associated with the rolling cell adhesion.
PIV is a visualization and measurement method. Its purpose is to obtain proper-ties of the uid [30]. Today, PIV has a wide application in various research elds. The use of PIV includes micron-resolution particle image velocimetry (µPIV) and applications in such areas as micro systems, biomedical ows, 3D-PIV, compari-son with CFD, household appliances, turbo machinery, internal combustion, car industry, complex aerodynamics, supersonic ows and naval applications [28][31]. Particle Image Velocimetry (PIV) is a method, which measures velocity gradient tensor du
dx of uid (see Figure 7.1). Therefore, advance ow properties can be
investigated by PIV because PIV maps ow uid to set of particles and then cal-culates the displacement between suitable particles [9]. The latest achievements
Figure 1.1: Result of using Particle Image Velocimetry algorithm
of PIV are summarised in [31]. This paper describes PIV image algorithms. Moreover, PIV images and analysis method are available in [4]. What is more, general cell image analysis algorithms are described by [15]. In particular, the algorithm, Circadian Gene Expression algorithm responsible for object detection will be considered.
to investigate rolling cell adhesion using PIV?
PIV was created to analyze the poor PIV images (PIV images are images with short time image and high density). Unfortunately, the poor PIV cell images are not available due to high cost and complex process of using PIV technique. Nev-ertheless, live cell images from The Swiss Federal Institute of Technology Zurich and PIV images from PIV Challenge [4] have been applied for the purpose of this thesis. The aim of this master thesis is to indetify the challenges of using PIV in live cell images and propose the algorithm, which may analyze rolling cell adhesion problem.
It seems that the main problems of this thesis are the rolling cell adhesion model and overpassing the challenges, which appears when PIV algorithm is adopted to the rolling cell adhesion model.
1.2 Aims and objectives
The aim of the master thesis is to adopt the Particle Image Velocimetry to ana-lyze the rolling cell adhesion biological problem.
Objectives of this research are:
O1: To identify the rolling cell adhesion problem; O2: To explore PIV algorithm;
O3: To propose the algorithm, which adopts PIV;
In order to adopt the Particle Image Velocimetry to analyze the biological prob-lem, the biological problem (O1) and Particle Image Velocimetry (O2) have to be properly understood. It is indispensable to know what exactly have to be ana-lyzed and how. If an algorithm has to be created or modi ed, the main important is to nd out its advantages, disadvantages, limitations and assumptions. If the algorithm, which has to be adopted, is well known, and the problem, which has to be analyzed, is properly understood, the algorithm can be proposed (O3).
1.3 Research questions
To achieve the objectives of the following research questions are introduced: RQ1: What is the rolling cell adhesion problem?
RQ2: How to adopt PIV algorithm to investigate the biological images? RQ2.1 What are the challenges of PIV?
RQ2.2 What are the eective algorithms of cell detection?
RO2.3 How to compare the adopted PIV algorithm with optical ow al-gorithm?
RQ3: How to adopt the PIV algorithm to the rolling cell adhesion?
1.4 Expected outcomes
EO1: Presentation of rolling cell adhesion problem; EO2: Explanation of PIV;
EO2.1: Identi cation of the PIV challenges;
EO2.2: Evaluation of the selected image processing algorithms used for cell detection;
EO2.3: Comparison of the adopted PIV algorithm with the optical ow one;
EO3: Proposition of the algorithm, which adopts the PIV;
1.5 Research methodology
The research has been derived into three research questions. In order to answer to the rst research question, theoretical review related to biological aspects is performed. However, the traditional theoretical review is insu cient, therefore some additional information is gathered through the consultation with expert. In order to answer to the second research question, the theoretical review is performed but the review is focused on the elucidation of PIV algorithm and indication of PIV challenges. What is more, additional information and some technical aspects, are explained using mail messages with the authors of gathered articles or programmers of some applications.
What is more, to answer to RQ2.2 and RQ2.3, two experiments are performed. The last part of this research is the algorithm synthesis.
analysis of the thesis needs. The rst need has to answer the question: "How to gather the main important information and data related to this thesis". The second need is associated with evaluation of speci c algorithms.
Due to complex nature of this thesis, known literature review approaches, such as systematic literature review or traditional literature review, are not su cient. The main characteristic of this thesis is that some advanced technical problems has to be deeply analysed. Therefore the literature review has to explain some-thing, not collect all literatures related to some topics. At the beginning kind of snowball technique has been used, because an expert recommend some books such as [25]. During the study some questions and doubts are appeared. The answer to these questions is looked for literature and any doubts are discussed with experts. Image analysis depends on preprocessing method in order to, for example, sup-press image noise. In order to use some image algorithms, rstly data (image) has to be manipulated. Only during the experiment variables are manipulated. Observational study and survey do not manipulate any variable. Therefore, as a natural consequence of these, the experiment is selected.
1.6 Validity threats
It seems that validity threats [35] concern the following phases of research process - Literature Review and Experiments.
Validity threats related to literature One of the main threats to the valid-ity of literature review is lack of literature closely related to the rolling cell adhesion. In particular, there is no direct source of information about physical dependencies occurring in the rolling cell adhesion, therefore, the literature recommended by an expert has been analysed as a basis for a further research.
Validity threats related to the fact, that the author is not a biologist The author is not a biologist, therefore any conclusions and each doubts have been discussed with the expert.
Research problem
2.1 Rolling cell adhesion problem
Common problem in many biological areas seems to be cell motility [24]. In fact, cell motility seems to be a fundamental evolution achievement. However, many body cells are stationary. In order to defend body against any infection, the cells move and search for foreign organism. What is more, an uncontrolled cell movement reveals a cancerous cell [17]. One of the issues closely associated with motility is rolling cell adhesion. Rolling cell adhesion is manifested by mechanical work. Therefore, according to analysis cell images, in particular cell movement and displacement, rolling cell adhesion can be measured and evaluated. Further-more, the image analysis result allows assessing the division level, absorption and collision of the relevant cell classes.
From the chemical point of view, the rolling cell adhesion is a process, which depends on behaviour between cells' structures [7] [20]. The importance of rolling cell adhesion is most noticeable in cancer cells [33]. The major cause of cancer mortality is a metastasis. Cell adhesion plays the most important role in the metastasis. At the beginning, the cancer cells form malignant tumor. Then some cells detach from the tumor and move to other part of the body through lymphatic system or bloodstream. Consequently, they penetrate a new tissue or organ and form new malignant tumor. If some cancer cells detach themselves from the tu-mor, the adhesion between cancer cells decreases. However, when cancer cells form a new malignant tumor, the adhesion between cancer cells increases [33]. For this reason, a thorough understanding the variation of adhesion mechanism, especially during metastastis, can contribute to a discovery of a new possibil-ity of cancer therapy. Sometimes one single cell is damaged and in exchange malignant cells are produced. The malignant cells behave abnormally during mi-tosis. The mitosis is a cell division of nucleus [12]. During the cell division, the cell rounds up, therefore, the cell contrast on phase contrast images increases [12]. In this case, a conclusion can be derived that cell division increases contrast, therefore the sequence of phase contrast images will be characterized by high
variation of intensity.
2.1.1 Phase Contrast Images
Image analysis helps scientists answer some biological problems. Nevertheless, the image analysis seems to be di cult and time-consuming, when performed manually. Natural solution is an automated image analysis. Unfortunately, due to a complex biological image characteristic, there is no optimal algorithm, which seems to generate an eective result. Furthermore, biologists analyze many types of real images, such as uorescent, DIC or phase contrast images. From biological point of view, the most important images seem to be taken in the phase contrast microscopy of live cells as the phase contrast images. The main advantage of the phase contrast image is that this kind of images present original cells unchanged by any chemical. On the other side, there are many types of phase contrast images, therefore, it seems to be almost impossible to create a fully automated algorithm due to the fact that cellular periphery and nucleus are surrounded by the feature called halo [3].
2.1.2 Rolling cell adhesion model
For the purpose of this thesis, the following model has been designed. The rolling cell adhesion problem is described by means of mathematical language. Firstly, the assumption the tumor is set of cells is made. Secondly, it is generalized that if adhesion increases, the dierence between cells decreases. On the other hand, if adhesion decreases, the dierence between cells increases. Moreover, if the dif-ference increases, the cell speed increases.
Let Ij be the phase contrast image, where j = 1, ..., n. The number n is equal to
the number of phase contrast images. Let cij be the i-th cell on j-th image, where
i = 1, ..., m. The number m is equal to the number of cells. Now the adhesion can be expressed as follows:
|cij − ckj| < (2.1)
where i 6= k
In order to apply the 2.1 Equation, the intensity variation has to be removed from images. Then cells have to be detected and, at the end, the cell track has to be calculated.
2.2 Particle Image Velocimetry
with two velocity dimensions, namely: - time and length. On the other hand, this technique indirectly describes particle velocity instead of the uid velocity [25]. Therefore, PIV estimates the instantaneous and average particle velocity vectors throughout a ow eld, which is actually a set of particles which create the ow eld. In this case, uid mechanical problems can be considered as a particle motion problem.
PIV measures the displacement of small tracer particle during short time interval. In order to keep uid properties, the particle tracers should be su ciently small [9].
In order to calculate the distance between particle pattern in image time, the cross-correlation function is used and applied not for the whole image but for a small part of an image called an interrogation window. The result of cross-correlation for one interrogation window is one velocity vector. The relation of velocity vector u and particle displacement d is de ned by [25]:
u = d
M δt (2.2)
where:
M is magni cation δ t is image time
Figure 2.1 illustrates the correlation process. Implementation of cross-correlation can be expressed by the following equation:
f (x, y) ◦ k(x, y) = w 2 X i=−w2 h 2 X j=−h2 f (x + i, y + j)k(i, j) (2.3) where: w - width of template k; h - height of template h; f - has origin at the top left; k - has origin in the middle;
The cross-correlation technique assumes that the uid motion is parallel during short time interval. Moreover, the technique seems to be a time consuming oper-ation because cross-correloper-ation function has to be calculated for each point in the image. The complexity of cross-correlation is equal to O(n2). In order to reduce
the computation of cross-correlation operations to O(Nlog2N) operations, Fast
Fourier Transformation is applied to cross-correlation [25].
f (x, y) ◦ k(x, y) =f (x + i, y + j) \\ k(i, j) (2.4) where:
f (x + i, y + j)\ is a Fourier Transform of f(x+i,y+j) \k(i, j)is a Fourier Transform of f(i,j)
Therefore, the alternative for calculation of the cross-correlation de ned by Equa-tion 2.3 is to use the cross-correlaEqua-tion of two funcEqua-tions as a complex conjugate multiplication of their Fourier transforms (see [25]). This dependency is proved by cross-correlation theorem and expressed by equation 2.4.
Cross-correlation considers two images and does not seem to present the problem with ambiguity, zero velocity and decreased resolution.
2.2.1 PIV challenges
Particle Image Velocimetry is a technique, which maps ow in the uid [24], but Particle Image Velocimetry algorithms seems to focus on analysis of poor PIV im-ages [31] [25]. The main characteristic of PIV imim-ages is that the imim-ages present short time images in order to achieve small displacement between objects on each image [25].
The images analyzed in this thesis are a sequence of phase contrast images. This image sequence was collected during one hour. The images present cell motion, which can be distorted by rotation or adhesion. In this case, an assumption that the uid motion presented in the image sequence is parallel should be excluded, applying a cross-corelation technique to this image sequence would probably re-fute and skew the results.
Analysing the Figure 2.2 the pattern of cells motion is not parallel, but is
dis-Figure 2.2: Illustration of particle displacement in two consecutive images
torted. Moreover, the cross-correlation between the two images from Figure 2.2 is very small, only cell denoted as A overlap itself. Therefore, applying fundamental cross-correlation technique to a motion analysis with some disturbances does not seem to be su cient enough to obtain e cient results.
2.2.2 Cell detection algorithms
The cell detection is associated with isolating the object from the background. For this reason, the cell detection process can be considered as an image segmen-tation process, because image segmensegmen-tation is typically focused on locating the object in the background [26]. According to literature review, the segmentation can be distinguished as two categories. First category of segmentation is based on abrupt changes in intensity, the other one is based on dividing the image into parts de ned by the criteria. It means that the segmentation process divides the image into homogeneous regions with respect of some properties, such as color or object. The cell segmentation is mainly performed with combination of morphological op-erations [11] [38] [29]. However, it has to be known that the cells and surrounding uid impact on the close cells physically and chemically. Physical properties of the cell environment coordinate the mentioned processes. Especially the cell environ-ment in uences the cell adhesion [32]. In motile cells, the adhesion is constantly assembled and disassembled, when the cell starts new contact, it breaks an old one. Cell motility and cell shapes depend on cell adhesion. If cell moves, the morphology of the cell is changed.
The algorithm which uses the based-region segmentation approach is called the watershed segmentation [26]. The watershed segmentation refers to a morpho-logical method of image segmentation. The watershed idea can be imagined as a landscape immersed in the lake with holes pierced in local minimum [26]. Basins are lled up with water starting at each local minimum but dams are built where water meets from dierent basins. The results of watershed segmentation is that separat individual basins of the landscape engage. The watershed segmentation process in illustrated in Figure 2.3.
Fundamental feature of image, such as direction, shape, characteristics, are con-tained in image edge [39], [1]. Generally, the edge can exist in the following situation:
between object and object between object and background between area and area
Figure 2.3: Immersion process of watershed
Canny algorithm is combined with mathematical morphology.
From practical point of view, Canny edge detection algorithm with initial edge threshold is used to identify the cell edges. The obtained detected edges are op-tionally subjected to morphological operations such as erosion and dilation, which connect discontinuous edge segments. Calculated and connected components are used to obtain the object contour.
2.3 Related work
This thesis is focused on the analysis of rolling cell adhesion among the images sequence. There is no additional information such as range of cell diameter or chemical aspects of uid cell. Therefore, the purpose of this research is to inter-pret and explore using proper algorithms the sequence of cell contrast images. It seems that the most important information for further analysis is the fact that there is no any additional information about chemical and biological aspects of analysed cells.
Theory
3.1 Pre- and post- processing methods
3.1.1 Segmentation
Let R denote the entire image region. The segmentation divides R into n subre-gions R1, ... Rn such as [11]:
Sn
i=1Ri = R
Ri is a connected region i = 1, .. n
Ri∩ Rj = ∅ for all i and j, i 6= j
P (Ri) = T RU E
P (Ri∪ Rj) = F ALSE for any adjacent region Ri and Rj
where ∅ is a null set, but P (Ri)
The rst condition indicates completion of the segmentation. It means that
Figure 3.1: Illustration of segmentation
every pixel belongs to the region. The second condition about connected regions requires that each region Ri, in topological sense, cannot be represented by any
union of two or more disjont nonempty regions.
The third condition indicates that a part of region cannot be part of another one. The fourth condition requires that each pixel in the region Ri has to present some
common characteristics, for example, each pixel in the region which has the same gray level [11]. Figure 3.1 presents the segmentation. On left hand side, an entire image region is visible, on the right hand side, there is a region divided into three partitions.
3.1.2 Interrogation window
Interrogation window is an image neighborhoods of size (2ωx + 1) × (2ωy + 1),
where ωx, ωy ∈ N. It means that interrogation window is a part of an entire
image, which is explored in order to nd image features, e.g. to detect an object.
3.1.3 Mathematical morphology
Mathematical morphology is [38] [21] a mathematical theory associated with geo-metrical structures analysis and processing. This mathematical theory is adopted to image analysis and identi cation [38]. Generally, mathematical morphology is used to extract some components from images. The extracted components de-scribe shape in respect of boundaries or skeletons. Moreover, the mathematical morphology is used in pre- and postprocessing such as morphological ltering [11]. One of the basic mathematical morphology operations are erosion and dilation, open and closed.
The binary erosion A by B is de ned as: A B = \
b∈B
A−b (3.1)
The binary dilation A by B is de ned as: A ⊕ B = [
b∈B
Ab (3.2)
The binary open is de ned as:
The binary close is de ned as:
A • B = (A ⊕ B) B (3.4) where A is a set of binary images, but B is the set of structure element [38].
3.1.4 Threshold
Let f(x,y) denotes the image, whose intensity level of the image is grouped into two dominant modes. One mode characterizes objects (foreground) and the other one - the background. In order to exact the objects, the modes have to be separated by threshold T [11]. Therefore, the threshold can be de ned as follows:
f (x, y) =( 1 if f(x, y) ≥ T
0 if f(x, y) < T (3.5)
3.1.5 First and second derivatives
The rst derivative of the image is a gradient. The image gradient is de ned by: ∇f = ∂f
∂xx +ˆ ∂f
∂yyˆ (3.6)
The second derivative of image is de ned by: ∇2f = ∂
2f
∂x2x +ˆ
∂2f
∂y2yˆ (3.7)
3.1.6 Local standard deviation
Standard deviation is a statistical value, which determines the sample data spread, or, how far the data are from the sample mean.
Let X presents pixel set of image I with mean value E[X]=µ. Local standard de-viation is a standard dede-viation of 3-by-3 neighborhood around the corresponding pixel from X in image I. The standard deviation is expressed by the following simple formula:
σ = pE[(X − µ)2 (3.8)
3.2 Detection and tracking algorithms description
Circadian Gene Expression
method assumes that the cell shape is convex. The cell contour is constructed by n nodes that are interpolated linearly. The nodes are speci ed by their polar coordinates (ρ, θ) along rays R starting at the known center c. Due to the unique parametrization the contour detection problem is expressed as a shortest path problem [27].
Canny algorithm
The fundamental image features, such as direction, shape, characteristics, are contained in an image edge [39], [1].
The aim of object detection, actually, edge detection, is to extract the image edge using rst and second derivative, high pass ltering and morphology [38] [29]. There are many dierent edge detection methods such as:
Edge detection based on gradient operator the edge is identi ed as a placed of the rapid changing gray value. The example of a gradient operators are Sobel or Prewitt [11].
Edge detection based on optimum operator the edge is identi ed as a point whose second derivative is equal to 0. The example of optimum operator is the Canny operator.
What is more, according to the literature the most important and useful edge detection algorithm is Canny algorithm [38]. The disadvantage of the Canny algorithm is that the result of that algorithm depends on texture and image noise. In order to improve the result and achieve a more accurate edge extraction, Canny algorithm is combined with mathematical morphology.
Lucas Kanade algorithm (optical ow)
The aim of the the Lucas Kanade algorithm is to track the identi ed point in the consecutive images. Let A and B denote two gray scale images, such as A is taken in time t but B is taken in time t + δt. In order to nd the displacement vector of some interesting pixels, rstly, the pixels have to be indicated in the A image and then the set of pixels has to be detected in the B image. If the proper pixels are identi ed in the A and B images, then the displacement vector between them can be calculated.
Let u be the image point such as u = [uxuy]T in the A image. Let v be the
corresponding image point in the B image expressed as v = u + d = [ux+ dxuy+
vector d is the vector that minimizes the error function de ned by Equation 3.9 (d) = (dx, dy) = ux+ωx X x=ux−ωx uy+ωy X y=uy−ωy (A(x, y) − B(x + dx, y + dy))2 (3.9)
where ωx and ωy are two integares, usually ωx and ωy ∈ 2, 3, 4, 5, 6, 7.
Analysing the Equation 3.9 seems to present di culty in achieving a
compro-Figure 3.2: Idea of Lucas Kanade algorithm
mise between large movement detection and algorithm speed. When high-speed motion is investigated, intuitively a large interrogation window is expected. How-ever, if the interrogation window is extended, the time algorithm execution will be increased. On the other hand, the Equation 3.9 indicates that dx ≤ ωx and
dy ≤ ωy. In order to solve this problem, the image piramidal representation is
Gaussian lter
Gaussian lter is a lter, which smoothes an image and remove noise from the image by calculating weighted average in a lter box. The Gaussian lter is derived from the same equation used in Gaussian distribution and is expressed by Equation 3.10.
f (x, y) = 1 σ√2π exp
−x2+y2
2σ2 (3.10)
3.3 Performance evaluation of cell segmentation
3.3.1 Supervised evaluation method
Supervised evaluation method is used in segmentation algorithm evaluation. The method compares the image segmentation results against the manual segmenta-tion results [37]. The manual segmentasegmenta-tion refers to an object detecsegmenta-tion and is called ground-truth. The similarity between manual detection and image seg-mentation result indicates the quality of segseg-mentation algorithm.
The clear advantage of supervised evaluation method is believed to obtain better evaluation. However, the manual segmentation seemes to be time-consuming, subjective and quite di cult. Moreover, in particular in biological images, it can-not be a guarantee that segmentation performed manually is better than others. What is more, many dierent supervised evaluation methods have been proposed. In this thesis the ROC curve, especially precision-recall method has been used.
Receiver Operating Characteristics (ROC) curve
ROC curve is a two dimensional graph, where true positive rate is presented in the Y axis and false positive rate is presented in X axis. Therefore, the graph draws tradeo between bene ts (true positives) and cost (false positives), means the y-axis corresponds to Sn and y-axis corresponds to 1-Sp, where Snis a sensitivity
and Sp is a speci city. Sensitivity and speci city are de ned by the following
equations: Sn = T P T P + F N (3.11) Sp = T N T N + F P (3.12) where,
TP true positives - detected cell correspond to ground-truth cell;
TN true negatives - no ground-truth cell does not correspond to the detected cell;
However, there are many interpretations of ROC curve and it seems that more usefull from the thesis point of view is relation between correctly identi ed cells over all ground-truth cells (recall) to correctly identi ed cells over all detected cells (precision). The relation is named precision-recall and is expresed by the following equations [22]
recall = T P
T P + F N (3.13)
precision = T P
T P + F P (3.14)
The clear advantage of precision-recall seems to be simple analysis positives de-tection vs. negatives dede-tection, because if the number of false negative increases, the recall decreases. Moreover, if the number of false positives increases, the pre-cision decreases. What is more, in case when the number of negatives increases, it is not visible in ROC curve.
The disadvantage of ROC curve is that the ROC curve is sensitive to class dis-tribution and error costs.
3.3.2 System-level evaluation
Alternative evaluation method examines the impact of the segmentation method on the system [37]. In this thesis, the segmentation prepares some data to track analysis. Therefore, this method seems to be quite interesting and applicable to the considered problem. In this case, better segmentation method is selected according to the emiprical system result. In order to assess empirical system result, in particular, in order to answer the question how the segmentation method imapcts the track analysis, some criteria are introduced.
C1: The segmentation result has to minimize the number of false positives, even if the false positives minimization entails true positive minimization. C2: The shape of the detected cell should cover the ground-truth cell as much
as possible. However, in order to avoid discrepancies associated with cell shape, in further examination, only the cell velocity will be taken into ac-count.
C3: The segmentation result should be applicable to further analysis.
3.4 Performance evaluation of cell motion
track-ing
According to literature review, the performance evaluation of cell motion tracking is a challenging problem due to lack of a unique metric to assess dierent asspects of motion tracking. In order to overcome this problem, the aim of the tracking evaluation has been identi ed as the eectiveness evaluation of a high precision tracking based on the comparison of the tracked data to the manually marked data. Therefore, the performance evaluation of cell motion tracking has been consisted of two parts. The rst part has been assessed the manually marked data, the second one has been evaluated the tracked data as a result of Lucas Kanade and PIV algorithm.
The manually marked data evaluation assesses two independently hand marked trajectories of the same cell during time motion. Both trajectories have been marked by the same person, by clicking on the image.
Let denote T1 and T2 as the two independently hand marked trajectories. The
track T1 has been de ned as: T1 = {(xi1, yi1) : 1 ≤ i ≤ N }. The track T2has been
de ned as: T2 = {(xi2, yi2) : 1 ≤ i ≤ N }, where i describes the frame number, but
{(xi1, yi1)} and {(xi2, yi2)} identify the cell center. Then the dierence between
trajectories has been calculated by the following equation: di =
p
(xi1− xi2)2+ (yi1− yi2)2 (3.15)
In order to assess the two independently hand marked trajectories the mean expressed by Equation 3.16, median, minimum and maximum have been analyzed. Median separates the higher half of data from the lower half, but minimum and maximum are expressed by Equation 3.17 and 3.18 respectively.
ˆ d = 1 N N X i=1 di (3.16) min(x, y) =( x if x ≤ y y if y ≤ x (3.17) max(x, y) =( x if x ≥ y y if y ≥ x (3.18)
EC1: The cell trajectory fragmentation breaks down the cell motion overview; EC2: The close trajectory estimation is su cient condition to analyze the cell
velocity;
Analysing the de ned evaluation condition EC1, it seems, that the proper eval-uation metric is a metric that track the trajectory fragmentation [36]. Track fragmentation identi es the trajectory discontinuity. The best result of tracking algorithm is a trajectory with lack fragmentation. It means, that tracking algo-rithm produces the continuous trajectory.
Experiment 1
The objective of the experiment is to evaluate three detection algorithms selected from literature review. The aim of the experiment is to indicate the most eec-tive detection algorithm. The eeceec-tiveness is measured by supervised evaluation method and system-level evaluation.
4.1 Planning the experiment
Due to lack of random participants assigned to the treatment or control group, the quasi-experiment is selected [35]. In order to formalize the experiment, the following hypotheses are formulated.
H0: The segmentation method (algorithm) does not aect nal result.
(detec-tion objects)
If the algorithms results are similarly to each other, the ratio of true posi-tive and false posiposi-tive will be the same for the result from each algorithm, therefore segmentation method does not aect nal result.
HA: The segmentation method (algorithm) aects nal result.
It is expected that the algorithm maximizes detection of correct object (ex-act cells) and minimizes the incorrect detection of cells, then the algorithm is more eective. However, the alternative hypothesis can be stated, when ratio of true positive to false positive is going to 1 and in this case the more eective algorithm is when the ratio is higher than others.
If the algorithm nds more proper objects (cells), the result from algorithm that assesses the rolling cell adhesion will be more valuable. By measure-ment the correct detection (true positive) and incorrect detection (false positive) we understand true positive rate and false positive rate in ROC.
The de ned null hypothesis H0against alternative hypothesis HAare tested using
independent and dependent variables. Independent variables are a sequence of phase contrast images. The dependent variables are results from algorithms exe-cution, therefore there are images contained segmented regions (detected objects). In order to execute the algorithms, the experimental data (sequence of phase contrast images) is at the beginning prepared so as remove noise and intensity variation from images (see Table 4.1).
Number Algorithm Preprocessing methods No. 1 Detecting a cell using image segmentation Open morphological operation No. 2 Circadian Gene Expression Parametrization of the active rays No. 3 Canny algorithm Average Background Subtraction
Table 4.1: Structure of image preprocessing
The preprocessing method depends on the nature of the study, and for this reason there is no unique method that could be applied to prepare an image to be used by each algorithm (each method and each algorithm is described in Chapter 3.1). Appropriate tools, which could help conducting the experiment were choosen during literature review and own experience. Moreover the following selection criteria were introduced:
availability of documentation and speci cation; own experience with the tool such as MATLAB; preprocessing, detection and tracking algorithms;
If some speci cation were not available or were not understood, the authors of the tool were asked via mail to contribute and share needed documents. Each asked authors replied for mail and helped. Among many available tools and after their analysis the CellTrack [1], MATLAB and Circadian Gene Expression [2] were selected.
Subject of the experiment is the number of proper selected cell via each de-tected algorithm and by the expert.
Independent variable is a sequence of phase-contrast images. However, De-pendent variable is a sequence of images with selected cells.
Measurement criteria used in this Experiment are described in Chapter 3.1 in Section entitled Performance evaluation of cell segmentation.
4.2 Conducting the experiment
Firstly, the selected tools were installed on Windows 7 Home Premium Operation System (64 bits) with the procesor Intel(R) Core(TM) i5 CPU 2.53GHz.
The tools instalation required additional programs such as Fiji or ImageJ. More-over not all algorithms were available in that tools, therefore some additional scripts had to be implemented.
The data which were selected to analysis are a sequence of phase contrast images. The images present cell motion. Moreover, the images were taken 60 times per one hour - each image in one minute.
The experiment contains two parts. First part is the preprocessing and the other one is a segmentation itself.
Analysing sequence of images, the image noise and non-uniform illumination are visible. During the preprocessing stages, the unexpected images features such as the noise and non-uniform illumination are removed. Moreover, due to cell motion over time, the intensity of foreground is changed. In method No. 1 (see Table 4.1) the mathematical morphology operation is used in order to remove the mentioned image disturbances. This morphological operation is called open and can estimate the image background. The Algorithm 2 presents pseudo code of open operation, where is a single structural element, but I is an image. The Algorithm 1 Pseudo code of open operation
A ← dilation(I, s) A ← erosion(A, s)
obtained result of open operation is subtructed from the orginal image and rea-sonable background is produced (see Algorithm 6).
In order to remove the image noise and prepare the image to the execute the Cir-cadian Gene Expresion algorithm, the Median Filter, Gaussian Smoothing Filter or Non-Linear Diusion Filter are used. Moreover, a time-lapse interval is de ned in minutes and typical radius of nucleus in pixels is estimated.
gener-Algorithm 2 Pseudo code of removing the non-uniforgm illumination from the image se ← CreateStructuralElement f 1 ← imopen(im, se) f 2 ← imsubtract(im, f 1) im ← im − f 2 bw ← CreateBinaryImage(im) LocalStandardDeviationOf (bw)
ates the similar result to the threshold result and indicates many regions, which are not interesting from the experimental point view. The Table 4.2 compares average standard deviation from the orginal image and from the result of each mentioned preprocessing method. As it can be seen, each method decreases the image average standard deviation. However, it does not mean that the best method is a method with the smallest average standard deviation result.
Average standard deviation from the result of:
Original image Method No.1 result Method No.2 result Method No.3 result
15.9684 10.3 9.3 2.69
Table 4.2: Average standard deviation
The Table 4.3 compares the nal result of applying the detection algorithms to the preprocessed images. The recall value for the each algorithm result is sim-ilar in contrast to the precision value. The precision result of Canny algorithm and detecting a cell using the image segmentation is quite low. The comparison of the precision-recall average values, seems to corroborate with the assumption that smallest standard deviation image value does not seem to imply at least satis ed nal detection result. The Figure 4.1 compares sample of the detection
Number Algorithm Precision Recall
No. 1 Detecting a cell using the image segmentation 33% 87% No. 2 Circadian Gene Expression 100% 100%
No. 3 Canny algorithm 35% 90%
Table 4.3: Precision-recall average
(1) Original chase contrast image (2) Example of Circadian Gene Expression
(3) Example of Cany edge detector with average background subtraction
(4) Example of Cany edge detector with Gaussian background subtraction
(5) Example of cell extraction using morphological operations
4.3 Analysing the experiment result
At the beginning, the algorithms results are measured by a supervised evalua-tion method. The supervised evaluaevalua-tion method is expressed by precision-recall average. The precision-recall average presents the relation between correctly iden-ti ed cells over all ground-truth cells (recall) to correctly ideniden-ti ed cells over all detected cells (precision). The precision-recall average presents Table 4.3. Ac-cording to the brief analysis, the results from Table 4.3, the recall average is high and can be accepted for each detection algorithm. However, according to the precision average, the rst and third algorithm is not su cient because many false positives are identi ed and the precision average is small, less than 50%, therefore the result is not e cient from the thesis poinf of view.
According to the result interpretation from supervised evaluation method, it is quite di cult to assess if one algorithm result is su cient or better than others. In order to resolve this inconsistency, the indirect method, namely the system-level evaluation method (Chapter 3.1), is applied to the result analysis. The system-level evaluation method assesses the impact of the segmentation on the system. In this case, the system expresses the rolling cell adhesion which consists of cell tracking.
The rst criterion de ned in the system-level evaluation method says that the segmentation result has to minimize the number of false positives, even if the false positives minimization entails true positive minimization. It means that the su cient algorithm minimizes the number of false positives, therefore, according to the result analysis from Table 4.3, the rst and third algorithms should not be taken into consideration because the algorithms increase the number of false positives.
The second criterion of the system-level evaluation method says that the shape of the detected cell should cover the ground-truth cell as much as possible. How-ever, in order to avoid discrepancies associated with the cell shape in further examination, only the cell velocity will be taken into account. Unfortunately, it is almost impossible to assess how much the detected cell covers the ground-truth cell. However, if the cell velocity will be taken into consideration, the algorithm which decreases numbers of false positives seems to be more e cient. In this case the second criterion is closly related to the rst criterion.
some additional operations.
To sum up, according to the supervised evaluation and the system-level evalua-tion methods the segmentaevalua-tion algorithm aects nal result (detecevalua-tion objects) and the ratio of true positive and false positive is dierent from each algorithm. Therefore, there seems to be enough evidence to reject the null hypothesis. What is more, using the rst and second criterion from the system-level evaluation method, the most useful segmentation algorithm is Circadian Gene Expression, owing to the fact that it decreases the false positives detection and increases the precision average.
4.4 Discussion
Proper cell extraction is indispensable to explore the cell motion from the im-ages. Among many imaging modalities, the most common is non-invasive contrast microscopy. The automatic cell extraction and tracking from phase-contrast images is di cult due to non-uniform intensity and the halo eect sur-rounding a cell. Therefore, the automatic cell segmentation (exctraction) is chal-lenging and the standard detection algorithms, such as simple morphological op-erations, are ineective. The basic morphological operations are not e cient in phase-contrast images segmentation because the contrast between the cell and background is low. For this reason, the basic morphological operation cannot dis-tinguish the exact cell edge and some arti cial edges, eg. the object edge which is not the cell. Moreover, the halo eect surrounding the cell produces the spurious edge. In this case, the nal result of using basic morphological operation, contains many false positives FP (see Table 4.4).
The pixel is: The pixel is classi ed The pixel is classi ed as no ground-truth as ground-truth
not ground-truth TN FP
ground-truth FN TP
Table 4.4: Confusion matrix
The better result seem to be obtained using Canny detection algorithm. Canny algorithm is the multi-step edge detection operator, which follows:
1 removing noise by convoluting the orginal image with average subtraction; 2 nding the image gray derivatives;
4 calculating the threshold value;
However, the threshold cannot be used in the phase-contrast segmentation be-cause some eects stemming from the phase-contrast microscopy be-cause back-ground and the cell to have a similar gray level.
The approach, which is applied in the Circadian Gene Expression (CGE), seems to overcome the threshold problem by using the active rays and active contour. The drawback of this approach is that, the cell's centroid has to be marked af-ter the removing image noise and before the cell's contour detection. Thus, the method is time-consuming because the cell's centroid has to be marked manually. For this reason the CGE is a semi-automatic cell detection algorithm. On the other side, the CGE is time-consuming but the accurate cell extraction is more e cient than cell extraction using basic morphological operations or using Canny detection algorithm.
Experiment 2
The objective of this experiment is to evaluate the results obtained from Particle Image Velocimetry (PIV) and Lucas Kanade algorithms. What is more, the aim is also to indicate the similarities and dierences between the algorithms results. In other words, the PIV and Lucas Kanade algorithms are applied to track the cells between the images sequence.
5.1 Planning the experiment
The algorithms result are two trajectories TA (trajectory comes from PIV
algo-rithm) and TB (trajectory comes from Lucas Kanade algorithm). Primary eect
under study is the algorithms eectiveness [16]. The eectiveness of the tracking algorithms is based on the accuracy. The accuracy describes the better approx-imation of the exact result (the trajectory obtained manually and denoted as T0). Therefore metrics, which evaluates the algorithms result (the trajectories),
identi es how similar or how dierent the trajectories are. Moreover, during the experiment execution the impact of two dierent techniques on the eects of the tracking cells in the biological images can to verify whether the trajectories ob-tained from PIV and Lucas Kanade match the exact trajectory.
Due to lack of random participants assigned to the treatment or control group, the quasi-experiment is selected [35]. In order to formalize the experiment, the following hypotheses are formulated.
H0: PIV and Lucas Kanade algorithms are eective
HA: PIV and Lucas Kanade algorithms are not eective
It is expected that the algorithms behave similarly. However, the nature of each algorithm is dierent, eg. fundamental PIV application is in mechanical uid, therefore, its result can be dierent than the result from the Lucas Kanade al-gorithm. The alternative hypothesis can be stated, when trajectory coming from
the PIV algorithm does not cover trajectory from the Lucas Kanade algorithm. In order to enhance the experiment evaluation, additional statistical method is proposed to the experiment analysis (see Chapter 6). If the cell displacement is expressed as a time series, the exact solution (exact displacement between cell in time t) comes from a distribution, eg. α stable distribution. Therefore, the algorithm result should come from the same distribution as the exact solution. The de ned null hypothesis H0 against alternative hypothesis HA are tested
us-ing independent and dependent variables. Independent variables are a sequence of segmented images. The dependent variables are results from the algorithm execution, therefore, there are images, which contain cell movement trajectories. In order to properly analyze the experiment result, the performance evaluation of cell motion tracking has been divided into two parts. The rst part assessed the manually marked data, the other one evaluated the tracked data as a result of the Lucas Kanade and PIV algorithms.
Subject of the experiment is the similarities between cell trajectory obtained as an algorithm results and by the expert.
Sampling techniques is not directly used by the author of this thesis. Sequence of phase-contrast images were taken by the experts in The Swiss Federal Insti-tute of Technology Zurich. The author of this thesis received already a proper prepared sample of the images.
Independent variable is a sequence of phase-contrast images. However, De-pendent variable is a sequence of images there are images, which contain cell movement trajectories.
Measurement criteria used in this Experiment are described in Chapter 3.1 in Section entitled Performance evaluation of cell motion tracking.
5.2 Conducting the experiment
Firstly, the selected tools have been installed on Windows 7 Home Premium Op-eration System (64 bits) with the procesor Intel(R) Core(TM) i5 CPU 2.53GHz. The tools instalation required additional programs such as Fiji or ImageJ and PIVlab1_31 tool [5]. Moreover, not all algorithms were available in that tools, therefore some additional scripts had to be implemented.
Step 1 To select proper cell on each image and remove background; Step 2 To calculate centroid of the selected cell from step 1;
Step 3 To use PIV algorithm implemented in PIVlab to analyze motion of the selected cell from Step 1;
Step 4 To save the results obtained from Step 2 and 3 in .txt le and then plot on the same gure results obtained from Step 2 and step 3;
In order to use Lucas-Kanade algorithm, which is implemented in CellTrack to analyze cell motion, the following sequence has been covered:
Step 1 To select proper cell in rst image of the images sequence; Step 2 To use Lucas-Kanase algorithm;
Step 3 To save the results obtained from Step 2 and then plot the results in the same gure, what the results from PIV algorithm and centroids are; The context of this experiment will be characterized according to toy vs. real problem and speci c vs. general. This means that the problem in the exper-iment is addressed to a problem of toy size in order to make the studies valid to the general software engineering domain [35]. In general, trajectories coming from PIV and Lucas Kanade are evaluated. However, in particular, the dier-ence between estimated trajectories TAand TBand exact solution T0 is compared.
Figure 5.1 presents the example of exact (manual) cell identi cation denoted as X0, X1, X2 (blue points) and estimated cell identi cation (red points) denoted as
Y0, Y1, Y2. From the experiment point of view, the most important is the distance
between exact and estimated solution, which means distance between X0 and Y0,
X1 and Y1 and X2 and Y2.
The variables have been manipulated and controlled [35] in order to build a body of knowledge [6].
In order to execute the experiment, the considered medical images have been prepared to a further analysis during a pre-processing. Size of the medical image (see Figure 5.3) is equal to: 1344 x 1024. Although this image presents strong variation in contrast, the beads from the image can be clearly distinguished in semi-automatic way. In order to avoid some errors in PIV calculations, the back-ground subtraction is applied to the considered image, so that a static backback-ground is obtained.
5.2.1 Manually marked data preparation and observation
Figure 5.1: Distance calculations
selected in Figure 5.3. However, the other type is a cell set, which moves quickly and the sample is showed in Figure 5.4.
At the beginning two independent trajectories, T1 and T2, of the manually marked
Figure 5.2: Distance between two independent trajectories
5.2.2 The tracked data preparation and observation
Lucas Kanade algorithm
The Lucas Kanade algorithm uses dierential method for an optical ow esti-mation. The algorithm is available in the CellTrack application. The results obtained from Lucas Kanade depends on three coe cients such as pyramid level, window size and termination criteria. From the thesis point of view, the most important seems to be pyramid level. Table 5.4 presents dependencies between piramid levels, which is accuracy of trajectory obtained from the Lucas Kanade algorithm. Firstly, the cell in only one image has been selected and then the Lucas Kanade algorithm has been performed for the whole images sequence. PIV algorithm
At the beginning, high distance variation is caused by the fact, that cell position has been obtained by clicking any point belonging to the cell. However, later the cell position has been approximated by clicking the center of the cell. Then, the distances are quite similar and the variation is small. Mean, minimum and maximum of the results presented in Figure 5.2 are represented in Table 5.1, 5.2 and 5.3.
Metric Mean Min Max d(T1, T2) 4.2592 0 27.8927
Table 5.1: Metric for all results presented in Figure 5.2
Metric Mean Min Max d(T1, T2) 17.0992 5 27.8927
Table 5.2: Metric for the rst ten results presented in Figure 5.2
Metric Mean Min Max d(T1, T2) 1.6912 0 3.6056
Table 5.3: Metric for the last fty results presented in Figure 5.2
Cell, which moves slowly Cell, which moves quickly Piramid Level (PL) PL 1 PL 2 PL 3 PL 1 PL 2 PL 3 Track Fragmentation 0 0 0 0 1 3 mean(d(TB, T0)) 4.1887 4.1945 4.2082 574.7709 554.3287 563.6714 min(d(TB, T0)) 1.4142 1.4142 1.4142 3.6056 3.6056 3.6056 max(d(TB, T0)) 7.6158 7.6158 7.6158 982.2932 916.1577 1203.8
Table 5.4: Comparison of a cell with slow motion vs. cell with quick motion
Cell, which moves slowly Cell, which moves quickly Interrogation Window Size 64x32x16 128x32x16 64x32x16 128x32x16
Track Fragmentation 0 0 1 4
Figure 5.3: Marked cell moves slowly (the image is taken in 0 minut)
Figure 5.5: Results obtained from PIV algorithm
Figure 5.6: Paths calculated by PIV algorithm, Lucas Kanade Method and manually
5.3 Analysing the experiment results
The experiment consists of two parts. First part evaluates manually marked data and the other part examines tracked data by two dierent algorithms for motion tracking of a single particle (cell). One algorithm, namely Lucas Kanade, tracks vertices of cell boundaries including intra-cellural feature points. However, the other one estimates the velocity vector eld of the change in position of the cell using cross-correlation.
Manually marked data rstly distinguishes two kinds of cells - cells, which move slowly, and cells, which move quickly. Analysing results showed in Figure 5.2, is observed, that rst ten distances present quite high variation. Other observations are relatively similar, which means, other observations present small dierentia-tions. The variation between distances is caused by the fact, that at the beginning ( rst ten observations) cell position obtained by clicking any point belonging to the cell. However, later the cell position has been approximated by clicking the center of the cell. According to mean, minimum (min) and maximum of results presented in Table 5.1, 5.2 and 5.3, can be conclude that the best result are obtained in metric for the last fty results presented in Figure 5.2 because the maximum value is the smallest one.
Therefore, in order to achieve quantitative manually marked data of cell position, one characteristic point, such as centroid, should be approximated.
The tracked data trajectories consists of trajectories obtained by the PIV algo-rithm, TA, and trajectories obtained by Lucas Kanade, TB.
According to analysis results presented in Table 5.4 and 5.5, both algorithms do not seem to guarantee, that appropriate points locations will be obtained for cells, which move quickly. Therefore, the results from Lucas-Kanade optical ow algorithm and the PIV algorithm can be inconsistent, but only for cell with high velocity. Results obtained for cell with slow velocity cover the manually marked data.
Starting the experiment, two hypothesis have been stated. However, experiment conduction reveals that the hypotheses did not take into account the dierences between cells velocities. Therefore, the analysis using the stated hypotheses di-vided into two kinds of cell such as:
1: For cell, which moves slowly, there is enough evidence to reject the alternative hypothesis. It means, that each algorithm, PIV and Lucas Kanade, is eective.
eective.
5.4 Discussion
The experiment deals with non-trivial performance evaluation problem of cell mo-tion tracking obtained by two dierent algorithms. The rst algorithm, namely PIV, calculates the uid velocity vector eld, however, the other one, called Lu-cas Kanade, is an algorithm applied to optical ow estimation. The basic PIV assumption is the short time-lapse interval and the high density image [9]. How-ever, the Lucas Kanade algorithm assumes that the projection of the same point is the same in every frame, the object does not move quickly and the point move-ment is equalized to its neighbors. Therefore, common part of both algorithms assumes that the cell moves slowly. Nevertheless, the PIV takes a step forward and uses a fact that if the cell moves slowly, the dispacement between the cell location in time t and t+δt is small. It can be achieved by taking images at a speci c slow interval, which means, with small δ. However, if time-lapse interval is not short, such as time-lapse interval of the considered images sequence, the algorithm result can produce some inconsistency. If the cell is far displaced in δt, the algorithm can loose to track the proper cell. This problem, namely drift problem, is visible in Figure 5.6 by the Lucas-Kanade algorithm and appears classically in the analysis of long sequences. Therefore, drift problem is one of the drawback in the Lucas-Kanade algorithm. However, the biggest disadvantage of the PIV algortihm is the fact that, the algorithm cannot track more than one cell during the sequence images. Moreover, in order to fully characterize the cell movement, including rotation, correlation, divergence, adhesion, the fully charac-terized velocity eld of cell displacement is required. It means, that, as much as possible, points belonging to the cell should be indicated and then tracked among the sequence images. It is quite di cult, because tracking many parallel points is a time-consuming operation. In order to decrease the time algorithm execution, only the cell centroids should be taken into consideration.
Considered medical images are not poor PIV images. Moreover, the images present low density. In this case, actually using disadvantage of the PIV algo-rithm, tracking of a single cell could be obtained because single cell was treated as one object which PIV algorithm had to track.
PIV is based on correlation between two consecutive images. The disadvantage of the technique is that the size of interrogation window has to be found exper-imentally. As the calculations showed, if the interrogation window is too small, sometimes "empty places" can be obtained.
order to calculate the structure of the ow, some additional operations have to be applied. Generally, Proper Orthogonal Decomposition [25] is applied to the PIV algorithm in order to present the trajectory structure. Nevertheless, the method is not useful, if the single cell is tracked as the value of each vector is similar and the most valuable vectors (modes) cannot be found.
In order to present the trajectory structure, the centroid was calculated. The cen-troid results are almost the same as a manual solution. Moreover, each cencen-troid lies in the vector elds obtained by the PIV algorithm and it can be assumed that centroid indicates the structure (trajectory) of the cell movement.
Performance evaluation of cell motion
using time series
This chapter presents a novel methodology for tracking performance evaluation using converting image into time series. The aim of the method is to examine how sensitive the tracking algorithms is to noise.
6.1 Theory
Time series is a sequence of data points. Each datum is measured at a speci c time t. Actually, time series is a set of observations, which is denoted as {Xt}.
There are two types of time series:
1. A discrete-time time series - in this case a set of time t is a discrete set. Discrete set S is discrete in a larger topological space X if every point x ∈ S has a neighborhood U such that S ∩ U = x. The points of S are then said to be isolated. Typically, a discrete set is either nite or countably in nite. 2. A continuously-time time series - in this case set of time t is a continuously
set.
α stable distribution - theory
α stable distribution can be described by the following property:
Sum of independent alpha stable random variables is a independent alpha stable random variable [34].
Characteristic function of alpha stable distribution has below form: α has been called as stability index.
1. If α = 2, then X has gaussian distribution
2. If 0 < α < 2, then X has distribution with heavy tail If X has α stable distribution, then:
1. E|X|p < ∞for 0 < p < α
2. E|X|p = ∞for α < p < 2
β was called as skewness parameter. 1. if β > 0 then X has right skewed tail 2. if β < 0 then X has left skewed tail
3. if β = 0 and µ = 0 then X has symmetric alpha stable distribution
σwas called as scale parameter. Standard deviation in Gaussian distribution and scale parameter in alpha stable distribution has similar role.
µis in charge of shift and for α > 1 is a expectation.
In order to illustrate the α - stable densities, some example have been created.
Figure 6.2: Densities of skewed α - stable distribution for many dierent coe cients
