Contributions to 3D Image Analysis using Discrete Methods and Fuzzy Techniques: With Focus on Images from Cryo-Electron Tomography

Full text

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8) .

(9)

(10)

(11)

(12)

(13)

(14) !

(15)

(16)

(17)

(18)

(19)

(20)

(21) "#$%"&.

(22)

(23) . %%#'()'('* %+#,-,'))*(-* . ...

(24) /''),.

(25) .

(26)

(27)

(28) ! "#"## #$%&

(29)

(30) &

(31)

(32)

(33) &

(34)

(35) '

(36) (

(37) ) ' * !'"##'+

(38)

(39)

(40) ,- . !

(41) /00 1'2 /

(42)

(43) - &

(44) +

(45) )

(46)

(47)

(48) '. . '

(49)

(50)

(51)

(52)

(53)

(54)

(55) 3"3'3#' '-4567387%%933:89' 2

(56) & ( 1 (

(57)

(58)

(59) ;

(60) & ( ( (

(61)

(62) '

(63)

(64) & 1

(65)

(66)

(67) ;

(68)

(69) < '-

(70) .

(71)

(72) &

(73) & .

(74) 1 . 1 . '-

(75) (

(76) &

(77)

(78) &

(79) (

(80)

(81) =

(82) 0

(83) &

(84) ; >

(85) =,> (

(86) (

(87)

(88)

(89) =46?>

(90) &

(91)

(92)

(93)

(94)

(95) =

(96) ) > &

(97)

(98)

(99) =/!>' &

(100)

(101)

(102)

(103) &

(104) (

(105) <

(106)

(107)

(108) &

(109)

(110) .

(111) & . ' !

(112) (

(113)

(114)

(115)

(116)

(117)

(118) &

(119)

(120) ) '

(121) (

(122) -

(123)

(124) * =-*>

(125) !)

(126) '

(127) (

(128)

(129)

(130) (46? 1 &

(131)

(132) (

(133) &00 .

(134)

(135) &

(136)

(137) '

(138) .

(139)

(140) (

(141)

(142)

(143) & .

(144)

(145)

(146)

(147) & &00 .

(148)

(149)

(150) '

(151)

(152) & !)

(153) (

(154) .

(155)

(156) & <'

(157) & .

(158) &

(159)

(160) &

(161) &

(162) /!

(163) & (

(164) '5

(165)

(166) .

(167)

(168) &

(169) .

(170)

(171) (

(172) <

(173)

(174) (

(175)

(176) ' * ( &

(177)

(178)

(179) & .

(180) (

(181) &

(182)

(183) ( &

(184)

(185) (. (.

(186) & '@ (

(187)

(188)

(189) .

(190) ( ( ( (

(191)

(192) '

(193) (

(194)

(195)

(196) (

(197)

(198) .

(199) ( & .

(200) & &

(201)

(202)

(203) ) /!'

(204) ,&00

(205) (

(206)

(207) (

(208)

(209)

(210) &

(211)

(212)

(213) ! " #

(214) $

(215)

(216)

(217)

(218) #

(219) %$ & #'

(220) ( ))*# #+*,-., # A! * "## -446:%:"9 -4567387%%933:89 $ $$$ "%37=. $BB '<'B

(221) C D $ $$$ "%37>.

(222) List of Papers. This Thesis is based on the following papers, which are referred to in the text by their Roman numerals.. I. II. III. IV. V. VI. VII. VIII. Gedda, M. (2005) Clustering of Objects in 3D Electron Tomography Reconstructions of Protein Solutions Based on Shape Measurements. In Proceedings of the 3rd International Conference on Advances in Pattern Recognition (ICAPR 2005), Lecture Notes in Computer Science, 3687(2):377–383. Gedda, M., Svensson, S. (2006) Separation of Blob-Like Structures using Fuzzy Distance Based Hierarchical Clustering. In Proceedings of the Swedish Symposium on Image Analysis (SSBA 2006), 73–76. Gedda, M., Svensson, S. (2006) Fuzzy Distance Based Hierarchical Clustering Calculated using the A* Algorithm. In Proceedings of the 11th International Workshop on Combinatorial Image Analysis (IWCIA 2006), Lecture Notes in Computer Science, 4040:101–115. Fouard, C., Gedda, M. (2006) An Objective Comparison between Gray Weighted Distance Transforms and Distance Transforms on Curved Spaces. In Proceedings of the 13th International Conference on Discrete Geometry for Computer Imagery (DGCI 2006), Lecture Notes in Computer Science, 4242:259–270. Gedda, M., Svensson, S. (2007) Flexibility Description of the MET Protein Stalk Based on the Use of Non-Uniform B-Splines. In Proceedings of the 12th International Conference on Computer Analysis of Images and Patterns (CAIP 2007), Lecture Notes in Computer Science, 4673:173–180. Gedda, M., Vallotton, P. (2010) Three-Dimensional Tracing of Neurites in Fluorescence Microscopy Images Using Local Path-Finding. In Proceedings of the 35th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2010), IEEE. Gedda, M. (2010) Heuristics for Grey-Weighted Distance Computations. In Proceedings of the Swedish Symposium on Image Analysis (SSBA 2010), 43–46. Gedda, M., Öfverstedt, L.-G., Skoglund, U., Svensson, S. Image Processing System for Localising Macromolecules in Cryo-Electron Tomography. Submitted for publication..

(223) IX Gedda, M. Algorithms for Grey-Weighted Distance Computations. Submitted for publication. Reprints were made with permission from the publishers..

(224) Related Work. In the process of performing the research leading to this Thesis, the author has contributed also to the following publications, significantly so for Paper 2, and minor parts of Papers 1 and 3. 1. Sintorn, I-M., Gedda, M., Mata, S., Svensson, S. (2005) Medial GreyLevel Based Representation for Proteins in Volume Images. In Proceedings of the 2nd Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA 2005), Lecture Notes in Computer Science, 3523(2):421–428. 2. Svensson, S., Gedda, M., Fanelli, D., Skoglund, U., Sandin, S., Öfverstedt, L-G. (2006) Using a Fuzzy Framework for Delineation and Decomposition of Immunoglobulin G in Cryo Electron Tomography Images. In Proceedings of the 18th International Conference on Pattern Recognition (ICPR 2006) 4:687–690. 3. Bongini, L., Fanelli, D., Svensson, S., Gedda, M., Piazza, F., Skoglund, U. (2007) Resolving the Geometry of Biomolecules as Seen by Cryo Electron Tomography. Journal of Microscopy, 228(2):174–184..

(225)

(226) Contents. 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1 Molecular analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1.1 Cryo-electron tomography . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1.2 Biological specimen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2 Nerve cell analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2.1 Fluorescence microscopy . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2.2 Biological specimen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3 Image analysis concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.1 Image acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.2 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.1.3 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.1.4 Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.1.5 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2 Topics closely related to contributions in this Thesis . . . . . . . . . 27 3.2.1 Distance measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2.2 Fuzzy set theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2.3 Grey-weighted distances . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.2.4 Analysis of linear features . . . . . . . . . . . . . . . . . . . . . . . . 35 3.3 Image analysis in electron tomography . . . . . . . . . . . . . . . . . . 36 3.4 Image analysis of neurites . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.1 Grey-level–based methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.1.1 Decomposing objects with blob-like sub-domains . . . . . . 39 4.1.2 Efficient algorithm implementation . . . . . . . . . . . . . . . . . 41 4.1.3 Impact of heuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.1.4 Grey-level–based distance comparison . . . . . . . . . . . . . . . 44 4.1.5 Algorithms for grey-weighted distance computations . . . . 46 4.2 Macromolecular analysis in cryo-ET . . . . . . . . . . . . . . . . . . . . 47 4.2.1 Localisation of macromolecules in vitro . . . . . . . . . . . . . . 47 4.2.2 Flexibility representation of the MET protein . . . . . . . . . . 49 4.2.3 Classification of macromolecules . . . . . . . . . . . . . . . . . . . 51.

(227) 4.3 Tracing neurites using local path-finding . . . . . . . . . . . . . . . . . 5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Summary in Swedish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52 55 57 59 63 65.

(228) 1. Introduction. Most research in cell and molecular biology aims at increasing knowledge of the low-level bodily functions to beat diseases and enhance the quality of life. An essential step in figuring out these low-level functions is by visually inspecting various processes of objects involved. To draw any conclusions of biological behaviour from these inspections, statistical studies are needed that can either confirm or reject the theories involved. As image acquisition has become increasingly digital, it is now easy to generate huge amounts of data for analysis. However, manual analysis of large datasets is a tedious task, resulting in observer fatigue and introduces human bias which is hard to characterise. It is also subjective and highly dependent on observer experience and knowledge. The increase in computer power and subsequent development of image analysis techniques has greatly helped to introduce objectivity and coherence in measurements and decision making. Digital image-based analysis can provide structural information, objectiveness and accuracy. It has become an essential tool for facilitating both large-scale quantitative studies and qualitative research. This is vital in the study of cells and macromolecular structures to understand their biological role. This Thesis presents the results of work related to method development and analysis of low-resolution three-dimensional (3D) images applied to electron tomography images and fluorescence microscopy images.. 1.1. Motivation. Although image analysis has surfaced as a vital tool in cell and molecular biology research, there is still a lack of automated methods for analysing 3D images with low resolution and high degree of noise. Many of the sophisticated analysis methods developed for two-dimensional (2D) images become complex and time-consuming when generalised to three dimensions, limiting the possibilities of large-scale studies. This issue is apparent in both electron tomography, where existing methods often reduce dimensionality by limiting the analysis to symmetric objects, and neurite research, where global methods like skeletonisation are time-consuming and do not scale well with image size. One essential principle when dealing with this issue in settings with low contrast and low signal-to-noise ratio (SNR) is to emphasise the use of grey-level– based representations. This is to avoid the risk of loosing important properties. 9.

(229) when dealing with crisp (binary) representations. Another important principle is to always incorporate any a priori shape information available to guide the analysis.. 1.2. Objectives. The main objective of the work presented in this Thesis is to develop image analysis methods of low complexity, that take into account both grey-level and shape information, to facilitate large-scale studies of 3D images with low resolution and low SNR. To enable the development of such analysis methods, there is also a need to examine both the theoretical and practical properties of the steps involved. To fulfil the main objective, the following goals need to be reached: • Develop methods for locating objects of interest in noisy low-resolution 3D images. • Address the issues of segmenting complex macromolecules in noisy lowresolution 3D images. • Develop methods for analysing structural properties of macromolecules and nerve cells. • Develop local approaches to global problems to keep the time complexity low and scale well with image size. • Examine theoretical and implementational details of computationally heavy approaches to keep time complexity low.. The emphasis in this Thesis is on the development and implementation of methods. In fact, we did not pursue any involvement in image acquisition, which is also an important aspect for successful results. However, large parts of the work involve strong collaborations with the Cell and Molecular Department at the Karolinska Institutet in Stockholm, Sweden, where they focus on electron tomography to gather information for solving problems in structural biology.. 1.3. Thesis outline. Chapter 2 outlines the background to the application areas we have in mind. Chapter 3 covers basic image analysis concepts as well as previous work related to the new work presented in this Thesis. Concepts closely related to the contributions are covered in higher detail. Chapter 4 describes the contributions of the included Papers. Chapter 5 summarises the Thesis, and in 10.

(230) Chapter 6, future work is discussed together with thoughts on where the field is heading.. 11.

(231)

(232) 2. Background. The main focus of the work in this Thesis is analysis of images from electron tomography and, to a lesser degree, from fluorescence microscopy. In this Chapter, a brief background on these imaging techniques and the biological specimen used is provided.. 2.1 2.1.1. Molecular analysis Cryo-electron tomography. Transmission electron microscopy (TEM) allows for the direct imaging of macromolecules in solution as well as in a frozen hydrated state, often referred to as cryo-electron microscopy (cryo-EM) [1]. The transmission electron microscope (see Figure 2.1 (a)) measures the scattering from the electron beam and produces a 2D projection (micrograph) depicting the electrostatic potential of the biological specimen. Multiple views can be collected by tilting the specimen. A tilt series is acquired by repeated recording and tilting the specimen in fixed angular steps within a tilt range (usually -60◦ to +60◦ ). Cryo-electron tomography (cryo-ET) [64] is an individual particle reconstruction technique which allows 3D imaging of an individual object, not to be confused with single particle reconstruction which does not provide the reconstruction of one single particle. Instead, it provides an average of many thousands of macromolecules that are rigid enough to behave as one single particle, with higher resolution as a result. The electron tomographic reconstruction in cryo-ET is carried out using the principle of filtered back-projection (see the work by Klug [36] for a review). The back-projection at a point in space is defined as the sum of all the projected values along all lines through that point. Hence, simply carrying out back-projection of an object gives us a smeared version of the object. Figure 2.2 shows how two homogeneous spherical objects are projected from three directions. Figure 2.3 illustrates how the projections are back-projected, giving smeared version of the objects, and Figure 2.4 shows how the representation improves with increasing number of projections. The back-projection at a point outside the objects is not necessarily zero, since there will be directions along which one has non-zero projections that contribute to the summation. This problem is addressed by applying a convolution (filtering) oper-. 13.

(233) .

(234) .

(235) . . (a). (b). Figure 2.1: (a) Transmission electron microscope. At the top sits an emission source, the electron gun. The condenser is an electromagnetic lens which allow for the formation of the electron beam to the desired size and location. The beam passes through the specimen, which can be set at any angle within the tilt range. The transmitted beam is focused by the objective lens into an image, which is enlarged by the projector lenses. The beam strikes a fluorescent screen, allowing the user to see the image, before it hits the detector. (b) Fluorescence microscope (image courtesy of Wikimedia Commons). The excitation filter filters out excitatory light, which is reflected by the dichroic mirror onto the specimen. The fluorescence in the specimen gives rise to emitted light, which is focused by the objective lens. The emission filter filters out the excitation light and lets the fluorescent light through, focused by the ocular, to the detector.. ation, thereof the filtering part of the algorithm name. This technique allows reconstructing an object from its tomographic data. There is a physical limit to the tilt angles that can be reached in a transmission electron microscope. The limited angular range decreases the resolution in z-direction and is known as the missing wedge or missing data problem [39] (see Figure 2.5). The three-dimensional reconstruction from a tilt series of micrographs is referred to as the density function/map/image, since it represents the density distribution of the biological specimen. Electron tomography (ET) is often seen as the most promising imaging technique for study of large macromolecular complexes within the cellular context [56], but can also be successfully used to access the structure of individual bio-molecules in solution. However, a drawback in analysis of macromolecules by cryo-ET, apart from the missing wedge problem, is that it suffers 14.

(236) Figure 2.2: Projections from -45◦ , 0◦ , and 45◦ .. Figure 2.3: Reconstruction from projections in Figure 2.2.. from low resolution and low signal-to-noise ratio (SNR). The biological material is very sensitive to the electron beam and is rapidly damaged by high electron doses, limiting the number of images that can be recorded from the specimen. The experimental conditions of low dose, low contrast, and few projections, give a low SNR that produces large errors in the reconstruction. The quality of the reconstruction can be increased by using, for example, the iterated regularisation method COnstrained Maximum Entropy Tomography (COMET) [52, 61]. The method aims to produce the most featureless recon15.

(237) (a). (b). (c). (d). Figure 2.4: (a) Two spheres back-projected using (b) 7 projections, (c) 13 projections, and (d) 121 projections respectively. The tilt angle is -60◦ to 60◦ .. (a). (b). Figure 2.5: The missing wedge problem illustrated with back-projections of two spheres. (a) Ideal reconstruction case with information from all angles (tilt angle 90◦ to 90◦ ). (b) Real world case with information missing (tilt angle -60◦ to 60◦ ).. struction that fits the projection data. In particular, the COMET procedure will minimise the detrimental effects of errors in the measured data. Despite all the problems mentioned above, by using ET, it is possible to examine proteins and other macromolecules in three dimensions at a resolution of a few nanometres [57].. 2.1.2 Biological specimen The Immunoglobulin G (IgG) antibody is a glycoprotein with a molecular mass of 150 kDa, which binds to foreign agents, such as viruses, by subdomains named fragment antigen-binding arms (Fab arms). Hinges connect two Fab arms to a stem (Fc stem). The antibody is believed to be a highly flexible structure, being characterised by a wide range of variability of FabFab and Fab-Fc angles [30, 51]. The met proto-oncogene (hepatocyte growth factor receptor) (MET) protein controls growth, invasion, and metastasis in cancer cells. Activating MET mutations is a predisposition to developing cancer. See the work by Birchmeier et al. [6] or Lai et al. [38] for reviews of the MET protein. In its mature form, MET consists of a β -propeller structure and stalk, named from its stalk-like structure. Recent structural studies indicate that MET is a flexible 16.

(238) structure and that this flexibility is affected by the binding to its ligand, hepatocyte growth factor/scatter factor (HGF/SF) [26]. The protein has recently been used in structural studies [26, 57]. Cryo-ET reconstructions of IgG antibodies are used for the work in Papers II, IV, and VIII. Reconstructions of MET proteins are used for the work in Papers V, and VIII.. 2.2 Nerve cell analysis 2.2.1 Fluorescence microscopy In regular absorption light microscopy, the observed light originates from a light source, and the specimen is seen as darker areas where the light is attenuated. Fluorescence microscopy (FM) differs from normal light microscopy mainly due to the fact that the light registered by the camera originates from fluorochromes in the specimen. Light from a light-source hits the specimen, and the fluorochromes in the specimen emit fluorescence, which is detected by the camera (see Figure 2.1 (b)). A dichroic mirror reflecting high energy and transmitting low energy light together with band-pass emission and excitation filters control what fluorochromes are excited and what emission spectra are imaged. Fluorescence microscopy images are often blurred by light leaking from out-of-focus fluorescence. Confocal microscopy [13] is an optical method for enhancing the image quality where the fluorochromes in the specimen are excited by a well-focused laser and light from out-of-focus planes is blocked by a pinhole. Deconvolution microscopy [2] is a computational method for enhancing image resolution. The light contribution from out-of-focus planes is estimated and removed from the focus plane. When a 3D object is imaged, a series of well-focused 2D images can be acquired and stacked to form a 3D image of the specimen.. 2.2.2 Biological specimen In physiological neuroscience, it is common to use nerve cells (neurons) taken from the brains of rats cultivated from the Wistar strain (Wistar rats). A study on neurons has been performed to examine the morphological and electrophysiological properties of such neurons [20]. The samples consist of thin slices from the lateral nuclei of the amygdala, which have been implicated with a variety of functions including memory and attention. In the morphological analysis, the neurons are stained with a fluorescent dye and imaged in vitro (in an artificial environment outside the living organism) through fluorescence microscopy. Image data from the study is used for the work in Paper VI.. 17.

(239)

(240) 3. Image analysis concepts. This Chapter covers some basic image analysis concepts and gives a more detailed description of some topics more closely related to the contributions in this Thesis.. 3.1. Basic concepts. Digital image analysis is the extraction and analysis of information from digital images. It is any form of signal processing for which the input signal is a digital image. Due to the specific characteristics of digital images, such as spatial correlation and object representation, compared to time signals in traditional signal processing, image analysis has become a distinct field with an abundance of methods (many of which are closely linked to digital geometry). Solving an image analysis problem can be approached in two ways. There is the continuous approach and the discrete approach. In the continuous approach, a mathematical model is created from information about the physical scene using, for example, differential equations. The discrete approach argues that the continuous model does not hold well for the discretised mappings of the scene and instead leans towards discrete mathematics for creating methods tailored to the specific circumstances of the imaging setup. The continuous approach is popular when the objects of interest in the imaged scene are easy to model and computation time generally is not an issue. The discrete approach is popular in analysis of low-resolution images, where discretisation is a large factor, and in real-time scenarios, where complex mathematical models are often too computationally costly. The work flow for solving an image analysis problem can be divided into five steps. Not all steps apply to all problems, but the work flow applies well to most cases. The steps are image acquisition, preprocessing, segmentation, representation, and classification. The steps can be generalised into low-level, intermediate-level, and high-level analysis. Figure 3.1 shows a flow chart illustrating the steps and levels.. 3.1.1 Image acquisition Acquiring digital images generally means capturing a physical scene onto a digital medium. A common example is the digital camera which captures in-. 19.

(241) coming light through a lens and onto a charge-coupled device (CCD). Each sensor on the CCD grid measures the amount of incoming light over a (short) period of time. The CCD samples the analogue rays by discretising their intensities to produce a digital representation of the scene, i.e., a digital image. This is referred to as digitising and each element in a two-dimensional (2D) image is generally referred to as a pixel (short for picture element) and in a three-dimensional (3D) image a voxel (short for volume picture element). For consistency, the name pixel is used for both 2D and 3D image elements throughout this Thesis. The intensity or brightness in each image element is generally represented by integer values corresponding to the amount of light captured by the sensor. The contrast is a measure of the difference in brightness between parts of the image. It is desirable to have high contrast images since low contrast can make it difficult to distinguish different parts. Figure 3.2 (a) shows a digital image of a physical scene. The magnification shows the individual pixels of the grid and Figure 3.2 (b) is the digital representation of the magnified region with intensities between 0 and 255. 3D images are often constructed by stacking 2D images on top of each other or, as in electron tomography (described in Chapter 2), mathematically reconstructed from 2D projections. In mathematical terms, a digital image may be regarded as a discrete integer-valued function f (x, c), where x = [x, y, z] are the spatial coordinates and c is the spectral (colour) component. Here, we mainly work with 3D grey-level images, which is reflected in the notation since there is only one colour component. See Figure 3.3 for the structure of a 3D image. Spatial resolution The spatial resolution determines how closely points can be resolved in an image. This depends on properties of the imaging system. The response of an imaging system to a point source is described by the point spread function (PSF). The degree of spreading (blurring) the point source by a focused optical system is a measure for the quality of the imaging system. A high resolution imaging system must thus contain an optical system with a narrow PSF and a large sensor array. Figure 3.4 shows a setup where a point source affects multiple sensors on the CCD array due to a wide PSF. Note that the sensor size defines the lower limit for the resolution, i.e., as long as the PSF is narrow enough for a point light source to only affect a single sensor, then the image resolution cannot be improved by narrowing the PSF. Constructing a setup with a narrow PSF and large sensor array is rather straightforward when it comes to imaging systems of low complexity, such as digital cameras. However, for other imaging techniques, the resolution can be difficult to determine and even vary over the image or between different images acquired by the same system. For TEM, the scattering of the electron beam depends on the local density in the sample, and any regularisation technique used in ET remodels the reconstructed data to fit some criterion. This alters the digital representation and makes the resolution impossible to deduce 20.

(242) Problem. Image acquisition. Segmentation. Classification High level. Preprocessing. Representation. Low level. Intermediate level. Result. Figure 3.1: Work flow in image analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . (a). (b). Figure 3.2: (a) Digital image of a zebra with the eye region magnified (courtesy of Berkeley Segmentation Dataset [43]). (b) Digital representation of the eye region.. Figure 3.3: Three-dimensional (3D) image.. 21.

(243) Figure 3.4: Point spread function (PSF). The ray, coming from a point source, gets spread (blurred) by the lens to cover an area of the CCD.. theoretically. Instead, the resolution has to be determined by first imaging objects of known sizes and then estimating the resolution. With the emergence of new imaging techniques, researchers are always eager to push the boundaries by examining objects either smaller or further away than what was previously possible. This means that low resolution will always be an issue in image analysis. Fuzzy theory (described further in Chapter 3.2.2) has emerged as a popular framework for dealing with low-resolution images.. 3.1.2 Preprocessing Preprocessing is usually performed to reduce noise or enhance edges. This is generally accomplished by spatial filtering where a filter is convolved with the input image. That is, a filter contains coefficients that are multiplied with the corresponding values in the input image and the result is a weighted sum of the elements covered by the filter mask: g(x) =. ∑. w(s) f (x + s),. s∈N. where w is the filter mask, f the input image, and g the output image (see Figure 3.5). A popular noise reduction filtering technique is the Gaussian smoothing. Gaussian smoothing uses a Gaussian filter to smooth the image. However, the Gaussian filtering reduces both noise and edge information. To reduce the noise without removing the edge information, an edge preserving technique is needed. A popular edge preserving technique is anisotropic diffusion [48] (see Figure 3.6) which uses scale-space, i.e., information at different scales 22.

(244) Image f (x). x. Filter w w(-1,-1) w(0,-1) w(1,-1). (x, y). w(-1, 0) w(0, 0). w(1, 0). w(-1, 1) w(0, 1). w(1, 1). y Figure 3.5: Spatial filtering with a 3×3 filter mask at position x = (x, y).. (resolution levels), to reduce image noise without removing significant parts of the image content.. Figure 3.6: Edge preserving filtering. Noisy image (top) filtered using Gaussian filtering (left) and anisotropic diffusion (right).. 23.

(245) 3.1.3 Segmentation Segmentation is the task of partitioning the image into separate regions. All regions representing something of interest in the image are referred to as the foreground and the remaining regions are referred to as the background (see Figure 3.7). Naturally, neither the foreground nor the background necessarily is one connected region. The foreground regions are often referred to as objects as each region generally represent some object of interest.. Figure 3.7: Segmentation example. The original image (left) has been segmented (right) into foreground (coloured regions) and background (black region). Image courtesy of MATLABTM (The MathWorks, Natick, MA, USA).. What makes up a connected region depends on what connectivity is used. For 2D images, we have 4-connectivity when only neighbours sharing an edge are considered adjacent pixels and 8-connectivity when both edge and vertex neighbours are considered adjacent pixels, see Figure 3.8. For 3D images, we have 6-, 18-, and 26-connectedness for face, face and edge, and face and edge and vertex neighbours respectively, see Figure 3.9.. (a). (b). Figure 3.8: Connectivity in 2D images. (a) Neighbouring pixels in 4-connected images. (b) Neighbouring pixels in 8-connected images.. When imaging a physical scene, it is important to adjust the environment, i.e., lighting, background, etc., to be as beneficial as possible for the segmentation step. Another important factor is the image quality. Images with low contrast and low signal-to-noise ratios (SNR) are most likely very difficult to segment. Therefore, it is important to ensure that the image acquisition system is calibrated and fine-tuned for optimal image quality for the particular scene. Segmentation is a classical image analysis problem and there are an abundance of approaches to it [28]. A common group of segmentation methods 24.

(246) (a). (b). (c). Figure 3.9: Connectivity in 3D images. (a) Neighbouring pixels in 6-connected images. (b) Neighbouring pixels for 18-connected images. (c) Neighbouring pixels for 26-connected images.. is histogram-based methods. A histogram is a graphical representation of the grey-level distribution in the image (i.e., the number of pixels of each grey level). The simplest of the histogram-based methods is thresholding, where all pixels with grey levels above some threshold is assigned to the foreground and the rest are assigned to the background, or vice versa. Another thresholding method is minimum-error thresholding [35] (evaluated in the work by Glasbey [27]), where the threshold value is determined automatically. It is based on the optimisation of a criterion function related to the average pixel classification error rate. The pixel classification makes the assumption that the intensities in the background and foreground regions are normally distributed. Another group of segmentation methods is template-based segmentation methods, where a pre-defined template is used to search for objects or objectlike features in the image (referred to as template matching). Another group is region-growing methods, where similar pixels are grouped into regions. Watershed segmentation (WS) [69, 70] is a common example in this group. In WS, the image is considered a landscape and water rises from the bottom of each valley, each with its own label. When the water from two valleys meet, a watershed is created to refrain the valleys from merging. The watersheds will then define the borders between the different regions (see Figure 3.10). In seeded watershed segmentation (also referred to as marker-based watershed segmentation), selective region-growing is enabled based on an initial seeding, allowing water to rise only from the pre-defined seeds instead of every minima, yielding much less over-segmentation (provided that the seeds are chosen appropriately).. 3.1.4 Representation To facilitate analysis of the segmented image regions (objects), a representation is generally constructed to describe the object. By representation we here refer to any digital representation which can be used to distinguish different objects from each other but contain much less information than the object itself. This can be, e.g., a set of features (descriptors), where each feature is 25.

(247) Figure 3.10: Watershed segmentation (WS). Intensity profile (left), WS (centre), and seeded WS with seeds indicated by arrows (right).. represented by a single value, or a model capturing the distinctive properties of the object. When using a set of features, the process of calculating the feature values is referred to as feature extraction. The features can be based on shape, such as width, height, perimeter, compactness, eccentricity, or various statistical moments. Other popular features are based on spectral (intensity or colour) or textural properties. All descriptors x j ( j = 1 . . . n), where each x j measures some feature of an object i, are put in a feature vector constituting the representation of the object, vi = [x1 , x2 , . . . , xn ]T . The feature vectors vi reside in an n-dimensional feature space (see Figure 3.11) where similar objects end up spatially close.. Figure 3.11: Feature extraction. The morphological features width and height are calculated for an object (left) and visualised in the feature space (right).. Representations other than feature vectors can be models, such as geometrical objects or polynomials, where the model parameters are used to describe the objects.. 3.1.5. Classification. Classification is used to determine the type of an object based on its representation. A class is a collection of objects that share some common pattern or properties. The classification methods can be categorised as either supervised 26.

(248) or unsupervised. In supervised classification, such as minimum error classification, some objects belonging to a known class are used for training the system and calculating decision boundaries between different classes in feature space. Any new objects are classified using these decision boundaries. In minimum error classification, any new objects are classified as belonging to the spatially closest class in feature space. In unsupervised classification (also referred to as clustering), all objects belonging to the same class are assumed to have similar properties and, thus, lie spatially close in feature space. The feature vectors are ordered into ’natural’ clusters representing the classes. The natural clusters are then compared with some reference data to identify the classes. Two common clustering techniques are k-means clustering [19], which is an iterative method converging on k clusters by minimising the variance within clusters and maximising the variance between clusters, and hierarchical clustering [19], which starts with one cluster for each object and merges clusters iteratively. Hierarchical clustering results in a dendrogram, which is a tree diagram illustrating the order in which the clusters merge. When the number of classes is not known, various cluster validation methods [29] can be used to determine the number of clusters which best partition the feature space.. 3.2 Topics closely related to contributions in this Thesis 3.2.1 Distance measures Distance calculations are widely used to extract shape and size information [11, 34]. In continuous (Euclidean) space Rn , the distance between two points is usually given by the Euclidean distance. For a point x = (x1 , x2 , . . . , xn ) and a point y = (y1 , y2 , . . . , yn ) the Euclidean distance is defined as the p-norm (∑ni=1 |xi − yi | p )1/p , where p = 2. The p-norm is often denoted by || · || p , e.g., the Euclidean distance between x and y can be written as ||x − y||2 . Digital images can often be considered a square lattice making up the discrete space Zn ⊂ Rn and the discrete distance is defined as the length of the shortest discrete path from p to q. A discrete path P of length l − 1 going from pixel p to pixel q is defined as an l-tuple (x1 , . . . , xl ) of pixels such that x1 = p, xl = q, and (xi , xi+1 ) defines adjacent pixels for all i = 1, . . . , l − 1. The distance d(p, q) represents the lowest sum of all distance weights wi = w(xi , xi+1 ) along all possible paths linking p to q, d(p, q) = { min (C (P)) | C (P) = P∈P pq. l−1. ∑ wi },. (3.1). i=1. where P pq is the set of all possible paths from p to q and the distance weights wi define the spatial distance between two neighbouring pixels. 27.

(249) The discrete distance between two pixels, thus, depends on both image connectivity and the choice of distance weights. Discrete distance is generally referred to as weighted distance due to the impact of the distance weights. Common weighted distance definitions are the city block/Manhattan distance (1-norm), chessboard/Chebyshev distance (∞-norm), 3-4-5 distance, or one of the definitions using the ’optimal’ distance weights designed to be rotation invariant over large distances [10, 12, 68]. A distance transform (DT) is the calculation of the shortest spatial distance from a set of pixels, generally referred to as seeds, markers or even features, to all other pixels in an image. The transform result is a distance map (often referred to as distance transform as short for distance transform result) where the value at each pixel is the shortest distance from the seeds (see Figure 3.12).. Figure 3.12: Distance transform. Input image (left), seeds (red, centre), and resulting distance map with iso-contours (right). Image courtesy of MATLABTM (The MathWorks, Natick, MA, USA).. The geodesic distance between two points included in a set is the length of the shortest path, or geodesic [59], linking these points and included in the set. The set is referred to as a geodesic mask. Algorithms Due to hardware limitations, the first methods for computing distance transforms were based on the classic raster scan approach [50] referred to as the chamfer algorithm. It uses a window containing a weight mask (chamfer mask) which is slided across the image updating the central pixel at each position. The scan consists of a forward pass and a backward pass. Figure 3.13 shows the masks used for the forward and backward passes for both 2D and 3D images. The set of chamfer weights {w1 , w2 , w3 } is determined by the distance definition used, e.g., the city block distance has chamfer weights {1, 2, 3} and chessboard distance has chamfer weights {1, 1, 1}. Later, graph-based region-growing techniques, such as the image foresting transform (IFT) [21] have become popular. They use a graph representation of the image, as shown in Figure 3.14. The chamfer weights are assigned to the graph arc weights ci j , and then some version of the theoretically optimal Dijkstra’s algorithm [17] is used to find the shortest paths in the graph. The 28.

(250) Dijkstra’s algorithm is optimal in the sense that it only needs to visit each node once. When calculating point-to-point distances, i.e., when only concerned about the distance between two points in the image, there is no need for a complete distance transform. The optimal choice is then the A algorithm [31] which uses heuristic information to guide the search. z. z x. y. w2 w1 w2 w1 X w3 w2 w3 w2 w1 w2 w3 w2 w3. x. z=0 y z = −1. w3 w2 w3 w2 w1 w2 w3 w2 w3 X w1 w2 w1 w2. z=1. z=0. (b) (a) Figure 3.13: The masks for calculating distance using the chamfer algorithm for 3D images. The pixel position is marked with an X and w1 , w2 , and w3 are the chamfer weights for face-, edge-, and vertex neighbours respectively. (a) The mask used in the forward pass. (b) The mask used in the backward pass. The chamfer masks for 2D images are the masks above for z = 0.. a b c d. a. cab. b. cad cbd. cac ccb. c. ccd. d. Figure 3.14: Graph representation of an image. Image (left) with four pixels labelled a, b, c, and d. The respective pixels in the graph representation (right) with arc weights ci j , where i, j ∈ {a, b, c, d}.. Heuristics When computing the distance between two points, heuristics can be utilised to guide the search. A heuristic uses available information to rapidly find a solution. When using graph-based techniques, a heuristic stores an estimate of the distance to the goal node at each position. Which node to expand next is decided based on both the distance travelled and the estimated distance left to the goal node. To guarantee that no unnecessary nodes are expanded, the estimate needs to be a lower bound on the true distance left. 29.

(251) For example, if we want to travel from point p to point q in Euclidean space Rn , the lower bound on the distance left from any intermediate point n is h = ||n − q||2 (i.e., a straight line from n to q). We also know that we have travelled the distance d = ||n − q||2 to get to our intermediate position n. We can obtain an estimate t of the total distance for travelling from p to q through n by just adding d and h, t = d + h. This means that at any intermediate position n on the path from p to q, there is no reason to travel in a direction that will increase the estimate t of the total distance. In this example, the heuristic is the estimate of the distance left from n to q. In Euclidean space, we always know the exact distance left to q and the heuristic is not an estimate but the true distance left. This means that when using a heuristic in Euclidean space, the search will never deviate from the straight line between p and q, since travelling in any other direction will increase the total distance t (see Figure 3.15).. Figure 3.15: The estimated total distance from p to q via n is the distance d travelled plus the estimated distance h to the goal. In Euclidean space, the estimate t will always be larger than ||p − q||2 for all positions n not on a straight line from p to q.. For weighted distances in Zn , which do not correspond to the Euclidean norm, using heuristics becomes a bit more complex. The limited directions, denoted by the chamfer vectors vi (e.g., in Z3 : v1 = [1, 0, 0], v2 = [1, 1, 0] and v3 = [1, 1, 1]), and their associated weights wi on the discrete path from p to q make it difficult to estimate the lower bound h on the distance left from any intermediate node n. A heuristic h which is determined to be a lower bound on the weighted distance from any node p to q can be constructed by taking the length of the unit step, wi wˆ i = , ||vi ||2 in the shortest direction, mini (wˆ i ), and multiply it with the Euclidean distance to the node q. See Figure 3.16 for an illustration of the unit step. In mathematical terms, wi h = min (3.2) · ||p − q||2 ≤ min {||p − q||} , i=1...n ||vi ||2 30.

(252) where n is the number of dimensions, wi are the distance weights, vi are the chamfer vectors, ||·||2 is the Euclidean norm, and ||·|| is the weighted distance using weights wi .. Figure 3.16: The unit steps vˆ i , wˆ i for 2D images.. 3.2.2 Fuzzy set theory Pixels generally do not have a true/false belongingness to an imaged object but can belong to the object to a certain degree, i.e., images are in nature fuzzy. This is apparent in medical images where it can be difficult to find distinct borders between organs and other substances in the body, or in electron tomography images (hampered by low contrast and low signal-to-noise ratios) where imaged objects often stick together with indeterminable border regions. Generally, discretisation of continuous objects introduce some level of uncertainty. Fuzzy set theory, introduced by Zadeh in 1965 [74], is a framework for handling this uncertainty. From fuzzy set theory, we have the following definitions: Let X be a reference set, then a fuzzy set A in X is defined as a set of ordered pairs A = {(x, μA (x)) |x ∈ X}, where μA : X → [0, 1] is the membership function of A in X. An n-dimensional fuzzy digital object O is a fuzzy subset defined on Zn , i.e., O = {(p, μO (p)) | p ∈ Zn }, where μO : Zn → [0, 1]. The fuzzy membership function for digital images can be any function mapping intensity values to the interval [0, 1]. One such example is the membership function used by Saha et al. [54, 55], GmO ,σO ( f (p)), if f (p) ≤ mO , μO = (3.3) 1, otherwise, 31.

(253) where f is the image intensity function, mO and σO are mean and standard deviation of intensity values within the object mask and GmO ,σO is an unnormalised Gaussian function with mO and σO as its mean and standard deviation parameters. In the case of a fuzzy image, a common generalisation is that the grey levels of the foreground pixels can be seen as the degree of belongingness of the pixels to the object. Such a description gives a better representation of objects than binary versions due to better handling loss of information unavoidable during the discretisation step [62] (see Figure 3.17). Fuzzy set theory can be applied to many image analysis concepts, e.g., fuzzy segmentation [14], fuzzy connectedness [49, 66], and fuzzy distance [55] (described further in Chapter 3.2.3).. Figure 3.17: Fuzzy representation (left) and a crisp representation of the same object (right).. 3.2.3 Grey-weighted distances Grey-weighted distances are distances defined so that the grey-level information in the image is taken into account. Grey-weighted distances can be considered a type of geodesic distance where the geodesic mask is the input image. The definition of grey-weighted distance resembles the definition of weighted distance (Equation (3.1)), but weighted distance determine the weights from spatial information while grey-weighted distance incorporate both spatial and membership/grey-level information. The grey-weighted distance cost ci = c(xi , xi+1 ) (also referred to as arc weight in analogy with weighted distances) is defined as the cost of travelling from a pixel xi to an adjacent pixel xi+1 on the geodesic mask. Computing the grey-weighted distance d(p, q) then consists of finding the path with the lowest sum of arc weights ci along all possible paths linking p to q, d(p, q) = { min (C (P)) | C (P) = P∈P pq. l−1. ∑ ci },. i=1. where ci is the cost function on the geodesic mask. In analogy with the distance transform (covered in Chapter 3.2.1), a grey-weighted distance trans32.

(254) form calculates the smallest grey-weighted distance from each pixel in the image to the closest pixel in a set of pixels. Rutovitz [53] first proposed a grey-weighted distance where the arc weight is equal to the grey level of the destination pixel of each step along the path. Levi and Montanari [40] extended this definition when defining a grey-weighted medial axis transform (GRAYMAT) by weighting the distance between adjacent pixels with their grey levels. In their definition, the length of a path is defined as a discretisation of the integral of the pixel values along the path, and the arc weight is defined as 1 c(xi , xi+1 ) = ( f (xi ) + f (xi+1 )) · ||xi − xi+1 ||, 2. (3.4). where || · || refers to the spatial distance between two adjacent nodes in the image graph, i.e., any choice of chamfer weights (see Chapter 3.2.1). Saha et al. [55] proposed a theoretical framework for distances on fuzzy sets inspired by Levi and Montanari’s definition. In a fuzzy framework, the fuzzy objects are used as geodesic masks and the distance is referred to as fuzzy distance, 1 c(xi , xi+1 ) = (μ (xi ) + μ (xi+1 )) · ||xi − xi+1 ||, 2. (3.5). where μ (·) is the fuzzy membership value. When using the fuzzy distance definition, the grey-weighted distance transform is generally referred to as the fuzzy distance transform (FDT). The FDT facilitates concepts such as skeletonisation and object thickness for fuzzy objects [54, 55]. Toivanen [65] proposed two definitions where the path between two points is defined as a path lying on the surface defined by the grey levels. The first is the distance on curved space (DOCS), c(xi , xi+1 ) = | f (xi ) − f (xi+1 )| + ||xi − xi+1 ||, and the second is the weighted distance on curved space (WDOCS), c(xi , xi+1 ) = | f (xi ) − f (xi+1 )|2 + ||xi − xi+1 ||2 .. (3.6). (3.7). While GRAYMAT propagates fast for low grey levels, DOCS and WDOCS account for the roughness of the ’height map’ and represent the minimal amount of ascents and descents to be travelled to reach a neighbouring pixel. Figure 3.18 shows a one-dimensional (1D) example of how the grey-weighted distance from a pixel p to a pixel q is calculated. For GRAYMAT, the distance is an approximation of the area under the grey-level function. For DOCS and WDOCS, the distance is equal to the length of the red lines. Figure 3.19 illustrates how the integrating property of the GRAYMAT definition gives rise to large distance values towards the centre of bright regions, while WDOCS be33.

(255) haves like a spatial distance transform over homogeneous regions and shows a sharp increase in distance when traversing an edge. . . . Figure 3.18: Grey-weighted distances in the 1D case. For GRAYMAT, the distance is equal to the combined area of the bars. For DOCS and WDOCS, the distance is equal to the length of the red lines.. Figure 3.19: Grey-weighted distance transforms. Original image with seeds (top), GRAYMAT distance map (left) and WDOCS distance map (right).. When using the GRAYMAT definition for a grey-weighted transform of a single object, the integrating property makes the local maxima on the distance map resemble the centre of mass (see Figure 3.20). This property is utilised extensively in Papers II, III, V, and VIII. Other distances have been defined on grey-level images. For example, Bloch [7] detailed several distances between fuzzy sets. She also proposed a new geodesic distance for fuzzy sets [8] based on Rosenfeld’s definition of fuzzy connectedness [49]. Soille [63] also defined a geodesic measure inspired by Levi and Montanari’s definition [40]. 34.

(256) intensity DT GRAYMAT. Figure 3.20: Topographic illustration of the 1D case showing how the local maximum of the GRAYMAT distance transform is more representative to the underlying intensity profile than is the local maximum of the distance transform (DT in legend). The arrows indicate the local maxima positions.. Algorithms Due to hardware limitations, early methods for computing grey-weighted distance transforms were based on the classic, but slow, raster scan approach described in Chapter 3.2.1, iterating until stability. The approach works well for spatial distance calculations where a distance transform only need two passes through the image, but for grey-weighted distance, where the domain is usually not convex, the number of passes through the image becomes dependent on content. Like for distance transforms, propagation using graph-search techniques based on Dijkstra’s algorithm have become popular to reduce computation time [22, 33, 67]. There is a wealth of data structures available for Dijkstra’s algorithm which affect the performance. A thorough investigation on which algorithms and data structures perform best in different scenarios for computation of grey-weighted distances is performed in Paper IX. A similar study is presented by Nyul et al. [46] for fuzzy connectedness [66]. A common design choice associated with the data structures is whether to choose memory efficiency over speed, or vice versa. A method for efficiently computing point-to-point fuzzy distances by using heuristics is presented in Paper III and heuristics for the DOCS and WDOCS definitions are presented in Paper VII.. 3.2.4 Analysis of linear features For analysing linear features in 3D images, such as images of neurons and vasculature, Al-Kofahi et al. [3] covers the three most common approaches. The first is based on skeletonisation [60], where the object is eroded iteratively by some structuring element to extract a medial representation called the skeleton. The second approach is based on edge enhancement and identi35.

(257) fying contours by chaining edge pixels together. Both the skeletonisation and edge enhancement approaches require processing of every pixel in the image with numerous operations per pixel. The third approach, generally referred to as tracing but sometimes vectorisation or vectorial tracking, is exploratory by extracting an initial point and then tracing the structures recursively based on local image properties. A common tracing approach is to use template matching (covered in Chapter 3.1.3) for local identification and tracing of linear features. A value indicating the strength of a linear feature can be measured for each pixel by looking at the eigenvalues of the Hessian matrix [24, 41]. In image analysis, the Hessian matrix is a square matrix of second order partial derivatives of an image, ⎡ ⎤ ∂2 f 2 ⎢ ∂∂ 2x f ⎢ H( f ) = ⎣ ∂ y∂ x ∂2 f ∂ z∂ x. ∂2 f ∂ x∂ y ∂2 f ∂ y2 ∂2 f ∂ z∂ y. ∂2 f ∂ x∂ z ⎥ ∂2 f ⎥ , ∂ y∂ z ⎦ 2 ∂ f ∂ z2. where f = f (x, y, z) is a 3D image. The eigenvalue decomposition of the Hessian provides a useful geometric interpretation. The Hessian maps a spherical neighbourhood to an ellipsoid whose axes are along the directions given by the eigenvectors and the corresponding axis’ lengths are the magnitude of the respective eigenvalues. The ellipsoid locally describes the second order structure (variation) of the image and can be used as a tool for determining the local geometry. A linear feature in the geometric interpretation of the Hessian for 3D images means, |λ1 | ≈ 0 |λ1 | << |λ2 | λ2 ≈ λ3 where λk is the eigenvalue with the k-th smallest magnitude (|λ1 | ≤ |λ2 | ≤ |λ3 |). The variation along the linear feature should be low (|λ1 |) while the variations in the orthogonal plane should be high (|λ2 |, |λ3 |). Frangi et al. [24] used these eigenvalue relations to construct a dissimilarity measure referred to as the vesselness value for 3D vessel enhancement filtering.. 3.3 Image analysis in electron tomography The recognition of macromolecules in 3D density images from ET has lately become a vital issue. Recent ET analysis have been focused on rigid, and mostly symmetric, macromolecular structures in the scale 500-1000 kDa using various template matching techniques [5]. To deal with the massive com36.

(258) putational complexity inherent in template matching, various approaches have been used: reduction of search space using non-linear anisotropic diffusion prior to template matching [9]; and a rough matching, complemented by either cluster analysis [23] or segmentation to refine the result [47] are two examples. Template matching is unsuitable for localising non-rigid structures due to mainly two reasons. Firstly, allowing for flexibility in the template increases the already high computational complexity. Secondly, not all macromolecules are available as high-resolution templates. The influence of the missing wedge will also have to be incorporated into the template model. There exists a number of segmentation methods for ET data which do not use template matching [25, 71, 73]. These focus more on the identification of sub-domains in macromolecules. In the work by Volkmann [71], seeds are chosen as local maxima after pre-processing the image with a Gaussian smoothing algorithm and the sub-cellular structures are identified by seeded watershed segmentation. In the work by Yu and Bajaj [73], the symmetry inherent in the macromolecular complexes imaged in that study is utilised using averaging. In the work by Garduño et al. [25], fuzzy set methods, in terms of fuzzy segmentation, are used to segment selectively stained, plastic embedded spiny dendrites imaged by ET by manually placing seed points in the density images. Fuzzy set theory is also used in the related work in Paper 3 for describing inter-domain flexibility of macromolecules. One of the drawbacks in analysis of macromolecules in cryo-ET is that it suffers from low SNR (see Chapter 2.1.1). Various denoising algorithms are popular to improve the conditions for morphological analysis. See the work by Narashima et al. [45] for a recent evaluation. Another problem in the ET field in general is the validation of large experiments, which is a monumental task even with the help of experts to manually create a ground truth dataset by visual inspection. Therefore, using simulated data for evaluation is essential and frequently used in this research field, e.g., see the work by Scheres et al. [58].. 3.4. Image analysis of neurites. Analysis of neurites has been performed using various approaches for handling linear features (covered in Chapter 3.2.4). The skeletonisation approach, first used in an attempt for automatic tracing of neurons from 3D FM data by Cohen et al. [15], is perhaps the most commonly used approach. Its utility can be illustrated by its use in three works on morphology analysis of neurites. In the first work, Koh et al. [37] perform morphology analysis of dendritic spines (small protrusions on the dendritic backbone) from FM images of rat neurons. The dendrites are segmented by manual thresholding and the skeleton is found through erosion of the connected components. The skeleton is then pruned by identifying short spurs 37.

(259) and loops on the medial axis. The backbone of each dendrite is extracted by tracing the medial axis and employing a decision based on minimum deviation angle whenever a dendritic branching point is encountered. Finally, the dendritic spines are categorised by morphological analysis of all protrusions from the dentritic backbone. In the second work, Dima et al. [18] automatically segment and skeletonise neurons from insects in FM images. A 3D wavelet transform is used for multiscale edge detection and the edges are used to derive a segmentation algorithm by filling the regions between boundary edges. The skeleton is extracted from the segmentation together with branching points and sharp bending points. The individual parts are then put together to a labelled graph representation for morphology analysis. In the third work, He et al. [32] perform 3D tracing of cat neurons in FM images. They assume sparseness of the structures and exploit this by limiting the skeletonisation to a N × N × N neighbourhood to decrease the computational complexity. The skeleton is used to extract a graph representation of the neuron, which is pruned to eliminate spurs. Although the title claims that the method is automatic, it requires a semi-automatic interaction step for reconnecting dendritic fragments from the skeletonisation. An exploratory approach is used by, e.g., Al-Kofahi et al. [3] for tracing neurons in FM images using a generalised 3D cylinder model to guide the tracing. This limits the computational complexity compared to the global skeletonisation approach despite the extra complexity introduced by the many degrees of freedom of matching a cylindrical template. The extraction of linear features using the Hessian matrix is used successfully by Meijering et al. [44] to guide the semi-automatic tracing of neurites from 2D FM images. In Paper VI, the approaches from Al-Kofahi et al. [3] and Meijering et al. [44] are combined to examine whether automatic tracing of neurons from 3D images can be facilitated using local path-finding. Instead of the computationally intense template matching, the method relies on linear feature values from the Hessian.. 38.

No results found