Towards automatic detection and visualization of tissues in medical volume rendering

Full text

(1)Examensarbete LITH-ITN-MT-EX--06/012--SE. Towards Automatic Detection and Visualization of Tissues in Medical Volume Rendering Erik Dickens 2006-02-03. Department of Science and Technology Linköpings Universitet SE-601 74 Norrköping, Sweden. Institutionen för teknik och naturvetenskap Linköpings Universitet 601 74 Norrköping.

(2) LITH-ITN-MT-EX--06/012--SE. Towards Automatic Detection and Visualization of Tissues in Medical Volume Rendering Examensarbete utfört i medieteknik vid Linköpings Tekniska Högskola, Campus Norrköping. Erik Dickens Handledare Claes Lundström Examinator Anders Ynnerman Norrköping 2006-02-03.

(3) Datum Date. Avdelning, Institution Division, Department Institutionen för teknik och naturvetenskap. 2006-02-03. Department of Science and Technology. Språk Language. Rapporttyp Report category. Svenska/Swedish x Engelska/English. Examensarbete B-uppsats C-uppsats x D-uppsats. ISBN _____________________________________________________ ISRN LITH-ITN-MT-EX--06/012--SE _________________________________________________________________ Serietitel och serienummer ISSN Title of series, numbering ___________________________________. _ ________________ _ ________________. URL för elektronisk version. Titel Title. Towards Automatic Detection and Visualization of Tissues in Medical Volume Rendering. Författare Author. Erik Dickens. Sammanfattning Abstract The technique. of volume rendering can be a powerful tool when visualizing 3D medical data sets. Its characteristic of capturing 3D internal structures within a 2D rendered image makes it attractive in the analysis. However, the applications that implement this technique fail to reach out to most of the supposed end-users at the clinics and radiology departments of today. This is primarily due to problems centered on the design of the Transfer Function (TF), the tool that makes tissues visually appear in the rendered image. The interaction with the TF is too complex for a supposed end-user and its capability of separating tissues is often insufficient. This thesis presents methods for detecting the regions in the image volume where tissues are contained. The tissues that are of interest can furthermore be identified among these regions. This processing and classification is possible thanks to the use of a priori knowledge, i.e. what is known about the data set and its domain in advance. The identified regions can finally be visualized using tissue adapted TFs that can create cleaner renderings of tissues where a normal TF would fail to separate them. In addition an intuitive user control is presented that allows the user to easily interact with the detection and the visualization.. Nyckelord Keyword. 3D Graphics, Transfer Function Design, Volume Rendering, Medical Image Processing, Tissue Detection, A priori knowledge, Tissue Visualization.

(4) Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/. © Erik Dickens.

(5) Towards Automatic Detection and Visualization of Tissues in Medical Volume Rendering Erik Dickens 17th February 2006.

(6) Abstract The technique of volume rendering can be a powerful tool when visualizing 3D medical data sets. Its characteristic of capturing 3D internal structures within a 2D rendered image makes it attractive in the analysis. However, the applications that implement this technique fail to reach out to most of the supposed endusers at the clinics and radiology departments of today. This is primarily due to problems centered on the design of the Transfer Function (TF), the tool that makes tissues visually appear in the rendered image. The interaction with the TF is too complex for a supposed end-user and its capability of separating tissues is often insucient. This thesis presents methods for detecting the regions in the image volume where tissues are contained. The tissues that are of interest can furthermore be identied among these regions. This processing and classication is possible thanks to the use of a priori knowledge, i.e. what is known about the data set and its domain in advance. The identied regions can nally be visualized using tissue adapted TFs that can create cleaner renderings of tissues where a normal TF would fail to separate them. In addition an intuitive user control is presented that allows the user to easily interact with the detection and the visualization..

(7) Acknowledgements I would like to give great and warm thanks to the following persons: Claes Lundström, my supervisor at Sectra Imtec AB, for great support and many fruitful discussions that made this thesis to what it is. All the colleagues at Sectra, especially Aron Ernvik and Fredrik Karlsson, for all their support regarding the development environment and making my stay at Sectra a great time. My academic supervisor Anders Ynnerman for valuable feedback and ideas. Rasmus Ewehag for reading and giving feedback on the report. Finally all my friends and my family, you gave me support just by being there.. 1.

(8) Contents 1 Introduction 1.1 1.2 1.3 1.4. Problem Description Objectives . . . . . . Outline of Report . . Reader Prerequisites. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. 2 Background 2.1 2.2 2.3. Medical Imaging . . . . . . . . . . . . 2.1.1 Modalities . . . . . . . . . . . . Medical Volume Visualization . . . . . 2.2.1 Direct Volume Rendering . . . The Use of Medical Volume Rendering. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Applications. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 3 Related Work 3.1 3.2. One-dimensional Transfer Functions . . . . Multidimensional Transfer Functions . . . . 3.2.1 Derivatives . . . . . . . . . . . . . . 3.2.2 Curvature . . . . . . . . . . . . . . . 3.2.3 Local neighborhoods and structures. 4 Methods 4.1 4.2 4.3 4.4. 4.5 4.6. Overview . . . . . . . . . . . . . . . 4.1.1 System Pipeline . . . . . . . A priori knowledge . . . . . . . . . . 4.2.1 The description of a tissue . . 4.2.2 The content of the data set . Automatic Peak Detection . . . . . . 4.3.1 Partial Range Histogram . . 4.3.2 Peak Search . . . . . . . . . . Partial Range Histogram Processing 4.4.1 Objective . . . . . . . . . . . 4.4.2 Processing concepts . . . . . 4.4.3 Merging . . . . . . . . . . . . 4.4.4 Splitting . . . . . . . . . . . . 4.4.5 Morphology . . . . . . . . . . Classication . . . . . . . . . . . . . Visualization . . . . . . . . . . . . . 4.6.1 Adaptive Trapezoids . . . . .. 2. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . .. . . . .. . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. 6 6 7 7 7. 8. 8 8 9 11 12. 15 15 16 16 17 18. 20. 20 20 22 22 23 23 24 25 26 26 26 27 29 29 32 33 33.

(9) 4.6.2 4.6.3 4.6.4. Local Transfer Function . . . . . . . . . . . . . . . . . . . Block Volume Visualization . . . . . . . . . . . . . . . . . Histogram Visualization . . . . . . . . . . . . . . . . . . .. 5 Implementation 5.1 5.2 5.3. Application Environment . . . . . . . . . . . . . . . . . PRH Processing . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Splitting and merging schemes . . . . . . . . . . 5.2.2 Content dependent morphology . . . . . . . . . . Graphical User Interface . . . . . . . . . . . . . . . . . . 5.3.1 Automatic Peak Detection and Visualization Tab 5.3.2 Merging and Splitting Tab . . . . . . . . . . . . 5.3.3 Morphology Tab . . . . . . . . . . . . . . . . . . 5.3.4 Classication Tab . . . . . . . . . . . . . . . . .. 6 Results 6.1 6.2 6.3 6.4 6.5 6.6. Test environment and data sets Automatic Peak Detection . . . Merging and Splitting . . . . . Morphology . . . . . . . . . . . Classication . . . . . . . . . . Visualization . . . . . . . . . .. 7 Discussion 7.1 7.2. Conclusions . . . . . . . . 7.1.1 Related Work . . . 7.1.2 Proposed Methods Future Work . . . . . . .. . . . .. . . . .. . . . .. A Appendix. . . . . . . . . .. . . . . . . . . .. . . . . . . . . .. . . . . . . . . .. . . . . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. 34 34 35. 37. 37 38 38 42 42 43 43 43 44. 46. 46 46 48 50 51 52. 54. 54 54 55 55. 57. 3.

(10) List of Figures 2.1 2.2 2.3. 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 5.1. 5.2 5.3 5.4 5.5 5.6 6.1 6.2 6.3 6.4. The tube and the detector in a CT scanner . . . . . . . . . . . . The coronal (1), sagittal (2) and axial (3) slices . . . . . . . . . . Ray casting. For each pixel a ray is cast. The ray is sampled and the nal pixel intensity is determined by the integration of all these samples. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 9 10. The system pipeline . . . . . . . . . . . . . . . . . . . . . . . . . In this data set no obvious peaks are present in the global histogram that can guide the user in the TF setting . . . . . . . . . A PRH that has two PRH objects. . . . . . . . . . . . . . . . . . The Contact attribute of these two PRH is low, but their Connectedness is maximum. . . . . . . . . . . . . . . . . . . . . . . . The 2D analogue of the structuring element used in this processing. A part of the aorta is dilated content dependently using κ = 0.40 The Adaptive Trapezoid. A trapezoid that ts the content of the PRH (the Gaussian curve approximating the content is not present in the gure) . . . . . . . . . . . . . . . . . . . . . . . . . The block volume visualization and the DVR synchronized in the same scene. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The histogram visualization . . . . . . . . . . . . . . . . . . . . .. 21. A not equally densed contrast agent in an angiographic data set. Red visualizes the regions with the highest intensity, i.e. highest density of contrast, whereas the regions with somewhat lower intensity are visualized in green. . . . . . . . . . . . . . . . . . . The histogram of a block containing two dierent tissues . . . . . The APD and visualization tab page . . . . . . . . . . . . . . . . The merging and splitting tab page . . . . . . . . . . . . . . . . . The morphology tab page and the condence slider . . . . . . . . The classication tab page . . . . . . . . . . . . . . . . . . . . . . In the rst test data set the bile duct is visible as the high intensity region. Other tissues are also identiable. . . . . . . . . . . . The global histograms of the test data sets. The upper corresponds to the rst test data set and the lower to the second (the angiography). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A result of manually setting a 1D TF for the rst test case. . . . A subset of the output PRH of the APD algorithm processing the rst test data set. . . . . . . . . . . . . . . . . . . . . . . . . 4. 12. 24 27 28 30 31 34 36 36. 39 41 43 43 44 45 47 47 48 48.

(11) 6.5. A block visualization of three PRH directly after the APD algorithm has processed the rst test data set. . . . . . . . . . . . . . 6.6 The merging in Scheme 1 applied to the second test case. . . . . 6.7 Dierent merging and splitting schemes applied to the initial set of PRHs of the rst test case. (a) Scheme 2.1, (b) Scheme 2.2 and (c) Scheme 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 A block visualization of (a) before and (b) after a merge using the Ellipsoid attribute. . . . . . . . . . . . . . . . . . . . . . . . . 6.9 The condence level used on a PRH containing parts of the liver 6.10 The condence level used on a PRH containing a part of the aorta 6.11 DVR using (a) global TF in comparison with (b) local TF that makes use of the known region of the PRH. The stain of the color of the kidney on the liver is due to that the regions of the PRHs are actually dilated to avoid block artefacts . . . . . . . . . . . .. 5. 49 49 50 51 52 53. 53.

(12) Chapter 1. Introduction This initial chapter introduces the thesis to the reader. A description of the problem that this thesis addresses is followed by a short summary of its main objectives. Finally the outline of the report is presented and a recommendation of reader prerequisites is given.. 1.1 Problem Description The making of 3D visualizations of medical image volumes using the technique of volume rendering has become an increasingly important tool in work of the radiology departments of today. Its strength of eectively capturing 3D internal structures as the blood vessels and the skeleton in one single 2D image provide the users with valuable information when analyzing and making diagnosis of medical image data. Furthermore the user can interact with this 3D rendering in dierent ways. Rotating and zooming, i.e. changing the point of view, enhances the understanding of the shapes of the organs and their locations relative to one another. Other interaction tools are available to determine what organs that should be visible or to visually separate them by changing their colors. Despite that the applications that use this technique have existed in the area of medical imaging for quite some years they have not yet become a natural tool in the every-day diagnostic work of the radiology departments. Instead their use is limited to a few specic types of examinations. There are naturally several reasons, but this thesis will address two major obstacles for this lack of use. Firstly the tool that makes dierent organs or tissues visually appear can often fail to separate these tissues when dealing with more complex medical data sets. This tool is called Transfer Function (TF) and its most common implementation is to transfer the intensities in the image volume to a color and opacity in the nal rendering. As mentioned it works ne is some types of examinations where each tissue of interest has a well separated range of intensities. In other image volumes, these intensity ranges are widely overlapping and a more sophisticated TF is needed. Since the TF determines the appearance of the rendered image, this is the most natural step in the rendering process where the user can interact and determine what should be visualized. This is the second obstacle that this thesis addresses; the controls given to the user in order to design the TF. Today, even. 6.

(13) for the simplest kind of TF as described above, the interaction is too complex. The time that the users can spend on modifying the TF in addition to their often not too extensive computer experience constrain the complexity of such an interaction. Furthermore, if a more complex TF is to be used, its consequence is certainly a more complex user interaction if it is not deliberately simplied.. 1.2 Objectives Given the above stated problem description, the objectives of this thesis are the following:. • Evaluate existing techniques for Transfer Function design. • Further develop a promising approach to the problem, both in terms of improving and extending the algorithm, and in terms of the interface, i.e. controls and visualizations, nally exposed to the user. • Implement and test the proposed tools and methods.. 1.3 Outline of Report Chapter 2 gives an introduction to the area of medical imaging in terms of how the image data is produced and visualized. A description of the volume rendering technique and how it is used is also given. Chapter 3 presents the previous work that is related to the problem of TF design. The following chapter describes the ideas and methods proposed by me and my supervisors that try to solve this problem. The implementation of these methods is described in chapter 5, i.e. how these methods can be used and the look of the graphical user interface. Chapter 6 is dedicated to present the achieved results of the proposed methods and their implementation. Finally chapter 7 ends this report by stating the conclusions drawn about the previously made work in this area of research. It also concludes and discusses the fulllment of the proposed methods with respect to their objectives and how these methods can be further developed.. 1.4 Reader Prerequisites For a reader to fully appreciate the content of this thesis a good knowledge of image processing and analysis, e.g. the concept of morphology, is recommended. Furthermore, knowledge of 3D computer graphics and the concept of volume rendering and its use in medical imaging will also help the understanding while reading this report.. 7.

(14) Chapter 2. Background This chapter introduces the reader to the area of medical imaging explaining the main causes to the problems that gave rise to this thesis. The rst section will describe the characteristics of medical image data and how it is produced. Furthermore, dierent tools that visualize medical image volumes will be presented, for example the volume rendering technique that is highly relevant for the objective of this thesis. The following section will explain why the medical volume rendering applications often fail to become an everyday diagnostic tool for its assumed users. Finally a more thorough presentation of the TF design tools available in the medical volume rendering applications of today will be given.. 2.1 Medical Imaging Medical imaging has been a powerful diagnostic tool in medicine for a long time. The traditional way of producing medical images uses a at-panel x-ray detector visualizing the bone structures inside the body. The lms produced by this apparatus were put in front of a light screen facilitating the analysis. Nowadays many radiology departments produce digital images. The cost- and time-eciency that a clinic can gain by managing and storing digital images in comparison with the traditional lms is convincing.. 2.1.1 Modalities The equipment that can produce digital images is called a modality. There are several types of modalities, the primary are Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound, PET and Mammography. CT and MRI scanners will be further described since they have the ability of producing image data interesting for this thesis, namely 3D medical image data sets.. Computed Tomography The Computed Tomography, also known as CAT. (Computed Axial Tomography) scanning, is one of the most commonly used modality type [7]. A CT modality uses x-rays to produce its images and has therefore a characteristic of capturing bone that absorbs the radiation. Besides this it is also 8.

(15) used in the analysis of blood vessels. Before the scanning, the blood vessels are injected with a contrast material. This material attenuates the x-rays during the scanning and the contrast-lled blood vessels become easily registered highintensity regions in the resulting image. The CT also has an ability of capturing soft tissue, although this is not what it is mainly used for. The images produced by the CT scanner are slices with a normal in the direction of the main axis going through the body. These are accomplished by letting a tube rotate in a spiral around the patient emitting x-ray radiation while on the opposite side a detector measures the amount of attenuation (g 2.1). A slice is produced each time the tube has rotated 360◦ . An advantage of. Figure 2.1: The tube and the detector in a CT scanner computed tomography is that the scale of intensity values is calibrated and has a dened unit, called the Hounseld unit. This means that the intensity range of a tissue is approximately the same, regardless of patient. The dierences from case to case are only dependent of the properties of the examined tissues.. Magnetic Resonance Imaging The other major modality that can produce. 3D image data sets is a scanner that uses the technique of Magnetic Resonance Imaging (MRI) [20]. The images such a modality produces are somewhat noisy but it has an ability of eciently separating dierent kinds of soft tissues. The patient is surrounded by a strong magnetic eld. A radio pulse is emitted with a frequency that puts the hydrogen molecules in the body into a high-energy state. When the pulse is turned o the molecules will go back to their normal state, emitting their acquired extra energy. The time it takes for this emission to decay is detected by the modality in order to generate the nal image. In comparison to an image produced by a CT scanner, the MR data values can hardly be expressed with a dened unit and a calibrated scale. This is a characteristic of the MRI data sets; the same tissue of the same patient can have totally dierent intensity values from case to case.. 2.2 Medical Volume Visualization The image volumes that these multi-slice scanners produce need to be visualized to the user. The most straight-forward way of working is to simply view each one of these slices one after another. To make this kind of 2D visualization more exible one can produce a slice with an arbitrary normal direction, interpolating 9.

(16) its values from the volume data. This plane can then be moved along its normal, interactively exploring the content of the data set. The technique is called Multi Planar Reconstruction (MPR); an image can be produced from any plane in the volume data. A common use of MPR in medical 3D applications is to have three dierent MPR views; the axial, coronal and sagittal view. Each view presents a slice with a normal in the direction of the x-, y- or z-axis (g 2.2). Despite the fact that the MPR technique is very exible, it still cannot be called. Figure 2.2: The coronal (1), sagittal (2) and axial (3) slices true volume visualization since it only visualizes values from a 2D plane in the volume. Exploring the volume data and getting a good understanding of its content can be a time consuming process when only a 2D plane is visualized in each image. Instead, there is certainly a need to capture more information in the rendered image displayed to the user. A technique called Maximum Intensity Projection (MIP) can visualize a slice of arbitrary thickness. The rendered image is parallel to the slice. To produce the image a ray is cast from each pixel. The maximum intensity value that the ray encounters through the thick slice will set the new pixel value. Now the rendered image can capture the high intensity regions within this thick slice. This is for example useful when analyzing angiographic data sets where the blood vessels are the objectives. When speaking of visualization of volumes with arbitrary content, the objective is often to visualize surfaces of materials present in the volume data. Techniques have been developed to approximate surfaces with geometrical primitives in order to render them using normal graphics accelerating hardware. One of these techniques is the Marching Cubes algorithm [16]. It approximates an isosurface, a surface dened by a single value. I.e. all points on an isosurface have the same value. However these techniques are often not very useful by themselves when dealing with medical image data. In this domain, the rendered image should be able to capture the spatial variations of the intensities throughout the volume. Firstly, the surface of a tissue in a medical volume data set can rarely be described as discrete, i.e. that a voxel either belongs to the tissue or not. Instead, the smooth transitions from one material into another need to be visualized. Moreover, the boundaries are not the only objective of medical volume visualization; also the inner intensity variations of a tissue are interesting. To this end the characteristics of Direct Volume Rendering come in 10.

(17) handy.. 2.2.1 Direct Volume Rendering Direct Volume Rendering (DVR) is a process that can allow the entire content of a volume to be visualized in a rendered 2D image, i.e. every voxel in the volume has a possibility to contribute to the nal rendering. In comparison to surface rendering, which uses geometrical primitives to describe features in the data set, the volume rendering operates directly on the individual data values of each voxel [8]. Most often the desirable result is not to let all voxels in the volume contribute to the rendered image. Instead, prior to the actual rendering a Transfer Function (TF) maps the properties of a voxel to optical properties such as opacity and color that will make it visible in the nal rendering. This is the most important step in the rendering process. Here the user is allowed to interact and decide what to visualize. Each voxel can now be represented with a color that will yield a resulting color image. In medical imaging, this kind of visualization of medical image volumes is referred to as the Volume Rendering Technique (VRT). When performing the actual rendering, the volume rendering integral needs to be evaluated [4]. This integral solves how the light is aected along a ray from a light source to the eye of the viewer travelling through the volume. I.e. it is evaluated considering both emission and absorption along this ray. In its denition this integral is continuous. Although, dealing with discrete data sets, a numerical approximation is made and samples are taken along the ray where each sample emits and/or absorbs the light energy. This integral is evaluated by a rendering technique. The object-order approach is to calculate how the individual voxels in the volume will contribute to the rendered image. A technique that uses this approach is when rendering volumes on consumer graphics hardware where the volume data is stored as 2D or 3D textures [2]. Proxy geometry, quadrants with this texture mapped onto them, represents the volume in the rendering scene. In normal geometrybased rendering the object order approach is implemented; each object, i.e. a geometrical primitive, is projected down onto the image plane. The same thing is made with this proxy geometry, taking into consideration that the textures have an alpha channel that determines its opacity. The evaluation of the volume rendering integral is performed when these slices are projected to the image plane, integrating their individual opacity and color channels. Another approach is the image-order rendering. Such a technique determines for each pixel in the nal image how much the volume contributes to it, i.e. an opposite approach compared to object order rendering. The most commonly used technique implementing this approach is ray casting, a quite straight-forward evaluation of the volume rendering integral. For each pixel in the image a ray is shot into the volume. Along this ray samples are taken. Each sample is transferred to a color and opacity according to the dened TF. Now, determining the nal pixel values these samples are integrated along the ray. A simple optimization of this algorithm can be done by integrating in a front-to-back order starting from the sample closest to the user. If the integration at some point reaches full opacity, it can be terminated since the samples lying behind will not aect the resulting pixel. Summing up, the volume rendering of a medical image volume provides 11.

(18) Figure 2.3: Ray casting. For each pixel a ray is cast. The ray is sampled and the nal pixel intensity is determined by the integration of all these samples. the user with a comprehensible 2D image that can visualize the internal 3D structures and the variations of intensities throughout the volume data.. 2.3 The Use of Medical Volume Rendering Applications In the beginning of this thesis, I made three visits at dierent radiology departments and met users with varying amount of experience of medical volume rendering applications. Discussions gave me a deeper insight about the gap between these applications and its assumed users; what can be done to make these applications the powerful tool it can be. The primary users of a medical volume rendering application are using it as a tool to assist the making of diagnosis. Most often these users are radiologists. But also nurses and technicians that produce the image data can be included in that user group, especially when no radiologist is at hand, e.g. at night times or at smaller departments. In recent years these applications also have started to reach out to other clinics as the orthopedist and can for example function as a support in surgery planning. There are some main factors that determine whether the medical volume rendering application can serve as the supportive tool it is supposed to be [14]. 1. The level of complexity in learning the tool must t the level of computer 12.

(19) experience that the users have. The profession of a radiologist should mean deep knowledge in the human anatomy and its diseases and malfunctions, not necessarily an interest in computer science. 2. The level of complexity in using tool, once learned, must correspond to the time that the users can spend on examining. Normally a radiologist only uses from 30 seconds up to a couple of minutes examining one case. There is understandably little time to spend on adjusting and tuning parameters that will render an image that can support the diagnosis. 3. The application must contain tools that can give a suciently good support in the tasks it is supposed to aid. In some cases, it does not matter how much time the user spends on trimming parameters; the tools of the application are too weak and it will be impossible to render a comprehensive image of the volume data. 4. The tools must provide the user with results that the user can trust. If a tool contains a lot of automatic processing and few interaction possibilities are left to the user, it is a risk that the user will loose the condence in the result. On the other hand, if too much interaction is needed from the user, the process can easily be too time-consuming to t the demands of the current working situation. 5. Economical availability of the application In some cases specialized hardware or applications developed to solve a very specic problem are needed to make volume rendering a supportive tool. This will make the option of using volume rendering too expensive for many departments. This thesis will present tools that will try to improve a medical volume rendering application with respect to the initial three problems stated above.. Transfer Function Design As described the TF setting is currently the most important step in the volume rendering process with respect to user interaction. The TF determines what properties a voxel must have to be visible in the nal image. With this tool the user can interact with the rendering and make it visualize what is of his/her interest. This is why the TF design is and has been a large area of research and also the underlying objective of this thesis. Mathematically described it is a function mapping from a set of properties P1 . . . Pn of a voxel v to a set of optical properties O1 . . . On .. T F : (P1 (v), P2 (v) . . . Pn (v)) → (O1 , O2 . . . On ). (2.1). The medical volume rendering applications of today oer the user the possibility of setting a 1D TF that maps the intensity of a voxel to a color and opacity. I.e.. T F : f (v) → (R, G, B, A) 13. (2.2).

(20) The user can select a predened TF setting and is also oered the possibility to manually try to construct and adjust the TF. The predened TF, also called presets, try to reect certain tissues, i.e. the intensity range that this tissue is believed to have is set opaque and colored. These presets are static and work ne in the case when the intensity ranges of the tissues are non-overlapping and known in advance. They are best applied to CT data where the scale is calibrated. Unfortunately there is still a need for manual adjustment since the properties of the tissues can vary from patient to patient. The manual setting of the TF is often performed under the guidance of the histogram. Since the peaks allocated in the high-intensity regions of the histogram often correspond to tissues of interest, the user can identify their intensity ranges and set them opaque. A shortcut to the TF modication is the Window level tool. By moving the mouse in dierent directions the user can change the width and location of the opaque intensity range. This is a more commonly used interaction since it requires little understanding of the TF domain and is fast to use. For further description of tools for user interaction in medical volume rendering applications the reader is referred to [14]. As stated in the problem description these tools are in many cases not sufcient. In MRI data sets the intensity ranges of the tissues are unpredictable from case to case. Even in CT volumes they vary. This problem urges the need of active presets, i.e. presets that are generated by analyzing the content of the data set to examine. The other main problem that makes these tools insucient is that the tissues of interest often have overlapping intensity ranges. A TF solely based on intensity fails to separate these tissues in the rendering. A solution to this is to make use of more properties of a voxel to determine its optical properties. Methods that tries to overcome these stated problems and other related work are presented in the following chapter.. 14.

(21) Chapter 3. Related Work The content of this chapter will present the research related to the problem of TF design. The rst section will describe some developed tools and research for designing a 1D TF. The following section is dedicated to a more complex TF design that uses more than one property of a voxel to determine its color and opacity. This chapter is an important part of the thesis since one of the objectives mentioned in section 1.2 is the evaluation of the research made in this area. The conclusions of this evaluation can be found in chapter 7.. 3.1 One-dimensional Transfer Functions As described in section 2.3 the 1D TFs is the most common kind of TF used in medical volume rendering applications of today. This section will present some important research made regarding tools that use a 1D TF solely based on the intensity of the voxel. The focus of this research is instead on providing the user with tools that will facilitate the nding of the optimal TF. Since the intensity is the only property of the voxels that will aect their appearance in the rendered image, Bajaj introduced in [1] a natural approach by analyzing the isosurfaces that dierent intensity values will form. The concept is called the contour spectrum and it visualizes properties of all 3D isosurfaces in a 2D interface. These properties are all scalar values and can be modeled as a function of the isovalue. The most important properties used to analyze the isovalues are the surface area and the gradient integral. The gradient integral is the accumulated gradient for a specic isosurface. Since a larger gradient often indicates a boundary between materials, the local maxima of this function is used to determine interesting isosurfaces. Bajaj also proposes an automated TF design by setting opaque colors at the intensities corresponding to these local maxima. Another isosurface property that can be useful is the hyper Reeb graph, presented in [3]. The hyper Reeb graph describes how the topology changes varying the isovalue, i.e. visualizing when an isosurface splits up into two or more disjointed objects. A problem with these methods is that they try to nd object boundaries in terms of an isosurface; this is not the objective of a TF design in medical volume rendering. In medical image data the intensity of a pixel can be aected. 15.

(22) by more than one tissue. In these cases it is not correct to determine a discrete surface of a tissue since a pixel at the boundary suering from this eect cannot be classied as entirely belonging to that tissue or not. This characteristic of medical image data is called the partial volume eect and exists indierently of what type of modality that produced it. A dierent approach is presented in [5] where the user's grading of the image quality is the centre of the design, not the actual data values contained in the data set. The used technique is a user-guided stochastic search algorithm that tries to converge to the image that the user thinks best visualizes the objectives. The user is set to choose among an initial set of renderings produced by dierent TFs. The visualizations that the user is most satised with will be the input to a new set of renderings. I.e. the TFs that produced these images will be the parents of the new population of TFs. This process will iterate until the user is satised with the results. The disadvantage of this method is its time consuming process, both in terms of user interaction and image rendering since a vast amount of renderings must be produced. As previously mentioned, the 1D TF is most often not sucient to visualize the interesting tissues due to overlapping intensities. The research described in the following section tries to approach this problem.. 3.2 Multidimensional Transfer Functions The insuciency of the 1D TFs to solve the problem demands the use of letting more properties of a voxel determine its color and opacity. These TFs are called Multidimensional Transfer Functions. This section will describe TFs depending on properties such as the derivatives, the curvature and the local neighborhood of a voxel.. 3.2.1 Derivatives The concept of multidimensional TFs was initially introduced by Levoy in [13] where the second dimension is the absolute value of the gradient. Once found a value corresponding to an interesting isosurface, the paper proposes that a voxel having a value close to this isovalue will have opacity dependent on its gradient magnitude; the larger this magnitude is, the further away from the isovalue the voxel value is allowed to be. This will produce renderings that better captures the smooth transitions between tissue boundaries in comparison to a pure isosurface. A somewhat dierent use of the gradients is described in [17]. Instead of letting the gradient aect the color and opacity, they assign the voxel dierent lighting properties that will become visible if surface shading is applied in the rendering. This can be used to emphasize surfaces behind another surface that would have been eectively obscured only using a normal TF. At the Visualization 2000 conference there was a competition called the Transfer Function Bake-O [15]. Four dierent teams tried to convince a panel that their approach to solving the TF design problem showed the most promising results. The winner of this bake-o was Gordon Kindlmann that used not only the rst but also the second order derivatives along the gradient direction in order 16.

(23) to semi-automatically generate a TF. His paper [9] introduces the concept of the histogram volume that is a 3D histogram; the value of each bin is the number of voxels having a certain combination of f (v), f 0 (v) and f 00 (v). The f (v) is the intensity, f 0 (v) and f 00 (v) the magnitude of the rst respectively the second order derivatives in the direction of the gradient. Based on this histogram volume, a function p(f (v), f 0 (v)) describes how close a voxel is to a boundary in terms of its value, gradient magnitude and the average second derivative of this (f (v), f 0 (v))-couple. The user interacts with the process by dening a boundary emphasize function that maps the function p to opacity. This could be described as deciding the appearance of the boundaries in the rendering. A great benet of this interaction is that it is in a 1D domain, even though the TF is 2D. In [11] this method is further developed in terms of user interaction tools. The histogram volume can be projected onto one of its axis, i.e. all slices parallel to the plane of two axes are summed. The resulting 2D histogram shows the relation between either (f, f 0 ), (f, f 00 ) or (f 0 , f 00 ). The used histograms are the ones showing the intensity in relation to a derivative. The user can place an overlay with rectangular or triangular shape onto this histogram. The color inside the overlay is constant, but the opacity is maximal along the center line of the overlay and linearly diminishes down to zero at the edges. This is similar to the tool described in the previously mentioned paper of Levoy [13]. A great disadvantage with this kind of interaction is that the user interacts in one domain, in this case the domain of the TF, but then the actual result is visualized in another domain, namely the spatial domain of the data volume. This is confusing, especially when the interaction tool involves derivatives which may be dicult to comprehend. To facilitate the understanding of the connection between these two domains, the user is provided with a dual-domain interactive tool. Now the user can interact in the spatial domain using a data probe and receive feedback of its correspondence in the TF domain. This tool is used to automatically set a TF. Now the user sets the appearance of the rendered image by interacting in the very same domain.. 3.2.2 Curvature Another property that can be used to inuence the optical properties of a voxel is the shape of an isosurface around it. Hladuvka et al. [6] proposes this approach by introducing a TF based on the principal curvatures. This approach does not consider the gradient of the voxel itself but instead the positions of adjacent samples on an isosurface going through the voxel. For a point on a surface, the principal directions give a good indication of how the surface bends around it. The principal curvatures, k1 and k2 , instead reveals how much the surface changes. The k1 describes the amount of bending in the direction of the tangent at the point where the surface bends the most. k2 on the other hand, represents the smallest bending. The calculation of the principal curvatures proposed by Hladuvka is unfortunately computationally very expensive. Kindlmann et al. presents in [10] a simplication of this calculation using rst and second derivatives extracted through convolution with lter kernels. A TF based on the curvature eectively separate dierent shapes, but again, in the application area of this thesis the objective is not to visualize discrete surfaces. 17.

(24) 3.2.3 Local neighborhoods and structures So far the presented research has only considered intensity and derivatives when assigning optical properties to a voxel. But there is often a need to further improve the performance of the TF. An analysis of the local neighborhood can give valuable information useful in this classication. In fact, the derivatives contain information about the neighborhood. In [22] Sato et al. presents a method for determining what kind of local intensity structure a voxel belongs to. They make use of the Hessian matrix, which is a 3 × 3 matrix containing the partial second derivatives in the position of a voxel. By analyzing the eigenvalues of this matrix the authors state that dierent combinations indicate dierent local intensity structures. The four main structures are edge, line, sheet and blob. To each voxel will now be assigned a value describing how similar its neighborhood is to each one of those. The TF can therefore successfully separate two voxels with similar intensity but that belong to dierent structures. Due to the partial volume eect, previously described in section 3.1, a voxel cannot always be classied as part of one and only one tissue. Surrounding tissues will aect the intensity of the voxel. Research that specically tries to identify the mixture of tissues in such a voxel has been made. In [21] and [12] the authors present techniques for determining the mixture of tissues in a voxel. Letting the TF consider this combination of materials in a voxel would yield a rendering that better separates dierent tissues. An approach covering the entire TF design procedure, from manual user interaction to the nal application of the TF using local neighborhoods is presented in [23]. The interaction is similar to the mentioned data probing in [11], i.e. the user does not need to interact in the domain of the TF. In this interface the user selects voxels in the volume believed to be part of the tissue and voxels believed to not be a part of the tissue. The user has now dened a training set of classied voxels. An online training of a neural network is made while the user interacts. The task of this network is to determine the probability of a voxel belonging to the very same tissue the user is selecting. The input of the network is the features of a voxel in terms of intensity, gradient magnitude and a representation of the local neighborhood, i.e. the neighboring voxels' position and corresponding intensity. Consequently, a well-trained network will assign a high probability of a voxel that has similar features to the voxels the user classied as part of the tissue, and vice versa a probability close to zero if they correspond to the features of the voxels the user believed not to be part of the tissue. A drawback any system using a neural network suers from is its information hiding characteristic, known as the black box. All knowledge that the network gains during the training is stored in its internal nodes, information that unfortunately is totally uninterpretable. This means that nothing can directly change its stored knowledge; feeding the network with new training data is the only way. The proposed methods in the following chapter are based on a paper which further extends the use of local histograms in TF design [18]. Firstly it introduces a novel scheme for detecting materials in an unknown data set by grouping neighborhoods with similar intensity content. This algorithm will be thoroughly described in the next chapter. Secondly it presents an approach that adds a second dimension of the TF 18.

(25) based on local intensity properties. The TF does not depend on any derivatives, i.e. the primary goal is not to display boundaries as it often is in other TF design research. Instead the objective is to separate two tissues with overlapping intensity ranges. Two voxels belonging to dierent tissues but have similar intensities are separated through an analysis of the intensities of the voxels in their neighborhood. This analysis is simply a calculation of how many of these intensities that are within a dened interval. E.g. suppose we want to separate a narrower vessel from a thicker one, e.g. a coronary artery from the aorta. Then the interval is the principal range of values that the vessel contains. A voxel in the coronary artery will therefore have a lower amount of voxels in its neighborhood having a value in this interval, in comparison with a voxel in the aorta. Although this is most often not sucient since a border voxel at the aorta probably will have an amount similar to the voxels in the thinner vessel. A second criterion can then be used to separate these voxels. Assume we know that the aorta is in the vicinity of a tissue with another intensity range. Now, the greater amount of neighborhood values a voxel has in this range, the greater is its probability of belonging to the aorta (assuming that the rst criterion is fullled, i.e. the voxel belongs to the intensity range of the tissue and its neighborhood has a sucient amount of similar voxels). Finally a linear combination of the two probabilities derived from these amounts is used as the second dimension of the TF making a better separation of these vessels in the rendered image.. 19.

(26) Chapter 4. Methods This chapter presents a method that tries to solve the problem of TF design. All sections in this chapter except section 4.3 are developed by me in collaboration with my tutors. Firstly an overview of the method is presented. The following sections in this chapter give more detailed descriptions of the individual steps and the main concepts used in the method.. 4.1 Overview This method does not try to accomplish fully automatic tissue detection and TF generation. It is rather a method equipped with tunable tools that aim to nd properties of tissues of interest, such as the regions in the volume where they are present and their approximate distribution of intensities. The method tries to accomplish the following:. • Find regions in the volume where each region aims to entirely contain a tissue. • Identify those regions most likely to contain a tissue of the user's interest. • Create informative visualizations of these classied regions, the tissues. • Provide the user with intuitive, easy-to-handle tools to control this tissue detection and visualization.. 4.1.1 System Pipeline The architecture of the proposed method is formed as a pipeline. Figure 4.1 presents a functional model visualizing the ow of data in this pipeline using symbols that each represents the functionality of dierent entities.. The functional model In the beginning, a source, visualized as a parallelogram, produces some kind of data. The information this data contains is represented in a data object, a circle, that stores and provides access to it. The process object operates on 20.

(27) Figure 4.1: The system pipeline a data object, processes it in some way, and produces an output data object. Finally the data ends up in a sink, which only consumes an input data object without producing any new. The sink is typically a visualization that presents the data to a user.. Description of the pipeline A very central concept in this pipeline is the Partial Range Histogram (PRH). It is more thoroughly explained in section 4.3, but for now it can be described as the histogram of a region in the volume that aims to contain an entire tissue. The objective of the pipeline is to produce an output of a set of PRHs, where each of these PRHs contains a tissue of the user's interest. Initially the input medical data set is processed by an algorithm named Automatic Peak Detection, originally presented in [18]. The outcome from this algorithm is the initial set of PRHs. This is followed by the PRH Processing step where this set is rened so that the PRHs are more likely to contain entire tissues. The classication is the nal step in the pipeline. It tries to nd the tissues that are of the user's interest among the PRHs. The a priori knowledge base that for example can contain representations of tissues and their locations relative to one another in the human anatomy is of important use. It is necessary in the classication, but it can also aect and improve the performance of the PRH processing. Finally, the visualization serves as an important tool in this pipeline. If a user wants to interact with the dierent steps of the algorithm it is necessary to get a good understanding of the current state. Dierent visualizations that present the properties of the PRHs have been developed to support the user.. 21.

(28) Interaction Modes A user can interact with the pipeline in two dierent modes. The advanced interaction mode is available for users that are well-acquainted with the method. This consists of interacting with the PRH processing phase, choosing methods and determining their parameters with the guidance and the use of tools for exploring the current PRHs. In simple mode the user relies initially on a fully automatic processing. The automatic processing is a sequence of predened algorithms and parameters that is known to work well with the current type of examination. The user can afterwards easily explore among the results of the classication. In addition, the user can post-process the classied PRHs by using an intuitive control described in section 5.3.. 4.2 A priori knowledge A priori knowledge is something that is known before the actual processing and analysis begin. In this case the a priori knowledge is regarding the input of the pipeline, the medical data set. For example, knowing the content of the input data set and how this content can be described as can both simplify and enhance the performance of the method. In other words, it plays a very important role in this method. The a priori knowledge used in this method consists of describing tissues with dierent attributes and knowing what tissues that should be present in the input data set.. 4.2.1 The description of a tissue A tissue in a medical volume data set can be described with two dierent kinds of attributes; spatial- and value-descriptive attributes. The spatial-descriptive attributes represent the geometrical and topological properties of the tissue:. • Volume • Absolute position • Shape • Relation to other known tissues in terms of position and connectedness A tissue's distribution of values is described with the value-descriptive attributes. These attributes are based on how the histogram of the tissue looks like.. • Mean value • Variance • Location of the mean in the histogram relative to other known tissues The latter mentioned attributes are more consultative since one of the fundamental problems is that the values of a tissue vary from case to case. Although,. 22.

(29) comparing with an almost identical previously made examination, the valuedescriptive attributes can give a good indication about the content of the current one. In CT scanned data, where the scale is calibrated and has a dened unit, the content of a tissue varies less from case to case and these attributes can give good advices for the processing. A tissue can have more than one description in terms of the above mentioned attributes. For example, the spatial-descriptive attributes of a tissue can vary since its geometrical properties dier from patient to patient. The type of examination, e.g. what modality is used and if some contrast material is present, highly aects the value-descriptive attributes. Suppose that an MRI scanner is used to examine the paths from the gall bladder. Then the time passed since the insertion of the contrast material and the age of the patient are parameters that will aect the value-descriptive attributes.. 4.2.2 The content of the data set Since the number of tissue descriptions can be quite numerous, the work load for an a priori based processing and analysis will be quite heavy if it is supposed to consider all those descriptions. Instead, knowing what type of examination the data set reects, and consequently knowing what tissue descriptions that can be taking into consideration, will greatly simplify this process. In medical imaging a standard called DICOM (Digital Imaging and Communications in Medicine) is used. This standard is created for facilitating the distribution and viewing of medical images. The image data and its corresponding metadata are included in a DICOM le. The metadata denes the format and describes the content of the image data. This content description should give sucient information for knowing what tissues of interest that should be present in the data set. As seen in g 4.1 the a priori knowledge is something that can aect great parts in the system pipeline. An a priori knowledge guided processing will make decisions based on what the user is looking for. I.e. a processing is made only if the result is a step closer to the nal objective. Intuitively this seems like a lot of extra calculations since there is a need to somehow evaluate these decisions. But the a priori knowledge can also be used to discard a lot of decisions. Suppose we are looking for a tissue that for sure is a high intensity region in the data set, then we could ignore all processing that involves low intensity regions. In the following sections, where each step of the pipeline is described, examples are given of how the a priori knowledge can simplify the calculations and improve the performance.. 4.3 Automatic Peak Detection The rst step that processes the input data volume is an algorithm named Automatic Peak Detection (APD). It is originally presented in [18]. In this thesis one of the steps of the algorithm is moved to the PRH Processing step in the pipeline because it conceptually ts better there. As described previously in chapter 2, a common support when manually adjusting a 1D TF is to use the histogram of the volume. The peaks of the histogram are supposed to indicate where on the intensity scale the tissues are 23.

(30) positioned. However, this is often not sucient because other regions in volume may have intensities overlapping the range of the tissue, partially or entirely obscuring its corresponding peak (g 4.2). This, together with the fact that in these cases a 1D TF entirely based on the intensity is most often not sucient for visualizing only the tissue of interest, makes it hard for the user to design a TF appropriate for his/her need. The algorithm aims to nd those peaks that correspond to the tissues of interest. This is done by subdividing the global histogram nding sub volumes represented as groups of non-overlapping neighborhoods with similar intensity content. Some of these groups, or sub volumes, will hopefully entirely or partially contain the tissues. Before describing how to achieve this state, it is important to introduce a key concept of the method.. Figure 4.2: In this data set no obvious peaks are present in the global histogram that can guide the user in the TF setting. 4.3.1 Partial Range Histogram A Partial Range Histogram (PRH) is a group of (non-overlapping) neighborhoods in the volume. The neighborhoods are grouped together because they share a similar content in terms of intensity. The shape of the neighborhood can be arbitrary, but it is important that they do not overlap each other and that they ll up the entire volume. I.e. a voxel in the considered volume must belong to one and only one neighborhood. In this thesis cubical neighborhoods are used, i.e. it is very easy to subdivide the volume into all these cubes. These cubes will be addressed as blocks through the rest of the thesis, i.e. each side of a block will always have equal length. If the size of the volume is not evenly divisible with the side of a block, the volume will be padded with extra zeros so the remaining voxels also can be considered. Now, all the blocks that are assigned to a PRH have a common property. They must all contain at least a certain amount of voxels in an intensity range. These are two properties of the PRH. A high amount, e.g. 0.95, in combination with a tight intensity range, will make only blocks with a very similar content assigned to it. Consequently, if this threshold is lower and the intensity range is larger, blocks with more diering and widely distributed content can be added, meaning that the PRH can contain more inhomogeneous regions of the volume. The amount of voxels that a block has in the intensity range I of the PRH is henceforth referred to as the range weight, wr . This can formally be expressed as |{v : f (v) ∈ I ∧ v ∈ B}| wr (B, I) = |B| where v is a voxel, f (v) the intensity of the voxel and B the set of voxels contained in a block.. 24.

(31) 4.3.2 Peak Search The objective of this algorithm is to nd all these peaks corresponding to the interesting tissues. In order to do this, an initial assumption is made; a tissue has a normal distribution of intensities. With this in mind the rst step is to nd the tissue corresponding to the highest peak in the histogram by tting a Gaussian curve to the tip of it. This Gaussian approximation is described by its height h, the mean µ and the standard deviation σ , and can be seen as an approximation of the distribution of the tissue. A block will be assigned to this peak if it has a range weight of an intensity range dened by the curve, [µ − α · σ, µ + α · σ], larger than a certain threshold ². In the case when no blocks have been assigned to the peak, the threshold will be incrementally decreased until at least one block is added. When the algorithm has found all blocks corresponding to this peak, the PRH is created. The interval, [µ − α · σ, µ + α · σ], and the nal range weight threshold ² are properties of the new PRH. The assigned blocks will then be removed from the global histogram, exposing a new peak. An identical process will be made for this peak, with the dierence that all blocks already assigned to a PRH are not available; during this algorithm a block can only be assigned to one and only one PRH. This process will iterate until the histogram has emptied. Three parameters are necessary to set prior to the algorithm; the length of a block side b, the initial range weight threshold ² and a parameter α determining the intensity range given a mean and a deviation. α = 1 in this thesis. A too large value means that blocks that should belong to dierent tissues could easily end up in the same PRH, resulting in few PRHs with many blocks assigned to them. The problem of having many tissues represented in the same PRH is harder to solve than having one tissue divided into many PRHs. This fact will become more evident in the following sections. The other two parameters should depend on what content the volume is assumed to have. To be able to capture thinner tissues, the block size needs to be so small that the highest amount of this tissue a block can contain is suciently high. Also, if the tissues of interest are assumed to be inhomogeneous, a larger block size is advisable so it can contain the wider spectra of intensities that the tissue is assumed to have. The homogeneity is also the characteristic that determines the initial threshold range weight; a lower value is needed if inhomogeneous tissues should be possible to capture. Summarized, the sequential steps of the algorithm: 1. Find the highest peak in the histogram. 2. Create a PRH by tting a Gaussian curve dened by h, µ and σ to this peak. The intensity range I is equal to [µ − σ, µ + σ]. 3. Find all blocks in the volume having wr (B, I) > ² and assign them to this peak. If no blocks are assigned, ² is incrementally decreased until at least one block is assigned. 4. Remove all assigned block from the histogram. This will expose a new highest peak. 5. Repeat steps 1-4 until the histogram is empty. 25.

(32) 4.4 Partial Range Histogram Processing To be able to determine the interesting tissues' intensity ranges and the regions in the volume where they are contained, the APD algorithm is mostly not sucient. This section describes the further processing of the output PRHs. One can describe the output of the APD algorithm as a volume of blocks where each block belongs to one PRH. This block volume can be seen as a coarsely down sampled version of the original volume but where now each sample is multi-valued, i.e. properties describing the content of the block and the PRH it is assigned to. The low resolution in combination with the simple content representation makes it easily accessible for processing and analysis.. 4.4.1 Objective The goal of the PRH Processing is to try to isolate all the blocks of a tissue into one PRH so that no other blocks than those are present in that PRH. The blocks that are assigned to a PRH contain at least a certain amount of voxels, wr , in an intensity range I specied by the Gaussian curve of the PRH. This property means that each block ts the Gaussian approximation of the PRH. An initial assumption was that a tissue in the data set has a normal distribution. If the PRH contains all the blocks of a tissue and only those, then its Gaussian approximation will quite well describe the distribution of the tissue. In practice this is rarely the case after the use of the APD algorithm. Instead we have to deal with the following three problems: 1. The same tissue can be present in two or more PRHs. 2. A PRH can contain two or more tissues. 3. A PRH can contain blocks that don't belong to a tissue of interest for the user (garbage blocks). In order to solve the above mentioned problems two dierent tools are presented that can be used either in combination or by themselves. First the PRH splitting and merging processing is presented. This kind of processing modies the set of PRH by transferring blocks between them. Henceforth the size of the blocks and their one-to-one relationship with a PRH are preserved. This is not the case in the other kind of processing which is a set of content dependent morphological operations that operates on a single PRH.. 4.4.2 Processing concepts The PRH processing makes use of some new concepts important to introduce before further reading.. PRH Block Volume The block volume of a specic PRH is a binary volume inferred from the original block volume where each block either belongs to the specied PRH or not.. 26.

(33) PRH Object Another new concept that will appear is the PRH object. A PRH object is a collection of blocks in the same PRH that are connected through 6-connectivity (g 4.3).. Figure 4.3: A PRH that has two PRH objects.. 4.4.3 Merging The merging aims to solve problem 1, i.e. collecting all the blocks of a tissue in one single PRH. Depending on the parameter setting and the data set, the APD algorithm most often produces a lot of PRHs and an interesting tissue can be present in many PRHs. The merging joins two PRHs if they fulll the criteria that are based on dierent merging attributes. These attributes of a pair of PRH are either based on the content of the PRH or on the geometrical relationship between them. Important properties of the attributes that facilitate the merging decision are the following:. • It is symmetric; Attribute(P RHA, P RHB) = Attribute(P RHB, P RHA) • Its value range is [0, 1] The symmetry of the attribute is needed because the estimation of how good the result of merging A with B should be the same as the result of merging B with A. The value range should be limited to [0,1] to facilitate the comparison between dierent merging pairs.. Content-based merging attributes These attributes describe how similar the content of the two PRH is, i.e. their histograms. This consists most often of comparing the Gaussian approximations of the PRHs; they are described with very few values that make them very fast to process.. 27.

(34) Geometrical merging attributes The geometrical merging attributes of a merging pair are determined using a description of the distribution of the blocks of each PRH. One estimates a value for how well they t together and how likely it is that they should be merged by analyzing the two distributions. This can of course be done in many ways, here some attributes are proposed.. Contact The Contact of a PRH A to another PRH B is the surface area that A has in contact with the surface of B divided by the total surface area of A. The surface area in contact is all the sides of all the blocks in A that are shared with a block in B. The nal Contact score is determined using the max operation of the two Contact values. The max operation makes the function symmetric. Figure 4.4 present a PRH pair with a low Contact value. Connectedness The Connectedness attribute uses the concept of PRH ob-. jects. The Connectedness between two PRH A and B is described semantically as how big part of PRH B each object in PRH A is connected to. The eect each PRH object's connectedness to the other PRH has on the total connectedness of the PRH is proportional to its size. Two blocks in a collection of blocks are connected when it is possible to make a path with 6-connectivity of the blocks in the collection. Thanks to these criteria the Connectedness attribute is symmetric. In gure 4.4 the two PRH has the maximum Connectedness, i.e. all blocks in each PRH is connected to all the blocks in the other PRH.. Figure 4.4: The Contact attribute of these two PRH is low, but their Connectedness is maximum.. Ellipsoid There is a need in both the processing and the later described classication step to represent the distribution of the blocks in a PRH. It is interesting to know: • What is the variance in each direction? • How strongly do the spatial dimensions correlate to each other, what is the dependency between the dimensions?. 28.

(35) All information about the dependency and variance of the dimensions of a signal is summarized within its covariance matrix. The eigenvalues and the eigenvectors of this matrix describe a Gaussian approximation of the distribution. In our case, the signal is the binary block volume (3D) and its Gaussian approximation takes the shape of an ellipsoid, a 3D ellipse. The Ellipsoid merging score of a pair of PRH is retrieved by calculating the similarity of the ellipsoid of the merged PRH with respect to the ellipsoids of the two individual PRHs. The best similarity score will set the value of the attribute. The calculation of the similarity of two ellipsoids is based on the angle between the major directions and the dierences of the quotients between the magnitudes of the major directions and the other orthogonal directions that dene the ellipsoid. These parameters can then be tuned depending on what kind of structure we are looking for. E.g. if the desirable structure is vessel-like, a larger dierence in the quotients is allowed if the dierence of magnitude is in the major direction. The Ellipsoid attribute is henceforth an example of an a priori knowledge guided processing because it is dependent on what tissue we are looking for.. 4.4.4 Splitting A splitting of a PRH tries to solve problem 2 and 3, i.e. removing blocks in a PRH that do not belong to its dominating tissue. In comparison to the merging, the splitting processing is only based on geometric criteria. E.g. a PRH can be split by letting each one of its PRH objects form a new PRH.. 4.4.5 Morphology The previously presented splitting and merging algorithms does not make any attempt to split individual PRH objects. Since a PRH object is supposed to contain all the blocks of a tissue, two object specic problems are left unsolved. Namely 1. A PRH object can contain blocks that do not belong to the tissue 2. The same tissue can be present in two or more PRH objects This section describes tools for manipulating individual PRH objects aiming to solve these two problems and fulll the objective of the processing phase. These tools are extended morphological operations. In image analysis morphological operations are used to modify and analyze the structure of an image. Usually the objective is to identify, represent and modify the shape of objects in the image. Two fundamental operators, erosion and dilation, are often used in combination in morphological processing. These operations involve two input arguments, the image and a structuring element. The structuring element is convoluted over the entire image, producing the nal output image. The shape of the structuring element also determines the shape of the objects in the output image. In the later described operations a structuring element of seven blocks is used. In g 4.5 the 2D analogue of this structuring element is shown. For further description of mathematical morphology in signal processing the reader is referred to [19]. 29.

(36) Figure 4.5: The 2D analogue of the structuring element used in this processing. Previously in this chapter, it was stated that the set of blocks of a PRH can be seen as a binary volume, where a block in the block volume either belongs to a PRH or not. A morphological operation on this set of blocks will add or delete blocks from the set. When dealing with binary images, a morphological operation is purely a combination of logical operations since the value of an element is either 1 or 0. In our case a block is also equipped with metadata that describes its content. This can be used to determine whether a block should be added or deleted from the set; the normal binary morphology is extended to content dependent morphology. The content criterion for addition or deletion of a block is based on how well it ts the content description of the entire PRH with respect to how well the rest of the blocks t to it. Concretized, the range weight of a block is compared to the mean range weight of the PRH.. Content dependent erosion If the morphological erosion operation is applied on a binary image it will shrink the existing objects in the image, i.e. the erosion deletes blocks. The formal denition as a set-theoretical operation is. B ª S = {b|(S)b ⊆ B}. (4.1). where S is the set of blocks in the structuring element and B the set of blocks in the input image. (S)b means the translation with the point b of the set S . In short, using the structuring element previously described, a block b is removed if one of its six neighbor blocks does not belong to the set of blocks in the input image. In the content dependent erosion operation a block that would have been removed by the normal erosion operation is saved if its range weight is suciently high with respect to the mean range weight. I.e.. ˜ = {b|(S)b ⊆ BP RH ∨ (b ∈ I ∧ wr (b, IP RH ) ≥ λ · w BP RH ªS ¯rP RH )}. (4.2). where BP RH is the binary volume of blocks dened by the PRH and IP RH the intensity range of the PRH dened by its Gaussian approximation. λ is a sensitivity parameter. The erosion operation aims to solve the rst of the above mentioned object specic problems by removing blocks from a PRH object whose content diers from the content of the rest of the blocks. Two tissues that have overlapping intensity ranges and located quite close to each other can end up in the same PRH object due to the size of the blocks. This object will have a loose connection between the tissues that can be opened with the erosion operation. The PRH object is then divided into two separate objects; each forming a new PRH.. 30.

No results found