Extraction and Quantiﬁcation of Features in XCT Datasets of Fibre Reinforced Polymers using Machine Learning Techniques

(1)

Extraction and Quantification of Features

in XCT Datasets of Fibre Reinforced

Polymers using Machine Learning

Techniques

Miroslav Ivanov Yosifov (Yusuf Senturk)

Miroslav Ivanov Yosifov (Yusuf Senturk)

Project: Master of Science (two years) in Computational Science and Engineering, 30 ECTS Credits Spring 2020

Supervisor: Prof. Paolo Bientinesi, Ph.D.

(2)

(3)

Abstract

This master’s thesis shows the extraction, quantification and visual anal-ysis of pores and individual fibres in fibre reinforced polymer (FRP) materials. The core methods used and advanced for this purpose are tai-lored deep learning techniques, which are coupled with interactive visu-alisation. These techniques were applied to X-ray Computed Tomogra-phy (XCT) data to extract pores and fibres of carbon (CFRP) and glass fibre reinforced polymers (GFRP). Although segmentation is widely ex-amined, there is still a high necessity to come up with improved meth-ods, especially given the huge potential of machine learning. In this thesis, we aimed at designing efficient and powerful segmentation mod-els with reasonable performance on consumer-grade GPU systems.

At the heart of the studied machine learning techniques for segmen-tation was U-Net [39], a deep convolutional neural network based seg-mentation technique. The main contributions of this thesis are seen in modifying and improving U-Net’s layer architecture to facilitate the seg-mentation of pores and fibres in 3D-XCT data of FRPs. Furthermore, a hyper-parameter optimization was completed through a parameter anal-ysis with a tuning function. The results with highest accuracy from all suitable hyper-parameters were used for the final training process. The trained model (prediction model) has finally been implemented and in-tegrated as a segmentation filter in open iA [13], in order to facilitate an efficient segmentation of XCT datasets with the fully trained model.

(4)

(5)

Acknowledgements

First of all, I want to thank both of my supervisors Prof. Paolo Bientinesi, Ph.D. and FH-Assistenzprof. DI (FH) Dr. Christoph Heinzl for challenging, supporting and motivating me during my thesis. I would like to also thank Prof. Eddie Wadbro, Ph.D to supporting me for all general information.

I would like to express my gratitude especially to FH-Assistenzprof. DI (FH) Dr. Christoph Heinzl and FH-Prof. PD DI Dr. Johann Kastner for giving me the opportunity to work in the research group of Computed Tomography.

Several colleagues contributed to this thesis in different subjects. Firstly, I would like to thank DI Bernhard Fr¨ohler for support in open iA filters and for proof-reading my thesis. My sincere thanks also go to MSc., Patrick Weinberger for his continued support with his technical knowledge. Lastly, I must thank to DI (FH) Bernhard Plank, MSc, and DI Julia Maurer for providing the XCT datasets.

The research leading to these results has received funding from the Austrian Research Promotion Agency (FFG) within the program line ”TAKE OFF” under the grant number 874540 ”BeyondInspection” and also from the Research Foundation Flanders (FWO) and the Austrian Science Fund (FWF) under the grant numbers G0F9117N and I3261-N36 ”Quantitative X-ray tomography of advanced polymer composites” respectively. This thesis also was supported partly by the project PSSP - Photonic Sensing for Smarter Processes.

(6)

SYMBOLS AND ABBREVIATES

ML Machine learning DL Deep learning

XCT X-ray computed tomography FRP Fibre reinforced polymer CFRP Carbon fibre reinforced polymer GFRP Glass fibre reinforced polymer

SOD The distance between source and object in an XCT device SDD The distance between source and detector in an XCT device DL Dice loss

DSC The Dice score coefficient

(7)

1 Introduction and Motivation

1.1 Advanced Composite Materials

Advanced composite materials such as glass or carbon fibre reinforced composites are of-ten used in the aeronautics, construction, automotive industry, and several other application areas [33], due to their superior characteristics over conventional materials. Advanced com-posite materials are typically manufactured by integrating high strength fibres as reinforce-ments into polymeric matrices. The requirereinforce-ments in all of these application areas include a high strength of the material, stiffness, as well as being cost-effective and lightweight. Lightweight materials are especially important in automotive, aeronautics and aerospace industry in order to reduce fuel consumption and thus elevate efficiency in comparison to conventional materials. Carbon and glass fibre reinforced polymer laminates (CFRP, GFRP) are therefore playing a major role in aeronautics and the evolution of aeronautic components.

The reinforcements of such materials, e.g., carbon or glass fibres, increase the strength of the final composite material, its stiffness and elasticity while decreasing weight. The properties as required by the final product determine the selection of the reinforcement ma-terial and are thus the integral part in the advanced composite system. GFRP and CFRP materials include many kinds of pores, which are prevalent and characteristic for the re-spective FRP manufacturing process [27]. The pore structure therefore describes content of the pores, pore size and pore size distribution and pore morphology [34]. So, the analysis of pore structure is essential for quality assurance. In addition to the structure, also quantitative derived parameters such as volume and shape factors need to be evaluated for an in-depth FRP analysis.

To facilitate a thorough analysis, reliable segmentation masks are required in the first place. Numerous global, local or image segmentation techniques have been presented over the years, such as k-means, watershed, and Otsu (threshold) methods. Although segmen-tation has been studied for a long time resulting in a whole body of work in this domain, there is still a high necessity to improve the segmentation methods, especially for purpose built segmentation and challenging application areas [35]. In the presented work, the main focus was put on the segmentation and analysis of CFRP and GFRP using machine learn-ing, and more specifically on U-Net [39] as deep learning method. U-Net was applied, advanced and tested on various material science application scenarios, evaluated regarding its performance as well as the quality and usefulness of its results.

The main goal of this thesis was to generate a precise and efficient segmentation method for pores and fibers with appropriate performing time and suitable models with appropriate performance on consumer-grade GPU systems with applying deep learning methods.

1.2 X-ray Computed Tomography

(10)

Chapter 1. Introduction and Motivation

rotary plate for placing the sample. Figure 1.1 shows a fully shielded industrial X-ray CT system. The arrangement of these 3 main components and especially the distances between them determines the geometrical magnification of the XCT device [43]. The distance from source to object (SOD) and from source to detector (SDD) can typically be modified, and determines the achievable voxel size together with the resolution and pixel pitch of the detector. Equation (1.1) shows that the resolution of an XCT scan is depending on both SDD and SOD. To increase the geometrical resolution of the sample in the XCT scan, either SDD should be increased, or SOD should be decreased. SOD and SDD together need to be optimized in a way that the specimen always fits onto the detector image to ensure error-free reconstructions of the sample.

V xl=SOD_SDD· Pxl; Pxl = 50µm (1.1) where Vxl is the actual size and dimension of a single voxel in the resulting back-projected volume (i.e. its spacing), and Pxl is determined as the actual size and dimension of pixels on the detector (pixel pitch / pixel spacing of the detector). Additional parameters that have a major influence on the generated XCT data and which need to be set before starting a XCT scan are: number of projection images, detector shift, binning, image size and others.

Figure 1.1: The Nanotom 180NF XCT device

1.3 Image Analysis

In order to identify, analyse and classify features in XCT datasets, image analysis is es-sential. Segmentation is an image analysis method which allows to separate an image into regions of similar characteristics, e.g., by employing similar grey values for extracting pores or fibers. Several segmentation methods may be applied for segmenting features of inter-est in XCT scans of fibre-reinforced polymers. Most of these methods are threshold based techniques.

(11)

1.4. Machine Learning

techniques, as it is a simple but effective method to separate pores or fibres from the back-ground [37]. For this reason this technique is often seen in related work on pore extraction of FRPs: Recently, e.g., Plank et at. compared different types of threshold methods applied to the segmentation of CFRPs [35]. In computer vision and image processing, the Otsu method [31] is useful to segment images or volumes. It automatically determines the thresh-old value to be used for segmentation and is best described by the category histogram-based methods [41]. It computes the threshold according to the distribution of the greyvalues: The algorithm therefore separates voxels by minimizing intra-class variance, while maximizing the inter-class variance. It works best if the histogram of the underlying data has deep and sharp valleys between the two peaks describing object and background [21]. In the pre-sented work, the Otsu method was used to determine labelled images for training, testing and validation sets.

1.4 Machine Learning

(12)

Chapter 1. Introduction and Motivation

1.5 Goals of This Work

The following tasks were defined in order to facilitate and evaluate a U-net based segmen-tation, classification and analysis of XCT data of FRPs:

• Task 1: Implement a modified 3D U-Net for an effective segmentation of pores and fibres.

• Task 2: Find an effective neural network model to implement a testing function (pre-diction) in open iA (see appendix Figure A.4 and A.5).

• Task 3: Use the generated neural network model to segment unseen data of different resolution and big volume size.

• Task 4: Visualise the generated result and compare them with reference methods (Otsu method).

• Task 5: Analyse the generated pore data quantitatively (statistically) and qualitatively (visual analysis) according to shape and volume size of pores.

(13)

2 Related Work

In the following section we describe related work about feature extraction and quantification in section 2.1, and analysis and visualisation in section 2.2.

2.1 Feature Extraction and Quantification

Feature extraction and quantification is an essential prerequisite for the quantitative and vi-sual analysis of individual voids, pores and fibres in the material sciences. Every feature contains specifically different characteristic properties. This characteristic information may contain many properties depending on the feature. 3D volume segmentation techniques are important for extraction and quantification of features, and especially for detailed determi-nation of feature characteristics in interesting areas [5, 16]. To analyse pores, properties such as shape and volume are more important, while for the fibre analysis, diameter and orientation need to be considered. These properties are computed together with their distri-butions.

Segmentation methods can be categorized into 4 sub-groups: region-, pixel-, edge-, and model-based segmentation methods [19]. Watershed transform method is an example of this group which is used for image segmentation by Beucher [3]. Pixel based segmentation is one of the easiest and fastest methods, where the segmentation just depends on the grey value level of a pixel. Thresholding methods are included in this group.

Nowadays, deep learning method is a widely used approach in image and volume seg-mentation. It has been applied firstly on medical image segmentation (virus identification, Brain MR) [39]. Successful results open new doors for other areas such as material sci-ence and linguistics. It has even been applied to detect the recent Corona Virus Disease [46] (COVID-19) from XCT images which is a common problem of the whole world right now. In the computational linguistics area, deep learning is applied for Chinese word seg-mentation, as presented by Zheng et al.[50]. The term image segmentation is a important part of the deep learning methods in 2D and 3D. Image segmentation, transfer learning and instance segmentation methods are presented and explained by Garcia et al.[14]. Trans-fer learning helps in decreasing the time consumption (helping to reach convergence). 3D Instance segmentation methods are used for segmentation and classification of multi-class segmentation with Metric learning algorithm via Lahoud et al.[24]. They used multi-task loss functions which are called Feature Embedding Loss, Directional Loss and Joint loss.

2.2 Analysis and Visualisation

(14)

Chapter 2. Related Work

fibres via separating the fibres from each other to determine characteristic features. They applied Gaussian filtering, gradient magnitude, Hesse matrix computation and medial axis extraction coupled with tailored heuristics to extract and characterize individual fibres. Fur-thermore, it has been shown by Oberpeilsteiner et al. that Finite Element method could be applied to the simulation and analysis of short fiber reinforced materials [30]. Porosity maps as proposed by Reh et al. can be used for the characterization of pores in CFRP [37]. Feature Scout (aka. Fibre Scout) [48] is a tool for visualising and analyzing feature characteristics with parallel coordinate plots and scatter plot matrices linked to a 3D rep-resentation of the analysed specimen (see Appendix Figure A.3). Visual reprep-resentations of different data such as spatial and derived data for visual computing in material science are presented by Heinzl and Stappen [15].

(15)

3 Background

3.1 Otsu Threshold

In this section, the mathematical background for the Otsu Threshold method is discussed. As it was mentioned earlier, thresholding is straight forward, simple, effective and most importantly easy to apply for segmentation [31]. Now, we will define Otsu’s method from a mathematical point of view starting with thresholding [1]. First, the number of the pixels is defined by:

n= ∑L−1i=0 ni (3.1)

where n is total number of the pixels, and nistands for the number of pixels having the grey

value i. L is the count of different grey values. Note that this assumes that the image has a fixed number of discrete grey values (e.g., 0..255 or 0..65535). The grey level histogram is normalised and observed as a probability distribution through the following equation:

h(i) =ni

n (3.2)

The grey values range from 0 (black) to L-1 (white). It is worth to mention here, there are many ways to determine the threshold, which separates the object from the background. Given a threshold t, we can define and determine the probability of the two classes as follows:

ω1(t) = ∑ti=0h(i) ω2(t) = ∑L−1i=t+1h(i) (3.3)

The mean and variance of the foreground and background are respectively defined in the following equations:

µ1(t) = ∑ti=0ih(i), σ21(t) = ∑ t

i=0(i − µ1(t))2h(i)

µ₂(t) = ∑L−1_i=t+1ih(i), σ₂2(t) = ∑L−1_i=t+1(i − µ2(t))2h(i)

(3.4)

The intra (within) class variance (σ2w) and the inter (between) class variance (σ2B) are defined

as follows:

σ2_w(t) = ω1(t)σ21(t) + ω2(t)σ22(t)

σ2_B(t) = ω1(t) (µ1(t) − µT(t))2+ ω2(µ2(t) − µT(t))2

(3.5) To simplify the equation, another version of inter-class variance is shown in the following equation:

σ2_B(t) = ω1(t)ω2(t) (µ2(t) − µ1(t))2 (3.6)

(16)

Chapter 3. Background

Figure 3.1: Grey level value histogram with a threshold from Otsu method.

3.2 Sørensen-Dice Similarity Coefficient for Image Segmentation

The Sørensen-Dice similarity coefficient was independently and separately developed by Thorvald Sørensen [45] and Lee Raymond Dice [9]. It is used to compare two sets X and Y and defined as follows:

dice(X ,Y ) = 2|X ∩Y |

|X| + |Y | (3.7) where |X | represents the cardinality of set X. In our case, the Dice similarity coefficient (DSC) is used to determine the segmentation accuracy between input data and ground truth (reference segmentation).

3.3 Loss Functions for Unbalanced Data

As DSC determines the segmentation quality, we also used DSC as a loss function in the training process. The loss function is crucial for the training process to learn properly from the given(input) data. A formulation of the Dice loss [28] for 2 classes that can be used as loss function presents itself as follows:

DL2= 1 − ∑ N n=1pnrn+ ε ∑Nn=1pn+ rn+ ε −∑ N n=1(1 − pn) (1 − rn) + ε ∑Nn=12 − pn− rn+ ε (3.8) The ε term is used to avoid divisions by 0. A generalisation of the Dice Loss function (GDLF) is presented by Crum et al. [7] as a way of evaluating multiple class segmentation with a single result.

GDLF = 1 − 2 ∑

2

l=1wl∑nrlnpln

∑2l=1wl∑nrln+ pln

(3.9) where wl is expressed to define invariance to various label value properties. The notation

GDLFv is adapted while wl = 1/ ∑N_n=1rln

2

(17)

3.4. The 3D U-Net Architecture

chose the GDLFvas weighting, the contribution of each label is corrected by the inverse of

its volume. Therefore, it reduces the correlation between region size and Dice score [44]. In training, the gradient(with stochastic gradient descent) with respect to piis:

∂ GDLF ∂ pi = −2 (w2₁−w2 2)[∑Nn=1pnrn−ri∑Nn=1(pn+rn)]+Nw2(w1+w2)(1−2ri) [(w1−w2) ∑Nn=1(pn+rn)+2Nw2] 2 (3.10)

3.4 The 3D U-Net Architecture

U-Net was originally developed for biomedical image segmentation at the computer science department of Freiburg university Germany, in 2015 [39]. The network architecture consists of two parts: a contracting path (analysis) and an expansive path (synthesis). It features a U-shape and contains 52 layer in total. The architecture starts with the image input layer, which is followed by encoder, decoder, final convolution, softmax and output layers. Each encoder includes two sets of convolutional layers, two batch normalisation layers, two ReLU layers and a 2-by-2-by-2 max-pooling layer. Each decoder sub-network contains respectively two sets of convolution layer, two batch normalisation, two ReLU layers and a upsampling layers (deconvolution). The decoder can be seen as reverse operation of the encoder.

(18)

Figure 3.2: Schematic of the U-Net architecture

3.4.1 Notation for the Definition of Layers

Assume that we have the l-th layer, the inputs form an order 3 tensor xl ∈ RHl_×Wl_×Dl

. Therefore, we use three index sets il, jl, dl to hold a location for each specific value in xl. The index set il, jl_{, d}l_{addresses one value in x}l_{, which is in the d}l _{-th channel, and at}

spatial location il, jl (with other words, at the il-th row, and jl-th column. The separation of training data into small batches is called the mini batch strategy and typically used in learning algorithms. In addition, we need to modify in this case xl with an order 4 tensor in RHl×Wl×Dl×N. N is representing the mini-batch size. We assume that N = 1 in this section. The following assumptions and notations are used to simplify the notations and determinations:

• We use zero based indexing convolution which means that 0 ≤ il< Hl, 0 ≤ jl < Wl and 0 ≤ dl< Dl

• Notice that y and xl+1in fact refer to the same object due to the transformation of the input xl to an output y. This is also the input to the next layer.

• We assume that the output size of the result is Hl+1× Wl+1_{× D}l+1_{, and an element}

value in the output result is indexed by il+1, jl+1, dl+1 , 0 ≤ il+1< Hl+1, 0 ≤ jl+1< Wl+10 ≤ dl+1< Dl+1

(19)

3.4.2 Rectified Linear Unit (ReLU)

ReLU is the simplest layer in the network architecture. A ReLU layer operates a threshold for each element of the input data. ReLU layers set their output values to zero when their input value is negative. ReLU always has the same gradient [26]. A ReLU layer is linear and inexpensive to use in the architecture, as compared to a Sigmoid layer (see figure 3.3 and 3.6). It is also easy to calculate, we only need to determine max(0,input) value.

Figure 3.3: Plot showing the function represented by a ReLU layer.

A ReLU layer does not modify the size of the input data [49], therefore, xl and y have the same size. It can be defined for every value of the input data as in the following:

yi, j,d= max

n 0, x_{i, j,d}l

o

(3.11) with 0 ≤ i < Hl = Hl+1, 0 ≤ j < Wl= Wl+1, and 0 ≤ d < Dl = Dl+1. If we re-write the equation 3.11, it becomes:

dyi, j,d dxl

i, j,d

=hxl_{i, j,d}> 0i (3.12)

According to equation 3.12, this function is returning 0, if its argument is negative, passes through the input otherwise. Hence, we can write the function

h ∂z ∂xl i i, j,d= ( h ∂z ∂y i i, j,d if x l i, j,d> 0 0 otherwise (3.13)

There is a small issue in equation 3.13 in the theory. The function is not differentiable at x= 0. However, this is not a problem in practice.

3.4.3 Convolutional Layer

(20)

The extracted features from input are defined by the training of the network. With other words, convolutions layers extract the features from the input volume, ReLU decides if these features are useful or not.

Figure 3.4: Schematic representation of a convolutional layer.

The convolutional layer’s functionality is shown in Figure 3.4. A convolutional layer contains a filter which scans all input data in a kernel of specific size and determines thereof the respective values output matrix. More details on the mathematical background is pre-sented by Krizhevsky et.al [23]. Let’s assume that we have a volume of size H1×W1_{× D}1_.

Four hyper-parameters are then presented in the formula: the number of filters is represented by K, their spatial extent by F, the stride as S, the amount of zero padding is P. Determi-nation of the output volume of size of the convulutional layer H2× W2_{× D}2 _{is shown in}

following equations:

H2= H1_{− F + 2P /S + 1} _(3.14)

W2= W1− F + 2P /S + 1 (3.15) Equation 3.14 and 3.15 show that width and height are determined by equivalent by the symmetry

(21)

depth slice (of size H2×W2_{) is intertwine a result of performing a valid convolution of the}

d-th filter over scanning the input volume with a stride (step size) of S, and then offset by d-th bias.

The following equation shows the general formula of Convolutional layers (note that stride=1) (for a detailed explanation see [49]):

y_il+1_{, j}l+1_,d= ∑H_i=0_∑W_j=0_∑D l

dl₌₀f_{i, j,d}l_,d× xl

il+1_{+i, j}l+1_{+ j,d}l (3.17)

Equation 3.17 is replicated for all 0 ≤ d ≤ D = Dl+1, and for all spatial locations il+1, jl+1 satisfying 0 ≤ il+1< ¯Hl− H + 1 = Hl+1_{, 0 ≤ j}l+1_{< W}l_{−W + 1 = W}l+1_{. In this equation,}

xl

il+1_{+i, j}l+1_{+ j,d}l assigns for the element of x

l _{indexed by the triplet i}l+1_{+ i, j}l+1_{+ j, d}l_a

bias term bd, which is usually added to yil+1, jl+1_,d.

3.4.4 Max Pooling

Simply explained, max pooling layers take the maximum value of the input which comes from the ReLU by convolving a filter over an image (see Figure 3.5). It is important to reduce the amount of parameters and computation time by decreasing the spatial size in the CNN.

To explain the max pooling layers, the same notation is used as with the ReLU and convolution layers. Let us assume that xl∈ RHl_×Wl_×Dl

is the input data to the l-th layer. Now we use this as a pooling layer. There is no learning parameter in this layer , therefore tensor wiis null [49]. Assume that strides are defined respectively in the vertical and horizontal by Hdivided by Hland W divided by Wl. The output value ( xl+1) of the pooling will be again 3 tensors of size Hl+1×Wl+1_{× D}l+1_{with strides}

Hl+1= H_Hl, Wl+1=W_Wl, Dl+1= Dl (3.18) A pooling layer performs a xl channel by channel independently. The matrix with size Hl× Wl _{elements are divided into H}l+1_{× W}l+1 _{regions in each channel. Each}

sub-region size is H ×W . The pooling operator then maps a sub-sub-region into a single number (for details, see [49]). There are two kinds of operators in pooling: max pooling and average pooling. While the max pooling operator maps a sub-region to its maximum value, the average pooling maps a sub-region to its average value. The mathematical expression is shown in the following equations:

max : y_il+1_{, j}l+1_,d= max_{0≤i<H,0≤ j<W}xl

il+1_{×H+i, j}l+1_{×W + j,d}

average : y_il+1_{, j}l+1_,d= 1

HW∑0≤i<H,0≤ j<Wxlil+1_{×H+i, j}l+1_{×W + j,d}

(3.19)

where 0 ≤ il+1< Hl+1, 0 ≤ jl+1< Wl+1, and 0 ≤ d < Dl+1= Dl_{. In our modified CNN,}

max pooling layers were used. 3.4.5 Up Sampling

In the up sampling layers, interpolation is used for connecting coarse outputs to dense pixels. Using a linear mapping, simple bilinear interpolation calculates each output yi, j from the

(22)

Figure 3.5: Schematic representation of a max pooling and upsampling layer.

3.4.6 Sigmoid Function

The Sigmoid function is an activation function which determines output values in a range from 0 to 1 [47]. Due to its nature, it can be used only for two classes. Therefore, this function is often used for artificial neural networks that require an output values in the interval of 0 to 1 (see Figure 3.6). The function is formulated in the following expression:

f(x) =_1+e1−x (3.20)

This function is important to decrease the difficulty in learning CNN parameters and improving its accuracy [49]. For a multi-class segmentation, the Softmax function can be used as activation function [10].

(23)

4 Method

4.1 XCT Data Generation

All of the samples were scanned using a sub-µm-XCT laboratory device, a GE Nanotom 180 NF. This device is in operation at University of Applied Sciences Upper Austria, Campus Wels, since February 2008. A picture of the device is shown in Figure 1.1. The system includes an X-ray tube which can be operated with various focal spot sizes at a maximum of 180 kV. To keep the scanned specimen at a controlled temperature, a cooling system was added to the tube housing. The Hamamatsu detector [8] uses the scintillation principle, has 2304 × 2304 pixels with an edge length of 50 µm and a dynamic range of 12 bit. The software tool “datos −x2 acquisition”(2.2.1 RTM) was used for acquisition of projection images and for setting the scanning parameters.

4.2 Samples and Data Characteristics

4.2.1 Carbon Fibre Reinforced Polymers

(24)

Chapter 4. Method

Figure 4.1: (a) 3D views of Sample 1, Sample 2 and Sample 3. (b), (c) and (d) respectively show axis-aligned 2D slices of Sample 1, 2 and 3.

Table 4.1 Dataset information and usage for pore segmentation of CFRP samples Resolution Size Data type Usage Sample 1 3.3 µm 1220x854x976 unsigned 16 bit Training Sample 2 3.3 µm 1220x854x976 unsigned 16 bit Prediction Sample 3 10 µm 366x244x244 unsigned 16 bit Prediction

4.2.2 Glass Fibre Reinforced Polymers

(25)

4.3. Modification of U-Net Architecture

Figure 4.2: (a) 3D views of Sample 4, Sample 5 and Sample 6. (b), (c) and (d) respectively show axis-aligned 2D slices of Sample 1, 2 and 3.

Table 4.2 Dataset information and usage for fibre Segmentation of GFRP samples Resolution Size Data type Usage Sample 4 2.7 µm 1220x854x976 unsigned 16 bit Training Sample 5 2.7 µm 610x610x610 unsigned 16 bit Prediction Sample 6 3 µm 632x632x632 unsigned 16 bit Prediction

4.3 Modification of U-Net Architecture

The first trial was performed with a model which is based on the original version of 3D U-Net. The original U-Net (CNN) prediction model was successful in segmenting subvolumes with small size (such as 122x122x122), however it was not able efficiently segment larger volumes (such as 1000x1000x1000). Even when doing hyper-parameter optimization, all of the trained models failed to segment input efficiently in case of larger volumes. One of the results of CNN prediction from trained model is shown with a comparison to the modified CNN prediction in Appendix (see Figure A.2). Note that same hyper-parameters are used in both models with small differences in the training and validation dataset.

(26)

Chapter 4. Method

To fulfill Task 1, we implemented a modified U-Net architecture in Python using Keras and Tensorflow. The network architecture consists of two parts: a contracting path (en-coders) and an expansive path (de(en-coders). It contains 46 layers in total. A schematic figure of the architecture is illustrated in Figure 4.3.

Figure 4.3: Schematic of the modified U-Net architecture

The contracting parts start with an input layer and continues with encoder stages. Each of the encoders contains two 3x3x3 convolutions, followed by a rectified linear unit (ReLU) and a 2x2x2 max pooling layer. The first decoder section of the expansion stage contains an up-sampling, a convolutional layer and a ReLU layer. Each decoder stage contains three sets of convolutional layers, three ReLU layers and an up-sampling layer. The last decoder stage also contains an activation function, which is connected to the output layer by a Sigmoid function. Each encoder stage is connected with a decoder stage, also called concatenation. In the encoder stages the input data is analysed, the decoder stages predict the output data. The network architecture input size is 132x132x132, and the output size (prediction) is 122x122x122. A method employing overlapping tiles is used to improve the prediction accuracy at the borders of the test volumes. Each test volume is overlapping with 5 voxels in every dimension which explains the difference between input and output size. The main differences between the original U-net and the proposed modified U-Net are therefore found in the following points:

(27)

4.4. Data Preprocessing and Training

• In the expansive path, in each decoder one more convolution layer and ReLU layers were added.

• Instead of a softmax function, a sigmoid function was used. • Input size and output size were modified.

Normally, batch normalisation layers improve the training speed, performance, and sta-bility of convolutional neural networks for 2D input. For 3D input, it is not possible to choose a large batch size number because of memory limitations, the maximum possible value to use on our hardware was 3. Therefore, the result did not differ so much whether batch normalisation layers where used or not. A Sigmoid function is used instead of a Soft-max function, because we only performed binary segmentations. SoftSoft-max functions can be used for multi-class segmentation, but its complexity unnecessarily increases the train-ing time. The changes done in the expansive path resulted from the observations of our experimental results.

4.4 Data Preprocessing and Training

The modified CNN predicts a volume with a shape of 122x122x122 voxels to reduce mem-ory consumption in the training process. Pre-processing of the data starts with a normali-sation of the input volume and Otsu method segmentation. Experiments with the multiple Otsu method have been performed. However, we observed that multiple Otsu resulted in over-segmentation, see Appendix, Figure A.1. Therefore, it was decided to only use the sin-gle Otsu threshold. After normalisation, 5 voxels mirror-padding are added, and the XCT scan is split into sub-volumes with an overlapping of 5 voxels (132x132x132 input volume size). The labeled image was also split into sub volumes for training (Figure 4.4). The whole pre-processing was implemented in python1. For training, we used only the sub-volumes of Sample 1, which were split into 2 parts: 80% of the sub-volumes for training and 20% for test. From training sub volumes, 20% of the training data were used for validation.

Figure 4.4: Overview of data preprocessing and training.

(28)

Chapter 4. Method

To determine suitable parameters for the training process, a heuristic hyperparameter tuning with Talos2 _{was performed. The ”Adam” optimizer[22] was used with a learning}

rate of 4e-5. For this training, it was important to have a loss function that can handle highly imbalanced classes (only 3,75% of the volume represent pores in the training data). The overlap measures proposed by Sudre et al. [8] are suited best for a highly imbalanced dataset. The Dice coefficient was used as an overlap measure for the training’s loss. After the training, the results have been evaluated with the test dataset.

4.5 Integration into open iA

To fulfill Task 2, we implemented a deep learning segmentation filter in open iA. This al-lows users with no programming knowledge to use and apply the proposed modified U-Net. open iA was extended to support the industry standard format onnx (Open Neural Network Exchange)3. It performs segmentation by running a prediction based on our modified 3D U-Net architecture with a chosen pre-trained model. Figure 4.5 shows the series of steps a user needs to perform in the software to execute this segmentation in open iA:

Figure 4.5: Deep learning segmentation filter in Open iA

2_{Autonomio Talos Hyperparameter Optimization for Keras https://github.com/autonomio/talos}

(29)

4.6. Quantitative Data Analysis of Extracted Features

To segment any kind of data in 3D, first choose Filter → Segmentation → AI. Then select the pre-trained prediction model in onnx format4_{(see also A.4,A.6 and A.7 ). Figure}

4.5 also shows the input data and output segmentation results.

4.6 Quantitative Data Analysis of Extracted Features

(30)

(31)

5 Results and Evaluation

In our first results the predictions of our initial model based on the original version of 3D U-Net are presented in comparison with the results of Otsu thresholding, which was used as the reference segmentation method. Here, U-Net was used as a prediction model. The outcomes suggested that U-Net is not able to efficiently segment features in XCT data (Appendix Figure A.2). To more efficiently segment XCT data, we further modified U-Net, as described in the previous chapter (see section 4.3).

The Sørensen–Dice coefficient function (section 3.2) was used for evaluation of the segmentation accuracy and loss function for both training and validation (Table 5.1).

Table 5.1 Training Results

Training Loss Training accuracy Val. Loss Val. accuracy Results CFRP 0.9848 0.9991 0.9825 0.999

Results GFRP 0.9307 0.9871 0.9286 0.9854

open iA was used for normalisation of XCT input and reference data (Otsu segmenta-tion) both in the training and the testing process for all the samples. The normalisation of XCT scan data has been completed by re-scaling the grey values to a range between 0 to 1. The reference data segmentation were binarized into 2 classes with a value of 1 for pores and O for the background matrix material. The average Dice score of pores was determined as 0.9990 on 112 testing datasets (sub-volumes). The average Dice score of fibres (accu-racy) was determined as 0.985 on 130 testing datasets (sub-volumes). The testing dataset’s input size for each volume was 122x122x122.

In the following sections, we present the generalizability our prediction results for pore segmentation (section 5.1), and for fibre segmentation (section 5.3).

5.1 Visualisation and Visual Inspection Analysis of Pores

According the task 4, in this section, it is shown visualisation of the generated result and comparison of them with reference methods (Otsu method) of pore segmentation.

5.1.1 Sub-volume: 3.3 µm

(32)

segmen-Chapter 5. Results and Evaluation

appearance into rounded, long and thin, and into big shapeless pores. It has been found that the result was similar in both Otsu and the neural network, however when big pores were compared, small differences were detected between Otsu and our neural network model (Figure 5.1 (a,b), zoomed region).

Figure 5.1: 2D and 3D XCT slices of one of the sub-volumes from Sample 2. (a) shows the input sub-volume and its predictions from the modified U-Net (yellow) overlaid on Otsu segmentation (blue) in 3D. (b), (c) and (d) respectively show axis-aligned 2D slices of predictions overlaid on the input volume and Otsu method. The detail images show a 4x zoom of the edge region marked in red). Yellow colour represents the predictions. Blue colour represents ground truth segmented with Otsu method. Dark grey colour represents regions where the prediction and the ground truth agree.

(33)

5.1. Visualisation and Visual Inspection Analysis of Pores

Figure 5.2: Binary version of Figure 5.1 (blue: Otsu segmentation, yellow: neural network prediction, grey: Otsu and modified U-Net predictions agreed, black: background).

5.1.2 Prediction of Sample 2 (3.3 µm) and Visual Inspection Analysis

(34)

Chapter 5. Results and Evaluation

Figure 5.3: 2D XCT slices visualisation of the xy-axis from Sample 2. (a), (b), (c) and (d) respectively show input image, Otsu segmentation (blue) overlaid on input image, pre-diction (yellow) from modified U-Net model overlaid on input image and binary prepre-diction overlaid on Otsu segmentation. Region 1 and 2 are shown enlarged in the figures below.

(35)

Figure 5.5: Zoomed (8x) version of region 2 (blue: Otsu segmentation, yellow: modified U-Net prediction).

Figure 5.6: Zoomed (8x) version of the binary image comparison between modified U-Net prediction and Otsu segmentation (blue: Otsu segmentation, yellow: modified U-U-Net prediction, grey: modified U-Net and Otsu agree, black: background).

All in all, our data shows that Otsu method was not able to fully segmented all kind of shapes and the lower resolution images suggest that modified U-Net prediction performance is better than the reference method.

5.1.3 Prediction of Sample 3 (10 µm) and Visual Inspection Analysis

(36)

Figure 5.7: 2D XCT slices of the y-z plane from Sample 3. (a), (b), (c) and (d) respectively show the input image, Otsu segmentation (blue) overlaid on the input image, prediction (yellow) from modified U-Net overlaid on the input image and the binary prediction over-laid on the Otsu segmentation.

(37)

(38)

5.2 Quantitative Analysis of Pore Segmentation

To fulfil task 5, quantitative (statistical analysis) and qualitative (visual analysis) analysis are presented in this section. The statistical results are presented in the following 3 tables according to the introduced samples. Sample 2 was cut into two parts due to the large dataset size. One half of the volume was used from Sample 2 to analyse the properties and shapes of the pores. Table 5.2 and 5.3 shows similar behavior, in that pore percentages are most similar in both methods (with prediction showing a bit higher values). There are however distinct differences in the number of segmented pores. The modified U-Net is able to segment more pores in comparison to Otsu thresholding: In the visualisation section, an under-segmentation problem is shown for the Otsu thresholding on this sample. This data explains that even small differences in the percentage of the pore segmentation may result in under-segmentation (Table 5.2, 5.3 and 5.4). Interestingly, Sample 3 shows different results than sample 2 in terms of the number of segmented pores for prediction and Otsu method (Table 5.3 and 5.4). The percentage of voxels segmented as pores is different between Otsu and the modified U-Net (with a difference of 2%) (see Table 5.4). Although the CNN results show a higher volume percentage for pores, Otsu has segmented more pores than the modified U-Net because Otsu thresholding segmented single pores as divided pores especially in lower resolution images (Figure 5.11). The data suggests that while the modified U-Net is segmenting a single pore, Otsu thresholding identifies several pores in the same region.

Table 5.2 Statistics for sub-volume from Sample 2

Prediction Otsu How much percent of the volume is pore? 5.7% 5.4% How many pores are segmented? 33 21

Table 5.3 Statistics for Sample 2 (half volume)

Table 5.4 Statistics for Sample 3

(39)

5.2. Quantitative Analysis of Pore Segmentation

geometric properties of the pores such as volume and roundness (shape). The FeatureScout tool (see appendix, Figure A.3) from open iA was then used for the visual analysis and classification.

Figure 5.9: Visualisation of Sample 2 (half size). (a), (b), (c), (d) respectively show the modified U-Net prediction, connected component, pores grouped into 3 classes according to shape and pores grouped into 4 classes according to volume.

5.2.1 Volume Analysis

Figure 5.10 shows the visualisation results from analysing Sample 2 (half size) for the dif-ferent volume size of the pores with using the modified U-Net predictions.

Figure 5.10: (a), (b), (c), and (d) respectively show 4 different classes which group the pores according to their volume. Each colour represents a class, except black, which rep-resents rendering artifact. (a) Class 1 (blue): 0-40 voxels, (b) Class 2 (green): 40 − 103 voxels, (c) Class 3 (yellow): 103− 104_{voxels, (d) Class 4 (red): 10}4_{− 10}6_voxels.

(40)

Figure 5.11: 2D slices taken from Sample 3 (blue: Otsu segmentation, yellow: modified CNN prediction).

Table 5.6 Volume analysis statistics for subvolume

Bar Plot Number of pores

Otsu (GT) Prediction Class 1 0 9 Class 2 10 8 Class 3 10 13 Class 4 3 3 C1: Class 1: v < 40 C2: Class 2: 40 < v < 103 C3: Class 3: 103< v < 104 C4: Class 4: 104< v < 106 v: volume(voxel)

Table 5.7 Volume analysis statistics for Sample 2

(41)

5.2. Quantitative Analysis of Pore Segmentation

Table 5.8 Volume analysis statistics for Sample 3

Otsu (GT) Prediction Class 1 2852 1427 Class 2 1307 1599 Class 3 146 234 Class 4 7 10 C1: Class 1: v < 40 C2: Class 2: 40 < v < 103 C3: Class 3: 103< v < 104 C4: Class 4: 104< v < 106 v: volume (voxel) 5.2.2 Shape Analysis

Figure 5.12 shows a visualisation of the different classes according to the shape of the pores from modified U-Net predictions from analysed Sample 2. The pores were classified into rounded pores (blue), long and thin pores (yellow), and uncertain pores (green).

Figure 5.12: (a), (b), (c) respectively shows rounded pores (Class 1: blue), long and thin pores (Class 2: yellow) and complex pores (Class 3: green).

(42)

Table 5.9 Shape analysis statistics for subvolume

Prediction Otsu (GT) Class 1 5 3 Class 2 10 13 Class 3 8 17 C1: Class 1: r < 0.21 C2: Class 2: 0.21 < r < 0.5 C3: Class 3: 0.5 < r < 1.8 r: roundness

Table 5.10 Shape analysis statistics for Sample 2

Otsu (GT) Prediction Class 1 1020 1181 Class 2 1884 1818 Class 3 10020 1352 C1: Class 1: r < 0.21 C2: Class 2: 0.21 < r < 0.5 C3: Class 3: 0.5 < r < 1.8 r: roundness

Table 5.11 Shape analysis statistics for Sample 3

(43)

5.3. Visualisation and Visual Inspection Analysis of Fibres

Figure 5.13: 2D slices taken from Sample 3 (blue: Otsu segmentation, yellow: Modified U-Net prediction pores which smaller than 40 voxels

The figure 5.13 shows an example slice from Sample 2 which contains a pore with a size of 3 voxels. This pore is segmented only by the modified U-Net.

5.3 Visualisation and Visual Inspection Analysis of Fibres

In the following section, we evaluate GFRP samples 5 and 6. The modified U-Net was trained on Sample 4 and tested on samples 5 and 6. Note that Sample 5 and 6 have different voxel size and resolution. To fulfil task 4, we show the generated results and a comparison of them with reference methods (Otsu) regarding fibre segmentation.

5.3.1 Prediction of Sample 5 (2.7 µm) and Visual Inspection Analysis

The results for Sample 5 (GFRP) again shows that modified U-Net predictions are clearly better than Otsu thresholding (see Figure 5.14 and 5.15). The observed behaviour of the Otsu method was similar as in pore segmentation. It is clearly under-segmenting fibres for Sample 5.

(44)

Figure 5.15: Zoomed (8x) version of the region 4 (blue: Otsu segmentation, yellow: mod-ified U-Net prediction).

A 3D visualisation of the segmentation using Otsu Threshold and the modified U-Net method are presented in Figure 5.16.

Figure 5.16: 3D visualisation of the fibre segmentation (Blue: Otsu segmentation, yellow: modified U-Net prediction).

5.3.2 Prediction of Sample 6 (3 µm) and Visual Inspection Analysis

(45)

5.4. Quantitative Analysis of Fibre Segmentation

Figure 5.17: 2D slice visualizations of the xz-plane from Sample 6. (a), (b), (c) respectively show input image, Otsu thresholding (blue) overlaid on the input image and the prediction (yellow) from modified U-Net as overlay.

Figure 5.18: Zoomed (8x) version of the region 5 (blue: Otsu thresholding, yellow: modi-fied U-Net prediction) taken from Figure 5.17.

5.4 Quantitative Analysis of Fibre Segmentation

As fibres were intertwined in the data, it was impractical and/or impossible for the users to analyse them. In order to facilitate a single fiber analysis, a technique for fibre separation was required. To do that, connected component and binary thinning filters as included in open iA were used. Separated fibres are shown in Figure 5.19. Table 5.12 and 5.13 presents the percentages of the volume and the total fiber count. The main differences in the results of Sample 6 are that Otsu method has 5 percent more segmented fiber volume than the modified U-Net. This renders another proof of the over-segmentation of the Otsu method. Table 5.12 Statistics for Sample 5

(46)

Table 5.13 Statistics for Sample 6

Prediction Otsu How much percent of the volume is fibre? 25.7% 30.7% How many fibres are segmented? 1108740 1196878

Figure 5.19: 3D visualisation of the fibre segmentation from Sample 5: (a) 3D renderings, (b) binary thinning, (c) fibers are colored by their orientation.

5.5 Performance

The training was performed using a Nvidia Quadro RTX 6000 with Keras12.3.0 and Ten-sorflow2-GPU 2.1.0 in Python3. The training over 10 epochs with 358 training samples, 90 validation samples and a batch size of 3 has taken approximately 1 hour. For the predic-tion, a different system with a consumer-grade GPU (Nvidia RTX 2060 Super) was used. The prediction of a volume with dimensions of 1220x854x986 (560 sub-volumes) takes approximately 4 minutes in open iA using the ONNX runtime4. This time includes the normalisation, the splitting into sub-volumes, the prediction, and the creation of the result volume out of the sub-volumes (Task 6).

1_{Keras, Chollet, Franc¸ois and others, 2015. Software available from https://keras.io.} 2_{TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. tensorflow.org.} 3_{Python Software Foundation. Python Language Reference, version 3.6. http://www.python.org.}

4_{ONNX Runtime is a performance-focused inference engine for ONNX (Open Neural Network Exchange)}

(47)

6 Discussion

In this work, we developed a new segmentation method using deep learning algorithms. The quantification of the feature statistics and the visual analysis suggested that our ap-proach is effective and robust regarding the segmentation of pores or fibres, especially on low resolution datasets. The segmentation results of pores and fibres in XCT dataset were better when modified U-Nets were used than using Otsu thresholding. Other segmentation methods were also tested, such as Multiple Otsu, K-means and watershed (data is shown in Appendix in Figure A.1). All together, the outcomes from visual inspections suggested that among all those methods, Otsu thresholding is the most applicable and versatile method for the segmentation of fibres and pores in XCT data. The network is able to segment ob-jects with different resolutions even in the barely visible pores and fibres, such as 10 µm resolution, which the network was not trained on.

When applying deep learning algorithms, the input data quality and the ground truth properties are crucial for learning the right information in the trained network model (pre-diction model). Likewise, to improve the pre(pre-diction model, we need to improve the quality of the input dataset.

The quantitative analysis of the features evidences that shape and volume of the pores are critical properties for the segmentation. Segmentation of lower resolution images are most challenging in all methods. Separating the different labeled regions is arduous even by human visual inspection. Indeed, the resolution of the volume or the image has to be suf-ficient for separating fibres and pores from the background. Therefore, both neural network and Otsu method segmentation quality gets worse in the lower resolution images. It was observed that this situation is worse with the Otsu Threshold method, as shown in Section 5.1. The grey value peaks are relatively close to each other in the low resolution image.

(48)

(49)

7 Conclusion

As shown in this thesis, the modified U-Net approach demonstrates a reasonable perfor-mance also with a small set of training data. Comparing with the results in chapter 5, it can be seen quantitatively (statistically) and qualitatively (visual analysis) that modified U-Net performs better than Otsu method for pore and fibre segmentation. The network also showed a reasonable performance on the datasets with lower resolution (µm), on which it was not trained. The modified U-Net model also handles big datasets with a size of over 1000x1000x1000 (Approx 4GB). This process takes approximately 4 minutes on cunsumer-grade GPUs.

(50)

(51)

References

[1] M. Al-Bayati and A. El-Zaart. Mammogram images thresholding for breast cancer detection using different thresholding methods. Advances in Breast Cancer Research, 2:73–77, 2013.

[2] J. Ba, V. Mnih, and K. Kavukcuoglu. Multiple object recognition with visual attention. ICLR, 2:1–10, 2014.

[3] S. Beucher and C. Mathmatique. The watershed transformation applied to image seg-mentation. Scanning. Microsc., 6, 2000.

[4] S. Carmignato, W. Dewulf, and R. Leach. Industrial X-Ray Computed Tomography. Springer-Kindle Edition, 2018.

[5] Ö. Ç içek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. Medical Image Com-puting and Computer-Assisted Intervention – MICCAI. Lecture Notes in Computer Science. Springer, Cham, 9901:424–432, 2016.

[6] D. C. Ciresan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. CoRR, pages 3642–3649, 2012.

[7] W. R. Crum, O. Camara, and D. L. G. Hill. Generalized overlap measures for evalua-tion and validaevalua-tion in medical image analysis. IEEE Transacevalua-tions on Medical Imaging, 25(11):1451–1461, 2006.

[8] Hamamatsu datasheet. Flat panel sensor. https://www.hamamatsu.com.

[9] L. R. Dice. Measures of the amount of ecologic association between species. Ecology, pages 297–302, 1945.

[10] K. Duan, S. S. Keerthi, W. Chu, S. K. Shevade, and A. N. Poo. Multi-Category Clas-sification by Soft-Max Combination of Binary Classifiers. Springer Berlin Heidelberg, 2003.

[11] S. Bakas et al. Identifying the best machine learning algorithms for brain tumor seg-mentation, progression assessment, and overall survival prediction in the BRATS chal-lenge. CoRR, abs/1811.02629, 2018.

(52)

References

[14] A. G. Garcia, S. O. Escolano, S. Oprea, V. V. Martinez, and J. G. Rodr´ıguez. A review on deep learning techniques applied to semantic segmentation. CoRR, abs/1704.06857, 2017.

[15] C. Heinzl and S. Stappen. Star: Visual computing in materials science. Computer Graphics Forum, 36:647–666, 2017.

[16] J. Hu, A. Razdan, G. Nielson, G. E. Farin, P. Baluch, and D. Capco. Volumetric segmentation using Weibull E-SD fields. Visualization and Computer Graphics, IEEE Transactions on, 9:320–328, 2003.

[17] F. Isensee, P. Kickingereder, W. Wick, M. Bendszus, and K. H. Maier-Hein. Brain tu-mor segmentation and radiomics survival prediction: Contribution to the BRATS 2017 challenge. In Proc. 3rd Int. Workshop on Brainlesion (BrainLes), Held in Conjunc-tion with Medical Image Computing for Computer Assisted IntervenConjunc-tion (MICCAI), 10670:287–297, 2018.

[18] F. Isensee, P. Kickingereder, W. Wick, M. Bendszus, and K. H. Maier-Hein. No new-net. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2018. Lecture Notes in Computer Science. Springer, Cham, 11384:234–244, 2019.

[19] D. Kaur and Y. Kaur. Various image segmentation techniques: A review. International Journal of Computer Science and Mobile Computing, 3:809–814, 2014.

[20] J. Ker, L. Wang, J. Rao, and T. Lim. Deep learning applications in medical image analysis. IEEE Access, 6:9375–9389, 2018.

[21] V. Kharchenko, N. Kuzmenko, A. Kukush, and I. Ostroumov. Kernel density estima-tion for foreground detecestima-tion in dynamic video processing for unmanned aerial vehicle application. Conference: 2019 IEEE 5th International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), pages 71–74, 2019.

[22] D. Kingma and J. Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations, 2014.

[23] A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012.

[24] J. Lahoud, B. Ghanem, M. Pollefeys, and M. R. Oswald. 3D Instance segmentation via multi-task metric learning. volume abs/1906.08650, pages 9255–9265, 2019. [25] A. Lex, M. Streit, C. Partl, K. Kashofer, and D. Schmalstieg. Comparative

analy-sis of multidimensional, quantitative data. IEEE Transactions on Visualization and Computer Graphics, 16:1027–1035, 2010.

(53)

References

[27] G. Mayr, B. Plank, J. Sekelja, and G. Hendorfer. Active thermography as a quantitative method for non-destructive evaluation of porous carbon fiber reinforced polymers. Ndt & E International - NDT E INT, 44:537–543, 2011.

[28] F. Milletari, N. Navab, and S. A. Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV), pages 565–571, 2016.

[29] R. H. Nielsen. III.3 - Theory of the Backpropagation Neural Network**Based on “nonindent” by Robert Hecht-Nielsen, which appeared in Proceedings of the Interna-tional Joint Conference on Neural Networks 1, 593–611, © 1989 IEEE. In Harry Wechsler, editor, Neural Networks for Perception, pages 65–93. Academic Press, 1992.

[30] S. Oberpeilsteiner, T. Reiter, and D. Salaberger. Coupling of in-situ-ct with virtual testing by fem of short fiber reinforced materials. 5th Conference on Industrial Com-puted Tomography (iCT) 2014, 25-28 February 2014, Wels, Austria (iCT 2014), pages 299–305, 2014.

[31] N. Otsu. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1):62–66, 1979.

[32] D. Padfield and J. Miller. A label geometry image filter for multiple object measure-ment. GE Global Research, The Insight Journal, 2013.

[33] L. Pilato and M. J. Michno. Advanced composite materials. Springer-Verlag, New York, NY, 1994.

[34] M. Qian and F. H. (Sam) Froes. Titanium Powder Metallurgy. Imprint Butterworth-Heinemann, OSHA, 2015.

[35] G. Rao, B. Plank, and J. Kastner. Evaluation of cfrp-reference samples for porosity made by drilling and comparison with industrial porosity samples by means of quan-titative xct. 7th International Symposium on NDT in Aerospace, pages 1–10, 2015. [36] W. Rawat and Z. Wang. Deep convolutional neural networks for image classification:

A comprehensive review. Neural Computation, 29:2352–2449, 2017.

[37] A. Reh, B. Plank, J. Kastner, E. Gr¨oller, and C. Heinzl. Porosity maps–interactive exploration and visual analysis of porosity in carbon fiber reinforced polymers. Com-puter Graphics Forum, 31:1185–1194, 2012.

[38] M. Reiter, C. Gusenbauer, R. Huemer, and J. Kastner. At-line x-ray computed tomog-raphy of serial parts optimized by numerical simulations. Digital Industrial Radiology and Computed Tomography, pages 9375–9389, 2019.

[39] O. Ronneberger, P. Fischer, and T. Brox. U-Net: Convolutional networks for biomed-ical image segmentation. In Proceedings of the International Conference on Medbiomed-ical Image Computing and Computer-Assisted Intervention (MICCAI), 1:234–241, 2015. [40] D. Salaberger, K .Kannappan, J. Kastner, J. Reussner, and T. Auinger. Evaluation of

(54)

References

[41] M. Sezgin and B. Sankur. Survey over image thresholding techniques and quantitative performance evaluation. J. Electronic Imaging, 13:146–168, 2004.

[42] E. Shelhamer, J. Long, and T. Darrell. Fully convolutional networks for semantic seg-mentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39:640– 651, 2017.

[43] S. R. Stock. X-ray microtomography of materials. International Materials Reviews, 4:141–164, 1999.

[44] C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M. J. Cardoso. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. DLMIA 2, ML-CDS. Lecture Notes in Computer Science. Springer, Cham, 10553:240–248, 2017.

[45] T. Sørensen. A method of establishing groups of equal amplitude in plant sociology based on similarity of species and its application to analyses of the vegetation on dan-ish commons. Kongelige Danske Videnskabernes Selskab, Biologiske Skrifter, pages 1–34, 1948.

[46] S. Wang, B. Kang, J. Ma, X. Zeng, M. Xiao, J. Guo, M. Cai, J. Yang, Y. Li, X .Meng, and B. Xu. A deep learning algorithm using ct images to screen for corona virus disease (covid-19). 2020.

[47] A. Wanto, A. Windarto, D. Hartama, and I. Parlina. Use of binary sigmoid function and linear identity in artificial neural networks for forecasting population density. In-ternational Journal Of Information System & Technology IIJISTECH., 1:43–54, 2017. [48] J. Weissenb¨ock, A. Amirkhanov, W. Li, A. Reh, A. Amirkhanov, E. Gr¨oller, J. Kast-ner, and C. Heinzl. Fiberscout: An interactive tool for exploring and analyzing fiber reinforced polymers. Visualization Symposium (PacificVis), 2014 IEEE Pacific, pages 153–160, 2014.

[49] J. Wu. Introduction to convolutional neural networks. National Key Lab for Novel Software Technology, 2017.

(55)

A First Appendix

(56)

Appendix A. First Appendix

Figure A.2: Modified U-Net(yellow) vs. U-Net(red). Inverted version of Sample 5 is trained in original U-Net because of the architecture parameters. Therefore, input is the same data with inverted version.

(57)

Figure A.4: open iA showing 2D slices and 3D views of Sample 5 with AI segmentation filter.

(58)

Appendix A. First Appendix

Figure A.6: AI segmentation filter in open iA. Step 2: choosing the trained model.

Extraction and Quantiﬁcation of Features in XCT Datasets of Fibre Reinforced Polymers using Machine Learning Techniques

Extraction and Quantification of Features

in XCT Datasets of Fibre Reinforced

Polymers using Machine Learning

Techniques

Miroslav Ivanov Yosifov (Yusuf Senturk)

Contents

1 Introduction and Motivation

2 Related Work

3 Background

4 Method

5 Results and Evaluation

6 Discussion

7 Conclusion

References

A First Appendix