IMAGE SEGMENTATION
1/25V´aclav Hlav´aˇc
Czech Technical University, Faculty of Electrical Engineering Department of Cybernetics, Center for Machine Perception
121 35 Praha 2, Karlovo n´am. 13, Czech Republic hlavac@fel.cvut.cz, http://cmp.felk.cvut.cz
LECTURE PLAN
1. Segmentation needs image interpretation.
2. Thresholding.
3. Region-based segmentation.
4. Edge-based segmentation.
5. Template matching.
2/25
WHAT IS IMAGE SEGMENTATION ?
Segmentation is a collection of methods allowing to interpret parts of the image as objects.
The object is everything what is of interest in the image
(from the particular application point of view). The rest of the image is background.
The approach is similar to that used in pattern recognition, i.e. division of the image into set of equivalence classes.
Segmentation is not based on a real theory.
3/25
SEGMENTATION, EXAMPLE 1
4/25
SEGMENTATION, EXAMPLE 2
5/25
MAJOR APPROACHES TO SEGMENTATION
Threshold-based.
Spatial coherence-based (≈ clustering).
• Connecting edgels.
• Region merging/splitting.
• Line, circle, parametric model detection.
Unusual phenomena-based.
• Camouflage detection based on unusual texture.
• Region segmentation for image compression.
Detection of an a priori know template in the image.
6/25
SEGMENTATION IS APPLICATION DEPENDENT
Segmentation depends on an application, its semantic and a priori information available.
Methods are not universally applicable to all images.
The complicated physics related to image formation is given up due to its enormous complexity.
Instead, a pragmatic approach utilizing all semantic
information is used.
7/25
REGIONS AND SEGMENTATION
Regions as compact sets can represent spatial closeness.
Objects to be segmented in a 2D image very often
correspond to regions.
8/25
COMPLETE VS. PARTIAL SEGMENTATION (1)
Complete segmentation – divides an image into
non-overlapping regions that match to the real world objects.
Complete segmentation divides an image R into the finite number S of regions R
1, . . . , R
SR =
S
[
i=1
R
i, R
i∩ R
j= ∅ , i 6= j .
Partial segmentation – it is possible to find only parts with
semantic meaning in the image (e.g., regions, collection of
edgels) which will lead to interpretation in later analysis.
9/25
COMPLETE VS. PARTIAL SEGMENTATION (2)
To proceed from partial to complete segmentation it is
needed to explore the higher-level of information processing.
This can be performed iteratively in a feed-back loop.
The information about the semantics of the specific application is used.
Partial segmentation reduces the amount of data which need to be processed.
Examples of a complete 2D segmentation
Seek contrast objects in a homogeneous background – printed
characters, cell kernels, back-light illuminated details inspected
in industry .
10/25
EXAMPLE – DIGITAL PROFILE PROJECTOR
11/25
BACK-ILLUMINATED DETAIL
12/25
PROFILE PROJECTOR – USER’S SCREEN
13/25
A PRIORI INFORMATION USED IN SEGMENTATION
The more the better Possibilities: nic
Required shape of the region.
Required position, orientation.
Known initial and final point of the boundary (e.g., a drop of a polymer).
Relation of the region considered to other regions with required properties (e.g., above, inside).
Examples: nic
Remote sensing: Look for ships in the water. Typical properties of railway lines, highways (minimal curvatures). Rivers do not cross.
Medical: Blood vessels ar roughly parallel. Relate to anatomic atlas (model-based approach).
14/25
TAXONOMY OF IMAGE SEGMENTATION METHODS
Thresholding according to a global property. Typical thresholding
according to an intensity. The global knowledge is represented by the histogram of intensities.
Region-based. The regions are created by pixels with similar properties (homogeneity criterion).
Edge-based. Edges bear often important information about objects (cf.
human visual system).
Template matching. The template is sought in the image. The measure of similarity between the template and the image window is needed.
15/25
THRESHOLDING
Input image f (i, j), output image g(i, j) For each pixel (i, j)
g(i, j) = 1 for f (i, j) ≥ Threshold 0 for f (i, j) < Threshold + Simple technique, long time and more often used
+ Easy in hardware, intrinsically parallel.
– The threshold is a parameter which is difficult to adjust automatically in general.
– Works only for subclass of images in which objects are distinct from background in intensity.
16/25
EXAMPLE, THRESHOLD INFLUENCE
Original image. Threshold segmentation.
Threshold too low. Threshold too high.
17/25
THRESHOLDING, MODIFICATIONS
Local adaptive thresholds, e.g. divide an image into subimages and find threshold in each of them.
Band thresholding, let D be a set of intensities, e.g. an interval of intensities
g(i, j) = 1 for f (i, j) ∈ D 0 otherwise
Multiple thresholds
Semi-thresholding , to suppress background, useful in cases in which image is analysed by a human
g(i, j) = f (i, j) for f (i, j) ≥ Threshold 0 for f (i, j) < Threshold
18/25
EXAMPLE
BORDER REGIONS BY BAND THRESHOLDING
Original image. Border regions detected.
19/25
AUTOMATIC THRESHOLD DETECTION, USE OF HISTOGRAM
p-tile thresholding, if we know that the objects cover 1/p of the image, e.g. printed characters on a sheet =⇒ 1/p area of a histogram.
Histogram shape analysis, distinct objects on background correspond to a bi-modal histogram. Find middle between the modes.
20/25
AUTOMATICALLY FOUND THRESHOLD ACCORDING TO A BI-MODAL HISTOGRAM
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
0 32 64 96 128 160 192 224
Frequency
Intensity
Threshold
21/25
SMOOTHING OF THE RAGGED HISTOGRAM
Let hf(zi) be a histogram which is ragged.
Smoothing before finding the threshold automatically eases the problem with many local minima.
The 1D smoothing using sliding averaging of the window size 2K + 1 is often used.
The new histogram h0f(zi) is calculated as
h0f(zi) = 1 2K + 1
K
X
j=−K
hf(zi+j) ,
22/25
OPTIMAL THRESHOLDING BY MIXTURE OF GAUSSIANS Motivation:
optimal threshold
optimal optimal
optimal
optimal conventional
optimal
threshold conventional
conventional ???
distribution of objects distribution of background
threshold
(b) (a)
23/25
LOCAL THRESHOLDS BY MIXTURE OF GAUSSIANS
hregion – local histogram
hmodel – approximation of a histogram by n Gaussian distributions
hmodel(g) =
n
X
i=1
aie
(−(g−µi)
2 2σ2i
)
Criterial function for optimization
F = X
g∈G
(hmodel(g) − hregion(g))2
24/25
EXAMPLE, SEGMENTATION OF THE BRAIN NMR
Input: T1-weighted NMR images.
Desired classes: white matter, grey matter, celebro-spinal fluid (CSF)
0 50 100 150 200 250
Gray level
0 50 100 150 200 250
Gray level
0 50 100 150 200 250
Gray level White matter Gray matter
CSF
(a) (b) (c)
% Volume
% Volume
% Volume
16 12 8 4 0
16 12 8 4 0 16
12 8 4 0
25/25
BRAIN NMR, SEGMENTATION RESULT