Technical report, IDE1156, October 2011
Cylindrical Surface Analysis with White Light Interferometry
Master’s Thesis in Microelectronics and Photonics
Ethem Bora
School of Information Science, Computer and Electrical Engineering
Cylindrical Surface Analysis with White Light Interferometry
Master's Thesis in Microelectronics and Photonics Ethem BORA Supervisor:
Lars BÅÅTH
Cylindrical Surface Analysis with White Light Interferometry
Master's Thesis in Microelectronics and Photonics
School of Information Science, Computer and Electrical Engineering
Halmstad University
Box 823, S-301 18 Halmstad, Sweden
June 2011
Acknowledgement
First of all, I am heartily thankful to my supervisor, Prof Lars Bååth, whose encouragement, guidance and support from the initial to the final level enabled me to develop an understanding of the subject. On the other hand, it is honor for me to work with him and to be one part of his team. The idea of project which is a non – contact measurement system in the cylindrical surface is also excellent and one of the worthy subjects to study on.
I would also like to thank to my ―colleague Français‖ Guillaume Imbert who has been always together with me and study on this project. I am also thankful to my colleagues in the laboratory Sameera Atraqji who has been around us to point the importance of this project, Frederic Cabanettes who has never have doubt to help us about technical parts and MATLAB, Joseph Semere who has encouraged me to end this project in a good way, and other respectable people in Laboratory. I would also thank to Veronika Biro and my family because of their patient and endless support.
Finally, I am very thankful to my thesis coordinator Prof. Håkan Petersson who advises me to get this thesis and give me a chance to work with my supervisor Prof Lars Bååth.
Abstract
At present, one of the big challenges is to develop a precise surface measurement method for mechanical parts. Especially, to study cylindrical surface, the cause of many difficulties because of its geometry shape. This thesis presents a quite good solution for analyzing topography of cylindrical surface with White Light Interferometry optical system which is one of the important and suitable tools in optics.
In the construction period, the aim was to build a system which can be easily mounted on the sample. This is done by a very simple and compact design that also enables us to use it in research laboratories.
In the project, a cylindrical surface analysis is achieved by taking subsequent images with different nano-scale distance from the sample and stitched the acquired images. To achieve this implementation, subsequent images with the highest intensity are first determined and then located in a single image. In the stitching process, cross correlation method that is extremely useful to find out relative point of the images is used to merge the acquired images.
Additionally, stitching process is helped us to extend the area where research can be done.
In the project, MATLAB & LABVIEW are used for analyzing the images and controlling the motors, respectively.
CONTENT
1. INTRODUCTION 1
2. SUPERPOSITION AND INTERFERENCE OF THE WAVES 2
2.1. Theory of Optical Interference ... 2
2.2. Necessary Condition for Interference ... 5
2.2.1. Coherence ... 5
2.2.1.1. Temporal Coherence ... 6
2.2.1.2. Spatial Coherence ... 6
2.2.2. Monochromatic of Light ... 7
3. WHITE LIGHT INTERFEROMETRY 8
3.1. Mirau Objective... 9
3.2. Advantages of WLI system ... 11
3.2.1. No contamination ... 11
3.2.2. Resolution... 11
3.2.3. Size ... 11
3.2.4. Time Consumption ... 11
3.2.5. White light source vs. different light sources ... 12
3.3. Construction of the WLI System... 12
3.3.1. Interferometry Objective ... 13
3.3.2. Motors ... 14
3.3.3. Camera ... 14
3.3.4. White Light Source ... 14
3.3.5. Control... 14
4. POSITIONING AND CALIBRATION 15
5. MEASUREMENT 17
6. IMAGE STITCHING 23
6.1. Correlation Method ... 23
6.2. Fourier Method... 24
6.3. Point Mapping ... 25
6.4. Elastic Model-Based Matching ... 25
6.5. Final Measurement ... 25
7. RESULT 31
8. DISCUSSION 32
9. CONCLUSION 34
10. REFERENCES 35
LIST OF FIGURES
Figure 1: a) and b) Constructive & Destructive Interference respectively, c) Indicate both Interference on the Fringe Pattern (Point A corresponds constructive interference whereas
point B corresponds destructive interference [13,2]) ... 3
Figure 2: Young Double Slit and Fringe pattern on the Viewing Screen [2]... 4
Figure 3: Geometric Construction of Double Slit Experiment [2]... 4
Figure 4: Perfect Temporal and Spatial Coherence [1]... 6
Figure 5: Perfect Spatial Coherence and Poor Temporal [1] ... 7
Figure 6: a) Perfect Spatial Coherence, b) Poor Spatial & Perfect Temporal Coherence[10] .. 7
Figure 7: Interference Pattern with (a) White Light Source and (b) Monochromatic (Green) light source (www.itp.uni-hannover.de) ... 7
Figure 8: Vertical Scanning White Light Interferometer [18] ... 8
Figure 9: Interference Fringes (www.madsci.org) ... 9
Figure 10: Michelson Interferometry [10] ... 10
Figure 11: Schematic Diagram of Mirau Interferometer [13] ... 10
Figure 12: Construction of the White Light Interferometry System ... 13
Figure 13: The Bottom View of the Sample ... 15
Figure 14: Illustration of Misalignment of the Optical system ... 16
Figure 15: The Cylindrical Sample ... 17
Figure 16: User Interface on LabVIEW for Motors Stage Control... 18
Figure 17: The Schematic Diagram of the Control Part from LabVIEW ... 19
Figure 18: Illustration of Vertical Scanning Process a) and b) the Intensity Changing in Different Distance [16] c) Fringe Intensity Profile of the Images[11]... 20
Figure 19: The Algorithm for Vertical Scanning [17] ... 21
Figure 20: The Topography of Cylindrical Image ... 22
Figure 21: Illustration of the Misalignment Camera Position... 23
Figure 22: Illustration of the Geometric Primitives for Coordinate Transformation from Cartesian to Cylindrical... 26
Figure 23: The Algorithm in MATLAB to Transform the Cartesian to Cylindrical Coordinate ... 27
Figure 24: a) Before and b) After applying the program to Transform to the Image ... 28
Figure 25: Subtraction of the Images (Before – After) ... 28
Figure 26: The Algorithm in MATLAB to Merge the Topographic Images ... 29
Figure 27: The Stitching of the Topographic Images ... 30
Figure 28: Observation of the Dust particles on the Optical System ... 33
1. INTRODUCTION
Different kind of instruments has been developed to create accurate surface measurement in industrial applications, e.g. stylus that needs to have a contact to the sample. However, white light interferometry that is one of the widely used instruments in topography identification is brought different perspective with non – contact optical systems. Furthermore, WLI yields not only precise measurement but also provide high resolution that is in single nano scale.
Although the interferometry system has been used for many years in topography identification system, I believe that its capability is more than nowadays is achieved – it is one of the worthy subjects to study on. Our research with WLI optical system is one of the instances to prove that.
In the project White Light Interferometry optical system is constructed that involves vertical scanning method which sense the coherence peaks on to the cylindrical surface. Later on, stitching process is applied to increase the researchable area because of the limited field of view of the interferometry system. However, to protect the accuracy of the stitching process, the cylindrical form of the surface has been traced by the transferring to cylindrical coordinates. Additionally, very simple method is applied to overcome the misalignment of optical system.
2. SUPERPOSOTION AND INTERFERENCE OF THE WAVES
Superposition of wave states behavior of the sinusoidal waves which overlaps in the same medium. In the same manner, Interference explains the interaction of these waves. Thus, these highly relative subjects are mentioned in the same part.
2.1. Theory of Optical Interference
When two or more waves move in the same linear medium, the net displacement of the field (that is, the resultant wave) at any point equals the algebraic sum of all displacement caused by the individuals waves [2]. This definition clarifies the principle of the superposition of the wave which helps us to understand the interference phenomenon.
Mathematical definition for the superposition of the wave is below;
Two wave functions and are introduced as:
>
According to the mathematical description, if the waves are in phase they are in constructive interference, whereas if the waves are out of phase then destructive interference occurs as shown below;
(a) (b)
(c)
Figure 1: a) and b) show Constructive & Destructive Interference, respectively, c) shows the Interference Fringe Pattern from two emitters (Point A corresponds constructive interference
whereas point B corresponds destructive interference [13, 2])
The first experiment about the theory of optical interference was done by Grimaldi in the 17th century. In his experiment, Grimaldi allowed the Sun light to pass through two very small and relatively close pinholes. The aim of the experiment was to get sharp fringes to prove the interference. However, because of the insufficient sun light on the pinholes he couldn‘t get a good result. After many years the scientist Thomas Young was interest in the interaction of light waves and made up his mind to have an experiment that removes all questions about the nature of light. He, therefore, followed all papers about Grimaldi‘s studies. After all necessary consideration was taken into account, his experiment showed almost similar results to
Grimaldi‘s. The only difference was that he used an initial much smaller pinhole which provides high spatial coherence on the viewing screen as shown below;
(a) (b)
Figure 2: Young Double Slit and Fringe pattern on the Viewing Screen [2]
In the double slit experiment, the light from the source passes through from the first barrier to create a regular light wave. When the acquired light wave strikes two pinholes, S1 and S2, respectively, the light beam from these holes interfere with each other and create fringes that is possible to see with naked eye on the viewing screen.
Figure 3: Geometric Construction of Double Slit Experiment [2]
In the figure 3, d is distance between S1 and S2; L is the distance between double slit and viewing screen. δ is the path difference between lower slit and higher slit;
Constructive interference that corresponds to the bright fringes at point P is given by;
Destructive interference that corresponds to the dark fringes because of the out of phase (overlap) at point P is given by;
In the equation, order number (m) is 0, 1, 2..., and L>>d and d >>
2.2. Necessary Conditions for Interference
Frequency difference between two beams is one of the important considerations for optical interference because huge frequency difference can be reason huge phase difference which is time dependent. These two beams thus, must have approximately same frequency.
The amplitude of interfering waves is another considerable subject to create obvious fringes on the viewing screen. Thus, they must have nearly same amplitudes. However, these two considerations in optical system corresponds to the coherence and monochromatic of light.
2.2.1. Coherence
The visibility of the interference is the measure of optical coherence. This is to say, coherence is the necessary condition which enables us to see interaction of the waves. Thus, to obtain the sharpest fringes in interference pattern the waves must be perfect coherence. If the interference is less than perfect coherence, it is called as partially coherence [10]. However, when two beams combine (if there is no interference), the light beam is called as incoherent.
In this part, optical coherence is divided in to two parts temporal and spatial coherence.
2.2.1.1 Temporal Coherence
Temporal coherence is the measure of sinusoid light wave that oscillates nicely when net field exists in the medium and before its phase is getting random in time. Temporal coherence is strongly dependent on coherence time which can be described as oscillation time of this light wave before it changes its phase randomly. Thus, there is direct proportion between coherence time and temporal coherence i.e. Long coherence time is equal to great temporal coherence.
Figure 4: Perfect Temporal and Spatial Coherence [1]
No light source has the ability to expand its energy with infinite coherence length infinite temporal coherence in time. However, if the light source did exist, as shown in the figure 4, with infinite coherence time, just looking at any point in space it would be possible to guess what would happen in any other point. Even small changes in point 2 (P2‘) could be detected, with time, from point 4 (P4‘).
2.2.1.2 Spatial Coherence
Spatial coherence is the measure of the visible fringes that is the result of the interference of multiple emitting sources. It also shows how uniform the phase of the wave front that we have is in space. In temporal coherence, required condition is to have a monochromatic light, whereas in spatial coherence, this is not necessary because the points as illustrated below, has to be in phase and protect their case during expansion.
Figure 5: Perfect Spatial Coherence and Poor Temporal [1]
Figure 6 is shown below to make clear the idea about poor - perfect temporal and spatial coherence.
(a) (b)
Figure 6: a) Perfect Spatial Coherence, b) Poor Spatial & Perfect Temporal Coherence [10]
2.2.2. Monochromatic of Light
In optic, monochromatic light is described as the light source with one frequency or single wavelength. In interference, the reason of using monochromatic light is to have the sharpest fringes that increase the contrast between dark and light regions. As seen in the figure (a) if a light source with broad range of frequency is used, multi-color fringes are seen on the viewing pattern.
Figure 7: Interference Pattern with (a) White Light Source and (b) Monochromatic (Green) light source
3. WHITE LIGHT INTERFEROMETRY
The necessity to work on different surface requires different measurement technique which yields the most accurate and reliable result for each application. White light interferometer is one of the instruments which can yield accurate measurement, i.e. measurement at nano – scale level. Furthermore, combining with modern computer and software, its capabilities is improved.
In white light interferometer system, there are two conditions that have to be met to be real a light achromatic system, i.e. independent wavelength at the location of the zero order fringe and independent wavelength at the separation of the interference fringes, respectively. Many of interferometer systems provide only the first condition, because there is no really existing perfect achromatic system which can yield the second condition.
Optical interference is a commonly used one of the techniques. The basic technique involves splitting an optical beam from the same source into two separate beams – one of the beams is passed through, or reflected from, the object to be measured whilst the other beam (the reference) follows a known and constant optical path. The same basic principle is shown in figure 8 [3].
Figure 8: Vertical Scanning White Light Interferometer [18]
The light beam in the figure, from white light source is reflected by a beam splitter which behaves as a part mirror. The incoming light then, strikes the plate beam splitter. One beam hits and reflects back from the sample and the other hits reference mirror point in the Mirau objective. Finally, the beams are combined in the plate beam splitter – interference pattern with the interference fringes are captured by CCD camera system.
In the white light interferometer, the coherence peak sensing mode of operation using a a broad spectral width light source is used. Due to the large spectral bandwidth of the source, the coherence length of the source is short, and good contrast fringes will be obtained only when the two paths of the interferometer are closely matched in length [4].
Figure 9: Interference Fringes
This required maximum constructive point can be found by monitoring the intensity of the fringe modulation and determining when the fringe modulation intensity is maximized. To implement this, the objective lens is scanned toward the sample; each pixel on the CCD array is monitored for fringe modulation intensity as a function of the objective position.
Subsequent to the scan, the modulated intensity signal for each pixel is interrogated to determine the maximum modulation at which point the pixel is assigned a vertical position corresponding to the position of the objective when the maximum modulation occurred. At this point, the coordinate for each pixel in the field of view can be determined and a 3-D reconstruction of the surface topography can be constructed [11].
3.1. Mirau Objective
Interference fringes occur within the Mirau objective and are subsequently captured by the CCD camera. Mirau objective basically involves a beam splitter and a reference mirror that is inserted between the long working – distance objective and sample. Medium magnification can yield that is varied 10X, 20X or 40X, respectively.
Principle of Mirau interference is similar to the Michelson interferometer. According to this principle, the incident beam of light from the source, as shown below, strikes a beam splitter that reflects a half of the incident light and transmits the other part. Then, half of the wavelength is reflected back from M2 mirror, whereas transmitted other part of incident light is reflected from M1 mirror. These complementary beams, after that, combine and create light and dark bands as known fringes, on the interference pattern (B). Here is the distance to the
M2 is a fixed, whereas distance between beam splitter and M1 is adjustable which provides changing on the interference pattern.
Figure 10: Michelson Interferometry [10]
However, in the Mirau interferometer, the reference point is at the center of the system as illustrated below;
Figure 11: Schematic Diagram of Mirau Interferometer [13]
The prepared optical alignment is the main advantage of Mirau objective system because the required optical path lengths to the target and reference mirror have to be matched.
Furthermore, to focus the target is difficult while maintaining the optical paths. However, Mirau interferometer obviates the need for precision alignment result in three-dimensional motions with great precision and ease [12].
Although the preset alignment provides efficiently surface measurement, the Mirau interferometer requires a long working – distance microscope objective to accommodate the two plates, which consist of the reference surface and the beam splitter plate. This limits the numerical aperture. The reference surface modifies the optical transfer function of the objective because the reference surface is a central obscuration in the beam. This has the effect of reducing the contrast somewhat for larger field-of-view optical systems [7].
3.2. Advantages of the White light Interferometer
White light interferometers are extremely powerful optical tools which have been used for many years as a reliable non-contact optical profiling system for measuring step heights and surface roughness in many precision engineering applications [3]. The reasons that make the system attractive can be explained under five headlines as below.
3.2.1. No contamination
WLI systems are non – contact and provide no sample contamination which makes it possible to provide extremely accurate measurement of the surface.
3.2.2. Resolution
The resolution capability of WLIs systems is about 10-40 nm This scale is sufficient to make measurement on many different surfaces even in semiconductor technology.
3.2.3. Compact Size
Small size and reproducible yield other advantages to the WLIs systems that enable us to use them easily.
3.2.4. Time consumption
Vertical scanning WLI system can give extremely fast response (approximately a minute) comparable to other surface analyzer. That is explaining why WLI systems are preferable in surface measurement systems.
3.2.5. White light vs. Different light source
White light systems have also advantage over other light source systems, because white light is mixture of Red-Green-Blue colors that provide broad wavelength range when compared to other systems. This functionality enables us to remove the glooms which provides ordering problem between the adjacent data point.
In some cases, the white light has also advantage over laser, because the light beam from laser has long coherence length i.e. obvious interference fringes. However, this advantages sometime can also be problem because laser can cause of artificial fringes incorrect measurement, whereas coherence length of white light is short comparable to the laser but if the phase match can yield, the interference fringes enable us to have very precise and powerful measurement on the surface.
3.3. Construction of the White Light Interferometer System
In the project, we aimed to construct a system which can yield very simple and precise measurement on a cylindrical surface. To implement this, most of time has been spent to create the required design which is portable and easily mounted on the sample
The construction basically involves different motors which provide the system to move inside of the sample both in vertical and horizontal direction. Furthermore, White LED has been used as a light source to remove ambiguities caused by narrow wavelength. Here, light beam is directed by a right angle mirror which enables us to scan the sample of the surface even if the system is designed vertically. Complete design and detail about the equipment are mentioned below.
Vertical Motor Camera
Horizontal Motor Motor Holder White Light Source
Rotation Motor
Objective Sample
Figure 12: Construction of the White Light Interferometry System
3.3.1. Interferometry Objective
The objective is the most important part in the optical system. In the project, a Nikon interferometry Objective is used. Features of the objective is respectively, 10X magnification, 7.4 mm working distance, 0.3mm numerical aperture and 0.88 X 0.66 field of view.
3.3.2. Motors
Piezo drive system is one of the important parts in the project because it provides scanning towards the sample and enables us to create 3D topography map of the surface. Furthermore, scanning range is limited at 17 μm and resolution capability is in nano – scale.
To provide vertical movement for the construction, highly repeatable ‗Thorlabs NRT 150 Nano motor’ is used. Positioning resolution is limited at 100 nm and capability of the motor is 25.600 micro steps per tour.
To rotate precisely the interferometer system in horizontal direction, ‗Thorlabs BSC101 Nano rotator’ that provides continuously rotation (360°) is used.
3.3.3. Camera
In the project, to capture the sequential images, ‗Basler CCD piA2400 -12gm’ camera is used with 2448 x 2050 pixels.
3.3.4. White Light Source
The light source in the interferometry system is one of the necessary equipment to identify the surface. Thus, in the WLI construction, white LED is used as a light source which has a broad bandwidth with fiber optic cable that enables us to collimate the intensity of the light in a one point.
3.3.5. Control
In the implementation of the vertical scanning WLI system, the motors and camera are controlled by the ‗Virtual Interface of Lab View software’.
4. POSITIONING AND CALIBRATION
Positioning and calibration stages are highly important two subjects that play an active role to get precise measurement of the surface. However, here the calibration process precedes the positioning because the story of calibrations starts with the design of the construction. This is to say, the construction must be designed very well so that the calibration could maintain its setting during the measurement. During the construction, this stage is taken into account and all the dynamic instruments are fixed. Furthermore, the instruments in the prototype are controlled by Virtual LABVIEW software in order to have exact measurements.
In the positioning process, the misalignment problem of the optical system is overcome by manual tests. To achieve this, the reference point is determined on the sample and the view from the CCD camera is observed in the Virtual LAB VIEW software – the piezo motor is controlled to find the best view toward to the sample. After that the rotation motor rotates in 90 degree steps around the surface. The same observation is done in the LabVIEW to see whether the resolution is changed or not because the axis of the optical system must be same axis with the sample. The sample holder thus, is used to arrange the required changes as illustrated below;
Figure 13: The Bottom View of the Sample
Another misalignment of the optical system is solved with a tilt plate. Here the problem is that the optical system and the piezo motor must be perpendicular to each other so that precise measurements can be made. As a result, the acceptable result is achieved from the positioning even if the process is done by manually.
Figure 14: Illustration of Misalignment of the Optical system
Furthermore, to find the real FOV of the interferometry system, the vertical motor has been used. To implement this, the reference point is selected manually in the screen and the image captured by CCD camera and then by running the vertical motor in known distance, the second image which includes the reference point is captured. Finally, the sequential images are examined in MATLAB that enables us to determine the real field of view of the interferometry system by pixel alignment.
5. MEASUREMENT
In the project, white light interferometry is constructed to study on the inside surface of a cylinder with diameter 131mm as shown below. The first aim is to get topography of the cylindrical surface and then in the next section the image stitching is proposed to enlarge the field of view of the interferometry system as a mosaic of topographic maps.
Figure 15: The Cylindrical Sample
In the vertical scanning process, to get topography of the cylindrical sample, first the maximum intensities of each pixel from the different layers have been determined and then gathered in a single image which yields precise surface measurement. To implement this, initially Laboratory Virtual Instrument Engineering Workbench (LabVIEW) software that simplifies scientific computation, process control, and measurement applications is used to control the piezo motor which provides scanning toward to the sample. After that the data which is recorded by LabVIEW is analyzed in MATLAB.
In LabVIEW, the schematic construction for vertical scanning, as shown below, is designed that yields uncompressed movie which is recorded 20 frames/second. In our experiment scanned range is limited 17 μm which is highly enough for the roughness which is about 8 μm and other parameters for the scanning process can also be changed before the implementation such as number of frames.
f ile f d
-
'{lew-
.!'.roject Qperate !ools '{{l ndow !::!elp•R.b-.- ;
SN: 41827512: Vl.0.31(2.0.3)Volts PoWon
-
Zero EnableJog _j
II'"
Sc;
Set Posrtion
loop
...
1:: 2 u 0
1) on ...
1/J
.
rJ..:.l ...
0
....
Recordong Rotative motor I NS360·M
...
00 ...-1
-- - Jog •
'"""'
--- •
'"""'.!]
• •
Sit Ql29856: V!.O.J1(2.0 S• 4Q8l7119· Vl.O1. 1(2.
Jo;
v - J - .!l .!l
v •t] J - -
...:1 ::::::
0
1)
u
1)
.1....:..:.
....
1) rJ:l
v:;
1/"Ciostd" on the "loop" button 2/ select camera "cam4"
51clic "Set Posrtion'(delauvalues:15; 25; 20;17) 6/ elk on tht "Recording" button
1-.
....
MG17Piflo
v
..0...s:: 0
r/J
..r../.J.
Select
Select Path to Save AVIfile
t !
;
r
...
...:l
8 0 c.!:1
p.., 0 I-<
u
<])
... Frames per second I I I I g !True
.;:;
t.,_.
>
·!!MQlP lfZO' ! .-' - r:--- 0 O'l...:l SetPosOutput , , ,
=
§
('(j
...-!
• ICha n!D , 1
status!
0
i
: 0
... 5b Cl c.!:1
0
u
.;:;
Record!:--- Resolution
!i2EJ
-·--1? <{;>---· .
·-- P> ·I
@f
l tRe<,ording l§
·
u.g
8 <])
...s:: u
(ZJ
t.,_.
0
..(..'(..j
"d
-5
r/J<])
t5
OJ ·---g--e .
Vertical motor I NRT150-M
t::i:=====
=== -rt7l,+M.GM Rotative motor I NS3i60·M=f
:i ·-l --·---·---·---·---·- --·---' ;
[]] '-·-·· ---·------·---·-----·---·- --·---·-'
<])
r-
...
The implementation of the coherence peak sense is illustrated in the following figure. The intensity differences are obvious when interferometry optical system is moved toward to the sample. The process that is done in the project corresponds to this illustration.
(a) (b)
(c)
Figure 18: Illustration of Vertical Scanning Process a) and b) the Intensity Changing in Different Distance [16] c) Fringe Intensity Profile of the Images [11]
In MATLAB, the sequential frames are examined one by one and the highest intensity points, as seen in the real image, are integrated in a single image that enables us to acquire 3D surface topography.
The coherence peak sensing algorithm is shown below;
Figure 19: The Algorithm for Vertical Scanning [17]
3D topography of the cylindrical surface is shown below.
Figure 20: The Topography of Cylindrical Image
6. IMAGE STITCHING
The field of view (FOV) of the white light interferometry is quite small, less than 1 . There are other conventional interferometers which can yield larger FOV. However, the spatial resolution of these interferometers is low, and therefore for us incorrect measurement.
Another way to enhance the field of view is Image Stitching which enables us to merge the subsequent images to each other into a mosaic and acquire the topography of the sample with high spatial resolution.
However, there are some cases that must be considered before starting the image stitching.
One of them is the overlap region between the subsequent images because small overlap area may result in an incorrect mosaic. However, it yields less time consumption because of the less number of measurements. Whereas if the overlap area increases the result will be more accurate but the number of measurement increases. Thus, the rate of overlap range between the two images must be carefully selected to have both accurate measurement and the least time consumption.
Another consideration is the misalignment of the optical system because it can cause difficulties in the stitching process such as deviation in the image because of the small changes of camera‘s position. This factor is significant when many of subsequent images need
to be merged in vertical and horizontal direction. Otherwise, this consideration can be neglect.
Required
Geometric Shape of the Image
The Image captured by the Camera
Figure 21: Illustration of the Misalignment Camera Position
In some of scientific articles, image stitching refers to the ‘Image registration’ which can be defined as a mapping between two images both spatially and with respect to intensity [9]. The key point in the image registration is to find the relation of the subsequent images. To implement this, many techniques have been developed. In the following paper, the methods that is used for image registration is mentioned.
6.1. Correlation Methods
Cross-correlation is the basic statistical approach to registration. It is often used for template matching or pattern recognition in which the location and orientation of a template or pattern is found in a picture. By itself, cross-correlation is not a registration method. It is a similarity
measure or match metric, i.e., it gives a measure of the degree of similarity between an image and a template [9]. Mathematical definition of the normalized cross correlation is given by;
Where w(x, y) represents a pixel value of the image to be placed; w is the mean value of all pixels included in the box area; f (x+i, y+j) represents a pixel value of the composite image inside the box area; f (i, j) is the mean value of all pixels of the composite image within the box area and parameters K, L represent the box dimensions in number of pixels included [8].
The result of cross correlation is varied between 0 and 1.1 corresponds a perfect correlation, whereas 0 corresponds no similarities between the subsequent images. To be a good correlation states 0.7. Here, because of the influence of the local intensity, cross correlation must be normalized. Cross correlation yields the smallest error among the other methods.
However, it is limited to the registration methods where a small affine transformation misaligns the images. Additionally the images that contain noise could be a problem for cross correlation methods. To overcome this problem, the filter must be used.
Another approach to compute correlation of the two or more images is ‘Correlation Theorem’
that involves ‘Fast Fourier Transform (FFT)’. The difference from usual correlation theorem is that some parts of the process are achieved in the frequency domain instead of spatial domain. Combination of the Cross correlation and Fast Fourier method enable us to achieve precise and fast image registration. However, the FFT requires high capacity and cost that is direct proportion to the size of the image.
The Stitching implementation is not only applied to 2D but also to 3D images. Only difference is that the overlap region of the 3D subsequent images must be larger (that is about 50%) comparable 2D since the brightness level of these images is different.
6.2. Fourier Methods
In the previously method which is a combination of the cross correlation and FFT is that the frequency domain is used as a target to find the correlation. However, in this method, translation, rotation, reflection, distributives and scale all have their counterpart in the Fourier domain [9].
Phase correlation is the one of the basic methods that has been used all process in frequency domain. And it refers to Shift Theorem which is very effective method in the case where the
narrow frequency is used in the noisy image. However, it is limited by white noise that can be described as a noise spread all frequency range in the image. Even if Fourier method yield faster response comparable the cross correlation, it is only possible to use when well defined transformations exist.
6.3. Point Mapping
Point mapping method is used when there is a significant amount of spatially local variation.
It is an extremely useful method to register two subsequent images which has unknown misalignment.
The general method for point mapping consists of three stages. In the first stage features in the image are computed. In the second stage, feature points in the reference image, often referred to as control points, are corresponded with feature points in the data image. In the last stage, a spatial mapping, usually two 2D polynomial functions of a specified is determined using these matched feature points [9]. Accuracy of this system strongly depends on point matching. The rest process is only for interpolation. Even if the method yields very good result in unknown registration case, it is quite complicated process because of the matching process. Any control points must be used otherwise the methods do not provide precise measurement.
6.4. Elastic Model Based Matching
Elastic model method is based on point matching. However, in this method the control points do not compute directly and is not directly mapping from one image to another. Instead of this, it models the distortion in the image as deformation of an elastic material such as bending, stretching.
Elastic methods, because they mimic physical deformations, register images by matching structures. Thus, it has been developed and is often used for problems in shape and motion reconstruction and medical imaging. In these domains, the critical task is to align the topological structures in image pairs removing only the differences in their details. Thus, elastic methods are capable of registering images with some of the most complex distortions, including 2D projection of 3D objects [9].Comparable to the point select more stable and accurate.
6.5. Final Measurement
In the project, cross correlation method that provides stitching much better than any of methods is used to merge the subsequent images of the cylindrical surface. In the cross correlation method, the key point is to obtain the best correlation point between two overlapped sequential images.
However, before starting the image stitching, there is an important problem to overcome because of geometric shape of the sample. In the project, the sample that is worked on has cylindrical shape. The interferometry images that are captured by the camera are in Cartesian coordinates. Therefore also the resulting topographic map will be in Cartesian coordinates.
Our first aim thus is to, by applying geometric primitives; transform the Cartesian shape to the true cylindrical surface coordinates which enables us to achieve precise image stitching. To perform this, roughness of the sample that is expanded in ‗z’ direction is registered in corresponded pixel. This is to say, the optical interferometry system for each mosaic topographic map is assumed to be in the center of the curvature as illustrated below;
θ
Roughness
.
Azimuth Angle
Figure 22: Illustration of the Geometric Primitives for Coordinate Transformation from Cartesian to Cylindrical
Formulas are shown below;
( )
The process for coordinate transformation from Cartesian to cylindrical coordinate system is performed in MATLAB. Algorithm is given below;
Figure 23: The Algorithm in MATLAB to Transform the Cartesian to Cylindrical Coordinate
As seen in the algorithm, the reference point is determined in the center of the picture. The theta is found in every movement to the left and right side. This enables us to find the roughness of surface ( ). After that, ΔR is registered to corresponded pixel in the new image.
Cartesian coordinate Cylindrical coordinate
Figure 24: a) Before and b) After applying the program to Transform the Image
As seen the images, it is hard to see the reality whether the transformed is performed or not.
However, to overcome this problem, the images that consist of matrix are subtracted from each other – the result is shown in the figure as below;
In the stitching process, the sequential topography images can be stitched either extracting a strip in one of the frames of the image data cube or determining a small area that is about 200x200 pixels. In the project, the stitching process is performed by a small area because of the increased visibility of the overlap area in the image. The overlap range between two images is determined to 50% which is sufficient to find the best correlation point by observing the camera that is controlled by motorized stage. Because of the many number of images, MATLAB, that saved us from many complicated calculations, has been used. The algorithm of the image stitching is as below;
Start
Read the Images
Choose the Similar Regions of Each Image ‗One of the Images must be smaller than
other‘
Find Normalized Cross – Correlation and Coordinates
of Peak
Find Total Offset of the Images
Find the point where the Small image is located
Remove overlap area of the Large Image
Merge the Images
End
Figure 26: The Algorithm in MATLAB to Merge the Topographic Images
Figure 27: The Stitching of the Topographic Images
As seen the Figure 27, the sequential topographic images which is overlapped 60% is merged with each other. When magnification is increased, it is easy to see that there is an residual offset between the two images because of the misalignment of the optical system. However, the result is at an acceptable level.
7. RESULT
In this project, vertical scanning, transformation to true cylindrical coordinates and image stitching processes are applied to sequential images which are captured by CCD camera in a WLI optical system. The result from all three steps is at an acceptable level.
In the vertical scanning, the uncompressed movies that is 20 frames / per second are analyzed in MATLAB to get topography of the surface, and here the scanning range toward to sample is limited 17 μm which is highly sufficient to examine the roughness of the sample. After that, by using MATLAB as mentioned before, the topography of the sample is acquired.
In the stitching process, firstly the Cartesian coordinate is transformed to Cylindrical coordinate without losing any data. After that, the stitching process is applied. To transform the coordinates, geometric primitive has been used as mentioned previously section. Then, the sequential topographic images which are about 60 % overlapped are merged each other by Cross Correlation method that is highly precise method to find the correlation of the images.
The project aim is to create 3D image stitching by consists many of images that surrounds the sample both in vertical and horizontal direction. However, the stitching process is only applied in the horizontal direction, as shown Figure 27, because of insufficient time.
8. DISCUSSION
In the WLI optical system, the way of getting a good result in the image stitching is to have a well aligned optical system. In our structure, this is achieved manually. The result was acceptable as observed from the view of CCD camera in the Lab VIEW and when the overlapped region is compared in MATLAB. However, the construction can be changed to achieve better and more precise measurement. In our construction one of the problem is aberration which creates blurry at the edge of images because of the length of the optical systems. This could be shortened. However, the design which is already constructed could not let us to make some changes because of the thickness of the rotation motor which is about 6 cm. This is also limiting the vertical movement of the optical system.
According to many researches, the quality of the stitching process is determined by the visibility of the seam between the merged images. The image stitching is therefore often applied with as an image blending process so that the lines between two sequential images can be made invisible. However, in our experiment in order to protect the integrity of the topographic images, no blending has been used.
Vertical scanning is another significant part of the project. The acquired topography is completely precise. However, the process suffers from being too time consuming. This can be decreased a bit either by reducing the number of frames or by reducing scanning range which is an important parameter to get precise topography of the sample.
Another consideration is dust build up inside the optical system that can yield inaccurate measurement. In the project, special spray has been used to protect the optical system from dust and other kind of stain. However, it is hard to have perfectly clean optical system without having ‗clean room’ conditions. The dust in our optical system is shown below;
Figure 28: Observation of the Dust particles on the Optical System
9. CONCLUSION
In the project, a White Light Interferometry system has been designed that consist of vertical scanning and image stitching processes in cylindrical coordinates, respectively. First of all, by a vertical scanning process the topography of the sample is acquired. Transformation is then performed from Cartesian to Cylindrical coordinate by geometric primitives. Finally, the image stitching is applied in order to increase limited view of the interferometry system.
The result from all three processes is acceptable especially, vertical scanning and coordinate transformation. However, in the stitching process, we have faced many different problems because of the misalignment of the optical system. To overcome the misalignment problems, common methods is applied manually. Furthermore, even if the stitching process is applied only in one direction, feasibility of the image stitching in the cylindrical surfaces is proved.
10. REFERENCES
[1] Eugene Hecht, Optics, Fourth Edition, ISBN 0-321-18878-0.
[2] Serway and Beichner, Physics for Scientists and Engineers, Fifth Edition, ISBN-10:
003026961X.
[3] Roy T. Blunt, White Light Interferometry – a production worthy technique for measuring surface roughness on semiconductor wafers, April 24-27, 2006.
[4] James C. Wyant, White Light Interferometry, Optical Sciences Center.
[5] Akiko Harasaki, Joanna Schmit, and James C. Wyant, Improved vertical-scanning interferometry, Vol. 39, No. 13 y APPLIED OPTICS, 2000.
[6] Dae Wook Kim, Vertical scanning interferometers using white light, Oct.27 2009
[7] Bharat Bhushan, James C. Wyant, and Chris Koliopoulos, Measurement of surface topography of magnetic tapes by Mirau interferometry, Vol. 24, No. 10 / APPLIED OPTICS, 1985.
[8] Vladan Rankov†, Rosalind J. Locke, Richard J. Edens, Paul R. Barber and Borivoj Vojnovic, An algorithm for image stitching and blending, 2005.
[9] Lisa Gottesfeld Brown, A Survey of Image Registration Techniques, Vol 24, No 4, 1992.
[10] http://skullsinthestars.com/optics-basics-coherence/.
[11] Santo Padula, Measurement Variability of Vertical Scanning Interferometry Tool Used for Orbiter Window Defect Assessment, NASA/TM—2009-215636.
[12] Hong, S.S., Surface Acoustic Wave Optical Modulation, February 2001.
[13] Richard Leach, Guide to the Measurement of the Smooth Surface Topography using Coherence Scanning Interferometry, NPL.
[14] http://www.rp-photonics.com/coherence.html.
[15] T. Y. Chen and L. C. Yeh, Larger surface profile measurement of microstructures by white-light phase-shifting interferometry and image stitching, June 1-4, 2009.
[16] Daniel Kapusi, Torsten Machleidt, Kart-Heinz Franke, Eberhard Manske, Rainer Jahn, Measuring Large Areas by White Light Interferometry at the Nanopositioning and
nanomeasuring Machine, Sept. 12th, 2007.
[17] Karthick Sathiamoorthy and Tanjim Ahmed, Construction and Validation of a White Light Interferometer, June 2010.
[18] Inna Kurganskaya, Andreas Luttge, Andrew R. Barron, The Application of VSI (Vertical Scanning Interferometry) to the Study of Crystal Surface Processes, Jul 13, 2009.
[19] Wikimedia Commons.
