Orientation Estimation in Ambiguous Neighbourhoods

Orientation Estimation in Ambiguous

Neighbourhoods

MatsT. Andersson & HansKnutsson

Computer VisionLaboratoryLinkoping University

581 83Linkoping, Sweden

Abstract

This paper describes a new algorithm for local orientation estimation. The

proposed algorithm detects and separates interfering events inambiguous

neigh-bourhoods and produces robust estimates of the two most dominant events. A

representationsuitableforsimultaneousrepresentationoftwoorientationsis

intro-duced. Themainpurposeofthisrepresentationistomakeaveragingofoutputsfor

neigbourhoodscontainingtwoorientationspossible.

Thefeatureextraction is performed by aset ofquadraturelters. Amethod

toobtainalargesetofquadraturelterresponsesfromalimitedbasisltersetis

introduced. Theestimationoftheneighbourhoodandtheseparationofthepresent

eventsarebased uponthe quadratureresponsesintermsoflocal magnitudeand

phase. Theperformanceofthealgorithmisdemonstratedusingtestimages.

1 Introduction

ItiswidelyacceptedthatthemostpowerfulwaytosolvediÆcultimageprocessing

prob-lemsistouseahierarchicalprocessingstructure. Oneofthemostprofoundoperations

in such ascheme is thelocal orientationestimate for line and edge elements, which is

thesubjectforthispaper. Mostalgorithmsfororientationestimationassumethe

neigh-bourhoodto belocallyone-dimensionali.e. consistingof alineoranedge. Fornatural

images this assumption is correct in most cases. Ambiguous neighbourhoods such as

cornersandcrossesdo,however,containimportantinformationfortheinterpretationof

thescene athigherlevelsin thehierarchy.

Thepurposeoftheproposedalgorithmistoseparateambiguousneighbourhoodsinto

several one-dimensionaleventsthat can be propagated to thenext level asalternative

events. Insuchawaythedegreeoffreedomin themodelisincreasedwhilethebenets

oftheone-dimensionalapproachispreserved. Thedescriptionofthealgorithmwillcome

in twoparts. First a basis lterset is introduced that enablesan approximationof a

quadraturelterpairinanarbitrarydirection. Sincethealgorithmwillrequireadense

feature extraction such alter set is computationally eectivebut not necessary. The

secondpartofthepaperdescribeshowsingleeventscanbeextractedin anambiguous

environment by observing thequadrature responses in termsof magnitude and phase.

For eachneighbourhood thealgorithmwill calculatetwoorientationestimates. The

2 Feature Extraction

To perform a more exhaustive analysis of a neighbourhood, a denser partitioning of

theFourierdomainis necessarycomparedtomodelsbasedonlocal one-dimensionality.

The partitioning of the Fourierdomain is basedon quadrature lter responses overa

largenumberoforientations. Quadraturelterscanwith advantagebedened inpolar

coordinates and the response in terms of phase and magnitude enable a continuous

representationofedgeandlineresponses. Aneighbourhoodofagrayscaleimagei(x;y)

hasaHermitianFouriertransformI(!)=I(!

x ;! y )=I  ( ! x ; ! y

) which implythat

the real part R e I(!) is even and the imaginary part Im I(!) is odd. The energy

contribution ofthe local neighbourhood i(x;y) canin theFourierdomain beobtained

as: jI(!)j 2 =jI( !)j 2 =[R eI(!)] 2 +[ImI(!)] 2 (1) As jI(!)j 2

is even, it is suÆcientto estimate the energy in one halfplane, and the

evenandoddpartsofI(!)canbeextractedseparately. A generalquadratureltercan

beexpressedas: H e (;') = 1 2
G()
[ (' ' k )+ (' ' k +)] H o (;') = 1 2
G()
[ (' ' k ) (' ' k +)] H(;') = H e (;')+H o (;')=G()
(' ' k ) (2) where:

G()denesthefrequencycharacteristicsof thequadraturelter,= q ! 2 x +! 2 y . ' k

isthemain directionofthelter,'=tan 1 ( !y ! x ).

(')controlstheangularperformanceofthelter. (')=0forj'j=2.

Theaimisnowtoproducealargesetofquadraturelterresponses which can

sup-port the proposed model. Tocompute each of these lter responses separately would

lead to heavy and redundant computations. A more attractive way is to produce a

small basis lter set from which quadrature responses can beapproximatedin an

ar-bitraryorientation('

k

)andwith alternativeangulardiscrimination functions ('). A

straightforwardandrobust method to accomplish thistask is to produce abasis lter

set,[F

e (i);F

o

(i)],ofharmoniclters,i.e. ltersthatintheFourierdomainhaveangular

functionscorrespondingtocos(i') andsin(i')such that:

F e (i) = G()
cos(i') i=[0;1:::n] F o (i) = G()
sin(i') i=[1;2:::n] (3)

The angular bandwidth, n, of the basis lter set will determine the upperbounds

for the angular resolution ('). An increase of the angular bandwidth will, however,

requirelargerconvolutionkernelswhich reduce thespatial resolutionaccording to the

An expression forthequadrature response (Eq. 2) in anarbitraryorientation ('

k )

cannowbeapproximatedbyusingelementaryfeaturesofharmonicfunctions.

H(;')=!(0)F e (0)+ n X i=1 !(i)[F e (i)cos(i' k )+F o (i)sin(i' k )] (4)

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

-180

-135

-90

-45

0

45

90

135

180

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

-180

-135

-90

-45

0

45

90

135

180

Figure1: (') forthreedierentweightvectors!(i)

Theweightvector!(i)i=[0;1:::n]denesthe (')-function. Figure1showthree

normalized plots of (') corresponding to dierent !(i). The DC-level is controlled

by!(0)whichmust betuned toan optimalapproximationof aquadrature lter. The

remainingweightcoeÆcientsisinthis exampledenedby(fromwidetonarrow (')):

!(i)=cos 4

(i=2n+2); !(i)=cos 2

(i=2n+2)and!(i)=1. Inallcasesn=6. Theripple

eectthatmayoccurfornarrow (')functionscorrespondstoaphaseshiftof180 Æ

. As

will be explainedlater this eect is notcritical for this algorithm. To obtainH

e (;')

andH

o

(;') separately,equation4issplitin termsofoddandevencomponentsof(i)

sothat: H e (;') = P n i=0;2;4::: !(i)
[F e (i)cos(i' k )+F o (i)sin(i' k )] H o (;') = P n 1 i=1;3;5::: !(i)
[F e (i)cos(i' k )+F o (i)sin(i' k )] (5)

Thedesiredquadratureresponsecannowbecalculatedfromtheconvolutionresponse

betweentheimageandtheinverseFouriertransformofthebasisfunctions. Figure2show

aquadraturekernelpairh

e

(x;y)andh

o

(x;y)correspondingtoEq. 5. Theintermediate

Figure2: Wireplotofaresultingquadraturekernelh

e

(x;y)andh

o (x;y).

Fromabasisset consistingof 2n+1kernelsitissubsequentlypossibletocalculate

quadratureresponsesinanarbitrarynumberoforientations,where(n)denestheupper

boundsinthetrade-obetweendirectionalandspatialresolution. Inthisexamplen=6

resultsin 13 basiskernels. It may be interestingto compare this computationaleort

towhatisrequiredforsimpleone-dimensionalneighbourhood. Itcanbeshown,see[2],

thatthedominantorientationofaonedimensionalneighbourhoodcanbeestimatedby

aminimum ofthree quadrature lter pairs. Forcomplexornoisy neighbourhoods the

performanceisconsiderablyenhancedbyfourlterpairs,whichresultsin6 8kernels.

Thecomputational cost forthe convolutionin relation to a onedimensional approach

iswellbelowafactorof twowhich must be consideredfairly lowin comparison to the

benetsgained.

3 Estimation ofOrientationin Ambiguous

Neighbour-hoods

ThepurposeisnowtousethesamplesoftheFourierdomainprovidedbythequadrature

responsestoestimatethepresenteventsintheneighbourhood. Theambitionisthatthe

algorithmshould describetheneighbourhoodin termsofone-dimensionaleventsasfar

aspossible. An ambiguous neighbourhood will subsequently be interpreted asseveral

superimposed one-dimensionalevents,ofwhichthetwomostdominantareproducedas

output.

Toinitializetheprocessinganumberof (N)quadraturelterresponses,evenly

dis-tributedin onehalfplane of theFourierdomain, arecalculated. It mayseemoddthat

theknowledgeof the neighbourhoodincreases when (N)increases asall responses are

calculatedfrom thesamebasislterset, butasthese calculationsinvolvenonlinearities

(Eq.6) thisis trueto acertainextent. Noqualitativeinvestigationof therelationship

betweenthecalculatednumberoflterresponses(N)andtheangularresolutionofthe

The quadrature response in direction ' k = k=N k =[0;1:::N 1] is denoted X k =X k e +j
X k o ,whereX k e andX k o

refertotheevenandoddlterresponse. The

magnitudejX

k

j andtheFourierphase

k

areforeachlterpairdened as:

jX k j= q X 2 k e +X 2 k o  k =tan 1 ( X k o X k e ) (6)

θ

Figure3: Continuousline/edgedetectionbytheFourierphase.

TheFourierphase

k

canaccordingtoFigure3beinterpretatedasashapeestimate

forthepartoftheneighbourhoodthatcorrespondsto (' '

k

). For

k

equaltozeroor

thecorrespondingpartoftheneighbourhoodwillbeinterpretedasalightrespectively

darkline, while 

k

==2correspondsto edgesof `dierent sign'. The Fourierphase

thusoersacontinuousshapedescriptionofapartofaneighbourhoodintermsof

one-dimensionalevents. ThemagnitudejX

k

j,whichisphaseindependent,re ectstheenergy

presentinthesamepartoftheneighbourhood.

Tobeableto separateinterfering events,acontinuousdescriptionofboththeshape

and the dominant orientation for every part of the neighbourhood are required. The

shape criterion is well met by the Fourier phase (), but the orientation

representa-tionrequiresomeconsideration. Assumefor amomentthat theneighbourhood is

one-dimensional.ThiswillintheFourierdomainresultinaconcentrationoftheenergyupon

alineinsomedirection'2[0;],andthemagnitudeofthecorrespondingquadrature

re-sponseswillbehigh. Theangle'is,however,ambiguousas'

N 1 ' 0 =(1 1=N), while ' N 1 and' 0

correspond to practicallythe sameorientation. Toavoid this

am-biguity the concept of the double angle representation is introduced, see [2]. For a

one-dimensionalneighbourhoodthedominantorientationcanbecalculatedas

R= N 1 X k =0 jX k j
e j
2'k

where jR jcorresponds tothe energyin theneighbourhood and arg(R )is an

unam-biguousestimation of theorientation as'

0 and '

N 1

aremapped nextto eachother.

Whatwillhappenifseveraleventsarepresent? Duetothenatureofthevector

summa-tion,orthogonaleventswillbecomeoppositeintheproposed representationandhavea

tendencytocancel each other. Theresultwill beareductionin themagnitudejR j for

Fourierphase. Inother casesthe estimated orientation may notberelevantfor either

ofthe eventsdue to phaseinterference. The optimalresultfor such aone-dimensional

model in anambiguous neighbourhood is thus zero, asneither themagnitude nor the

estimatedorientationmayberelevantfor anyof theevents. Zero outputdo, however,

onlyoccurforaverylimitedset ofambiguousneighbourhoods.

Magnitude

0

0.5

1

1.5

2

-24

-12

0

12

24

Fourier Phase

-180

-90

0

90

180

-24

-12

0

12

24

Detected Event

0

0.5

1

1.5

2

-24

-12

0

12

24

Unwrapped Phase

0

45

90

135

180

-24

-12

0

12

24

Figure4: Illustrationofphaseandmagnitudeforanambiguousneighbourhood.

Thecoreoftheproposedalgorithmistoseparatetheglobalvectorsummationabove

intotwosummationsas

R a = a 2 X k =a 1 jX k j
e j
2' k R b = b 2 X k =b 1 jX k j
e j
2' k

wheretheelementsofeachsummationcorrespondtoasingleevent. Eachsummation

isperformedoveraclosed regionin the Fourierdomain andaspecic magnitude jX

k j

can at most contribute to one summation. To obtain a criterion for separation of a

neighbourhoodin termsof phase,itmaybehelpful tostudy themagnitudeand phase

response of atypically ambiguous neighbourhood. Theuppercurves ofFigure 4show

themagnitude and phaseresponses for aneighbourhood consisting of across between

aline and anedge whose orientation diersby 30 Æ

. Themagnitude of thetwoevents

dierafactorof twoandthere is noisepresent. Forclarity thesamplescorresponding

tothewholeFourierdomainareillustratedalthoughitisonlynecessarytoperformthe

calculations over one halfplane. From these curves it can be deduced that the phase

curveismoreorlessconstantintheareathat correspondstothepresentevents,while

itis rather unstable betweenthe eventsand in the noise. Todevelopthis observation

k = k X i=0 ( i  i 1 ) 2 (7)

Inthis operationcaremustbetakentodisengagenaturalphasejumps of2 which

areirrelevant. Theunwrappedphase
(')isillustratedinthelowerleftpartofFigure4.

Thealgorithmcannowbedescribedas: chosetwoarbitraryorientations'

l and'

h who

fulllthecondition:
('

h

)
('

l

)andcalculate thesum:

R a = h X k =l jX k j
e j
2'k (8)

This sumiscalculatedfor allpossiblecandidates of'

l and '

h

. As therst output

thenchoosetheresultingsumR

a

that obtainedthelargestmagnitude. Notethatifthe

(')-curve is atin an environment, the summation will engage moreterms and the

resultantmagnitude is morelikelyto belarge. If theparameter ischosenwith care

R

a

willcorrespondto themostdominanteventin theneighbourhood. Thepartof the

magnitude curve corresponding to R

a

is then disengaged and the algorithm estimates

thesecondevent,ifpresent,bythesamemethod. Todiscriminatenon-relevantoutputs,

especiallyforthesecondevent,anadditionalcriterionmaybeusedwhichdemandsthat

aneventmustspanacertainareaof theFourierdomain tobeaccepted. Theproposed

phasecriterionwouldtheoreticallyfailiftheneighbourhoodconsistsoftwoblackortwo

whitelines, as no phaseshiftswould thenoccur. Inpractice, however,there is always

noisepresentbothintheimageandinthelterswhichwillhelp thealgorithmtofocus

Figure 5 show the results of the algorithm on a test image. The original image is

displayed to the left and contain ambiguous neighbourhoods where both the relative

orientationandrelativephaseoftheeventsvary. Onthetoptotherightthemagnitude

oftwodominanteventsjR

a

j andjR

b

jareestimated. Note that jR

b

j iscloseto zerofor

one-dimensionalneighbourhoods. Theargument(orientation)oftheestimatedeventsis

encodedin colourwhich unfortunatelycannotbereproducedhere.

It is naturally possible go on and estimate more than two events in many cases,

asthere are no builtin limitationsin thealgorithm nor in the proposed model. It is,

however, important to consider the possibility to average the results in a meaningful

way. The output images on top of Figure 5 corresponding to R

a and R

b

can not be

averagedasthereisnoguaranteethattheeventsmaynot ipbetweentheimages. For

two eventsthis problem canbesolved bythe followingbijectivetransformation which

permitameaningfulaveraging:

[R a +R b ] Doubleangleof [R a R b ] (9)

The result is displayed in the lower part of Figure 5. Formore than two events the

averagingmustbeperformedin threedimensions,(x;y;').

Tosumup,theproposedalgorithmwillperformarobustandmorecomplete

estima-tionofacomplexandnoisyneighbourhoodcomparedtoearliermethodsbasedonlocal

one-dimensionality. Theresultscanbeaveragedandtswellinahierarchicalprocessing

structure. Thisconceptcanfurtheron,bysimplemodicationsbeusedtodetectspecial

casesof ambiguousneighbourhoods in a direct way. It is forexample possible that in

aone level operation detect orthogonal crossesbetweena lightand adark line, orto

estimatetheanglebetweentwocrossinglines.

References

[1] GostaH. Granlund: InSearchofaGeneralPictureProcessingOperator.Computer

GraphicsandImageProcessing,Vol.8,No.2,pp155-178,October1978.

[2] Hans Knutson: Filtering and Reconstruction in Image Processing, Thesis No.88,

LinkopingUniversity,1982.

[3] WilliamT.Freeman&EdwardH.Adelson: TheDesignandUseofSteerableFilters

for Image Analysis, Enhancement, and Wavelet Representation, Technical Report

126a,MassachusettsInstituteofTechnology,September1990.

[4] PietroPerona: DeformablekernelsforEarlyVision,ProceedingsofCVPR June91,

MauiHawaii.

[5] RonaldN.Bracewell: TheFouriertransformandItsApplications.McGraw-Hill,New