Backprojection of Some Image Symmetries Based on a Local Orientation Description

Based on a Local Orientation Description

Bjorn Johansson

Computer VisionLaboratory Department of ElectricalEngineering

LinkopingUniversity, SE-581 83Linkoping, Sweden bjorn@isy.liu.se

LiTH-ISY-R-2311 2000-10-30 ISSN1400-3902

Backprojection of Some Image Symmetries

Based on a Local Orientation Description

Bj¨orn Johansson

Computer Vision Laboratory

Department of Electrical Engineering

Link¨oping University, SE-581 83 Link¨oping, Sweden

bjorn@isy.liu.se

October 30, 2000

Abstract

Some image patterns, e.g. circles, hyperbolic curves, star patterns etc., can be described in a compact way using local orientation. The features mentioned above is part of a family of patterns called rotational symmetries. This theory can be used to detect image patterns from the local orientation in double angle representation of an images. Some of the rotational symmetries are described originally from the local orientation without being designed to detect a certain feature. The question is then: given a description in double angle representation, what kind of image features does this description correspond to? This ’inverse’, or backprojection, is not unambiguous - many patterns has the same local orientation description.

This report answers this question for the case of rotational symmetries and also for some other descriptions.

Contents

1 Introduction 3

2 Local orientation in double angle representation, z 3 3 Some symmetries on local orientation z 4

3.1 General assumption . . . 5

3.2 Rotational symmetries, z = ei2β(ϕ) . . . 5

3.2.1 Theory . . . 5

3.2.2 Experiments . . . 8

3.3 Polar symmetries, z = ei(n1ln r+n2ϕ+α) _{. . . . 9}

3.3.1 Theory . . . 9

3.3.2 Experiments . . . 10

3.4 Cartesian symmetries, z = ei(a1x+a2y+α) _{. . . . 11}

3.4.1 Theory . . . 11

3.4.2 Experiments . . . 12

1

Introduction

Human vision seems to work in a hierarchical way in that we first extract low level fea-tures such as local orientation and color and then higher level feafea-tures. No one knows for sure what these high level features are but there are some indications that curvature, circles, spiral- and star patters are among them [7], [13]. Perceptual experiments also indicate that corners and curvature are very important features in the process of recog-nition and one can often recognize an object from its curvature alone [1], [3].

One way to detect the features mentioned above is to use the theory of rotational sym-metries. They have been described in the literature a number of times, see e.g. [6], [2], [5], [11], [12], [4], [9], [10]. Hyperbolic-, circular-, star-, and other patterns can be described in a compact way using local orientation in double angle representation. They can therefore be detected using simple correlations on the local orientation im-age. These descriptions can be generalized to include a larger class of image patterns called rotational symmetries. This larger class is described on the local orientation and therefore we do not know directly what image patterns they describe. The most useful rotational symmetries has been proven to describe the patterns mentioned above, but there are still unexplored symmetries. The question is basically: given a description on local orientation, what class of image patterns does this correspond to? This report answers this question for the case of rotational symmetries and also for some other descriptions. The double angle representation is non-linear and the answer is therefore not trivial.

This report contains a lot of tedious mathematical calculations, but the idea is fairly easy: Start with a local orientation description in double angle representation. In each point, decode this representation into an orientation angle and assume that the image gradient has the same angle. When we know the image gradient we can solve a differ-ential equation to get the underlying image pattern (the solution is not unique).

2

Local orientation in double angle representation,

z

There exists a number of ways to detect local orientation in images. We will not deal with these methods in this report but rather concentrate on orientation representations. The classical representation of local orientation is simply a 2D-vector pointing in the dominant direction with a magnitude that reflects the orientation dominance (e.g. the energy in the dominant direction). An example of this is the image gradient.

A better representation is the double angle representation, where we have a vector, or a complex number z, pointing in the double angle direction. This means that if the ori-entation has the direction θ we represent it with a vector pointing in the 2θ-direction, i.e. z = ei2θ_{. Figure 1 illustrates the idea.}

This representation has at least two advantages:

• We avoid ambiguities in the representation: It does not matter if we choose to say that the orientation has the direction θ or, equivalently, θ + π. In the double

Re z Im z

Figure 1: Double angle representation of local orientation.

angle representation both choices get the same descriptor ei2θ_.

• Averaging the double angle orientation description field makes sense. One can argue that two orthogonal orientations should have maximally different represen-tations, e.g. vectors that point in opposite directions. This is for instance useful in color images and vector fields when we want to fuse descriptors from several channels into one description.

This descriptor will in this paper be denoted by a complex number z. The argument

arg z points out the dominant orientation and the magnitude |z| usually represent the

certainty or energy of the orientation.

3

Some symmetries on local orientation

z

Lets repeat the question we are trying to answer: given a description on local orienta-tion, what class of image patterns does this correspond to?

This section start with the orientation image z and go backwards to the original image,f, from which the orientation image was calculated, i.e.

Double angle representation z ⇒ Grayscale image f

This ’inverse’ is of course not unambiguous, but we get a hint by making the assump-tion that the image gradient is parallel to the dominant orientaassump-tion.

There are two assumptions made in this report. The first one, mentioned above, is not very restrictive and says that the image gradient is parallel to the local orientation. The second one assumes that the image pattern can be described as a separable function in some suitably chosen coordinate system which depend on the selected symmetry.

3.1

General assumption

The assumption that the image gradient ∇f is parallel to the local orientation β = β(x, y) can be written mathematically as

∇f =  f_x f_y  //  cos β sin β  , where β = 1 2arg z (1) i.e.  f_x f_y  = ±qf2 x+ fy2  cos β sin β  (2) We have to divide the phase (arg z) by two to get rid of the double angle representation. The price is the direction unambiguity (±).

Equation 2 squared gives

( f2 x = fx2+ fy2
cos2_β f2 y = fx2+ fy2

sin2_{β ⇔}fxsin β = ±fycos β (3)

From equation 2 we see that the ’-’-solution is false and we arrive at the final equation: Backprojection equation:

f_xsin β = f_ycos β (4)

Every pattern description based on the local orientation is described by a function β(x, y). This function can be put into equation 4 which can be solved to get the corre-sponding gray-image pattern f (x, y).

The remaining subsections solve the backprojection equation for some special cases of β(x, y). The first subsection deals with the rotational symmetry class. The other two subsections deals with polar and Cartesian symmetries, which may not be as useful as the rotational symmetries, but are included more as curiosity and art.

3.2

Rotational symmetries,

z = e

3.2.1 Theory

The name rotational symmetries is not entirely logical but it is an established name and should not be changed. They are actually a special case of polar symmetries described in section 3.3. They can be described using the local orientation description as

z = ei2β(ϕ) ₍₅₎

The orientation β only depends on ϕ and is constant along the r-dimension. The most useful ones are the zeroth, first, and second order rotational symmetries:

They are proven to describe patterns like corners, curvature, circles, and stars (see e.g. [8]). The other rotational symmetries has not been thoroughly examined before. We shall now solve the backprojection equation 4 assuming β = β(ϕ). In this case it is easier to switch to polar coordinates:

(

x = r cos ϕ

y = r sin ϕ (7)

The derivatives f_xand f_ycan be written in polar coordinates using the chain-rule:

( f_x= ∂f ∂x = ∂f ∂r∂x∂r+ ∂f ∂ϕ ∂ϕ ∂x = frcos ϕ − fϕsin ϕr f_y=∂f ∂y = ∂f ∂r∂r∂y+ ∂f ∂ϕ ∂ϕ

∂y = frsin ϕ + fϕcos ϕr

(8)

If this is inserted in equation 4 we get



f_rcos ϕ − fϕ

sin ϕ

r



sin β =f_rsin ϕ + f_ϕcos ϕ

r



cos β (9)

After some re-shuffling we get the polar version of equation 4: Polar backprojection equation: f_rr sin(β − ϕ) = fϕcos(β − ϕ)

(10)

This equation is still quite difficult to solve, but if we make the assumption that f is polar separable, i.e. f (r, ϕ) = R(r)Φ(ϕ), we get

R0_{(r)Φ(ϕ)r sin(β − ϕ) = R(r)Φ}0_{(ϕ) cos(β − ϕ)} ⇔ R0_(r) R(r)r = Φ0_(ϕ) Φ(ϕ) coth(β − ϕ) (11)

Since β only depends on ϕ we know that the left side only depends on r and the right side only depends on ϕ. Therefore both sides have to be constant:

( _R0_(r) R(r)r = K Φ0_(ϕ) Φ(ϕ) coth(β − ϕ) = K ⇒ ( R(r) = C₁rK Φ(ϕ) = C₂eKRtan(β−ϕ)dϕ (12) And we get Rotational symmetries: f(r, ϕ) = C(reRtan(β(ϕ)−ϕ)dϕ₎K (13)

Provided a function β(ϕ), this function can be solved numerically to get the final solu-tion.

There are some cases where we can solve equation 13 analytically. Suppose z is the n:th order rotational symmetry ei(nϕ+α)_{, i.e β(ϕ) =} 1

2nϕ +12α. Then we get Φ(ϕ) = C₂eKRtan((n₂−1)ϕ+α₂)dϕ_{= / ∗ / =} = C₂e−Kn1 2 −1ln | cos(( n 2−1)ϕ+α2)| ₌ = C₂| cos((n 2− 1)ϕ +α2)| −_nK 2 −1 (14)

/ ∗ / There is one exception to the solution above: If n = 2 we get

Φ(ϕ) = C₂eKRtan(α₂)dϕ₌

= C₂eK tan(α₂)ϕ (15) If we choose choose K = 1−n₂ and skip the| . | in the case n 6= 2 and K = 1 in the case n = 2 we get the final solution:

n:th order rotational symmetry: f(r, ϕ) = Cr(1−n 2)cos((n 2 − 1)ϕ +α2) n 6= 2 f(r, ϕ) = Cretan(α 2)ϕ n = 2 (16)

What does the patterns in equation 16 look like?

One way to visualize them is to plot trajectories or isobars (inspired by [5]). To get the trajectories we can for instance plot

g(r, ϕ) = 1 + cos(ωf(r, ϕ))

2 (17)

in the case n6= 2. ω determines the frequency of the repetitive pattern. For the case n = 2 is turns out that

g(r, ϕ) = 1

2(1 + cos(ω cos(α2) ln f(r, ϕ))) = = 1

2(1 + cos(ω(cos(α2) ln r + sin(α2)ϕ)))

(18)

is a better, well behaved, choice. It is easy to show that if f (r, ϕ) is a solution to the backprojection equation 4 then every function g(r, ϕ) = h(f (r, ϕ)) is also a solution. We can thus generate a bigger class of functions than polar separable functions that solves the symmetry equation. Figure 2 shows some examples of functions from equa-tion 16.

Another choice of g could be a log-norm function: g(r, ϕ) = e−ω ln2_{(100·f(r,ϕ)/ω)}

(19) This will give a non-repetitive pattern (a trajectory of f ). Different ω gives different patterns, e.g different ω when plotting the first order symmetry (n = 1) will give various degree of curvature, see figure 3.

3.2.2 Experiments

Figure 2 and 3 contains some examples of the functions described in equation 16.

n = −4

α

= 0

α

=

π

/2

α

=

π

α

= 3

π

/2

n = −3 n = −2 n = −1 n = 0

n = 1

n = 2

n = 3

n = 4

n = 5

Figure 2: Some examples of rotational symmetries z = ei(nϕ+α)_{. The backprojection}

is found in equation 16 and they are plotted using trajectory functions 17 and 18.

ω = 5 ω = 10 ω = 20 ω = 40 ω = 80

Figure 3: Some examples from equation 16 using function 19.

There are a lot of other patterns that can be described as a rotational symmetry. The figure shows some examples. In these cases we have to use equation 13 and approxi-mate the integral.

z = e−2iϕ_{+ e}2iϕ _{z = e}−1iϕ_{+ e}2iϕ _{z = e}−4iϕ_{+ 1.5e}2iϕ

z = −1 + e2iϕ_{+ 0.5e}4iϕ _{z = −e}−iϕ_{+ e}iϕ

3.3

Polar symmetries,

z = e

Just out of curiosity it could be interesting to see what happens if we also let the sym-metry description depend on the radius. Therefore we try to examine the description

z = ei(n1ln r+n2ϕ+α) ₍₂₀₎

(We select ln r instead of just r because that gives easier equations below.)

3.3.1 Theory

As before we start from the polar backprojection equation 10: f_rr sin(β − ϕ) = fϕcos(β − ϕ)

In this case we have

β = m₁ln r + m₂ϕ + v where m₁=n1 2 , m2= n₂ 2 , v = α 2 (21)

Instead of switching to polar coordinates as we did in the previous case we make the following substitution: ( s = m₁ln r + (m₂− 1)ϕ t = (1 − m2) ln r + m1ϕ (22) This gives ⇒ ( f_r = f_sm1 r + ft1−mr 2 f_ϕ = f_s(m₂− 1) + ftm1 (23)

If this is inserted in the polar backprojection equation we get

(m₁f_s+ (1 − m2)ft) sin(s + v) = ((m2− 1)fs+ m1ft) cos(s + v)

⇔

f_s(m₁sin(s + v) + (1 − m2) cos(s + v)) = ft(m1cos(s + v) − (1 − m2) sin(s + v))

⇔ f_scos(s + v − arctan m1

1−m2) = −ftsin(s + v − arctan

m1 1−m2)

This finally gives

f_scos ϑ = −ftsin ϑ (24)

where ϑ = s + v− arctan m1 1−m2

Now assume the function f is separable in the s, t variables, that is

If this is inserted in equation 24 we get S0_{T cos ϑ = −ST}0_{sin ϑ ⇔ −}S0 S cos ϑ sin ϑ = T0 T = const = K (26) The solution becomes:

(

S(s) = C₁eK ln | cos ϑ|_{= C}

1| cos ϑ|K

T (t) = C₂eKt (27)

Which give the final formula:

f(r, ϕ) = CrK(1−m2)eKm1ϕ_{| cos(m}

1lnr+(1−m2)ϕ+v−arctan1−mm12)|

K ₍₂₈₎

3.3.2 Experiments

Some functions from equation 28 is plotted in figure 5.

n2 = −3 n1 = −3 n2 = −2 n2 = −1 n2 = 0 n2 = 1 n2 = 2 n2 = 3 n1 = −2 n1 = −1 n1 = 0 n1 = 1 n1 = 2 n1 = 3

Figure 5: Some examples of polar symmetries z = ei(n1ln r+n2ϕ)_{. The}

back-projection is found in equation 28 and they are plotted using trajectory functions g = (1 + cos(ω log f))/2.

3.4

Cartesian symmetries,

z = e

3.4.1 Theory

Start from the backprojection equation 4:

f_xsin β = f_ycos β

Make the substitution

(

u = eγ_{cos β}

v = eγ_{sin β} where γ is defined by

( γ_x = −βy γ_y = β_x (29) which gives f_x = f_x=∂f ∂u∂u∂x+ ∂f ∂v∂v∂x= = f_ueγ_(γ

xcos β − βxsin β) + fveγ(γxsin β + βxcos β) =

= f_u(−βyu − βxv) + fv(−βyv + βxu)

f_y = f_y= ∂f ∂u∂u∂y +

∂f ∂v∂v∂y =

= f_ueγ_(γ

ycos β − βysin β) + fveγ(γysin β + βycos β) =

= f_u(β_xu − βyv) + fv(βxv + βyu)

(30)

If this is inserted in the backprojection equation we get f_u(β_xu − βyv) + fv(βxv + βyu)
u = f_u(−βyu − βxv)fv(−βyv + βxu)
v ⇔ f_u(β_xu2_{− β} yuv + βyuv + βxv2) = fv(−βyv2+ βxuv − βxuv − βyu2)

This finally gives

f_uβ_x= −fvβy (31)

In this case we have β = 1

2(a1x + a2y + α) ⇒ γ = 1

2(−a2x + a1y) (32)

Equation 31 then becomes

a₁f_u= −a2fv ⇔ (a1_∂u∂ + a2_∂v∂ )f = 0 (33) If we make the substitution

(

u = a₁U + a₂V

we have_∂U∂ = a_1∂u∂ + a_2∂v∂ and the differential equation becomes ∂

∂Uf = 0 (35)

which has the solution

f(U, V ) = h₁(V ) = h₁(a2u−a1v a21+a22 ) = h2(a2u − a1v) = = h₂(a₂eγ_{cos β − a} 1eγsin β) = = h₂(eγp_a2 1+ a22cos(β + arctanaa12)) (36)

And the final formula becomes f(x, y) = e1

2(−a2x+a1y)_cos(a1

2x + a22y +α2 + arctanaa12) (37)

3.4.2 Experiments

Some functions from equation 37 is plotted in figure 6.

a2 = −3 a1 = −3 a2 = −2 a2 = −1 a2 = 0 a2 = 1 a2 = 2 a2 = 3 a1 = −2 a1 = −1 a1 = 0 a1 = 1 a1 = 2 a1 = 3

Figure 6: Some examples of polar symmetries z = ei(a1x+a2y)_{. The}

backprojec-tion is found in equabackprojec-tion 37 and they are plotted using trajectory funcbackprojec-tions g =

4

Acknowledgment

This work was supported by the Swedish Foundation for Strategic Research, project VISIT - VIsual Information Technology.

References

[1] F. Attneave. Some informational aspects of visual perception. Psychological Review, 61, 1954.

[2] H. B˚arman and G. H. Granlund. Corner detection using local symmetry. In Pro-ceedings from SSAB Symposium on Picture Processing, Lund University, Sweden, March 1988. SSAB. Report LiTH–ISY–I–0935, Computer Vision Laboratory, Link¨oping University, Sweden, 1988.

[3] Irving Biederman. Recognition-by-components: A theory of human image un-derstanding. Psychological Review, 94(2):115–147, 1987.

[4] J. Big¨un. Optimal orientation detection of circular symmetry. Report LiTH–ISY– I–0871, Computer Vision Laboratory, Link¨oping University, Sweden, 1987. [5] J. Big¨un. Local Symmetry Features in Image Processing. PhD thesis, Link¨oping

University, Sweden, 1988. Dissertation No 179, ISBN 91-7870-334-4.

[6] Josef Big¨un. Pattern recognition in images by symmetries and coordinate trans-formations. Computer Vision and Image Understanding, 68(3):290–307, 1997. [7] Jack L. Gallant, Jochen Braun, and David C. Van Essen. Selectivity for polar,

hyperbolic, and cartesian gratings in macaque visual cortex. Science, 259:100– 103, January 1993.

[8] G. H. Granlund and H. Knutsson. Signal Processing for Computer Vision. Kluwer Academic Publishers, 1995. ISBN 0-7923-9530-1.

[9] Bj¨orn Johansson and G¨osta Granlund. Fast Selective Detection of Rotational Symmetries using Normalized Inhibition. In Proceedings of the 6th European Conference on Computer Vision, volume I, pages 871–887, Dublin, Ireland, June 2000.

[10] Bj¨orn Johansson, Hans Knutsson, and G¨osta Granlund. Detecting Rotational Symmetries using Normalized Convolution. In Proceedings of the 15th Interna-tional Conference on Pattern Recognition, volume 3, pages 500–504, Barcelona, Spain, September 2000. IAPR.

[11] H. Knutsson and G. H. Granlund. Apparatus for determining the degree of vari-ation of a feature in a region of an image that is divided into discrete picture elements. US-Patent 4.747.151, 1988, 1988. (Swedish patent 1986).

[12] H. Knutsson, M. Hedlund, and G. H. Granlund. Apparatus for determining the degree of consistency of a feature in a region of an image that is divided into discrete picture elements. US-Patent 4.747.152, 1988), 1988. (Swedish patent 1986).