JaphetNiyobuhungiro OptimalDecompositioninRealInterpolationandDualityinConvexAnalysis

(1)

Linköping Studies in Science and Technology.

Thesis No. 1604

Optimal Decomposition in Real

Interpolation and Duality in Convex

Analysis

Japhet Niyobuhungiro

Department of Mathematics

Division of Mathematics and Applied Mathematics

Linköping University, SE–581 83 Linköping, Sweden

(2)

Thesis No. 1604

Optimal Decomposition in Real Interpolation and Duality in Convex Analysis Japhet Niyobuhungiro

japhet.niyobuhungiro@liu.se www.mai.liu.se

Mathematics and Applied Mathematics Department of Mathematics

Linköping University SE–581 83 Linköping

Sweden

(3)

To my sisters and my brothers; To my friends.

(4)

(5)

Abstract

This thesis is devoted to the study of mathematical properties of exact mini-mizers for the K–,L–, and E– functionals of the theory of real interpolation. Re-cently exact minimizers for these functionals have appeared in important results in image processing.

In the thesis, we present a geometry of optimal decomposition for L– func-tional for the couple(`2, X), where space`2is defined by the standard Euclidean normk·k₂and where X is a Banach space onRn. The well known ROF denois-ing model is a special case of an L– functional for the couple L2, BV where L2 _{and BV stand for the space of square integrable functions and the space of}

functions with bounded variation on a rectangular domain respectively. We pro-vide simple proofs and geometrical interpretation of optimal decomposition by following ideas by Yves Meyer who has used a duality approach to characterize optimal decomposition for ROF denoising model.

The operation of infimal convolution is a very important and non–trivial tool in functional analysis and is also very well–known within the context of convex analysis. The L–, K– and E– functionals can be regarded as an infimal convolution of two well defined functions but unfortunately tools from convex analysis can not be applied in a straigtforward way in this context of couples of spaces. We have considered infimal convolution on Banach couples and by using a theorem due to Attouch and Brezis, we have established sufficient conditions for an infimal convolution on a given Banach couple to be subdifferentiable, which turns out to be the most important requirement that an infimal convolu-tion would satisfy for a decomposiconvolu-tion to be optimal. We have also provided a lemma that we have named Key Lemma, which characterizes optimal decompo-sition for an infimal convolution in general.

The main results concerning mathematical properties of optimal decomposi-tion for L–, K– and E– funcdecomposi-tionals for the case of general regular Banach couples are presented. We use a duality approach which can be summarized in three steps: First we consider the concerned functional as an infimal convolution and reformulate the infimal convolution at hand as a minimization of a sum of two specific functions on the intersection of the couple. Then we prove that it is sub-differentiable and finally use the characterizaton of its optimal decomposition.

We have also investigated how powerful our approach is by applying it to two well–known optimization problems, namely convex and linear programming. As a result we have obtained new proofs for duality theorems which are central for these problems.

(6)

(7)

Populärvetenskaplig sammanfattning

I avhandlingen studeras de matematiska egenskaperna hos exakta minimerare till K–, L– och E–funktionalerna från reell interpolationsteori. Exakta minimerare till dessa funktionaler har nyligen visat sig vara viktiga inom bildbehandling.

I avhandlingen presenteras en geometrisk beskrivning för den optimala upp-delningen av den L–funktional som ges av paret (`2, X), där rummet`2är se-kvenser av reella tal med standardnormen k·k2 och X är ett Banachrum på

Rn_{. Den välkända brusreduceringsmodellen ROF kan ses som ett specialfall där}

motsvarande L–funktional ges av paret (L2_{, BV}₎_{, där L}2 _{är kvadratintegrabla}

funktioner och BV betyder funktioner med begränsad variation. Vi ger enkla bevis och en geometrisk tolkning av den optimala uppdelningen av funktiona-len. Idéer har hämtats från Yves Meyer som tidigare använt dualitetsteori för ett karakterisera den optimala uppdelningen av den operator som svarar mot brusreduceringsmodellen ROF.

Infimalfaltning är en viktig, men komplicerad, operator inom funktionalana-lysen. Denna operatator förekommer också inom konvex analys. Funktionaler av K–, L– och E–typ kan ses som infimalfaltningar av två väldefinierade funk-tioner men tyvärr är det inte enkelt att använda de verktyg som utvecklats inom den konvexa analysen för att studera dessa funktionaler. I avhandlingen stu-derar vi infimalfaltning i fallet där de båda ingående funktionerna tillhör olika Banachrum. Vi använder ett resultat av Attouch och Brezis för att ge tillräckliga villkor som garanterar att en infimalfaltning, definierad på ett Banachpar, har en subgradient, vilket visar sig vara det viktigaste kriteriet en infimalfaltning måste uppfylla för att den skall representera en optimal uppdelning. Vi formu-lerar och bevisar också ett lemma som karakteriserar optimala uppdelningar för infimalfaltningar generellt.

Vi tillämpar ovanstående teori genom att presentera ett antal satser som ka-rakteriserar den optimala uppdelningen av K–, L– och E–funktionaler för fallet där funktionalerna definieras av par av Banachrum. Vårt angreppssätt kan be-skrivas med följande steg: Först betraktar vi den aktuella funktionalen som en infimalfaltning och omformulerar den som ett problem där en summa av två funktioner minimeras på genomskärningen av paret av Banachrum. Vi bevisar sedan att infimalfaltningen har en subgradient och slutligen använder vi detta, tillsammans med en karakterisering av dess optimala uppdelning, för att be-visa kapitlets huvudresultat. Dessutom tillämpar vi vår teori på två välkända optimeringsproblem. Resultatet är två nya bevis för för dualitetssatserna inom konvex och linjär programmering.

(8)

(9)

Acknowledgments

I thank my main advisors Natan Kruglyak and Irina Asekritova for their guid-ance and advice as my teachers and in the writing of this thesis. They intro-duced me to the subject and have beeen providing me with support along the way. Thanks to Natan for teaching me a lot about writing too. Irina’s helpful ideas and discussions were invaluable and suggested many improvements.

I thank my assistant advisors Fredrik Berntsson and Froduald Minani for the helpful part they played in my mathematical development as my teachers.

I thank Bengt Ove Turesson, Björn Textorius and all the administrative staff members of the Department of Mathematics for their help whenever need arises at work.

I thank everyone at home. My parents, sisters, brothers, and friends for their personal care and attention.

I would also like to thank all my fellow PhD students for making Linköping such a nice place to live and work.

Lastly I wish to acknowledge the financial support I received through Sida/Sarec funded National University of Rwanda-Linköping University cooperation. All involved institutions and people are hereby acknowledged.

Linköping, June 3, 2013 Japhet Niyobuhungiro

(10)

(11)

0

Introduction

0.1 Background

Several functionals such as L–, K– and E– functionals are very important in the theory of real interpolation. A more or less detailed theory on these functionals can be found for example in the books [1, 2]. Another good reference is the book [3]. Given a couple of Banach spaces (X0, X1), an element x ∈ X0+X1 and a

positive parameter t, the K– functional is defined by the formula

K(t, x; X0, X1) = inf w∈X1 kx−wk_X 0+tkwkX1 .

The K– functional is at the center of the so–called K– method of real interpola-tion that is basically concerned with the construcinterpola-tion of suitable families of real interpolation spaces between X0and X1. The K– functional is a particular case

of the more general L– functional which is defined by

Lp0,p1(t, x; X0, X1) =_w∈Xinf 1 kx−wkp0 X0+tkwk p1 X1 , (1) for 1≤ p0, p1<∞.

Definition 0.1 (Exact and near minimizers). We say that the element (which depends on x and t) wt ∈ X1 is a near minimizer for the functional (1) if there

exists C>0 independent of x and t such that

kx−wtkp_X0₀+tkwtk_Xp1₁ ≤CLp0,p1(t, x; X0, X1).

(14)

If C=1, then wtis called exact minimizer. If wt∈ X1is an exact minimizer, then

we will call

x=wt+ (x−wt), (2)

optimal decomposition for (1) corresponding to x.

The E– functional is basically seen as a distance functional and is defined by the expression

E(t, x; X0, X1) = inf

kwk_X1≤tkx−wkX0.

Remark 0.1. It is important to note that the optimal decomposition does not always exist.

Let (X0, X1) be a regular Banach couple, i.e., X0 and X1 are both Banach

spaces which are linearly and continuously embedded in the same Hausdorff topological vector space and moreover the intersection X0∩X1is dense in both

X0 and X1. Given an element x in X0+X1 and some parameter t > 0, we

consider the following L– functional

Lp0,p1(t, x; X0, X1) =_x=xinf 0+x1 1 p0 kx0kXp00+ t p1 kx1kXp11 , (3) for 1≤p0, p1<∞.

Problem. Suppose that the optimal decomposition x=x0,opt+x1,optfor K– functional

(respectively for L– and E– functionals) corresponding to the element x exists. Give a characterization of this decomposition. In other words, what are the mathematical properties of x0,optand x1,opt. For example, in the case of the L– functional (3), we want

the mathematical properties of the decomposition x=x0,opt+x1,optsuch that

Lp0,p1(t, x; X0, X1) = 1 p0 x_0,opt p0 X0+ t p1 x_1,opt p1 X1.

The L– functional is deeply connected to the well–known Rudin–Osher– Fatemi (ROF) image denoising model. Denoising is the problem of removing noise from an image. The most commonly studied case is with additive white Gaussian noise, where the observed noisy image f ∈ L2is related to the under-lying true image f∗by

f = f∗+η,

where the noise η∈ L2_{. The ROF model, also known as Total Variation (TV)}

regu-larization technique, proposes to approximate the true image f∗by the function

ft∈BV which minimizes the L2,1– functional for the couple L2, BV:

L2,1 t, f ; L2, BV= inf g∈BV kf −gk2_L2+tkgk_BV , (4)

(15)

0.1 Background 3

where L2 and BV stand for the space of square integrable functions and the space of functions with bounded variation on a rectangular domain respectively. Since its appearance in 1992, the ROF model [4] has been successful and popular and it has since been applied to a multitude of other imaging problems (see for example the book [5]). The problem of constructing exact minimizer for the functional (4) is difficult. Let us mention that when the following estimate of the noise is known

kηk_L2 ≤ε,

the so–called Morozov discrepancy principle (see [6]) suggests choosing t > 0 such that

kf −ftkL2 =ε.

The underlying idea of the Morozov principle can be explained from the point of view of interpolation theory. This has been done in the paper [7] by F. Cobos and N. Kruglyak who provided an algorithm that constructs an exact minimizer for the E–functional

E(t, f ; L∞, BV) = inf

kgk_L∞≤tkf − −gkBV, (5)

where L∞and BV are spaces of bounded functions and functions with bounded variation on an interval[a, b], respectively. They have also discussed connections of their results with the Rudin–Osher–Fatemi denoising model.

Different approaches such as PDE and wavelet–based approaches have been pro-posed (see for example the books [5, 8] and the paper [9]) for approximately constructing ft. Recently Kislyakov and Kruglyak in their book [10] considered

a similar problem for the couple of Sobolev spaces Lp, ˙Wq,k, however their approach gives only near minimizer, not exact minimizers.

In 2010, I. Asekritova and N. Kruglyak also presented an algorithm for the con-struction of a near minimizer for the couple L2, BV based on piecewise con-stant approximation and the Besicovitch covering theorem [11]. In 2002, in his book [12], Yves Meyer obtained a characterization of optimal decomposition for the ROF functional by using duality.

It is clear that the ROF model is a particular case of the L– functional (3) for p0 = 2, p1 = 1 and for the spaces X0 = L2(D) and X1 = BV(D) for some

rectangular domain D. Thus it is an interesting problem to study the properties of exact minimizer for L– functional in its general formulation on regular Banach couples.

(16)

0.2 Summary of main results

To begin with, let us once again consider the Rudin, Osher and Fatemi image denoising model. Let D= [a, b] × [c, d]be a rectangular domain inR2. Suppose that the unknown initial image f∗∈BV and we observe

fob= f∗+η,

where η ∈ L2(D) corresponds to noise. In order to reconstruct approximately the initial image f∗, the ROF model suggests considering the functional

L2,1 t, fob, L2(D), BV(D) = inf g∈BV 1 2kfob−gk 2 L2+tkgk_BV ,

and taking as approximation to f∗ the function ft which minimizes this

func-tional, i.e., the function ft∈BV such that

L2,1 t, fob, L2(D), BV(D) = 1 2kfob−ftk 2 L2+tkftkBV.

Note that for a function f of class C1, the seminorm in the space BV can be defined by kfk_BV = Z Z D ∂ f ∂x(x, y) + ∂ f ∂y(x, y) dxdy.

We call the expression

fob= (fob− ft) + ft,

the optimal decomposition for L2,1 t, fob, L2(D), BV(D) corresponding to fob. In

2002, Yves Meyer obtained a mathematical characterization of this optimal de-composition by using duality [12].

Based on Yves Meyer’s approach and aiming at an understanding of the geometrical interpretation of this optimal decomposition, in Chapter 2 of this thesis we have considered the same model for the couple `2_{, X on}_Rn_{, i.e., we}

consider the L– functional L2,1(t, x;`2, X) =_x=xinf 0+x1 1 2kx0k 2 `2+tkx1kX = inf x0∈`2 1 2kx0k 2 `2 +tkx−x0kX ,

where space`2is defined by the usual Euclidean normkxk_`2 =

∑n i=1|xi|2

1/2 for x∈Rn_{, X is a Banach space on}_Rn_{and t is a positive parameter. In order to}

formulate the result, let us consider a closed ball of radius t in the normk·k_X∗

of X∗centered at 0 and denote it by tB_X∗:

tBX∗ ={y∈Rn :kyk_X∗ ≤t}.

Our problem is to find x0,opt ∈ X0and x1,opt ∈ X1 such that x = x0,opt+x1,opt

and L2,1(t, x;`2, X) = 1 2 x0,opt 2 `2 +t x1,opt _X. It appears that we must consider two cases:

(17)

0.2 Summary of main results 5

(Case 1) kxk_X∗ ≤t

In this case the optimal decomposition for L2,1 t, x;`2, X is given by

x0,opt=x and x1,opt=0

(Case 2) kxk_X∗ >t

In this case the optimal decomposition for L2,1 t, x;`2, X is characterized

by the following theorem:

Theorem 0.1. Let x0,optbe an exact minimizer for L2,1(t, x;`2, X). Then x0,optis

the nearest element of tBX∗to the point x in the metric of`2, i.e.,

Et, x;`2, X∗= inf

kx0kX∗≤t

kx−x0k`2 =

x− −x0,opt _`₂.

The following figure gives the geometry of the optimal decomposition for couple `2_{, X.}

Figure 1: Illustration of Theorem 0.1.

Remark 0.2. Suppose that the given x is an image, like in the ROF model. Then x0,opt represents the noise component and x1,opt represents the image

component. Therefore ifkxk_X∗ ≤t then the whole image is treated as noise

and image component is regarded as nonexistent. Ifkxk_X∗ >t then x0,opt

is the nearest element of tBX∗ to the point x with respect to the metric of

`2, i.e., x_0,opt

X∗ =t and x0,opt, x−x0,opt=t x− −x_0,opt

(18)

Note that the noise component x0,opt belongs to the boundary of the set

tBX∗.

Remark 0.3. By a naive analogy, it seems from Theorem 0.1, that for the couple

(`p, X)for p6= 2 we would have the optimal decomposition x = x0,opt+x1,opt

for the L– functional

Lp,1(t, x;`p, X) = inf x=x0+x1 1 pkx0k p `p+tkx1kX , (6) is such that inf kx0kX∗≤t kx−x0k`p = x−x_0,opt `p. (7)

This conjecture would be illustrated geometrically in Figure 2.

Figure 2: Wrong geometry of optimal decomposition for Lp,1(t, x;`p, X); n=2;

p=3.

However, as we will see in Chapter 4, the situation in (7) does not hold true and the situation in Figure 2 is wrong. Therefore to address this issue we needed a different approach.

Below we will give a statement of the result. The approach used will be describe later in this section. Let(X0, X1)be a compatible Banach couple. i.e., X0and X1

(19)

in some Banach space X. Let x ∈ X0+X1, let 1 ≤ p < +∞ and t > 0. We

consider the L– functional

Lp,1(t, x; X0, X1) =_x=xinf 0+x1 1 p kx0k p X0+tkx1kX1 , (8)

We need to find a characterization of optimal decomposition for this L– functional. i.e., x=x0,opt+x1,optsuch that

Lp,1(t, x; X0, X1) = 1 p x_0,opt p X0+t x_1,opt X1.

Let(X0, X1)be a regular couple (X0∩X1is dense in both X0and X1). Then it is

a known fact from interpolation theory that X∗₀, X∗₁ also form a Banach couple and(X0∩X1)∗=X∗0+X∗1. The norm of the dual spaces is defined by:

kyk_X∗ j =sup n hy, xi: x∈Xj, kxkXj ≤1 o , j=0, 1.

The spaces X0+X1and X0∩X1are Banach spaces with respect to the following

norms kxk_X 0+X1 =_x=xinf₀_+x₁ n kx0kX0+kx1kX1 o ,

where the infimum extends over all representations x = x0+x1of x with x0in

X0and x1in X1, and kxk_X 0∩X1 =max n kxk_X 0,kxkX1 o .

Theorem 0.2. Let1 < p< +∞. The decomposition x =x0,opt+x1,optis optimal for

Lp,1(t, x; X0, X1)if and only if there exists y∗ ∈X∗₀∩X∗₁such thatky∗k_X∗

1 ≤t and ( ₁ p x0,opt p X0 = hy∗, x0,opti − − 1 p0ky∗kp 0 X∗ 0; t x_1,opt X1 = hy∗, x1,opti, where 1_p+ _p10 =1.

In order to illustrate the correct version of the wrong geometry which was illustrated in Figure 2, let us consider the particular case of couple (`p, X) on Rn_{. We have the L– functional}

Lp,1 t, x;`p, X= inf x=x0+x1 1 pkx0k p `p+tkx1kX ,

where 1<p< +∞. Consider the following function F0and its gradient:

F0(u) = 1 p kuk p `p, ∇F0(v) = n |v|p−1sgn(v)o. Let us define the setΩ by

Ω={v∈Rn _: _∇_F

(20)

As before, we also need to consider two cases: (Case 1) x∈Ω.

In this case, the optimal decomposition for Lp,1 t, x;`p, X is given by

x0,opt=x and x1,opt=0.

(Case 2) x /∈Ω.

In this case, the optimal decomposition for Lp,1 t, x;`p, X is characterized

by the following theorem:

Theorem 0.3. Let x be such that k∇F0(x)kX∗ > t. Then decomposition x =

x0,opt+x1,optis optimal for Lp,1 t, x;`p, X if and only if

(a) ∇F₀ x_0,opt X∗=t (b) x1,opt,∇F0 x0,opt =t x1,opt _X.

The Figure 3 gives the correct geometry of optimal decomposition for couple

(`p, X). The element x1,opt is orthogonal to the supporting hyperplane to tBX∗

at y∗= ∇F0 x0,opt.

(21)

Remark 0.4. For the case p=2, the setsΩ and tBX∗ coincide and we obtain the

situation as presented above for the couple `2, X.

It is interesting to note that in a general situation the setΩ could be noncon-vex and of rather complicated structure as illustrated by the following example:

Example 0.1

We present an example of illustration inR2_{. Consider the couple} _`3_{, X in the}

plane where the unit ball of X is the rotated ball of`1_{by the rotation matrix}

Rθ = cos θ sin θ −sin θ cos θ ,

for θ=30◦. We have that

kxk_X= R −1 θ x _`1 = √ 3 2 x1− 1 2x2 + 1 2x1+ √ 3 2 x2 . ∇F0(u) = h |u1|2sgn(u1),|u2|2sgn(u2) i . The setΩ can be written as

Ω= v∈R2: h |v1|2sgn(v1),|v2|2sgn(v2) iT _X∗ ≤t ,

where the norm in X∗is given by

kyk_X∗ = R −1 θ y _`_∞ =max ( √ 3 2 y1− 1 2y2 , 1 2y1+ √ 3 2 y2 ) . −1 −0.5 0 0.5 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.5 0 0.5 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

(22)

The Theorem 0.3 is illustrated in Figure 5. So we see that in this situation the setΩ is not convex.

−5 −4 −3 −2 −1 0 1 2 3 4 5 −4 −3 −2 −1 0 1 2 3 4 x Ω x0opt x1opt O area 1 area 3 area 4 −6 −4 −2 0 2 4 6 −5 −4 −3 −2 −1 0 1 2 3 4 5 tB X* y_∗ O area 1 area 3 area 4

Figure 5: Geometry of Optimal Decomposition for the Couple(`p, X)for p=3, t=2, X= R_θ `1_{and θ}₌₃₀◦_{. The set}_{Ω is illustrated on the left and t}_B

X∗on

the right.

In Chapter 4, we have obtained the results concerning optimal decomposition for K–, L– and E– functionals in a general case. For example the L– functional (8) is a particular case of the following general L– functional:

Lp0,p1(t, x; X0, X1) =_x=xinf 0+x1 1 p0 kx0k_Xp0₀+ t p1 kx1k_Xp1₁ , (9) where 1≤ p0, p1<∞.

Theorem 0.4. Let x ∈ X0+X1, 1 < p0, p1 <∞ and let t> 0 be a fixed parameter.

The decomposition x=x0,opt+x1,optis optimal for

Lp0,p1(t, x; X0, X1) =_x=xinf 0+x1 1 p0 kx0kXp00+ t p1 kx1kXp11 ,

if and only if there exists y∗∈X₀∗∩X₁∗such that

   1 p0 x_0,opt p0 X0 = hy∗, x0,opti − 1 p0₀ky∗k p0₀ X∗₀; t p1 x1,opt p1 X1 = hy∗, x1,opti − t p0 1 y∗ t p0₁ X∗₁. where 1_p+_p10 =1.

Below we describe in summary the approach used to prove the results. It is shown in Section 1.2 that the K–, L– and E– functionals can be considered as particular cases of infimal convolution, well–known in convex analysis.

(23)

Let(X0, X1)denote a regular Banach couple. Then there exist two specific

con-vex, lower semicontinuous and proper functions ϕ0 : X0 −→ R∪ {+∞} and

ϕ1: X1−→ R∪ {+∞}for each of the K–, L– and E– functionals such that they

can be written as a function F : X0+X1−→R∪ {+∞}defined by

F(x) = (ϕ0⊕ϕ1) (x) = inf x=x0+x1

(ϕ0(x0) +ϕ1(x1)), (10)

where the infimum extends over all representations x=x0+x1of x with x0and

x1 in X0+X1 and where ϕ0 : X0+X1 −→ R∪ {+∞} and ϕ1 : X0+X1 −→

R∪ {+∞} are respective extensions of ϕ0and ϕ1 on X0+X1 in the following

way ϕ0(u) = ϕ0(u) if u∈X0; +∞ if u∈ (X0+X1) \X0. (11) and ϕ1(u) = ϕ1(u) if u∈X1; +∞ if u∈ (X0+X1) \X1. (12)

For example, the L– functional (9) can be written as the infimal convolution Lp0,p1(t, x; X0, X1) = (ϕ0⊕ϕ1) (x), (13) where functions ϕ0and ϕ1are both defined on the sum X0+X1as follows

ϕ0(u) = ( ₁ p0 kuk p0 X0 if u∈X0; +∞ if u∈ (X0+X1) \X0. (14) and ϕ1(u) = ( _t p1 kuk p1 X1 if u∈X1; +∞ if u∈ (X0+X1) \X1. (15) In this case the functions ϕ0: X0−→R∪ {+∞}and ϕ1: X1−→R∪ {+∞}are

defined by ϕ0(u) = 1 p0 kukp0 X0 and ϕ1(u) = t p1 kukp1 X1. (16) However, it is important to notice that the extended functions ϕ0 and ϕ1could

stop to be lower semicontinuous even if ϕ0 and ϕ1 are. It turns out that the

most important requirement for an optimal decomposition corresponding to x is subdifferentiability of the function F = ϕ0⊕ϕ1 at x. However, since two

different Banach spaces are involved, some technical difficulties appear when you would like to apply known results in convex analysis. In this regard, we reconsider the infimal convolution F(x) = (ϕ0⊕ϕ1) (x)as follows:

F(x) = (ϕ0⊕ϕ1) (x) = inf y∈X0∩X1

(S(y) +R(y)), (17)

where S and R are functions defined on X0∩X1with values inR∪ {+∞}by

S(y) =ϕ0(a0−y) and R(y) =ϕ1(a1+y). (18)

The following theorem establishes conditions for which the function F=ϕ0⊕ϕ1

(24)

Theorem 0.5(Subdifferentiability of infimal convolution). Let the functions S and R be defined as in (18) and be convex, lower semicontinuous and proper. Let ϕ∗₀and

ϕ∗₁be the respective conjugate functions of ϕ0and ϕ1. Suppose that

(1) the sets dom S and dom R satisfy [

λ≥0

λ(dom S−dom R) =X0∩X1 (19)

(2) The conjugate function S∗of S is given by

S∗(z) =

ϕ∗₀(−z) +hz, a0i if z∈X∗0;

+∞ if z∈ X₀∗+X₁∗

\X₀∗. (20)

(3) The conjugate function R∗of R is given by

R∗(z) =

ϕ∗₁(z) +h−z, a1i if z∈X∗1;

+∞ if z∈ X₀∗+X₁∗

\X₁∗. (21)

Then the function(ϕ0⊕ϕ1)is subdifferentiable on its domain in X0+X1.

Finally, we apply the following lemma to the functions ϕ0and ϕ1.

Lemma 0.1(Key lemma). Let F0and F1be convex and proper functions from a Banach

space E with values inR∪ {+∞}such that their infimal convolution F = (F0⊕F1)

defined by

F(x) = (F0⊕F1) (x) = inf x=x0+x1

{F0(x0) +F1(x1)} (22)

is subdifferentiable at the point x∈dom(F0⊕F1). Then the decomposition x=x0,opt+

x1,optis optimal for (22) if and only if there exists y∗∈E∗that is dual to both x0,optand

x1,optwith respect to F0and F1respectively. i.e.,

F0 x0,opt= hy∗, x0,opti −F0∗(y∗);

F1 x1,opt

= hy∗, x1,opti −F1∗(y∗).

In the last chapter of the thesis, we have investigated how powerful our ap-proach is by applying it to two well–known optimization problems, namely con-vex and linear programming. The approach is as follows: First we reformulate the optimization problem at hand as an infimal convolution of two well–defined functions. Secondly, we check the subdifferentiability of the infimal convolution by Theorem 0.5 and finally use Lemma 0.1. As a result we have obtained new proofs for duality theorems which are central for these problems. Usually, in mathematical optimization theory, dual problem refers mostly to the Lagrangian dual problem or the Wolfe dual problem and the Fenchel dual problem for ex-ample. We have not found in literature proofs based on infimal convolution.

(25)

1

Preliminaries

In this chapter, we present some definitions and results from convex analysis and interpolation theory that are needed for the subsequent chapters. Throughout, E will denote a Banach space with the normk·k_E. The dual space E∗is the space of all bounded linear functionals y : E −→ R. Given y ∈ E∗, for y(x) we will writehy, xi with h· , ·idenoting the dual product between E∗ and E. We note that E∗ is a Banach space equipped with the norm

kyk_E∗=sup{|hy, xi|: x∈E; kxk_E ≤1}. (1.1)

In a finite dimensional case, one has to keep in mind that E=Rn=E∗, hy, xi =y · x, and norm in E∗is defined as in (1.1), where

kxk_E= (hx, xi)1/2.

1.1 Convex analysis

Throughout this section, we will consider convex functions on a Banach space E with values inR∪ {+∞}.

Definition 1.1. Let F be a function from E intoR∪ {+∞}. F is said to be convex if it satisfies:

F(λx1+ (1−λ)x2) ≤λF(x1) + (1−λ)F(x2), ∀x1, x1∈E;∀λ∈ (0, 1).

(26)

Definition 1.2. The effective domain or simply domain of the function F from E intoR∪ {+∞}is denoted dom F and defined by

dom F={x∈E : F(x) < +∞}. The effective domain of a convex function is convex [13].

Definition 1.3. The function F from E intoR∪ {+∞}is said to be proper if dom F6=∅.

Definition 1.4. If S is a subset of E, the indicator function δSof S is defined as:

δS(x) =

0 if x∈S;

+∞ if x∈E\A.

Clearly S is a convex subset of E if and only if δSis a convex function.

Definition 1.5. The epigraph of a function F from E intoR∪ {+∞}is the set: epi F={(x, α) ∈E×R | F(x) ≤α}.

It is the set of points of E×R which lie above the graph of F. Note that the

projection of epi F on E is dom F. Therefore if epi F is a convex set in E×R then

dom F is a convex set in E, as the image of the convex set epi F under a linear transformation [14]. As mentioned in [13], the epigraph is a useful concept in the study of convex functions because of the following result:

Proposition 1.1. A function F from E intoR∪ {+∞} is convex if and only if its epigraph is convex.

The following proposition gives some trivial results when you do some ma-nipulations on convex functions:

Proposition 1.2(Preservation of convexity). (1) If(Fi)i∈I, Fi : E−→R∪ {+∞}

is any family of convex functions and(λi)are positive real numbers then∑i∈IλiFi

is convex.

(2) If(Fi)i∈I, Fi : E−→R∪ {+∞}is any family of convex functions, their pointwise

supremum F=sup_i∈IFiis convex.

Definition 1.6. Let F be a function from E intoR∪ {+∞}. We say that F is lower semicontinuous (l.s.c.) at x∈ E if for every ε> 0 there exists a neighborhoodO

of x such that f(y) ≥ f(x) −εfor all y∈ O. Equivalently, this can be expressed

as

F(x) ≤lim inf

y→x F(y),

The function f is called lower semicontinuous if it is lower semicontinuous at every point of its domain.

(27)

1.1 Convex analysis 15 Example 1.1 (1) Let E=R and F(x) = ₋ 1 if x≥0; +1 if x<0. Then F is lower semicontinuous onR.

(2) The indicator function δS of a subset S⊂E is lower semicontinuous if and

only if S is closed.

Proposition 1.3(Closedness and lower semicontinuity). Let F be a function F : Rn _−→_R_{∪ {+}_∞_}_{. Then the following statements are equivalent:}

(a) The function F is lower semicontinuous on E;

(b) The level setS VF(α) ={x∈E : F(x) ≤α}are closed for every α∈R;

(c) The epigraph epi F of F is a closed set in E×R.

Definition 1.7. The infimal convolution of two functions F0 and F1 from E into

R∪ {+∞}is the function denoted(F0⊕F1)defined from E intoR∪ {−∞,+∞}

by

(F0⊕F1) (x) =_x=xinf

0+x1

{F0(x0) +F1(x1)}. (1.2)

Remark 1.1. It is clear that the infimal convolution can be defined directly in terms of addition of epigraphs:

(F0⊕F1) (x) =inf{µ∈R : (x, µ) ∈ (epi F0+epi F1)}. (1.3)

Definition 1.8. We say that the infimal convolution is exact if the infimum in (1.2) is attained when(F0⊕F1) (x)is finite.

Remark 1.2. Notice that (1.2) can be written as

(F0⊕F1) (x) =inf_x

1

{F0(x−x1) +F1(x1)}. (1.4)

If(F0⊕F1)is exact and(F0⊕F1) (x)is finite, we write

(F0⊕F1) (x) =F0(x−x˜1) +F1(x˜1),

implicitly defining ˜x1as a minimizer of (1.4). The decomposition x= (x−x˜1) +

˜

x1is hence referred to as optimal decomposition for(F0⊕F1) (x).

Definition 1.9. Let F be the function from E into R∪ {+∞}. The conjugate function of F is the function denoted F∗ and defined from E∗intoR∪ {+∞}by

F∗(y) =sup

x∈E

{hy, xi −F(x)}, ∀y∈E∗.

The conjugate is always convex as a pointwise supremum of the family of continuous affine functionsh· , xi −F(x).

(28)

Example 1.2

The conjugate of the indicator function is the following function

δ_S∗(y) =sup

x∈S

hy, xi.

The function y7−→sup_x∈Shy, xiis the so-called support function of S. The sup-port function is clearly continuous when S is bounded.

Symmetrically, for any function G from E∗ intoR∪ {+∞}, its conjugate G∗ is the function from E intoR∪ {+∞}defined by

G∗(x) =sup

y∈E∗

{hy, xi −G(y)}, ∀x∈E. (1.5)

Definition 1.10. We say that y is dual to x with respect to F if F(x) +F∗(y) =hy, xi,

and we say that x is dual to y with respect to F∗if F∗∗(x) +F∗(y) =hy, xi.

Below we give a definition of supporting points and supporting functionals Definition 1.11. Let E be a Banach space and S a closed convex subset of E and assume that S6=E and S6=∅.

(1) A point x0 ∈ S is said to be a supporting point if there exists a bounded

linear functional f ∈E∗ such that sup

x∈S

f(x) = f(x0).

(2) A given functional f ∈ E∗, which assumed to be different from zero func-tional is said to be a supporting funcfunc-tional if there exists x0∈S such that

sup

x∈S

f(x) = f(x0).

The terminology "supporting" in the definition above, comes from the geo-metric fact that the hyperplane H determined by f ,

H={x∈ E : f(x) = f(x0)},

touches S at x0 and leaves S in one of the half spaces determined by H. The

following theorem which can be found in [15, 16] gives a characterization of sets of supporting points and supporting functionals for a given closed and convex subset S of E.

(29)

1.1 Convex analysis 17

Theorem 1.1. (1) Let S be a closed convex subset of a Banach space E. Then the set of supporting points of S is dense in bd(S)(the boundary of S).

(2) Let S be a closed bounded convex subset of a Banach space E. Then the set of continuous linear functionals which support S is dense in X∗.

Let S be a nonempty subset of E and x∈S. We will define the following sets which are often associated with S.

Definition 1.12. (a) S is a cone ifR+S⊂S, whereR+is the set of nonnegative

reals. In particular 0∈S when S is a cone.

(b) The polar set of a subset S of E is a set S◦ in E∗defined as

S◦= ( y∈E∗: sup x∈E |hy, xi| ≤1 ) .

(c) The dual cone S+of a subset S of E is the set

S+={y∈E∗ : ∀x ∈S : hy, xi ≥0}.

(d) The orthogonal space S⊥of subset S of E is defined as follows S⊥={y∈E∗ : ∀x ∈S : hy, xi =0}.

(e) The tangent cone of a convex set S at a point x∈S is given by TS(x) =cl{λ(z−x): z∈S and λ≥0}.

(f) The normal cone of a convex set S at a point x∈S is given by NS(x) ={y∈E∗: ∀z∈S : hy, z−xi ≤0}.

Remark 1.3. It is observed that y∈NS(x) \{0}if and only if

H={z∈E : hy, zi = hy, xi},

is a supporting hyperplane to S at the point x and leaves S in one of the half spaces determined by H.

The following two results show that the operations of addition and infimal convolution of convex functions are dual to each other [14]. However the fact that the conjugate of infimal convolution is the sum of the conjugates holds with-out any requirements on the convex functions, but the fact that the conjugate of the sum is the infimal convolution of the conjugates requires some qualification conditions. We will state this under sufficient conditions of H. Attouch and H. Brezis [17].

(30)

Theorem 1.2(Conjugate of infimal convolution). Let F0 and F1 be convex

func-tions from E intoR∪ {+∞}. Then

(F0⊕F1)∗=F0∗+F1∗.

The following theorem by H. Attouch and H. Brezis [17] will be restated in Chapter 3. It provides a sufficient condition for the conjugate of the sum of two convex, lower semicontinuous and proper functions to be equal to the exact infimal convolution of their conjugates.

Theorem 1.3(Conjugate of a sum). Assume ϕ, ψ : E −→R∪ {+∞}are convex, lower semicontinuous, and proper functions such that

[

λ≥0

λ(dom ϕ−dom ψ) is a closed vector space.

Then

(ϕ+ψ)∗ =ϕ∗⊕ψ∗on E∗,

and, moreover, the infimal convolution is exact.

Definition 1.13. A point y∈ E∗is said to be a subgradient of a convex function F at a point x if

F(z) ≥F(x) +hy, z−xi, ∀z∈E. (1.6) We refer to condition (1.6) as the subgradient inequality, and it has a simple geometrical meaning when x∈dom F: it says that the graph of the affine func-tion

g(z) =F(x) +hy, z−xi, ∀z∈ E

is a non-vertical supporting hyperplane to the convex set epi F at the point(x, F(x)). Definition 1.14. The set of all subgradients of F at x is called the subdifferential of F at x and is denoted by ∂F(x). When x /∈dom F, we define ∂F(x) =∅.

In general, even when x ∈ dom F, it is possible that ∂F(x) may be empty or it may consist of just one vector.

If ∂F(x)is not empty, then F is said to be subdifferentiable at x.

An important example (see [18]) of a subdifferential is the normal cone to a convex set S⊂E at a point x∈S which can be defined by

NS(x):=∂δS(x).

Indeed, we have that y∈∂δS(x)if and only if

(31)

1.1 Convex analysis 19

This is trivially satisfied for z /∈ S. The meaningful case is when z∈S. This, for y to be a subgradient of δS(·)at x it is necessary and sufficient that

hy, z−xi ≤0, for all z∈S. Hence

hy, di ≤0 for all d∈cone(S−x).

Therefore the subdifferential of δS(·)at x is represented as the normal cone

∂δS(x) = − (cone(S−x))+=NS(x).

The following two examples give functions that are not subdifferentiable at 0, although 0 belongs to their domains.

Example 1.3

The function F :R−→R∪ {+∞}; with dom F=R+= [0,+∞)defined by

F(x) =    1 if x=0; 0 if x>0; +∞ if x<0.

is not subdifferentiable at 0. The only supporting hyperplane to epi F at(0, F(0))

is vertical.

Example 1.4

The function F :R−→R∪ {+∞}; with dom F=R+= [0,+∞)defined by

F(x) =

−√x if x≥0;

+∞ if x<0.

is not subdifferentiable at 0. The only supporting hyperplane to epi F at(0, F(0))

is vertical.

In the context of convex functions, subdifferentiability constitutes a generaliza-tion of differentiability (see the book [13])

Definition 1.15 (Directional derivative). Let F be a function F : E −→ R∪ {+∞}. The directional derivative of F at x in the direction of y denoted F0(x; y)is defined by

F0(x; y) = lim

ε→0+

F(x+εy) −F(x) ε ,

(32)

If there exists x∗∈E∗such that

F0(x; y) =hy, x∗i, ∀y∈E,

we say that F is Gâteaux-differentiable at x. In this case x∗is called the G ˆateaux-differential of F at x and is simply denoted by F0(x). The following propo-sition, which can be found in the book [13] shows that the case of Gâteaux-differentiability is the same as that of the uniqueness of the subgradient. Proposition 1.4. Let F : E −→ R∪ {+∞}be a convex function. If F is Gâteaux-differentiable at x∈ E, it is subdiffentiable at x and ∂F(x) = {F0(x)}. Conversely, if at the point x∈ E, F is continuous and finite and has only one subgradient, then F is Gâteaux-differentiable at x and ∂F(x) ={F0(x)}.

Proposition 1.5. Any points y∈ E∗ and x∈ E satisfy the inequality, well known as Young-Fenchel inequality

F(x) +F∗(y) ≥hy, xi. Equality holds if and only if y∈∂F(x):

y∈∂F(x) ⇐⇒F(x) +F∗(y) =hy, xi.

Definition 1.16. The core of a set S written core S is the set of points x∈S such that for any direction d in E, x+td lies in S for all small real t.

Note that this set clearly contains the interior of S. The following result is very useful.

Proposition 1.6. Let the function F from E intoR∪ {+∞}be convex, lower semicon-tinuous and proper. Then its conjugate function F∗defined from E∗intoR∪ {+∞}is also convex, lower semicontinuous and proper. Moreover

F∗∗=F.

The following proposition shows that the operation of infimal convolution preserves convexity.

Proposition 1.7. Let the functions F0 and F1 be convex, lower semicontinuous and

proper functions from E intoR∪ {+∞}. Then the function F(x) = (F0⊕F1) (x) =_x=xinf

0+x1

{F0(x0) +F1(x1)}

(33)

1.2 Interpolation theory 21

1.2 Interpolation theory

1.2.1 Some important definitions and results

Let X0 and X1 be Banach spaces endowed with norms k·kX0 and k·kX1 re-spectively. The spaces X∗₀ and X∗₁ will denote the dual spaces to X0 and X1

respectively. They are equipped with the following dual norms

kyk_X∗ 0 =sup n hy, xi: x∈X0, kxkX0 ≤1 o and kyk_X∗ 1 =sup n hy, xi: x∈X1, kxkX1 ≤1 o

respectively. The expressionhy, xidenotes the value of the functional y in X∗_j on the element x∈Xj, j=0, 1.

Definition 1.17. A pair(X0, X1)of Banach spaces X0and X1is called a

compat-ible couple if there is some Hausdorff topological vector space, say X, in which each of X0and X1is linearly and continuously embedded.

Example 1.5

• L1, L∞ is a compatible couple because both L1_{and L}∞ _{are linearly and}

continuously embedded in the Hausdorff spaceX of almost everywhere finite measurable functions.

• Any pair(X0, X1)of Banach spaces for which X0 is linearly and

continu-ously embedded in X1(resp. X1is linearly and continuously embedded in

X0) is a compatible couple because we may choose for the Hausdorff space

X the space X1(resp. X0) itself.

More examples can be found in the books [2, 1, 3].

Definition 1.18. Let X0 and X1 be a compatible couple, with corresponding

Hausdorff spaceX. Let X0+X1denote the sum of X0and X1, that is, the set of

elements x∈ X that are representable in the form x=x0+x1for some x0in X0

and x1in X1:

X0+X1={x∈ X : x=x0+x1; X0∈X0, X1∈X1}.

For each x∈X0+X1, set

kxk_X 0+X1 =_x=xinf₀_+x₁ n kx0kX0+kx1kX1 o , (1.7)

where the infimum extends over all representations x = x0+x1of x with x0in

X0and x1in X1. For each element x in the intersection X0∩X1of X0and X1, set

kxk_X₀_∩X 1 =max n kxk_X₀,kxk_X 1 o . (1.8)

(34)

Remark 1.4. Note that X0+X1and X0∩X1and their norms do not depend on

the particular choice of Hausdorff spaceX associated with the couple(X0, X1).

It is known [3, 1] that

Theorem 1.4. If(X0, X1)is a compatible couple, then X0+X1and X0∩X1are Banach

spaces with respect to the norms (1.7) and (1.8), respectively.

Definition 1.19. The compatible Banach couple(X0, X1)is said to be regular if

X0∩X1is dense in Xj for j=0, 1.

Remark 1.5. For a regular couple(X0, X1), the dual spaces X∗0 and X1∗also form

a compatible couple X₀∗, X₁∗. In this situation, both X∗

0and X1∗embed

continu-ously in(X0∩X1)∗[3].

Definition 1.20. If(X0, X1)is a compatible couple, then a Banach space X is said

to be an intermediate space between X0 and X1 if X is continuously embedded

between X0∩X1and X0+X1:

X0∩X1,→X,→X0+X1.

The symbol,→denotes a linear and continuous embedding, and if we denote X,→Y,

it means that the inequality

kxk_Y≤ckxk_X, ∀x∈ X, with c≥1, holds.

Remark 1.6. It is clear that X0 and X1 are always intermediate spaces for the

couple(X0, X1):

X0∩X1,→ Xj,→X0+X1, j=0, 1, with c=1.

Theorem 1.5(Duality Theorem). Let(X0, X1)be a regular couple of Banach spaces.

Then(X0∩X1)∗ is isometrically isomorphic to X0∗+X∗1 and (X0+X1)∗ is

isometri-cally isomorphic to X∗₀∩X₁∗. In formulas (where equality means isometrically isomor-phic): (1) (X0∩X1)∗=X∗0+X∗1, (2) (X0+X1)∗=X∗0∩X∗1. More precisely kyk_X∗ 0+X1∗=kyk(X0∩X1)∗ = sup x ∈X0∩X1 x 6=0 hy, xi kxk_X 0∩X1 and kyk_X∗ 0∩X1∗=kyk(X0+X1)∗ = sup x∈X0+X1 x6=0 hy, xi kxk_X₀_+X 1

(35)

1.2.2 Interpolation functionals

Recall that from Theorem 1.4 the spaces X0+X1and X0∩X1are Banach spaces

with respect to the norms (1.7) and (1.8), respectively.

The K- and J- functionals

The Peetre K- and J- functionals are constructed from the expressions (1.7) and (1.8) by introducing a positive weighting factor t, as follows:

Definition 1.21. Let(X0, X1)be a compatible couple of Banach spaces.

(1) The K- functional is defined for each x∈X0+X1and t>0 by

K(t, x; X0, X1) = inf x=x0+x1 kx0kX0+tkx1kX1 , (1.9)

where the infimum extends over all representations of x = x0+x1 of x

with x0∈X0and x1∈ X1.

(2) The J- functional is defined for each x∈X0∩X1and t>0 by

J(t, x; X0, X1) =max kxk_X 0, tkxkX1 .

Remark 1.7. (1) The K- functional plays a central role in the theory of inter-polation spaces [1]. It is used for the approach so-called K- method to the theory of interpolation spaces.

(2) In view of (1.7), we can write

K(t, x; X0, X1) =kxkX0+tX1, x∈X0+X1, t>0, where tX1denotes space X1with the norm x7−→tkxkX₁.

(3) The functionals x7−→K(t, x; X0, X1),(t>0)define a family of equivalent

norms on X0+X1because

min{1, t} kxk_X

0+X1 ≤K(t, x; X0, X1) ≤max{1, t} kxkX0+X1. (4) For any arbitrary decomposition x = x0+x1 of x ∈ X0+X1, it follows

from (1.9), that

K(t, x; X0, X1) ≤kx0kX0+tkx1kX1. (1.10)

(5) Since every x ∈ X0 has a trivial decomposition x = x+0 as an element

from X0+X1, it follows immediately from (1.10), that

(36)

(6) Similarly every x∈X1has a trivial decomposition x=0+x as an element

from X0+X1, and it follows immediately from (1.10), that

K(t, x; X0, X1) ≤tkxkX1, x ∈X1, t>0. (1.12) (7) If x∈X0∩X1then it follows from (1.8), (1.11) and (1.12) that

K(t, x; X0, X1) ≤min{1, t} kxkX0∩X1, x∈X0∩X1, t>0. The following properties of the K- functional are also very useful Proposition 1.8. For each x∈ X0+X1,

(1) The K- functional K(t, x; X0, X1)is a nonnegative concave function of t>0 and

1 tK(t, x; X0, X1) =K 1 t, x; X1, X0 . (2) K(t, x; X0, X1)is non-decreasing function of t on(0,+∞), (3) 1_tK(t, x; X0, X1)is non-increasing function of t on(0,+∞), (4) K(t, x+y; X0, X1) ≤K(t, x; X0, X1) +K(t, y; X0, X1).

As for the J- functional, it is also the basis of another approach to interpola-tion spaces so-called J- method. The J- funcinterpola-tional satisfies the following estimates Proposition 1.9. For all s, t>0 and for all x∈X0∩X1we have

min{1, t} kxk_X 0∩X1 ≤ J(t, x; X0, X1) ≤max{1, t} kxkX0∩X1, and K(t, x; X0, X1) ≤min n kxk_X 0, tkxkX1 o ≤min 1,t s J(s, x; X0, X1). E- functional

Let(X0, X1)be a compatible couple of Banach spaces. The E- functional is defined

for each x∈X0+X1and t>0 by

E(t, x; X0, X1) =inf n kx0kX0 : x0∈X0, x1∈X1, kx1kX1 ≤t, with x=x0+x1 o . (1.13) The K- functional and E- functional are closely related by the following

K(t, x; X0, X1) =inf s>0{E(s, x; X0, X1) +st, ∀x∈X0+X1, ∀t>0}, and E(t, x; X0, X1) =sup s>0 {K(s, x; X0, X1) −st, ∀x∈X0+X1, ∀t>0}.

(37)

Proposition 1.10(Properties of the E-functional). 1. The E- functional E(t, x; X0, X1)

is a nonnegative decreasing convex function of t>0, 2. E(t, x+y; X0, X1) ≤E ₂t, x; X0, X1+E ₂t, y; X0, X1

For some 1≤ p0<∞, we can write the E- functional in more general way as

follows Ep0(t, x; X0, X1) = inf kx1k_X1≤t 1 p0 kx0kp_X0₀. (1.14) L- functional

Let(X0, X1)be a compatible couple of Banach spaces. The L- functional is defined

(see for example [19]) for each x∈ X0+X1and t>0 by

L(t, x; X0, X1) = inf x=x0+x1 kx0kXp00 +tkx1k p1 X₁ , (1.15)

for 1≤p0, p1<∞. It is sometimes useful to define L- functional in the following

way Lp0,p1(t, x; X0, X1) =_x=xinf 0+x1 1 p0 kx0kp_X0₀+ t p1 kx1kp_X1₁ . (1.16)

1.2.3 Interpolation functionals as infimal convolutions

In this section we present the K-, E- and L- functionals in the context of infimal convolution. We will denote by R+ = [0,+∞) the nonnegative real numbers

and byR++= (0,+∞)the positive real numbers.

Let us define the functions Fj, j= 0, 1 as extensions of the normsk·kXj by+∞ to the whole of X0+X1, i.e.,

F0(x) = kxk_X 0 if x∈X0; +∞ if x∈X0+X1\X0. (1.17) and F1(x) = kxk_X 1 if x∈X1; +∞ if x∈X0+X1\X1. (1.18) The K- functional

Let the functions F0and F1be defined as in (1.17) and (1.18) respectively. Then

the K- functional can be regarded as a function from(X0+X1)intoR+∪ {+∞}

defined in terms of infimal convolution by

(38)

The E- functional

Let the functions F0and F1be defined as in (1.17) and (1.18) respectively. Then

the E- functional (1.13) can also be defined from(X0+X1)intoR+∪ {+∞}in

terms of infimal convolution E(t, · ; X0, X1) =

F0⊕δtB_X1

(·), for t∈R++,

where δtB_X1 is the indicator function of unit ball in X1scaled by t:

tBX1 = n

x∈X0+X1: kxkX1 ≤t o

. In the case of the Ep0 in (1.14), we have that

Ep0(t, · ; X0, X1) = 1 p0 Fp0 0 ⊕δtB_X1 (·), for t∈R++. The L- functional

Let the functions F0 and F1 be defined as in (1.17) and (1.18) respectively, and

let 1≤ p0, p1< +∞. Then the L- functional (1.15) defined from(X0+X1)into

R+∪ {+∞}can be written in terms of infimal convolution as follows

L(t, · ; X0, X1) = Fp0 0 ⊕F p1 1 (·), for t∈R++.

In the case of the Lp0,p1 in (1.16), we have that

Lp0,p1(t, · ; X0, X1) = 1 p0 Fp0 0 ⊕ t p1 Fp1 1 (·), for t∈R++.

Special case: Lp0,1- functional

Let us set 1< p0 <∞ and p1 =1 in (1.16). We obtain the following special

L-functional Lp0,1(t, x; X0, X1) =_x=xinf 0+x1 1 p0 kx0kpX00+tkx1kX1 .

In terms of infimal convolution, it is written as follows Lp0,1(t, · ; X0, X1) = 1 p0 Fp0 0 ⊕tF1 (·), for t∈R++.

Special case: L1,p1- functional

Let us set p0 =1 and 1 < p1< ∞ in (1.16). We obtain the following special

L-functional L1,p1(t, x; X0, X1) =_x=xinf 0+x1 kx0kX0+ t p1 kx1kpX11 .

(39)

In terms of infimal convolution, it is written as follows

L1,p1(t, · ; X0, X1) = F0⊕ t p1 Fp1 1 (·), for t∈R++.

(40)

(41)

2

Geometry of Optimal

Decomposition for the Couple

`

2 , X

on

R

n

In this chapter, we will investigate the geometry of optimal decomposition for L– functional L2,1(t, x;`2, X) = inf x=x0+x1 1 2kx0k 2 `2+tkx1kX , (2.1)

where space`2is defined by the standard Euclidean normk·k₂and where X is a Banach space on Rn and t is a given positive parameter. Our proof is based on some geometrical considerations and Yves Meyer’s duality approach (see the book [12]) which was used for the couple L2, BV in connection with the famous in image processing Rudin-Osher-Fatemi (ROF) denoising model. Our goal is also to investigate possibility to extend Meyer’s approach to more general couples than L2, BV.

2.1 Introduction

The denoising problem is a problem to extract the ideal “clean” image (sig-nal) from its noisy version [5]. The problem of denoising is fundamental in image (signal) processing. From a mathematical point of view, it consists of reconstructing approximately an initial, unknown image (signal) f∗ from the

observed, noised image (signal)

fob= f∗+η,

where the term η corresponds to the noise. In a very successful in image pro-cessing Rudin-Osher-Fatemi (ROF) denoising model [4, 8, 5] it is supposed that

(42)

the initial image f∗and observed image fobare functions defined on a

rectangu-lar plane domain D. Moreover it is also supposed that the noise η is a function from L2 = L2(D)and the initial image f∗ belongs to the well-known in

analy-sis space of functions of bounded variation BV, in which the seminorm for the differentiable function f is equal to

kfk_BV = Z Z D ∂ f ∂x(x, y) + ∂ f ∂y(x, y) dxdy.

For the non-differentiable functions, norm in BV is more complicated and we will not discuss it here. The Rudin-Osher-Fatemi model suggests to use as ap-proximation to the initial image f∗a function ftsuch that

L2,1 t, fob; L2, BV = 1 2kfob− ftk 2 L2+tkftkBV,

for some t>0. Such a function ftis called exact minimizer for the functional

L2,1 t, fob; L2, BV = inf f ∈BV 1 2kfob− fk 2 L2+tkfk_BV .

Unfortunately even for the known parameter t > 0, the construction of the minimizer ft is not simple. One of the approaches to this problem is based

on a very interesting fact that the minimizer ft is also a minimizer for the dual

problem (see, for example [8, 12]).

The main idea of this chapter is to prove a result which gives a geometri-cal characterization of such exact minimizer in the case of a finite dimensional couple (`2, X)on Rn, where X is defined by an arbitrary norm. Our proof in this situation is rather simple and provides a clear explanation of the general phenomenon.

This chapter is organized as follows: In Section 2.2 we obtain an important result characterizing the exact minimizer under the duality. The proof will be detailed in Section 2.3. Finally, Section will outline some remarks and future considerations.

2.2 Main result

Let X be a Banach space on Rn equipped with norm k·k_X. Let x be a fixed element fromRnand consider the following well known in interpolation theory L– functional (see the book [1])

L2,1(t, x;`2, X) = inf x0∈`2 1 2kx0k 2 `2+tkx−x0kX ,

where t is a positive parameter. Let x0,optbe an exact minimizer for L2,1(t, x;`2, X)

i.e., L2,1(t, x;`2, X) = 1 2 x_0,opt 2 `2+t x−x_0,opt X.

(43)

2.2 Main result 31

Let us define X∗to be the dual space of X equipped with the dual normk·k_X∗

defined by

kyk_X∗ =sup{hy, xi: x∈X; kxk_X≤1},

where

hy, xi =y · x and kxk_X = (hx, xi)1/2.

Note that in the case when k·k_X is a seminorm, the dual normk·k_X∗ could be

equal to+∞ for some elements. Let us consider a closed ball of radius t in the

normk·k_X∗of X∗centered at 0 and denote it by tBX∗:

tBX∗ ={y∈Rn:kyk_X∗≤t}.

Theorem 2.1 (Stability of a minimizer under duality). Let x0,optbe an exact

min-imizer for L2,1(t, x;`2, X). Then x0,opt is the nearest element of tBX∗ to the point x in

the metric of`2: inf kx0kX∗≤t kx−x0k`2 = x−x0,opt _`2.

The situation is illustrated by Figure 2.1 below, where x1,opt=x−x0,opt.

(44)

Remark 2.1. Theorem 2.1 can be formulated in terms of the E-functional on the couple `2, X∗. More precisely, let x0,optbe an exact minimizer for L2,1(t, x;`2, X).

Then Et, x;`2, X∗= inf kx0kX∗≤t kx−x0k`2 = x−x0,opt _`₂. (2.2) The Theorem 2.1 provides ways to compute the exact minimizers x0,opt for

L2,1(t, x;`2, X) using E t, x;`2, X∗. For illustration, let us consider a simple

example. Example 2.1

Consider the couple `2_,_`1_{and consider the problem of calculating the exact}

minimizer for the functional

L2,1(t, x;`2,`1) = inf x0∈`2 1 2kx0k 2 `2+tkx−x0k`1 .

In this case, X= `1and the ball tBX∗is defined by

tBX∗ ={y∈Rn:kyk_`_∞ ≤t}.

From Theorem 2.1 ( see also Remark 2.1 ), the solution x0,opt to this problem is

an exact minimizer for the E-functional E(t, x;`2,`∞) = inf

{x0: kx0k`∞≤t}

kx−x0k`2.

It can be calculated analytically as follows

x0,opt_i =    t, if xi>t xi, if −t≤xi≤t −t, if xi< −t , for i=1, 2, . . . , n.

Remark 2.2. The paper by P. Nilsson and J. Peetre [20] provides ways to find the exact minimizer for the couple(L1, L2)of the functional

K(t, x; L1, L2) = inf

x0∈L1

(kx0kL1+tkx−x0kL2).

2.3 Proof of the main result

In this section we will proceed to prove Theorem 2.1. The proof will be based on three lemmas that we will prove first. Assume that x0,optis an exact minimizer

for L(t, x;`2, X). This means that L2,1(t, x;`2, X) = 1 2 x_0,opt 2 `2+t x−x_0,opt X. (2.3)

(45)

2.3 Proof of the main result 33

Moreover, we will assume that both x0,opt 6= 0 and x−x0,opt 6= 0. The more

general case will be investigated in Chapter 4. Let us put the following notation

α= 1 2 x_0,opt 2 `2 and β=t x−x_0,opt X. (2.4)

Consider the closed balls B1= n u∈Rn:kuk_`2 ≤ √ 2αo and B2= u∈Rn:kx−uk_X ≤ β t . (2.5)

We note that B1and B2are nonempty convex subsets ofRn. Let T1be the tangent

hyperplane to B1passing through the point x0,opt as illustrated in Figure 2.2.

Figure 2.2: Illustration of lemma 2.1.

It is clear that since T1is orthogonal to the vector x0,opt. Therefore we have

u, x0,opt=x0,opt, x0,opt ∀u∈ T1.

The hyperplane T1dividesRn into two closed halfspaces. We will denote these

hyperplanes by H1and H2which contain the origin 0 and x respectively.

Lemma 2.1. Let H1and H2be defined as above and let B1and B2be defined as in (2.5).

Then B1is contained in H1and B2is contained in H2.

Proof. As T1is a tangent hyperplane to B1therefore B1⊂H1is obviously

satis-fied. So we need only to prove that B2⊂ H2as illustrated in Figure 2.2. Suppose

(46)

from definition of B2 and (2.4) follows that x0,opt ∈ bd(B2), where bd(B2) is

the boundary of B2. Consider the segmentv, x0,opt. From convexity of B2, this

segment must be contained in B2 and therefore since B1 is a ball of `2 and T1

is tangent to B1 through x0,opt, then we can find on this segment a point which

belongs to the interior of B1. We will denote this element by w and this situation

is illustrated in Figure 2.3.

Figure 2.3: The impossible situation where B2contains an element of H1\H2.

So, in this case we have:

kwk_`2 <

√

2α and tkx−wk_X ≤β. (2.6)

From (2.6) and (2.4), it follows that 1 2kwk 2 `2+tkx−wk_X< 1 2 x_0,opt 2 `2+t x−x_0,opt X.

Which is a contradiction of (2.3), because x0,optis the exact minimizer for L2,1(t, x;`2, X).

Therefore T1separates B1and B2.

Lemma 2.2. The exact minimizer x0,optsatisfies

x0,opt, x−x0,opt=t x−x_0,opt X. (2.7)

Proof. Put x1,opt∈X such that

(47)

Then the expression (2.4) can be rewritten as

L2,1(t, x;`2, X) = 1 2 x−x1,opt 2 `2+t x1,opt _X, Let us consider arbitrary ε∈ (−1, 1)and put

xε 1=x1,opt−εx1,opt. It is clear that L2,1(t, x;`2, X) ≤ 1 2kx−x ε 1k2`2+tkx₁εk_X. This is equivalent to 1 2kx−x ε 1k2`2+tkxε₁k_X≥ 1 2 x−x1,opt 2 `2+t x1,opt _X. Now putting the expression for xε

1, we get 1 2 x−x1,opt+εx1,opt 2 `2+t x1,opt−εx1,opt _X≥ 1 2 x−x1,opt 2 `2+t x1,opt _X. Expanding we get 1 2 x−x1,opt 2 `2 +εx−x1,opt, x1,opt +ε 2 2 x1,opt 2 `2 +t(1−ε) x1,opt _X ≥ 1 2 x−x_1,opt 2 `2+t x_1,opt X. Equivalently, ε x−x1,opt, x1,opt−t x_1,opt X + ε 2 2 x_1,opt 2 `2 ≥0. (2.9) Let us consider two cases:

• Case 1: ε≥0.

If ε≥0, then dividing the two sides of (2.9) by ε yields x−x1,opt, x1,opt−t x_1,opt X+ ε 2 x_1,opt 2 `2 ≥0. Taking limit when ε tends to zero from the right, we obtain

x−x1,opt, x1,opt−t x_1,opt X ≥0. (2.10) • Case 2: ε≤0.

If ε≤0, then a similar reasoning as in case 1 yields x−x1,opt, x1,opt −t x1,opt X ≤0. (2.11)

(48)

Combining (2.10) and (2.11), we get x−x1,opt, x1,opt −t x1,opt _X=0. Therefore in view of (2.8), we obtain that

x0,opt, x−x0,opt =t x−x0,opt _X, which completes the proof of (2.7).

Remark 2.3. By definition of β (see (2.4)) we see that the equality in Lemma 2.2 can be rewritten as

x0,opt, x−x0,opt=β. (2.12)

Lemma 2.3. The exact minimizer x0,optsatisfies

x_0,opt

X∗=t.

Proof. In view of (2.12), it is enough to prove that

β t x0,opt _X∗=x0,opt, x−x0,opt .

By definition of the normk·k_X∗, we have

x_0,opt X∗ = sup kzk_X≤1 x0,opt, z ,

which, by homogeneity property of the norm, is equivalent to

β t x0,opt _X∗ = sup kzk_X≤β t x0,opt, z . (2.13) Since x−x_0,opt X = β

t (see (2.4)), we have in particular that

β t x_0,opt X∗≥x0,opt, x−x0,opt . (2.14)

We need to show that

β t x0,opt _X∗≤x0,opt, x−x0,opt .

Assume, by contradiction that

β t x_0,opt X∗>x0,opt, x−x0,opt .

Using (2.13), this can be rewritten as: sup

kzkX≤βt

(49)

Denoting by Px0,optv the vector projection of v onto the vector x0,opt and by putting z=x−u, the expression (2.15) becomes

sup u∈B2 Px0,opt(x−u) _`2 > Px0,opt x−x0,opt _`2. (2.16) From Lemma 2.1 we know that T1separates B1and B2and x−x0,opt _X = β_t. In Figure 2.4, T₁00is parallel to T1.

Figure 2.4: Orthogonal Projections. The situation in (2.16) is impossible because we must have

sup u∈B2 Px0,opt(x−u) _`2 ≤ Px0,opt −x+x0,opt _`2. Therefore we must have

sup u∈B2 Px0,opt(x−u) _`2 ≤ Px0,opt x−x0,opt _`2, or equivalently sup kzkX≤βt

x0,opt, z≤x0,opt, x−x0,opt . (2.17)

From (2.13), the expression (2.17) can be rewritten as

β t x_0,opt X∗ ≤x0,opt, x−x0,opt . (2.18)

(50)

Combining (2.14) and (2.18), we get β t x0,opt _X∗=x0,opt, x−x0,opt .

Using the fact thatx0,opt, x−x0,opt = β(see (2.12) in Remark 2.3), we obtain

that

x0,opt

_X∗=t.

This completes the proof of Lemma 2.3. Now we are ready to prove Theorem 2.1.

Proof of Theorem 2.1. To prove Theorem 2.1, we consider the closed balls B3={u∈Rn:kukX∗ ≤t} and B4= n u∈Rn :kx−uk_`2 ≤ x−x0,opt _`₂ o . In Figure 2.5 we illustrate these balls. Moreover we consider a hyperplane T2

tangent to B4through the point x0,opt(and of course orthogonal to the vector x−

x0,opt because B4is a ball of`2), then we have two closed halfspaces H3and H4

which contain the origin 0 and the point x respectively. From this consideration, the ball B4is obviously contained in the halfspace H4.

Figure 2.5: Closed balls B3and B4ink·kX∗ andk·k_`2 respectively. It will be enough to prove that

(51)

as illustrated in Figure 2.5. Suppose by contradiction that there exists another element v∈ H4\H3such that

kx−vk_`2 ≤

x−x0,opt

_`₂ and kvk_X∗ ≤t, (2.19)

This situation is illustrated in Figure 2.6, where T₂00is parallel to T2.

Figure 2.6: The situation that B3contains an element v∈ H4\H3is impossible.

Sincekx−vk_`2 ≤

x−x_0,opt

`2, i.e., v∈ B4, by considering orthogonal

projec-tions of the vectors x0,optand v on the direction of x−x0,opt, we obtain that

Px−x0,opt x0,opt _`2 < Px−x0,optv _`2, which is equivalent to

x−x0,opt, x0,opt<x−x0,opt, v .

It follows from Lemma 2.2 that t

x−x0,opt _X <x−x0,opt, v .

Using Cauchy-Schwartz inequality, we get t x−x0,opt _X <x−x_0,opt, v ≤ x−x0,opt _Xkvk_X∗.

Which implies that

JaphetNiyobuhungiro OptimalDecompositioninRealInterpolationandDualityinConvexAnalysis

Linköping Studies in Science and Technology.

Thesis No. 1604

Optimal Decomposition in Real

Interpolation and Duality in Convex

Analysis

Japhet Niyobuhungiro

Department of Mathematics

Division of Mathematics and Applied Mathematics

Linköping University, SE–581 83 Linköping, Sweden

Abstract

Populärvetenskaplig sammanfattning

Acknowledgments

Contents

0

Introduction

0.1

Background

0.2

Summary of main results

1

Preliminaries

1.1

Convex analysis

1.2

Interpolation theory

1.2.1

Some important definitions and results

1.2.2

Interpolation functionals

1.2.3

Interpolation functionals as infimal convolutions

2

Geometry of Optimal

Decomposition for the Couple

`

2

, X

on

R

n

2.1

Introduction

2.2

Main result

2.3

Proof of the main result