iköigS dieiS ie eadTe h

DissertationNo. 1294

Aspe ts of a Constraint Optimisation

Problem

by

Johan Thapper

DepartmentofComputerandInformationS ien e

Linköpingsuniversitet

In this thesis we study a onstraint optimisationproblem alled the

max-imum solution problem, hen eforthreferred to asMax Sol. It is dened

asthe problem of optimising alinear obje tive fun tion over a onstraint

satisfa tionproblem(Csp)instan eonanitedomain. Ea hvariableinthe

instan e isgiven anon-negativerationalweight,and ea h domain element

isalsoassignedanumeri alvalue,forexampletakenfromthenatural

num-bers.Fromthispointofview,theproblemisseentobeanaturalextension

ofintegerlinearprogrammingoveraboundeddomain. Westudy boththe

time omplexity of approximating Max Sol, and the time omplexity of

obtainingan optimal solution. In the latter ase, wealso onstru t some

exponential-timealgorithms.

The algebrai methodisapowerfultool forstudying Csp-related

prob-lems. It was introdu ed for thede ision versionof Csp, and hasbeen

ex-tended to a number of other problems, in luding Max Sol. With this

te hniqueweestablishapproximability lassi ationsfor ertainfamiliesof

onstraint languages, based on algebrai hara terisations. We also show

howthe on eptofa oreforrelationalstru tures anbeextended inorder

to determine when onstant unary relations an beadded to a onstraint

language,without hangingthe omputational omplexityofndingan

opti-malsolutiontoMaxSol. Usingthisresultweshowthat,inaspe i sense,

whenstudyingthe omputational omplexityofMax Sol,weonlyneedto

onsider onstraintlanguageswithall onstantunaryrelationsin luded.

Someoptimisationproblemsareknowntobeapproximablewithinsome

onstantratio,butarenotbelievedto beapproximablewithinan

arbitrar-ilysmall onstantratio. Forsu hproblems,itisofinteresttondthebest

ratiowithinwhi htheproblem anbeapproximated,orat leastgivesome

boundsonthis onstant. Westudythisaspe tofthe(weighted)Max Csp

problem for graphs. In this optimisation problem the numberof satised

onstraintsissupposedtobemaximised. Weintrodu eamethodfor

study-ingapproximationratioswhi h isbased onanew parameteron thespa e

of allgraphs. Informally, wethink of this parameter asan approximation

distan e; knowingthedistan e betweentwographs,we anbound the

IwanttothankmysupervisorPeterJonssonfortheopportunitytodomy

thesisworkinhisgroup,andfor ountlessinspiringdis ussions. Mystudies

were alsoin nosmall partinuen edbySvante Linusson, whoadvisedmy

li entiatethesisandgavemea ertain ombinatorialviewonthings.

Fortheir ontributionofideas,en ouragement,andperspe tive,Iwould

furtherliketo thankthefollowingpeople:

•

my o-authors: Ola,Peter,Gustav,Tommy,andRobert;

•

Ola, Rustem, and Tommy, for reading and ommenting on parts of themanus ript;

•

Tommyformanyinterestingdis ussionsanden ouragement;

•

everyoneatTCSLABfortheopenand reativeworkatmosphere.

Mu h gratitude also goes to my friends and family, and to Aurélie for

herendless loveandsupport, andfor herpatien e in me during these last

Partsofthisthesishavepreviouslybeenpublishedinthefollowingrefereed

papers:

•

Ola Angelsmarkand Johan Thapper. Algorithms for themaximum hammingdistan eproblem. InBoiFaltings,FrançoisFages,Fran es a

Rossi, and Adrian Pet u, editors, Constraint Satisfa tion and

Con-straint Logi Programming: ERCIM/CoLogNet International

Work-shop (CSCLP-2004), Lausanne, Switzerland, June 2325, 2004,

Re-vised Sele ted and Invited Papers, volume 3419 of Le ture Notes in

Computer S ien e,pages128141. SpringerVerlag,Mar h 2005.[7℄

•

Ola Angelsmark and Johan Thapper. A mi rostru ture based ap-proa h to onstraint satisfa tion optimisation problems. In Ingrid

Russell and Zdravko Markov, editors, Re ent Advan es in Arti ial

Intelligien e. Pro eedings of the Eighteenth International Florida

Ar-ti ialIntelligen eResear hSo ietyConferen e(FLAIRS-2005),May

1517, 2005, Clearwater Bea h, Florida, USA,pages155160. AAAI

Press,2005.[8℄

•

Peter Jonsson, GustavNordh, and Johan Thapper. The maximum solutionproblem on graphs. InLud¥k Ku£era andAntonínKu£era,

editors, Mathemati al Foundations of Computer S ien e, 32nd

Inter-national Symposium (MFCS-2007), ƒeský Krumlov, Cze h Republi ,

August 2631, 2007, Pro eedings, volume 4708 of Le ture Notes in

Computer S ien e,pages228239. SpringerVerlag,2007.[85℄

•

TommyFärnqvist, Peter Jonsson, andJohan Thapper. Approxima-bility distan e in the spa e of

H

- olourability problems. In Anna Frid,AndreyMorozov,Audrey Rybal henko,andKlausW. Wagner,

editors, Computer S ien e  Theory and Appli ations, 4th

Interna-tional Computer S ien e Symposiumin Russia (CSR-2009),

Novosi-birsk,Russia,August1823,2009,Pro eedings,volume5675ofLe ture

NotesinComputerS ien e,pages92104.SpringerVerlag,2009.[49℄

•

PeterJonssonandJohanThapper. Approximabilityofthemaximum solutionproblem for ertainfamiliesof algebras. In Anna Frid,

An-puterS ien eSymposiuminRussia(CSR-2009), Novosibirsk,Russia,

August 1823, 2009, Pro eedings, volume 5675 of Le ture Notes in

ComputerS ien e,pages215226. SpringerVerlag,2009.[86℄

•

RobertEngström,TommyFärnqvist,PeterJonsson,andJohan Thap-per. Properties of anapproximability-relatedparameter on ir ular

ompletegraphs. InThomasM.Liebling,JaymeL.Szwar ter,Carlos

E.Ferreira,andFábioProtti,editors,LAGOS'09VLatin-Ameri an

Algorithms, Graphs, and OptimizationSymposium, Gramado, Brazil,

November37,2009,volume35ofEle troni NotesinDis rete

Math-emati s,pages115120,2009.[45℄

Additionally,theresultsfrom thefollowingpaperarealso in luded:

•

Peter JonssonandJohan Thapper. Approximatingintegerprograms with positive right-hand sides. Information Pro essing Letters,

I Introdu tion 1

1 Introdu tion 3

1.1 Constraintsatisfa tion . . . 10

1.2 Optimisation . . . 14

1.2.1 Example: Maximum Independent Set . . . 14

1.2.2 Example: Max ut . . . 15

1.3 Approximability . . . 16

1.4 Themaximumsolutionproblem. . . 17

1.5 Methodsandresults . . . 18

1.6 Outlineofthethesis . . . 22

2 Preliminaries 25 2.1 Basi denitions andnotation . . . 25

2.1.1 Graphs . . . 25

2.2 Constraintsatisfa tion . . . 28

2.2.1 Constraintlanguagesandsatisfyingassignments . . . 28

2.2.2 Relationalstru turesandhomomorphisms. . . 29

2.3 Themaximumsolutionproblem. . . 29

2.4 Approximabilityandredu tions . . . 30

2.4.1 Some ompleteproblems. . . 33

2.4.2 Theuniquegames onje ture . . . 33

2.5 Thealgebrai method . . . 34

2.6 Post'slatti e . . . 36

II Approximability 39 3 Ane Algebrasand Approximability 41 3.1 Universalalgebra . . . 44

3.1.1 Endomorphismrings . . . 45

3.2 Anealgebras . . . 45

3.3 Linearequationsoverabeliangroups . . . 48

3.3.1 The ase

M = 0 ×

R

A

. . . 51

3.4 Supplementarytoolbox. . . 59

3.4.1 Two-elementdomain . . . 61

3.5 Stri tlysimplesurje tivealgebras. . . 62

3.6 Symmetri algebras. . . 65

3.7 Dis ussion . . . 68

4 MinimisationofInteger LinearPrograms 71 4.1 Unboundeddomain. . . 73

4.1.1 Lowerbounds . . . 75

4.2 Boundeddomain . . . 76

4.3 Dis ussionandfuture work . . . 78

5 Approximation Distan e 79 5.1 Approximationdistan e forMax

H

-Col . . . 83

5.2 Propertiesoftheparameter

s

. . . 84

5.2.1 Exploitingsymmetries . . . 85

5.2.2 Asandwi hinglemma . . . 88

5.2.3 Thespa e(

G

≡

, d

). . . 90

5.2.4 Alinearprogramformulation . . . 91

5.3 Cir ular omplete graphs . . . 94

5.3.1 Mapsto

K

2

. . . 97

5.3.2 Constantregions . . . 98

5.3.3 Mapsto odd y les . . . 99

5.4 Approximationbounds forMax

H

-Col . . . 100

5.4.1 Sparsegraphs . . . 102

5.4.2 Denseandrandomgraphs . . . 105

5.5 Fra tional overingby

H

- uts . . . 108

5.5.1 An upperbound . . . 111

5.5.2 Conrmationofas ale . . . 112

5.6 Approximationdistan eforMaxSol . . . 114

5.7 Dis ussionandopenproblems . . . 118

III Computational Complexity 123 6 Constant Produ tion 125 6.1 Preliminaries . . . 127

6.1.1 Weightedpp-denability. . . 128

6.1.2 Themeasure

m(f )

asabilinearform . . . 131

6.1.3 Conne tedsubstru tures. . . 134

6.2 Constantprodu tion . . . 138

6.3 Appli ation: graphs . . . 142

6.3.1 Pathsand y les . . . 145

6.3.2 Graphsonat mostthreeverti es . . . 149

6.3.3 ProofofLemma6.40. . . 151

6.4 Dis ussion . . . 158

7 ConservativeConstraintLanguagesand Arbitrary Weights161 7.1 Basi redu tions . . . 163

7.2 Undire tedgraphswithloops . . . 165

7.3 A omplete lassi ation. . . 171

7.4 Con lusions . . . 175

8 Exponential-TimeAlgorithms 177 8.1 Themi rostru ture . . . 180

8.2 The overingmethod . . . 181

8.3 AlgorithmsforMax Sol . . . 184

8.4 TheMax Hamming Distan eproblem . . . 189

8.4.1 AlgorithmforMax HammingDistan e

(2, 2)

-Csp . 190 8.4.2 AlgorithmforMax HammingDistan e

(d, l)

-Csp. . 200

8.5 Dis ussionandopenproblems. . . 204

Introdu tion

Intheresear held of omputational omplexitytheoryonestudies

prob-lems, andthe e ien ywith whi h these problems an be solved. Both

of these terms ome with non-standard interpretations whi h really have

quite little to do with what one would normally onsider a problem or

e ient. Letusthereforebeginwithsomeexamplesin ordertohighlight

thisdis repan y,andtointrodu esomerelatedvo abulary.

Problems,instan es, and algorithms

Everydayween ounter problemsofdierentkinds. Thismaybeanything

from appropriatelydividingourtimeand energybetweenanumberof

dif-ferenttasks,orguringoutour urrentlo atingonamapofalesserknown

ity,todeterminingthepre isesetofdo umentsrequiredintheappli ation

foraRussianvisa. Someproblemsareeasy,othersarehard. Mostofthese

problemsdonotlendthemselvesverywellto formalisationinto thesortof

problemswewillbedealingwith;forus,itwillbeimportantthatboththe

problem and a solution an be pre isely des ribed, and that the method

with whi h we wantto obtainthesolution hasapredi table out ome. To

simplify things, wetherefore narrowourdis ussion to problems whi h are

in some sense mathemati al, or  ombinatorial. As in all s ien es, su h

idealised models and approximations of real-world problems always ome

equipped with limitationsof appli ability, but theyhavetheadvantageof

beingmu hmoreeasilyanalysedinaformalsetting. Letushavealook at

thefollowingfamiliarproblem:

Solvefor

x

:

x

2

− x − 1 = 0

From the viewpointof omplexity theory, this is in fa t nota problem at

all, but an instan e of a problem. A problem onsists of a olle tion of

su hinstan es. Wesolveaproblembymeansofanalgorithm: asequen e

the problem of solving arbitrary quadrati equations. This problem an

besolvedbymeans ofthe ompleting thesquare-algorithm. Tospe ifya

problem we thus need to spe ify the set of instan es of the problem, but

wealsoneedto spe ify what onstitutesasolution. Intheaforementioned

example,thealgorithmwilldelivertwosolutions:

x =

1 ±

√

5

2

≈ 1.6180, −0.6180

If wearehappywithsolutionsin radi alsandelementaryoperations, then

the rst answer is a eptable, but sin e both solutions are irrational, we

annot produ e a orre t, nite, de imal expansion,for either of them. If

wede ideto settlefor four orre tde imal pla es, thenthese ond answer

is a eptable. In either ase, we need to make sure that the form of the

desiredoutput ispre iselyspe ied,and wehaveto hooseanappropriate

omputerrepresentationforit. Similar onsiderationsmustbegiventothe

representationoftheinstan es.

We anof oursedeneotherproblemswhi hin ludethegivenequation

x

2

_{− x − 1 = 0}

asaninstan e. Considertheproblemofsolvinganarbitrary

polynomialequation, regardlessofdegree. Weknowthat ifweaskfor the

solutionsintermsofradi alsandelementaryoperations,thenfor

polynomi-alsofdegreegreaterthanfour,weareataloss. Here,wewillhavetosettle

for approximate solutionsin the general ase, even if we an obtainexa t

solutionsformanysu hproblems.

E ien y and resour es

Theideaof onsideringthegeneral,worst aseappearsagainwhenwestudy

thee ien yofour hosenalgorithm. Therearemanywaysofdeningthe

e ien y of an algorithm, but they all involve restri ting the resour es

whi h thealgorithm needsto performits task. Twoof themain resour es

in modern day omputersare time and memory. Either hoi e leads to a

ri htheoryandposesmanyfas inatingopenproblems, butwewill restri t

ourselvestothestudyoftime. Infa t,time isaslightlymisleadingterm.

Onegenerallystartsoutobservingthatthesamealgorithmwill ompletein

dierentamountsoftimeondierent omputerhardware,orevenwhenrun

twi eonthesamehardware,giventoday'selaborate omputerar hite tures.

We remedy this by ounting not the number of lo k y les it takes to

perform atask, but ratherthe numberofinstru tions, orsteps,whi h are

arried out by the algorithm. One therefore settles for some `reasonable'

formalmodelof omputation, andregardsdieren esin multipli ativeand

additive onstantsbetweensu hmodelsasuninteresting.

Inprin iple, we ouldnowstudy afun tion

t

whi h assumesthe num-berofstepsaspe i algorithmwillneed,givenaninstan eoftheproblem.

Thiswouldprobablybe omeavery ompli atedfun tiontodetermine,even

gathermoreinformation,orastobetterapproximatesomereal-world

prob-lem. Thus, itwould makemoresense tostudy thenumberofstepsneeded

forthe algorithm to ompleteits omputation asafun tion of thesize of

theinstan e. Thesize ofaninstan eis afun tionwhi hisdire tly related

tothe on reterepresentationthathasbeen hosenfortheproblem. We an

formallydeneitasthenumberofbitsneededto en odetheinstan e, but

wewillusuallysettleforafun tionwhi hgrowsasymptoti allyasthea tual

size. Thefun tion

t

is thenafun tion from naturalnumbers, representing thesizeoftheinstan e,tonaturalnumbers,representingthemaximal

num-berofstepsrequiredtosolveaninstan e ofaspe i size. This is alled a

worst aseanalysis;theremaybeinstan esofagivensize

n

whi hthe al-gorithm ansolveveryqui kly,butingeneral it andonobetterthan

t(n)

. Alsonotethat

t

maybeapartialfun tion sin etheremaybenoinstan es atallforsomeparti ularvaluesof

n

. Now,wearenota tuallyinterestedin theexa tvalueswhi h

t

assumes,but ratherin howit behavesas

n

grows large. This is formalisedin the followingasymptoti notation. Forpartial

fun tions

t

and

g

,wewrite

t(n) ∈ O(g(n)),

ifthereare onstants

C

and

n

0

su hthat

t(n) ≤ C · g(n)

forall

n > n

0

.

Thisnotation,atthesametime,takes areofreasonableapproximationsin

the hoi eofinstan esize,andtheuninterestingadditiveandmultipli ative

onstantsdue tothe omputationalmodel. Fornotational onvenien e, we

write `

t ∈ O(n

2

₎

' insteadof `

t(n) ∈ O(g(n))

forafun tion

g(n) = n

2

'. In

this ase,wealsosaythatthetime omplexityofthealgorithmasso iated

to

t

is

O(n

2

₎

.

Example1.1 Consideragaintheproblemofsolvinggeneralquadrati

equa-tions. Aquadrati equation anbegivenonastandardformas:

ax

2

+ bx + c = 0.

An instan e an thus be en oded by three numbers:

a

,

b

, and

c

. If we assume that

a

,

b

, and

c

are rational oe ients, ea h given by the ratio of two relatively prime integers, then the numberof bits to representthe

instan eisapproximatelythesumofthe2-logarithmsoftheabsolutevalues

of the six integers. Sin e we are disregarding onstants, we an a tually

let the size be the 2-logarithm of the largest among the absolute values

of the six integers. If we hoose to produ e anexpression in radi als and

elementaryoperations,thenessentiallyalloperationsofthealgorithmarea

xednumberofadditions,subtra tions,andmultipli ations. Itisreasonable

to assumethat additionsand subtra tions anbe arriedoutin anumber

that for this algorithm,

t(n) ∈ O(n

2

₎

; we say that the algorithm runs in

quadrati time. It is also ommon to ount the number of arithmeti al

operations, asopposed to the number of bit operations. In this ase we

wouldsaythatthepreviousalgorithmrunsin onstanttime. Normally,we

aremu hlessspe i inhowwerepresentinstan esofourproblems,relying

heavily ona epted onventions, and theabsorption of onstants into the

O

-notation.

`How about a ni egame of hess?'

Thegameof hesstakespla eonanidealisedbattleeldwheretheopponents

pra tise theirstrategi and ta ti al skills. It is thear hetypalwargame.

A hessproblem,orpuzzle,isaprespe iedpositiongivenona hessboard,

oftena ompaniedbyashortinstru tionontheform`whitetoplayandwin'.

Theobje tiveoftheproblem isto omeupthethebest movein thegiven

situation. Forthe`...toplayandwin'-instan es,we andeneabestmove

asonewhi halwaysleadstoawinfortheplayer. Anexamplepuzzleisgiven

inFigure1.1 1

. Thisis learlyinlinewithwhatwemeanbyaproblem;whois

Figure 1.1: White toplayandwinin 2.

toplay,thelegalpositions,andthemovesbetweenthemareallwell-dened

andeasytoen odein a omputerreadableformat. Howeverfromthepoint

ofviewof omputational omplexitytheory,theproblemasstatedistrivial:

it an besolvedbyanalgorithmwhi hrunsin a onstantnumberofsteps.

Thiswasobservedalreadyin1950byClaudeShannon[125℄,wherehegives

aroughestimateonthenumberofpossiblepositionsin hesstoabout

10

43

.

1

Wemaywant to onsider alsosome positions whi h annevero ur from

normal play but whi h still pose interesting hess problems. In any ase,

sin ethisnumberisboundedbysome(admittedlyverylarge) onstant,the

numberofpossibleinstan estoa hesspuzzle-solvingalgorithmis onstant.

Butthen, we an simplyhard- ode ea h positionand its solutioninto one

giganti table. This table may takea longtime to produ e, and it would

hardlytintoanyexisting omputermemory,butthepointisthatitisnite

in size and ould, in prin iple, be onstru ted. The asymptoti growth of

thenumberof stepsrequired foran algorithm, whi h givenapuzzle looks

uptheanswerinthistableisthusalso onstant: thereissomepuzzlewhi h

takesthemostnumberof omputationalstepstosolve. The hesspuzzle

problem anbeturnedintoa( omplexitytheoreti ally)non-trivialproblem,

forexamplebyhavingitplayedonageneralised

n

times

n

hessboard. Linear programming

Wenowtakealookatperhapsthe mostwell-knownproblemin

ombinato-rialoptimisation: linearprogramming. An instan eof linearprogramming

isspe iedbyave tor

c

ofsize

n

,ave tor

b

ofsize

m

,andan

m × n

matrix

A

. All entriesin theve torsandin thematrixaresupposedto berational numbers. Thelinearprogram orrespondingtothisinstan e anbewritten

asfollows:

Maximise

c

T

_x

subje tto

Ax ≤ b,

x

_{≥ 0.}

The entries in the ve tor

x

= (x

1

, . . . , x

n

)

T

are alled variables, and the

aim is to assign rational values to the variables so that

x

resides in the polytopespe iedbytheinequalities,andsothattheobje tivefun tion

f (x) = c

T

_x

ismaximised.

The lassi algorithmforsolvinglinearprogramsisthe simplex

algo-rithm, des ribed by George Dantzig in 1947. (First mention of the term

`linearprogramming'seemstobeDantzig[36℄. Seealso[37℄.) Thesimplex

algorithmperformsverywellinpra ti e,andsimplex-basedalgorithmsare

stillat theheartofmanylinearprogramsolverstoday. Thealgorithm an

roughlybedes ribedasfollows: arstobservationisthat eitherthelinear

obje tive fun tion angrow arbitrarily, in whi h ase a maximal solution

doesnotexist,or it mustattain itsmaximumin avertexof thepolytope.

Furthermore,fromanon-optimalvertex,we analwaysrea habetter

ver-texalongsomeedgeofthepolytope. Theideaisthereforetopi kastarting

vertex,and towalkalongtheedgesofthepolytope,from vertextovertex,

alwaysinadire tionwhi hin reasestheobje tivefun tion. Unfortunately,

this strategy has an, at least theoreti al, draw-ba k; it is possibleto

everyvertexof thepolytopebefore rea hingthemaximum. This

onstru -tionworksinanydimension,andin thisfashionwe anprodu eaninnite

sequen e ofpolytopeswhosevertex ountgrowexponentiallywiththesize

of theinstan es. Theworst asetime omplexityofthesimplex algorithm

is thereforeexponential,andthealgorithm,from ourpointofview, annot

be onsiderede ientatall.

Ittookoverthreede ades,anda ompletelydierentapproa h,to ome

up with a omplexity theoreti ally satisfying improvement (during whi h

time, of ourse, the simplex algorithm ontinued to solvean ever

in reas-ing number of instan es in pra ti al appli ations all over the world.) In

1979, Leonid Kha hiyanpresented the rstpolynomial-time algorithm for

linear programming, based on the the ellipsoid methodintrodu ed by

NemirovskyandYudin,andindependentlybyShor,inthe1970s.

Paradox-i ally, this method proved to perform worse in pra ti ethan the simplex

algorithm. Advan es in s ien e, however, must be judged both on their

immediate pra ti al impa t as well ason the ideas and possibilities they

bringtolight,andtheimportan eofthisresultishardtooverstate. Today,

interior point methods, introdu ed by Karmarkar [90℄, provide linear

programsolverswhi hbothperformwellinpra ti e,andrunin worst ase

polynomialtime.

Nowthat wehaveabetterunderstanding ofwhat wemeanbyaproblem,

andhowwemeasuree ien y,wewillmoveontothemostfamousquestion

in omputers ien e, ompa tlystated`P

6=

NP

?

'. Complexity lasses,redu tions and ompleteness

One of the interestsof the omputational omplexity theorist is to de ide

uponsomerelevantmeasureofe ien y,andthentotryandexpressinsome

moreorlessexpli itwaywhi hproblems an(andoftenmoreimportantly,

whi hproblems annot)besolvedwiththise ien y. The olle tionofall

su hproblems onstituteswhatwe alla omplexity lass. Thein lusion

ofaproblemina omplexity lass anbeprovedbywritinganalgorithmfor

theproblem, andto provethatthealgorithmise ientintherightsense.

Webeginby onsideringonlyde ision problems,whi hareproblems

where the set of solutionsis takento be

{

yes, no

}

. The mostbasi lass ofproblemsis alled P,whi hstandsforpolynomialtime. Thisisthe lass

ofde isionproblemsforwhi hwe anwriteanalgorithmthat, intheworst

ase,runsin anumberofstepsthat isboundedbyapolynomialin thesize

oftheinput. Inthis lasswendproblemssu has`Is

x

aprime number?', andde idingwhetherthereisa onne tedpathinanetworkfromonepoint

to another. Manyproblems are morenaturallystated as omputing some

fun tionvalueinalargerdomainthan

{

yes,no

}

. Forexample,thegreatest ommondivisorof twointegers anbe omputedin polynomialtimeusing

`Is thegreatest ommondivisor of

a

and

b

greaterthan orequalto

c

?' It isgenerally onsidered that ifaproblem annot besolvedin apolynomial

numberofsteps,thenasthesizeoftheinputgrows,itwillqui klybe ome

infeasibleto obtainasolution,evenwithtomorrow's omputers 2

. Wehave

seenthatlinearprogrammingfallsintothis lass,althoughitwasnotproved

before1979. Wehavealsoseenthat eventhoughthesimplex algorithmin

general annotbe guaranteed torun within apolynomialnumberofsteps,

it isnonetheless the algorithm of hoi e overthe ellipsoid method for this

problem. Thisissueseemstoarisepartlyfromthefa tthatwestudyworst

ase omplexity. Thesimplexalgorithmmayindeed runin anexponential

numberofstepsfora leverly ontrivedinput,butsu hinputsdonoto ur

frequentlyinpra ti e.Itisworthmentioningthattheworst ase omplexity

approa h is not the only one taken, and that, for example, the study of

average ase omplexity is a growing and interesting eld in its own

right. Nevertheless,thedistin tionbetweenproblemsinside andoutsideof

P,andthe ontinuedeorttotryandunderstandthisboundary,hasproved

extraordinarilyfruitful,bothfromatheoreti alandpra ti alpointofview.

The se ond lass whi h we will look at is alled NP, whi h stands for

non-deterministi polynomialtime. It anberoughlydenedasthe lassof

(de ision)problemsforwhi ha orre tsolution anbe he kedinanumber

ofstepswhi hisboundedbyapolynomialinthe sizeofthe solution 3

.

The lass NP is known to ontain, in addition to all of P, also quite

afew problemswhi h arebelieved notto bein P. Toreturn to our hess

example,althoughitdidnot onstituteaveryinterestingproblemfromour

point ofview, we intuitively ndthat it should beeasier, in somesense,

toverifya orre tsolutionthanto ndone. Yet,almost40yearsafterthe

on eption of the lass NP [29, 104℄, we still have no luehow to de ide

whetheror notP

6=

NP,i.e.whether itisreallyhardertosolveaproblem inNP froms rat h, thanitisto verifyasolutionwhi his giventous. Of

ourse, westrongly believe that it is harder, and it is not a ontroversial

hypothesistobuildone'sworkupon. Still,sin emostofthisthesisisabout

tra ing this boundary between easy and hard, if it were to turn out that

P

=

NP, i.e. that there is no boundary at all, then many of our results wouldberenderednullandvoid.

Oneofthereasonswhythequestion`P

6=

NP

?

'seemssohardtoanswer is that aproofseems torequirean expli itproblem in NP whi h isnot a

memberof P ,and aproofof su h apropertymust ne essarily onsiderall

possibleknownandunknownalgorithmsforsolvingthisparti ularproblem.

There ishoweverate hniqueforsingling outproblems whi h areharder

thanothers(oratleastnoteasier.)Thebasi omponentsinthiste hnique

has be ome known as redu tions, and thestudy of omplexity theory is

somu hbuiltaroundthemthat it ould,withsomea ura y,be alledthe

2

However,quantum omputingmayputadentinthislongheldtruth.

3

study of redu tions. We willmeetmany typesofredu tionsin this thesis,

but wewill mentionhereonlythe Karp redu tion, namedin thehonour

of omputers ientistRi hard Karp, and more ommonly referredto asa

polynomial-timeredu tion.

Denition 1.2 Apolynomial-time redu tion fromaproblem

Π

1

toa prob-lem

Π

2

isa fun tion, omputablein polynomial time, whi h takes as input an instan e

I

1

of

Π

1

andprodu esasoutput aninstan e

I

2

of

Π

2

su hthat the solutionto

I

1

in

Π

1

isyesifandonlyifthe solutionto

I

2

in

Π

2

isyes. Itfollowsthatsu hredu tionspreservebothPandNP,inthesensethat

if

Π

2

is in P (NP ), then

Π

1

is in P (NP). If

Π

1

is polynomial-time redu ibleto

Π

2

,then wethinkof

Π

1

asbeingnoeasierthan

Π

2

to solve. Indeed,toanyalgorithmfor

Π

2

we anaddtheredu tionasarststepand get an algorithm for

Π

1

whi h is at most apolynomialfa tor slower than theoriginalalgorithm. Hen e,if

Π

1

isnotinP,thenneitheris

Π

2

.

A quiteremarkablefa t is that there areproblems

Π

in NPwhi h are harder than any other problem in NP in the sense that every problem

in NP isKarpredu ibleto

Π

. We allsu h problemsNP- omplete,and hundredsofthemhavebeenidentiedsin etheseminalpapersofCook[29℄,

Levin [104℄,andKarp[91℄,manyofwhi happearinthestandardreferen e

by GareyandJohnson[56℄. The notionof ompletenesso urs in many

other omplexity lassesaswell,and ompleteproblemsareusedasabasis

forprovingotherproblemshardand/or omplete.

Weendthisbriefintrodu tionto omplexitytheorybymentioningthatjust

astherearemanymorenotionsofe ien y,therearemanymore omplexity

lasses, mostof whi h wewill not onsider at all. In fa t,so many lasses

havebeendenedovertheyears,forvariouspurposes,thattheynowreside

in aComplexityZooadedi atedwiki urrentlykeeping489 lasses 4

,and

ounting!

1.1 Constraint satisfa tion

In thisthesis we onsider aparti ular lassof problems alled onstraint

satisfa tion problems,orCsps. The formaldenition ofaCsp is given

in thepreliminaries, Se tion 2.2. Forthe purposes of this introdu tion, it

will su e with aninformaldes ription,whi h is givenafter thefollowing

example.

Example1.3 Thestru tureinFigure1.2isknownasa( ombinatorial)

graph. Su hagraph onsistsofasetof points, alledverti es,and aset

oflines onne tingpairsofpoints, allededges. They anbeusedtomodel

relationshipsbetweenvarious entities. Asanexample,we anthinkof the

graph in Figure 1.2 as a model of live variables in a omputer language

u

1

u

2

u

3

u

4

u

5

v

1

v

2

v

3

v

4

v

5

Figure 1.2: ThePetersengraph.

ompiler. The register allo ation stepin a ompilerassigns physi al CPU

registers to the variables. Inthe graph, welet the verti es represent the

variables,andanedgebetweentwoverti esrepresentsthefa tthattheirlive

ranges overlap,i.e.thereissomepointduringtheexe utionoftheprogram

when both variables may arry important values. An edge between two

verti es thus implies that these two verti es may not be allo ated to the

samephysi alregister.

A proper vertex olouring of agraphis an assignmentof  olours

totheverti esofthegraphsothatnotwoverti eswhi hshareanedgeare

assignedthesame olour. Inourexample,the oloursrepresentthephysi al

registers,andaproper olouring orrespondstoarealisablevariable-register

allo ation. Wewill use theset

{

reg1, reg2, reg3

}

as olours, and show that three physi al registerssu e to allo ate all tenvariables of the

Pe-tersengraph. Thefollowingassignment

f

isaproper olouring:

f (u

1

) = f (u

2

) = f (v

3

) = f (v

5

) =

reg1

f (u

3

) = f (u

4

) = f (v

2

) =

reg2

f (u

5

) = f (v

1

) = f (v

4

) =

reg3

Theproblem3-Colouringisthatofde iding,foranarbitrary inputgraph,

whether there exists a proper olouring using no morethan three distin t

Informal denition

A Cspinstan e is determined by three omponents, aset of variables, a

set ofvalues,andaset of onstraints. Thesetofvariables,whi hwewill

denote by

V

, is in the ase of Example 1.3 given bythe set of verti esof thegraph:

{u

1

, . . . , u

5

, v

1

, . . . , v

5

}

. Wealsoneedasetofvalues,ordomain elements. Thisset,whi h wewilldenote by

D

, is alledthe domain. In the example, it is given by

{

reg1, reg2, reg3

}

. Finally, we need a set of onstraints,whi hwewill denoteby

C

. Constraintsare lo al onditions onthevaluesallowedforea hvariable. The onstraintsin Example1.3are

givenbythe rulethat notwoverti essharinganedge may havethesame

olour. For a spe i graph, this rule an be modelled in many dierent

ways. It turns out that the most natural way is to have ea h onstraint

apply only to a single pair of verti es, e.g. 

u

1

and

u

3

must be assigned dierent olours. The examplethen ontains15dierent onstraints,one

forea hedgeinthegraph.

A fun tion

f : V → D

is alled an assignment. The assignment

f

satises a onstraint

c ∈ C

if it givesvalues to the variables in su h a way that the given onstraint

c

holds. In the example, ea h onstraint

c

appliesto twoverti es,say

v

1

and

v

2

, sowe anverifythat

f

satises

c

by he king that

f (v

1

) 6= f(v

2

)

. A satisfying assignment,or solution,to a Cspinstan e is anassignmentwhi h satises all ofthe onstraintsin

C

simultaneously. In the 3-Colouring example,the satisfying assignments

arein one-to-one orresponden ewithproper olourings.

For a given Csp instan e, and asuggested assignment

f

, it is easy to he k, in polynomial time, whether or not

f

is satisfying. Hen e Csp is in the lass NP, and from Example 1.3 we on lude that the problem of

de idingwhetherornotagivenCspinstan ehasasatisfyingassignmentis

NP- omplete.

Parametrisation and the di hotomy onje ture

Sin e the Csp framework an be used to model so many interesting and

importantproblems,butisNP- omplete ingeneral,one antrytorestri t

its power in dierent ways. One possibility is to restri t the size of the

domain

D

to obtain a parametrised problem in whi h any instan e must satisfy

|D| ≤ k

. Unfortunately eventhe problem Csp restri tedto a two-element, or Boolean, domain is NP- omplete. It ontains the problem of

de idingthesatisabilityofapropositionallogi formula,whi hwasamong

therstproblemsshowntobeNP- omplete. Adierentrestri tiononCsp

istoallowonly ertaintypesof onstraints. Forexample,wehaveseenthat

3-Colouring anbedes ribed with onstraintsof the form

f (u) 6= f(v)

. Thebinaryrelation

6=

onthedomain

D

isthustheonlytypeof onstraint weneed,andtherestri tionofCsptoonlysu h onstraintswillbedenoted

Csp

({6=})

. Ingeneral,we antakeanyset

Γ

of onstraintsandletCsp

(Γ)

denote the restri tion of Csp where only onstraints from the set

Γ

are

allowed. S haefer [119℄ managed to lassify the restri tion of Csp over a

Booleandomainany xedsetof onstraints

Γ

asbeingeitherinP,orbeing NP- omplete. This di hotomytheoremisquiteremarkable. ByLadner's

theorem [101℄, we know that, unless P

=

NP, there are problems whi h areneitherin P ,norin NP. Yet,theCspproblemoveraBooleandomain

ontainsonlytwotypesofproblems: easyandhard.

Asimilarphenomenono ursforanother lassofCsps,thistimerelated

to graphs. A graph homomorphismfrom a graph

G

to agraph

H

is a fun tion fromtheverti esof

G

to theverti esof

H

su hthat theimage of ea h edge in

G

is an edge in

H

. Thegraph homomorphism problem parametrisedbyagraph

H

,anddenotedHom

(H)

,istheproblemof de id-ing,foragivengraph

G

,whetherthereexistsagraphhomomorphismfrom

G

to

H

. Inparti ular,theproblemHom

(K

3

)

,where

K

3

denotesthe omplete graphonthreeverti es,isnothingmorethanthe3-Colouring-problemin

disguise. HellandNe²et°il[71℄showedthatforundire tedgraphs,Hom

(H)

isinPif

H

iseitherbipartite,orifithasaloop. Inallother ases,Hom

(H)

is NP- omplete. Note that the ase

H = K

3

is notbipartite, and hasno loop,hen eHom

(K

3

)

isNP- omplete.Thisisinagreementwithour laim inExample1.3that 3-Colouringis NP- omplete.

Guided by the o urren e of su h a di hotomy in twoimportant

sub- lasses of Csp , Feder and Vardi [53℄ onje tured that Csp

(Γ)

in general mustexhibitadi hotomy. Theirargumentwasbasedonastudyofa lass

named MMSNP (monotone monadi SNP without inequalities). They

showedthatif eitherof thethree restri tionsmonotoni ity, monadi ity, or

noinequalities isremovedfromMMSNP,theresulting lassis

polynomial-timeequivalenttoallofNP. InthissensetheyidentiedMMSNPasthe

largestsub lassofNPwhi h anpotentiallyexhibitadi hotomy. They

fur-thershowedthatCspandMMSNParepolynomial-timeequivalent. This

means that while in a stri t sense Csp is a sub lass of MMSNP, there

exists a polynomial-time redu tion from every problem in MMSNPto a

problem in Csp . Thus theyaskedthe question: Is every problem in Csp

either in P or NP- omplete? The onje ture that this isindeed the ase

hasbe omeknownas `thedi hotomy onje ture':

Conje ture 1.4 (The Di hotomy Conje ture) Given afamily of

on-straints

Γ

,the problemCsp

(Γ)

iseitherin P,or itisNP- omplete. Thedi hotomy onje turehasinspiredmu hofthetheoreti alresear h

on the onstraintsatisfa tion problem. Progress towards settling

Conje -ture1.4haslargelybeendrivenbyamethodwhi hhasbe omeknownasthe

algebrai approa h,orthealgebrai method. Anearlyandinuential

paperinthisdire tion isJeavons,Cohen,andGyssens[79℄. Bulatov,

Jeav-onsandKrokhin[22℄ use hara terisationsfrom universalalgebrato study

languages[18℄, onstraintlanguagesoverathree-elementdomain[20℄,and

apolynomial-timealgorithmforso- alledMal'tsev onstraints[21℄.

1.2 Optimisation

Tomerelyknowtheexisten eofasolutiontoagivenproblemmaybe

insuf- ient. Onemaywishto obtainthebest solution,in somespe i sense.

This is the asein parti ular forlinear programming,where the existen e

of asolution onlyassertsthat the polytopespe iedby theinequalitiesis

non-empty,whilethegoalgenerallyistooptimiseanobje tivefun tionover

thepolytope.

Similar need for optimisationo urs for the solution spa es of general

onstraint satisfa tion problems. There are several approa hes to

intro-du ing an optimisation riterion on Csp instan es. We will primarily be

on ernedwiththefollowingtwotypesof riteria:

1. One an assign weights to the variables and domain elements and

introdu eanobje tivefun tion basedonthese weights.

2. One anallow onstraintstobeunsatisedinasolution,andbasethe

obje tivefun tion onthenumberofsatised onstraints.

Forbothoftheseapproa hes,theresultingproblemisin generalnoteasier

than its non-optimising ounterpart. In the rst ase, if we an nd an

optimal solution to a parti ular problem using some restri ted resour es,

then we an learlyndone. Inthese ond ase,thereare solutionsto the

de ision problemifand onlyif amaximalsolutionsatises all onstraints.

Wewillnowillustratethesetwoapproa heswithtwoexamples,theproblems

MaximumIndependentSetandMax ut. Bothexamplesalsoillustrate

that an optimisationproblem an bestri tly harder(in this aseprovided

P

6=

NP)thantheoriginalproblem.

1.2.1 Example: Maximum Independent Set

Asubset

S

oftheverti esofagraphis alled anindependentsetifthere is no edge between any pair of verti es

u, v ∈ S

. For thePetersen graph in Figure 1.2 it iseasy to ndan independentset of size four, e.g.theset

{u

1

, u

2

, v

3

, v

5

}

,whi histhesetofverti esof olour reg1inExample1.3. Wealsoseethatitisnotpossibleto hoosemorethantwoverti esfromthe

outerpentagon,normorethantwoverti esfromtheinnerpentagram,ifwe

wish toobtainanindependentset. Hen ethesize ofalargestindependent

set inthePetersengraphisexa tlyfour.

The problem Maximum Independent Set is to nd an independent

set of maximumsizein agiveninput graph

G

. Thisoptimisationproblem will appearmanytimesthroughoutthisthesisasatypi alhard problem;

parti ularone anndapolynomial-timeredu tionfrom anNP- omplete

problemtoMaximum Independent Set. Weoftenexpressthisbysaying

thatMaximumIndependent SetisNP-hard. However,itis learlyeasy

tondone solution: theemptysetisasubsetoftheverti esofanygraph,

anditisalwaysanindependentset.

ToobtainaCsp-formulationoftheproblemMaximumIndependentSet,

weagain taketheverti esasvariables

V

, but thistime welet thedomain

D

be

{0, 1}

. Ourintendedinterpretationofanassignment

f

from

V

to

D

is thatifavertex

v

isassignedthevalue1by

f

, then

v ∈ S

, andif

f (v) = 0

, then

v 6∈ S

. Forea hedgebetween

u

and

v

,wehavethe onstraint:

(f (u), f (v)) ∈ {(0, 0), (0, 1), (1, 0)}.

Inorderto solveMaximum Independent Set,wethusneedtomaximise

the numberof ones assigned bya solution

f

subje t to onstraintsof the form

R = {(0, 0), (0, 1), (1, 0)}

. Thisproblem ofmaximisingthenumberof onesinasolutiontoaCspinstan e,expressedusingonlythe onstraint

R

, is alledMax Ones

({R})

.

TheproblemMax OnesisanoptimisationproblemforCspoftherst

typewith thelinearobje tivefun tion

X

v∈V

f (v).

1.2.2 Example: Max ut

Themoststudied problemin these ond ategoryis alled Max ut. Let

G

be agraphwith vertexset

V

and edge set

E

, and let

S

be asubset of

V

. Wesaythat

S

indu esa utin

G

,whi histhesubsetofedgesthatare atta hedto onevertexin

S

andonevertexin the omplement,

S = V \ S

. Theproblem Max utisthentonda utoflargestsizein agiveninput

graph

G

. ThisproblemisalsoNP-hard,andhasbeenextensivelystudied. SimilartoourpreviousCsp-formulations,weidentify theset ofverti es

V

asvariablesandweletthedomain

D

be

{a, b}

. Forea hedgebetweentwo verti es

u

and

v

,wehavea onstraint

(f (u), f (v)) ∈ {(a, b), (b, a)}.

Itis easyto seethat theresultingCsp instan eis satisableifand onlyif

G

is bipartite. This is pre isely when the maximum ut in

G

is given by theentireset

E

. Theproblem ofdeterminingwhether ornotagraph

G

is bipartiteisin P .

WeobtaintheproblemMax utbyaskingnotiftheinstan e is

satis-able, butinsteadfor anassignment

f

whi h satises asmany ofthe on-straintsaspossible. Ea h onstraintsatisedby

f

orrespondsto anedge inthe utindu edby

f

. Theset

S

anbere overedfrom

f

,forexampleby letting

S = f

−1

_(a)

1.3 Approximability

Inthepreviousse tion,ween ounteredtwooptimisationproblems:

Maxi-mum Independent Setand Max ut. Bothof theseproblems are

NP-hard to solve to optimality. There is however a way to dierentiate the

hardnessofthetwo,ifwe onsiderapproximationalgorithmsinpla eof

exa talgorithms.

Example1.5 Let

G = (V, E)

beagraph. Pla e ea hvertex

v ∈ V

into

S

or

S

with equalprobability. Forea h edge

e ∈ E

, let

X

e

= 1

if the edge

e

is in the ut indu ed by

S

, and let

X

e

= 0

otherwise. By linearity of expe tation,wehave

E

X

e∈E

X

e

!

=

X

e∈E

E(X

e

) =

1

2

· |E|.

Sin e the expe ted measure of a solution, pi ked at random, is half of the

total number of edges, it follows that there exists some solution with a

measure at leastthis big. We an obtainsu h asolutionby thefollowing

re ursivepro edure. If

G

onsistsof onlytwoverti es, thenweput oneof themin

S

,andtheotherin

S

. Thisisasolutionwithmorethanhalfofthe edges in the ut. Otherwise, arbitrarily pi k avertex

v

, and remove

v

, as wellastheedges onne tedto

v

, fromthegraph

G

. Wethus obtainanew graph

G

′

_{= (V}

′

_{, E}

′

₎

, withfewerverti es. Re ursivelyapply thispro edure

to obtainasolutionforthegraph

G

′

with, byassumption,ameasureofat

least

1

2

· |E

′

|

. Inthissolution,someoftheneighboursof

v

arein

S

,andthe edgesfrom

v

totheseverti es onstituteaset

E

1

. Theremainingneighbours of

v

arein

S

, andtheedgesfrom

v

tothese verti es onstituteanotherset

E

2

. The nal solution to

G

is obtained by assigning

v

to either

S

or

S

, dependingon thesizeof thesets

E

1

and

E

2

. The measureof thissolution isboundedfrom belowby

1

2

· |E

′

| + max {|E

1

|, |E

2

|} ≥

1

2

· |E|,

sin e

E = E

′

_{∪ E}

1

∪ E

2

.

Wesay that an algorithm approximates amaximisation problem within

a onstant

r

, ifit alwaysreturns asolution

f

, su hthat themeasure of

f

divided by the measure of an optimal solution is bounded from below by

r

. We all

r

the approximation ratio of the algorithm 5

. We similarly

saythat an algorithm approximates a maximisationproblem within some

fun tion

r(n)

,iftheratiobetweenthemeasuresofthesolutionprovidedby the algorithm, and that ofthe optimalsolutionis bounded from below by

r(n)

. Here,thebound

r(n)

varieswiththeinstan esize

n

. 5

Theapproximation ratioforan algorithmofa maximisationproblem issometimes

denedastheratiobetweenthemeasureofanoptimalsolutionandthatofthealgorithm,

Therearetwonatural omplexity lassesforoptimisationproblems,

or-respondingtothe lassesPandNPforde isionproblems. Theseare alled

PO and NPO. The omplexity lass APX is the lass of optimisation

problemsinNPOwhi h anbeapproximatedwithin some onstant

r > 0

. InExample 1.5, we gave anexample of an algorithm whi h approximates

Max utwithin

1

2

. Hen ewehaveshownthattheproblemMax utisin APX. It anfurther be shown that Maximum Independent Set is not

inAPX ,andso,wehavedierentiatedthehardnessofthesetwoproblems.

TheproblemMaximumIndependentSetresidesinsteadina omplexity

lass alled poly-APX, whi hstri tly ontainsAPX (if P

6=

NP.) Both of these lasses have omplete problems under the appropriate redu tions

(see Se tion 2.4.) In fa t, Max ut is APX - omplete, and Maximum

Independent Setispoly-APX- omplete.

The twoproblems Max ut and Maximum Independent Set were

hosenasrepresentativesfortwodistin tapproa hestooptimisationon

in-stan esofCsp. Khannaet al.[93℄ studiesbothofthese approa hesin the

aseof Boolean Csp , from aperspe tiveof approximation. They onsider

bothmaximisationand minimisationproblems, resultingin thefour

prob-lems Max Ones, Min Ones, Max Csp, and Min Csp. In all asesthey

nd that the problems, parametrised by a onstraint language, fall into a

small numberof approximation lasses. Inthe aseof Max Ones, whi h

is of primary on ern to us, they nd that every problem Max Ones

(Γ)

is either in PO, it is APX- omplete, it is poly-APX- omplete, it is an

NP- ompleteproblem to ndasolutionof non-zeromeasure,orthe

prob-lem of nding anysolution is NP- omplete. This situation resembles the

di hotomyphenomenon, onamorereneds ale.

1.4 The maximum solution problem

Inthis se tion, wewill nally meetthe main problem of the thesis. The

Maximum Solution Problem, or Max Sol for short, an be seen

ei-ther asa naturalgeneralisation of Max Ones to larger domains, or as a

generalisation of integer linear programming. It is most easily dened as

themaximisingoptimisationversionofCsp , with valuesasso iatedto the

domainelements,andwithalinearobje tivefun tion. Itisoftennaturalto

take

D = {0, 1, . . . , m − 1}

,sothatthenumeri alvaluesareimpli itlygiven bythe domain elementsthemselves. The obje tivefun tion, ormeasure,

ofasatisfyingassignment

f : V → D

,isin this asegivenby

X

v∈V

ω(v) · f(v),

wherethe oe ients

ω(v)

arenon-negativerationals,andare onsideredto beapartoftheinstan e.

Example1.6 Assumethat

V = {v

1

, . . . , v

n

}

,that

a

= (a

1

, . . . , a

n

)

isarow ve torofrationalnumbers,andthat

b

is arationalnumber. Forea hsu h

hoi eof

a

,and

b

,we andenean

n

-ary onstraint(arelation)

Ineq

a,b

as follows:

Ineq

a,b

= {(d

1

, . . . , d

n

) ∈ D

n

|

n

X

i=1

a

i

· d

i

≤ b},

If welet

Γ = {Ineq

a,b

| a ∈ Q

n

, b ∈ Q}

,then Max Sol

(Γ)

is theproblem integerlinearprogrammingoveraboundeddomain

D

. Notethat

Γ

ontains anite numberof onstraints,even thoughit isnotimmediately apparent

from itsdenition;thereareonly

2

|D

n

_|

distin trelationsofarity

n

. Exa talgorithms forMax Sol (underthename ofMax Value) were

givenbyAngelsmarkandThapper[8℄. Forthe onstraintlanguage

param-eterisation,theproblemhasbeenstudiedforlanguagesdes ribingsolutions

toequationsoverabeliangroups[99℄,forproblemsinmulti-valuedlogi [83℄,

andfor ertainundire tedgraphs[85℄. Thealgebrai approa hforstudying

the approximability of the maximum solution problem was introdu ed in

Jonssonet al.[82℄. It wasusedthereforstudyingmaximalaswellas

ho-mogeneous onstraint languages. Building on this work, Jonssonand

Thapper [86℄ used the algebrai approa h to lassify the approximability

of Max Sol for onstraint languagesinvariantunder twolargefamiliesof

algebras. Fromtheapproximationside,allobtainedresultsforthegeneral

maximumsolutionproblemalignni elywiththoseofKhannaetal.[93℄on

the Max Ones problem. In parti ular, theysuggestthefollowing

onje -ture:

Conje ture 1.7 Let

Γ

be a onstraintlanguage overa nite domain

D ⊆

N

. The problem Max Sol

(Γ)

is either in PO, it is APX- omplete, it is poly-APX- omplete, it isNP-hardto obtain asolution of non-zero

mea-sure, oritisNP-hardtoobtain anysolution.

Arelativelyup-to-datesurveyofthe urrentlyknownresultsrelatingto

theproblem anbefoundinJonssonandNordh [84℄.

1.5 Methods and results

In this thesis, westudy the maximum solutionproblem from anumberof

dierentpointsofview. Weemploybothwell-establishedandnovelmethods

ofanalysis,aswellasafewdierentdierentnotionsofe ien y. Arough

division ofthethesisdistinguishesthepartthatfo usesonapproximability

and the partthat dealswith omputational omplexity. More spe i ally,

in the rst part, we look at a morene grained s ale of e ien y, where

we aim to determine how hard it is to approximate an optimal solution,

example,twoofthe haptersonapproximabilitydealwithspe i

approx-imation onstantsforproblems that areknown tobehard to approximate

arbitrarilywell. Ontheotherhand,inthethird hapteron

approximabil-ity,Chapter3,wetreatamoregeneral lassofproblems, and onsequently

wefo uson lassifyingtheseintheappropriateapproximation lasses: PO,

APX, poly-APX, et . Similarly, while the main subje tof the thesis is

themaximumsolutionproblem,wemake onta twith,andsometimesgive

more attention to other problems, su h as Max

H

-Col (a restri tion of Max CSP) in Chapter 5, and Max Hamming Distan e in Chapter 8.

Wenowbrieypresentthedierentte hniquesused,anddes ribeourmain

resultsand ontributions.

The algebrai approa h

Thisisoneofthemainte hniquesused,anditappearsseveraltimes

through-outthethesis. Itexploitsa onne tionbetween onstraintlanguages

repre-sentedassetsofrelations,anduniversalalgebra,whi hdealswithsetsof

polymorphisms: fun tions operating onthe domain. Thete hniquehas

beenusedprimarily fortheCspde isionproblem,withimpressivesu ess.

It was extended to the maximum solution problem by Jonsson et al. [82℄

andused, amongother things,to lassifyhomogeneous onstraint

lan-guages. These are onstraintlanguageswhi h in lude a onstraintof the

form

{(d, π(d)) | d ∈ D},

for everypermutation

π : D → D

. We ontinuethis eortby lassifying the approximability of Max Sol

(Γ)

for onstraint languages

Γ

whi h are preserved by families of ane algebras. Su h onstraint languages are

des ribedby ertainsystemsof linearequationsoverabeliangroups. This

resultallowsustogiveasimpliedproofforthe lassi ationofMax Sol

for homogeneous onstraint languages(as aspe ial ase of languages

pre-servedbysymmetri algebras.) Wealso lassifylanguagespreservedby

stri tlysimplesurje tivealgebras,whi h anbeseenasbuildingblo ks

formore omplexalgebras[22℄.

There exists another, more general, optimisation framework for Csp ,

alled the valued onstraint satisfa tion problem, or VCSP. It was

intro-du edbyS hiexetal.[120℄,andin ludesMaxSolasaspe ial ase. An

al-gebrai approa hbasedontwogeneralisedpolymorphism on epts,namely

multimorphisms and fra tional polymorphisms, has been developed

forthe study of VCSP. Whilethese showgreat promise, wewill notdeal

Approximationby relaxation and rounding

Thisisa ommonlyappliedte hniquefor onstru tingapproximation

algo-rithms forvariousrestri tionsofintegerlinearprogramming. Itisbasedon

the ideaofrelaxing theintegrality onstraintsof theprogram, after whi h

a solution to the relaxed program an be found in polynomialtime. The

obtainedsolutionis in generalfra tional, andsomekind ofrounding

pro- edure isneededtogenerateanintegersolution. Weusethiste hniquefor

variousrestri tionsofthefollowingintegerlinearprogram:

Minimise

c

T

_x

subje tto

Ax ≥ b,

x

_{∈ X,}

where

c

isnon-negative,

A

isanintegermatrix,and

b

≥ 1

( omponentwise.) Weshowthatsu hprograms,forwhi hthemaximumabsoluterowsumsare

boundedby

k

, anbeapproximatedwithin

k

when

X =

N

n

,but annotbe

approximatedwithin

k − ε

,

ε > 0

,ifKhot'sUniqueGamesConje tureholds ( f.Se tion2.4.2.) Wealsoshowthatndingafeasiblesolutiontothesame

problemisNP-hardinalmostall aseswhen

X = {0, . . . , a − 1}

n

. Previous

studies have fo used mainly on the spe ial ase when

A

is non-negative, so- alled overing integer programs.

Thisproblem isan exampleof aCspoptimisationproblem overan

in-nite domain. Stri tly speaking, it is a Min Sol, rather than a Max

Sol-problem.

Approximationdistan e

Weintrodu eanovelmethoddesignedto extendknownapproximation

ra-tiosforoneproblemtoboundsontheratioforotherproblems. Themethod

worksequallywellforpositiveapproximationresultsasfor negative,

inap-proximability results. The basis forthe method is abinary parameter on

graphs whi h, in a sense, measures a distan e between the graphs. This

distan e an be used to relate the approximation ratios of the problems

parameterised by the orresponding graphs. Our main appli ation of this

method isto the problem (Weighted) Maximum

H

-Colourable Sub-graph(Max

H

-Col,)whi hisarestri tionofthegeneralMaxCSP prob-lem to a single, binary, and symmetri relation. We also show how the

method anbeappliedtothemaximumsolutionproblem, althoughinthis

settingmu hlessisknownabouttheapproximationratios,whi hlimitsthe

appli ability.

Example1.8 Goemans and Williamson [58℄ have designed an algorithm

forMax utusingsemidenite programming,whi h approximatesthe

op-timum within

0.87856

. Ourte hnique allowsus to measure thedistan e between

K

2

, whi h is thegraph orrespondingto Max ut, and

C

11

, the y leon11verti es. Thisdistan eturnsouttobe

s(K

2

, C

11

) = 10/11

,and

we an applyLemma 5.3to ndthat Max

C

11

-Col anbeapproximated within

0.87856 · s(K

2

, C

11

) ≈ 0.79869

.

Conversely, Khot et al. [95℄ have shown that, if the Unique Games

Conje ture holds ( f. Se tion 2.4.2,) then it is NP-hard to approximate

Max ut within any onstant greater than

0.87856

. We an now ap-ply our te hnique to this negative result, and nd that, onditioned on

the same onje ture, it is NP-hard to approximate Max

C

11

-Colwithin

0.87856/s(K

2

, C

11

) ≈ 0.96642

.

Inaddition tothe appli ationof thete hniqueto Max

H

-Colfor awide rangeof graphs

H

, we alsoestablish a lose onne tionto work by ’ámal on ubi al olourings[122,123℄. This onne tionshowsthat ourparameter

s(H, G)

hasthe same numeri alvalue as

1/χ

H

(G)

, where

χ

H

is aspe ial typeof  hromati number. We believethat this insightmayturn out to

be ru ial for understanding thebehaviour of

s

, and in theextension, for understandingtheapproximabilityofMax

H

-Col.

Constant produ tion

Givenadomain

D

, dene

C

D

= {{(d)} | d ∈ D}.

Theset

C

D

ontainsall onstantunaryrelations. Forthede isionproblem Csp

(Γ)

,itisknownthatwhen

Γ

isa ore(i.e.whenallendomorphismsare surje tive,)thenCsp

(Γ)

isinP(NP-hard)ifandonlyifCsp

(Γ ∪ C

D

)

isin P(NP-hard.) Furthermore,therestri tionto a ore anbedonewithout

lossofgenerality. Previously,nosu hresultwasknownforMax Sol. The

on eptofa orehaspreviouslybeengeneralisedto thatofamax- ore( f.

Se tion 2.5,) but the possibility of adding onstantsto the languagedoes

not arry over. We presenta newproperty whi h also aims to mimi the

ore on ept, but is morerestri tivethan themax- ore (in thesense that

beingamax- oreisane essary,butnotsu ient onditionforsatisfyingthe

property.) Weshowthatthispropertyidenties aseswhenaddingthe

on-stantrelations anbedonewithout hangingthe omputational omplexity

oftheproblem. Wealsoshowthat foralanguage

Γ

withoutthisproperty, we an dedu ethe omputational omplexityofMax Sol

(Γ)

from that of itssmallerendomorphi images.

By onsidering what happens when onstants annot be added to the

onstraintlanguage,weshowthat the generalMax Sol

(Γ)

exhibits a di- hotomy if and only if Max Sol

(Γ ∪ {C

D

})

does. Forthe purpose of an-swering the di hotomy question for Max Sol, we an therefore assume

that all onstants are in the language. The te hnique is used to lassify

Max Sol

({H})

for ertainundire tedgraphs

H

withasimplestru ture. Example1.9 Let

H

be a y li graphon aneven numberof verti es, at least six. If we an add the onstant relations to

H

, then it is easy to

see that we an redu e to this problem from the retra tion problem on

H

,andthisproblemisNP- omplete[130℄. Ifwe annotaddthe onstants, thenourresultsshowthatthe omputational omplexityofMax Sol

({H})

is determined by that of the set of problems Max Sol

({g(H)})

for non-surje tiveendomorphisms

g

on

H

. Butfor all su h endomorphisms,

g(H)

is apath,andforpaths,themaximumsolutionproblemisinPO. We an

on ludethatMax Sol

(H)

isinPOinthis ase,hen ewehaveestablished adi hotomyfor y les.

The overingmethod

In addition to thepreviously mentioned omplexity analyses, wewill also

addresstheissueof onstru tingexa t,butexponential-timealgorithmsfor

Csp relatedproblems. Inthis ontext,the ommonand preferred

parame-terisationistorestri ttheproblemto

(d, l)

-Cspinstan es,where

d

indi ates themaximalalloweddomainsizeand

l

themaximalarityofany onstraint. The overing method is a te hnique for extending an

exponential-time algorithmwhi hsolvessomespe i problemfor

(d

′

_{, l)}

-Cspinstan es

to an algorithm whi h solves the same problem for

(d, l)

-Csp instan es, where

d

isstri tly greater than

d

′

. The overingmethod anbeseenasa

wayofderandomisingastandardprobabilisti te hniquefordoingthesame

thing. The basi idea is as follows: assume that we restri t the domain

of ea h variable of the

(d, l)

-Csp instan e to a random

d

′

-subset of the

domain, hosenwithuniformdistribution. Under somemild onditionson

thespe i problem,we anthenusetheexistingalgorithmforsolvingthis

resulting instan e. The probabilityof su ess, i.e. theprobabilitythat the

intended solutionstill remains in the restri tedinstan e, an be bounded

by repeating this pro ess a prespe ied number of times. Typi ally, one

will need to do about

(d/d

′

₎

n

repetitions to obtain a onstant bound on

this probability, where

n

is thenumberof variablesof theinstan e. What the overing method does, is to repla e this probabilisti algorithm with

a deterministi one, with a small penalty on the running time. We use

this te hnique to onstru t algorithms for Max Sol

(d, 2)

, the versionof Max Sol restri tedto

(d, 2)

-Csp instan es. It has also been su essfully appliedtoalargenumberofde ision, ounting,andoptimisationproblems,

f. Angelsmark[5℄.

1.6 Outline of the thesis

We on ludethis introdu tionwith aroad map of the thesis. Themain

body of thesis is, as has already been mentioned, divided into two main

parts, roughly separating onsiderations of approximability from those of

Part I

Thispart ontainstheintrodu tion,aswellassomepreliminaries. In

Chap-ter2,webeginby(re-)introdu ing onstraintsatisfa tionandthemaximum

solution problem, and provide a properformal framework for these

prob-lems. Wethenpro eedwithformaltreatmentsofapproximability,andthe

relatedredu tions. Finally,weintrodu ethebasi on eptsofthealgebrai

approa h. This hapter also serves to establish onventions and notation

whi h willbeassumedthroughoutthethesis.

Part II

Intherstmain partwe onsiderapproximabilitypropertiesofMaxSol,

integer linear programming, and of Max Csp for undire ted graphs. In

Chapter3,weapplythealgebrai approa htotwolargefamiliesofalgebras:

stri tly simple surje tive algebras, and symmetri algebras. The

main ontributionofthis hapteristheresultsforanealgebras.

Chapter 4studies somerestri ted integer linear programs overan

un-boundeddomain. ModulothetruthofKhot'sunique games onje ture,

this hapterprovidestightapproximationratiosfortheseintegerlinear

pro-gramsusingadeterministi roundingte hnique.

Thenal hapteronapproximability,Chapter5,introdu esthe on ept

ofapproximationdistan e, anovelte hniquefor studyingapproximability.

Weapplythiste hniquetoMax

H

-Col(arestri tionofMaxCsp)aswell asto Max Sol. Themain on reteresultsareobtainedforMax

H

-Col, but there is also a substantialpart whi h dealswith the te hnique itself.

A se tion is also devoted to the Max Sol problem. The fa t that the

approximability of Max Sol is mu h less studied, however, restri ts the

appli abilityofthete hnique.

Part III

In this part, we take a oarser view on Max Sol, and try to determine

simply when it is polynomial-time solvable to optimality, and when it is

not. Jonsson et al.[85℄ onje tured that Max Sol witharbitrary weights

andparameterisedbya onservative onstraintlanguageispolynomial-time

equivalent to Min Hom for the same onstraint language. Thanks to a

re entresultbyTakhanov[126℄,weprovethis onje ture,inChapter7.

Chapter 6 dealswith thete hnique of onstant produ tion. It extends

theideas from thepaper`The maximum solutionproblem ongraphs' [85℄

to arbitrarynite onstraintlanguages. Appli ations are given for ertain

undire tedgraphswithrelativelysimplestru ture.

Finally, in Chapter 8,we givesomeexa t,exponential-timealgorithms

fortwoCspproblem: Max Soland theMaxHamming Distan e

Preliminaries

Inthis hapterweformallydene onstraintsatisfa tionandthemaximum

solutionproblem. We alsogive therelevanttheoreti al ba kgroundto

ap-proximability, and its related redu tions. Finally, we introdu e the basi

on epts of the algebrai approa h. This hapter also serves to establish

onventionsandnotationwhi hwillbeassumedthroughout thethesis.

2.1 Basi denitions and notation

Integers,naturalnumbers,rationalnumbers,andrealnumbersaredenoted

by

Z, N, Q,

and

R

, respe tively. Positive integers, rational numbers and real numbersare denoted by

Z

+

,

Q

+

,

and

R

+

, respe tively. Non-negative rationalnumbers,

Q

+

∪ {0}

,aredenotedby

Q

≥0

.

Let

S

beanyset,and

k

apositiveinteger. Wedenoteby

S

k

,thesetof allsubsets

S

′

_{⊆ S}

su hthat

|S

′

_{| = k}

. Wedenote by

[N ]

theset

{1, . . . , N}

. 2.1.1 Graphs

Denition2.1 An(undire ted) graph

G

isatuple

(V, E)

,where

V

isa niteset ofverti es,and

E ⊆

V

2

isasetof edges.

Fora graph

G = (V, E)

, welet

n(G) = |V |

denote the numberof verti es of

G

,and

e(G) = |E|

denotethenumberofedges. When

e(G) > 0

,wesay that

G

isnon-empty. Theneighbourhood,

N

G

(v) = N (v)

,ofthevertex

v ∈ V

in the graph

G

, is dened as the set

{u ∈ V | {v, u} ∈ E}

. The degree,

deg

G

(v) = deg(v)

,ofavertex

v ∈ V

, anbedenedas

|N

G

(v)|

. A graph

G = (V, E)

is alled

d

-regularif

deg(v) = d

forall

v ∈ V

.

Let

G = (V, E)

be a graph. If

E

′

_{⊆ E}

, then we say that the graph

G

′

_{= (V, E}

′

₎

isasubgraphof

G

,andwedenotethisby

G

′

_{⊆ G}

. If

V

′

_{⊆ V}

isasubsetof theverti esof

G

, thenthegraph

G|

V

′

= (V

′

_{, E ∩ V}

′

_{× V}

′

₎

is

alledthesubgraph indu edby

V

′

Attimes,wewillallowedges, alledloops,whi hgofromavertex

v

to itself. Sometimesonealso allowsformorethanoneedge togofrom

u

to

v

foragivenpairofverti es

u, v ∈ V

. Su hedgesare alledparalleledges,and itis learfromourdenitionthat wedonotallowthem. A graph

(V, E)

is alled simpleifitdoesnot ontainanyloopsorparalleledges.

Example2.2 The omplete graph,

K

n

, on vertexset

[n]

, is the graph with edgeset

E(K

n

) =

[n]

2

. Itis

n − 1

-regular,and ontains

n

2

= n(n −

1)/2

edges. The y le graph,

C

n

,onvertexset

[n]

,isthegraphwithedge set

E(K

n

) = {{a, b} | |a − b| ≡ 1

mod

n}

. It is

2

-regular, and ontains

n

edges. Thegraphs

K

n

and

C

n

arebothsimple.

C

n

isasubgraphof

K

n

,but notanindu ed subgraph. Thegraphs

K

5

and

C

7

aredepi tedinFigure2.1.

1

2

3

4

5

1

2

3

4

5

6

7

Figure2.1: Thegraphs

K

5

(left),and

C

7

(right).

Denition 2.3 Let

G

and

H

begraphs. A graph homomorphismfrom

G

to

H

is afun tion

f : V (G) → V (H)

su hthat

{u, v} ∈ E(G) =⇒ {f(u), f(v)} ∈ E(H).

Theexisten eofagraphhomomorphismfrom

G

to

H

isdenotedby

G → H

. In this ase,wesay that

G

ishomomorphi to

H

.

A vertex olouring of agraph

G

is a fun tion

f : V (G) → C

, from the verti es of

G

to a set of olours,

C

. If

f

is su h that for any edge

(u, v) ∈ E(G)

, we have

f (u) 6= f(v)

, then

f

is alled a proper vertex olouring, or simply a proper olouring. Clearly, whether ornot

G

has a proper olouring with olours

C

depends only on the ardinality of

C

. This observation suggests the denition of the following important graph