Market and resource allocation algorithms with application to energy control

IT Licentiate theses 2001-005

Market and Resource Allocation Algorithms

with Application to Energy Control

P^ER C^ARLSSON

UPPSALA UNIVERSITY

Department of Information Technology

with Application to Energy Control

BY

PER CARLSSON

May 2001

DEPARTMENT OFCOMPUTING SCIENCE

INFORMATION TECHNOLOGY

UPPSALA UNIVERSITY

UPPSALA

SWEDEN

Dissertation for the degree of Licentiate of Philosophy in Computing Science at Uppsala University 2001

with Application to Energy Control

Per Carlsson

Per.Carlsson@cs.lth.se

Department of Computing Science Information Technology

Uppsala University Box 337 SE-751 05 Uppsala

Sweden

http://www.it.uu.se/

c

Per Carlsson 2001 ISSN 1404-5117

Printed by KFS i Lund AB, Sweden Printing sponsored by EnerSearch AB

The energy markets of todayare markets with ratherfew activeparticipants.

Theparticipantsare,withfewexceptions,largeproducersanddistributors. The

market mechanisms that are used are constructedwith this kind of amarket

situation in mind. With an automatic or semiautomatic approach, the mar-

ket mechanismwould be ableto incorporate a largernumber of participants.

Smaller producers,and evenconsumers, couldtakeanactivepartin themar-

ket. Thegainisinmoreecientmarkets,andduetosmalleructuations in

demand betterresourceusagefromanenvironmentalperspective.

Theenergy markets of the Nordiccountries (as wellasmany others)were

deregulated during the last few years. The change has been radical and the

situation is still rather new. We believe that the market can be made more

ecientwiththehelp ofthedynamicsofthesmallactors.

Theidealisedworldoftheory(ofeconomics)oftenreliesonassumptionssuch

as continuous demand and supply curves. These assumptions are useful, and

they do not introduce problems in the powermarket situation of today, with

relatively few, large, participants. When consumers and small producers are

introduced on themarket,the situation is dierent. Then it is adrawbackif

themarketmechanismcannothandlediscontinuoussupplyanddemand.

The growth in accessibility to computational powerand data communica-

tionsthatwehaveexperiencedinthelastyears(andareexperiencing)couldbe

utilised whenconstructingmechanismsfortheenergymarketsoftomorrow.

Inthis thesiswesuggestanewmarketmechanism,ConFAst, thatutilises

the technological progressto make it possible to incorporate a large number

of active participants on the market. The mechanism does not rely on the

assumptions above. Thegainis amore ecient market with lessuctuations

in demandovertheday.

Tomakethis possiblethere is aneedfor ecient algorithms, in particular

this mechanismrelies on anecientaggregationalgorithm. An algorithm for

aggregationofobjectivefunctionsispartofthisthesis. Thealgorithmhandles

maximisationwith non-concave,evennoisy,objectivefunctions. Experimental

results show that the approach, in practically relevant cases, is signicantly

faster thanthestandardalgorithm.

1 Introduction 1

1.1 TheProject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 TheProblem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.1 ConFAst, anApproach to Ecient and yet Fast Power ControlMarkets . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.2 ResourceAllocationwithNoisyFunctions . . . . . . . . . 3

1.3.3 Implementation. . . . . . . . . . . . . . . . . . . . . . . . 4

2 An ApproachtoEcientand yetFastPowerControl Markets 7 2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 TheStrainsonMarketMechanismsinExtreme Situations 7 2.1.2 TheDynamicsoftheSmallActors . . . . . . . . . . . . . 8

2.1.3 TheTechnologicalProgress . . . . . . . . . . . . . . . . . 10

2.1.4 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2 MarketMechanismsTheoryandPractice . . . . . . . . . . . . 10

2.2.1 IncreasingCostandDecreasingUtility . . . . . . . . . . . 11

2.2.2 Information . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.3 TheMarket ofTodayandTomorrow . . . . . . . . . . . . 13

2.3 ConFAst,A NewMarketMechanism . . . . . . . . . . . . . . . 14

2.3.1 ConcaveandNon-ConcaveUtility . . . . . . . . . . . . . 15

2.3.2 OutlineoftheConFAstMechanism . . . . . . . . . . . . 16

2.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Resource Allocationwith NoisyFunctions 29 3.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.1.1 ResourceAllocation . . . . . . . . . . . . . . . . . . . . . 29

3.1.2 AggregatingObjectiveFunctions . . . . . . . . . . . . . . 30

3.1.3 Anoteonthecomplexityofthegeneralproblem . . . . . 32

3.2 MainResult . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.3 ExperimentalResults. . . . . . . . . . . . . . . . . . . . . . . . . 36

3.3.1 Tree-structuredAggregation. . . . . . . . . . . . . . . . . 36

3.3.2 TheExperiments . . . . . . . . . . . . . . . . . . . . . . . 37

3.3.3 SummaryoftheExperiments . . . . . . . . . . . . . . . . 43

3.4 TechnicalDetails . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.4.1 AlgorithmWithout NoiseFiltering . . . . . . . . . . . . . 44

3.4.2 AlgorithmBasedonNoiseFiltering . . . . . . . . . . . . 46

3.4.3 ConstructingSegmentsBasedonHullFunctions . . . . . 48

3.4.4 FindingallPossibleCandidatesinSegmentsWithConvex andConcaveHullFunctions. . . . . . . . . . . . . . . . . 50

3.4.5 ProofofTheorem3.4.1 . . . . . . . . . . . . . . . . . . . 63

3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Bibliography 65

Introduction

1.1 The Project

The thesis work started in a project, DiMALLOC 1

, that was nancedby the

Swedish National Board for Industrial and Technical Development, NUTEK.

As this project ended, the work hasbeencompleted within the IT in Energy

ProgrammeoftheSwedishResearchInstituteforInformationTechnology,SITI.

ThisprogrammeisoneoffourprogrammesofSITI.ThefocusofITinEnergyis

on(i)localcommunicationsoverthepowerlines,(ii)vulnerabilityandsecurity

in electronic commerce (with focus on power line communication), and (iii)

electronicenergymarkets(withfocusoneconomicalandcomputationalaspects

ofelectronicpowermarkets). Thethesispresentssomeoftheresultsinthelast

area.

A Swedish research company, EnerSearch AB, plays a central partin the

programme together with Blekinge Institute of Technology, Lund University,

andUppsalaUniversity.

The aim of EnerSearch AB  that is owned by interests in the energy

sector is to initiate and coordinate research in areasthat in dierent ways

are related to the energy and business sectors. As in the case of the IT in

EnergyProgramme,theresearchisconductedincloserelationwithuniversities.

Current owners of EnerSearch AB are: ABB Automation Products, ECN -

Netherlands Energy Research Foundation, EDP - Electricidade de Portugal,

Iberdrola,IBMUtilityServices,E.ON-Energie,andSydkraft.

1.2 The Problem

Powermarketsaroundtheworldarederegulating. Howtodesignecientpower

marketsisanissuethatisdiscussedamongauthorities,energyutilities,andre-

searchers. Itisaninterdisciplinaryproblem,withmainaspectsineconomicsas

1

DistributedMarketalgorithmsandresourceALLOCation

wellastechnicalelds. Theproblemhasinterestingcomputationalaspectstoo,

andduetothegrowingaccesstocomputationalpoweranddatacommunica-

tioncapabilitiesthe computerscienceapproachisableto opennewmarket

possibilities.

A wayto enhancetheeciencyofpowermarkets istoincorporate alarger

numberof participantsonthe activemarket. Today theend consumeris not

presentonthemarket(withfewexceptions)and,asaconsequence,themarket

has to rely onmore orless accurate predictionsof consumption patterns and

volumes. Thepricesat e.g. theNordic spot market,Nordpool 2

, variessigni-

cantlyfromhourto hour,duetouctuations indemand. Ontheotherhanda

largeshareoftheloadshassuchpropertiesthatthetimeproleoftheirdemand

could be utilised to even outasignicantpartof the uctuations (if they are

incorporatedintothe active powermarket), and this canbedonewith no (or

minor)lossin comfortoreciency fortheconsumer.

The gains of introducing the small participants on the active market are

concerningmostoftheaspectsthatwecanthinkof;whentheyareintroduced

supportedbyapropermarketmechanismtheuctuationsindemand,and

hence in price, becomes smaller, the planning of production will be based on

moreaccuratenumbers,thereislessstressonthetransmissionanddistribution

gridonhighpeakhours,wegetabetterresourceusagefromanenvironmental

perspective,andsoon;awinwinsituation.

Whenconsumersandsmallscaleproducersareintroducedontheactivemar-

ket,specialcarehastobetakenconcerningdiscontinuoussupplyanddemand.

Market mechanisms oftoday, based onstandardtheory of economics, assume

that supplyand demand iscontinuous. Due to this assumptionit ishardand

risk prone to participate on the market for any actor with a non-continuous

curve. Howtoconstructamechanismthat handlesnon-continuoussupplyand

demandis nontrivial.

Thepossibilityofalargescaleintroductionoftheconsumptionsideonthe

activemarketisaconsequenceofthedevelopmentincomputationalpowerand

data communicationsthat hastakenplace during the last years. Theaccessi-

bility to the Internet is growingand can be utilised, great eorts are put on

developmentofinexpensivelocalareanetworks,andmoreandmoreequipment

isenhancedwith(whatisoftenreferredas)intelligence. Inourhomeswehave

(orsoonwearehaving)computationalcapacityinwashingmachinesandrefrig-

eratorsaswellasheating/coolingsystems. Ifitispossibletocommunicatewith

the equipment,there is apotentialto introduceit onthemarket,represented

bysmallpiecesofsoftware. Consumers,otherthanlargeones,willneverpartic-

ipateonthemarketiftheyhavetocalculatetheirdemandbyhandandexpress

itasmathematicalfunctions. Whenincorporatingconsumersonthemarketwe

think ofthem asrepresentedbysoftwareagents,i.e. small pieces ofcomputer

software,thatgivetheirinputto thesystem.

2

1.3 Contributions

This thesisconsists of twopapersandan implementation ofthe algorithmin-

troducedin oneofthem;therstpaperintroducesanovelmarketmechanism,

ConFAst, a (semi) automatic mechanism with properties that meet the de-

mands that we focused on in the previous section. The second paper is an

articlewhichintroducesanalgorithm foraggregationof averygeneralclassof

(separable)objectivefunctions.

Thecharacterofthetwopapersdierssignicantly. Itisnotonlythetopic,

but the perspectivetoo. The rstone, dealingwithpowercontrolmarkets, is

on aratherhigh level, whilethesecond one,focusingon theconstructionand

behaviourofanalgorithm,israthertechnical.

Thethird partof the thesisis an implementation of the aggregationalgo-

rithm,animplementationthatwasproducedforcomparisonbetweenouralgo-

rithm and the standard algorithm for the problem. The work on the market

mechanismincludedimplementationtoo,but itis notin aformthat is suited

forpublication.

1.3.1 ConFAst, an Approach to Ecient and yet Fast

Power Control Markets

The ConFAst mechanism is designed with large computational markets in

mind. It is well suitednotonly formarketswhere thenumber ofparticipants

countsintensorhundreds,butitscalestohundredsofthousandsormore,both

producers,distributors,andconsumers. Themarketcomputationisdistributed

overcomputersinthenetwork,i.e. itisnotnecessarytogatherallinformation

needed at acentral spot. It handles non-continuous supplyand demand, and

henceitreducestheriskofparticipatingwithsuchcurvessignicantly.

Thepaperhasbeenpresented ataseminaronInformation andCommuni-

cation Technology in the Energy Sector, held by the Nordic Energy Research

Scientic Program, in Trondheim, Norway, March 8  9, 2001. A conference

versionofthepaperisundersubmission.

TheConFAst mechanismdepends onthe aggregationof utility functions

functionsthatexpresstherelativevalueofconsumptionsbundles,i.e. akind

ofobjectivefunctions. Computationallythisisthemostdemandingpartofthe

workanditisessentialthat theaggregationisperformedinanecientway.

1.3.2 Resource Allocation with Noisy Functions

Alotofeortshavebeenputontheaggregationofseparableconcaveobjective

functions 3

. Thegeneralcase,whennothingcanbesaidabouttheobjectives,is

hard. Apracticallyrelevantclassoffunctionsisseparablenon-concaveobjective

functions. Standard algorithms for aggregation of this class of functions are

basically constructed onapairwise aggregationof functions and abrute force

3

Concaveinthemaximisationversionoftheproblem,convexobjectivesintheminimisation

testing of allcombinations of thetwofunctions. Moreecientaggregation of

separableobjectivefunctions thatmightnotbeconcaveisnontrivial.

Thearticlepresentsanalgorithmthat focuses onthefactthat whenaggre-

gating non-concavefunctions the resulting function very fast becomes almost

concave,but noisy. The algorithm utilisesregularitiesin the aggregatedfunc-

tionswhenthefunctionisalmostconcaveorisclosetoanothersmoothcurve

sothat thesearchspacecanbeprunedwhenaggregatingthefunctions.

The algorithmis generic, but developed withresourceallocationand com-

putationalmarketsin mind. Thealgorithmcanbeusedasasubroutineofthe

ConFAstmarketmechanism.

Thearticleisrathertechnical. For thereadernotinterestedinalltechnical

detailsitisprobablysucienttoreadtheintroductionthroughtheexperimental

results, and skip the technical description of the algorithm. Hopefully this

partgivesenoughto grasp themain ideasbehind thealgorithm, and givesan

impressionoftheperformance.

Thearticleissubmittedforjournal publication.

1.3.3 Implementation

Implementationsandtestsof boththemarket mechanismandtheaggregation

algorithmispartofthethesiswork.

TheJavaclasses(andtestdata)neededtorunsometestsontheaggregation

algorithm are available on the Internet and on the CD-ROM included in the

thesis. Theimplementationisdonetotestandcomparetheperformanceofthe

algorithmondierentinput(thecomparisonisdoneonthestandardalgorithm

foraggregationof non-concaveobjectivefunctions). Theresults,that arepart

of the article, showthat the algorithm indeed is competitive. This holds for

bothadversarydataand practicallyrelevantdata. Itshouldbenotedthat the

standardalgorithmthat weuseforcomparisoncanbeexpected tobecloseto

optimalduetoitssimplicity. Muchcouldprobablybedonetoimprovethisrst

testimplementationofouralgorithm,asitisrathercomplicated.

Thethreepartsofthethesisare,insomeways,ratherdisparate. Atthesame

timetheyareheld togetherbytheaim todevelopalgorithmsand mechanisms

for resourceallocation and markets. As saidearlier, theapplication areathat

wehavein mindisthepowermarkets.

Acknowledgements

The thesis work has been partof twodierent projects. It startedin theDi-

Malloc project, sponsored by the Swedish National Board for Industrial and

Technical Development, NUTEK 4

. The work wasnished within the Swedish

National Programme on IT in Energy, aprogramme of the Swedish Research

InstituteforInformation Technology,SITI 5

.

ThemasterobjectiveoftheprogrammeITinEnergyistodemonstratehow

informationtechnologycanimprovetheeciencyin futureenergysystems.

EnerSearch AB 6

, a Swedish research company owned by interests in the

energysector,hasplayedacentralpartintheprojectsascoordinatingpartner

ofUppsalaUniversityandothers.

IwanttothankmyadvisorsFredrikYggeandArneAnderssonforsupport,

ideas,discussions,andmuchmore,andThoreHusfeldt,thathasbeenavaluable

local support in Lund. Both the articles of this thesis are joint work with

AnderssonandYgge. FurthermoreIwanttothankmyfriendsattheComputer

Science Department,Lund University,whereI livemyeveryday(working)life.

EventhoughIamnotatmyhomedepartmentinUppsalatooften,Iappreciate

beingpartoftheteam.

Finally, Iwantto thankmyfamily.

4

Duetorestructuring,the partofNUTEKthatsponsoredtheprojectisnowpartofthe

SwedishAgencyforInnovationSystems,Vinnova,http://www.vinnova.se.

5

http://www.siti.se

6

An Approach to Ecient and

yet Fast Power Control

Markets

Powermarketsaroundtheworldarederegulating. Theissueofhowtoproperly

designpowermarketsisadelicateone,intensivelydiscussedamongauthorities,

researchers,energyutilities,etc. Inthispaperwefocusontwomajoraspectsof

powermarket design: (i)managementofdiscontinuousdemand/supplycurves,

and(ii)computationaldesignofmarketsofhugesizewithrelativelyshorttime

frames.

Weintroduce amechanism, ConFAst, for highly dynamic powermarkets

thatallowsforhugenumbersofactiveparticipants(alsoconsumers)inthemar-

ket,eveniftheyhavediscontinuousdemand/supplycurves. Thecomputational

eciencyoftheapproachenablesusagealsoinmarketswithshorttimeframes.

Wearguethat ConFAst signicantlycanimprovetheeciency ofenergy

systems.

2.1 Introduction

2.1.1 The Strains on Market Mechanisms inExtreme Sit-

uations

Around January 24, 2000, the weather in all of Sweden was so cold that the

demand for electricity wasextremelyhigh andthe pressureon the wholepower

systemwasequallyhard. Asaconsequencethepriceonthespotmarketroseto

levels where energy utilities with no or small production capacity of their own

riskedtolose allof their annualprotduring onesingleday.

Thiswasthehardesttestofthenewmarket systemsofar,sincetherecent

deregulationoftheenergymarketsofFinland,Norway,andSweden. However,

in the times of deregulation the energy system is pushed closer to its limits

andwewillsee moreof thesetypesofsituations in thefuture. Skeptics ofthe

deregulationseeanewdawnin thisphenomenon.

Inanextremesituationlikethisone,newproblemsariseorcomeintofocus.

Onesuch problem is how torelease thepressureon thesystem in awaythat

aects the society and theeconomyas littleaspossiblewhen the demand for

energyistoohigh.

Today,inSweden,thereisanongoingdiscussiononhowtobettercopewith

this type of situations. One of the main issuesis how to properly utilise the

dynamics of the energy consumers. This questionis an aspect of the issue of

howtoconstructmoreecientpowermarkets. Theuctuationsindemandand

pricesonthemarketsoftodayaresignicantevenundernormalconditions,see

Figure2.1.

Figure2.1: Sincedemandvariesverymuchfromhourtohourthemarketprices

ofaderegulatedmarketvarysignicantlyaswell. Thesegraphsshowthemarket

priceoftheNordicspotmarket(left,peakprice55Euro/MWh)andtheSwedish

regulatingmarket(right,peakpricecloseto190Euro/MWh)aweekinJanuary

2001.

2.1.2 The Dynamics of the Small Actors

Today theloadside of themarket, theconsumerdemand, is essentiallytaken

as exogenous and the suppliers and distributors have to relyon moreor less

accuratepredictions. Thisarticleexploitspossibilitiesfortheloadside,aswell

assmallscaleproduction,to takeanactivepartin themarket.

A large portion of the loads within the power grid are controllable (in a

controlsystemsense). Thetermcontrollableloadcanbeillustratedbyexamples

fromeverydaylife: A lightbulboraTVsetisnotacontrollableloadwhen

the userswitches iton it is supposed to turn on instantly, and theother way

aroundwhenitisswitchedo. Ontheotherhandtheheatersofanapartment,

or any equipment that supply a comfortable indoor temperature, are typical