Disturbance Aspects of Iterative Learning Control

Mikael Norrlof, Svante Gunnarsson

Department of Electrical Engineering

Linkoping University, SE-581 83 Linkoping, Sweden

www: http://www.control.isy.liu.se

email: mino@isy.liu.se, svante@isy.liu.se

2000-06-19

REG

LERTEKNIK

AUTO

_{MATIC CONTR}

OL

LINKÖPING

Submitted(revisedversion)toEngineeringApplications ofArticialIntelligence. T

echni-calreportsfromtheAutomaticControlgroupinLinkopingareavailablebyanonymousftp

at the address 130.236.20.24 (ftp.control.isy.liu.se/pu b/Re por ts/). This report

Control

M. Norrlof and S. Gunnarsson

Department of Electrical Engineering

Linkoping University

SE-58183 Linkoping, Sweden

Email: mino@isy.liu.se, svante@isy.liu.se

Fax: +46-13-282622

Abstract

Disturbance aspects of Iterative Learning Control(ILC) are considered. By

using a linear framework it is possible to investigate the in uence of the

dis-turbances in thefrequency domain. The eects of thedesign lters in the ILC

algorithm on the disturbance properties can then be analyzed. The analysis is

supported bysimulationsand experiments.

The word learning has many interpretations in engineering in general and in control

in particular. In the control area learning in general means a procedure where a

representation of a dynamic system or a strategy for controlling a dynamic system

isadapted insome way inorder to improvethe overall performance.

In Iterative Learning Control (ILC) the assumption is that the controlsystem is

sup-posedtocarry outthesame operationrepeatedly. This isacommonsituationinmany

robotics applications where a robotis supposed to dothe same action, e.g. a welding

orcuttingoperation,overandoveragain. Applicationsinotherpartsofmanufacturing

can alsobefound, butindustrialrobots havebeen the major applicationareafor ILC.

Byusing experiencefrom one cycle the idea is toadjust the input signal in an

appro-priateway such that the performance of the system inthe next cycle is improved. By

measuringe.g the path errorof arobotmovementin onecycle the jointtorquesinthe

next cycle are adjusted such that the path error is reduced. An important dierence

between ILC and e.g. adaptive control or neural network modeling is that the whole

inputsignal isadapted inILC whilethe parameters inaparameterized representation

of the controller or the model of the system is adapted in the other cases. ILC is a

feed-forward (open loop) control methodwhich means that the whole input sequence

isprecomputed beforethe cycle begins.

The standard assumption in ILC is that each cycle is carried out during anite time

intervalt=0;::: ;N. Sincethe implementationisdoneusingacomputer theproblem

z k (t)=T r (q)r(t)+T u (q)u k (t)+T d (q)d k (t)+T n (q)n k (t) (1) wherez k (t);r(t)andu k

(t)denotethesystemoutputsignal,thereferencesignalandthe

ILCinputsignalrespectively. Thevariablesd

k

(t)andn

k

(t)denoteloaddisturbanceand

measurementdisturbancerespectively. Theaiminthepaperistoanalyzetheeects of

thedisturbancesontheoverall systemperformance. A furtherstepwouldbetodesign

the ILC algorithmusing a description of the statistical properties of the disturbance

signals. The subscript k denotes iteration(cycle) number. The reference signal r(t) is

the same in all iterations, which means that the whole sequence is known before the

rst iterationbegins. All the othersignalswillchange fromiterationtoiteration. The

variablesT r (q);T u (q);T d (q)and T n

(q)denotestable discretetime lters. It isofcourse

a restriction toconne the problem tolinear systems, but on the other hand this will

makeit possible toobtainfrequency domain insight intothe disturbanceproperties of

ILC. Inmany situationssuch insightis useful from anengineering viewpoint.

The formulation in equation (1) is taken from (Norrlof 1998), and it covers a wide

class of situationsranging froman open loopcontrolproblem toa closed loopsystem

operating under both feed-back and feed-forward control. In (Arimoto et al. 1984),

which is often referred to as one of the original papers in the eld of ILC, the ILC

algorithm was used to generate the input to the system directly. In the framework

here this corresponds to

T u =G T r =0 T n =0 T d =0 (2)

where G isthe transfer function of the system to be controlled. In (2) and the sequel

theargumentoftheinvolvedtransferfunctionswillsometimesomittedforconvenience.

k

system. Hence ILCis used asacomplementtothe conventional robotcontrol system.

The structure depicted in Figure1 corresponds to the situation

T r = (F +F f )G 1+FG T u = FG 1+FG T d = G 1+FG T n = FG 1+FG (3) where G;F and F f

are the transfer functions of the system to be controlled and the

feed-back andfeed-forwardregulators respectively. A slightmodicationofthesystem

structure shown in Figure 1 is to let the ILC input signal be used as a feed-forward

signal added to the control signal generated by the feed-back and feed-forward parts

of the controller. This just corresponds toa redenition of the transfer functionT

u (q)

inequation (1).

ThefundamentalprobleminILCis todesign anupdate algorithmforthe inputsignal

u

k

(t)such that the error

e

k

(t)=r(t) z

k

(t) (4)

isreduced insome appropriate sense as the iterationsproceed.

A generalformulafor updatingthe ILC input signal is given by

u k+1 (t)=Q(q)(u k (t)+L(q)e k (t) (5) wheree k

(t) denotes the measured error signal i.e.

 e k (t)=r(t) y k (t)=e k (t) n k (t) (6)

andQ andL arelinear, possibly non-causal,lters. Since u

k+1 (t)iscomputed o-line, with both e k (t) and u k

(t) given over the whole time interval, non-causal lteringcan

havebeenpublished. Amongearliercontributionsonends(Craig1988),(Hideg1992),

(Moore 1993) and (Horowitz 1993), while recent surveys of the area of ILC are given

in (Moore 1998) and (Bien and Xu 1998). The results presented in the paper are

extensions of the results presented in (Gunnarsson and Norrlof 1997) and (Norrlof

1998). The main contribution in this paper is the frequency domain analysis of the

disturbance properties of ILC.

The paper is organized as follows. In Section 2 the ILC algorithm properties in

gen-eral are discussed, while in Sections 3 and 4 the eects of load and measurement

disturbances are investigated. Section 5 then contains simulations that supports the

theoreticalanalysis. Experimentscarriedoutonanindustrialrobotarethenpresented

inSection 6. Finally some conclusions are given inSection 7.

2 Algorithm Properties

The question is now the following. Given the system dened by equation (1), how

shallthe ltersQ andLinthe update equation(5)bechosensuchthat the errore

k (t)

decreases in anappropriate way?

The main issues when choosing Q and L are convergence, robustness and in uence of

disturbances. In this paper the convergence and robustness issues will be mentioned

brie ywhilethemainattention isondisturbanceaspects. Whiletheearliestpaperson

ILC considered situations where almost no knowledge of the system to be controlled

was available the use of some kind of a priori modelin the design of ILC algorithms

choos-methodsforrobustcontroldesignareappliedandtheltersaredesigned togivea

con-vergentILC algorithmdespiteuncertainties inthe process model. Ine.g. (Gorinevsky

et al. 1995) the ILC input signal is formulated as an optimization problem, using an

a priori model, resultingin a time domain updating equation for the input signal. In

(Norrlof 1998) it is shown how system identicationcan be used tobuild a low order

linearmodelof anindustrialrobotwhichthenisused fordesignofanappropriateILC

algorithm,which is alsotestedin experiments.

2.1 Error equation

In this section the aim is to derive the fundamental error equation that describes

how the error evolves as the iterations proceed. The derivation of the error updating

equation isformulated as the following lemma.

Lemma 2.1 Let e~be dened as

~

e=(1 T

r

)r (7)

i.e. thedisturbancefree error signalobtainedwithout any ILC input,i.e. whenu

0 0.

Thenthe error signal is updated according to

e k+1 =Q(1 LT u )e k +(1 Q)~e+T d (Qd k d k+1 )+T n (Qn k n k+1 )+QLT u n k (8)

Proof 2.1 Combining equations (4), (1), and (5) gives

e k+1 =e~ T u Qu k T u QLe k +T u QLn k T d d k+1 T n n k+1 (9)

T u u k =e k ~ e+T d d k +T n n k (10) gives e k+1 =Q(1 LT u )e k +(1 Q)~e+T d (Qd k d k+1 )+T n (Qn k n k+1 )+QLT u n k (11)

which is the desired result.

The dierence equation (8) has three types of driving terms. The rst term is e~i.e.

the error obtained when there is no ILC present. The other driving terms come from

the load and measurement disturbances respectively, and they enter the equation in

slightlydierentways. Thelongruneects ontheerrorofthesetermswillbeanalyzed

in Sections 3 and 4 below. A result corresponding to equation (8) is presented in

(Panzieri and Ulivi 1995) forthe open loop case and for load disturbancesonly.

2.2 Convergence condition

The convergence propertiesare determinedby the homogeneouspart of the dierence

equation (8)and a suÆcient condition for convergence is that

jQ(e i!T )jj1 L(e i!T )T u (e i!T )j<1 8 ! (12)

which means that

j1 L(e i!T )T u (e i!T )j< 1 jQ(e i!T )j 8 ! (13)

This criterion can be viewed in a Nyquist diagram, and with Q = 1 the condition is

that the Nyquistcurve L(e i!T

)T

u (e

i!T

frequency range,thestabilityregioncanbeincreased. Theprize paidfortheincreased

stability regionisthatthe errorcan notbe eliminatedcompletely. This willbefurther

discussed below.

2.3 Asymptotic properties

Togetsomeinitialinsightintotheasymptoticpropertiesofthealgorithmthe loadand

measurementdisturbancesareinitiallyneglected. Providedthattheiterativeprocedure

converges the asymptoticerror signal, using equation(8), becomes

E 1 (e i!T )= 1 Q(e i!T ) 1 Q(e i!T )(1 L(e i!T )T u (e i!T )) ~ E(e i!T ) (14) where E 1 and ~

E denote the Fourier tranforms of the asymptotic and nal error

re-spectively. It isclearly seen that by usingQ6=1itis impossibletoeliminatethe error

completely,but as willbeseen later there are otheradvantages with this choice.

Typ-icallyQ is chosen asa lowpass lter with unit gain for lowfrequencies which implies

that the low frequency part of e~can bereduced substantially.

3 Load Disturbances

A number of observations concerning the handling of load disturbances can be made

using equation(8). Consider rst the case Q1 and n

k

0, which means that there

are nomeasurement disturbances. This implies the update equation

e k+1 =(1 LT u )e k +T d (d k d k+1 ) (15)

tions. Ifa disturbance isof repetitivenature inthe sense that the disturbance signals

d

k

(t)=d

k+1

(t) for allk, the contributiontothe error dierenceequationis zero. This

assumption islikelyfor the load disturbance where forexample load disturbances due

togravitationalforces can beexpected to berather similarduring dierent iterations.

Consider also the situation when Q 6=1, there are no measurement disturbances and

d

k

(t)=d(t) 8 k. This corresponds tothe errordierence equation

e k+1 =Q(1 LT u )e k +(1 Q)~e+T d (Q 1)d (16)

Theloaddisturbancewillactasadrivingtermsimilartotheinitialerrore.~ Inatypical

case T

d

, and the lter Q are both of lowpass type, which means that 1 Q is of high

pass type. Multiplying these two willgive aband pass lter with a pass band around

the bandwidth of the closed loopsystem. The possibilitiestoreduce the eects of the

load disturbance hence depends on the relationship between the frequency content of

the load disturbance and the cut-o frequency of the lter Q.

Usingequation (16) the asymptotic error,similar to(14), becomes

E 1 (e i!T )= 1 Q(e i!T ) 1 Q(e i!T )(1 L(e i!T )T u (e i!T )) ( ~ E(e i!T ) T d (e i!T )D(e i!T )) (17)

whereDdenotes theFouriertransformoftheloaddisturbance. Thisexpression clearly

illustratestheobservationabovethatthefrequencycontentofd,thecut-ofrequencies

of T

d

and 1 Q respectively will determine how well the ILC algorithm handles load

Measurementdisturbancesaremorelikelytobeofrandomcharacterwhichmeansthat

n

k+1

(t)6=n

k

(t)ingeneral. Therewillhencealways bedrivingtermsonthe righthand

side of equation (8) that prevent e

k

from tending to zero. In order to concentrate on

the in uenceof the measurementdisturbanceit ishereassumed that r0and d0.

This impliesthat ~e0,and using equation (1) this givesthat

e k = T u u k T n n k (18)

The second term on the right hand side is caused by the conventional feed-back, and

since this term is always present the interest from an ILC viewpoint is on the rst

term. Equations (5) and (1) together with Figure 1give

u k+1 =Q(1 LT u )u k QLSn k (19)

whereS is the transfer function

S = 1

1+FG

(20)

Assume then that n

k

is a stationary stochastic process with spectral density 

n (!),

and let, in stationarity, 

u

(!) denote the spectral density of u

k . Equation (19) then gives  u (e i!T )= jQLS j 2 1 jQ(1 LT u )j 2  n (e i!T ) (21)

Hence the spectral density of error inequation (18) becomes

 e (e i!T )= jT u QLSj 2 1 jQ(1 LT u )j 2  n (e i!T )+jT n j 2  n (e i!T ) (22)

It is clear from equation (22) that the magnitude of the error spectral density will be

very large for frequencies where j Q(1 LT

u

L= ^ T u (1 H B ) where H B

is a high-pass lter determining the desired convergence

properties ofthe ILCalgorithmand ^

T

u

isamodelofthe transferfunctionT

u

. The left

plot in Figure 2 shows the principal behavior of jQ(1 LT

u

)j when Q = 1 while the

right plot shows the same quantity when Q is a low pass lter. It is clear that when

the Q lter isintroducedthe high frequency disturbance impactcan bereduced.

It is seen that by introducingQ it is possible toreduce the measurement disturbance

impact on the ILC algorithm, but with the price that the error in the trajectory

following will not tend to zero. The demand of trajectory following and disturbance

reduction willinthis sense be contradictory.

5 Simulations

To illustrate the disturbance properties a simulation example will be studied. The

exampleisasimplieddescriptionofasinglerobotjointmodeledasadoubleintegrator.

The discretetime representation is given by

G(z)= T 2 (z+1) 2J(z 1) 2 (23)

where J = 0:0094 is the moment of inertia. The system is controlled by the discrete

time PD-regulator F(z)=K P + K D T (z 1) z (24) where K P = 12:7 and K D

= 0:4. The feed-forward lter is a double dierentiation

represented by F f (z)= J  (z 1) 2 T 2 z 2 (25)

where J is the assumed moment of inertia. The model is simulated using 1 kHz

samplingfrequency. The appliedreference trajectory shown in Figure3.

The lter L ischosen in a heuristic, but modelbased, way by choosing

L= ^ T 1 u (1 H B ) (26) where ^ T u

denotes a nominalclosed looptransfer function obtained using the modelof

the open loopsystem. It is assumed that there is a 30% error inJ 

, i.e. the moment

of inertia in the model of the system. The lter H

B

is a Butterworth high pass lter

(hereofsecondorder)forwhichthegain tendstoonefor highfrequencies. ChoosingL

according tothis designrule, with cut-o frequency of the highpass lter equalto0:4

times the Nyquist frequency, gives the Nyquist curve depicted in Figure4, which also

shows T

u

for comparison. Figure 4 also shows the right hand side of the convergence

criterion(13) for Q=1 and Q chosen as alowpass lter respectively.

The rst goal isto investigatehowthe loaddisturbance in uences the ILC algorithm.

TheleftpartofFigure5showsthe appliedloaddisturbanceasafunctionoftime. This

disturbance signal isapplied atthe same time every cycle as arepetitive disturbance.

TherightpartofFigure5showsthemaximumerrorforeachiterationwithandwithout

the Q-lter. Figure 6shows the spectrumof E

k

as afunction of iteration. Clearlythe

introductionofalterQreduces theconvergence speedoftheILCalgorithmandthere

willalsonot be convergence tozero of the error. Already after one iterationwhenthe

lterQisusedtheenergyintheerrorhas almostreached itsnal valueandalsointhe

spectrumit is possible tosee that thereis not much change after the rst iteration.

The next step is to introduce a measurement disturbance, n

k

, and this is chosen as

discretetime whitenoise process with standard deviation3:2
10 4

. The timedomain

iteration. The plotof the energy clearly illustratesthat the use of Qreduces the error

energy caused by the measurement disturbance. The fact that the maximum error

behaves inasimilarfashioninthetwocasesisexplainedbythatthe maximumerroris

of low frequency character and hence not aected by the choice of Q. In Figure 8the

spectrum of the position error signal r z

k

is shown with and without Q lter. The

reduction of the high frequency part of the error spectrumis clearly shown.

6 Experiments

The results presented in the previous sections will now be illustrated using an ABB

IRB1400industrialrobot. Foramorethorough descriptionofthe technicalpartofthe

experimental setup see (Norrlof 1998).

Inthis example ILCis appliedtothreejoints. The robothas atotal of6-DOF but for

the three wrist jointsILC is not applied. Each of the joints are modeled as a transfer

functiondescriptionfromtheILCcontrolinputtothemeasured motorpositiononthe

robot. The feedback controller, implemented by ABB, is working in parallel with the

ILC and since the controller is working well, the closed loop from reference angular

position to measured angular can be described using a low order linear discrete time

model. The models are obtained by applying system identication to the closed loop

system.

In Figure 9 the program used in the experiment is shown together with the resulting

trajectory on the arm-side of the robot. Note that the program in Figure 9 is not

pro-producesa straight lineon the arm-sideof the robot. The linestartsfrom the current

position, not explicitly stated, and ends in p2. The speed along the path is in this

case programmedto be 100 mm/s. The lastparameter z1 indicatesthat the pointp2

is a zone point. this means that the robot will pass in a neighborhood of the point

with a distance not more than 1 mm. This can also be seen in Figure 9. The moveL

instructionisasimpliedversionofthecorrespondinginstructionMoveLintheRAPID

programminglanguageThe actualposition of p1 inthe base coordinatesystem isx =

1300 mm, y =100 mm, and z= 660 mm.

The aim of the experiment is to illustrate the importanceof the Q-lter with respect

to measurement disturbances. Since experiments on real systems always will contain

measurement disturbances it will not be possible to compare the performance with a

situation without measurement disturbances. The aim will therefore be to illustrate

the positivein uence of the introduction of the lter Q. In the experiments the lter

Lischosen asL(z)=0:9z 4

i.e. ascalingtimesatime shiftfoursamplesforward. This

hence corresponds to a non-causal ltering. This choice gives moderate performance

requirements. The ILC algorithm is run during ten iterations with Q = 1 and Q

chosen as a low pass lter respectively. Figure 8 illustrates the performance for the

threemotorsinvolved. ChoosingQasalowpassltertheerrorsettlesatasteadylevel

while the choice Q = 1 gives large uctuations in the error. It should be noted that

the conditions in the experiments diers from the simulation since the undisturbed

positionerror z

k

isnot available. Instead the measured position errorr y

k

has tobe

Disturbance aspects of Iterative Learning Control algorithm has been studied using

frequency domainmethods. It has been how the choice of design lters, in particular

thelterQ,aectsthe wayloadand measurementdisturbancesaect theperformance

of ILC algorithms. The results have been supperted by simulations and experiments

ona real robot.

8 Acknowledgments

ThisworkwassupportedbyCENIITatLinkopingUniversityandABBRoboticswithin

ISIS atLinkoping University.

References

Arimoto,S., S.Kawamura andF.Miyazaki(1984).\BetteringOperationofRobotsby

Learning". Journal of Robotic Systemspp. 123{140.

Bien,Z.and J.X. Xu(1998). Iterative LearningControl. Analysis, Design, Integration

and Applications. KluwerAcademic Publishers. Dordrecht, The Netherlands.

Craig, J. (1988). Adaptive Control of Mechanical Manipulators. Addison-Wesley

Pub-lishing Company.

de Roover, D. (1996). \Synthesis of a Robust Iterative Learning Controller Using an

H

1

approach". In: Proc. of the 35th IEEE Conference on Decision and Control.

of feedforward dependence on the task parameters: Experiments in direct-drive

manipulatortracking". In: Proc. ACC 1995. Seattle,Washington. pp. 883{887.

Gunnarsson, S. and M. Norrlof (1997). \Some Experiences of the Use of Iterative

Learning Control for Performance Improvement in Robot Control Systems". In:

IFAC Symposium in Robot Control 1997. Nantes, France. pp. 379{383.

Hideg, L.(1992). Stability of LearningControlSystems. PhDthesis. Oakland

Univer-sity. Rochester, Michigan.

Horowitz, R. (1993). \Learning Control of Robot Manipulators". ASME Journal of

DynamicSystems, Measurement, and Control115, 403{411.

Moore, K.L. (1993). Iterative Learning Control for Deterministic Systems. Springer

Verlag.

Moore, K.L. (1998). \Iterative Learning Control: An Expository Overview". Applied

and Computational Controls, Signal Processing and Circuits.

Norrlof,M.(1998).Onanalysisandimplementationofiterativelearningcontrol.

Licen-tiate thesis LIU-TEK-LIC-1998:62 Linkoping Studies in Science and Technology.

Thesis No727. Departmentof Electrical Engineering,Linkopings universitet.

Panzieri, S. and G. Ulivi (1995). \Disturbance rejection of Iterative Learning Control

Applied to Trajectory for a Flexible Manipulator". In: Proc. ECC 1995. Rome,

+

+

+

+

-T

Frequency (rad/s)

dB

10

−4

10

−2

10

0

10

2

−300

−250

−200

−150

−100

−50

0

50

Frequency (rad/s)

dB

10

−4

10

−2

10

0

10

2

−300

−250

−200

−150

−100

−50

0

−0.5

0

0.5

1

1.5

2

2.5

3

−30

−20

−10

0

10

20

30

sec

rad

−0.5

0

0.5

1

1.5

2

2.5

3

−60

−40

−20

0

20

40

60

80

100

sec

rad/s

0

0.5

1

1.5

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Re

Im

learning circle

stab. circle

Lfor Ldesigned usinga nominalmodel. Dash-dotted:

1=jQ(e i!T

)j for Q=1and Q chosen as a lowpass lter respectively.

−0.5

0

0.5

1

1.5

2

2.5

3

−20

−15

−10

−5

0

5

10

15

20

sec

Nm

0

2

4

6

8

10

10

−8

10

−6

10

−4

10

−2

10

0

10

2

Iteration

|| e

k

||

∞

with Q filter

without Q filter

0

100

200

300

400

500

0

2

4

6

8

10

−22

−20

−18

−16

−14

−12

−10

−8

−6

−4

−2

Iteration

Hz

log10 of power (arb. unit)

0

100

200

300

400

500

0

2

4

6

8

10

−22

−20

−18

−16

−14

−12

−10

−8

−6

−4

−2

Iteration

Hz

log10 of power (arb. unit)

Figure6: Upper: ErrorsignalspectrumwithoutQlter. Lower: Errorsignalspectrum

0

2

4

6

8

10

10

−7

10

−6

Iteration

Error signal energy

with Q filter

without Q filter

0

2

4

6

8

10

10

−3

10

−2

10

−1

Iteration

|| e

k

||

∞

with Q filter

without Q filter

0

100

200

300

400

500

0

2

4

6

8

10

−14

−13

−12

−11

−10

−9

−8

−7

−6

−5

Iteration

Hz

log10 of power (arb. unit)

0

100

200

300

400

500

0

2

4

6

8

10

−14

−13

−12

−11

−10

−9

−8

−7

−6

−5

Iteration

Hz

log10 of power (arb. unit)

Figure8: Upper: Errorsignalspectrum,withoutQlter. Lower: Errorsignalspectrum

%% starting at p1 moveL p2,v100,z1; moveL p3,v100,z1; moveL p4,v100,z1; moveL p5,v100,z1; moveL p6,v100,z1; moveL p1,v100,fine;

−10

−5

0

5

10

15

20

25

−5

0

5

10

15

20

25

30

35

x [mm]

y [mm]

p1

p2

p3

p4

p5

p6

Figure9: The program used to produce the trajectory used in the example (left) and

theresultingtrajectoryonthe arm-sidetranslatedsuchthat p1isinthe origin(right).

0

5

10

0

0.05

0.1

0.15

0.2

0.25

Motor 1

einf

0

5

10

0

0.5

1

1.5

2

e2

Iteration

0

5

10

0

0.1

0.2

0.3

0.4

Motor 2

einf

0

5

10

0

1

2

3

4

e2

Iteration

0

5

10

0

0.05

0.1

0.15

0.2

0.25

Motor 3

einf

0

5

10

0

0.5

1

1.5

2

e2

Iteration