Some results on Iterative Learning Control with disturbances

Some results on Iterative Learning Control with disturbances

M. Norrlof and S. Gunnarsson Department of Electrical Engineering Linkoping University, S-581 83 Linkoping, Sweden

WWW: http://www.control.isy.liu.se

Email: mino@isy.liu.se, svante@isy.liu.se

March 17, 1998

REGLERTEKNIK

AUTOMATIC CONTROL LINKÖPING

Report no.: LiTH-ISY-R-2020

Submitted to CCSSE'98 (First Conference on Computer Science and Systems Engineering in Linkoping)

Technical reports from the Automatic Control group in Linkoping are available

by anonymous ftp at the address

. This report is

contained in the compressed postscript le

.

Some results on Iterative Learning Control with disturbances

M. Norrlof and S. Gunnarsson

Department of Electrical Engineering, Linkoping University, Linkoping, Sweden

Email: mino@isy.liu.se, svante@isy.liu.se

Abstract

Iterative Learning Control is presented briey together with a review of some results on convergence and some new results on Iterative Learning Control with load disturbances and measurement disturbances. In the example presented, Iterative Learning Control is used in combination with conventional feed-back and feed-forward control, and it is shown that learning control is highly a ected by the di erent disturbances. Some ideas on how the lters in the ILC algorithm should be chosen are also discussed.

Keywords

: Robot control, iterative learning control, disturbance a ects

1 Introduction

In many applications manipulators repeat their motions over and over in cycles, this is the case in for example laser cutting. Often the robot manufacturer gives a measure of the accuracy of the robots repeatability, this measure tells how well the robot repeats the same motion. If there is an error in the absolute accuracy, i.e. a dierence between the programmed path and the actual path, this error is repeated with the same accuracy as the repeatability. The basic idea behind iterative learning control (ILC) is that the controller should learn from previous cycles and perform better every cycle

. ILC was rst introduced by Arimoto et al 2] in 1984 and since than many papers have been addressing robot control in combination with iterative learning control, e.g. 4], 11], 13], 7], and 9]. The convergence properties of the algorithm were addressed already in 2], and are further covered in many articles e.g. 10], 1], and 8].

In this paper the fundamental properties of the ILC algorithm applied as a complement to conventional feed-forward and feed-back control will be illus- trated. Focus will be on disturbance aects but also convergence and robustness will be discussed.

1

Better means here with respect to the absolute accuracy.

2 Problem Statement

Consider the representation of a control system shown in Figure 1. After we have introduced a load disturbance and a measurement disturbance we get

Y

=

(

+

) +

(1)

where

,

,

, and

represents input, output, load disturbance, and mea- surement disturbance respectively. G is the transfer function of the system, in the example in this paper a linear model of a one degree of freedom robot manipulator. The system is controlled using a combination of feed-forward and feed-back control. The control law is given by

U

=

+

(

) (2) where

denote the reference signal.

and

denote the transfer function of the feed-forward and feed-back regulators.

+ + + +

+ -

F

f

F G

y

d

(

)

d

(

)

y

(

)

n

(

)

u

(

)

z

(

)

Figure 1: A system with load disturbance and measurement disturbance, con- trolled with feed-forward and feed-back regulators

To evaluate the properties of the applied algorithm we use the control sig- nal generated by the feed-back regulator when no measurement disturbance is applied,

E

=

(

) (3)

This error signal will of course not be used in the feedback for the algorithm because it is not available in the real system. If the feed-back controller is of PD type the error will be a linear combination of the position error and the velocity error, which is reasonable, the control objective is to minimize the position error and the velocity error. Using equations (1), (2), and (3), the error can be described as

E

=

((

)

+

) (4) where

is the transfer function of the closed loop system given by

G

C

= 1 +

(5)

Note that the error the controller will use in the feedback is dened by,

E

con

=

((

)

) (6) how this will aect the properties will be discussed in section 5.

2

When capital letters are used in the following it indicates that the signals are transformed,

the discussion covers both continuous and discrete time signals unless otherwise stated.

3 Outline of the ILC Method

As stated in the introduction the input signal,

, in many applications is repet- itive. This means that if the control system is time invariant

and there is an error in the trajectory following in the rst iteration, this error will be repeated cycle after cycle. If the dynamics of the system is largely repeatable a control algorithm that improves performance from trial to trial can be constructed. A new control signal 

is added to the control signal

in Figure 1 and the input signal to the system will thus be given by

U

k

=

+

(

) + 

(7) The index k indicates iteration or cycle number. Considering only linear oper- ations the updating of the correction signal can, in the frequency domain, be expressed as



=

j=0



H

j

(

) (8) where 

= 0

are linear lters. Note that the denition of

in equa- tion (3) implies that we have to compensate for the measurement disturbance by subtracting

. The updating formula used in this paper, though, is



=



+

(

) (9) where

and

are linear lters. The choice of the lters

and

is the main task when designing an iterative learning control algorithm, since the lters determine the convergence and robustness properties. One method for choosing appropriate lters in the update equation is presented in 5] where methods from design of robust controllers are applied and the lters are designed to give a convergent ILC algorithm despite uncertainties in the process model. In 6] the problem is considered from a dierent viewpoint and the choice of the ILC input signal is formulated as an optimization problem, resulting in a time domain updating equation for the input signal. A variant of this approach is also discussed in 8].

4 Convergence Properties

We shall now investigate how the error signal behaves when the update equation (9) is used. If we dene ~

as the disturbance free error signal obtained in the

rst iteration when 

0 we get

~

E

=

(

)

(10) From equations (4), (7), and (9) the following can be derived

E

k +1

= ~



(

)

+

(11) By adding and subtracting relevant terms on the right hand side we arrive at

E

k +1

= (1

) ~

+

( ~



+

) (12)

; G

C H

2

(

) +

+

3

no adaption

3

by expanding 

using equation (9) and then use equation (11) we get the following error update equation

E

k +1

= (1

) ~

+ (

)

(13) +

(

) +

(

+ (

)

)

A corresponding equation is presented in 13] for the open loop case and for load disturbances only.

The convergence properties are determined by the homogeneous part of the dierence equation (13) and referring to 4] the convergence condition, in the continuous-time case, is that

jH

1

(

)

(

)

(

)

1

(14) This criterion can be viewed in a Nyquist diagram, and the condition is that the Nyquist curve of

should be held within the circle with center in +1 and radius 1. To the right in Figure 2 the amplitude curve of a

lter is shown above and the resulting convergence property for the ILC algorithm, 1

, is shown below. It is obvious that the convergence is best for low frequencies and that it is very slow for frequencies close to the Nyquist frequency.

−0.4 −0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

−1

−0.5 0 0.5 1

Nyquist curve

Real

Imag

GcH

G_c learning circle

stab. circle

10⁰ 10¹ 10² 10³ 10⁴

0 2 4 6 8 10 12 14

Amplitude curve for the filter H

10⁰ 10¹ 10² 10³ 10⁴

0 0.2 0.4 0.6 0.8 1

Amplitude curve for the error updating filter

Figure 2: Learning circle and stabilizing circle (left). Example of lters,

1,

H

2

(upper right), the resulting updating lter 1

(lower right) Provided that the learning procedure converges the error signal becomes



E

= 1

1

+

~ (15)

We see that by using

= 1 we are not able to eliminate the error completely, but as will be seen later other advantages are obtained by this choice. An alternative parameterization of the lters in the learning law was presented in 5], where

H

1

=

=

(16)

Q

and

are linear lters. In 12] this formulation is used with

dened as

Q

= 1 1 +

(17)

and

a scalar. The condition for convergence, based on 5], becomes

k

1

=

1 +

(18)

and it is obvious that the stability region can be extended by a suitable choice of the lter

, resulting in a so called stabilizing circle (see Figure 2).

By letting

be frequency dependant the stability region can be extended in a frequency dependant way and equation (18) shows that if we choose

as a low-pass lter,

will extend the stability region for high frequencies. In 5] it is also shown that the lter

can be found through a 'model matching problem', i.e. solving the

problem

L

= arg min

L2H1

kQ

(1

)

(19) This minimization will result in

kQ

(1

)

=

1 (20) It should be noted that the smaller

is the faster the convergence of 

and

e

.

5 Disturbance Aects

We will now study input and output disturbances separately and we will start by exploring the load disturbance. First we recall the equation describing how the error is updated

E

k +1

= (1

) ~

+

(1

)

(21) +

(

) +

(

+

(

1)

)

here with the denition of the lters

and

as is found in equation (16).

5.1 Load disturbance

A number of observations can be made using equation (21). Let us rst con- sider the case

1 and

0, which means that we have no measurement disturbance. This implies the update equation

E

k +1

= (1

)

+

(

) (22) The disturbances contribute to the error equation by their dierences between the iterations. If a disturbance is of repetitive nature in the sense that the disturbance signals

(

) =

(

) for all

, the contribution to the error dier- ence equation is zero. This assumption is likely for the load disturbance where for example load disturbances due to gravitational forces can be expected to be rather similar during dierent iterations.

Let us also consider the situation with

= 1, neglect measurement dis- turbances and assume that

(

) =

(

)

. This corresponds to the error dierence equation

E

k +1

= (1

) ~

+

(1

)

(23)

; G

C

D

(1

)

The load disturbance will act as a driving term similar to the initial error ~

, and we can draw some conclusions about the frequency content of this driving term. We know that the closed loop system

, and the lter

are both of low pass type and this means that 1

is of high pass type, multiplying these two will give a bandpass lter with a passband around the bandwidth of the closed loop system. However, if the load disturbance is of low frequency type or the

lter

can be chosen with a higher crossover frequency than the bandwidth of the system, this will not be a problem.

5

Frequency (rad/sec)

Singular Values (dB)

Singular Values

10⁻⁴ 10⁻² 10⁰ 10² 10⁴

−300

−250

−200

−150

−100

−50 0 50

1 − G c L

Frequency (rad/sec)

Singular Values (dB)

Singular Values

10⁻⁴ 10⁻² 10⁰ 10² 10⁴

−300

−250

−200

−150

−100

−50 0 50

Q(1 − G c L)

Figure 3: The amplitude plot of

(1

)

with

1 (left), and the same amplitude plot but with

of low pass type (right)

5.2 Measurement disturbance

Measurement disturbances, on the other hand, are more likely to be of random character which means that

(

)

=

(

) in general, and there will hence always be a driving term on the right hand side of equation (21) that prevents

E

k

from tending to zero. Consider the case when we only have a measurement disturbance, this means that

0 and

0. The fact that

0 implies

~

E

0, and we get

E

k +1

=

(1

)

+

(

+

(

1)

) (24) this equation can also be written as a nite sum

E

k +1

=

(

+

(1

)

j=0 Q

k ;j;1

(1

)

) (25) A sucient condition for stability is

lim

k !1 kE

k k

1

<M

(26)

which can be proved to hold when

kQ

(1

)

=

1

(27) because then the sum has an upper bound

k

X

j=0

kQ

(1

)

(28)

 k

X

j=0

 j

KK

1

1

(29)

Let us choose

1 and

= ^

(1

) where

is a high-pass lter

which decides the desired convergence properties of the ILC algorithm. ^

is a

model of the closed loop system including

and a model of

referred to as ^

.

In the left plot in Figure 3 the principal behavior of

1

is shown and we

see that in this case we will have problems at high frequencies. If the

lter is

introduced the high frequency disturbance impact can be reduced. In the right

plot of Figure 3 the aect when Q is a low pass lter is shown.

We see that by introducing

we can reduce the measurement disturbance impact on the ILC algorithm, but with the price that the error in the trajectory following will not tend to zero. The demand of trajectory following and distur- bance reduction will in this sense be contradictory and we have to have this in mind when designing the lters

and

.

−0.5 0 0.5 1 1.5 2 2.5 3

−30

−20

−10 0 10 20 30

sec

rad

yD − reference signal

−0.5 0 0.5 1 1.5 2 2.5 3

−60

−40

−20 0 20 40 60 80 100

sec

rad/s

dyD/dt − velocity reference signal

Figure 4: The reference signal,

(left) and the derivative, _

(right).

6 Simulation Example

We shall consider a simplied description of a single robot joint modeled as a double integrator, i.e.

G

(

) = 1

Js

2

(30)

Since the system is computer controlled we shall use the discrete time represen- tation given by the transfer function

G

(

) =

(

+ 1)

2

(

1)

(31)

where

= 0

0094 is the moment of inertia. The system is controlled by a discrete time PD-regulator given by

F

(

) =

+

T

(

1)

z

(32) with

= 12

7 and

= 0

4. The feed-forward lter is chosen as a double dierentiation represented by

F

f

=

(

1)

T 2

z

2

(33)

where

is the estimated moment of inertia. The correction signal will be updated according to equation (9) where

(

) =

(

) and

(

) =

(

)

(

) are lters that both may be non-causal. The model is simulated using 1 kHz sampling frequency. For evaluation of the algorithm we shall apply the reference trajectory shown in Figure 4.

6.1 Design of the

lter

For

(

) = 1 the ideal choice of

would be to choose it as the inverse of

G

C

(

), which, theoretically, would result in convergence to zero in one step.

7

This is however an unrealistic choice since it requires exact knowledge of the system and results in a lter with very high gain for high frequencies. Instead we consider

L

(

) = ^

(

)(1

(

)) (34) where ^

(

) denotes the closed loop transfer function we obtain by using the model of the open loop system. We will consider the case when we have a 30%

error in

, the estimated moment of inertia in the model of the system. The

lter

(

) is a Butterworth high pass lter (here of second order) for which the gain tends to one for high frequencies. Choosing

(

) according to this design rule, with cut-o frequency of the high pass lter equal to 0

4 times the Nyquist frequency, gives the Nyquist curve depicted in the left plot of Figure 2. Figure 2 also shows

(

) for comparison. The whole Nyquist curve is now inside the learning circle while it for large frequencies tends to the origin.

−0.5 0 0.5 1 1.5 2 2.5 3

−20

−15

−10

−5 0 5 10 15 20

sec

Nm

d − load disturbance

0 1 2 3 4 5 6 7 8 9 10

10⁻¹² 10⁻¹⁰ 10⁻⁸ 10⁻⁶ 10⁻⁴ 10⁻² 10⁰

Iteration

Energy (arb. unit), log−scale

Energy in the error signal

with Q filter without Q filter

0 200

400

600 0 2 4 6 8 10

−20

−18

−16

−14

−12

−10

−8

−6

−4

−2

Iteration number Error signal spectrum (Ek) without Q

Hz

log10 of power (arb. unit)

0 200

400

600 0 2 4 6 8 10

−20

−18

−16

−14

−12

−10

−8

−6

−4

−2

Iteration number Error signal spectrum (Ek) with Q filter

Hz

log10 of power (arb. unit)

Figure 5: Load disturbance (upper left), error signal energy (upper right), error signal spectrum without Q lter (lower left), and error signal spectrum with Q

lter (lower right)

6.2 Load disturbance

The rst goal is to investigate how the load disturbance in!uences the ILC algorithm. In Figure 5, the upper left plot shows the applied load disturbance as a function of time. This disturbance signal is applied at the same time every cycle as a repetitive disturbance. Recall equation (23)

E

k +1

= (1

) ~

+

(1

)

; G

C

D

(1

)

If

1 the load disturbance will not have any aect on the result, except in the

rst iteration. By introducing the lter

= 1 we can see that the disturbance

will have an aect on the resulting error. Normally, as discussed in section 5, the

lter

is chosen as a low-pass lter which means that 1

will have high-pass characteristics. In Figure 5 the result when we have introduced a lter

can be seen. In the upper right plot the energy in

as function of iteration is shown and in the lower plots the spectrum of

is shown as a function of iteration.

Clearly the introduction of a lter

reduces the convergence speed of the ILC algorithm and there will also not be convergence to zero of the error. Already after one iteration when the lter

is used the energy in the error has almost reached its nal value and also in the spectrum it is possible to see that there is not much change after the rst iteration.

0 100

200 300 400

500 0

2 4

6 8

10

−14

−13

−12

−11

−10

−9

−8

−7

−6

−5

Iteration number Position error signal spectrum (Epk) without Q filter

Hz

log10 of power (arb. unit)

0 100

200 300 400

500 0

2 4

6 8

10

−14

−13

−12

−11

−10

−9

−8

−7

−6

−5

Iteration number Position error signal spectrum (Epk) with Q filter

Hz

log10 of power (arb. unit)

0 1 2 3 4 5 6 7 8 9 10

10⁰ 10¹

Iteration

Energy (arb. unit), log−scale

with Q filter without Q filter

0 1 2 3 4 5 6 7 8 9 10

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5

Iteration

max(abs(ek)) − [rad]

Speed of convergence

with Q filter without Q filter

Figure 6: Position error signal spectrum, without Q lter (upper left), position error signal spectrum with Q lter (upper right), position error signal energy (lower left), and maximum absolute position error (lower right)

6.3 Measurement disturbance

The measurement disturbance,

, introduced is a time discrete white noise process with standard deviation 3

2

10

. The aect of introducing a measure- ment disturbance is very important because, when the algorithm is tested on a real robot, measurement disturbances will always be present. Therefore it is of great importance to really investigate how the lters should be chosen to reduce the in!uence of the measurement disturbance. First recall equation (25)

E

k

=

(

+

(1

)

j=0 Q

k ;j;1

(1

)

)

If the lters are chosen as in equation (16) it is possible to shape the transfer function

(1

) by introducing

= 1, as shown in Figure 3. By looking

9

at equation (25) it is possible to see that the in!uence of the measurement disturbance at high frequencies can be reduced by using

as a low-pass lter.

In Figure 6 the spectrum of the position error signal

is shown without Q lter (upper left) and with Q lter (upper right). In the lower left plot the energy of the position error signal is shown as a function of iteration, and in the lower right plot the maximum absolute error in the time domain is shown as a function of iteration. It is obvious in Figure 3 that introducing the

lter will make the aect of measurement disturbances decrease at certain frequencies. In the convergence plot of Figure 6, it is possible to see again that nearly all the aect of the size of the error is in the rst iteration. This is clearly an aect of introducing the measurement disturbance. In the previous section it was clear that if

1 the error would decrease to zero, but here we can see that this will not be the case when there is a measurement disturbance.

7 Conclusions

The Iterative Learning Control algorithm has been studied when there are dis- turbances in the system and it is shown that the choice of lters in the algorithm is very much dependent on what kind of disturbance we have. In the example it is shown that nearly static values in the error are already reached in the rst iteration of the algorithm. This result has also been found in experiments, e.g.

in 3] where an ILC algorithm is applied on a DELTA robot. Some new results on ILC with measurement disturbances are presented and some ideas of how the lters in the ILC algorithm should be chosen are also discussed.

8 Acknowledgments

This work was supported by CENIIT at Linkoping University and ABB Robotics within ISIS at Linkoping University.

References

1] N. Amann, D. H. Owens, and E. Rogers. Iterative learning control for dis- crete time systems with exponential rate of convergence. Technical Report 95/14, Centre for Systems and Control Engineering, University of Exeter, 1995.

2] S. Arimoto, S. Kawamura, and F. Miyazaki. Bettering operation of robots by learning. Journal of Robotic Systems, 1(2):123{140, 1984.

3] E. Burdet, L. Rey, and A. Codourey. A trivial method of learning control.

In Preprints of the 5th IFAC symposium on robot control, volume 2, Nantes, France, September 1997.

4] J. J. Craig. Adaptive Control of Mechanical Manipulators. Addison-Wesley Publishing Company, 1988.

5] D. de Roover. Synthesis of a robust iterative learning controller using an

H

1

approach. In Proc. 35th CDC, pages 3044{3049, Kobe, Japan, 1996.

6] D.M. Gorinevsky, D. Torfs, and A.A. Goldenberg. Learning approxima-

tion of feedforward dependence on the task parameters: Experiments in

direct-drive manipulator tracking. In Proc. ACC, pages 883{887, Seattle,

Washington, 1995.

7] S. Gunnarsson and M. Norrlof. On the use of learning control for improved performance in robot control systems. In Proceedings of the European Con- trol Conference 1997, Brussels, Belgium, July 1997.

8] S. Gunnarsson and M. Norrlof. A short introduction to iterative learn- ing control. Technical Report LiTH-ISY-R-1926, Department of Electrical Engineering, 1997.

9] S. Gunnarsson and M. Norrlof. Some experiences of the use of iterative learning control for performance improvement in robot control systems. In Preprints of the 5th IFAC symposium on robot control, volume 2, Nantes, France, September 1997.

10] L. M. Hideg. Stability of Learning Control Systems. PhD thesis, Oakland University, 1992.

11] R. Horowitz. Learning control of robot manipulators. Journal of Dynamic Systems, Measurement, and Control, 115:402{411, June 1993.

12] T. Mita and E. Kato. Iterative control and its application to motion control of robot arm - a direct approach to servo-problems -. In Proceedings of 24th Conference on Decision and Control, pages 1393{1398, December 1985.

13] S. Panzieri and G. Ulivi. Disturbance rejection of iterative learning control applied to trajectory tracking for a !exible manipulator. In Proceedings of 3rd European Control Conference, pages 2374{2379. ECC, September 1995.

11