IN
DEGREE PROJECT ELECTRICAL ENGINEERING, SECOND CYCLE, 30 CREDITS
,
STOCKHOLM SWEDEN 2016
Phase-based Extremum
Seeking Control
SUYING WANG
Phase-based Extremum Seeking Control
SUYING WANG
Stockholm 2016
Automatic Control
Phase-based Extremum Seeking Control
Abstract
Extremum Seeking Control (ESC) is a model-free adaptive control method to locate and track the optimal working point for nonlinear plants. However, as shown recently, traditional ESC methods may not work well for dynamic systems. In this thesis, we consider a novel ESC loop to locate the optimal operating point for both static and dynamic systems. Considering that the phase-lag of the system undergoes a large shift near a steady-state optimum and reaches the value of ⇡/2 at the optimal operating point, the novel ESC applies the phase-lag of the target system to track the optimum. An ex-tended Kalman filter is used to ensure the accuracy of the phase estimation. The structure of a phase locked loop (PLL) is employed in combination with an integral controller to lock the phase near ⇡/2, such that the target system will operate near the optimal working point. The controller is demonstrated by application to optimization of the substrate conversion in a chemical re-actor.
Sammanfattning
Acknowledgement
First of all, I would like to express my sincere gratitude to my supervisor Olle Trollberg, for his constant encouragement and guidance. He has walked me through all the stages of my thesis, leading me into the world of extremum seeking control. Without his consistent and illuminating instruction, the thesis could not have reached this final stage.
Second, I would like to express my heartfelt gratitude to my examiner Professor Elling W. Jacobsen, for his excellent instructions and guidance on my thesis project. Without his patient instruction, insightful criticism and expert guidance, I would not be able to complete my thesis.
I feel grateful to all the professors and teachers at KTH, who o↵ered me valuable courses and advice during my study.
Last but not least, I am truly grateful and thankful to Nan Qi and Diliao Ye, for giving me lots of suggestion during my thesis period. I would also like to thank my parents and my boyfriend for providing support. Their encouragement and unwavering support has sustained me through frustration and depression.
Contents
1 Introduction 1
1.1 Problem Statement . . . 2
1.2 Contribution . . . 3
1.3 Structure of the Report . . . 3
2 Background 4 2.1 Classic Extremum Seeking Control . . . 4
2.1.1 Sliding mode Extremum Seeking Control . . . 6
2.1.2 Perturbation based Extremum Seeking Control . . . . 7
2.1.3 Phase in Perturbation based Extremum Seeking Control 11 2.2 Phase-based Extremum Seeking Control . . . 12
3 Design of the Control Loop 18 3.1 Phase Locked Loops . . . 18
3.1.1 Adapting a PLL for ESC . . . 20
3.2 Phase Estimation . . . 21
3.2.1 Estimation by Demodulation . . . 21
3.2.2 Updated Estimation from Variant of EPLL . . . 23
3.2.3 Estimation by Kalman Filter . . . 24
3.3 Controller . . . 28
3.4 Selection of the Controller Structure . . . 28
4 Controller Tuning 31 4.1 Kalman Filter . . . 31
4.2 Controller . . . 34
4.3 Perturbation Signal . . . 35
4.4 Summary . . . 37
5 Example and Analysis 38 5.1 Performance Test . . . 39
5.2 Robustness Test . . . 42
6 Conclusions and Further Research 47 6.1 Disscussion and Conclusion . . . 47 6.2 Future Work . . . 47
Chapter 1
Introduction
In process industry today, optimization is essential for most aspects of process operation. For example, in the production process, manufacturers might prefer to either maximize the output of a process or minimize the power consumption; when designing a car, it is often desired to minimize the fuel consumption. In order to reduce the cost and maximize the profits, processes should be designed and operated both e↵ective and cost-efficient, preferably in combination with a low workload for the process operators. In this thesis, we specifically focus on optimization within the control layer of a process. We try to determine a control law to, for example, maximize process throughput or minimize the consumption of energy or raw materials.
The optimization problem could be solved either online or o✏ine. If the operating conditions of the plant are stable and the optimum does not vary over time, we could do a static o✏ine optimization to find the optimal oper-ating point and keep the process working at the optimum using a regulator. However, o✏ine methods cannot help us locate the optimal operating point for some systems due to disturbances. These disturbances may vary a lot, making the location of the optimum uncertain. In order to accurately lo-cate the optimum, we should try to do the optimization online, obtaining a feedback based solution. When feedback is introduced in the system, the sensitivity towards uncertainty and modelling errors would be reduced. In addition, we might be able to track the optimal operating point over time with the help of the feedback signals, which may improve the performance of the controller.
which can track the optimal operating point when the parameters of the system are uncertain. All these methods can be used to locate the optimal operating point when the model of a process is available. However, a model is not always available or possible to derive. Forms of system equations, as well as the parameters, are often unknown. In addition, the unmeasured disturbances may vary over time, e.g., solar power plant operation depends on the weather, dust on the panel, humidity, etc., which is hard to forecast and varies a lot. Moreover, the time and resources available may not be enough to develop an accurate model. Therefore, a model-free method is required to perform process optimization.
Extremum Seeking Control (ESC), which could be regarded as a branch of adaptive control, might be a choice for model-free optimization. ESC is an online model-free method, which only relies on output measurements. It mainly focuses on the gradient of the steady-state map. The gradient will be zero when the output of the system reaches the maximum/minimum point. Therefore, optimal operating point could be found when the gradient reaches zero. Although traditional ESC can be used for the optimization of online model-free systems, it has some limitations when applied to nonlinear dynamical plant. For example, the plant has to be quasi-static and the adaption gain should be small when the traditional perturbation ESC is applied [3].
Trollberg and Jacobsen introduced a novel idea about model-free opti-mization [4], which is the basic idea of this thesis. In [4], the relationship between dynamic and static properties of plants with general nonlinear dy-namics are investigated. It is shown that the steady-state optimal operating point is not only reflected by a zero gradient of the equilibrium map, but also in the local phase-lag of the system. At the optimal working point, the phase-lag of the system will approach±⇡/2 due to a bifurcation of the plants zero dynamics. The novel idea is so interesting that the idea is applied to do the optimization, i.e. phase-based extremum seeking control.
1.1
Problem Statement
1.2
Contribution
In this thesis report, we show that the optimization problem could be solved by the novel method, using the structure of phase locked loops (PLLs).
We analyze the impact of the parameters in the controller and provide a guide for controller tuning. This analysis is performed when the Kalman filter is used to do the phase estimation, and the control part of the controller is an integral controller.
We perform several simulations illustrating the performance of the method. These simulations show the impact of the various parameters, which is the same as our analysis.
1.3
Structure of the Report
The remainder of the thesis report is structured as follows. In Chapter 2, we provide the necessary background. Here we describe the ESC problem in more depth, and discuss how the phase may be utilized to locate the steady-state optimum.
In Chapter 3, we go on and consider the design of the novel control loop. The structure, the estimator as well as the controller are discussed in detail in this part.
Chapter 4 provides a guide for controller tuning. Guidance for parameter tuning for the estimator, the controller as well as the perturbation signal are analysed in detail.
In Chapter 5, several simulations are performed to illustrate the perfor-mance of the novel method.
Chapter 2
Background
This chapter provides the background of the thesis project. Section 2.1 in-troduces the classic extremum seeking control in detail. In Section 2.2, the working scheme of phase-based ESC is discussed.
2.1
Classic Extremum Seeking Control
Extremum seeking control is an online model-free optimization method based on the feedback from output measurement. It is used to locate and track the optimal operating point of a given plant, when no model information is available. We should note that an implicit assumption in ESC is that an optimum exists.
In addition, we only consider SISO (Single Input Single Output) systems in this thesis. In general, ESC relies only on the feedback from output measurements. For static plants, i.e., plants without dynamics, or memory, the control target of ESC is the output of the system. For dynamic plants, however, ESC tries to locate the extremum steady-state output. That is, if we consider the state space representation of a general nonlinear plant:
˙x = f (x, ✓),
y = h(x), (2.1)
where f is the state equation and h is the output equation. Then the steady states are defined when the derivative of the states is zero, i.e.,
f (xss, ✓) = 0,
yss= h(xss),
Figure 2.1: Relationship between control, adaptive control and extremum seeking control.
The idea of extremum seeking control could be traced back to 1922 [5]. Some studies were performed in Russia during the Second World War [6]. In the mid-20th century, researchers put a lot of e↵ort into ESC. For ex-ample, Draper and Li detailed an extremum seeking control algorithm for internal combustion engines and the performance was analyzed [7]. Obabkov discussed multichannel ESC [8]. The first rigorous assessment of stability of ESC was published in 2000 [3]. In this study, Krst´ıc and Wang proved local stability of a near optimal solution for a general set of dynamic plants. This study renewed the interest in the field of ESC as is evident in the large number of publications following the break through. Nowadays, quite a few applications are based on ESC, e.g., Anti-Lock Brake System(ABS) system, biology system, etc. [9].
Extremum seeking control may be considered as a subfield of adaptive control. In adaptive control, the model used by the controller is updated online, using the information available in the measurements. A typical adap-tive controller consists of two separate loops: one is a normal feedback loop, the other is a parameter-adjusting loop. In extremum seeking control, the model, which is adapted, is essentially the local gradient of the equilibrium map. A feedback loop is then applied to drive the system to a point, where the gradient is zero. Figure 2.1 relates extremum seeking control to adaptive control and control in general.
There are several extremum seeking control methods, e.g., gradient-based ESC, sliding mode ESC, perturbation-based ESC, etc. [10].
for the system is the point with zero gradient. These methods can be re-garded as an approximation of gradient descent. Therefore, the gradient descent method is introduced first. Gradient descent is a first-order opti-mization method. It works as follows. Assume that we want to minimize the performance function
y = J(✓), (2.3)
where y is the output, ✓ is the input and J is the performance function, which is di↵erentiable. Then, y decreases fastest if it goes from J(✓0) in the
direction of the negative gradient of J at (✓0, J(✓0)). rJ(✓0) represents the
gradient of J at point (✓0, J(✓0)). Assume is positive and small enough, we
can get J(✓1).
J(✓1) = J(✓0) rJ(✓0). (2.4)
The term rJ(✓0) is subtracted from J(✓0), i.e., the point is moved down
toward the minimum. If we preform this repeatedly, we can get a sequence of point J(✓0), J(✓1),· · · such that
J(✓n+1) = J(✓n) nrJ(✓n), n 0, (2.5)
i.e.,
y0 y1 y2 · · · . (2.6)
Therefore, the sequence may converge to the desired local minimum.
Gradient descent method can be used to find the minimum point. The initial point can be set at any point. However, this method has some limita-tions. The converge speed is relatively slow when the operating point is close to the minimum. Moreover, the value of can influence the convergence. Smaller leads to slower converge speed but on the other hand, the system may diverge if a large is employed.
Although many methods in extremum seeking control are based on the gradient, sliding mode ESC is a notable exception. It dose not rely on any estimation of the gradient.
2.1.1
Sliding mode Extremum Seeking Control
[12]. The basic idea of sliding mode ESC is described as follows. Assume we would like to maximize the output of a static system
y = J(✓), (2.7)
where J is referred as the performance function, ✓ 2 < is the control input and y 2 < is the output. The basic idea in sliding mode based ESC is to force the output to follow a given reference signal using a sliding mode controller. The aim of the controller is to force the output to be ever increasing with a specified rate k. By doing so, the output will eventually reach a local maximum. In sliding mode ESC, we should first choose any ever-increasing function g(t) with slope of k as reference. The error between the reference signal g(t) and the system output y(t) is e(t) = y(t) g(t). The input signal ✓ is adjusted at a certain rate and the direction is decided by the error e(t). Then, if the controller is tuned properly, it will lock onto a switching surface and approach the optimum at the same rate of g(t). When the working point is near the optimum, the increase rate of y(t) cannot be sustained and the controller will oscillate around the optimum.
2.1.2
Perturbation based Extremum Seeking Control
Perturbation based ESC is another kind of ESC, which is based on the gra-dient information. It is developed for static systems, like y = J(✓), where J is the performance function, y 2 < is the output and ✓ 2 < is the input. The optimal operating point can be found when the gradient of the performance function reaches zero. Therefore, perturbation based ESC tries to adjust the input to make the gradient approach zero. The method works essentially as follows.
The perturbation based ESC scheme is shown in Figure 4.3. This scheme has five elements, namely, target system, perturbation signal, high-pass filter, low-pass filter and the controller. This method is based on the gradient of the performance function. Therefore, to ensure that the gradient information is available in the output, a perturbation a sin(!t) signal is added to the input. The relationship among controlled input, estimated optimal input and the perturbation signal is
✓ = ˆ✓ + a sin(!t), (2.8) where ✓ is the controlled input, ˆ✓ is the estimated optimal input.
Figure 2.2: Structure of perturbation-based extremum seeking control. Therefore, a larger a is preferred. But on the other hand, large fluctuations in either the input or output is not what we desired. As the perturbation is made larger, nonlinearities will become more important. In addition, large fluctuations may put more wear on the equipment and it might be preferred to minimize the variation in the output in some applications. Therefore, a small a is preferred under this consideration. A trade o↵ should be made in order to strike a balance between these two considerations. The value of a should be chosen properly to get a good estimation while keeping the fluctuations as small as possible.
In the output signal, only the variation contains the gradient informa-tion. Therefore, a high-pass filter is applied to remove the bias of the target output such that only variation is kept. Thus, the break o↵ frequency of the high-pass filter should be set to a value lower than the frequency of the perturbation signal.
The amplitude of the variation is then extracted into a constant via de-modulation. It introduces high frequency mode as well, as shown in Equation 2.9,
frequency. The remaining constant term is proportional to the amplitude of the variation in the output and hence also the local gradient. Consequently, the output of the low-pass filter becomes an estimate of the gradient. It is not the exact value of the gradient, but proportional to the gradient.
An integral controller is employed to move the operating point towards the optimum based on the gradient information from the low-pass filter. The value of control gain may influence the whole control system. The gain es-sentially decides the bandwidth of the controller, which indicates the speed of the controller. The bandwidth of the controller should normally be lower than that of the gradient estimator, since the updated gradient information should be provided to the controller. If the speed of the controller is higher than that of the estimator, the controller cannot get the updated gradient in-formation. As a result, the controller may make the whole system unstable. The bandwidth of the gradient estimator is decided by the break-o↵ fre-quency of the low-pass filter, which is decided by the perturbation frefre-quency. Therefore, the value of the controller gain k should be chosen according to the value of perturbation frequency.
When these five elements are settled, the control loop can locate the optimal operating point automatically. Consider a static nonlinear system, for which the performance function can be represented as
y = J(✓), (2.10)
where y 2 < is the output, ✓ 2 < is the input and J is the static plant. Assume that the system output has a maximum value and that the controller gain k is positive. The optimal input is represented by ✓⇤ while the estimated
optimal input could be written as ˆ✓. The input error ˜✓ could be represented as
˜
✓ = ✓⇤ ✓.ˆ (2.11)
Since J(✓) has a maximum value, the linear part in its Taylor expansion is zero if we evaluate at ✓ = ✓⇤. If we do the second-order Taylor expansion
near ✓⇤ and drop the higher order elements, we have J(✓)⇡ J(✓⇤) + J
00 (✓⇤)
2 (✓ ✓
⇤)2, (2.12)
where J00(✓) is the second-order derivative of J(✓). As shown in Figure 4.3, the input ✓ equals to the sum of ˆ✓ and a sin(!t). Substituting ✓ in Equation 2.12 and expanding the square, we get
The signal y is passed through the high-pass filter, eliminating constant terms. y ⌘ ⇡ J 00 (✓⇤) 2 ✓˜ 2 aJ00 (✓⇤)˜✓sin(!t) J 00 (✓⇤) 4 a 2cos(2!t). (2.14)
The output of the high-pass filter is demodulated by multiplication with the original perturbation. The demodulated signal is then passed through the low-pass filter. Here we assume that the filter is perfect and completely removes any components with frequency above the breako↵ frequency. The controller input ⇠ can then be calculated as follows.
a sin(!t)(y ⇠) ⇡ aJ 00 (✓⇤) 2 ✓˜ 2sin(!t) a2J00 (✓⇤)˜✓sin2(!t) J00(✓⇤) 4 a 3cos(2!t) sin(!t), ⇠ ⇡ a 2✓J˜ 00 (✓⇤) 2 . (2.15)
After di↵erentiating on both sides of Equation 2.11, we could have ˙˜✓ = ˙ˆ✓. Therefore, after the controller, the rate of change of the input error ˜✓ can be represented as
˙˜✓ = ˙ˆ✓ = k ⇤ ⇠ ⇡ ka2J200(✓⇤)✓.˜ (2.16) To solve the Equation 2.16, we use separation of variables.
d˜✓ dt ⇡
ka2J00 (✓⇤)
2 ✓,˜ (2.17)
which can be rewritten as d˜✓ ˜ ✓ ⇡ ka2J00 (✓⇤) 2 dt. (2.18) Further, we have ln˜✓ ln˜✓0 ⇡ ka2J00 (✓⇤) 2 (t t0), (2.19)
where t0 is the initial time and ˜✓0 is the initial input error. Finally, ˜✓ is
obtained and can be given as ˜
✓ ⇡ eka2J 00
(✓⇤)
According to our assumption, k is positive while J(✓) has maximum value, hence, J00(✓) < 0, (2.21) which means ka2J00 (✓⇤) 2 < 0. (2.22)
From the equations above, it follows that lim t!1 ˜ ✓(t)⇡ lim t!1e ka2J00(✓⇤) 2 (t t0)+ln˜✓0 = 0. (2.23) which implies that limt!1✓(t)ˆ ⇡ ✓⇤. In other words, the input will approach
the optimal value when time goes to infinity. However, this analysis about the optimum is valid only locally. If the start point is far from the optimum, this analysis will not be valid any more.
2.1.3
Phase in Perturbation based Extremum Seeking
Control
The analysis above is valid only for static system. A di↵erence between static systems and dynamic systems is that the latter will introduce a phase lag. Perturbation based ESC works also for dynamic systems. Krst´ıc and Wang applied perturbation based ESC to a dynamic system in 2000 [3]. They used asymptotic methods that essentially brought the problem back to the static case. However, their method require very slow estimation and control.
We will now investigate the e↵ect of the perturbation on the various signal in the loop. Consider G(s) as a local linear approximation of the nonlinear system at a stationary solution. For simplicity, we assume that ˆ✓ is constant. When the perturbation signal ˆ✓ + a sin(!t) is set as an input of the control loop, the frequency response of the system is given by G(i!), and |G(i!)| is the gain of the frequency response. According to Figure 4.3, the stationary output of the high-pass filter can be written as
y ⌘ = a|G(i!)||HH(i!)|sin(!t + ), (2.24)
where a is the amplitude of the input signal, |G(i!)| is the gain of the fre-quency response, |HH(i!)| is the gain of the frequency of the high-pass filter,
is the phase-lag of the output of the high-pass filter and ! is the frequency of the perturbation signal.
Multiplied by the perturbation signal a sin(!t), y ⌘ is transformed into the signal shown below
a sin(!t)(y ⌘) = a
2
To remove the high frequency components, we proceed the signal through the low-pass filter and get the input of the controller ⇠, i.e.,
⇠ = a
2
2cos( )|G(i!)||HH(i!)||HL(0)|, (2.26) where HL(0) is the amplitude of the frequency response of the low-pass filter.
The aim of the controller is then to make the signal ⇠ be zero. The value of |HH(iw)| and |HL(0)| are known since the filters and the perturbation
frequency are selected by users. In addition, a is non-zero. Therefore, either |G(iw)| or cos( ) has to be zero. For static systems, when ! = 0, |G(0)| is the gradient. When ⇠ reaches zero, the gradient |G(0)| reaches zero as well. Then, the optimal operation point can be found. For dynamic systems, however, G(iw) is unlikely to be zero since it implies that all dynamics are gone at the optimal working point. Therefore, it is likely that it is the phase condition which is fulfilled at the optimum. This indicates that the phase-lag is tied to the optimal operating point and it will be further discussed in next section.
2.2
Phase-based Extremum Seeking Control
According to [4], there is a connection between phase and optimality. It is mentioned that there is a large phase-shift near the extremum point. The phase-lag will reach ±⇡/2 when the system is operating at the optimum. We are going to show how to make use of the phase-lag to find the optimal operating point.
Assume that a nonlinear system can be represented as ˙x = f (x, ✓),
y = h(x), (2.27)
where x is the state, ✓ 2 < is the input and y 2 < is the output. Assume that the steady state is parametrized by the input ✓,
˙x = 0, i↵x = I(✓). (2.28) Therefore, the steady-state input-output relationship is
y = h(I(✓)). (2.29)
Assume that the steady-state input-output relationship is as shown in Figure 2.3 and G✓(s) is the transfer function of the above defined system at the
steady state corresponding to ✓. We then have G✓(0) =
dJ
Figure 2.3: Steady-state output-input relationship. where J is the cost function.
Therefore, the gradient at point A is G✓A(0). Assume point B is the optimal operating point, then the gradient at point B, i.e., G✓B(0), is zero. The sign of the gradient will be changed when the operating point moves across point B. That is, G✓A(0) is positive and G✓C(0) is negative.
Assume a finite dimensional state space representation is linearized, a rational transfer function is in a form of
G✓(s) = K sm+ b m 1sm 1+· · · + b1s + b0 sn+ a n 1sn 1+· · · + a1s + a0 , (2.31)
where K is the gain of transfer function and a0, a1,· · · , an 1and b0, b1,· · · , bm 1
are the coefficients.
Then at the optimal point B, we have G✓B(0) = 0 = K
b0
a0
. (2.32)
In this equation, we have three parameters, K, a0 and b0. As the sign of
the gradient G✓(0) must switch when the input goes through the optimal
Figure 2.4: The relationship between z(✓) and ✓.
change sign. We will analyse them one by one. If K changes sign, the value of K will be zero at the optimal working point B. When this is the case, the system will not have any dynamic response at the optimum. This is rare in dynamic systems. Therefore, we don’t consider this situation. a0 is not the
parameter which changes sign, either. If a0 changes sign near the optimum,
it means that at least one pole in the transfer function should be infinity at the optimum and change sign from +1 to 1 or the other way around. It is impossible to move a pole from positive to negative through infinity. In other words, it is impossible to change the sign of a0 through infinity. Therefore,
a0 should not changes sign. Upon the analysis above, only b0 changes sign
around the optimal operating point. Hence, for the rest of the analysis, we assume only b0 changes sign.
If we rewrite Equation 2.31 as G✓(s) = K
(s + z1)(s + z2)· · · (s + zm 1)(s + zm)
(s + p1)(s + p2)· · · (s + pn 1)(s + pn)
, (2.33) where z1, z2,· · · , zm represents the zeros and p1, p2, p3,· · · , pnrepresents the
poles. Comparing Equation 2.31 and Equation 2.33, we have b0 = Qmi=1zi.
Since b0 switches sign, an odd number of zeros has to switch sign. Assume
that only one zeros changes sign. Hence, the optimality condition correspond to a zero switching sign through the origin. Assume the specific zero zm =
Then, we are interested in how the phase is a↵ected by such a zero-crossing. G✓(s) could be rewrite as
G✓(s) = G0(s)(s + z(✓)), (2.34) where G0(s) = k0 (s + z1)(s + z2)· · · (s + zm 2)(s + zm 1) (s + p1)(s + p2)· · · (s + pn 1)(s + pn) . (2.35) Therefore, the phase-lag can be expressed as the sum of arg(G0(i!)) and
arg(i! + z(✓)). The phase of G0(i!) will be discussed first.
According to Equation 2.35, we can obtain the phase of G0(s).
arg(G0(i!)) = arctan
! z1 + arctan ! z2 +· · · + arctan ! zm 1 (arctan ! p1 + arctan ! p2 +· · · + arctan ! pn 1 + arctan ! pn ). (2.36) Property 1. When ! ! 0, the value of arg(G0(i!)) will approach kG0⇡, where kG0 = 0,±1, ±2, · · · .
Proof. As we assumed before, there is only one zero zm in G✓(s) will be zero
when the system reaches the optimal operating point. According to Equation 2.34, G0(s) is a part of G✓(s), which does not contain z(✓). Therefore, zeros in
G0(s) cannot be zero. Assume that the system is stable. Then, poles in G0(s)
that cannot be zero either. Therefore, neither poles nor zeros in the G0(s)
can be zero. As a result, z!
i will approach zero, where i = 1, 2,· · · , m 1 when ! approaches zero. Additionally, p!j will approach zero, where j = 1, 2,· · · , n when ! approaches zero as well. It is easy to find that arctan(0) = 0. According to Equation 2.36, phase of G0(s) is the sum of the arctan results.
Hence, when ! ! 0, we have
arg(G0(i!))⇡ arctan(0) + arctan(0) + · · · + arctan(0)
(arctan(0) + arctan(0) +· · · + arctan(0)) = kG0⇡,
(2.37) where kG0 = 0,±1, ±2, · · · .
As for the arg(i! +z(✓)) = arctanz(✓)! , its di↵erent phase-shifts are shown in Figure 2.5. With fixed frequency, phase-lag will change as the working point ✓ changes. When the system reaches the optimum, the gradient of the steady-state input-output map will be zero. Based on the discussion above, G✓(s) will be zero, which indicates that the value of b0 will be zero as well.
Figure 2.5: Di↵erent phase shifts at di↵erent frequencies.
z(✓) will go through the imaginary axis as well, which means the value of arg(i! + z(✓)) will yield a large continuous phase shift around the optimal operating point. When the frequency decreases, the value of the shift will be close to ±⇡ near the optimal operating point, which will be explained in detail the the following parts.
According to the discussion above, it can be investigated that, at the optimal operating point, the phase-lag of the system will be
arg(G0(i!)) + arg(i! + z(✓)) = arg(G0)± ⇡/2. (2.38)
If small frequency ! is employed, the phase-lag will be close to
As discussed above, the value of arg(G0(i!)), the value of arg(i! + z(✓))
and the phase-shift are all influenced by the value of !. In what follows, we would like to show that as the frequency ! decreases, the phase-shift increases. As shown in Figure 2.5, considering the frequencies !1 > !2 >
!3 and z(✓1), z(✓2) are the value of the zero at two di↵erent near optimal
working points on either side of the optimum. The phase-shifts are di↵erent at di↵erent frequencies, when the input ✓ changes the same value. With the decrease of the frequency, the phase shift increases, i.e., 1 < 2 < 3. The phase changes more significantly at lower frequency. When the
frequency ! is zero, z(✓) moves on the real axis. If the working point moves from ✓1 to ✓2, the phase will change from 0 to ⇡ accordingly, which means the
phase shift is ⇡. When it moves the other way around, the phase shift will be ⇡ instead. Thus, when the frequency is close zero, the phase shift will be close to ±⇡. In addition, when lower frequency is set, only the optimum can make arg(i! + z(✓)) equal to±⇡/2. It is possible to find the optimum by the phase information. On the other hand, if large frequencies are employed, the phase shift will be small. When the frequency is set to infinity, the phase shift will be close to zero, which means a larger range of operating points can make arg(i! + z(✓)) close to ±⇡/2. Under this situation, it is almost impossible to find the optimal operating point any more. Therefore, it is easier to locate the optimal operating point for lower frequencies.
Chapter 3
Design of the Control Loop
In this chapter, we will discuss how estimation and regulation of the local phase-lag could be utilized for extremum seeking control. A Phase locked loop (PLL) has been used to accurately track the phase and frequency of noisy input signals. Here we note that a similar structure is useful also for phase-based ESC and thus we begin the chapter by reviewing the literature on PLLs. We then go on and discuss how the elements of a PLL may be adapted for our current purpose. Specifically, we discuss various alternative phase estimation schemes and the impact of di↵erent loop filters, i.e., controllers. Finally, we select a specific control structure which we analyse further in the later chapters.
3.1
Phase Locked Loops
The technique of phase locked loops was described by Henry de Bellescize [13] in 1932. The theory of PLL was well developed and widely used in modern communication systems in 1970’s [14], [15], [16]. PLL was generally used to detect and track the frequency of an incoming signal. An early application of PLL was in analogue television where it was used to synchronize local sweep rates with the frequency in the broadcast signal. Later, it was applied to tune integrated circuits [17]. Today, PLL is frequently used in a variety of applications ranging from space communications to network clocks [18].
Figure 3.1: A basic structure of phase locked loop.
acts as a feedback controller while the VCO behind changes the phase and frequency of the output according to the output of the loop filter. When the loop is locked, the frequency of the output signal is exactly the same as the frequency of the input signal, i.e., reference signal, and the phase error between reference signal and output signal is locked to a constant value as well.
PLL estimates the phase error based on demodulation by multiplication. It su↵ers from double-frequency errors introduced by the multiplication. In order to solve this problem, Karimi and Iravani introduced an Enhanced Phase Locked Loop (EPLL) structure [19], where the phase detection part is re-organized. This enhanced PLL is robust with respect to both internal settings and external noise. Furthermore, it allows the phase and amplitude of the input signal to be estimated directly and independently. In [20], Karimi did further improvement in EPLL, where an estimation loop is added. The enhanced loop is able to estimate the amplitude of the input signal, which can help the system get rid of the double-frequency errors. Patapoutian described a PLL with the structure of Kalman filter in [21]. This system achieves rapid acquisition and reliable tracking through replacing the constant gain with a time-varying Kalman gain [22]. A novel PLL method for single-phase system was proposed in [23]. Instead of the general structure, this method generates the orthogonal voltage system, using a structure based on second order generalized integrator. This method is easy to implement and is free from frequency influence.
on the phase error, whereas PLL often focuses on the frequency error. The di↵erence between the two are only an integrator, since phase is the integral of frequency. Thus, the problems are similar. In the following parts, we will consider how the elements of a PLL may be utilized for ESC.
Among the structures mentioned above, the basic PLL could be a suitable choice for dealing with the extremum seeking problem. Most structures dis-cussed above are based on the basic one, e.g., the structures proposed in [20], [19]. The majority of them enhanced one or two basic elements to improve the performance of the PLL, while others carried out di↵erent structures, e.g., the structure of Kalman filter, to achieve better performance. Among all the structures, the basic structure is the simplest one. The novel ESC focuses on locking the phase at a certain value, for which, the basic struc-ture of PLL is enough. Therefore, it is not necessary to apply more complex structures in the novel loop.
3.1.1
Adapting a PLL for ESC
Summarizing the discussion above, the ESC in the structure of PLL works as follows:
1) The target system gives the output signal to the phase detector;
2) The phase detector part estimates and tracks the varying phase accurately, and pass the phase information to the loop filter;
3) The loop filter adjusts the input signal of the target system with the phase information;
4) The phase of the output will be changed since the input of the system changes, entering the next loop.
When the phase of the output is equal to the reference phase, the control loop will be locked, at which the optimal operating point is achieved.
3.2
Phase Estimation
An accurate estimation is the base of the control loop since the novel method is phase based. In addition, the operation of the whole loop is based on the estimated result. Therefore, it is important to choose a proper estimator. In this section, we consider several di↵erent methods of phase estimator: Estimation by demodulation, updated estimation from variant of EPLL, and estimation by Kalman filter.
3.2.1
Estimation by Demodulation
We consider the basic estimator applied in the PLL first. Description
The estimator in PLL is usually a multiplier as shown in Figure 3.2. In a basic PLL, we have
Uref(t) = A1sin(!1t + 1),
Uo(t) = A2sin(!2t + 2),
(3.1) where Uref(t) is the reference signal to which we compare the phase of the
output, Uo(t) is the output signal for which we want to estimate the phase.
The output of the multiplier y(t) is
y(t) = Uref(t)⇥ Uo(t) = A1A2sin(!1t + 1) sin(!2t + 2),
=A1A2
2 [cos((!1 !2)t + 1 2) cos((!1+ !2)t + 1+ 2)].
Figure 3.2: The standard estimator in PLL.
When the frequency of the two input signals are equal to each other, e.g., !1 = !2 = !, the output of the multiplier is
y(t) = A1A2
2 (cos( 1 2) cos(2!t + 1+ 2). (3.3) High frequency term in the output of the multiplier y(t), i.e., cos(2!t + 1+ 2), is the double frequency error mentioned in the previous section. The
high frequency element could be attenuated by adding a low-pass filter with break o↵ frequency lower than 2!. Assume the filter is an ideal low-pass filter, we have
ˆerror = A1A2
2 cos( 1 2). (3.4)
Therefore, the phase di↵erence between two signals can be estimated. In the novel ESC, we aim at adjusting the phase di↵erence between the perturbation and the system output to the value ⇡/2. We could set the perturbation signal as the reference signal Uref(t), i.e.,
Uref(t) = a sin(!t), (3.5)
such that the structure is applied in a standard way. As the frequency of the perturbation signal is set by the user, the frequency of the output signal can be obtained if a suitable band-pass filter is added. The frequency of the output signal could be adjusted to be the same as the perturbation signal Uref(t), if the band-pass filter is designed to eliminate all the other
would be
ˆerror = aA
2 (cos( ) cos(2!t + )). (3.6) When an ideal low-pass filter is added, the high frequency term will be elim-inated and the estimation of could be obtained.
Properties
This demodulation method is able to estimate the phase in a simple way. However, we might encounter problems with this method. For example, we might get the double frequency errors by just multiplying two signals together. However, these errors may be attenuated by adding a low-pass filter after the multiplier. Another problem is that it is impossible to get the phase directly. As shown in Equation 3.3, what we can deal with is only the whole output signal, i.e., A1A2
2 cos( 1 2), instead of the phase information 1 2. Both the value of amplitude A12A2 and the value of phase 1 2 will
influence the output signal of the phase detector. Thus, when the amplitude
A1A2
2 is small, the value of the signal will be small as well, which may slow
down the converge speed.
3.2.2
Updated Estimation from Variant of EPLL
In [19], a new method for phase detection is proposed. We will discuss it in detail in following parts.
Description
The structure of the phase detector is shown in Figure 3.3, where Uref is the
reference signal, A represents the estimated amplitude, Uo is the output of
VCO and e is the intermediary signal. The multiplier in conventional PLL is replaced by three multipliers, one integration, one subtraction and a phase-shift of 90 degrees. Instead of multiplying the input signal by the output of VCO, a refined variant of the VCO signal is subtracted from the input signal to produce an intermediary signal. Then the intermediary signal is multiplied by the output of VCO. The estimated amplitude is the output of the integration block of phase detector. The output of the loop filter is the estimated time derivative of the total phase.
Properties
Figure 3.3: A new estimator in PLL.
not only locked, but also equal. Moreover, this method can immune noise and is robust with respect to externally imposed conditions. However, the calculation of the phase information is related to the estimated amplitude. Any change on amplitude will influence the estimation of the phase, which may lead to a poor tracking ability of the phase.
3.2.3
Estimation by Kalman Filter
Estimation using Kalman filter is another alternative. We are going to in-troduce how to estimate the phase-lag by Kalman filter in detail.
Description
Consider the nonlinear plant that we try to optimize, is operated with an input signal
✓ = ✓0+ a sin(!t). (3.7)
Assume that the target system is stable, then the output in the stationary situation is periodic, with the same base-frequency as the perturbation signal. The output can hence be described by a Fourier series expansion
y(t) = c0+ n
X
k=1
where c0 represent the constant part in y(t), Ak, Bkand k are the coefficients.
The higher harmonics in the equation are generated by the nonlinearity and they should be small if the amplitude of the input signal a is small.
At this point, the phase-lag is given by the fundamental components. Therefore, we have
y(t) = A sin(!t) + B cos(!t) = C sin(!t + ), (3.9) where
C = pA2+ B2, = arctanB
A. (3.10)
Therefore, the phase of the output could be calculated if the value of the coefficients A and B could be estimated.
A Kalman filter can be utilized to estimate these coefficients. It assumes that the model of a system is in the form
˙x(t) = F (t)x(t) + B(t)u(t) + w(t), w(t)⇠ N(0, Q),
y(t) = H(t)x(t) + v(t), v(t)⇠ N(0, R), (3.11) where u(t) is the control input, y(t) is the system output, w(t) is the process noise, v(t) is the measurement noise, x(t) is the state of the system, F (t) and B(t) are possibly time-varying and describe the state dynamics, H(t) is the measurement model, Q and R are covariance matrix of process noise and measurement noise respectively.
The predict-update model of Kalman filter is
˙ˆx(t) = F (t)ˆx(t) + B(t)u(t) + K(t)(y(t) H(t)ˆx(t)), ˙
P (t) = F (t)P (t) + P (t)F (t)T K(t)H(t)P (t) + Q, K(t) = P (t)H(t)TR 1,
(3.12)
where K(t) is the Kalman Gain.
The initialization of Kalman filter is ˆ
x(t0) = E[x(t0)],
P (t0) = V ar[x(t0)],
(3.13) where ˆx(t0) is the estimation of x(t) at time t0, P (t0) is the covariance matrix
of x(t) at time t0, E[·] represents the expected value and V ar[·] represents
the covariance.
section, the phase-lag is mainly given by the fundamental waveform and harmonics contribute little on the phase-lag. Therefore, the observed model should mainly be based on the fundamental waveform, i.e.,
y(t) = A sin(!t) + B cos(!t). (3.14) However, if the true signal is more complex than the fundamental waveform, the Kalman filter will try to fit this complex behaviour into just A and B, which might lead to poor estimation. Therefore, it could be useful to have additional terms in the Kalman filter. Then, the model of the output could be set as
y(t) = c0 + A1sin(!t) + B1cos(!t) + A2sin(2!t) + B2cos(2!t). (3.15)
We set the state in Kalman filter as x1 = c0, x2 = A1, x3 = B1, x4 = A2, x5 = B2. (3.16)
Since we assume all the coefficients are constant and fixed, we set the process model in the Kalman filter as
F (t) = 2 6 6 6 6 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 7 7 7 7 5. (3.17) B(t) = 0. (3.18)
The model of the output is set as
H(t) =⇥ 1 sin(!t) cos(!t) sin(2!t) cos(2!t) ⇤. (3.19) With proper F , B, H selected, we can represent y(t) in the form assumed by the Kalman filter. Then, the predict-update model can be written as
˙ˆx(t) = K(t)(y(t) H(t)ˆx(t)), ˙
P (t) = K(t)H(t)P (t) + Q, K(t) = P (t)H(t)TR 1,
H(t) =⇥ 1 sin(!t) cos(!t) sin(2!t) cos(2!t) ⇤.
All elements in Equation 3.20 are settled except the terms Q and R. We will discuss the choice of Q and R in Chapter 4.
The amplitude C and phase of the output can be calculated using Equation 3.10, and we could get the amplitude and phase as
C = q ˆ x2(t)2+ ˆx3(t)2, = arctanxˆ3(t) ˆ x2(t) . (3.21)
It is also possible to have other useful filters to filter out elements, such as bias terms, higher order harmonics, etc., depending on how we design the filter. A band-pass filter, with a lower cuto↵ frequency !L and a upper
cuto↵ frequency !H, could be added before the Kalman filter to allow a
lower order model for the Kalman filter. If the cuto↵ frequencies are set as !L < ! < !H, the output of the band-pass filter, i.e., the input of the Kalman
filter, contains only the elements with basic frequency. Therefore, the band-pass filter can modify the signal to make it suitable for the model of Kalman filter. In addition, higher order harmonics could be eliminated by the band-pass filter, which allows a lower order model for Kalman filter. However, if a band-pass filter is added, it will add a bias to the phase estimation. That is, when the output of the band-pass filter becomes the input of the Kalman filter, the phase-lag of the output of the target system true is not equal to
the estimated phase-lag provided by the Kalman filter any more. Instead,
true will be the combination of the estimated phase-lag from the Kalman
filter and the phase-lag of the band-pass filter. The estimated phase-lag
estimated is calculated from Equation 3.21. The phase-lag of the band-pass
filter band pass can be calculated since all coefficients in the band-pass filter
is set by users. Therefore, the true phase-lag is
true= estimated band pass. (3.22)
Properties
filter. Parameters, e.g., Q, R, can influence the performance of the estimator a lot. If parameters are not properly set, poor estimation may be obtained.
3.3
Controller
Once the phase-lag of the output is estimated, it should be compared with the reference phase. The controller should try to drive the operating point to a point, where the local phase-lag is ⇡/2 or ⇡/2. In order to make the control target unique, we may introduce a nonlinear transformation of the control input. We can chose a cosine function, an absolute function or any other even function to make the control target unique, as long as the reference is adjusted accordingly. In order to make the control loop simple, a simple even function is preferred. In this thesis, we choose the cosine function to make the control target unique. When the cosine function is applied, the controller should try to control the input of the controller to be zero. Other even functions, e.g., the absolute function, can also be an alternative.
As to the controller itself, there are many choices, e.g., proportional con-troller, integral concon-troller, PI concon-troller, PID concon-troller, etc.. In order to achieve good accuracy, the controller should achieve a zero static error. Ad-ditionally, the controller must be robust since the gain of the system varies a lot. Here, the gain means the ration of the change in the local phase-lag to the change in the input. Moreover, as mentioned in Section 3.1.1, the controller should work slower than the phase estimator, which indicates the bandwidth of the controller should be low compared to the phase estimator. Therefore, a simple controller might be sufficient. An integral controller is usually sufficient since it can always eliminate the steady-state error for the system. However, we should note that an integral controller is not the only choice, other controllers are also possible to do the control as well.
3.4
Selection of the Controller Structure
We have introduced di↵erent possible phase estimation schemes and con-trollers. In this section, we will compare them and choose one combination for the control loop.
the value of estimated amplitude may influence the whole control loop. The estimation method used in EPLL is able to provide the estimated phase in-dependently. However, the calculation of the phase information is related to the estimated amplitude, which means the value of estimated amplitude will influence the estimation of phase. This may make it hard to track the phase accurately. As for the Kalman filter, it can estimate both the phase and amplitude and get the value independently, which implies that the value of estimated amplitude will not influence the performance of the whole control loop. In addition, it can track variables well. One important characteristics of the estimator required is to estimate the phase accurately. The phase-lag of the target system is the base in the novel ESC method. Hence, it is pre-ferred to obtain the phase information without the influence of amplitude. Comparing the three methods, the first one is the worst, since its output is the combination of both amplitude and phase. Therefore, the second and third methods are preferred. Another important characteristics of the esti-mator is that the estiesti-mator should be able to track the phase. As mentioned before, the Kalman filter is able to track the phase well, while the method used in EPLL may have some problem in tracking. Therefore, the estimation method using Kalman filter is preferred to be the phase estimator in the ESC loop.
As for the controller, many structures are available. Simple controllers might be sufficient, since bandwidth of the controller should be low compared to the estimator. Therefore, we consider simple controllers, e.g., proportional controller and integral controller. Both of the two controllers could achieve zero static error if the gain of the system, i.e., the ratio of the change in the local phase-lag to the change in the input, is a constant. However, the gain of the system varies a lot in real situation. Then, proportional controller cannot achieve zero static error unless the proportional gain is changed. Therefore, proportional controller is not suitable in the ESC. On the other hand, an integral controller can achieve zero static error. Therefore, it can be a suitable choice for the ESC system. In the integral controller, the bandwidth is equal to the integral gain, which gives a hint of how the integral gain should be tuned. The tuning of the integral gain will be discussed in detail in the next chapter. Other controllers, e.g., PI controller, PID controller, etc., are also able to achieve zero static when the gain varies, but they might bring extra complexity to the design. Since a simple controller can control the loop well, there is no need to use a controller which adds more complexity to the design. Then, the final structure of the control loop is shown in Figure 3.4. The control loop works as follows:
Step 1. Initial value ˆ✓0 is set first. Then, the input ✓ is put into the target
Step 2. The output of the target system goes into the band-pass filter. After the filter, all the higher order harmonics, as well as the constant term are attenuated.
Step 3. The phase estimator, i.e., the Kalman fliter, estimates the phase of the output estimator with the model illustrated in Equation 3.15. The phase
of the band-pass filter band passis already known, since all the parameters in
the band-pass filter are set by users. According to Equation 3.22, the phase of the output true can be calculated.
Step 4. The nonlinear transformation function, i.e., the cosine function, deals with the input of the controller to make the control target unique. Step 5. The controller, i.e., the integral controller, moves the operating point of the target system towards the optimal operating point.
Chapter 4
Controller Tuning
There are several parameters that can be adjusted in the novel ESC loop: (1) Parameters in Kalman filter: the model of Kalman filter, measurement noises covariance R and process noises covariance Q; (2) Prameters in controller: the controller gain k; (3)Parameters in perturbation signal: the amplitude of the signal and the frequency of the signal. The values of these parameters may significantly influence the performance and stability of the controller. Therefore, it is important to investigate the e↵ects of these parameters and provide insight into how to select them.
4.1
Kalman Filter
Kalman filter is utilized to estimate the phase of the output and the estimated result is fed to the controller to control the target system. The parameters in the Kalman filter will influence the accuracy of the estimation and speed of the estimation, which significantly impacts the overall performance of the method.
We will discuss the model of Kalman filter first. A proper model of the input signal of the Kalman filter is essential for an accurate estimation. If the real signal is much more complex than the model, the estimator will try to put all the information to the limited model, which may lead to inaccurate estimation. As we assumed in section 3.2.3, the output y is periodic with the same base-frequency as the perturbation signal. The higher harmonics in the equation are generated by the nonlinearity and they should be small if the amplitude of the input signal a is small. Therefore, the phase-lag is mainly given by the components with frequency ! and we can write the input signal of the Kalman filter as
where
C =pA2+ B2, = arctanB
A. (4.2)
However, this model is valid only in the stationary point and that it is not able to capture transient behaviour in the system. Since the phase-lag is mainly given by the fundamental wave, a band-pass filter can be employed to allow a lower order model in Kalman filter.
With the band-pass filter, we can modify the input to the estimator to make the signal more suitable for the model of Kalman filter. As discussed in section 3.2.3, we choose
y(t) = A1sin(!t) + B1cos(!t) + A2sin(2!t) + B2cos(2!t) + c0 (4.3)
as the model of the input signal. The parameters of the band-pass filter should be chosen depending on the perturbation frequency utilized.
Consider a band-pass filter that can eliminate signals with frequency lower than !low and higher than !high. An ideal band-pass filter has a complete
flat passband and signals with frequencies within the passband will be kept completely. All the signals with frequencies outside the passband will be completely attenuated. However, there are no ideal band-pass filters in prac-tice. There is a region outside the passband where frequency are attenuated, but not rejected, which is called the filter roll-o↵. Therefore, we should take roll-o↵ into consideration when choosing the cut-o↵ frequency of the band-pass filter. In order to keep the intended information and achieve better estimation on Kalman filter, we require that: (1) !low should be lower than
or equal to the frequency of the perturbation signal !; (2) !high should be
higher than or equal to the frequency ! and lower than the frequency 2!, i.e., ! !high < 2!. Since the phase-lag is mainly given by the terms with
basic-frequency, we should leave the components with frequency !. There-fore, we could choose !low = !high = !, such that the signal with frequency !
would be kept due to the roll-o↵. Therefore, the model of the Kalman filter can be suitable for di↵erent systems, since high order harmonics as well as the bias are attenuated. In addition, the bias introduced by the band-pass filter could be easily calculated and removed since all filter parameters as well as the frequency are user determined.
With the model set, the matrices F (t), B(t), H(t) are settled as well. The covariance matrices Q, R and the initial conditions have to be determined by the user. We try to figure out how to set these values. By substituting
K(t) = P (t)H(t)TR 1, (4.4) into ˙P (t), we can get
˙
Since all the states in Kalman filter are assumed as constant, we have F (t) = 2 6 6 6 6 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 7 7 7 7 5. (4.6)
Therefore, Equation 4.5 can be simplified as ˙
P (t) = P (t)H(t)TR 1H(t)P (t) + Q. (4.7) P (t) will probably oscillate since H(t) contains sinusoids. From Equation 4.7, we can find that the increase in Q and R will result in the increase in the value of ˙P (t). If P (t) oscillates, a larger P (t) might imply a larger˙ amplitude. This in turn would mean that the covariance of the estimated error will oscillate with a larger amplitude. Then the Kalman filter may be unstable and cannot achieve an accurate estimation. Therefore, the value of Q and R should not be large in order to make the estimator stable. As the value of P (t) may oscillate, it may be easier to analyse the estimator with the average value of P (t). According to Equation 4.4, the value of K(t) would be influenced by the value of P (t), H(t) as well as the value of R. H(t) is a part of the model of the Kalman filter and P (t) is the error covariance matrix, which is decided by the estimated state. Therefore, for K(t), R is the only element that can be adjusted. From Equation 4.4, we can find that when R decreases, the value of K(t) will increase and vice versa. Since the value of F (t) and B(t) are set to zero in the model of Kalman filter, we have
˙ˆx(t) = K(t)(y(t) H(t)ˆx(t)). (4.8) We can find that K(t), the Kalman gain, will influence the converge rate of the estimated value ˆx(t). With larger K(t), ˆx(t) may converge faster. Since R is the covariance matrix of measurement noise, with smaller R, the measurements are trusted more. If the model is sufficiently accurate, this may lead to a faster convergence.
Figure 4.1: Two di↵erent situations of the phase-shift.
4.2
Controller
The integral controller has the transfer function G(s) = k
s. The sign and the
value of k would influence the performance of the controller.
We will investigate how the sign of the control gain is related to the system being controlled first. It is discussed in the section 2.2 that z(✓) will pass the imaginary axis when the system goes through the optimal working point ✓⇤. However, we cannot predict what direction z(✓) will go, when it crosses the imaginary axis. According to the characteristics of di↵erent systems, the phase-shift can be divided into two cases, as shown in Figure 4.1. We cannot be certain in which direction the phase-lag will change when we move the operating point. If the sign of control gain k is not chosen to match the target system, we will get an unstable closed loop and the operating point will never move to the optimal working point. Therefore, we could perform a quick initial experiment to determine the sign of k.
estimator. On the other hand, the value of k will influence the stability of the system. If a large k is chosen, the output of the controller may oscillate more, which may make the system unstable. If the value of k is small, the converge rate will be small, which may slow down the converge speed of the whole control loop. Therefore, there is a trado↵ between the speed and the robustness.
4.3
Perturbation Signal
The perturbation signal has two parameters, one is perturbation frequency and the other is amplitude.
As discussed in Chapter 3, the frequency of the input will influence the phase-lag of the output of the target system. According to section 2.2, the phase-lag of the system is
= arg(G0(i!)) + arg(i! + z(✓)) = g0+ arctan
!
z(✓) + kG0⇡, (4.9) where
kG0 = 0,±1, ±2, · · · . (4.10) With lower frequency, the value of g0 will be close to zero. Then the
phase-lag of the system is mainly based on the value of arctan !
z(✓). As introduced
in section 2.2, z(✓) will be zero when the operating point reaches the op-timum working point. Therefore, at the opop-timum, the value of arctan( !
z(✓))
will be close to±⇡/2. Hence, the phase-lag will be close to ⇡/2+kG0⇡, kG0 = 0,±1, ±2, · · · at the optimal operating point for low frequency. If the per-turbation frequency is large, the value of g0 cannot be ignored. As for the
arg(i! + z(✓)), with larger !, there will be a wider range of inputs ✓ that can make arg(i! +z(✓)) close to±⇡/2. As a result, the phase-lag at the optimum will be g0+±⇡/2 + kG0⇡ instead of ⇡/2 + kG0⇡, where kG0 = 0,±1, ±2, · · · . Hence, it is hard to locate the real optimal operating point with the reference phase ±⇡/2 when the perturbation frequency is large. In order to locate the optimal operating point with more accuracy, a lower perturbation frequency should be set.
The perturbation frequency will influence the converge speed of the sys-tem as well. With higher perturbation frequency, the bandwidth of the es-timator can be larger, which allows a faster controller. Hence, the converge rate of the whole system can be larger than the system with lower pertur-bation frequency. Therefore, a higher frequency can be used first to locate a working point ✓⇤
Figure 4.2: Di↵erent phase error at di↵erent frequency. the optimal working point u⇤
1 and the frequency can be decreased. By
re-ducing the perturbation frequency step by step, we can achieve the optimal operating point with less error and at a higher speed.
However, lower frequency may have a higher converge rate in some special situations. When the frequency is the only di↵erence, the lower frequency system may converge faster than the higher frequency system. It might be the phase error that result in this situation. The phase error between the true phase and the reference phase can be larger for lower frequency at the same working point, as shown in Figure 4.2. As the integral controller gain are the same, the converge speed will be faster for the system with larger phase error. Therefore, a system with lower frequency can converge faster than a higher frequency system.
input signal ✓. This may lead to a large fluctuation on the output signal, which is not preferred in some applications. Hence, the amplitude should be small. However, in a real system, noise always exists. Thus, the amplitude a should be large in order to distinguish the perturbation signal from the noise. In other words, the amplitude should be large enough to obtain a large signal to noise ratio.
4.4
Summary
According to the discussion above, some conclusions can be made.
1) With larger Q and R, the value of P (t) might oscillate with large ampli-tude. The value of QR will influence the performance of the estimator. If R is smaller than Q, measurement is trusted more and vice verse. If the model of is sufficiently accurate, smaller R may lead to a faster convergence. 2) The sign of the control gain k should be determined by a small experiment. Moreover, the value of K should be chosen carefully to make the controller work both fast and stable.
Chapter 5
Example and Analysis
In this chapter, an example is given to show how the control loop works. The following system comes from [4].
Example: Isothermal CSTR: Consider an isothermal perfectly mixed tank reactor with two consecutive reactions A! B, 2B ! C, with standard mass action kinetics
V ˙cA = F (cAf cA) V k1cA, (5.1)
V ˙cB = F cB+ V k1cA V k2c2B, (5.2)
where cA and cB are concentrations of A and B, respectively, cAf is the
con-centration of input flow. The parameters are V = 1.0, cAf = 1.0,k1 = 2.0
and k2 = 0.1.
The steady-state input-output relationship is shown in Figure 5.1. The optimal working point is around ✓⇤ = 0.375. We will now assume that the model is unknown and apply the ESC algorithm designed in Chapter 3 in order to locate the optimum.
In the example, some parameters are set first:
The measurement noises covariance matrix is set as R(t) = 0.1. A low-pass with break-o↵ frequency !l = 1.3! and a high-pass filter with break-o↵
frequency !h = 0.8! are employed to combine a band-pass filter, where ! is
the input frequency. The process noises covariance matrix is set as
Figure 5.1: The steady-state input-output realtionship.
With these parameters settled, we could run the control loop and analyze the performance.
5.1
Performance Test
Perturbation signals with di↵erent frequencies are tested in this example. The result is shown clearly in Figure 5.2. The comparison result is shown in Table 5.1. It is obvious that e1 < e2 < e3 < e4. We can draw the conclusion
that with lower frequency, the final control result will be closer to the exact optimal working point for this system, the same as discussed in Section 4.3. In addition, it can be inferred from the figure that as the frequency increases, the speed of converge decreases as discussed in the previous chapter.
Frequency ! Output of the controller ˆ✓ Steady-state error e = ✓ ✓⇤
0.02 0.3789 e1 = 0.0039
0.1 0.3792 e2 = 0.0042
0.2 0.3588 e3 = 0.0162
0.3 0.3138 e4 = 0.0612
Figure 5.2: Control result with frequency ! = 0.3, ! = 0.2, ! = 0.1 and ! = 0.02, with k = 0.0001.
Figure 5.3 shows that the phase error is larger for lower frequency at the same working point as we discussed in Chapter 4, which may lead to a faster convergence for lower frequency when other parameters are the same.
In Chapter 4, we suggested that an approach to achieving both fast con-vergence and accuracy would be to initially set the perturbation frequency high with a correspondingly aggressive tuning. And then, we successively detune the frequency as well as the controller, in order to eventually achieve accuracy. As shown in Figure 5.4, the blue line shows the result that the sys-tem starts with frequency ! = 0.1, controller gain k = 0.001 before t = 1500. When t reaches 1500, the system will change the frequency to ! = 0.02 and the controller gain will be k = 0.0001. The red line shows the result with frequency ! = 0.02 and controller gain k = 0.0001 throughout the entire sim-ulation. It is shown clearly that the system converges faster in first situation if other parameters are adjusted properly.
Interestingly, we find that the value of k cannot be increased much when ! = 0.02 as is illustrated in Figure 5.5. Comparing the curve in Figure 5.5, the only di↵erence between these two curves is the value of controller gain k. With certain perturbation frequency, large value of k may introduce instability to the system, as discussed in the Controller Tuning part.
Figure 5.3: Phase-lag with frequency ! = 0.3, ! = 0.2, ! = 0.1 and ! = 0.02.
Figure 5.5: Control result with frequency ! = 0.02 control constant k = 0.001 VS. the result with frequency ! = 0.02 and control constant k = 0.01 ESC. In the classic ESC, the control gain k is set to 100 to obtain a large convergence rate; the cuto↵ frequency of the low-pass filter is set to !l= 0.3!;
the cuto↵ frequency of the high-pass filter is set to !h = 0.6!. The result
is shown in Figure 5.6. It implies that in this particular case the system controlled by classic ESC converges slower than the one controlled by the phase-based ESC method. The steady-state error in the classic ESC is 0.0159 and the error in the phase-based ESC is 0.0042, which indicates that the phase-based method achieves a result with less steady-state error than the classic one.
5.2
Robustness Test
In order to test the robustness of the controller, we perform two di↵erent tests below.
Figure 5.6: The di↵erent control results by di↵erent ESC methods with fre-quency ! = 0.1
controller successfully adjusts the input in order to compensate for the added disturbance. Therefore, we can draw the conclusion that the novel extremum seeking controller has good robustness for disturbances in this situation.
Then comes the test where the first order inertial element is added to the system. The output in s domain becomes Y (s) = xbT s+1ki in the test case,
where xb represents cB. We set T = 1, ki = 0.1, and the result is shown in
Figure 5.9. We can see from the result that the controlled output converges to a certain value. With the same frequency and control gain, the converge rate is almost the same as the system without the inertial element. The steady-state error under this situation, i.e., e = 0.0228, is a little bit larger than the error without the first order inertial element. This result indicates that the control loop can deal with inertial element well and the operating point can be moved to a steady value, which is close to the optimal operating point.
5.3
Summary
Figure 5.7: The result of the controller output when disturbance d = 0.1 added at time 7000 with frequency ! = 0.1.
Chapter 6
Conclusions and Further
Research
6.1
Disscussion and Conclusion
A phase-based extremum seeking control loop is designed in this thesis report. The structure of PLL is employed to lock the control loop when the phase-lag is±⇡/2. Kalman filter is employed to ensure an accurate estimation. To achieve a zero steady-state error, an integral controller is utilized.
The impact of the parameters in the control loop are analyzed, as well as a guide for controller tuning. Simulations shown in Chapter 5 imply that this model free method can move the operating point of the target system to the optimum. In addition, this control loop is robust for both disturbance and inertial terms.
6.2
Future Work
The main purpose of this thesis work is to locate the optimal operating point of a dynamic system. Discussions in the previous chapters show that the goal has been achieved with the phase-based method. However, there are still some tasks left.
