' $
Department of Mathematics
Test results from Parallelization of Model
Building Algorithms for Derivative-free
Optimization
Per-Magnus Olsson
LiTH-MAT-R--2014/01--SE
Link¨oping University
Test Results from Parallelization of Model
Building Algorithms for Derivative-free
Optimization
Per-Magnus Olsson
February 13, 2014
1
Abstract
We present results from testing of parallel versions of algorithms for
derivative-free optimization. Such algorithms are required when an analytical
ex-pression for the objective function is not available, which often happens in simulator-driven product development. Since the objective function is un-available, we cannot use the common algorithms that require gradient and Hessian information. Instead special algorithms for derivative-free optimiza-tion are used. Such algorithms are typically sequential, and here we present the first test results of parallelization of such algorithms. The parallel ex-tensions include using several start points, generating several points from each start point in each iteration, alternative model building, and more. We also investigate whether we can generate synergy between the different start points through information sharing. Examples of this include using several models to predict the objective function value of a point in order to prioritize the order in which points are sent for evaluation. We also present results for higher-level control of the optimization algorithms.
2
Introduction
This report contains the test results for testing of parallelization of algo-rithms for derivative-free optimization. We have implemented the parallel
very little analysis in this report, the reader is referred to the dissertation for more analysis and discussion. Thus, this technical report can be seen as an appendix to the dissertation.
Due to the industrial applications that we are interested in, we test the algorithms on low-dimensional problem, where we do not have access to the objective function. Since the objective function is unknown, we do not have access to gradient or Hessian information. The test cases that we use are taken from the MVF test case suite [1].
To each section, there is a corresponding section in the dissertation that discusses the theory and reasoning for the extension, and there is also a section in the test results chapter that discusses and analyzes the test case results,
3
Test Case Specifications
Table 1 displays the test case name, number of dimensions and the intervals for the variables. The intervals are not considered bounds, but are rather used to limit the search to an interesting area. Test cases with variable number of dimensions have been marked with ‘var’ in the dimensions column.
Table 1: Test case specifications.
Name n Interval Characterization
Ackley var [−30.0, 30.0] Highly multimodal, noisy.
Beale 2 [−4.5, 4.5] Unimodal, smooth.
Bohachevsky1 2 [−100.0, 100.0] Multimodal, fairly smooth.
Bohachevsky2 2 [−50.0, 50.0] Multimodal.
Booth 2 [−10.0, 10.0] Multimodal, smooth.
BoxBetts 3 x1 ∈ [0.9, 1.2] Multimodal.
x2 ∈ [9.0, 11.2]
x3 ∈ [0.9, 1.2]
Branin 2 x1 ∈ [−5.0, 0.0] Multimodal with three global minima,
x2 ∈ [0.0, 15.0] smooth.
Branin2 2 [−10.0, 10.0] Multimodal.
Camel3 2 [−5.0, 5.0] Multimodal, smooth.
Camel6 2 [−5, 0, 5.0] Multimodal with two global and
four local minima, smooth.
Chichinadze 2 [−30.0, 30.0] Unimodal, edgy.
Colville 4 [−10.0, 10.0] Multimodal, smooth.
Corana 4 [−100.0, 100.0] Very multimodal.
Eggholder var [−512.0, 512.0] Multimodal, smooth.
FreudensteinRoth 2 [−15.0, 15.0] Multimodal, smooth.
Gear 4 [12.0, 60.0] Multimodal, edgy.
Generalized Rosenbrock var [−100.0, 100.0] Unimodal, smooth.
Goldstein-Price 2 [−2.0, 2.0] Multimodal, smooth.
Hansen 2 [−10.0, 10.0] Multimodal, noisy.
Hartman 3 3 [0.0, 1.0] Multimodal with three local minima.
Hartman 6 6 [0.0, 1.0] Multimodal with six local minima.
Himmelblau 2 [−5.0, 5.0] Multimodal with four global minima,
smooth.
Holzman 2 var [−10.0, 10.0] N/A.
Hosaki 2 x1 ∈ [0.0, 5.0] Smooth.
x2 ∈ [0.0, 6.0]
Hyperellipsoid var [−5.12, 5.12] Unimodal, smooth.
Kowalik 4 [−5.0, 5.0] Unimodal.
Leon 2 [−1.2, 1.2] Unimodal, smooth.
Levy7 var [−32.0, 32.0] Multimodal, smooth.
Matyas 2 [−10.0, 10.0] Unimodal, smooth.
Max mod 2 [−10.0, 10.0] Unimodal, edgy.
McCormick 2 x1 ∈ [−1.5, 4.0] Unimodal, smooth.
x2 ∈ [−3.0, 4.0]
Michalewitz 2 [0, π] Multimodal, smooth.
Multimod var [−10.0, 10.0] Unimodal, smooth.
Neumaier Perm0 var [−1.0, 1.0] N/A.
Neumaier Perm 4 xi ∈ [−i, i] N/A.
Neumaier Power Sum 4 [0.0, 4.0] N/A.
OddSquare5 5 [−5π, 5π] N/A.
OddSquare10 10 [−5π, 5π] N/A.
Plateau 5 [−5.12, 5.12] Unimodal, edgy.
Quartic Noise U var [−1.28, 1.28] Multimodal, noisy.
Rana var [−500.0, 500.0] Highly multimodal, smooth.
Rastrigin 2 2 [−5.12, 5.12] Multimodal, smooth.
Schaffer 1 2 [−100.0, 100.0] Multimodal.
Schaffer 2 2 [−100.0, 100.0] Multimodal.
Schwefel1 2 var [−10.0, 10.0] Unimodal, smooth.
Schwefel2 21 var [−10.0, 10.0] Edgy.
Schwefel2 22 var [−10.0, 10.0] Unimodal, edgy.
Schwefel2 26 var [−512.0, 512.0] Highly multimodal, smooth.
Shekel2 2 [−65.536, 65.536] Multimodal, smooth.
Shekel4 5 4 [0.0, 10.0] Multimodal, smooth.
Shekel4 7 4 [0.0, 10.0] Multimodal, smooth.
Shekel4 10 4 [0.0, 10.0] Multimodal, smooth.
Shubert 2 [−10.0, 10.0] Highly multimodal, noisy.
Shubert2 var [−10.0, 10.0] Highly multimodal, noisy.
Shubert3 2 [−10.0, 10.0] Highly multimodal, noisy.
Sphere var [−10.0, 10.0] Unimodal, smooth.
Sphere2 var [−10.0, 10.0] Unimodal, smooth.
Step var [−100.0, 100.0] Unimodal, edgy.
Stretched V var [−10.0, 10.0] Highly multimodal, smooth.
Sum Squares var [−32.0, 32.0] Unimodal, smooth.
Trecanni 2 [−5.0, 5.0] Unimodal, smooth.
Trefethen4 2 x1 ∈ [−6.5, 6.5] Highly multimodal, noisy.
x2 ∈ [−4.5, 4.5]
Watson 6 [−10.0, 10.0] Multimodal.
4
Test Case and Algorithm Settings
In the testing of the algorithms on the synthetic test cases, we used the
following settings. The start value of the trust region radius ρbeg was set to a
percentage of the variables’ interval. If there were different variable intervals,
then the smallest interval determined the value of ρbeg.
For each test case, the starting point was randomized within a region that allowed the complete model to be within the specified interval for each region. This means that the starting point was randomized within a smaller
region as the value of ρbeg increased. The trust region radius ∆ was set to
We used the three termination criteria: ρk ≤ ρend, at most 1000 NPEs
after building the initial model, and at most 1000 iterations in the main loop. If any of these values were fulfilled, then the algorithm was terminated. This means that if the algorithm failed to converge to a stationary point within the limits on function evaluations and iterations, it will be terminated due to the latter criteria.
We chose n = 4 for the test cases with variable dimensionality.
The testing on synthetic test cases was performed on a PC with 4GB RAM and a dual core Intel i5 CPU running at 2.67GHz. Each core has two hardware threads, therefore the operating system sees the CPU as having four separate cores. For this reason, the computer did most four parallel evaluations of the objective function during testing. However, in our tests we sometimes send more than four points for parallel evaluation. In such cases, the points are evaluated in batches of four at a time. Since we use syn-chronous parallelization in this testing, the algorithm does not know whether all points are really evaluated in parallel.
5
Interpreting the Test Results
Solving the problems in the setting of interest here is actually a multicriteria optimization in itself. It is desirable to find a low objective function value, but since each objective function evaluation is very time-consuming, it is also desirable to use as few function evaluations as possible. Whether a lower objective function value is “worth” a certain number of evaluations is difficult to determine in advance. In the testing that is presented in this chapter, we will often encounter the situation that one parameter setting will give a lower objective function value than another, but the number of required function evaluations is greater. Here we will favor a lower objective function value, even if this requires more evaluations. However, we are not happy with a large increase in the number of evaluations for a very small improvement in the objective function value, so the tradeoff is somewhat arbitrary. In practical applications, this tradeoff must be done by the user, and can depend on e.g. the particular problem at hand, available time and computational resources.
For the cases where there is a single optimization run that generates a single point in each iteration, NPE is always equal to one, so we do not
When we build the initial model, we need to evaluate p1 = 12(n + 1)(n + 2)
points. Therefore, when we test building the initial model in one step, we assume that we can evaluate this many points in parallel, since we must wait until all points have been evaluated before execution can continue.
In the tables, the first column displays the test case name and the second the number of variables. The third column shows the best objective function value. The fourth column shows the number of times one or more points were sent for evaluation (NPE ). The total number of function evaluations (NE ), including building the model, is displayed in the fifth column. The last column shows the total time in milliseconds for each test case.
To display how different settings affect the number of function evalua-tions that are used, we use data profiles [2]. A data profile is a cumulative distribution function, and thus monotonically increasing, step function with range [0,1]. The x-axis of a data profile displays the number of times one or more points were sent for evaluation (i.e. NPE ), and the y-axis displays the fraction of test cases. Each setting produces a data profile, and on each line, each marker symbolizes a particular test case. For each number of NPE , the line’s y-value displays the fraction of test cases that were solved using the specified number of function evaluations. However, an algorithm that is aborted as the number of available NPEs was exhausted is also considered to have solved the problem for the sake of showing the number of NPEs in a data profile. A user can use data profiles to determine which setting is most likely to solve a certain problem, given a budget on NPE , since a line with a higher fraction is likely to be better.
The number of function evaluations measure is a relevant measure of the time for solving the kinds of problems that we are interested in since it is assumed that the function evaluations take the majority of the solution time.
6
Results for Cache
Table 2: Test case results when not using a cache of evaluated points.
Name n f NPE NE Time [ms]
Ackley 4 19.7761 206 219 971.216
Beale 2 4.13235e-014 109 113 558.718
Bohachevsky1 2 2.55351e-015 38 42 100.684 Bohachevsky2 2 5.14588e-013 30 34 125.16 Booth 2 8.4665e-024 27 31 76.6352 BoxBetts 3 5.17169e-014 130 138 923.627 Branin 2 0.397887 35 39 133.006 Branin2 2 6.50309 38 42 116.151 Camel3 2 0.298638 50 54 226.156 Camel6 2 -0.215464 45 49 98.3349 Chichinadze 2 16.1736 104 108 218.808 Colville 4 6.68204e-014 1003 1016 10337.4 Corana 4 448908 184 197 342.891 Eggholder 4 -1627.68 126 139 253.284 Freudenstein-Roth 2 48.9843 68 72 533.732 Gear 4 0.0659891 197 210 138.9
Generalized Rosenbrock 4 3.01903e-012 474 487 2342.92
Goldstein-Price 2 30 38 42 176.886 Hansen 2 -7.61849 31 35 117.358 Hartman 3 3 -3.86278 1003 1011 531.847 Hartman 6 6 -3.322 100 126 503.453 Himmelblau 2 4.17678e-015 21 25 90.2529 Holzman2 4 5.10577e-030 417 430 543.889 Hosaki 2 -1.12779 27 31 137.774 Hyperellipsoid 4 6 175 188 81.4005 Kowalik 4 0.00159405 282 295 1875.76 Leon 2 2.18618e-013 142 146 670.949 Levy7 4 71.6882 213 226 1368.1 Matyas 2 5.5051e-024 66 70 592.809
Max Mod 2 1.25023e-008 69 73 283.705
McCormick 2 -1.91322 19 23 71.9507
Michalewitz 2 -0.801303 78 82 525.176
Multimod 4 2.54012e-008 270 283 472.751
Neumaier Perm0 4 5.13389e-016 188 201 1995.49
Neumaier Perm 4 550.469 276 289 832.372
Neumaier power sum 4 2.40524e-006 1003 1016 4515.54
Odd square5 5 -1.39764 991 1010 3056.73
Plateau 5 47 273 292 123.542 Quartic Noise U 4 1.56688 306 319 401.764 Rana 4 -986.464 1003 1016 3475.22 Rastrigin 4 54.956 133 146 276.105 Rastrigin2 2 23.7316 21 25 57.6509 Schaffer1 2 0.497082 124 128 196.405 Schaffer2 2 0.000224926 84 88 178.262 Schwefel1 2 4 1.88135e-023 216 229 249.798 Schwefel2 21 4 2.59801e-008 207 220 549.3 Schwefel2 22 4 22.0098 178 191 404.499 Schwefel2 26 4 63.2216 102 115 233.779 Shekel2 2 500 143 147 133.917 Shekel4 5 4 -2.68286 119 132 233.644 Shekel4 7 4 -10.4029 68 81 182.498 Shekel4 10 4 -3.83543 73 86 149.148 Shubert 2 -5.25711 18 22 60.0225 Shubert2 4 -6.38963 184 197 898.28 Shubert3 2 -5.25711 18 22 54.0927 Sphere 4 3.36749e-021 175 188 55.3142 Sphere2 4 7.72166e-020 150 163 1054 Step 4 0 261 274 339.453 Stretched V 4 1.4039e-018 167 180 2591.93
Sum Squares 4 2.07396e-024 261 274 121.617
Trecanni 2 3.79429e-015 32 36 282.567
Trefethen4 2 8.62521 23 27 199.812
Watson 6 0.00228767 822 848 6431.26
Table 3: Test case results with a cache of evaluated points.
Name n f NPE NE Time [ms]
Ackley 4 19.7761 162 175 612.437
Beale 2 4.13235e-014 94 98 437.165
Bohachevsky1 2 2.55351e-015 33 37 79.7964
Bohachevsky2 2 5.14588e-013 29 33 144.648
BoxBetts 3 5.17169e-014 96 104 671.245 Branin 2 0.397887 32 36 173.811 Branin2 2 6.50309 37 41 78.6388 Camel3 2 0.298638 49 53 140.772 Camel6 2 -0.215464 44 48 57.0041 Chichinadze 2 16.1736 89 93 147.961 Colville 4 6.68204e-014 195 208 8231.79 Corana 4 448908 130 143 310.236 Eggholder 4 -1627.68 99 112 205.32 Freudenstein-Roth 2 48.9843 55 59 556 Gear 4 0.0659891 113 126 126.578
Generalized Rosenbrock 4 3.01903e-012 444 457 2183.92
Goldstein-Price 2 30 37 41 112.294 Hansen 2 -7.61849 23 27 76.2868 Hartman 3 3 -3.86278 71 79 417.562 Hartman 6 6 -3.322 99 125 327.968 Himmelblau 2 4.17678e-015 19 23 60.0356 Holzman2 4 5.10577e-030 356 369 549.148 Hosaki 2 -1.12779 23 27 70.6461 Hyperellipsoid 4 6 92 105 65.5855 Kowalik 4 0.00159405 248 261 1178.62 Leon 2 2.18618e-013 134 138 523.349 Levy7 4 71.6882 211 224 1034.59 Matyas 2 5.5051e-024 37 41 328.191
Max Mod 2 1.25023e-008 69 73 157.588
McCormick 2 -1.91322 18 22 60.9303
Michalewitz 2 -0.801303 62 66 443.671
Multimod 4 2.54012e-008 270 283 493.392
Neumaier Perm0 4 5.13389e-016 156 169 1350.39
Neumaier Perm 4 550.469 269 282 614.911
Neumaier power sum 4 2.40524e-006 1003 1016 6110.75
Odd square5 5 -1.39764 828 847 3735.57
Odd Square10 10 -1.3585 1003 1067 16665.8
Plateau 5 47 153 172 372.46
Quartic Noise U 4 1.56688 238 251 568.27
Rastrigin 4 54.956 107 120 304.822 Rastrigin2 2 23.7316 20 24 59.6953 Schaffer1 2 0.497082 113 117 219.319 Schaffer2 2 0.000224926 84 88 185.462 Schwefel1 2 4 1.88135e-023 126 139 276.079 Schwefel2 21 4 2.59801e-008 207 220 746.325 Schwefel2 22 4 22.0098 178 191 484.292 Schwefel2 26 4 63.2216 83 96 264.636 Shekel2 2 500 97 101 155.32 Shekel4 5 4 -2.68286 108 121 267.462 Shekel4 7 4 -10.4029 66 79 174.668 Shekel4 10 4 -3.83543 67 80 144.273 Shubert 2 -5.25711 17 21 57.0126 Shubert2 4 -6.38963 152 165 997.928 Shubert3 2 -5.25711 17 21 62.8057 Sphere 4 3.36749e-021 92 105 67.0703 Sphere2 4 7.72166e-020 85 98 869.86 Step 4 0 199 212 405.918 Stretched V 4 1.4039e-018 121 134 2394.06
Sum Squares 4 2.07396e-024 152 165 198.416
Trecanni 2 3.79429e-015 31 35 212.628
Trefethen4 2 8.62521 22 26 97.9253
Watson 6 0.00228767 748 774 6397.14
7
Results When Building the Initial Model
in one Step
Table 4: Test case results when building the initial model in one step.
Name n f NPE NE Time [ms]
Ackley 4 19.7465 155 169 437.708
Beale 2 1.22194e-013 76 81 350.753
Bohachevsky1 2 5.55112e-016 42 47 114.372
Bohachevsky2 2 6.16174e-015 28 33 83.7518
Booth 2 3.50421e-024 16 21 62.3746 BoxBetts 3 6.65031e-015 97 106 443.326 Branin 2 0.397887 34 39 142.058 Branin2 2 0.102163 29 34 101.545 Camel3 2 0.298638 47 52 202.839 Camel6 2 2.10425 50 55 140.439 Chichinadze 2 -42.8686 79 84 1605.95 Colville 4 3.676 362 376 1573.83 Corana 4 473217 120 134 318.242 Eggholder 4 -1627.68 97 111 329.16 Freudenstein-Roth 2 48.9843 57 62 604.34 Gear 4 0.0659891 115 129 139.011
Generalized Rosenbrock 4 2.08716e-013 516 530 2533.62
Goldstein-Price 2 30 38 43 155.208 Hansen 2 -7.61849 15 20 81.3782 Hartman 3 3 -3.86278 66 75 375.183 Hartman 6 6 -3.322 111 138 352.196 Himmelblau 2 3.19638e-014 21 26 58.5012 Holzman2 4 1.898e-031 644 658 993.327 Hosaki 2 -1.12779 41 46 149.071 Hyperellipsoid 4 6 91 105 76.1112 Kowalik 4 0.00159405 246 260 1831.26 Leon 2 1.3682e-014 108 113 540.496 Levy7 4 -54.5009 939 953 2805.52 Matyas 2 2.01183e-023 36 41 322.188
Max Mod 2 7.33974e-009 72 77 193.625
McCormick 2 -1.91322 17 22 71.6312
Michalewitz 2 -0.801303 69 74 668.667
Multimod 4 6.2837e-009 216 230 563.622
Neumaier Perm0 4 1.5771e-014 180 194 1124.48
Neumaier Perm 4 550.469 246 260 754.06
Neumaier power sum 4 4.69884e-007 1002 1016 5363.53
Odd square5 5 -1.39764 494 514 1780.18
Odd Square10 10 -1.19557 1002 1067 12288.5
Plateau 5 47 135 155 263.837
Rana 4 -986.464 60 74 208.111 Rastrigin 4 50.2302 113 127 199.718 Rastrigin2 2 23.7316 19 24 63.4748 Schaffer1 2 0.497371 99 104 259.169 Schaffer2 2 0.000857251 78 83 148.197 Schwefel1 2 4 4.37302e-022 106 120 4549.65 Schwefel2 21 4 1.29166e-008 243 257 411.284 Schwefel2 22 4 22.3562 156 170 440.989 Schwefel2 26 4 63.2216 83 97 625.747 Shekel2 2 500 57 62 360.409 Shekel4 5 4 -10.1532 93 107 304.697 Shekel4 7 4 -10.4029 96 110 226.111 Shekel4 10 4 -10.5364 70 84 102.469 Shubert 2 -5.25711 19 24 42.1211 Shubert2 4 -6.38963 184 198 995.106 Shubert3 2 -5.25711 19 24 46.4376 Sphere 4 3.44782e-021 91 105 77.673 Sphere2 4 1.22867e-020 74 88 818.077 Step 4 0 158 172 229.101 Stretched V 4 5.46995e-015 137 151 1073.29
Sum Squares 4 3.94897e-023 119 133 4820.66
Trecanni 2 8.65569e-017 36 41 121.021
Trefethen4 2 8.12208 26 31 85.6554
Watson 6 0.00228767 742 769 5764.82
8
Results When Varying the Initial Trust
Re-gion Radius
Figure 1 shows the data profiles for all tested values of the initial trust region radius.
0 200 400 600 800 1000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1% 2% 5% 10% 15% 20% 25% 30%
−20 0 20 40 60 80 −60 −40 −20 0 20 40 60
Objective function value with ρbeg = 1%
Objective function value with
ρbeg
= 2%
(a) With ρbeg= 2%.
−20 0 20 40 60 80 −80 −60 −40 −20 0 20 40 60
Objective function value with ρbeg = 1%
Objective function value with
ρbeg = 5% (b) With ρbeg = 5%. −20 0 20 40 60 80 −60 −40 −20 0 20 40 60 80
Objective function value with ρbeg = 1%
Objective function value with
ρbeg = 10% (c) With ρbeg = 10%. −20 0 20 40 60 80 −80 −60 −40 −20 0 20 40 60 80
Objective function value with ρbeg = 1%
Objective function value with
ρbeg = 15% (d) With ρbeg = 15%. −20 0 20 40 60 80 −60 −40 −20 0 20 40 60 80
Objective function value with ρbeg = 1%
Objective function value with
ρbeg
= 20%
(e) With ρbeg = 20%.
beg
Name n f NPE NE Time [ms]
Ackley 4 19.8148 162 175 434.588 Beale 2 2.62455e-014 77 81 354.891 Bohachevsky1 2 2.93099e-014 46 50 86.155 Bohachevsky2 2 3.88578e-016 34 38 88.5371 Booth 2 8.26182e-022 40 44 181.093 BoxBetts 3 3.77925e-014 94 102 833.488 Branin 2 0.397887 27 31 118.685 Branin2 2 0.102163 30 34 87.875 Camel3 2 0.298638 53 57 189.719 Camel6 2 -0.215464 39 43 127.414 Chichinadze 2 -41.835 78 82 153.57 Colville 4 2.7466e-013 429 442 1234.97 Corana 4 407638 150 163 286.342 Eggholder 4 -1627.68 129 142 592.525 Freudenstein-Roth 2 48.9843 65 69 296.633 Gear 4 1.21644e-007 148 161 206.54
Generalized Rosenbrock 4 2.32861e-013 392 405 2019.3
Goldstein-Price 2 30 32 36 132.059 Hansen 2 -7.61849 18 22 95.9648 Hartman 3 3 -3.86278 58 66 439.732 Hartman 6 6 -3.322 65 91 309.338 Himmelblau 2 2.14177e-015 19 23 49.3563 Holzman2 4 3.97317e-030 267 280 418.919 Hosaki 2 -1.12779 52 56 269.609 Hyperellipsoid 4 6 91 104 67.9968 Kowalik 4 0.00159405 264 277 946.552 Leon 2 2.02736e-014 124 128 607.562 Levy7 4 -52.0615 475 488 2048.62 Matyas 2 4.81431e-025 42 46 389.556
Max Mod 2 1.49447e-009 67 71 234.086
McCormick 2 -1.91322 16 20 61.1501
Michalewitz 2 -0.801303 67 71 128.977
Multimod 4 4.24075e-008 233 246 503.07
Neumaier Perm 4 550.469 235 248 547.431
Neumaier power sum 4 1.07849e-005 1003 1016 3825.88
Odd square5 5 -1.39764 828 847 2265.45 Odd Square10 10 -1.3585 1003 1067 15036.5 Plateau 5 46 154 173 314.683 Quartic Noise U 4 1.56688 209 222 417.756 Rana 4 -765.143 90 103 398.201 Rastrigin 4 62.5311 62 75 143.088 Rastrigin2 2 14.6516 28 32 55.0309 Schaffer1 2 0.496753 109 113 215.244 Schaffer2 2 0.00054507 76 80 116.769 Schwefel1 2 4 2.98228e-023 115 128 4165.81 Schwefel2 21 4 2.68241e-008 207 220 305.084 Schwefel2 22 4 20.3203 164 177 405.073 Schwefel2 26 4 319.836 50 63 96.1442 Shekel2 2 500 122 126 122.479 Shekel4 5 4 -10.1532 69 82 132.172 Shekel4 7 4 -10.4029 70 83 94.368 Shekel4 10 4 -10.5364 63 76 100.337 Shubert 2 -6.32997 28 32 103.907 Shubert2 4 -5.3684 186 199 490.661 Shubert3 2 -6.32997 28 32 112.027 Sphere 4 3.31775e-023 104 117 97.6332 Sphere2 4 2.75733e-021 105 118 1112.45 Step 4 1 159 172 270.445 Stretched V 4 1.54555e-015 128 141 1184.52
Sum Squares 4 7.6635e-024 143 156 298.156
Trecanni 2 2.12171e-015 33 37 184.551
Trefethen4 2 8.35342 21 25 89.3336
Watson 6 0.00228767 815 841 5009.36
Table 6: Test case results with ρbeg = 5%.
Name n f NPE NE Time [ms]
Ackley 4 19.4364 184 197 453.574 Beale 2 4.55759e-015 75 79 367.704 Bohachevsky1 2 0 34 38 161.27 Bohachevsky2 2 0 37 41 187.974 Booth 2 6.53614e-025 12 16 81.7831 BoxBetts 3 1.67461e-014 89 97 631.935 Branin 2 0.397887 23 27 155.871 Branin2 2 9.80233e-014 36 40 133.126 Camel3 2 0.298638 33 37 147.14 Camel6 2 2.10425 35 39 106.419 Chichinadze 2 -43.3159 75 79 387.466 Colville 4 9.06405e-018 397 410 1750.51 Corana 4 184197 142 155 377.869 Eggholder 4 -1627.68 80 93 287.065 Freudenstein-Roth 2 48.9843 64 68 479.177 Gear 4 2.85776e-008 139 152 256.088
Generalized Rosenbrock 4 1.91252e-013 367 380 2178.96
Goldstein-Price 2 30 27 31 157.755 Hansen 2 -7.61849 21 25 71.8032 Hartman 3 3 -3.86278 52 60 299.545 Hartman 6 6 -3.322 137 163 589.198 Himmelblau 2 3.5564e-014 20 24 80.4612 Holzman2 4 2.31249e-023 386 399 748.893 Hosaki 2 -1.12779 66 70 102.516 Hyperellipsoid 4 6 104 117 80.8893 Kowalik 4 0.00159405 251 264 1182.12 Leon 2 1.93787e-015 114 118 768.499 Levy7 4 -64.6282 126 139 757.069 Matyas 2 2.41059e-026 41 45 357.483
Max Mod 2 1.25419e-009 65 69 233.528
McCormick 2 -1.91322 43 47 143.889
Michalewitz 2 -0.801303 60 64 433.246
Multimod 4 2.01106e-008 229 242 570.652
Neumaier Perm0 4 1.78131e-015 162 175 772.929
Neumaier Perm 4 1.8328e-015 228 241 510.21
Odd square5 5 -1.39764 828 847 3750.23 Odd Square10 10 -1.3585 1003 1067 14846.7 Plateau 5 43 173 192 359.385 Quartic Noise U 4 1.56688 249 262 446.631 Rana 4 -765.143 91 104 804.542 Rastrigin 4 37.9028 68 81 154.429 Rastrigin2 2 0.121099 34 38 45.5938 Schaffer1 2 0.487077 46 50 189.898 Schaffer2 2 0.000307125 82 86 126.466 Schwefel1 2 4 4.69129e-025 133 146 953.641 Schwefel2 21 4 3.0254e-008 177 190 330.342 Schwefel2 22 4 16.4176 144 157 334.188 Schwefel2 26 4 319.836 61 74 137.433 Shekel2 2 500 108 112 166.538 Shekel4 5 4 -10.1532 63 76 154.084 Shekel4 7 4 -10.4029 93 106 309.075 Shekel4 10 4 -10.5364 59 72 113.686 Shubert 2 -12.1543 18 22 65.8153 Shubert2 4 -15.5105 186 199 622.659 Shubert3 2 -12.1543 18 22 68.5496 Sphere 4 2.72574e-025 104 117 95.8497 Sphere2 4 2.36716e-024 72 85 782.743 Step 4 0 125 138 350.522 Stretched V 4 7.98293e-017 147 160 2578.23
Sum Squares 4 1.04744e-024 149 162 211.805
Trecanni 2 4.43456e-013 38 42 177.171
Trefethen4 2 4.03313 35 39 95.6668
Watson 6 0.00228767 815 841 6318.37
Table 7: Test case results with ρbeg = 10%.
Name n f NPE NE Time [ms]
Ackley 4 2.9015e-008 198 211 359.529
Beale 2 0.45343 976 980 4973.32
Bohachevsky2 2 0 25 29 76.2121 Booth 2 3.484e-026 37 41 219.086 BoxBetts 3 1.56498e-014 92 100 1002.07 Branin 2 0.397887 19 23 129.917 Branin2 2 1.90141e-014 33 37 104.908 Camel3 2 0.298638 37 41 177.84 Camel6 2 2.10425 33 37 109.181 Chichinadze 2 -42.8686 67 71 405.824 Colville 4 3.676 186 199 1013.48 Corana 4 16155.7 164 177 397.063 Eggholder 4 -1627.68 75 88 278.587 Freudenstein-Roth 2 48.9843 57 61 528.839 Gear 4 1.02647e-008 163 176 322.239
Generalized Rosenbrock 4 2.56283e-012 608 621 3033.91
Goldstein-Price 2 3 30 34 170.922 Hansen 2 -47.5604 25 29 75.2351 Hartman 3 3 -3.86278 60 68 296.018 Hartman 6 6 -3.322 61 87 333.572 Himmelblau 2 5.02543e-015 19 23 76.972 Holzman2 4 4.88468e-030 359 372 847.282 Hosaki 2 -1.12779 18 22 104.945 Hyperellipsoid 4 6 104 117 106.536 Kowalik 4 0.00159405 256 269 1877.11 Leon 2 1.38316e-014 88 92 553.73 Levy7 4 -44.334 77 90 311.903 Matyas 2 1.005e-027 12 16 109.711
Max Mod 2 4.58623e-009 73 77 158.548
McCormick 2 -1.91322 17 21 54.5104
Michalewitz 2 -0.801303 54 58 200.879
Multimod 4 3.92503e-008 184 197 519.633
Neumaier Perm0 4 3.81566e-015 153 166 1404.2
Neumaier Perm 4 1.06531e-013 169 182 559.43
Neumaier power sum 4 6.9891e-005 1003 1016 4147.89
Odd square5 5 -1.39764 828 847 3828.28
Odd Square10 10 -1.3585 1003 1067 10554.4
Quartic Noise U 4 1.56688 172 185 461.696 Rana 4 -933.357 83 96 319.721 Rastrigin 4 36 60 73 129.16 Rastrigin2 2 0.121099 26 30 57.9195 Schaffer1 2 0.495946 108 112 6769.94 Schaffer2 2 0.000163926 76 80 153.486 Schwefel1 2 4 6.9617e-029 135 148 158.673 Schwefel2 21 4 2.59289e-008 196 209 570.124 Schwefel2 22 4 2.43779e-008 211 224 668.554 Schwefel2 26 4 754.12 63 76 2584.64 Shekel2 2 500 126 130 254.192 Shekel4 5 4 -10.1532 87 100 173.919 Shekel4 7 4 -3.7243 61 74 131.932 Shekel4 10 4 -2.42173 107 120 207.316 Shubert 2 -12.1543 18 22 71.4256 Shubert2 4 -21.3887 167 180 1007.19 Shubert3 2 -12.1543 18 22 69.1694 Sphere 4 3.80073e-027 25 38 15.0709 Sphere2 4 9.20995e-028 104 117 1097.82 Step 4 0 124 137 223.954 Stretched V 4 1.0599e-017 143 156 2361.91
Sum Squares 4 5.63247e-027 153 166 164.184
Trecanni 2 1.46809e-016 26 30 115.891
Trefethen4 2 0.454589 29 33 104.484
Watson 6 0.00228767 597 623 3926.15
Table 8: Test case results with ρbeg = 15%.
Name n f NPE NE Time [ms]
Ackley 4 1.39849e-008 179 192 545.548 Beale 2 0.454379 913 917 4033.51 Bohachevsky1 2 1.92624e-013 39 43 113.628 Bohachevsky2 2 0 24 28 85.7293 Booth 2 5.75868e-029 39 43 236.123 BoxBetts 3 1.74905e-013 86 94 566.034
Branin 2 0.397887 23 27 203.918 Branin2 2 0.102163 24 28 155.686 Camel3 2 0.298638 31 35 107.923 Camel6 2 -0.215464 26 30 57.9338 Chichinadze 2 -42.8686 67 71 380.463 Colville 4 1.98091e-014 226 239 1015.04 Corana 4 845.301 162 175 263.007 Eggholder 4 -1627.68 89 102 173.639 Freudenstein-Roth 2 48.9843 72 76 430.851 Gear 4 1.79367e-007 155 168 209.223 Generalized Rosenbrock 4 3.70143 246 259 1928.72 Goldstein-Price 2 3 23 27 96.6662 Hansen 2 -145.478 23 27 67.5172 Hartman 3 3 -3.86278 65 73 325.862 Hartman 6 6 -3.322 53 79 294.045 Himmelblau 2 7.8119e-013 18 22 63.8601 Holzman2 4 3.31105e-031 452 465 679.919 Hosaki 2 -1.12779 23 27 98.1713 Hyperellipsoid 4 6 91 104 75.9192 Kowalik 4 0.0147138 426 439 855.714 Leon 2 2.40647e-016 119 123 704.8 Levy7 4 -70.5007 188 201 1453.47 Matyas 2 4.05553e-029 23 27 236.186
Max Mod 2 3.98401e-010 73 77 269.675
McCormick 2 -1.91322 16 20 126.439
Michalewitz 2 -0.801303 67 71 164.915
Multimod 4 1.33711e-008 204 217 652.265
Neumaier Perm0 4 3.38613e-017 169 182 1735.25
Neumaier Perm 4 550.469 168 181 696.42
Neumaier power sum 4 3.97809e-005 1003 1016 5754.21
Odd square5 5 -1.39764 828 847 3390.5 Odd Square10 10 -1.3585 1003 1067 10844.9 Plateau 5 37 175 194 318.816 Quartic Noise U 4 1.56688 213 226 477.039 Rana 4 -754.132 101 114 671.721 Rastrigin 4 36.9514 83 96 211.563
Rastrigin2 2 0.484396 22 26 34.8732 Schaffer1 2 0.495946 86 90 142.296 Schaffer2 2 0.000267038 73 77 117.683 Schwefel1 2 4 2.76453e-027 150 163 122.047 Schwefel2 21 4 4.69206e-009 184 197 510.421 Schwefel2 22 4 3.10386e-008 196 209 435.928 Schwefel2 26 4 596.298 108 121 9048.02 Shekel2 2 -7.47612e+009 115 119 49.8837 Shekel4 5 4 -10.1532 71 84 56.2669 Shekel4 7 4 -10.4029 76 89 66.1548 Shekel4 10 4 -10.5364 96 109 157.086 Shubert 2 -24.0625 21 25 16.936 Shubert2 4 -6.38963 196 209 490.151 Shubert3 2 -24.0625 21 25 14.363 Sphere 4 3.61101e-028 117 130 50.0734 Sphere2 4 2.592e-027 26 39 140.958 Step 4 0 138 151 92.2893 Stretched V 4 1.04706e-015 165 178 608.559
Sum Squares 4 1.00757e-026 153 166 126.916
Trecanni 2 3.4622e-019 26 30 81.0536
Trefethen4 2 -1.30511 33 37 63.7377
Watson 6 0.00228767 894 920 5153.71
Table 9: Test case results with ρbeg = 20%.
Name n f NPE NE Time [ms]
Ackley 4 8.81315 161 174 703.154 Beale 2 1.9177e-013 52 56 316.801 Bohachevsky1 2 0 32 36 86.1981 Bohachevsky2 2 0 30 34 73.4158 Booth 2 3.9443e-030 32 36 237.926 BoxBetts 3 2.28567e-016 83 91 505.772 Branin 2 0.397887 21 25 159.576 Branin2 2 8.33967e-014 30 34 78.8968 Camel3 2 2.03359e-015 27 31 135.722
Camel6 2 -1.03163 32 36 87.0939 Chichinadze 2 -42.8686 74 78 334.236 Colville 4 3.676 323 336 1592.6 Corana 4 0.416625 175 188 502.189 Eggholder 4 -1627.68 73 86 251.566 Freudenstein-Roth 2 3.46462e-016 61 65 407.724 Gear 4 2.0226e-006 154 167 310.859
Generalized Rosenbrock 4 2.00887e-014 362 375 2124.03
Goldstein-Price 2 3 34 38 146.539 Hansen 2 -145.478 25 29 66.8524 Hartman 3 3 -3.86278 72 80 485.167 Hartman 6 6 -3.322 109 135 523.59 Himmelblau 2 1.17803e-013 23 27 65.9316 Holzman2 4 4.1604e-031 246 259 464.354 Hosaki 2 -1.12779 19 23 83.2756 Hyperellipsoid 4 6 12 25 9.73852 Kowalik 4 0.00159405 210 223 1148.62 Leon 2 4.28486e-013 85 89 600.166 Levy7 4 -39.1032 114 127 448.444 Matyas 2 5.82278e-031 16 20 163.468
Max Mod 2 2.78597e-009 66 70 145.343
McCormick 2 -1.91322 15 19 65.6983
Michalewitz 2 -0.801303 65 69 247.282
Multimod 4 2.77489e-008 182 195 426.588
Neumaier Perm0 4 1.10445e-016 151 164 766.222
Neumaier Perm 4 550.469 162 175 764.163
Neumaier power sum 4 4.63972e-005 1003 1016 5396.26
Odd square5 5 -1.39764 828 847 3510.73 Odd Square10 10 -1.3585 1003 1067 9552.85 Plateau 5 31 180 199 188.333 Quartic Noise U 4 1.56688 151 164 256.677 Rana 4 -665.865 93 106 385.131 Rastrigin 4 36 64 77 135.506 Rastrigin2 2 1.08989 24 28 42.2043 Schaffer1 2 0.272741 42 46 87.0323 Schaffer2 2 0.000180323 73 77 166.875
Schwefel1 2 4 8.76129e-029 149 162 144.462 Schwefel2 21 4 3.43282e-008 212 225 486.313 Schwefel2 22 4 7.8526e-008 184 197 802.671 Schwefel2 26 4 714.659 50 63 126.76 Shekel2 2 -2.07533e+009 207 211 438.742 Shekel4 5 4 -10.1532 63 76 182.198 Shekel4 7 4 -3.7243 103 116 292.359 Shekel4 10 4 -2.42173 80 93 227.08 Shubert 2 -13.1027 26 30 113.937 Shubert2 4 -21.3887 168 181 678.761 Shubert3 2 -13.1027 26 30 110.427 Sphere 4 3.09628e-029 22 35 16.5079 Sphere2 4 5.4249e-028 115 128 1404.41 Step 4 0 135 148 193.838 Stretched V 4 4.47078e-018 179 192 2156.49
Sum Squares 4 5.60091e-029 139 152 204344
Trecanni 2 1.75218e-015 39 43 75.1539
Trefethen4 2 -0.19618 32 36 60.3759
Watson 6 0.00228767 821 847 4374.06
Table 10: Test case results with ρbeg = 25%.
Name n f NPE NE Time [ms]
Ackley 4 1.8568e-008 173 186 445.695 Beale 2 0.455668 887 891 3241.4 Bohachevsky1 2 0 31 35 112.93 Bohachevsky2 2 0 27 31 102.671 Booth 2 3.15544e-030 35 39 213.005 BoxBetts 3 6.64337e-014 94 102 648.436 Branin 2 0.397887 21 25 145.13 Branin2 2 1.06854e-014 39 43 205.053 Camel3 2 9.1781e-022 26 30 133.73 Camel6 2 -1.03163 30 34 112.298 Chichinadze 2 -41.835 82 86 383.157 Colville 4 3.676 281 294 1200.34
Corana 4 6.21563 149 162 363.641
Eggholder 4 -1627.68 74 87 258.247
Freudenstein-Roth 2 48.9843 73 77 570.978
Gear 4 0.000119815 155 168 321.721
Generalized Rosenbrock 4 9.31656e-014 320 333 2217.75
Goldstein-Price 2 3 29 33 109.79 Hansen 2 -38.5788 27 31 95.9809 Hartman 3 3 -3.86278 54 62 330.355 Hartman 6 6 -3.322 91 117 432.133 Himmelblau 2 6.30371e-014 21 25 93.6 Holzman2 4 1.98073e-029 260 273 589.294 Hosaki 2 -1.12779 21 25 86.259 Hyperellipsoid 4 6 20 33 14.724 Kowalik 4 0.00132144 747 760 2367.24 Leon 2 9.83949e-015 103 107 642.285 Levy7 4 -666.999 114 127 527.822 Matyas 2 5.1276e-030 14 18 177.308
Max Mod 2 6.21197e-009 63 67 177.194
McCormick 2 -1.91322 19 23 90.8734
Michalewitz 2 -0.801303 67 71 268.432
Multimod 4 2.2418e-008 174 187 569.502
Neumaier Perm0 4 2.49918e-017 176 189 707.817
Neumaier Perm 4 3.99737e-014 198 211 521.604
Neumaier power sum 4 9.28234e-006 1003 1016 6696.68
Odd square5 5 -1.39764 828 847 3519 Odd Square10 10 -1.3585 1003 1067 11369.8 Plateau 5 30 184 203 196.075 Quartic Noise U 4 1.56688 154 167 256.119 Rana 4 -1197.99 97 110 562.184 Rastrigin 4 36 50 63 100.04 Rastrigin2 2 0 25 29 58.1625 Schaffer1 2 0.459781 60 64 300.922 Schaffer2 2 0.000407909 70 74 201.44 Schwefel1 2 4 3.03231e-028 148 161 166.333 Schwefel2 21 4 3.07266e-008 201 214 474.714 Schwefel2 22 4 5.1674e-008 187 200 473.323
Schwefel2 26 4 872.623 81 94 612.768 Shekel2 2 -2.25005e+010 89 93 193.559 Shekel4 5 4 -10.1532 73 86 200.552 Shekel4 7 4 -3.7243 100 113 358.739 Shekel4 10 4 -3.83543 49 62 97.6093 Shubert 2 -6.39209 20 24 58.0423 Shubert2 4 -21.3887 198 211 1828.33 Shubert3 2 -6.39209 20 24 52.7639 Sphere 4 6.02431e-030 21 34 16.9918 Sphere2 4 6.5081e-030 20 33 255.933 Step 4 0 139 152 259.061 Stretched V 4 1.16138e-015 175 188 1713.92
Sum Squares 4 7.69139e-030 155 168 1583.11
Trecanni 2 5.55444e-015 43 47 139.962
Trefethen4 2 -1.57889 29 33 72.3587
Watson 6 0.00228767 569 595 14381.7
Table 11: Test case results with ρbeg = 30%.
Name n f NPE NE Time [ms]
Ackley 4 3.04259e-008 175 188 291.406 Beale 2 1.26708e-014 71 75 405.078 Bohachevsky1 2 0 30 34 126.332 Bohachevsky2 2 0 31 35 114.628 Booth 2 0 12 16 110.601 BoxBetts 3 5.02173e-013 75 83 966.747 Branin 2 0.397887 23 27 168.839 Branin2 2 3.5902e-015 40 44 162.117 Camel3 2 3.71063e-017 32 36 137.885 Camel6 2 -1.03163 25 29 78.2343 Chichinadze 2 -43.3159 147 151 587.271 Colville 4 8.15779e-014 371 384 1410.94 Corana 4 1.16663 150 163 281.99 Eggholder 4 -1627.68 66 79 259.55 Freudenstein-Roth 2 48.9843 82 86 636.769
Gear 4 8.60377e-006 149 162 282.859
Generalized Rosenbrock 4 4.62456e-014 520 533 2616.35
Goldstein-Price 2 3 30 34 94.4392 Hansen 2 -16.0556 25 29 87.7376 Hartman 3 3 -3.86278 68 76 428.648 Hartman 6 6 -3.322 132 158 729.625 Himmelblau 2 4.90828e-016 23 27 62.7199 Holzman2 4 1.78657e-030 208 221 342.245 Hosaki 2 -2.34581 21 25 90.7552 Hyperellipsoid 4 6 108 121 98.1679 Kowalik 4 0.000307486 392 405 2206.61 Leon 2 1.16261e-014 99 103 496.758 Levy7 4 -177.29 230 243 1091.6 Matyas 2 6.70532e-032 14 18 136.197
Max Mod 2 2.75928e-009 78 82 156.102
McCormick 2 -1.91322 18 22 82.0372
Michalewitz 2 -0.801303 59 63 523.054
Multimod 4 2.97597e-008 205 218 409.525
Neumaier Perm0 4 2.19742e-015 173 186 924.55
Neumaier Perm 4 2.11933e-012 195 208 382.819
Neumaier power sum 4 4.73013e-005 1003 1016 4630.43
Odd square5 5 -1.39764 828 847 3377.66 Odd Square10 10 -1.3585 1003 1067 11298.3 Plateau 5 30 182 201 207.437 Quartic Noise U 4 1.56688 127 140 241.623 Rana 4 -819.64 64 77 365.498 Rastrigin 4 36 73 86 158.017 Rastrigin2 2 0.484396 28 32 40.2703 Schaffer1 2 0.451776 104 108 178.42 Schaffer2 2 0.000594409 78 82 198.165 Schwefel1 2 4 9.86076e-031 147 160 141.072 Schwefel2 21 4 2.98613e-008 181 194 403.03 Schwefel2 22 4 5.29607e-008 178 191 397.941 Schwefel2 26 4 398.831 72 85 238.9 Shekel2 2 -5.3458e+009 96 100 172.172 Shekel4 5 4 -10.1532 59 72 228.84
Shekel4 7 4 -10.4029 87 100 214.063 Shekel4 10 4 -2.42734 53 66 140.422 Shubert 2 -12.1543 22 26 80.6194 Shubert2 4 -15.5105 180 193 511.736 Shubert3 2 -12.1543 22 26 91.5286 Sphere 4 1.38051e-030 12 25 9.39168 Sphere2 4 1.18329e-030 20 33 310.792 Step 4 0 135 148 233.836 Stretched V 4 9.71625e-016 165 178 1406.38
Sum Squares 4 6.12945e-028 150 163 153.271
Trecanni 2 1.58424e-016 31 35 135.516
Trefethen4 2 1.14371 27 31 138.495
Watson 6 0.00228767 533 559 4160.36
9
Results for Switching Trust Region Shape
Table 12 displays the result of using an elliptical trust region that changes the axis along which it is elongated, in every iteration. We can see that although some test cases are solved with fewer function evaluations and/or lower objective function values, there are also some test cases where this setting is considerably worse than the standard spherical trust region setting, e.g. Beale and Step.
Table 12: Test case results when using an switching trust region shape.
Name n f NPE NE Time [ms]
Ackley 4 18.5752 189 202 816.694 Beale 2 0.456447 933 937 3844.62 Bohachevsky1 2 8.12683e-014 42 46 274.9 Bohachevsky2 2 1.72085e-015 35 39 175.025 Booth 2 9.90037e-024 38 42 283.93 BoxBetts 3 5.7173e-015 92 100 465.617 Branin 2 0.397887 21 25 108.248 Branin2 2 1.10168e-014 36 40 185.329 Camel3 2 0.298638 37 41 144.445
Camel6 2 -0.215464 36 40 172.635 Chichinadze 2 -27.8253 39 43 342.52 Colville 4 3.676 339 352 1955.76 Corana 4 57674.2 1003 1016 3643.46 Eggholder 4 -1627.68 84 97 365.375 Freudenstein-Roth 2 48.9843 65 69 292.093 Gear 4 1.49651e-006 52 65 1060.74
Generalized Rosenbrock 4 1.3636e-014 395 408 1567.7
Goldstein-Price 2 30 28 32 130.574 Hansen 2 -12.3158 40 44 108.031 Hartman 3 3 -3.86278 59 67 364.972 Hartman 6 6 -3.322 120 146 533.659 Himmelblau 2 4.59451e-014 26 30 108.463 Holzman2 4 2.78933e-031 349 362 553.145 Hosaki 2 -1.12779 28 32 3365.18 Hyperellipsoid 4 6 105 118 432.952 Kowalik 4 0.00159405 255 268 1080.97 Leon 2 1.0472e-014 93 97 431.251 Levy7 4 -83.5006 190 203 756.129 Matyas 2 2.42119e-026 40 44 170.201
Max Mod 2 1.37465e-008 66 70 127.645
McCormick 2 -1.91322 22 26 87.9586
Michalewitz 2 -0.801303 59 63 153.542
Multimod 4 1.7058e-008 241 254 466.497
Neumaier Perm0 4 1.7549e-015 159 172 1156.1
Neumaier Perm 4 6.25318e-013 175 188 367.943
Neumaier power sum 4 4.99764e-005 1003 1016 4285.91
Odd square5 5 -1.39763 448 467 2679.65 Odd Square10 10 -1.38981 1003 1067 10180.9 Plateau 5 41 85 104 434.173 Quartic Noise U 4 1.56688 145 158 882.695 Rana 4 -973.049 123 136 838.486 Rastrigin 4 36.9514 94 107 346.722 Rastrigin2 2 1.93757 28 32 118.091 Schaffer1 2 0.49345 59 63 87.9636 Schaffer2 2 0.000189568 87 91 314.782
Schwefel1 2 4 4.72101e-025 190 203 2070.74 Schwefel2 21 4 3.37064e-008 225 238 676.403 Schwefel2 22 4 5.03683e-008 474 487 1104.17 Schwefel2 26 4 319.836 34 47 254.247 Shekel2 2 500 112 116 247.566 Shekel4 5 4 -10.1532 84 97 341.823 Shekel4 7 4 -10.4029 74 87 288.319 Shekel4 10 4 -3.83543 71 84 290.934 Shubert 2 -12.1543 25 29 126.423 Shubert2 4 -15.5105 157 170 946.166 Shubert3 2 -12.1543 25 29 110.383 Sphere 4 2.72761e-025 27 40 182.826 Sphere2 4 3.7685e-024 114 127 1182.09 Step 4 3 310 323 2326.75 Stretched V 4 3.70821e-016 166 179 1015.9
Sum Squares 4 4.26051e-025 133 146 5308.19
Trecanni 2 1.9792e-018 39 43 118.858
Trefethen4 2 6.79534 35 39 115.296
Watson 6 0.00228767 693 719 4369.01
10
Results for Elliptical Trust Regions
The tables is this section display the results for when using an elliptical trust region. Each table displays the result for a different value of γ. From Figure 3, we see that the values γ = 0.15, 0.35 and 0.60 are dominated most of the time.
0 200 400 600 800 1000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
γ = 0.0 (circular trust region)
γ = 0.05 γ = 0.15 γ = 0.25 γ = 0.35 γ = 0.50 γ = 0.60 γ = 0.75 γ = 1.0
Figure 3: Data profile when varying the forgetting factor γ.
Table 13: Test case results when using an standard hyperspherical trust region, i.e. γ = 0.
Name n f NPE NE Time [ms]
Ackley 4 3.04259e-008 175 188 394.998 Beale 2 1.26708e-014 71 75 387.497 Bohachevsky1 2 0 30 34 104.685 Bohachevsky2 2 0 31 35 98.4801 Booth 2 0 12 16 110.442 BoxBetts 3 5.02173e-013 75 83 964.03 Branin 2 0.397887 23 27 156.509 Branin2 2 3.5902e-015 40 44 156.403 Camel3 2 3.71063e-017 32 36 156.795 Camel6 2 -1.03163 25 29 81.3004 Chichinadze 2 -43.3159 147 151 592.054 Colville 4 8.15779e-014 371 384 1328.39 Corana 4 1.16663 150 163 259.234 Eggholder 4 -1627.68 66 79 264.917
Freudenstein-Roth 2 48.9843 82 86 677.456
Gear 4 8.60377e-006 149 162 309.644
Generalized Rosenbrock 4 4.62456e-014 520 533 2666.56
Goldstein-Price 2 3 30 34 87.0069 Hansen 2 -16.0556 25 29 87.4115 Hartman 3 3 -3.86278 68 76 491.74 Hartman 6 6 -3.322 132 158 786.33 Himmelblau 2 4.90828e-016 23 27 69.027 Holzman2 4 1.78657e-030 208 221 338.863 Hosaki 2 -2.34581 21 25 98.5089 Hyperellipsoid 4 6 108 121 103.658 Kowalik 4 0.000307486 392 405 2158.19 Leon 2 1.16261e-014 99 103 586.05 Levy7 4 -177.29 230 243 1204.93 Matyas 2 6.70532e-032 14 18 163.958
Max Mod 2 2.75928e-009 78 82 185.028
McCormick 2 -1.91322 18 22 94.674
Michalewitz 2 -0.801303 59 63 592.04
Multimod 4 2.97597e-008 205 218 466.284
Neumaier Perm0 4 2.19742e-015 173 186 1048.01
Neumaier Perm 4 2.11933e-012 195 208 459.596
Neumaier power sum 4 4.73013e-005 1003 1016 5518.87
Odd square5 5 -1.39764 828 847 3691.79 Odd Square10 10 -1.3585 1003 1067 15896.4 Plateau 5 30 182 201 513.724 Quartic Noise U 4 1.56688 127 140 405.425 Rana 4 -819.64 64 77 592.441 Rastrigin 4 36 73 86 262.017 Rastrigin2 2 0.484396 28 32 78.6365 Schaffer1 2 0.451776 104 108 417.497 Schaffer2 2 0.000594409 78 82 242.906 Schwefel1 2 4 9.86076e-031 147 160 206.597 Schwefel2 21 4 2.98613e-008 181 194 659.224 Schwefel2 22 4 5.29607e-008 178 191 735.36 Schwefel2 26 4 398.831 72 85 555.32 Shekel2 2 -5.3458e+009 96 100 270.226
Shekel4 5 4 -10.1532 59 72 291.929 Shekel4 7 4 -10.4029 87 100 379.916 Shekel4 10 4 -2.42734 53 66 207.812 Shubert 2 -12.1543 22 26 160.163 Shubert2 4 -15.5105 180 193 916.145 Shubert3 2 -12.1543 22 26 242.083 Sphere 4 1.38051e-030 12 25 12.3689 Sphere2 4 1.18329e-030 20 33 379.198 Step 4 0 135 148 303.901 Stretched V 4 9.71625e-016 165 178 2948.62
Sum Squares 4 6.12945e-028 150 163 196.821
Trecanni 2 1.58424e-016 31 35 158.231
Trefethen4 2 1.14371 27 31 124.372
Watson 6 0.00228767 533 559 5517.95
Table 14: Test case results for γ = 0.05.
Name n f NPE NE Time [ms]
Ackley 4 2.57993 140 153 941.149 Beale 2 1.17953e-016 57 61 409.788 Bohachevsky1 2 0 32 36 100.935 Bohachevsky2 2 0 28 32 86.391 Booth 2 0 12 16 66.4882 BoxBetts 3 4.43119e-014 84 92 920.369 Branin 2 0.397887 21 25 158.592 Branin2 2 1.3027e-014 52 56 176.978 Camel3 2 1.46095e-015 36 40 128.049 Camel6 2 -1.03163 26 30 67.4845 Chichinadze 2 -43.3159 146 150 690.02 Colville 4 2.23477e-014 377 390 1707.46 Corana 4 2.26463 154 167 305.03 Eggholder 4 -1627.68 69 82 387.297 Freudenstein-Roth 2 48.9843 75 79 638.467 Gear 4 4.5033e-009 155 168 574.381
Goldstein-Price 2 3 32 36 125.053 Hansen 2 -16.0556 25 29 81.3658 Hartman 3 3 -3.86278 54 62 419.096 Hartman 6 6 -3.322 248 274 3353.5 Himmelblau 2 3.72501e-013 27 31 59.6761 Holzman2 4 2.92411e-030 251 264 523.116 Hosaki 2 -2.34581 22 26 108.537 Hyperellipsoid 4 6 108 121 367.637 Kowalik 4 0.00117502 221 234 2240.66 Leon 2 3.93026e-015 96 100 541.798 Levy7 4 -243.29 198 211 1268.38 Matyas 2 6.70532e-032 14 18 142.555
Max Mod 2 2.20912e-008 60 64 136.027
McCormick 2 -1.91322 18 22 215.958
Michalewitz 2 -0.801303 73 77 544.052
Multimod 4 7.6492e-009 272 285 1073.89
Neumaier Perm0 4 2.13746e-014 145 158 1938.84
Neumaier Perm 4 1.76143e-012 171 184 2538.64
Neumaier power sum 4 0.0005123 629 642 5281.54
Odd square5 5 -1.39764 213 232 3428.5 Odd Square10 10 -1.35851 1003 1067 17453 Plateau 5 30 182 201 654.163 Quartic Noise U 4 1.56688 214 227 516.738 Rana 4 -819.64 65 78 327.187 Rastrigin 4 36 53 66 208.545 Rastrigin2 2 0.484396 25 29 93.1789 Schaffer1 2 0.441908 119 123 303.205 Schaffer2 2 0.000220845 71 75 193.027 Schwefel1 2 4 1.81203e-028 276 289 283.63 Schwefel2 21 4 1.65287e-008 193 206 505.966 Schwefel2 22 4 2.93257e-008 205 218 849.803 Schwefel2 26 4 398.831 108 121 469.718 Shekel2 2 -2.51259e+010 95 99 194.916 Shekel4 5 4 -10.1532 90 103 472.358 Shekel4 7 4 -10.4029 70 83 265.622 Shekel4 10 4 -2.42734 56 69 212.568
Shubert 2 -12.1543 23 27 106.396 Shubert2 4 -15.5996 201 214 1227.23 Shubert3 2 -12.1543 23 27 96.4252 Sphere 4 1.38051e-030 12 25 45.7062 Sphere2 4 1.18329e-030 108 121 1264.97 Step 4 0 137 150 166.01 Stretched V 4 1.27392e-015 163 176 1095.51
Sum Squares 4 6.18084e-027 297 310 288.934
Trecanni 2 7.01763e-016 40 44 124.381
Trefethen4 2 1.14371 28 32 90.4499
Name n f NPE NE Time [ms] Ackley 4 3.63048e-008 260 273 863.511 Beale 2 1.4649e-014 49 53 452.239 Bohachevsky1 2 0 27 31 91.3334 Bohachevsky2 2 0 32 36 63.8136 Booth 2 0 12 16 106.849 BoxBetts 3 4.8779e-015 86 94 1084.44 Branin 2 0.397887 22 26 175.16 Branin2 2 1.80997e-012 33 37 203.318 Camel3 2 1.36123e-015 33 37 196.797 Camel6 2 -1.03163 25 29 96.2142 Chichinadze 2 -43.3159 88 92 586.167 Colville 4 1.27696e-012 293 306 2249.13 Corana 4 0.42332 105 118 1246.96 Eggholder 4 -1627.68 55 68 272.885 Freudenstein-Roth 2 48.9843 91 95 635.483 Gear 4 1.98903e-006 150 163 249.501
Generalized Rosenbrock 4 2.16867e-013 450 463 3022.32
Goldstein-Price 2 3 32 36 121.664 Hansen 2 -16.0556 25 29 70.076 Hartman 3 3 -3.86278 70 78 527.408 Hartman 6 6 -3.322 284 310 3587.23 Himmelblau 2 2.09148e-012 22 26 68.5827 Holzman2 4 1.18447e-011 147 160 2096.63 Hosaki 2 -2.34581 23 27 110.482 Hyperellipsoid 4 6 108 121 513.671 Kowalik 4 0.000958304 1003 1016 4340.35 Leon 2 2.26203e-013 94 98 711.433 Levy7 4 -248.594 142 155 948.884 Matyas 2 6.70532e-032 14 18 128.997
Max Mod 2 5.72181e-008 58 62 150.635
McCormick 2 -1.91322 18 22 127.868
Michalewitz 2 -0.801303 66 70 429.911
Multimod 4 6.26214e-008 255 268 858.202
Neumaier Perm0 4 4.14486e-015 182 195 1059.74
Neumaier Perm 4 777.99 83 96 1789.72
Neumaier power sum 4 0.0594671 101 114 1175.44
Odd square5 5 -1.39762 241 260 4730.09 Odd Square10 10 -1.3585 217 281 14176.2 Plateau 5 30 189 208 446.345 Quartic Noise U 4 1.56688 148 161 1061.6 Rana 4 -819.64 74 87 456.667 Rastrigin 4 36 54 67 227.373 Rastrigin2 2 0.484396 27 31 85.543 Schaffer1 2 0.414668 105 109 416.316 Schaffer2 2 0.000376566 85 89 272.397 Schwefel1 2 4 5.08797e-030 145 158 213.226 Schwefel2 21 4 4.64232e-008 242 255 1087.86 Schwefel2 22 4 3.2485e-008 243 256 1041.99 Schwefel2 26 4 43.5149 86 99 694.822 Shekel2 2 -3.47527e+009 143 147 224.232 Shekel4 5 4 -10.1532 76 89 269.473 Shekel4 7 4 -10.4029 82 95 311.682 Shekel4 10 4 -2.42734 67 80 4340.9 Shubert 2 -12.1543 23 27 58.547 Shubert2 4 -15.5105 172 185 2417.67 Shubert3 2 -12.1543 23 27 78.0834 Sphere 4 1.38051e-030 12 25 77.449 Sphere2 4 1.18329e-030 108 121 1282.82 Step 4 0 124 137 568.766 Stretched V 4 2.24996e-005 49 62 984.591
Sum Squares 4 7.40863e-027 199 212 194.586
Trecanni 2 5.36285e-015 38 42 244.449
Trefethen4 2 1.14371 27 31 109.158
Watson 6 1.09831 158 184 1903.04
Table 16: Test case results for γ = 0.25.
Name n f NPE NE Time [ms]
Ackley 4 2.57993 1001 1014 2345.98 Beale 2 1.68108e-014 43 47 404.079 Bohachevsky1 2 0 28 32 110.118 Bohachevsky2 2 0 29 33 78.9726 Booth 2 0 12 16 118.746 BoxBetts 3 5.55182e-015 85 93 1281.82 Branin 2 0.397887 23 27 175.756 Branin2 2 3.07057e-013 32 36 197.3 Camel3 2 1.65485e-015 34 38 227.536 Camel6 2 -1.03163 22 26 85.1276 Chichinadze 2 -42.8686 69 73 308.974 Colville 4 4.56685e-016 420 433 2077.22 Corana 4 1.25963 171 184 321.089 Eggholder 4 -1627.68 99 112 1446.87 Freudenstein-Roth 2 48.9843 83 87 645.082 Gear 4 3.00591e-007 101 114 739.659 Generalized Rosenbrock 4 11951.4 85 98 1483.4 Goldstein-Price 2 3 42 46 119.305 Hansen 2 -16.0556 24 28 53.7921 Hartman 3 3 -3.86278 64 72 429.27 Hartman 6 6 -3.322 151 177 9172.84 Himmelblau 2 3.21655e-015 24 28 93.1823 Holzman2 4 0.00113104 71 84 2391.73 Hosaki 2 -2.34581 20 24 79.4877 Hyperellipsoid 4 6 108 121 264.793 Kowalik 4 0.000995146 227 240 2142.68 Leon 2 5.47266e-014 88 92 632.989 Levy7 4 -507.113 173 186 2888.92 Matyas 2 6.70532e-032 14 18 182.255
Max Mod 2 2.79995e-008 75 79 262.391
McCormick 2 -1.91322 17 21 127.587
Michalewitz 2 -0.801303 68 72 396.036
Multimod 4 0.00161189 165 178 2337.47
Neumaier Perm0 4 1.18324e-016 186 199 2163.52
Neumaier Perm 4 613.605 136 149 1464.07
Neumaier power sum 4 0.166504 105 118 2288.16
Odd square5 5 -1.39764 336 355 3893.09 Odd Square10 10 -1.35637 128 192 14197.2 Plateau 5 30 188 207 503.147 Quartic Noise U 4 1.56689 82 95 1627.36 Rana 4 -819.64 94 107 9524.23 Rastrigin 4 36 66 79 327.447 Rastrigin2 2 0.484396 27 31 88.9221 Schaffer1 2 0.396098 157 161 883.885 Schaffer2 2 0.000404092 82 86 202.042 Schwefel1 2 4 3.01739e-029 134 147 250.209 Schwefel2 21 4 7.48612e-008 231 244 823.68 Schwefel2 22 4 5.74008e-008 217 230 975.725 Schwefel2 26 4 -2343.09 209 222 2784.94 Shekel2 2 -4.53693e+009 122 126 309.269 Shekel4 5 4 -7.90819 59 72 1135.52 Shekel4 7 4 -2.75193 67 80 362.268 Shekel4 10 4 -2.42734 115 128 5439.2 Shubert 2 -12.1543 23 27 87.9043 Shubert2 4 -15.5105 83 96 1051.43 Shubert3 2 -12.1543 23 27 75.4684 Sphere 4 1.38051e-030 12 25 39.3021 Sphere2 4 1.18329e-030 108 121 725.815 Step 4 0 136 149 211.702 Stretched V 4 6.79e-017 144 157 2147.86
Sum Squares 4 3.45323e-027 174 187 160.274
Trecanni 2 4.36939e-015 50 54 448.625
Trefethen4 2 1.14371 28 32 104.416
Watson 6 9.24624 141 167 1724.72
Table 17: Test case results for γ = 0.35.
Name n f NPE NE Time [ms]
Ackley 4 3.51135e-009 347 360 1325
Beale 2 4.75252e-019 63 67 588.221
Bohachevsky2 2 0 27 31 102.976 Booth 2 0 12 16 140.039 BoxBetts 3 1.2602e-013 88 96 757.445 Branin 2 0.397887 26 30 272.824 Branin2 2 3.09491e-015 34 38 245.18 Camel3 2 2.16705e-017 35 39 220.857 Camel6 2 -1.03163 38 42 4942.65 Chichinadze 2 -42.2822 75 79 653.595 Colville 4 2.29677e-016 387 400 2337.96 Corana 4 0.464625 170 183 2472.03 Eggholder 4 -1627.68 66 79 371.791 Freudenstein-Roth 2 48.9843 93 97 707.298 Gear 4 2.09322e-007 104 117 2103.08
Generalized Rosenbrock 4 6.96306e-016 578 591 3074.77
Goldstein-Price 2 3 29 33 92.745 Hansen 2 -16.0556 25 29 81.2515 Hartman 3 3 -3.86278 74 82 700.506 Hartman 6 6 -3.322 87 113 1398.29 Himmelblau 2 1.79601e-015 24 28 96.0167 Holzman2 4 3.85671e-007 99 112 2379.89 Hosaki 2 -2.34581 21 25 115.268 Hyperellipsoid 4 6 108 121 1084.56 Kowalik 4 0.0532357 58 71 503.358 Leon 2 1.46345e-015 97 101 724.391 Levy7 4 -215.816 463 476 4655.97 Matyas 2 6.70532e-032 12 16 189.188
Max Mod 2 5.25103e-009 79 83 341.308
McCormick 2 -1.91322 18 22 159.501
Michalewitz 2 -0.801303 57 61 971.327
Multimod 4 2.42209 75 88 1714.54
Neumaier Perm0 4 1.05911e-016 177 190 2327.75
Neumaier Perm 4 797.325 132 145 2095.32
Neumaier power sum 4 0.341909 101 114 3793.76
Odd square5 5 -1.39763 130 149 2873.88
Odd Square10 10 -1.35783 136 200 15543.7
Plateau 5 30 61 80 3542.4
Quartic Noise U 4 1.56689 69 82 2475.25 Rana 4 -819.64 56 69 349.187 Rastrigin 4 36 62 75 432.494 Rastrigin2 2 0.484396 26 30 88.2488 Schaffer1 2 0.414668 144 148 247.951 Schaffer2 2 0.00216895 75 79 218.317 Schwefel1 2 4 7.29696e-030 143 156 200.328 Schwefel2 21 4 0.00023521 162 175 1892.54 Schwefel2 22 4 4.78034e-008 197 210 732.75 Schwefel2 26 4 -854.169 93 106 2667.92 Shekel2 2 -8.15297e+009 159 163 443.313 Shekel4 5 4 -10.1532 91 104 18618.5 Shekel4 7 4 -2.75193 106 119 1153.94 Shekel4 10 4 -2.42734 58 71 205.8 Shubert 2 -12.1543 23 27 90.3125 Shubert2 4 -15.5996 213 226 1145.02 Shubert3 2 -12.1543 23 27 71.8698 Sphere 4 1.38051e-030 12 25 61.9538 Sphere2 4 1.18329e-030 108 121 838.792 Step 4 0 138 151 275.972 Stretched V 4 4.11071e-016 158 171 1186.69
Sum Squares 4 5.62063e-027 154 167 169.237
Trecanni 2 3.29667e-014 42 46 219.132
Trefethen4 2 1.14371 28 32 127.961
Watson 6 2.14765 162 188 4488.81
Table 18: Test case results for γ = 0.50.
Name n f NPE NE Time [ms]
Ackley 4 2.44935e-008 336 349 1482.13 Beale 2 4.45085e-014 69 73 563.58 Bohachevsky1 2 0 32 36 148.071 Bohachevsky2 2 0 32 36 92.7685 Booth 2 0 12 16 74.8898 BoxBetts 3 1.15677e-012 86 94 1055.55
Branin 2 0.397887 21 25 241.094 Branin2 2 6.82071e-013 41 45 332.493 Camel3 2 3.67508e-021 29 33 152.929 Camel6 2 -1.03163 21 25 110.013 Chichinadze 2 -43.3159 71 75 587.366 Colville 4 3.19999e-013 266 279 2093.08 Corana 4 0.791625 153 166 2617.03 Eggholder 4 -1627.68 60 73 271.488 Freudenstein-Roth 2 48.9843 80 84 714.311 Gear 4 2.32883e-007 130 143 4345.89 Generalized Rosenbrock 4 4.86212 315 328 3582.54 Goldstein-Price 2 3 31 35 150.231 Hansen 2 -16.0556 28 32 129.873 Hartman 3 3 -3.86278 67 75 655.218 Hartman 6 6 -3.322 60 86 361.409 Himmelblau 2 4.74166e-014 22 26 110.534 Holzman2 4 0.0018523 43 56 1203.83 Hosaki 2 -2.34581 22 26 373.292 Hyperellipsoid 4 6 108 121 556.448 Kowalik 4 0.000959568 1003 1016 4652.97 Leon 2 4.2909e-015 81 85 733.767 Levy7 4 -335.39 260 273 1793.84 Matyas 2 6.70532e-032 14 18 145.244
Max Mod 2 1.45581e-008 84 88 256.351
McCormick 2 -1.91322 18 22 113.492
Michalewitz 2 -0.801303 64 68 435.677
Multimod 4 1.22344 90 103 1756.43
Neumaier Perm0 4 9.44101e-017 213 226 1891.48
Neumaier Perm 4 587.025 125 138 1270.9
Neumaier power sum 4 0.576549 115 128 1624.47
Odd square5 5 -1.39764 333 352 3000.43 Odd Square10 10 -1.3898 269 333 9303.63 Plateau 5 30 150 169 2020.81 Quartic Noise U 4 1.56688 62 75 1752.65 Rana 4 -819.64 75 88 456.671 Rastrigin 4 36 86 99 469.01
Rastrigin2 2 0.484396 26 30 102.382 Schaffer1 2 0.441908 148 152 251.157 Schaffer2 2 0.000301572 81 85 233.094 Schwefel1 2 4 9.86076e-031 142 155 407.55 Schwefel2 21 4 2.78867e-008 213 226 809.188 Schwefel2 22 4 4.99641e-006 216 229 2226.49 Schwefel2 26 4 -2226.55 106 119 678.658 Shekel2 2 -9.49749e+009 119 123 277.912 Shekel4 5 4 -8.99025 87 100 1630.96 Shekel4 7 4 -2.75193 92 105 491.252 Shekel4 10 4 -2.42734 73 86 754.468 Shubert 2 -12.1543 22 26 103.619 Shubert2 4 -25.7418 278 291 12790.2 Shubert3 2 -12.1543 22 26 95.5286 Sphere 4 1.38051e-030 12 25 65.4592 Sphere2 4 1.18329e-030 108 121 1009.75 Step 4 0 126 139 302.162 Stretched V 4 1.27775e-016 173 186 1526.94
Sum Squares 4 8.11647e-027 159 172 129.98
Trecanni 2 7.64733e-016 34 38 210.396
Trefethen4 2 1.14371 26 30 145.913
Watson 6 140.925 62 88 2539.65
Table 19: Test case results for γ = 0.60.
Name n f NPE NE Time [ms]
Ackley 4 6.20278e-009 310 323 1528.98 Beale 2 1.14446e-015 49 53 348.99 Bohachevsky1 2 0 30 34 150.529 Bohachevsky2 2 0 27 31 138.488 Booth 2 0 12 16 77.9132 BoxBetts 3 2.27159e-013 74 82 1155.56 Branin 2 0.397887 26 30 284.16 Branin2 2 3.86524e-014 31 35 249.943 Camel3 2 2.08851e-014 50 54 559.287
Camel6 2 -1.03163 20 24 169.364 Chichinadze 2 -42.8686 75 79 322.939 Colville 4 3.20472 75 88 2064.32 Corana 4 0.742651 95 108 1940.22 Eggholder 4 -1627.68 76 89 653.895 Freudenstein-Roth 2 48.9843 77 81 626.23 Gear 4 1.79367e-007 61 74 1907.99 Generalized Rosenbrock 4 93763.5 68 81 2715.6 Goldstein-Price 2 3 27 31 184.259 Hansen 2 -16.0556 26 30 114.973 Hartman 3 3 -3.86278 62 70 730.969 Hartman 6 6 -3.322 80 106 1747.21 Himmelblau 2 5.82085e-014 27 31 114.27 Holzman2 4 0.00011388 48 61 2159.01 Hosaki 2 -2.34581 21 25 100.279 Hyperellipsoid 4 6 108 121 622.785 Kowalik 4 0.0178262 673 686 6564.24 Leon 2 1.10058e-015 111 115 484.566 Levy7 4 -273.497 151 164 1762.16 Matyas 2 6.70532e-032 15 19 167.938
Max Mod 2 1.94532e-009 80 84 322.253
McCormick 2 -1.91322 18 22 160.4
Michalewitz 2 -0.801303 60 64 925.716
Multimod 4 5.97463 51 64 1725.45
Neumaier Perm0 4 3.32107e-014 184 197 1243.08
Neumaier Perm 4 181.225 231 244 2178.69
Neumaier power sum 4 1.14641 97 110 2376.15
Odd square5 5 -1.39693 198 217 1698.51 Odd Square10 10 -1.34446 161 225 15189.5 Plateau 5 30 288 307 11369.1 Quartic Noise U 4 1.56688 65 78 1531.54 Rana 4 -819.64 102 115 7579.7 Rastrigin 4 36 72 85 392.591 Rastrigin2 2 0.586463 14 18 1243.8 Schaffer1 2 0.126991 47 51 313.477 Schaffer2 2 0.000568414 83 87 222.587
Schwefel1 2 4 8.38867e-030 148 161 272.39 Schwefel2 21 4 7.23258e-008 189 202 814.896 Schwefel2 22 4 8.73436e-008 230 243 1068.95 Schwefel2 26 4 -11346.1 250 263 1130.93 Shekel2 2 -6.42616e+009 133 137 276.473 Shekel4 5 4 -10.1532 94 107 491.636 Shekel4 7 4 -2.75193 96 109 787.365 Shekel4 10 4 -2.42734 90 103 424.865 Shubert 2 -12.1543 30 34 96.21 Shubert2 4 -15.7164 1001 1014 1970.89 Shubert3 2 -12.1543 30 34 111.575 Sphere 4 1.38051e-030 12 25 106.563 Sphere2 4 1.18329e-030 108 121 1466.59 Step 4 0 38 51 17670.7 Stretched V 4 2.0443e-016 173 186 1012.17
Sum Squares 4 1.62256e-027 160 173 155.686
Trecanni 2 3.79813e-014 36 40 152.342
Trefethen4 2 1.14371 27 31 118.084
Watson 6 3.69617 138 164 3132.31
Table 20: Test case results for γ = 0.75.
Name n f NPE NE Time [ms]
Ackley 4 1.98824e-008 393 406 1540.14 Beale 2 4.81366e-015 60 64 493.757 Bohachevsky1 2 0 33 37 5967.23 Bohachevsky2 2 0 35 39 384.783 Booth 2 0 12 16 50.0985 BoxBetts 3 1.00152e-018 91 99 773.601 Branin 2 0.397887 27 31 183.635 Branin2 2 6.77056e-016 33 37 239.538 Camel3 2 5.50199e-018 43 47 191.685 Camel6 2 -1.03163 25 29 121.051 Chichinadze 2 -42.8686 586 590 788.469 Colville 4 42.8902 54 67 1662.19
Corana 4 1.45463 104 117 2225.07 Eggholder 4 -1627.68 49 62 149.86 Freudenstein-Roth 2 48.9843 93 97 628.152 Gear 4 2.35764e-009 777 790 4431.09 Generalized Rosenbrock 4 423828 25 38 1614.22 Goldstein-Price 2 3 36 40 205.016 Hansen 2 -16.0556 25 29 122.165 Hartman 3 3 -3.86278 54 62 496.564 Hartman 6 6 -3.322 61 87 508.967 Himmelblau 2 4.45249e-018 24 28 117.118 Holzman2 4 0.00314207 80 93 2162.25 Hosaki 2 -2.34581 28 32 155.099 Hyperellipsoid 4 6 108 121 968.17 Kowalik 4 0.00129087 179 192 2273.18 Leon 2 6.23523e-013 97 101 695.879 Levy7 4 -327.15 187 200 2160.74 Matyas 2 6.70532e-032 12 16 193.865
Max Mod 2 1.37708e-008 72 76 263.799
McCormick 2 -1.91322 18 22 202.435
Michalewitz 2 -0.801303 56 60 659.788
Multimod 4 2.80362 52 65 1719.54
Neumaier Perm0 4 4.12378 24 37 2023.44
Neumaier Perm 4 731.329 110 123 1664.44
Neumaier power sum 4 1.23694e-007 1003 1016 6498.74
Odd square5 5 -1.39442 144 163 2290.66 Odd Square10 10 -1.33874 103 167 9332.31 Plateau 5 30 444 463 7100.44 Quartic Noise U 4 1.56688 79 92 2311.94 Rana 4 -1015.76 108 121 563.629 Rastrigin 4 36 73 86 410.122 Rastrigin2 2 1.14255 12 16 1128.62 Schaffer1 2 0.345506 82 86 214.709 Schaffer2 2 0.000717202 84 88 243.261 Schwefel1 2 4 8.68592e-030 141 154 390.353 Schwefel2 21 4 4.91748e-008 260 273 1103.49 Schwefel2 22 4 4.00951e-008 237 250 728.381
Schwefel2 26 4 -3608.28 107 120 491.794 Shekel2 2 -1.19821e+010 108 112 238.642 Shekel4 5 4 -10.1532 78 91 341.213 Shekel4 7 4 -2.75193 108 121 946.929 Shekel4 10 4 -2.42734 78 91 1650.85 Shubert 2 -12.1543 22 26 107.515 Shubert2 4 -18.0381 107 120 2145.78 Shubert3 2 -12.1543 22 26 111.424 Sphere 4 1.38051e-030 12 25 156.021 Sphere2 4 1.18329e-030 20 33 282.677 Step 4 0 40 53 651.48 Stretched V 4 3.66234e-015 172 185 994.079
Sum Squares 4 6.10611e-027 161 174 158.339
Trecanni 2 3.62694e-015 41 45 247.679
Trefethen4 2 1.14371 26 30 124.968
Watson 6 4.49467 169 195 4244.59
Table 21: Test case results for γ = 1.00.
Name n f NPE NE Time [ms]
Ackley 4 1.9232e-008 316 329 1321.73 Beale 2 6.88287e-013 71 75 501.467 Bohachevsky1 2 0 33 37 5504.77 Bohachevsky2 2 0 29 33 122.148 Booth 2 0 12 16 50.2967 BoxBetts 3 1.88051e-013 97 105 513.992 Branin 2 0.397887 18 22 164.457 Branin2 2 6.80665e-016 30 34 193.247 Camel3 2 4.87948e-015 62 66 464.464 Camel6 2 -1.03163 23 27 166.712 Chichinadze 2 -41.835 69 73 447.386 Colville 4 4.96308e-015 379 392 1793.33 Corana 4 0.446625 84 97 2160.63 Eggholder 4 -1627.68 84 97 7796.54 Freudenstein-Roth 2 48.9843 84 88 632.401
Gear 4 7.00492e-007 96 109 690.565 Generalized Rosenbrock 4 423828 25 38 1241.68 Goldstein-Price 2 3 33 37 145.616 Hansen 2 -16.0556 25 29 83.5243 Hartman 3 3 -3.86278 64 72 457.036 Hartman 6 6 -3.322 60 86 432.992 Himmelblau 2 5.68138e-026 31 35 154.424 Holzman2 4 0.00823813 52 65 1657.72 Hosaki 2 -2.34581 28 32 125.243 Hyperellipsoid 4 6 108 121 915.223 Kowalik 4 0.00200208 162 175 1880.72 Leon 2 8.22807e-013 112 116 535.683 Levy7 4 -220.32 203 216 980.702 Matyas 2 6.70532e-032 12 16 179.183
Max Mod 2 1.12945e-008 82 86 287.079
McCormick 2 -1.91322 18 22 132.677
Michalewitz 2 -0.801303 71 75 356.462
Multimod 4 2.80362 52 65 1677.01
Neumaier Perm0 4 1.35345e-016 148 161 1982.41
Neumaier Perm 4 857.728 126 139 2260.25
Neumaier power sum 4 0.000138506 225 238 3670.12
Odd square5 5 -1.39751 100 119 2783.77 Odd Square10 10 -1.37159 189 253 10018.2 Plateau 5 30 93 112 2178.56 Quartic Noise U 4 1.56688 62 75 1620.38 Rana 4 -1015.76 109 122 1060.14 Rastrigin 4 36 87 100 371.835 Rastrigin2 2 1.14255 12 16 1590.21 Schaffer1 2 0.429723 49 53 202.799 Schaffer2 2 0.000976969 83 87 229.593 Schwefel1 2 4 1.67633e-029 133 146 227.14 Schwefel2 21 4 9.64061e-008 214 227 727.604 Schwefel2 22 4 3.4432e-008 192 205 743.231 Schwefel2 26 4 -523.349 51 64 1571.12 Shekel2 2 -3.01298e+009 111 115 233.551 Shekel4 5 4 -10.1532 65 78 304.753
Shekel4 7 4 -2.75193 89 102 419.127 Shekel4 10 4 -2.42734 74 87 318.408 Shubert 2 -12.1543 24 28 128.507 Shubert2 4 -25.3183 68 81 1061.73 Shubert3 2 -12.1543 24 28 121.531 Sphere 4 1.38051e-030 12 25 151.881 Sphere2 4 1.18329e-030 20 33 355.404 Step 4 0 56 69 1041.42 Stretched V 4 4.93701e-016 185 198 738.344
Sum Squares 4 2.61941e-027 150 163 129.691
Trecanni 2 1.47168e-013 44 48 210.909
Trefethen4 2 1.14371 27 31 140.614
Watson 6 0.00228767 807 833 6089.29
11
Results With Circular Level Curves
Here are the complete test results when using the transformation that create circular level curves.
Table 22: Test case results for circular level curves.
Name n f NPE NE Time [ms]
Ackley 4 2.50139e-008 317 330 4757.35 Beale 2 2.83022e-009 74 78 10507.7 Bohachevsky1 2 0 52 56 29.9056 Bohachevsky2 2 0 35 39 32.3659 Booth 2 0.00125039 19 23 147.655 BoxBetts 3 0.000129283 86 94 15140.8 Branin 2 6.94023 735 739 868.756 Branin2 2 1.06732e-015 65 69 94.863 Camel3 2 0.00288433 28 32 4343.53 Camel6 2 -1.03163 74 78 19.7405 Chichinadze 2 -41.9251 76 80 68.1644 Colville 4 1.77838 1003 1016 1492.19 Corana 4 68653.8 272 285 714.876
Eggholder 4 -2893.15 846 859 7191.01 Freudenstein-Roth 2 49.5138 238 242 497.263 Gear 4 4.78406e-005 26 39 6887.2 Generalized Rosenbrock 4 398.221 1003 1016 2459.55 Goldstein-Price 2 32.3977 465 469 235.818 Hansen 2 -62.2147 511 515 299.182 Hartman 3 3 -3.86278 179 187 78.4181 Hartman 6 6 -3.322 1000 1026 2432.53 Himmelblau 2 3.09077e-016 113 117 50.6205 Holzman2 4 2.72803e-007 129 142 1024.59 Hosaki 2 -2.34578 35 39 5379.9 Hyperellipsoid 4 6.16506 47 60 414.94 Kowalik 4 0.00825667 282 295 1591.43 Leon 2 3.10804 298 302 109.055 Levy7 4 -37.7635 908 921 1034.64 Matyas 2 0.0318099 34 38 9803.57 Max Mod 2 0.527321 12 16 1924.29 McCormick 2 -1.91322 39 43 216.803 Michalewitz 2 -0.801303 549 553 15025.9 Multimod 4 0.0286207 322 335 1742.98
Neumaier Perm0 4 6.51634e-005 645 658 1143.43
Neumaier Perm 4 566.135 1003 1016 1638.4
Neumaier power sum 4 0.578279 1003 1016 4184.87
Odd square5 5 -1.39762 525 544 428303 Odd Square10 10 -1.3898 620 684 129460 Plateau 5 34 182 201 304.628 Quartic Noise U 4 1.56688 547 560 1378.42 Rana 4 -8466.43 196 209 1510.25 Rastrigin 4 37.9028 251 264 187.108 Rastrigin2 2 1.08989 105 109 31.7072 Schaffer1 2 0.499996 603 607 591.787 Schaffer2 2 10.2585 998 1002 1038.65 Schwefel1 2 4 8.61568e-007 135 148 48089.3 Schwefel2 21 4 2.09245 531 544 476.961 Schwefel2 22 4 0.05837 545 558 493.057 Schwefel2 26 4 -20213.4 92 105 3780.25
Shekel2 2 -21.7846 997 1001 1024.42 Shekel4 5 4 -0.442672 41 54 731.557 Shekel4 7 4 -10.4029 459 472 405.016 Shekel4 10 4 -3.83543 262 275 189.952 Shubert 2 -11.8177 997 1001 818.722 Shubert2 4 -25.7418 219 232 191.984 Shubert3 2 -11.8177 997 1001 800.843 Sphere 4 0.173462 59 72 333.914 Sphere2 4 0.0492775 313 326 11511.8 Step 4 30 69 82 356.079 Stretched V 4 1.28341e-005 553 566 2741.24 Sum Squares 4 0.00499003 40 53 45848 Trecanni 2 2.42348e-006 48 52 159.95 Trefethen4 2 1.14371 73 77 16.7756 Watson 6 0.169902 1003 1029 4620.78
12
Results When Varying the Model Weight
Exponent
Figure 4 displays the data profiles for the different values of α that were tested, including α = ∞ (a single model with circular trust region) for refer-ence.
0 200 400 600 800 1000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
α = ∞ (circular trust region)
α = 1.0
α = 2.0
α = 3.0
α = 4.0
α = 5.0
Figure 4: Data profiles when varying the model weight exponent α. Table 23: Test case results when using circular level curves.
Name n f NPE NE Time [ms]
Ackley 4 2.48485e-008 178 191 379.119 Beale 2 4.20919e-014 64 68 657.528 Bohachevsky1 2 0 31 35 94.2397 Bohachevsky2 2 0 31 35 108.926 Booth 2 0 12 16 99.3204 BoxBetts 3 1.01949e-014 75 83 795.13 Branin 2 0.397887 37 41 203.885 Branin2 2 0.102163 27 31 103.411 Camel3 2 3.71063e-017 32 36 143.205 Camel6 2 -1.03163 23 27 75.1575 Chichinadze 2 -39.8579 469 473 799.436 Colville 4 3.69146e-014 633 646 2801.08 Corana 4 0.416625 148 161 470.641 Eggholder 4 -1627.68 285 298 517.482 Freudenstein-Roth 2 48.9843 82 86 807.036
Gear 4 1.79367e-007 147 160 298.985 Generalized Rosenbrock 4 0.000772866 1004 1017 4975.6 Goldstein-Price 2 3 33 37 114.34 Hansen 2 -16.0556 28 32 100.936 Hartman 3 3 -3.86278 86 94 613.094 Hartman 6 6 -3.322 114 140 726.49 Himmelblau 2 7.44883e-017 27 31 65.4227 Holzman2 4 8.30202e-029 286 299 691.588 Hosaki 2 -2.34581 24 28 84.0476 Hyperellipsoid 4 6 108 121 101.862 Kowalik 4 0.000795912 1003 1016 2668.54 Leon 2 2.69533e-014 119 123 937.858 Levy7 4 -133.071 88 101 488.378 Matyas 2 6.70532e-032 14 18 195.194
Max Mod 2 1.3471e-008 68 72 185.774
McCormick 2 -1.91322 17 21 107.04
Michalewitz 2 -0.801303 58 62 201.385
Multimod 4 9.37148e-009 175 188 288.203
Neumaier Perm0 4 3.24093e-015 158 171 2142.69
Neumaier Perm 4 5.96585e-013 258 271 561.458
Neumaier power sum 4 0.000107102 1003 1016 5471.39
Odd square5 5 -1.39763 950 969 3567.81 Odd Square10 10 -1.3898 1003 1067 14687.6 Plateau 5 30 181 200 243.444 Quartic Noise U 4 1.56688 217 230 444.732 Rana 4 -819.64 92 105 254.014 Rastrigin 4 36 66 79 161.668 Rastrigin2 2 0.484396 30 34 37.4769 Schaffer1 2 0.5 1002 1006 1076.39 Schaffer2 2 0.000288036 82 86 384.751 Schwefel1 2 4 9.86076e-031 145 158 146.581 Schwefel2 21 4 3.85931e-008 204 217 602.121 Schwefel2 22 4 1.10782e-007 159 172 431.48 Schwefel2 26 4 517.35 126 139 352.312 Shekel2 2 14.5631 99 103 152.113 Shekel4 5 4 -10.1532 63 76 214.767
Shekel4 7 4 -2.75193 59 72 184.655 Shekel4 10 4 -3.83543 92 105 334.665 Shubert 2 -15.7637 27 31 89.4401 Shubert2 4 -15.5105 195 208 1058.54 Shubert3 2 -15.7637 27 31 94.4822 Sphere 4 1.38051e-030 12 25 9.42447 Sphere2 4 1.18329e-030 20 33 374.867 Step 4 0 135 148 266.991 Stretched V 4 2.42098e-015 174 187 1273.75
Sum Squares 4 6.12945e-028 150 163 190.216
Trecanni 2 2.38991e-015 34 38 125.884
Trefethen4 2 1.14371 30 34 73.8833
Watson 6 0.00228767 679 705 6722.05
Table 24: Test case results for α = 2.
Name n f NPE NE Time [ms]
Ackley 4 3.10774e-008 206 219 531.544 Beale 2 2.43622e-014 62 66 442.574 Bohachevsky1 2 0 31 35 99.8227 Bohachevsky2 2 0 31 35 118.219 Booth 2 0 12 16 107.36 BoxBetts 3 1.01949e-014 75 83 814.372 Branin 2 0.397887 37 41 243.58 Branin2 2 0.102163 28 32 134.21 Camel3 2 3.71063e-017 32 36 179.789 Camel6 2 -1.03163 24 28 88.6017 Chichinadze 2 -39.8579 242 246 445.002 Colville 4 2.24188e-013 286 299 1754.51 Corana 4 0.416625 148 161 372.143 Eggholder 4 -1627.68 196 209 467.047 Freudenstein-Roth 2 48.9843 82 86 868.849 Gear 4 2.84507e-006 149 162 338.031 Generalized Rosenbrock 4 993.825 1003 1016 3545.68 Goldstein-Price 2 3 26 30 67.1912
Hansen 2 -63.4352 36 40 54.0088 Hartman 3 3 -3.86278 63 71 369.297 Hartman 6 6 -3.322 112 138 724.073 Himmelblau 2 4.60264e-017 30 34 71.1159 Holzman2 4 8.69203e-030 232 245 560.618 Hosaki 2 -2.34581 26 30 97.8868 Hyperellipsoid 4 6 108 121 124.191 Kowalik 4 0.000307486 1003 1016 3801.63 Leon 2 2.55408e-013 101 105 868.918 Levy7 4 -150.5 212 225 1364.57 Matyas 2 6.70532e-032 14 18 194.19
Max Mod 2 2.08013e-009 64 68 199.747
McCormick 2 -1.91322 17 21 91.6293
Michalewitz 2 -0.801303 45 49 1730.64
Multimod 4 3.61432e-008 167 180 535.523
Neumaier Perm0 4 2.81333e-014 200 213 1235.76
Neumaier Perm 4 550.469 159 172 560.525
Neumaier power sum 4 0.00116165 1002 1015 5759.89
Odd square5 5 -1.39763 1003 1022 4133.26 Odd Square10 10 -1.3585 1003 1067 15417.1 Plateau 5 30 181 200 237.954 Quartic Noise U 4 1.56688 167 180 256.589 Rana 4 -819.64 117 130 961.561 Rastrigin 4 36 67 80 165.862 Rastrigin2 2 0.484396 27 31 53.227 Schaffer1 2 0.441908 188 192 279.231 Schaffer2 2 0.000250228 81 85 223.231 Schwefel1 2 4 9.86076e-031 145 158 168.087 Schwefel2 21 4 1.42004e-008 196 209 519.484 Schwefel2 22 4 3.4309e-008 193 206 504.117 Schwefel2 26 4 398.831 96 109 293.664 Shekel2 2 -1.00064e+010 102 106 165.811 Shekel4 5 4 -10.1532 89 102 319.503 Shekel4 7 4 -10.4029 81 94 253.846 Shekel4 10 4 -2.42734 95 108 285.912 Shubert 2 -12.1543 36 40 182.376
