Matrix-Less Methods forComputing Eigenvalues of LargeStructured Matrices

UNIVERSITATISACTA UPSALIENSIS

UPPSALA 2018

Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1652

Matrix-Less Methods for

Computing Eigenvalues of Large Structured Matrices

SVEN-ERIK EKSTRÖM

ISSN 1651-6214 ISBN 978-91-513-0288-1 urn:nbn:se:uu:diva-346735

Dissertation presented at Uppsala University to be publicly examined in 2446 ITC, Lägerhyddsvägen 2, hus 2, Uppsala, Friday, 18 May 2018 at 10:15 for the degree of Doctor of Philosophy. The examination will be conducted in English. Faculty examiner: Professor Lothar Reichel (Department of Mathematical Sciences, Kent State University).

Abstract

Ekström, S.-E. 2018. Matrix-Less Methods for Computing Eigenvalues of Large Structured Matrices. Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1652. 81 pp. Uppsala: Acta Universitatis Upsaliensis.

ISBN 978-91-513-0288-1.

When modeling natural phenomena with linear partial differential equations, the discretized system of equations is in general represented by a matrix. To solve or analyze these systems, we are often interested in the spectral behavior of these matrices. Whenever the matrices of interest are Toeplitz, or Toeplitz-like, we can use the theory of Generalized Locally Toeplitz (GLT) sequences to study the spectrum (eigenvalues). A central concept in the theory of GLT sequences is the so-called symbol, that is, a function associated with a sequence of matrices of increasing size. When sampling the symbol and when the related matrix sequence is Hermitian (or quasi-Hermitian), we obtain an approximation of the spectrum of a matrix of a fixed size and we can therefore see its general behavior. However, the so-computed approximations of the eigenvalues are often affected by errors having magnitude of the reciprocal of the matrix size.

In this thesis we develop novel methods, which we call "matrix-less" since they neither store the matrices of interest nor depend on matrix-vector products, to estimate these errors.

Moreover, we exploit the structures of the considered matrices to efficiently and accurately compute the spectrum.

We begin by considering the errors of the approximate eigenvalues computed by sampling the symbol on a uniform grid, and we conjecture the existence of an asymptotic expansion for these errors. We devise an algorithm to approximate the expansion by using a small number of moderately sized matrices, and we show through numerical experiments the effectiveness of the algorithm. We also show that the same algorithm works for preconditioned matrices, a result which is important in practical applications. Then, we explain how to use the approximated expansion on the whole spectrum for large matrices, whereas in earlier works its applicability was restricted only to certain matrix sizes and to a subset of the spectrum. Next, we demonstrate how to use the so-developed techniques to investigate, solve, and improve the accuracy in the eigenvalue computations for various differential problems discretized by the isogeometric analysis (IgA) method. Lastly, we discuss a class of non-monotone symbols for which we construct the sampling grid yielding exact eigenvalues and eigenvectors.

To summarize, we show, both theoretically and numerically, the applicability of the presented matrix-less methods for a wide variety of problems.

Keywords: Toeplitz matrices, eigenvalues, eigenvalue asymptotics, polynomial interpolation, extrapolation, generating function and spectral symbol

Sven-Erik Ekström, Department of Information Technology, Division of Scientific Computing, Box 337, Uppsala University, SE-751 05 Uppsala, Sweden.

© Sven-Erik Ekström 2018 ISSN 1651-6214

ISBN 978-91-513-0288-1

urn:nbn:se:uu:diva-346735 (http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-346735)

https://doi.org/10.1080/10586458.2017.1320241

https://doi.org/10.1007/s11075-017-0404-z

https://doi.org/10.1007/s11075-018-0508-0

−∆u = λu

http://www.it.uu.se/research/publications/reports/2017-016

https://doi.org/10.1002/nla.2137

http://www.it.uu.se/research/publications/reports/2018-005

A u = b,

A u b

u

A_nun= bn,

A_n ∈ C^n×n A un ∈ Cⁿ

bn ∈ Cⁿ

n A_n

A_n

A_nun = bn

A_n

A_n A_n A_n

A_n

A_n

A_n A_k

k n

n× n

[ai−j]ⁿ_i,j=1=

⎡

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎣

a₀ a₋₁ a₋₂ . . . a_1−n a1

a₂

a₋₂ a₋₁ a_n−1 . . . a₂ a₁ a₀

⎤

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎦ .

f ∈ L¹(−π, π)

{Tn(f)}n T_n(f) = [ ˆf_i−j]ⁿ_i,j=1 fˆ_ω f

fˆ_ω= 1 2π

' π

−πf(θ)e^−iωθdθ, i² = −1, ω ∈ Z.

f

{Tn(f)}n {Tn(f)}n

f

T_n(f) T(f) = [ ˆf_i−j]^∞_i,j=1= [ ˆf_i−j]^∞_i,j=−∞

f T_n(f)

A_n = A^∗_n n f f(θ) = f(−θ) ∈ R

T_n(f) n

T_n(f) 2p + 1

(Tn(f))_i=⁽0, . . . , 0, ˆf_p, . . . , ˆf₁, ˆf₀, ˆf₋₁, . . . , ˆf_−p,0, . . . , 0⁾,

f(θ) = ^*p

ω=−p

fˆ_ωe^iωθ.

fˆ_ω = ˆf_−ω ω = 1, . . . , p fˆ_ω(e^iωθ+ e^−iωθ) = 2 ˆf_ωcos(ωθ)

f(θ) = ˆf0+ 2^*p

ω=1

fˆ_ωcos(ωθ).

fˆ_ω ∈ R ω = 0, . . . p

n

fˆ_−p fˆ0 fˆ_p p

f(θ) = 2 − 2 cos(θ).

f(θ) fˆ₀ = 2 ˆf₁ = ˆf₋₁ = −1 fˆ_ω = 0 ω ̸= {−1, 0, 1} 2p + 1 = 3 f(θ)

p= 1

n= 5 f

T₅(f) =

⎡

⎢⎢

⎢⎢

⎢⎢

⎣

fˆ0 fˆ₋₁ 0 0 0 fˆ₁ fˆ₀ fˆ₋₁ 0 0 0 ˆf₁ fˆ₀ fˆ₋₁ 0 0 0 fˆ1 fˆ0 fˆ₋₁ 0 0 0 fˆ₁ fˆ₀

⎤

⎥⎥

⎥⎥

⎥⎥

⎦

=

⎡

⎢⎢

⎢⎢

⎣

2 −1 0 0 0

−1 2 −1 0 0 0 −1 2 −1 0 0 0 −1 2 −1 0 0 0 −1 2

⎤

⎥⎥

⎥⎥

⎦.

T_n(f)

−∆u = −∂²u

∂x² ≈ 1

h²T_n(f)un = 1 h²

⎡

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎢⎣

2u1− u2

−u1+ 2u2− u3

−uj−1+ 2uj− uj+1

−un−2+ 2un−1− un

−un−1+ 2un

⎤

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎥⎦ ,

h= 1/(n + 1) un = [u1, . . . , u_n]^T

A₅=

⎡

⎢⎢

⎢⎢

⎣

−1 2 −1 0 0

0 −1 2 −1 0

0 0 −1 2 −1

0 0 0 −1 2

0 0 0 0 −1

⎤

⎥⎥

⎥⎥

⎦=

⎡

⎢⎢

⎢⎢

⎢⎢

⎣

fˆ0 fˆ₋₁ fˆ₋₂ 0 0 0 fˆ₀ fˆ₋₁ fˆ₋₂ 0 0 0 fˆ₀ fˆ₋₁ fˆ₋₂ 0 0 0 fˆ0 fˆ₋₁ 0 0 0 0 fˆ₀

⎤

⎥⎥

⎥⎥

⎥⎥

⎦ .

A₅ = T5(f)

f(θ) = ^*p

ω=−p

fˆ_ωe^iωθ= −1 + 2e^−iθ− e^−i2θ,

p= 2 ˆf1 = ˆf2 = 0 ˆf0 = ˆf₋₂ = −1 fˆ₋₁ = 2

−∆u = −∂²u

∂x² ≈ 1

h²T_n(f)un = 1 h²

⎡

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎢⎣

−u1+ 2u2− u3

−u2+ 2u3− u4

−uj+ 2uj+1− uj+2

−un−1+ 2u_n

−un

⎤

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎥⎦ ,

h = 1/(n + 1) un = [u1, . . . , u_n]^T

f ∈ L^∞(−π, π) T_n(f) λ_j(Tn(f)) j = 1, . . . , n

nlim→∞

1 n

*n j=1

F(λj(Tn(f))) = 1 2π

' _π

−π

F(f(θ))dθ,

F : R → C

f ∈ L^∞(−π, π) σ_j(Tn(f)), j = 1, . . . , n

n→∞lim 1 n

*n j=1

F(σj(Tn(f))) = 1 2π

' _π

−π

F (|f(θ)|) dθ, F : R → C

f T_n(f)

f ∈ L¹(−π, π) f ∈ L¹(−π, π)

f f : [−π, π] → C

A B U0

C\E R(f) E R(f )

f (−π) f (θd) f (π)

A B

U0

f : [−π, π] → C f ∈ L^∞(−π, π)

f θd Z = C\U0

Tn(f) Tn(f) n= 500

E R(f )

f Z= C\U⁰ = E R(f)∪A∪B

T_n(f)

ε > 0 T_n(f)

ε Z o(n) n → ∞

T_n(f) f

n= 500 Z

f E R(f )

f T_n(f)

(m_f, M_f) m_f M_f

f M_f > 0 f

T_n(f)

p ∈ [1, ∞]

∥ · ∥p p

p ∥ · ∥∞ = ∥ · ∥

A A^†

A

{An}n

A_n ∈ C^n×n n {An}n

A_n

{An}n ∼σ f {An}n

f : D ⊂ R^k → C D 0 < µk(D) < ∞

nlim→∞

1 n

*n j=1

F(σj(An)) = 1 µ_k(D)

'

D

F (|f(y¹, . . . , y_k)|) dy¹. . .dyk,

F ∈ C0(C) µ_k R^k C₀(C)

{An}n ∼λf

{An}n f

nlim→∞

1 n

*n j=1

F(λ_j(A_n)) = 1 µ_k(D)

'

D

F(f(y1, . . . , y_k)) dy1. . .dy_k,

F ∈ C0(C)

{Zn}n

{Zn} ∼σ 0 ∥Zn∥ → 0 {Zn}n ∼σ 0

n ∈ N a : [0, 1] → C

D_n(a) a n× n

(Dn(a))_i,i= a (i/n) , i= 1, . . . , n.

{An}n

{{Bn,m}n}m {{Bn,m}n}m

{An}n m {Bn,m}n

{An}n A_n B_n,m

{Bn,m}n −→ {An}n {{Bn,m}n}m {An}n

{An}n

f : [0, 1] × [0, π] → C {An}n∼ f {An}n

f

f : [0, π] → C

{Tn(f)}n∼λ f λ_j(Tn(f)) T_n(f) f(θ) θ_j,n j= 1, . . . , n n

λ_j(Tn(f)) = f(θj,n) + Ej,n,

E_j,n O(h)

h n θ_j,n [0, π] E_j,n

n→ ∞ O(h)

n θ_j,n

n j j ∈ {1, . . . , n}

j_S ⊆ {1, . . . , n} θ_jS,n θ_j,n

j ∈ j_S h

τ τ

{An}n ∼ f {An}n ∼σ f A_n {An}n ∼λf

{An}n ∼ f {An}n

A_n= X_n+ Y_n

∥Xn∥, ∥Yn∥ ≤ C C n

X_n

n→∞lim

∥Yn∥1

n = 0 {An}n ∼λ f

{Tn(f)}n ∼ f(x, θ) = f(θ) f ∈ L¹(−π, π) {Dn(a)}n ∼ f(x, θ) = a(x) a : [0, 1] → C {Zn}n∼ f(x, θ) = 0 {Zn}n ∼σ 0 {An}n ∼ f {A^∗n}n ∼ f¯ A^∗_n

A_n

{An}n ∼ f {Bn}n ∼ g A_n B_n {αAn+ βB_n}n ∼ αf+ βg α, β∈ C {AnB_n}n ∼ f g

{An}n ∼ f f ̸= 0 {A^†n}n∼ f⁻¹ {An}n ∼ f A_n

{F (An)}n ∼ F(f) F : C → C

{An}n ∼ f

{Bn,m}n ∼ f_m {Bn,m}n −→ {An}n f_m → f [0, 1] × [−π, π]

τn

j h

τn jπ/(n + 1) 1, . . . , n 1/(n + 1) τn(0, 0) τn−1 jπ/n 1, . . . , n − 1 1/n τn−1(0, 0) τn−2 jπ/(n − 1) 1, . . . , n − 2 1/(n − 1) τⁿ−2(0, 0)

τ_n−1⁰ (j − 1)π/n 1, . . . , n 1/n τn(1, 1) = 0 ∪ τn−1(0, 0) τ_n^π₋₁ jπ/n 1, . . . , n 1/n τn(−1, −1) = τⁿ−1(0, 0) ∪ π τ_n^0,π₋₂ (j − 1)π/(n − 1) 1, . . . , n 1/(n − 1) 0 ∪ τⁿ−2(0, 0) ∪ π

τ_n−1^0,π (j − 1)π/n 1, . . . , n + 1 1/n 0 ∪ τn−1(0, 0) ∪ π

A_n τ A⁻¹_n

τ τ

τ_n

fˆ₀, ˆf₁, ˆf₋₁∈ C τ

f(θ) = ˆf₀+ ˆf₁e^iθ+ ˆf₋₁e^−iθ. T_n(f) g T_n(f) ∼ Tn(g)

λ_j(Tn(f)) = g(θj,n) = ˆf0+ 2⁺fˆ1fˆ₋₁cos(θj,n), θ_j,n τ_n

λ_j(Tn(f))

xj,n=⁽x^(j,n)₁ , . . . , x^(j,n)_k , . . . , x^(j,n)_n ^)T,

x^(j,n)_k =^,+fˆ1/ ˆf₋₁ -k

sin (kθj,n) .

+fˆ₁fˆ₋₁ ⁺fˆ₁/ ˆf₋₁

fˆ₁ =^.._.fˆ₁^.._.e^iφ1 fˆ₋₁=^.._.fˆ₋₁^.._.e^iφ−1 φ1, φ₋₁∈ [0, 2π]

+fˆ1fˆ₋₁=⁺fˆ1

+fˆ₋₁=^/.._.fˆ1

..

....fˆ₋₁^.._.e^{i(φ1+φ−1)/2}, +fˆ₁/ ˆf₋₁=⁺fˆ₁/

+fˆ₋₁=^/.._.fˆ₁^.._./^.._.fˆ₋₁^.._.e^{i(φ1−φ−1)/2}.

fˆ₁ = ˆf₋₁= −1 = e^iπ +(−1)(−1) =√

−1√

−1 = i · i = −1, +(−1)/(−1) =√

−1/√

−1 = i/i = 1.

T_n(f) n = 5 f(θ) = 2 − 2 cos(θ) g(θ) = f(θ) τ_n θ_j,5 = jπ/6 j= 1, . . . , 5

T₅(f) T5(f)xj,5= λj(T5(f))xj,5 j= 1, . . . , 5

T5(f) f(θ) = 2 − 2 cos(θ) j λj(T5(f)) xj,5

1 2 − 2 cos (πh) [sin (πh) , sin (2πh) , sin (3πh) , sin (4πh) , sin (5πh) ]^T 2 2 − 2 cos (2πh) [sin (2πh) , sin (4πh) , sin (6πh) , sin (8πh) , sin (10πh)]^T 3 2 − 2 cos (3πh) [sin (3πh) , sin (6πh) , sin (9πh) , sin (12πh) , sin (15πh)]^T 4 2 − 2 cos (4πh) [sin (4πh) , sin (8πh) , sin (12πh) , sin (16πh) , sin (20πh)]^T 5 2 − 2 cos (5πh) [sin (5πh) , sin (10πh) , sin (15πh) , sin (20πh) , sin (25πh)]^T

f(θ) = 2 − 2 cos(θ) θ ∈ [0, π]

λj(T5(f)) f(θ)

τ₅

T_n(f)

{Tn(f)}n ∼ f

{Tn(f)}n ∼σ,λf T_n(f)

τ

T˜_n(f) τ_n(1, 1)

τ_n(1, 1) = τ_n⁰₋₁ f

T˜_n(f) = Tn(f) + Rn T_n(f)

f R_n R_n

(Rn)1,1 = ˆf1 = −1 (Rn)n,n = ˆf1 = −1 T˜_n(f) τ_n(−1, −1)

τ_n(−1, −1) = τn^π−1 f

T˜_n(f) = Tn(f) − Rn R_n

T_n(f)

{Tn(f)}n ∼ f, {Tn(f)}n ∼σ f, {Tn(f)}n ̸∼λ f.

{Tn(f)}n f

n→ ∞ E_j,n= λ_j(T_n(f)) − f(θj,n)

f {Tn(f)}n

f(θ) = 2 − e^iθ+ 2ie^−iθ, n= 5

T₅(f) =

⎡

⎢⎢

⎢⎢

⎣

2 2i 0 0 0

−1 2 2i 0 0 0 −1 2 2i 0 0 0 −1 2 2i 0 0 0 −1 2

⎤

⎥⎥

⎥⎥

⎦.

g(θ) T_n(f) ∼ Tn(g) n

f n

g

g(θ) = 2 + 2√

2e^i3π/4cos(θ), n= 5

T₅(g) =

⎡

⎢⎢

⎢⎢

⎢⎣

2 √2e^i3π/4 0 0 0

√2e^i3π/4 2 √2e^i3π/4 0 0

0 √2e^i3π/4 2 √2e^i3π/4 0

0 0 √2e^i3π/4 2 √2e^i3π/4

0 0 0 √2e^i3π/4 2

⎤

⎥⎥

⎥⎥

⎥⎦ .

T_n(g) T_n(f)

D_n T_n(g) = DnT_n(f)D_n⁻¹ fˆ₁fˆ₋₁̸= 0

n= 5 D₅

D5 =

⎡

⎢⎢

⎢⎢

⎣

1 0 0 0 0 0 γ 0 0 0 0 0 γ² 0 0 0 0 0 γ³ 0 0 0 0 0 γ⁴

⎤

⎥⎥

⎥⎥

⎦,

γ = ⁺fˆ₋₁/ ˆf1 = √2i/√−1 = 1 − i

{Tn(f)}n ∼ ,σ f {Tn(f)}n ̸∼λf {Tn(f)}n ∼λg {Tn(g)}n ∼ ,σ,λg

f

g g τ5

T₅(f) ∼ T5(g)

|f(θ)| |g(θ)| θ∈ [−π, π]

T_n(f) f : [−π, π] → C

g : [−π, π] → C n T_n(f)

T_n(g) f τ₅

g τ5

T₅(f) T₅(g)

T_n(f)

T_n(g) |f(θ)| |g(θ)|

θ ∈ [−π, π] |f| |g|

θ∈ [−π, π]

τ₅ [−π, 0] ¯θj,5

|f(¯θj,5)| σ_j(T5(f))

T₅(g) (A^∗A = AA^∗) σ_j(T5(g))

|λj(T5(g))| ¯θ_j,n τ5

T₅(g) |g(θ)| |g(¯θj,n)|

0−(a(x)u^′(x))^′= b(x), x ∈ (0, 1), u(x) = 0, x∈ {0, 1},

a(x) b(x) a(x) = 1 −u^′′(x)

−(a(x)u^′(x))^′ = −a^′(x)u^′(x) − a(x)u^′′(x) ≈ Anun = bn,

A_n =

⎡

⎢⎢

⎢⎢

⎢⎢

⎣

a_1/2+ a3/2 −a3/2

−a3/2 a_3/2+ a5/2 −a5/2

−a5/2

−an−1/2

−a_n−1/2 a_n−1/2+ an+1/2

⎤

⎥⎥

⎥⎥

⎥⎥

⎦ ,

(An)_i=⁽0, . . . , 0, −ai−1/2, a_i−1/2+ ai+1/2,−ai+1/2,0, . . . , 0⁾, a_i = a(xi) xi = ih i = 0, . . . , n + 1 h = 1/(n + 1) bi = b(xi) un = [u1, . . . , u_n]^T bn= [b1, . . . , b_n]^T

{An}n

D_n(a) a(x)

T_n(f) f(θ) = 2 − 2 cos(θ)

D_n(a) =

⎡

⎢⎢

⎢⎣ a₁

a2

a_n

⎤

⎥⎥

⎥⎦, T_n(f) =

⎡

⎢⎢

⎢⎢

⎣

2 −1

−1

−1

−1 2

⎤

⎥⎥

⎥⎥

⎦,

D_n(a)Tn(f) =

⎡

⎢⎢

⎢⎢

⎢⎢

⎣

2a1 −a1

−a2 2a2 −a2

−a3

−an−1

−an 2an

⎤

⎥⎥

⎥⎥

⎥⎥

⎦ .

A_n= Dn(a)Tn(2 − 2 cos(θ)) + En

(En)i,j=

⎧⎪

⎪⎪

⎪⎨

⎪⎪

⎪⎪

⎩

a_(2i−1)/2− 2a2i/2+ a(2i+1)/2, i= j, i = 1, . . . , n, a_2i/2− a_(2i−1)/2, i= j + 1, i = 2, . . . , n, a_2i/2− a(2i+1)/2, i= j − 1, i = 1, . . . , n − 1,

0, .

a(x)

E_n ∥En∥ → 0 n → ∞

{En}n ∼σ0 {En}n {En}n∼ 0 {Dn(a)}n ∼ a(x), {Tn(f)}n ∼ f(θ), {En}n ∼ 0

{An}n ∼ a(x)(2 − 2 cos(θ)) A_n

{An}n ∼σ,λa(x)(2 − 2 cos(θ))

a(x)

a(x) =

⎧⎪

⎪⎨

⎪⎪

⎩

2 + x/2, x∈ [0, 1/3), e^πx/2, x∈ [1/3, 2/3), 2 + cos(3x), x ∈ [2/3, 1].

a(x)

An Dn(a)Tn(f) n = {5, 1000}

ξr r= 100 ξr(θj,5)

a(x) A5

D₅(a)T5(f)

ξ₁₀₀

100² = 10000 τ₅ ξ_r

r

Gr= {(i/n, j/(n + 1)); i, j = 1, . . . , r} ,

a(x)f(θ) (x, θ) ∈ Gr r² ξ_r

ξ_r ξ(θ) r→ ∞

ξ_r D_n(a)Tn(f) ξ(θ)

ξ(θ) θ_j,n

ξ_r θ_j,n

n= 1000 A_n D_n(a)Tn(f)

ξ₁₀₀