Selberg Zeta Functions and Transfer Operators for Modular Groups Jeremie Brieussel

Selberg Zeta Functions and Transfer Operators for Modular Groups

Jeremie Brieussel

U.U.D.M. Project Report 2004:14

Examensarbete i matematik, 10 poäng Handledare och examinator: Andreas Juhl

Juni 2004

Department of Mathematics

Uppsala University

OPERATORS FOR MODULAR GROUPS

J. BRIEUSSEL

UNDER THE DIRECTION OF A. JUHL

Contents

1. Introduction 1

2. The Selberg zeta function 1

3. Reduction theory 2

4. The transfer operator Ls 4

5. Selberg zeta functions and transfer operators 7

6. A dynamical point of view on Z_Γ0 10

References 15

1. Introduction

In this paper we are interested in the Selberg zeta functions for mod- ular groups. We first define it from a purely algebraical point of view, following [ManMar], as a product over conjugacy classes of primitive elements. Then, we show how it is related to the so-called transfer operator (sections 4 and 5). To prove the relation, we need a good way to represent the primitive conjugacy classes. This is the object of section 3, which follows [LewZag]. Finally, we show that the Selberg zeta functions can also been defined from a dynamical point of view, as a product over the closed geodesics of the quotient spaces Γ\H.

2. The Selberg zeta function

Definition 2.1. Let GL₂(Z) denote the set of 2 by 2 invertible matrices γ with integer entries such that γ⁻¹ also has integer entries. If γ ∈ GL2(Z), then det(γ) and det(γ)⁻¹ are both integers, so that det(γ) =

±1. The discriminant of γ ∈ GL₂(Z) is defined to be ∆(γ) := tr (γ)²− 4 det(γ) and we call γ hyperbolic if ∆(γ) ≥ 0 and also tr (γ) > 0. If γ is in SL₂(Z) this coincides with the usual definition. We set

N (γ) :=  tr (γ) + ∆(γ)^1/2 2

²

, χ_s(γ) := N (γ)^−s

1 − det(γ)N (γ)⁻¹, s ∈ C

Date: June, 2004.

1

We notice that if γ is conjugate to  λ 0 0 1/λ



with λ > 1, then N (γ) = λ².

We will define the Selberg zeta function for a subgroup Γ of finite in- dex in GL₂(Z) and a subgroup Γ0 of finite index in SL₂(Z). P (respec- tively P₀) will denote the coset space Γ\GL₂(Z) (resp. Γ0\SL₂(Z)). We also define ρP (resp. ρP0) to be the left regular representation of GL2(Z) (resp. SL2(Z)) in the space of functions on P , that is ρ^P(γ)(Γγ⁰) = Γγ⁰γ⁻¹ (resp. ρ_P0(γ)(Γ₀γ⁰) = Γ₀γ⁰γ⁻¹) and, if u : Γ\GL₂(Z) → C is a function ρ_P(γ)u(Γγ⁰) = u(Γγ⁰γ) (resp. ρ_P0(γ)u(Γ₀γ⁰) = u(Γ₀γ⁰γ) with u : Γ₀\SL₂(Z) → C).

P rim (resp. P rim₀) will denote a set of representatives of all GL₂(Z) (resp. SL₂(Z)) conjugacy classes of primitive hyperbolic elements of GL₂(Z) (resp. SL2(Z)). Here an element of a group is said to be primitive if it is not a non-trivial positive power of another element of the group.

Definition 2.2. The Selberg zeta function of Γ and Γ₀ are defined as Z_Γ(s) = Y

γ∈P rim

∞

Y

m=0

det[1 − det(γ)^mN (γ)^−s−mρ_P(γ)]

and

ZΓ0(s) = Y

γ∈P rim0

∞

Y

m=0

det[1 − N (γ)^−s−mρP0(γ)]

where R(s) > 1.

The definition clearly does not depend on the choice of the represen- tatives in P rim or P rim₀.

The convergence of the product will be shown in the proof of theo- rem 5.1.

3. Reduction theory

We need to find an appropriate set of representatives of conjugacy classes. Following [LewZag] we are interested in the following set of matrices :

Red := { a b c d



∈ GL₂(Z) : 0 ≤ a ≤ b, c ≤ d}

An element of Red is said to be a reduced matrix. The following propositions are parallel to the theory of the expansion in continued fraction of the roots ^a−d±

√

∆(γ)

2c of γx = x, which are quadratic irra- tionalities. The reader interested in such topic should refer to [Efr] and [Zag]. Here, we run the calculations directly on the matrices.

Proposition 3.1. For any γ ∈ Red there exist n₁, ...n_l ≥ 1 integers such that

γ = 0 1 1 n₁



· · · 0 1 1 n_l

 .

Moreover, the l-tuple (n1, ...nl) is unique. We call l =: l(γ) the length of γ.

Proof. Let γ =  a b c d



∈ Red. We first remark that if a = 0, we have nothing to prove, hence we can assume a 6= 0.

We prove that given γ in Red there is a unique n such that γ = 0 1

1 n



γ⁰, γ⁰ ∈ Red

and the sum of the entries of γ⁰ is less than the sum of the entries of γ, which proves the proposition by induction.

Indeed,  0 1 1 n

−1

 a b c d



=  c − an d − bn

a b



implies that 0 < d − bn ≤ b , hence n = _d

b − 1 is uniquely defined.

Straightforward computation show that 0 ≤ c − an ≤ a ≤ b and c − an ≤ d − bn and that the sum of the entries has decreased.

 Proposition 3.2. Every conjugacy class of hyperbolic matrices in GL2(Z) contains reduced representatives γ, which all have the same length l(γ).

There are l(γ)/k(γ) of them, where k(γ) is the largest k such that γ is a k-th power of an element of GL₂(Z).

Proof. We only sketch it and refer to [LewZag] for more details. Let γ = a b

c d



∈ GL₂(Z), as | det(γ)| = 1, there is a unique n ∈ Z such that n ≤ ^d_b,_a^c ≤ n + 1. We define F (γ) = 0 1

1 n

−1

γ 0 1 1 n

 with n as above.

If γ =  0 1 1 n₁



· · · 0 1 1 n_l



, then n = n₁ and

F (γ) = 0 1 1 n₂



· · · 0 1 1 n_l

  0 1 1 n₁

 .

In this case, the sequence {Fⁿ(γ)}_n≥1 is periodic of period l(γ)/k(γ).

(Indeed, if γ = γ₀^k, then γ is the k-fold concatenation of γ₀ = 0 1

1 n₁



· · · 0 1 1 n_l/k

 .)

The proposition follows from the facts (i) if det(γ) = 1, there exist some N such that F^N(γ) is reduced and (ii) if γ and γ⁰ are reduced and conjugate in GL2(Z), there exist some N such that F^N(γ) = γ⁰, which

means that the only reduced matrices γ⁰ conjugate to γ are obtained by permuting circularly the n_i.

The fact (i) is proved by a series of elementary inequalities. Let F (γ) = a⁰ b⁰

c⁰ d⁰



= d − bn c − an + n(d − bn)

b a + bn

 ,

then 0 ≤ a⁰ ≤ b⁰ is straightforward, as well as 0 ≤ c⁰ ≤ d⁰ (after maybe conjugating γ by  1 0

0 −1



we can assume b ≥ 0). One also proves c ≥ a ⇒ c⁰ ≥ a⁰, c < a ⇒ c − a < c⁰ − a⁰ and d ≥ b ⇒ d⁰ ≥ b⁰, d < b ⇒ d − b < d⁰− b⁰, so that for n big enough, F^N(γ) is reduced.

The fact (ii) is proved by the same kind of argument (cf [LewZag]).  Corollary 3.3. Every conjugacy class of hyperbolic matrices in SL₂(Z) contains reduced representatives γ, which all have the same length l(γ).

There are l(γ)/2k₁(γ) of them, where k₁(γ) is the largest k such that γ is a k-th power of an element of SL₂(Z).

Proof. We write γ = γ₀^k where k > 0 and γ₀, which belongs to SL₂(Z), is primitive in GL₂(Z), and set l0 = l(γ₀). Then we have k(γ) = k, l(γ) = kl₀, and we can write

γ₀ = 0 1 1 n₁



· · · 0 1 1 n_l0



The previous proposition tells that the GL₂(Z)-conjugates of γ corre- spond to the l₀circular permutations of the n_iin the preceding formula.

We now distinguish two cases.

If det(γ₀) = +1, then k₁(γ) = k (indeed, γ₀ is then in SL₂(Z) and primitive there). Moreover, l0 is even and the number of γ⁰ ∈ Red which are SL2(Z) conjugate to γ is only l⁰/2. Indeed, in the following formula, k has to be odd so that the determinant of the conjugating function is +1 :

 0 1 1 n_k

−1

· · · 0 1 1 n₁

−1

γ₀ 0 1 1 n₁

  0 1 1 n_k



= 0 1 1 n_k+1



· · · 0 1 1 n_l0

  0 1 1 n₁



· · · 0 1 1 n_k



If det(γ₀) = −1, then k is even and k₁(γ) = k/2 because γ₀² is prim- itive in SL₂(Z). Moreover, all the l0 circular permutations of the n_i are SL₂(Z)-conjugate to γ0because conjugating by 0 1

1 n₁



· · · 0 1 1 n_r



is the same as conjugating by 0 1 1 n_l

−1

· · · 0 1 1 n_r− 1

−1

and one of them must have determinant +1 as l₀ is odd.

In both cases, l(γ)/2k₁(γ) is the number of reduced representatives.



4. The transfer operator Ls

Let D denote the disc {z ∈ C : |z − 1| < 3/2} and BC the set of functions on D × P which are holomorphic and continuous on the boundary on each sheet D × {t}, with t ∈ P . If k · k_∞,t denotes the supremum norm of holomorphic functions continuous to the boundary on the sheet D × {t}, we set a norm on B_C by k · k∞ = P

t∈Pk · k∞,t

(P is a finite set). Then (B_C, k · k∞) is a Banach space.

Definition 4.1. For s in C, we define the operator πs,k from B_C into itself by :

π_s,kf (z, t) := 1 (z + k)^2sf

 1

z + k, 0 1 1 k

 (t)



Moreover, if γ is an element of Red, say γ =  0 1 1 n₁



· · · 0 1 1 n_l

 , then γ is said to be reduced and we define π_s(γ) := π_s,n1· · · π_s,nl.

For <(s) > 1/2, the transfer operator Ls is given by Ls :=

∞

X

k=1

πs,k

Indeed, as any f in B_Cis bounded the seriesP∞

k=1πs,kf converges as soon as P∞

k=1(x + k)^−2sconverges, in particular it converges absolutely for <(s) > 1/2. Moreover, as for each k the operator π_s,k is holomor- phic, L_sis meromorphic for <(s) > 1/2. In fact, L_scan be analytically continued to the whole complex plane, and is meromorphic there. We refer to [May1] for the proof of this analytic continuation.

Lemma 4.2. When γ is reduced, π_s(γ) is a trace-class operator, and Tr (π_s(γ)) = χ_s(γ)τ_γ,

where τ_γ = tr (ρ_P(γ)) = Card{t ∈ P : ρ_P(γ)(t) = t}.

Proof. We first restrict our attention to the case when Γ = GL₂(Z). In this case, we write π_s(γ) = Π_s(γ) and B_C = B_C. Then

Π_s(γ)f (z) = 1

(cz + d)^2sf az + b cz + d

 and we apply

Lemma 4.3. If ϕ, ψ are holomorphic functions on D, such that ψ maps D strictly into itself, and if A : B_C → B_C is the operator defined by Ag(z) = ϕg(ψ(z)), then A is a nuclear operator of order zero, hence compact and trace-class.

Its eigenvalues are λ_n = ϕ(˜z)(ψ⁰(˜z))ⁿ, n ≥ 0, where ˜z is the unique fixed point of ψ in D. Moreover, each of them is a simple eigenvalue.

The trace of A is given by : Tr (A) =

∞

X

n=0

λ_n = ϕ(˜z) 1 − ψ⁰(˜z)

We refer to [May1] for a proof of this lemma.

Here, ϕ(z) = (cz + d)^−2s, ψ(z) = ^az+b_cz+d and ψ⁰(z) = _(cz+d)^det(γ)2, so that Tr (Π_s(γ)) = 1

(c˜z + d)^2s 1 1 −_(c˜^det(γ)_z+d)2

To prove that this equals χ_s(γ), we just need to check that N (γ) = (c˜z + d)². But this can be done by direct computation :

N (γ) = a + d +p(d − a)²+ 4bc 2

!2

= (c˜z + d)²

In the case where P = Γ\GL₂Z = {t1, . . . , t_p} is non trivial, we identify f (z, t) to a Cⁿ-valued function f (z, t) =



 f₁(z)

... f_p(z)



, with f_i(z) = f (z, t_i). The representation ρ_P(γ) can be seen as a permuta- tion of {t₁, . . . , t_p}, and is identified to the induced permutation matrix which acts on Cⁿ. The action of π_s(γ) is described by :

πs(γ)f (z, t) = ρP(γ)





Π_s(γ)f₁(z) ... Π_s(γ)f_p(z)



.

But if A is a trace-class operator on a space B, M a matrix of size n and A the operator on Bⁿ defined by

A



 ϕ1

... ϕ_n



= M



 Aϕ1

... Aϕ_n



 then A is trace-class and Tr (A) = Tr (A) · tr (M ).

Hence we have

Tr (π_s(γ)) = Tr (Π_s(γ)) · tr (ρ_P(γ)) = χ_s(γ)τ_γ

 Corollary 4.4. For <(s) > 1/2, L_s : B_C→ B_Cis a nuclear operator of order zero. In particular, it is compact, trace class and has a Fredholm determinant which satisfies

det(1 − L_s) = exp −

∞

X

l=1

Tr (L^l_s) l

!

Proof. We denote by λ^k₀ the eigenvalue of biggest modulus of π_s,k. It is given by λ^k₀ = (˜z + k)^−2s, hence the series P

n≥0|λ^k₀| is converging for

<(s) > 1/2. Then, as each of the operators π_s,k are nuclear of order zero, compact and trace class P

k≥0π_s,k = L_s is also.

To have the formula for the Fredholm determinant, we just have to check P

l≥1|^{Tr (L}_l ^ls)| < ∞, which follows from the computations in the proof of Theorem 5.1.

 In fact, L_s is nuclear of order zero, compact, trace class and admits a Fredholm determinant on the whole complex plane (except at the poles of L_s), we refer once more to [May1] for a proof of this fact.

5. Selberg zeta functions and transfer operators In this section, we state and prove the main relation which links the Selberg zeta function with the transfer operator on B_C.

Theorem 5.1. We have

Z_Γ(s) = det(1 − L_s), Z_Γ0(s) = det(1 − L²_s), s ∈ C

In fact, we will prove the result for <(s) > 1. But det(1 − L_s) is meromorphic on the whole complex plane, so that the equality holds there, defining an analytic continuation of Z_Γ to the whole complex plane. Such an analytic continuation is usually proved using the Selberg trace formula.

The following proof of the theorem follows [ManMar].

Proof. We first consider the case of a subgroup Γ of GL₂(Z). We make a formal computation, and justify at the end the convergence of the products and sums involved.

− log(det(1 − L_s)) = Tr

∞

X

l=1

L^l_s l

!

= Tr





∞

X

l=1

1 l

∞

X

n=1

π_s,n

!l



= Tr X

γ∈Red

1

l(γ)πs(γ)

!

= X

γ∈Red

1

l(γ)Tr π_s(γ) where π_s(γ) = π_s,n1· · · π_s,nl(γ) with γ =  0 1

1 n1



· · · 0 1 1 n_l(γ)



(Indeed, given a fixed l,

∞

X

n=1

π_s,n

!l

= X

n1,...nl≥0

π_s,n1· · · π_s,nl.)

Moreover, we know from Lemma 4.2 that Tr (π_s(γ)) = χ_s(γ)τ_γ and from Proposition 3.2 that there are l(γ)/k(γ) elements of Red in the same conjugacy class of a hyperbolic matrix, so that

X

γ∈Red

1

l(γ)Tr π_s(γ) = X

γ∈Hyp

1

k(γ)χ_s(γ)τ_γ

where Hyp denotes a set of representatives of conjugacy classes of hy- perbolic elements of GL₂(Z). Any γ ∈ Hyp can be written uniquely as

˜

γ^k with ˜γ primitive and k ≥ 0.

(Indeed, maybe after conjugaison, we can assume γ ∈ Red, that is γ =  0 1

1 n₁



· · · 0 1 1 n_l



. But in this case, being a kth power means being a k-fold concatenation of 0 1

1 n₁



· · · 0 1 1 n_l/k



) Hence we get

X

γ∈Hyp

1

k(γ)χs(γ)τγ = X

γ∈P rim

∞

X

k=1

1

kχs(γ^k)τ_γ^k

= X

γ∈P rim

∞

X

k=1

1 k

N (γ^k)^−s

1 − det(γ^k)N (γ^k)⁻¹τ_γ^k

= X

γ∈P rim

∞

X

k=1

1 k

N (γ)^−ks

1 − det(γ)^kN (γ)^−kτ_γ^k

as N (γ^k) = N (γ)^k (this is straightforward for γ diagonalizable, and hence always true by continuity).

We have proved that

− log(det(1 − L_s)) = X

γ∈P rim

∞

X

k=1

1 k

N (γ)^−ks

1 − det(γ)^kN (γ)^−kτ_γ^k. On the other hand, we have

− log(Z_Γ(s)) = X

γ∈P rim

∞

X

m=0

− log det(1 − det(γ)^mN (γ)^−s−mρ_P(γ))

= X

γ∈P rim

∞

X

m=0

∞

X

k=1

1

k det(γ)^mkN (γ)^−(s+m)ktr (ρ_P(γ)^k)

= X

γ∈P rim

∞

X

m=0

∞

X

k=1

1

k det(γ)^mkN (γ)^−(s+m)kτ_γ^k

= X

γ∈P rim

∞

X

k=1

1

kτ_γ^kN (γ)^−sk

∞

X

m=0

det(γ)^mkN (γ)^−mk

= X

γ∈P rim

∞

X

k=1

1 k

N (γ)^−ksτ_γ^k 1 − det(γ)^kN (γ)^−k

= − log det(1 − Ls)

We have proved the result formally, and just need to check the con- vergence for <(s) > 1, which will prove the result for all s by analytic continuation. We consider the identity

− log(Z_Γ(s)) = X

γ∈Red

χ_s(γ) l(γ) τ_γ.

We will show that

M_s := X

γ∈Red

|χ_s(γ)|

converges for R(s) > 1. (τ_γ is the trace of a permutation matrix, hence bounded by the index of Γ).

This will imply the convergence of the product defining Z_Γ(s). In- deed, all the terms are replaced by their absolute value when s is re- placed by its real part, which justifies the interchanges of the orders of summation in the preceding computation.

We have χ_s(γ) = ^{N (γ)}_∆(γ)^1/2−s_1/2 (this is verified by straightforward com- putation for γ diagonal, and any hyperbolic matrix is conjugated to a diagonal matrix). Then

M_s = X

γ∈Red,det(γ)=−1

|N (γ)^1/2−s|

∆(γ)^1/2 + X

γ∈Red,det(γ)=+1

|N (γ)^1/2−s|

∆(γ)^1/2

=

∞

X

k=1

C_k⁻

√k²+ 4

k +√ k²+ 4 2

!1−2<(s)

+

∞

X

k=3

C_k⁺

√k²− 4

 k +√ k²− 4 2

^1−2<(s)

where C_k^±= CardE_k^± with E_k^± = {γ ∈ Red : Tr (γ) = k, det(γ) = ±1}.

In both sums, the asymptotic term is k^−2<(s)C_k^±, where <(s) > 1.

So that all we need to prove is C_k^± = O(k^1+ε) for any ε > 0.

We set A_N := Card{(k, l) ∈ N² : kl = N } and E_k,u^± := a b c d



∈ E_k^± : a = u

 . It is well known that A_N = O(N^ε). And we have

Card(E_k,a^± ) = Card({γ ∈ E_k^±: bc = a(k − a) ± 1} ≤ A_a(k−a)±1.

Moreover, a(k − a) ± 1 ≤ k²/2, and E_k = ∪^k/2_a=0E_k,a, so that C_k^± ≤

k

2O((^k₂²)^ε) = O(k^1+2ε) and the asymptotic term is k1−2<(s)+2ε. The series converges provided ε is small enough, because <(s) > 1. The convergence is checked.

In the case of a discrete subgroup Γ₀ of SL₂(Z), we have a similar computation.

− log det(1 − L²_s) = Tr





∞

X

l=1

1 l

∞

X

n=1

π_s,n

!2l



= Tr





X

γ∈Red,l(γ)even

2

l(γ)π_s(γ)





= X

γ∈Red∩SL2Z

2

l(γ)χ_s(γ)τ_γ

= X

γ∈Hyp0

1

k₁(γ)χ_s(γ)τ_γ And on the other side

− log(Z_Γ0(s)) = X

γ∈P rim0

∞

X

m=0

∞

X

k=1

1

kN (γ)^−(s+m)ktr (ρ_P0(γ)^k)

= X

γ∈P rim0

∞

X

k=1

1

kN (γ)^−skτ_γ^k

∞

X

m=0

N (γ)^−mk

= X

γ∈P rim0

∞

X

k=1

1

kχ_s(γ^k)τ_γk

= X

γ∈Hyp0

1

k₁(γ)χ_s(γ)τ_γ

= − log det(1 − L²_s).

The proof of the convergence being similar to the preceding one, the proof is complete.

 6. A dynamical point of view on ZΓ0

In this section, we give another definition of Z_Γ0, in terms of closed geodesics of the surface Γ₀\H. The reader interested in this dynamical point of view should refer to [Juh].

Let H = {z ∈ C : =z > 0} denote the Poincar´e half plane, that is the upper half plane equipped with the metric ds² = (dx² + dy²)/y². The group P SL₂(R) acts on H by

 a b c d



z = az + b cz + d

Given a discrete subgroup Γ of P SL₂(R), we denote by X(= XΓ) the quotient space of H under the action of Γ, that is X = Γ\H. As the

unit tangent bundle SH of H can be identified with P SL2(R), we get the following identification :

SX ' Γ\P SL₂(R)

We denote by Φt : SX 7→ SX the geodesic flow. It can be described in algebraic terms as Φ_t(Γg) = Γga⁻¹_t where a_t = e^t/2 0

0 e^−t/2

 , t ∈ R.

Let us denote by C the set of all closed oriented geodesics of X, which can also be seen as closed orbit of Φ_t on SX. If c is in C, we denote its length by |c|. That is |c| is the smallest t > 0 such that Φt(x) = x for any x in the corresponding orbit on SX.

Generally, given a flow ϕ_t : M → M on a manifold M , and a point x which is T -periodic, we define the linear Poincar´e map by P_x :=

d_x(ϕ_T) : T_xM → T_xM where T_xM is the tangent space of M at the point x.

It turns out that the geodesic flow Φ_t on such a surface has the Anosov property : T M has a Φ_t-invariant decomposition T M ' T⁰ ⊕ T^s ⊕ T^u such that dΦ_t acts trivially on T⁰, by expansion on T^u for t > 0 and by contraction on T^s for t > 0. Under this decomposition T_xM ' T_x⁰⊕ T_x^s⊕ T_x^u, P_x can be written as P_x = Id ⊕ P_x^s⊕ P_x^u. Lemma 6.1. If x and x⁰ belong to the same periodic orbit of Φ_t, then P_x and P_x⁰ are conjugate.

Proof. Let’s denote by at the right multiplication by a⁻¹_t (= Γa⁻¹_t ) in SX. Then Px = Dx(a|c|) : Tx(SX) → Tx(SX) and Px⁰ = Dx⁰(a|c|) : T_x⁰(SX) → T_x⁰(SX). Moreover, as x and x⁰ belong to the same orbit, there exists t₀ such that x⁰ = Φ_t0(x) = a_t0(x) and D_x(a_t0) : T_x(SX) → T_x⁰(SX). Moreover, we clearly have a_t0a_|c|a⁻¹_t0 = a_|c|, so that differenti- ating and using the chain rule, we get

D_x(a_t0)P_xD_x(a_t0)⁻¹ = P_x⁰

which is the desired conjugation. 

We are now able to rewrite the Selberg zeta function for Γ₀ = SL₂(Z) from a purely dynamical point of view. The case of subgroups of finite index is treated bellow.

Theorem 6.2. We have Z_SL2Z(s) =Y

c∈C

Y

m≥0

det(1 − (P_c^s)^me^−s|c|), <(s) > 1

The notation P_c^s just insists on the fact that det(1 − (P_x^s)^me^−s|c|) does not depend on the point x of c.

Lemma 6.3. There is a bijection between the set C of closed oriented geodesics of Γ\H and the set of all conjugacy classes {g}Γ of primitive hyperbolic elements of Γ.

Moreover, if c is the closed geodesic associated to the conjugacy class {g}_Γ of Γ, then e^|c|= N (g).

Proof. Given a closed orbit c of Γ\P SL2(R), we associate to it a con- jugacy class of Γ as follows. For x ∈ c we have Φ_T(x) = x for some T and we take T > 0 minimal with this property. If we write x = Γg, g ∈ P SL₂R, we get Γga⁻¹T = Γg, that is ga⁻¹_T g⁻¹ = g₀ ∈ Γ. We define f (c) := {g₀}_Γ. This is well defined (indeed, if x⁰ = Γg⁰ is in c, there exists t such that g⁰ = ga_t, and so g⁰a⁻¹_T g⁰⁻¹ = ga_ta⁻¹_T a⁻¹_t g⁻¹ = g₀).

Moreover, g₀ is conjugate to a⁻¹_T which is hyperbolic, hence is hyper- bolic itself. It is also primitive.

To prove primitivity, we first remark that given {g₀}_Γ a non triv- ial hyperbolic conjugacy class, there exists g ∈ P SL2(R) such that g₀ = ga⁻¹_T g⁻¹ for some a_T. (It is sufficient to have g in SL₂(R) which conjugates g0 to a diagonal matrix, which is clear since the character- istic polynomial det(XId − g0) = X²− Tr (g0) + det(g0) has a positive determinant ∆(g₀) = Tr (g₀)² − 4 det(g₀). The diagonal matrix is of the form a_T as det(g₀) = 1.)

Then, if we assume g₀ = h^k, with h ∈ Γ , we have g and u in SL₂(R) such that g0 = ga⁻¹_T g⁻¹ and h = ua⁻¹_t u⁻¹, which implies ga⁻¹_T g⁻¹ = g₀ = h^k = ua⁻¹_t u⁻¹ua⁻¹_t u⁻¹...ua⁻¹_t u⁻¹ = ua⁻¹_ktu⁻¹ hence a_T = (g⁻¹u)⁻¹a_ktg⁻¹u, so g⁻¹u = Id and T = kt. So h = ga⁻¹_t g⁻¹ is in Γ so that Γga⁻¹_t = Γg, that is Φ_t(x) = x with t < T which is a contradiction. So g₀ is primitive.

f is surjective. Indeed, if we write g₀ = ga⁻¹_T g⁻¹ then as g₀ ∈ Γ, we have Γga⁻¹_T = Γg, or Φ_T(Γg) = Γg, hence a periodic orbit of the geodesic flow.

f is injective. Indeed, if g0 and g1, corresponding to c and c⁰ re- spectively define the same conjugacy class, we have γ in Γ such that g₀ = γg₁γ⁻¹. But g₀ = ga⁻¹_T g⁻¹, hence Γg is in c. On the other hand, g₁ = γ⁻¹ga⁻¹_T g⁻¹γ hence Γγ⁻¹g = Γg belongs to c⁰. Hence c and c⁰ are the same geodesic.

f defines a bijection between the set of closed geodesics and the set of conjugacy class of primitive hyperbolic elements.

Now we show e^|c| = N (g) if f (c) = {g}Γ. But up to conjugacy, we can assume g = at for some t > 0. Then a lift of c to H is (the positive part of) the imaginary axis of C. And as at(z) = e^tz, the length of the closed geodesic c is the length of the geodesic in H which brings i to e^ti, that is |c| =Re^ti

i dy

y = ln[e^t] = t, and so e^|c|= e^t= N (a_t) = N (g).  Lemma 6.4. Let Γ be a discrete subgroup of P SL₂(R) and X = Γ\H.

If c is a periodic orbit of the geodesic flow Φ_ton SX with period T = |c|, then P_c^s = e^−|c|Id and P_c^u = e^|c|Id.

Here again, we use the notations P_c^s and P_c^u to emphasize that the result does not depend on which point x of c we choose.

Proof. We take an arbitrary point x = Γg₀ in c. We have Γg₀a⁻¹_T = a_T(Γg₀) = Γg₀ so that g₀a⁻¹_T g₀⁻¹ = γ is in Γ. Let α be defined by the commutative diagram

Γg −−−→^aT Γga⁻¹_T



 y^g

−1 0



 y^g

−1 0

Γgg⁻¹₀ −−−→ Γga^{α −1}_T g₀⁻¹

Then α(Γe) = Γg₀a⁻¹_T g⁻¹₀ = Γγ = Γe (where e represents the identity matrix) and

D_Γe(α) : T_Γe(Γ\P SL₂R) → TΓe(Γ\P SL₂R)

But T_Γe(Γ\P SL₂(R)) can be identified with sl2R, the Lie algebra of P SL₂(R) as follows. To each X ∈ sl2R we associate the fundamental vector field eX generated by X :

Xe_Γg = d dt

  0

(Γg exp(tX))

and then X 7→ eX_Γe realizes the isomorphism sl₂R ' TΓe(Γ\P SL₂R).

We show that D_Γe(α) acts as the adjoint action Ad(γ). We compute

DΓe(α)(X) = d dt

0α(Γ exp(tX))

= d

dt   0

Γ exp(tX)g₀a⁻¹_T g⁻¹₀

= d

dt   0

Γγ⁻¹exp(tX)γ But g⁻¹exp(tX)g = exp(tAd(g)X), so that

D_Γe(α)(X) = d dt

0Γ exp(tAd(γ)X) =(Ad(γ)X)^ _Γe = Ad(γ)X But Ad(γ) = Ad(g0a⁻¹_T g₀⁻¹) = Ad(g0)Ad(a⁻¹_T )Ad(g₀⁻¹), and

Ad(g0)⁻¹DΓe(α)Ad(g0) = Ad(a⁻¹_T ) : sl2R → sl²R

and all we need to do is find the spectral decomposition of Ad(a⁻¹_T ).

Therefore, we consider the adjoint action of elements of the Lie al- gebra, defined as : ad(X) = _dt^d

0Ad(exp(tX)), and which given by ad(X) = [X, ·]. For X₀ = 1 0

0 −1



, we have

ad(X₀) a b c d



=



X₀, a b c d



=

 0 2b

−2c 0



so that the eigenvalues are {−2, 0, 2}, and we have a decomposition sl₂R = N⁻⊕ N ⊕ N⁺, with respect to which ad(X₀) = −2Id ⊕ 0 ⊕ 2Id.

We come back to Ad(aT) via the formula Ad(exp X) = exp(ad(X)),

which implies Ad(a_T) = Ad(exp(tX₀/2)) = exp((t/2)ad(X₀)), so that with respect to the decomposition above, Ad(a_T) = e^−TId ⊕ Id ⊕ e^TId, and so P_x^s = e^−TId = e^−|c|Id and P_x^u = e^|c|Id.

 The two lemmas prove Theorem 6.2, once we remark that the as- sumption tr (γ) > 0 for γ hyperbolic in SL₂(Z) implies that only one of the two lifts of an hyperbolic element of P SL₂(Z) is counted in Z_Γ(s).

For subgroups Γ₀ of finite index p of SL₂(Z), Γ0\H is a p-covering of P SL₂(Z)\H and so the unit tangent bundle Γ0\P SL₂(R) is also a p-covering of P SL₂(Z)\P SL2(R). The canonical projection is denoted by π : Γ₀g 7→ Γg. The geodesic flow is described as the right action of A = { e^t/2 0

0 e^−t/2



, t ∈ R} on the sphere bundle of both surfaces and the following diagram is commutative

Γ₀\P SL₂(R) × A −−−→ Γ₀\P SL₂(R)



 y^π



 y^π

P SL₂(Z)\P SL2(R) × A −−−→ P SL2(Z)\P SL2(R)

Given a closed orbit c of the geodesic flow on P SL₂(Z)\P SL2(R) its lift to Γ₀\P SL₂(R) is a family of closed orbits of the geodesic flow. If x is a point in c, there is T > 0 minimal such that a_T(x) = xa⁻¹_T = x.

Let π⁻¹(x) = {x₁, . . . , x_p}, then for all i there is a unique j such that xia⁻¹_T = xj so that aT and x define a permutation of {1, . . . , p}, denoted by ρx.

Lemma 6.5. If x and x⁰ belong to the same orbit c of the geodesic flow on P SL₂(Z)\P SL2(R), then the permutations ρx and ρ_x⁰ of {1, . . . , p}

are conjugate.

Proof. Indeed, there exists some t such that xa⁻¹_t = x⁰. This im- plies that for each j, there is a unique i_j such that x_ja⁻¹_t = x⁰_ij. As a⁻¹_t a⁻¹_T a_t = a⁻¹_T , j 7→ i_j is a conjugating permutation between ρ_x and

ρ_x⁰. 

Let ρ_x also denote the corresponding permutation matrix of size p.

Theorem 6.6.

Z_Γ0(s) =Y

c∈C

Y

m≥0

det(1 − (P_c^s)^me^−s|c|ρ_c), <(s) > 1

where C denotes the set of closed oriented geodesics of P SL₂(Z)\H.

We note ρ_c to emphasize that det(1 − (P_c^s)^me^−s|c|ρ_x) does not depend on x ∈ c.

Proof. Let x belong to c, a_T as above. We only have to identify ρ_x with ρ_P0(a_T). Let π⁻¹(x) = {x₁, . . . , x_p} and write x_i = Γ₀g_i with g_i ∈ P SL₂(R). Both x1 and x_i project on x, so that Γg₁ = Γg_i where Γ denotes P SL₂(Z). This implies γi := g_ig⁻¹₁ belongs to Γ = P SL₂(Z) and x_i = Γ₀γ_ig₁. Then the right multiplication x_i = Γ₀g_i 7→ Γ₀g_ig₁ = Γ₀γ_i ∈ Γ₀\Γ is a bijection between π⁻¹(x) and Γ₀\Γ. It gives the desired identification. Indeed

Γ₀g_j = x_j = ρ_x(x_i) = x_ia⁻¹_T = Γ₀g_ia⁻¹_T and

ρP0(aT)(Γ0γi) = ρP0(g₁⁻¹aTg1)(Γ0γi) = Γ0γi(g₁⁻¹aTg1)⁻¹

= Γ₀g_ig₁g⁻¹₁ a⁻¹_T g₁ = Γ₀g_ia⁻¹_T g₁ = Γ₀g_jg₁ = Γ₀γ_j

 References

[Bea ] Alan F. Beardon, The Geometry of Discrete Groups, Springer-Verlag New York 1983.

[ChMay ] C. H. Chang, D. Mayer, Thermodynamic formalism and Selberg’s zeta function for modular groups. Regular and Chaotic Dynamics, vol. 5, No 3 (2000) 281-312.

[Efr ] I. Efrat, Dynamics of the continued fraction map and the spectral theory of SL₂Z. Inventiones math. 114 (1993) 207-218.

[Juh ] A. Juhl, Cohomological Theory of Dynamical Zeta Functions. Progress in Math (vol 194) 2001.

[Kna ] A. Knapp, Representation Theory of Semisimple Groups, Princeton Univer- sity Press 1986.

[LewZag ] J. B. Lewis, D. Zagier, Period Functions and the selberg zeta function for the modular group. In : The Mathematical Beauty of Physics, 83-97, Adv.

Series in Math. Physics 24, World Sci. Publ., River Edge, NJ, 1997.

[ManMar ] Yuri I. Manin, Matilde Marcolli, Continued fractions, modular symbols, and non-commutative geometry. Selecta Mathematica New Ser. (SM), Vol. 8 N.3 (2002) 475-520.

[May1 ] D. Mayer, Continued fractions and related transformations. In : Ergodic Theory, Symbolic Dynamics and Hyperbolic Spaces. T. Bedford, M. Keane, C.

Series (Eds). Oxford University Press 1991.

[May2 ] D. Mayer, The Thermodynamic formalism Approach to Selberg’s zeta func- tion for P SL₂Z. Bull. AMS 25 (1991) 55-60.

[Rat ] John G. Ratcliffe, Foundations of Hyperbolic Manifolds, Springer-Verlag 1994.

[Sel ] A. Selberg, Harmonic analysis and discontinuous groups in weakly symmetric Riemannian spaces with applications to Dirichlet series. J. Indian Math. Soc.

20 (1956) 47-87.

[Zag ] D. Zagier, Zetafunktionen und Quadratische K¨orper, Springer-Verlag Berlin 1981.

(J. Brieussel) 59 Rue S. Gryphe, 69007 Lyon, France E-mail address: jeremie.brieussel@ens-lyon.fr