# Cutting out arithmetic Teichmüller curves in genus two via Theta functions

###### Abstract.

We compute the class of arithmetic genus two Teichmüller curves in the Picard group of pseudo-Hilbert modular surfaces, distinguished according to their torsion order and spin invariant. As an application, we compute the number of genus two square-tiled surfaces with these invariants.

The main technical tool is the computation of divisor classes of Hilbert Jacobi forms on the universal abelian surface over the pseudo-Hilbert modular surface.

###### Contents

## 1. Introduction

The aim of this paper is to contribute to the classification of arithmetic Teichmüller curves and the computation of their basic invariants. The extension of the bundle of Jacobi forms to the universal family of abelian surfaces over pseudo-Hilbert modular surfaces and the computation of its class will be our main technical tool.

#### Arithmetic Teichmüller curves.

Square-tiled surfaces are covers of the square torus, ramified over at most one point. Affinely deforming the squares into parallelograms yields a curve in the moduli space of curves, called arithmetic Teichmüller curve. Non-arithmetic Teichmüller curves, which are generated by flat surfaces that do not arise via branched coverings of the torus, have been classified in genus two ([McM05], [McM06]), and in higher genus there is a growing number of partial results. For Teichmüller curves generated by square-tiled surfaces, the classification problem is solved only for genus two surfaces with a single ramification point ([HL06] for prime degree coverings and [McM05] in general). They are classified by two invariants, the number of squares and the spin.

#### Genus two, two ramification points.

Genus two square-tiled covers with two ramification points come with three obvious invariants. One is the spin invariant, the number of integral Weierstraß points. The other two are the torsion order of the two branch points in a minimal intermediate torus covering and the degree of this covering (see Section 2). It is conjectured (and well-supported by computer experiments of Delecroix and Lelièvre) that these are the only invariants, i.e. that the set of genus two degree covers of the torus with torsion order and spin is irreducible. For one ramification point, both [HL06] and [McM05] solved the irreducibility question combinatorially by exhibiting prototypes for the flat surfaces and connecting any two of the same invariants by a change of direction. This approach might work for two ramification points as well, but the combinatorial complexity is challenging.

This paper does not contain any picture of a flat surface. Instead we propose to tackle the classification problem by first computing the class of in the (rational) Picard group of a pseudo-Hilbert modular surface and in the second step to argue that this class is not too divisible and that potential summands cannot be Teichmüller curves.

#### Counting square-tiled surfaces.

In this paper, we complete the first step in this program for odd . As a result, we can solve the following counting problem. For this has been conjectured by Zmiaikou ([Zmi11, p. 67]).

###### Theorem 1.1.

The number of reduced square-tiled
surfaces of genus two, two ramification points, odd degree , torsion order
and spin invariant is given as follows.

If is odd, then

If is even, then there is no spin invariant and

If , then

###### Remark 1.2.

In principle, the same program can be carried out for even , but it requires performing similar computations as we present them for covering surfaces with an extra level of two (see Section 9.4). The conjectural values for the counting problem are as follows. For and even we have

and for and is even the values are

#### Classes in the Picard group

The above counting result is a consequence of the following statement that gives the class of the (union of) Teichmüller curves generated by the square-tiled surfaces of degree , with torsion order and spin on the compactified pseudo-Hilbert modular surface , whose open part parametrizes abelian surfaces with multiplication by a pseudo-quadratic order. See Section 3 for the definition of and the Hodge bundles .

###### Theorem 1.3.

Let be odd. The class of in is given as follows. If is odd, then

(1) | ||||

If is even then

(2) |

If , then

(3) | ||||

###### Remark 1.4.

The conjectural classes for the case even are given as follows. If is odd, then

(4) | ||||

If is even then

(5) |

If , then

(6) | ||||

#### Strategy of the proof.

Instead of locating a Teichmüller curve inside , we locate the branch points of the covering map from the flat surface to the torus inside the universal family of abelian surfaces over the open subset . The points that we want to single out lie on image of the flat surface in its Jacobian (i.e. on the theta divisor), they are branch points (i.e. the derivative of the theta function vanishes in some direction), and they have the property that their image in a certain intermediate elliptic curve is -torsion. Theorem 8.3 expresses that the image of this intersection of three divisorial conditions in is the Teichmüller curve. The basic idea to use theta functions builds on that in [Möl14], but there one could work entirely in the two-dimensional base, while most of the difficulties here come from performing the triple intersection in the four-dimensional total space. Of course, for intersection theory calculations, we need to work on a reasonable (normal, at most quotient singularities) compactification of . We recall the background on toroidal compactifications and construct in Section 5. The family comes with some obvious divisors (boundary components, Hodge bundle, zero sections), whose intersection product is readily computed. The goal is hence to express the ingredients of the triple intersection in these terms.

#### Jacobi forms for pseudo-Hilbert modular surfaces.

Hilbert Jacobi forms are functions on the universal covering of whose transformation law combines the elliptic behavior on and the modular behavior on in the usual way as for elliptic Jacobi forms. The precise definitions are given in Section 6.3. The basic example of a Jacobi form is the theta function, both in the elliptic and in the pseudo-Hilbert modular case. We would like to express the divisor class of a Jacobi form on in terms of the natural divisors mentioned above. We stress that, however, this question is not even well-defined. Only after making some artificial choice at the boundary (our choice is (30) in Section 6.3) we can determine the class of a Jacobi form in Theorem 6.1.

At the end of the day, we are only interested in the class of a divisor (the Teichmüller curve) generically lying in . Consequently, we have to determine and subtract in Section 9.2 the spurious boundary components, thereby compensating the arbitrariness in the boundary extension of Jacobi forms.

Finally, in the case of , the analogous statement of Theorem 8.3 is Theorem 8.4 and there two other spurious summands occur. One contribution is from the reducible locus in , whose class we determine in Section 7. The other contribution stems from square-tiled surfaces with only one branch point. The classes of the corresponding Teichmüller curves have been determined in [Bai07].

## 2. Origamis, Square-tiled surfaces and their spin structure

Let be the moduli space of flat surfaces and for any partition of , let be the stratum, where the divisor of has type . In this paper will always be an arithmetic Veech surface of genus . This is equivalent to requiring the existence of an origami map, a covering to an elliptic curve such that is branched over only one point and . The map is unique only up to isogeny and translation on (the latter can be dispensed with by translating the unique branch point to the origin). We call reduced, if it does not factor over an origami map that has strictly smaller degree. Equivalently, is reduced, if and only if the lattice of generated by relative periods

is equal to .

If is the particular elliptic curve with , then is called square-tiled surface. In this case, .

A covering to an elliptic curve is called minimal or optimal, if it does not factor over an isogeny of degree . A covering is minimal, iff the induced map on the first absolute homology is surjective.

Let denote the set of -torsion points, where , and let denote the hyperelliptic involution. Let denote the Weierstraß divisor on . From now on we restrict to the case of genus two surfaces.

###### Proposition 2.1.

For any arithmetic Veech surface of genus , there is a reduced origami map and a decomposition into a minimal covering of degree and an isogeny of degree .

The map , and a fortiori , is uniquely determined by the requirement that

We call the origami map with a factorization and location of branch points as in this proposition normalized.

###### Proof.

By [Kan03, Proposition 2.2], there is a uniquely determined minimal, normalized covering . Moreover, this covering satisfies

and since the ramification points of are not fixed by , their images , satisfy . Let be an isogeny with , or equivalently . Such an isogeny exists since is a Veech surface, and hence is of finite order. The minimal such is given by the quotient map , where is the subgroup generated by . ∎

It is possible that . In this case, the branching divisor is non-reduced, i. e. . The integers and are uniquely determined by the Veech surface. We call the degree and the torsion order of .

### 2.1. Spin structure

Let be an arithmetic Veech surface with reduced, normalized covering . A Weierstraß point is called integral, if is equal to the branch point of . The number of integral Weierstraß points is an invariant of the -orbit of , called the spin invariant . Depending on the parity of and , we determine when it distinguishes orbits.

Let factorize as with a minimal, normalized covering and an isogeny of degree . Let denote one of the branch points of . Then . If , then the induced map on the -torsion points is an isomorphism. Thus, if

and if , then

If on the other hand, , then is a primitive -torsion point, and the fiber of does not contain a -torsion point, since the equation has no solution . Thus in this case

Note that the preceding discussion applies both to arithmetic Veech surfaces in and to arithmetic Veech surfaces in . In the second case of course.

Next we consider the case that is a reducible genus two surface but with compact Jacobian, i.e. is the union of two elliptic curves joined at a node . In this case an origami map is simply defined to be a map that is non-constant on both factors, or equivalently is non-zero on both components. This implies that and (and ) are isogenous. If then obviously . We call Weierstraß divisor on the set of fixed points different from of the elliptic involutions on and with respect to the zero . Obviously as in the smooth case. This notion is justified since one easily checks that for any family of flat surfaces degenerating to , the Weierstraß divisor converges to . Again we let be the number of integral Weierstraß points, i.e. the number of points in with image equal to .

There are no integral Weierstraß points on a component iff is odd. If is even, there is three or one Weierstraß point, depending on whether factorizes through multiplication by two or not. The latter can happen only if is divisible by four. For consequently

since precisely one of the is odd. If is even, then both might be odd, resulting in no integral Weierstraß points. If both are even and one of the maps factors through multiplication by two, then factors through a two-isogeny. Consequently, if is a reduced origami map and , then

where corresponds to both odd.

## 3. Pseudo-Hilbert modular surfaces

In this section we introduce the surfaces containing the Teichmüller curves we are interested in. These are moduli spaces for Abelian surfaces with multiplication by pseudo-quadratic orders that we call pseudo-Hilbert modular surfaces . They admit a finite cover, which is a product of two modular curves. Consequently, many line bundles on arise from line bundles on the modular curves and we summarize the main properties. Next, we introduce the Teichmüller curves on and fix notation for all the divisors on we need. See also [Bai07], [Her91] or [McM07] for basic properties of pseudo-Hilbert modular surfaces.

### 3.1. Modular curves and modular forms

We let be the principal congruence group of level and be the (open) modular curve. Its smooth compactification is denoted by . If , the curve has cusps and genus .

We record that is a covering of degree

if we consider these curves as quotient stacks. (In terms of coarse moduli spaces, if we let denote the image of in , the covering is of degree , which is half the degree above for .)

The Hodge bundle on is , where is the (compactified) universal family (see Section 5). We also write if we want to emphasized the level. Global sections of are modular forms of weight for . Moreover, , where is the divisor of cusps and is the canonical bundle.

The discriminant is a modular form of weight for . It is non-zero on and vanishes to the order at each cusp () of . Thus

(7) |

The principal congruence group of level is conjugate to another congruence group

Consequently, the action of and on are equivariant with respect to the multiplication map by on and there is an isomorphism

The two-fold product of the groups appears naturally as subgroup of pseudo-Hilbert modular groups, as we will see next.

### 3.2. Pseudo-Hilbert Modular surfaces

Let and . Following the conventions for Hilbert modular surfaces, we let , whose subring

will be called a pseudo-quadratic order of discriminant . Let be the inverse different. The pseudo-Hilbert modular group is

and pseudo-Hilbert modular surface is the quotient^{1}^{1}1topologically, but not as a quotient stack, see Section 3.3

It is the moduli space parameterizing abelian surfaces with multiplication by the pseudo-quadratic order of discriminant as we will see in Section 4. Since

both inclusions being of degree , the pseudo-Hilbert Modular surface admits a useful covering given by

and a quotient given by

The factor group , and thus a fortiori , acts on the smooth compactification of and the quotient maps and extend to quotient maps

We will work with this normal (but not smooth) compactification of . In fact is the Baily-Borel compactification of . We now list the divisors on that will be important in the sequel.

#### Boundary divisors

The image of is
a curve and the image of
is a curve . Their closures are denoted by .
The curves are irreducible and isomorphic to
([Bai07, Proposition 2.4]).^{2}^{2}2There
are different indexing conventions for the boundary divisors in
[Bai07] and in [Her91].
As mnemonic for our convention, keep in mind that and
are pulled back via .

#### The Hodge bundles.

We let be the pullback of the Hodge bundle to the product. The next important divisor classes on are the Hodge bundles

By definition .

In the same way, we define as the pullback of the boundary divisors to . They consist of irreducible components , .

Pulling back (7) to the product and then taking its -pushforward we obtain the important relation

(8) |

in .

#### The product locus.

We denote by the product locus, the locus of abelian surfaces that split as a polarized surface. We will determine the class of this locus in Section 7. The complement consists of principally polarized abelian surfaces that are Jacobians of genus two curves.

#### The Teichmüller curves.

The projection of an -orbit of a square-tiled surface is a Teichmüller curve in . If is a minimal torus covering of degree , then the kernel of is a connected abelian subvariety of exponent (cf. (11)) by [BL04, Lemma 12.3.1, Corollary 12.1.5 and Proposition 12.1.9]. Consequently, by Proposition 4.1 below, a square-tiled surface that factorizes through such a map defines a point in and the corresponding Teichmüller curve is a curve in .

We let () be the union of Teichmüller curves generated by reduced square-tiled surfaces of degree where has a double zero. By the results in the preceding section, decomposes into spin components . The topology of is completely determined by the work of [McM05], [Bai07], and [Muk14]. In particular the spin components are irreducible.

We let be the union of Teichmüller curves generated by reduced square-tiled surfaces of degree such that has two simple zeros and has torsion order . By the preceding section, decomposes into its spin components .

### 3.3. On quotient stacks

Since we suppose throughout, the stack discussion on in the beginning of this section was inessential. The group however contains for all an element of finite order that acts trivially on , namely embedded diagonally. We want the main object of our studies, the pseudo-Hilbert modular surface to be a variety, rather than a stack with global non-trival isotropy group of order two. For this purpose we consider as the quotient stack . As a set, , as introduced above, but the morphism is of degree throughout this paper. In particular, it is also possible to define the Hodge bundles ’from above’ without invoking the orbifold bundles on by the relation . The equation (8) holds with this convention (and with the reduced scheme structure on ).

The reason for this discussion is that the diagonally embedded does no longer act trivially when considering the universal family, see (13) in the next section. So there is no choice but to let the universal family and its compactification be really the quotient stack by the group . In particular, the map is of degree . This has the irritating consequence that the map of the universal family is the composition of the forgetful map composed with a (pointwise identity) map of degree . This factor has to be taken into account in push-forwards, see Section 8.

## 4. Abelian surfaces with multiplication by pseudo-quadratic orders and modular embeddings

Here, we sketch how parametrizes abelian surfaces with multiplication by and describe the universal family

(9) |

where

(10) |

One should be aware that is the universal family only when considered as a quotient stack. The fibers of the underlying variety are Kummer surfaces, and in particular singular. Nevertheless, the open family and its compactification, introduced in Section 5, are both quotients of smooth varieties by finite groups and thus smooth when considered as stacks.

It will be convenient to compare this family to the universal family of all principally polarized abelian surfaces via a map that is equivariant with respect to a group inclusion . Such a pair is sometimes called modular embedding and it will be used in the next section to pull back theta functions.

Recall that the exponent of an abelian subvariety of dimension in a principally polarized abelian variety is defined as

(11) |

see [BL04, Section 1.2 and 12.1].

###### Proposition 4.1.

The pseudo-Hilbert modular surface surface is the moduli space of all pairs , where is a principally polarized abelian surface and is a choice of multiplication by .

Equivalently, is the moduli space of all pairs consisting of a principally polarized abelian surface together with a projection to an elliptic curve such that is a connected abelian subvariety of exponent .

For the convenience of the reader and to fix notations, we provide a sketch of the proof the first statement, following [Bai07, Theorem 2.2]. The second statement follows from [BL04, Proposition 12.1.1 and Proposition 12.1.9] after unwinding the definitions.

We want to provide with a polarization. For this purpose we define the ’Galois conjugation’ on by . With the usual definition of trace the pairing

on is unimodular, alternating and -valued, hence a polarization. Moreover, we let . Then, a symplectic basis of is

where is an arbitrary basis of . For , define the embedding

The image is a lattice in spanned by the columns of

where and where and . We will work throughout with the choice

The quotient is a principally polarized abelian surface (ppas), polarized by the hermitian form with matrix and the columns of are a symplectic basis for the pairing with matrix . The associated point in is , with the convention that corresponds to the ppas with lattice spanned by the columns of . It admits multiplication by via the diagonal action on the embedding . This justifies the claims made in Section 3.2.

Since both eigenspaces of multiplication by are defined over , the abelian surface is isogenous to a product of elliptic curves with an isogeny of degree . We give an explicit basis of the sublattice corresponding to the product decomposition. It is generated by the columns of

For an -basis of , define the elliptic curve . Then the isogeny between abelian varieties

is induced by the identity on the universal cover. The coordinate projections , induce the dual isogeny

which after composition with the isomorphism covered by , becomes multiplication by on .

This completes the sketch of the proof of Proposition 4.1.

#### Modular embeddings

The universal family is now easily obtained by pullback of the universal family of principally polarized abelian surfaces over via a modular embedding.

###### Lemma 4.2.

The embedding

is equivariant with respect to

where , and .

Note that the induced map does not depend on the choice of the matrix . If is another basis, , and is the embedding associated with , then

The proof of Lemma 4.2 is a straightforward calculation, once one fixes the precise definition of the group actions on source and target. We define the semidirect products by the rule

This semidirect product acts on the product by

(12) |

where and , and . The action is compatible with the projection on the first factor and standard action of on .

Next, we explicitly write out the action of on , or more generally of on , which is implicitly already given by (12) and the modular embedding. For , set . Then acts via

(13) |

where , , and and where

## 5. Compactifying the universal family over

We will compute the classes of the curves as the image of a locus cut out in the universal family of abelian surfaces over the pseudo-Hilbert modular surface. Over the open pseudo-Hilbert modular surfaces, this family is described as the quotient (see Section 4)

To perform intersection calculations, we need to work on a compact space and the aim of this section is to describe explicitly such a compactification of . Our strategy is as follows. The universal family over the modular curve has a simple compactification, by adding a ’-gon’ of rational curves at every cusp, the simplest instance of a toroidal compactification. In order to reduce from to such a situation, we have to pass from to a finite cover where this surface is a product, as explained in the previous section, and then to pass fiberwise to an isogenous abelian variety.

The aim of this section is to exhibit a compactification of by describing the action of the -step covering group on the product of two compactified universal elliptic curves. We thus present a compactification of as a quotient of a smooth compact variety by a finite group action. Along with this, we introduce local coordinates at the boundary that will be used to define bundle extensions in the next section.

For this purpose we note that has a normal subgroup that is equal to a product , where

The quotient is a product family