# Spin liquids on a honeycomb lattice: Projective Symmetry Group study of Schwinger fermion mean-field theory

###### Abstract

Spin liquids are novel states of matter with fractionalized excitations. A recent numerical study of Hubbard model on a honeycomb latticeMeng et al. (2010) indicates that a gapped spin liquid phase exists close to the Mott transition. Using Projective Symmetry Group, we classify all the possible spin liquid states by Schwinger fermion mean-field approach. We find there is only one fully gapped spin liquid candidate state: “Sublattice Pairing State” that can be realized up to the 3rd neighbor mean-field amplitudes, and is in the neighborhood of the Mott transition. We propose this state as the spin liquid phase discovered in the numerical work. To understand whether SPS can be realized in the Hubbard model, we study the mean-field phase diagram in the spin-1/2 model and find an -wave pairing state. We argue that -wave pairing state is not a stable phase and the true ground state may be SPS. A scenario of a continuous phase transition from SPS to the semimetal phase is proposed. This work also provides guideline for future variational studies of Gutzwiller projected wavefunctions.

## I Introduction

Traditional Landau’s theoryLandau and Lifschitz (1958); Ginzburg and Landau (1950) points out that states of matter can be classified by their symmetry. And the low energy excitations can be understood by either bosonic modes or fermionic quasiparticles, which carry integer multiples of the quantum numbers of the fundamental degrees of freedom. Fractional quantum Hall liquids (FQHLs) provide a striking counterexample of the Laudau’s paradigm: different FQHLs all have the same symmetry, yet they are very different since a quantum phase transition is required to go from one liquid to another. To understand their differences, one has to go beyond Laudau’s paradigm and the concept of topological order was introducedWen and Niu (1990); Wen (1995). The quasiparticle excitations in FQHLs carry only a fraction of the fundamental electric charge. Meanwhile these fractionalized quasiparticles obey neither bosonic nor fermionic statistics and are dubbed anyons consequently.

Can strong interactions lead to similar novel states of matter in the absence of magnetic field? After the original proposal of AndersonAnderson (1987), intensive theoretical studies have revealed that spin systems can realize such novel phases of matter: spin liquids(SL). And a few experimental systems have been identified to be likely in spin liquid phasesShimizu et al. (2003); Helton et al. (2007); Okamoto et al. (2007). A spin liquid is often defined to be a quantum phase of spin-1/2 per unit cell that does not break translation symmetry. These liquid phases of spins are also distinct from one another by their topological order. Although a rigorous theorem is lacking because we are still unable to classify all possible topological order, it is generally believed that the excitation of a topological ordered phase is fractionalizedOshikawa and Senthil (2006).

Although theoretical studies have shown that spin liquid ground states exist for artificial model HamiltoniansAffleck and Marston (1988); Read and Sachdev (1991); Rokhsar and Kivelson (1988); Moessner and Sondhi (2001); Kitaev (2006); Wen (2003), it remains unclear whether a simple or experimentally realizable Hamiltonian can host such novel states. Recently a remarkable quantum Monte Carlo simulation of Hubbard model on a honeycomb latticeMeng et al. (2010) indicates that a gapped spin disordered ground state exists in the neighborhood of the Mott transition. Although a honeycomb lattice has two spin-1/2 per unit cell, it is impossible to have a band insulator phase without breaking lattice symmetry. Therefore this spin disordered phase should be topologically ordered and have fractionalized excitations. We will still term it a spin liquid.

What is the nature of this spin liquid phase? In this paper we try to propose the candidate states by Schwinger-fermion (or slave-boson) mean-field approachBaskaran et al. (1987); Baskaran and Anderson (1988); Affleck and Marston (1988); Kotliar and Liu (1988); Affleck et al. (1988); Wen and Lee (1996); Lee et al. (2006), following the techniques developed in LABEL:PhysRevB.65.165113. Our results can be summarized as follows. We search for the fully gapped spin liquids which lead us to focus on the mean-field states. We first use Projective Symmetry Group (PSG)Wen (2002) to classify all 128 possible mean-field states that preserve the full lattice symmetry as well as time-reversal symmetry. Notice the spin liquid phase in the numerical work seems to be connected to the semi-metal phase by a second-order phase transition, which suggests this state to be in the neighborhood of a uniform Resonating-Valence-Bond (u-RVB) state. So we classify all the 24 possible states around the u-RVB states. Among these 24 states, we find only 4 states can have a full energy gap in the spinon spectrum, while other 20 states have symmetry protected gapless spinon excitations. We find that up to 3rd neighbor mean-field amplitudes, only one of the four fully gapped state can be realized, and we term it as Sublattice Pairing State (SPS). We propose this state to be the spin liquid state discovered in the numerical study. We also study the mean-field phase diagram of the antiferromagnetic spin-1/2 model on a honeycomb lattice to understand whether SPS can be more favorable than the u-RVB state while both states are in the neighborhood of the Mott transition. We find when a spinon gap opens up by -wave pairing on top of the u-RVB state. This -wave pairing state is not a stable phase and is an artifact of the mean-field study where gauge dynamics are ignored. On the other hand, the proposed SPS state is continuously connected to the -wave pairing state by making the pairing phase sublattice dependent. This suggests the ultimate fate of -wave pairing state may be SPS. We propose that a more careful projected wavefunction study, which includes the gauge fluctuations, may be able to find SPS state as the ground state in the model. The possible continuous phase transitions from SPS into semi-metal phase are discussed.

## Ii Schwinger-fermion approach and PSG

In Schwinger-fermion approach, a spin-1/2 operator at site is represented by:

(1) |

A Heisenberg spin Hamiltonian is represented as . Because this representation enlarges the Hilbert space, states need to be constrained in the physical Hilbert space, i.e., one -fermion per site:

(2) |

Introducing mean-field parameters , , where is fully antisymmetric tensor, after Hubbard-Stratonovich transformation, the Lagrangian of the spin system can be written asWen (2002)

(3) |

where two-component fermion notation is introduced for reasons that will be explained shortly. is a matrix of mean-field amplitudes:

(4) |

are the local Lagrangian multipliers that enforces the constraints Eq.(2).

In terms of , Schwinger-fermion representation has an explicit gauge redundancy: a transformation , , leaves the action invariant. This redundancy is originated from representation Eq.(1): this local transformation leaves the spin operators invariantAffleck et al. (1988) and thus does not change physical Hilbert space.

One can try to solve Eq.(3) by mean-field (or saddle-point) approximation. At mean-field level, and are treated as complex numbers, and must be chosen such that constraints Eq.(2) are satisfied at the mean field level: . The mean-field ansatz can be written as:

(5) |

where . A local gauge transformation modify but does not change the physical spin state described by the mean-field ansatz. By construction the mean-field amplitudes do not break spin rotation symmetry, and the mean field solutions describe spin liquid states if translational symmetry is preserved. Different ansatz may be in different spin liquid phases. The mathematical language to classify different spin liquid phases is PSGWen (2002).

PSG is the manifestation of topological order in the Schwinger-fermion representation: spin liquid states described by different PSG’s are different phases. It is defined as the collection of all combinations of symmetry group and gauge transformations that leave invariant (as are determined self-consistently by , these transformations also leave invariant). The invariance of a mean-field ansatz under an element of PSG can be written as

(6) |

Here is an element of symmetry group (SG) of the spin liquid state. SG on a honeycomb lattice is generated by time reversal , reflection , rotation and translations as illustrated in FIG. 1 (see also appendix A). is the gauge transformation associated with such that leaves invariant.

There is an important subgroup of PSG, Invariant Gauge Group (IGG), which is composed of all the pure gauge transformations in PSG: . One can always choose a gauge in which the elements in IGG is site-independent. In this gauge, IGG can be global transformations: , the global transformations: , or the global transformations: , and we dub them , and state respectively.

The importance of IGG is that it controls the low-energy gauge fluctuations. Beyond mean-field level, fluctuations of and need to be considered and the mean-field state may or may not be stable. The low-energy effective theory is described by fermionic spinon band structure coupled with a dynamical gauge field of IGG. For example, state with gapped spinon dispersion can be a stable phase because the low-energy dynamical gauge field can be in the deconfined phaseWegner (1971); Kogut (1979). But for a state with gapped spinon dispersion, the gauge fluctuations would generally drive the system into confinement due to monopole proliferationPolyakov (1977), and the mean-field state would be unstable. And an state with gapped spinon dispersion should also be in the confined phase because there is no known IR stable fixed point of pure gauge theory in 2+1 dimension. Because the purpose of this paper is to search for stable spin liquid phases that has a Schwinger fermion mean-field description, we will focus on states.

If and , by definition we have . This means that the mapping is a many-to-one mapping. In fact it is easy to show that mapping induces group homomorphismWen (2002):

(7) |

Mathematically is an extension of by .

Our definition of PSG requires a mean-field ansatz . With Eq.(7), one can define algebraic-PSG which does not require ansatz . An algebraic-PSG is simply defined as a group satisfying Eq.(7). Obviously a PSG (realizable by an ansatz) must be an algebraic-PSG, but the reverse may not be true, because sometimes an algebraic-PSG cannot be realized by any mean-field ansatz.

To classifying all possible Schwinger-fermion mean-field states, we need to find all possible group extensions of the with a IGG. Here is the direct product of the space group of honeycomb lattice and the time-reversal group. In appendix A we show the general constraints that must be satisfied for such a group extension. In appendix B, using these constraints, we find there are in total 160 algebraic-PSGs on honeycomb lattice. And at most 128 PSGs of them can be realized by an ansatz . These 128 PSGs are the complete classification of spin liquids on a honeycomb lattice.

## Iii Classification of states around the u-RVB state

Can one further identify the candidate states for the spin liquid discovered in the numerical studyMeng et al. (2010)? The answer is yes. Numerically the spin liquid phase is found close to the Mott transition and it seems to be connected to the semimetal phase by a continuous phase transition. What are the Schwinger-fermion states in the neighborhood of the semi-metal phase?

Are there Schwinger-fermion mean-field states that can be connected to the semi-metal phase via a continuous phase transition? This question was firstly discussed by Hermele in LABEL:hermele:035125. Using slave-rotor formalism, it was shown that the semi-metal phase can go through a continuous phase transition into an u-RVB state (also termed as algebraic spin liquid (ASL) in LABEL:hermele:035125) at the mean-field level. This u-RVB ansatz, in terms of -spinon, can be written as , is real and summation is over all nearest neighbor bond. The single-spinon dispersion of u-RVB state is similar to the electronic dispersion in the semi-metal phase, which is composed of four two-component Dirac cones at the corner of Brillouin Zone, two from spin and two from valley. Physically it is easy to understand u-RVB state connecting with the semi-metal phase: At the Mott transition, only the charge fluctuation becomes fully gapped and the spinon dispersion still remember the semi-metal band structure.

The u-RVB ansatz can be simply expressed as a graphene-like nearest neighbor hopping of -fermions: Fig.1:

(8) |

where is real. Beyond mean-field level, the low-energy effective theory of u-RVB state is described by two-component Dirac spinons ( gauge doublet) coupled with a dynamical gauge fieldHermele (2007), i.e. QCD. In the large- limit QCD has a stable IR fixed point with gapless excitations and can be a stable ASL phaseAppelquist and Nash (1990). When the pure gauge QCD is in a confined phaseGross and Wilczek (1973); Politzer (1973). This indicates a critical and when confinement occursAppelquist and Nash (1990). Although no controlled estimate of is available, a self-consistent solution of the Schwinger-Dyson equationsAppelquist and Nash (1990) suggests . We will assume that and therefore u-RVB state is not a stable phase.

Due to the lack of the knowledge of the confinement mechanism, it is difficult to reliably predict the ultimate fate of the u-RVB state (or ASL). But one possibility is that the strong gauge interaction induces Higgs condensation which breaks the gauge symmetry down to , so that the renormalization group flows into a stable fixed point of gauge theory. Based on this assumption, we can propose a scenario of a continuous phase transition from the semi-metal phase into a spin liquid phase: the critical point is still described by the slave-rotor critical theory discussed in LABEL:hermele:035125. But on the Mott insulator side, a dangerously irrelevant operator (for example, can be a four-fermion interaction term) becomes relevant and finally drive the RG flow away from the ASL fixed point and flow into a stable fixed point of a phase by Higgs mechanism. Here we assume the ASL still describes an unstable fixed point with relevant directions. This scenario is schematically shown in Fig.2, which has the same spirit of the deconfined quantum criticalitySenthil et al. (2004).

If this scenario is correct, the mean-field ansatz of the spin liquid should be connected to the u-RVB ansatz by a continuous Higgs condensation, which breaks the IGG down to . During this transition, the u-RVB ansatz and the amplitudes play the role of the Higgs boson. We define a state to be around (or in the neighborhood of) the u-RVB when the state can be obtained by an infinitesimal change .

The PSG of must be a subgroup of the PSG of the u-RVB state Eq.(8). In appendix C we classify all these possible PSG subgroups with the IGG, which allows us to construct all possible states around the u-RVB state. This technique was firstly developed by WenWen (2002). We find 24 gauge inequivalent PSGs as listed in Table 1 in appendix C.

Can these 24 SL states have a full energy gap? We find not all of them can have a gapped spinon spectrum. This can be understood starting from a Dirac dispersion of the u-RVB state. To gap out the Dirac nodes, at least one mass term in the low-energy effective theory of a given state must be allowed by symmetry. In appendix E we show that only 4 of the 24 states allow mass term in the low energy theory. Thus only these 4 states are fully gapped spin liquids around u-RVB state. The other 20 states have symmetry protected gapless spinon dispersions.

These four states are state #16,#17,#19, and #22 in Table 1 in appendix C. We can generate their mean-field ansatzs by these PSGs. We find that up to the 3rd neighbor mean-field amplitudes as shown in Fig.1, only one of these four states can be realized, which is state #19. As shown in appendix E.2, mean-field ansatzs up to the 3rd neighbor of the other three states actually have a IGG. Only after introducing longer-range mean-field bonds can these three states have a IGG. In particular, state #16 requires 5th neighbor, state #17 requires 4th neighbor and state #22 requires 9th neighbor amplitudes, while state #19 only requires 2nd neighbor amplitudes. Because the expansion of the Hubbard model give a rather short-ranged spin interaction for the SL phase found in numericsMeng et al. (2010) (), the other three states are unlikely to be realized in a Hubbard model on honeycomb lattice.

After choosing a proper gauge, the mean-field ansatz of #19 can be expressed as a sublattice dependent pairing of the -spinons, as shown in Fig.1:

(9) |

and we term it as sublattice pairing state (SPS). Note that , because otherwise the ansatz has IGG. We propose SPS to be the SL phase found in numerics.

## Iv Schwinger-fermion mean-field study of the model on honeycomb lattice

Can SPS be realized in the Hubbard model when , where numerics shows a gapped SL phase? In particular, by the Mott transition theory of HermeleHermele (2007), the u-RVB (or ASL) state is in the neighborhood of the Mott transition. Can SPS be more favorable than the ASL state? To address this question, we use expansion of the Hubbard modelMacDonald et al. (1988) to obtain an effective spin model on honeycomb lattice:

(10) |

where and are the 1st neighbor and 2nd neighbor antiferromagnetic coupling. Following LABEL:PhysRevB.37.9753, we find up to order, the effective and are:

(11) |

Naively plugging in gives .

We use the variationally mean-field ansatz Eq.9. Note that this mean-field study is biased towards spin disordered ground state. For example, we do not include Neel order which is known to be the ground state at , and we also do not include the spiral spin order which is found by semiclassical study of modelRastelli et al. (1979); Fouet et al. (2001). The purpose of the current mean-field study is to understand whether a gapped spin liquid can be more favorable compared to the gapless ASL state when is tuned up and frustration becomes important.

By minimizing the mean-field energy in Eq.(3), the phase diagram of model is obtained and shown in Fig.3, where we fix and is scaled from Eq.(3) by . We find that when (or ), the ground state is u-RVB(or ASL) state: and . When , the ground state is an -wave pairing state: and . The -wave pairing state opens an energy gap for spinons but has remaining gapless gauge fluctuation. Due to monopole proliferationPolyakov (1977) the -wave pairing state is not a stable phase. In this mean-field study, the gauge fluctuations are not considered and this is the reason why we find -wave pairing state as a ground state. Taking gauge fluctuations into account, the likely fate of the -wave pairing state is that becomes nonzero and the SPS state is realized.

We propose to study the model by Gutzwiller projected wavefunction variational approachGros (1989) because it can be viewed as a method to include the gauge fluctuation. We leave this projected wavefunction study as a direction of future research, which may realize SPS as the ground state. Projected wavefunctions are also classified by PSG, so the present work also provide guideline for the search of ground states in the projected wavefunction space.

## V Discussion

In this work we completely classified the mean-field states in the Schwinger-fermion approach. Using physical argument, we identify a single state: SPS, as the possible spin liquid phase found in the recent Quantum Monte Carlo study of the Hubbard model on a honeycomb latticeMeng et al. (2010). SPS is in the neighborhood of the semimetal phase and we propose a scenario for the continuous transition connecting the two phases.

In our mean-field study of the model, the -wave pairing state is realized for a fairly large , corresponding to a fairly large . A higher order spin-spin effective interaction such as the 6-spin ring exchange term and/or a more careful projected wavefunction study may realize SPS phase for a smaller .

In a recent workWang (2010), Wang study the mean-field states in the Schwinger-boson approach, and identify a zero-flux SL state, which is naturally connected to a Neel ordered state by a potentially continuous phase transition. Whether the SPS found in the present work is related to Wang’s result is unclear. And we leave the possible continuous transition from SPS to the Neel ordered phase as a subject of future research.

YR thanks Ashvin Vishwanath and Fa Wang for helpful discussions. YML thanks Prof. Ziqiang Wang for support during this work under DOE Grant DE-FG02-99ER45747. YR is supported by the start-up fund at Boston College.

## Appendix A General conditions on projective symmetry groups on a honeycomb lattice

As mentioned in section II, SG on a honeycomb lattice is generated by time reversal transformation , translations along : , plaquette-centered 60 rotation, and a horizontal mirror reflection as shown in Fig.1. In the present problem, the symmetry group can be represented as

where and , , since the generators satisfy

(12) |

Here stands for the identity element of . To completely determine the multiplication rule of this group, we need to identify the multiplication rule of two different generators in an order different from :

(13) | |||

(14) | |||

(15) | |||

(16) | |||

(17) | |||

(18) | |||

(19) |

The above relations can be written in an alternative way

(20) | |||

(21) | |||

(22) | |||

(23) | |||

(24) | |||

(25) | |||

(26) | |||

(27) |

which determines the inverse of all the group elements.

As introduced in section II, the mean-field ansatz of a spin liquid is invariant under the action of any element of a projective symmetry group (PSG). The multiplication rule of the symmetry group would immediately enforce the following constraints on a PSG by its definition: if then

(28) |

On the other hand, we know those pure gauge transformations, under which the mean-field ansatz is invariant, constitute a subgroup of PSG, coined the invariant gauge group (IGG):

(29) |

Therefore from (28) we have the following constraints on the elements of a PSG

The above condition holds for any two group elements of SG. Similar with SG, we can choose a set of generators in any given PSG: . Any given element in PSG can be written in the standard form:

(30) |

Since the multiplication rule of SG on a honeycomb lattice is completely determined by (12)-(19), or equivalently (20)-(27), the only independent constraints on the PSG generators are the following:

(31) | |||

or more specifically

(32) | |||

Above are all the general consistent conditions to be satisfied by the generators of a PSG on a honeycomb lattice.

We will use to label a site in a honeycomb lattice, where are the coordinates of the unit cell in basis and for and sublattice respectively. For convenience, we summarize the coordinate transformation of all the generators in the symmetry group on a honeycomb lattice as follows:

(33) | |||

## Appendix B Classification of projective symmetry groups on a honeycomb lattice

As discussed in section II, the problem of classifying all possible Schwinger-fermion mean-field states is mathematically reduced to finding all possible PSGs. Let us firstly find all algebraic PSGs.

### b.1 General discussions

In the case of spin liquids, the IGG of the corresponding PSG is a group: . The constraints listed in appendix A now becomes

(34) | |||

(35) | |||

(36) | |||

(37) | |||

(38) | |||

(39) | |||

(40) | |||

(41) | |||

(42) | |||

(43) | |||

(44) | |||

(45) | |||

(46) | |||

(47) |

where all the ’s take value of . Not all of these conditions are gauge independent. Because we can re-choose the gauge part of generators such as by multiplying them by (an element of IGG), only those conditions in which the same generator shows up twice are gauge independent. We can use this gauge dependence to simplify these conditions. Because () only show up once in the equation of (), we can always choose a gauge such that . All other ’s are gauge independent.

In the following we will determine all the possible PSG’s with different (gauge inequivalent) elements . These different PSG’s characterize all the different type of spin liquids on a honeycomb lattice, which might be constructed from mean-field ansatz .

First notice that under a local gauge transformation , the PSG elements transform as . Making use of such a degree of freedom, we can always choose proper gauge so that

Now taking (38) into account, we have and therefore

(48) | |||

Meanwhile, from (34), (39) and (40) we can immediately see that , and the gauge inequivalent choices of are the following

(49) |

where .

As discussed earlier, we can always choose a proper gauge so that . Then from conditions (41) and (42) we see that

(50) |

similarly from conditions (44) and (45) we have

(51) |

where . Note that (35) and (46) give further constraints to the above expression (51):

(52) |

Now we see the elements of PSG can be expressed as

(53) | |||

(54) | |||

Consistent conditions (35), (37), (43), (46) and (47) correspond to the following constraints on matrices :

(55) | |||

where and is a unit vector.

### b.2 A summary of 160 different PSG’s

Below we summarize all the 160 possible PSG’s obtained through
solving (55). We use capital Roman numerals
(\@slowromancapi@) and (\@slowromancapii@) to label and
respectively. Roman numerals (i) and
(ii) are used to label respectively. (A)
and (B) are used to label respectively.
Finally () and () are used to label respectively.

(\@slowromancapi@) :

It’s easy to see that