# Phase Structure of Anisotropic Antiferromagnetic Heisenberg Model on Layered Triangular Lattice: Spiral State and Deconfined Spin Liquid

## Abstract

In the present paper, we study spin- antiferromagnetic (AF) Heisenberg model on layered anisotropic triangular lattice and obtain its phase structure. We use the Schwinger bosons for representing spin operators and also coherent-state path integral for calculating physical quantities. Finite-temperature properties of the system are investigated by means of the numerical Monte-Carlo simulations. Detailed phase diagram of the system is obtained by calculating internal energy, specific heat, spin correlation functions, etc. There are AF Néel, paramagnetic and spiral states. Turning on plaquette term (i.e., the Maxwell term on a lattice) of an emergent U(1) gauge field that flips a pair of parallel spin-singlet bonds, we found that there appears a phase that is regarded as a deconfined spin-liquid state, though “transition” to this phase from the paramagnetic phase is not of second order but a crossover. In that phase, the emergent gauge boson is a physical gapless excitation coupled with spinons. These results support our previous study on AF Heisenberg model on a triangular lattice at vanishing temperature.

## 1Introduction

Study of quantum spin models has a long history. In particular after the discovery of the high-temperature superconductors, exotic quantum spin states have been intensively explored. Among them, spin-liquid state with a deconfined spinon has interested many theoretical and experimental researches[1]. Recently experiments of the anisotropic triangular antiferromagnet (AF magnet) CsCuCl revealed the existence of the incommensurate spiral order at low temperature () and also spinon-like excitations at intermediate [2]. One may think that this material can be a candidate for so-called Z spin liquid[4]. Furthermore very recently, evidences for a spin liquid in EtMeSb[Pd(dmit)] at very low were reported[5].

In the previous paper[6], we studied frustrated AF Heisenberg model on an anisotropic triangular lattice in two dimensions (2D) at . We used the Schwinger bosons for representing spin operators, and derived an effective model for low-energy region assuming existence of a short-range spiral order. Then low-energy excitations are spinons and an emergent gauge field with local gauge symmetry. We studied the effective gauge model for the quantum AF Heisenberg model by means of Monte Carlo (MC) simulations, and obtained phase diagram. There exist the spiral state, paramagnetic (PM) dimer state and the spin-liquid state in the phase diagram. These phases can be also labeled by the gauge dynamics, i.e., Higgs, confinement, and Coulomb phases, respectively.

In the present paper, we shall continue the study in the previous work and investigate closely related model, i.e., AF Heisenberg model on layered anisotropic triangular lattice. We are interested mostly in finite- properties of the model. By using the Schwinger bosons and CP path-integral methods, direct application of the MC simulations becomes possible *without* assuming any kind of (short-range) orders. We shall clarify the phase diagram of the model. Results obtained in this paper support the study in the previous paper[6].

This paper is organized as follows. In Sec.II, we shall introduce models and give a derivation of the effective gauge-theory model. By using the Schwinger bosons, local U(1) gauge symmetry naturally appears. Then we discuss possible phases in the frustrated AF spin systems. Section III is devoted to numerical studies. We investigated phase structure of the model by calculating the internal energy, specific heat, spin correlation functions, etc. These numerical calculations show that there exist AF, PM, spiral and spin-liquid phases. Detailed study on the critical behaviors between these phases is given. Section IV is devoted to summary and discussion.

## 2Quantum AF spin model, Schwinger bosons, CP representation and gauge theory

In the present paper we shall study a spin- anisotropic AF Heisenberg model on a layered triangular lattice shown in Figure 1. For simplicity, we first consider a 3D cubic lattice and then add diagonal links in the upper-right direction (1-2 direction) in 2D layers. Quantum Hamiltonian of the spin system is given as

where is spin operator at site and is a direction index and also denotes unit vector in 3D lattice, whereas and are those of the 2D lattice. Therefore, the -term in Eq.(Equation 1) is the 3D nearest-neighbor (NN) AF interaction, whereas the -term is the next-nearest-neighbor (NNN) AF coupling in 2D layers. There exists AF Néel order for at low temperature (), and it is expected that a quantum phase transition takes place as is increased.

In this paper, we shall investigate finite- properties of the system (Equation 1) in detail. To this end, we employ the Schwinger-boson representation and the coherent-path-integral methods[7]. By means of these methods, numerical study of the system can be performed straightforwardly. In terms of the Schwinger bosons at site , (where denotes transpose of the vector/matrix ), the spin operator is expressed as

where is the Pauli spin matrices. As the magnitude of the quantum spin is , the physical states of the Schwinger bosons, , have to satisfy the following constraint at each site ,

We use the coherent-state path integral for the study of the system (Equation 1) expressed in terms of the Schwinger bosons. To this end, we introduce CP variables corresponding to , which satisfy the constraint

as required by Eq.(Equation 3).

Then the partition function is given by

where is the imaginary time, , denotes the path-integral over CP variables and is obtained from Eq.(Equation 1) by using Eq.(Equation 2). in Eq.(Equation 5) is explicitly given as follows,

where , which is nothing but the time-reversal spinor of , and we have used the fact that and are an orthogonal and complete set of vectors in the CP space. If one tries to numerically study the system (Equation 5) by means of the MC simulations, one immediately encounters difficulties in the important sampling procedure because the first term in the action is pure-imaginary. For , it is known that by integrating out the CP variables at all odd (or even) sites of the cubic lattice by assuming a short-range AF order, resultant action has a quartic form of ( even sites) and has a lower bound[8]. This calculation, however, cannot be applicable for the case that we are interested in. Therefore we shall take another way to avoid the imaginary term in , i.e., we consider finite- properties of the system (Equation 5) and ignore the imaginary-time dependence of variables . Study of finite- properties of the system is not only interesting itself but also gives an important insight into low- properties of the system as it is expected that an ordered phase at finite survives at lower ’s[9]. In this approximation, the partition function is given as

where and .

For later discussion, it is useful to rewrite the action in Eq.([6]) as follows. We first rename CP variables at odd site of the cubic lattice as

and introduce a gauge field at link for the AF spin-pair channel. Then

One-link integral over the gauge field can be performed exactly,

where is the modified Bessel function. From the behavior of the modified Bessel function , relation between the parameters and is obtained as follows,

Action is invariant under the following *local gauge transformation*,

where is an arbitrary gauge-transformation parameter. In order to investigate the possibility of the appearance of a spin liquid with deconfined spinon excitations, study of the gauge dynamics and behavior of the gauge field is important and necessary.

We also add plaquette term of the gauge field to the action in Eq.(Equation 9) as

where the last plaquette term is the counterpart of the Maxwell term of the gauge field in the continuum, and corresponds to the ring-exchange terms of spins like

for the case of small value of . Higher-order terms of correspond to nonlocal interactions between spins. It should be noticed that the above plaquette term is defined on the 3D cubic lattice, and therefore the induced ring-exchange interaction is three-dimensional. The gauge field is related to the original Schwinger-boson operators as

i.e., corresponds to creation and destruction operators of spin-singlet bond at sites and . Therefore the -terms in the action (Equation 10) flip pairs of parallel nearest-neighbor spin-singlet bonds, and enhance appearance of the resonating-valence-bond (RVB) liquid[10].

From the previous studies[11], phase structure of the quantum spin models corresponding to in Eq.(Equation 10) is expected as follows,

For , a phase transition from a paramagnetic state to the Néel state with AF long-range order takes place as is increased[12]. In a gauge-fixed formalism, the AF Néel state corresponds to the state in which and .

As the value of is increased in the AF phase, a spiral state appears at some critical value of [13]. In the spiral state, is parameterized as

where is a constant, and the condensation of smoothly varying field takes place, .

Furthermore, as the value of is increased, a spin-liquid state with a deconfined spinon appears. In the spin-liquid phase, and the gauge dynamics of is in the Coulomb phase. Gapless gauge boson appears as a low-energy excitation coupled to spinons.

In the following section, we shall show the results of study on the phase diagram and physical properties of the models in Eq.(Equation 10) and in Eq.([6]), which support qualitatively the above expectation. As mentioned in the introduction, the experiments for frustrated quantum magnet CsCuCl observed the spiral state and deconfined spin-liquid state[2]. Experimental results suggest a crossover in nature of the excitations from spin- spin waves at low energies to deconfined spin- spinons at medium to high energies. Furthermore, EtMeSb[Pd(dmit)], which is studied intensively these days, is closely related to the present model. Therefore, results in this paper are relevant to these materials.

## 3Numerical studies

### 3.1 case

In the previous section, we have derived the effective models of the CP U(1) gauge variables from the AF Heisenberg model on layered triangular lattice. In this section we show results of the numerical study of the models obtained by means of the MC simulations. We employed the *free boundary condition* in the plane as the system may have an incommensurate spiral order with the layered structure.

We first consider the case with . We investigated phase structure of the model by calculating the internal energy and the specific heat for various values of and ,

where is the system size of the 3D lattice. In the practical calculation, we employed the local update by the standard Metropolis algorithm for the total system with size and performed measurement of physical quantities in the central subsystem[14].

We have found that exhibits no anomalous behaviors, whereas exhibits singular behaviors that indicate existence of second-order phase transitions as and are varied. Observed phase transition lines in the plane are shown in Figure 2.

We first consider the PM-AF phase transition. In Figure 3, we show and as a function of for . It is obvious that exhibits no anomalous behavior whereas has a peak at and the peak develops as the system size is increased. This behavior of indicates a second-order phase transition at . In order to verify existence of the phase transition and understand physical meaning of each phase, we investigated correlation functions of spins that are given as follows,

where . Numerically obtained results are shown in Figure 4. At , the correlation functions have no long-range order (LRO). On the other hand at , they exhibit AF LRO. (Please recall that we have changed variables .) From this result, we conclude that transition from the PM to AF phases takes place at .

We turn to the phase transition from the PM to spiral states as shown in Figure 5. For , calculation of the total specific heat as a function of is shown in Figure 5. We also measured the specific heat of each term of the action, which is defined similarly to in Eq.(Equation 12), in order to see the physical meaning of the phase transition.

where

See Figure 6. From these results, it is obvious that a second-order phase transition from the PM to spiral states takes place at . It is interesting to see that the -term of the action tends to fluctuate strongly at the phase transition point but the -term does not.

We measured the spin correlations at and . The results are shown in Figure 7. It is obvious that at the spin does not have a LRO, whereas at it has a spiral LRO. One should notice, however, that at the spin correlation has a short-range spiral order and therefore we call this “phase” a *tilted-dimer state*, though there is no sharp phase boundary between the ordinary PM (at ) and tilted-dimer state. This observation supports our previous study of the AF magnets on anisotropic triangular lattice assuming short-range spiral order[6]. We also measured the spin correlation in the inter-layer direction in the spiral state, and found that it has an ordinary AF correlation as it is expected. It is interesting to see a snapshot of spin configurations in the spiral and AF states. See Figure 8.

We calculated the specific heat and spin correlations for various values of the parameters and and have obtained phase transition line that separates the PM and spiral phases.

Finally let us turn to the spiral-AF phase transition (see Figure 9.). We show calculations of and the specific heat of and -terms. See Figs.Figure 9 and Figure 10. It is obvious that the total specific heat exhibits only very weak anomalous behavior but and both show sharp peak at as the system size is increased. From this result, we conclude that the transition from the spiral to AF phases is of second order.

It is also interesting to see how the spin correlation changes from the spiral to AF phases. Results in Figs.Figure 11 show that the spin correlation gradually changes from the spiral order to AF order.

We also numerically studied the original model in Eq.([6]), and obtained similar results to those of model . Obtained phase diagram is shown in Figure 12, and spin correlation functions in Figure 13. Phase transitions are of second-order and spin correlation functions have similar behavior to those of .

Result for the spiral state obtained in the present subsection obviously means that the -Néel state is realized in each layer for the case of the isotropic triangular case . This result is in good agreement with the previous study on the AF Heisenberg model on isotropic triangular lattice at [15]. On the other hand, some of the previous study on the anisotropic AF Heisenberg model on 2D triangular lattice at suggested the existence of spin-liquid phases[16]. However the results obtained in this subsection show that it does not exist in the present model. In order to address the possibility of the spin-liquid phase, we shall study effect of the plaquette term of the emergent gauge field in the following subsection.

### 3.2 case

In this subsection, we shall study the effects of the plaquette -term in the action . For the case, we found that there exist three phases, i.e., the AF, spiral and PM phases. It is expected that the gauge dynamics in both the AF and spiral phases is in the Higgs phase as the condensation of the spinon field suppresses fluctuations of the gauge field . Low-energy excitations are gapless spin wave in the both phases. On the other hand in the PM phase, it is known that the confinement phase is realized and low-energy excitations are *bound state of the spinons* like a spin-triplet because of the strong fluctuations of .

As explained in Sec.2, another possible phase in the quantum spin systems is the spin-liquid phase. In that phase, there exist no LRO’s, whereas low-energy excitation is the *deconfined spinon* . This means that in the spin liquid only small fluctuations of the gauge field are realized and the gauge dynamics is in the Coulomb phase. Knowledge of the gauge field theory suggests that such a spin-liquid phase may be realized by turning on the plaquette -term because this term suppresses large fluctuations of the gauge field[17]. In the previous study on the gauge model of the spiral and spin-liquid phases[6], we found that the deconfined spin-liquid phase is realized in the vicinity of the spiral and tilted-dimer states. In the present paper, we show the results of study on the U(1) gauge model in the parameter plane with the value of fixed.

We first show phase diagram obtained by the MC simulations for in Figure 14. Reason for choosing this value of is that the spiral and tilted-dimer states appear as the value of is varied for . As shown in Figure 14, there exists a crossover line emanating from the point in the vertical direction. This crossover line separates dense and dilute instanton regions, whereas the both regions belong to the confinement phase of the U(1) gauge model in 3D. Besides the crossover line, there exist two sharp second-order phase transition lines emanating from and , respectively. As shown in Figure 14, these are the spiral and AF phase transition lines, respectively. We also found another “transition line” emanating from , which we identify as a crossover to the spin-liquid phase.

We show the total specific heat as a function of for and in Figure 15. There are two peaks at and , and the second peak at develops as the system size is increased indicating a second-order phase transition. Calculation of the spin correlation given later on shows that it is the phase transition to the spiral state. On the other hand, the first peak at does not develop as the system size is increased. More detailed calculation is shown in Figure 16. There exists small system-size dependence, but we think that this size dependence comes from the free-boundary condition that we took for the calculation.

It is useful to see how specific heat of each term behaves. See Figure 17. The specific heat of the -term , which is defined similarly to and , is a decreasing function of and changes its behavior at . On the other hand, the specific heats of the -term and -term both have peaks at and and these peaks develop as the system size is increased. This result suggests that there is a second-order phase transition at besides at .

It is important to see how the spin correlation function behaves and verify properties of each phase observed by the measurement of . Obtained results of the spin correlation for are shown in Figure 18. At and , there exists no LRO, whereas at and the LR spiral order appears. Furthermore at , the spin correlation shows the AF LRO. All the above results verify the phase diagram shown in Figure 14

In order to investigate the gauge dynamics, it is useful to study instanton (monopole) density , which measures magnitude of topologically nontrivial fluctuations of the gauge field . is defined as follows for the gauge field configuration [18]. First we consider the magnetic flux penetrating plaquette

We decompose into its integer part , which represents the Dirac string (vortex line), and the remaining part ,

Then instanton density at the cube around the site of the dual lattice is defined as

where is the antisymmetric tensor.

In Figure 19, we show the calculation of

As the gauge dynamics is already in the dilute-instanton region of the confinement phase for , the value of is small, but it decreases at , and its behavior becomes clear as the system size is increased. This result indicates that the region between two peaks at and corresponds to the “deconfined Coulomb phase”. In this phase, the gapless gauge boson appears as a low-energy excitation besides the deconfined spinons.

The global phase diagram in Figure 14 is consistent with that of the spin-liquid model obtained in Ref.[6]. In the model, however, the phase transition from spiral phase to PM phase is of first order and there exists sharp phase boundary between spin-liquid and PM phases. Anyway, results obtained in this paper support discussions in term of models for frustrated quantum AF magnets at .

## 4Summary

In this paper we have studied phase structure of the AF spin model on the layered anisotropic triangular lattice. We used the Schwinger bosons for represent ting quantum spins and also employed the coherent-state path integral methods. We focused on finite- phase diagram and investigated it by means of the MC simulations. We calculated the internal energy, specific heat and spin correlation functions. In the absence of the -term, we found that there exist three phases, i.e., AF, PM and spiral phases. All phase transitions between them are of second-order.

Then we turned on the -term and investigated if the deconfined spin-liquid phase appears. Calculations of the specific heat and instanton density shows that there exists deconfined spin liquid in the vicinity of the spiral and tilted dimer states. This result is in good agreement with the results of our previous study[6] in which we assumed a short-range spiral order and focused on the region in the phase diagram shown in Figure 20. However, the present study indicates that there is not sharp phase boundary between the tilted-dimer state and spin liquid, i.e., this “transition” is a crossover. At very lower , it is possible that this crossover changes to a genuine phase transition as the imaginary time plays a role of another dimension.

It is very interesting to study how hole doping changes the observed phase diagrams and how doped holes behaves in various magnetic phases. This problem is under study and we hope that we shall report the results in a future publication.

This work was partially supported by Grant-in-Aid for Scientific Research from Japan Society for the Promotion of Science under Grant No.20540264.

### Footnotes

- Present address: Department of Applied Physics, Osaka University, Suita, Osaka, 565-0871 Japan

### References

- See for example, S.Sachdev, Nature Physics
**4**, 173(2008);

L.Balents, Nature**464**, 199(2010), and references cited therein. - R.Coldea, D.A.Tennant, A.M.Tsvelik, and Z.Tylczynski,

Phys. Rev. Lett.**86**, 1335(2001). - R.Coldea, D.A.Tennant, and Z.Tylczynski, Phy. Rev.
**B68**, 134424(2003). - Y.Qi, C.Xu, and S.Sachdev, Phys. Rev. Lett.
**102**, 176401(2009). - M.Yamashita, N.Nakata, Y.Senshu, M.Nagata, H.M.Yamamoto, R.Kato, T.Shibauchi, and Y.Matsuda, Science
**328**, 1246(2010);

T.Itou, A.Oyamada, S.Maegawa, and R.Kato, Nature Physics**6**, 673(2010). - K.Nakane, A.Shimizu, and I.Ichinose, Phys. Rev.
**B80**, 224425(2009). - See for example, G.Misguich, “Exact Methods in Low-dimensional Statistical Physics and Quantum Computing”, Les Houches summer school 2008.
- I.Ichinose and T.Matsui, Phys.Rev.
**B45**, 9976(1992). - See for example, K.Aoki, K.Sakakibara, I.Ichinose, and T.Matsui,

Phys. Rev.**B80**, 144510(2009). - P.W.Anderson, Science
**235**, 1196(1987). - S.Takashima, I.Ichinose, and T.Matsui, Phys. Rev.
**B72**, 075112(2005). - See for example, A.Auerbach, “
*Interacting Electrons and Quantum Magnetism*″ (Springer-Verlag, New York, 1994). - J.Merino, R.H.McKenzie, J.B.Marston, C.H.Chung, J.Phys.: Condens.Matter
**11**, 2965(1999). - N.Metropolis, A.W.Rosenbluth, M.N.Rosenbluth,

A.M.Teller, and E.Teller, J. Chem. Phys.**21**, 1087(1953). - See for example, S.R.White and A.L. Chernyshev, Phys. Rev. Lett.
**99**, 127004(2007),

and references therein. - S.Yunoki and S.Sorella, Phys. Rev.
**B74**, 014408(2006). - J.B.Kogut, Rev. Mod. Phys.
**51**, 659(1979). - T.A.DeGrand and D.Toussaint, Phys. Rev.
**D22**, 2478(1980).