# Exotic phase diagram of a topological quantum system

###### Abstract

We study the quantum phase transitions (QPTs) in the Kitaev spin model on a triangle-honeycomb lattice. In addition to the ordinary topological QPTs between Abelian and non-Abelian phases, we find new QPTs which can occur between two phases belonging to the same topological class, namely, either two non-Abelian phases with the same Chern number or two Abelian phases with the same Chern number. Such QPTs result from the singular behaviors of the nonlocal spin-spin correlation functions at the critical points.

###### pacs:

05.30.Rt, 03.65.Vf## I Introduction

A quantum phase transition (QPT) involves an abrupt change of the ground state in a many-body system due to its quantum fluctuations.Sachdev () Discovering and characterizing new QPTs in a two-dimensional topological quantum system have recently attracted considerable interest.HongYao (); XYFeng (); Kitaev06 (); xgwen () Because the ground states in some topological quantum systems (e.g., the Kitaev spin models on honeycombKitaev06 () and triangle-honeycombHongYao () lattices) are exactly solvable, QPTs in these systems can be analytically investigated. In these topological systems, the discovered QPTs include the transition between a gapped Abelian phase and a gapless phase,XYFeng (); SYang () the transition between Abelian and non-Abelian phases,CNash (); HongYao (); FABais (); JQYou (); SBChung () and the transition between two non-Abelian phases with different Chern numbers.Kitaev06 () Also, an unconventional QPT between two non-Abelian phases was foundGKells () in the Kitaev spin model on a triangle-honeycomb lattice by a fermionization method. Nevertheless, to the best of our knowledge, the QPT between two topological phases of the same Chern number (which belong to the same topological class) has not yet been found.

Here we show that, in the Kitaev spin model on a triangle-honeycomb lattice, a QPT can indeed happen between two gapped phases in the same topological class, in addition to the ordinary topological QPT between two phases of different Chern numbers. To demonstrate this, we focus on two parameter regimes as typical examples: (i) When the parameters vary across a critical curve separating two gapped phases of the same Chern number or , a first-order QPT occurs; (ii) when the parameters vary across a special critical point where several critical curves meet, a continuous QPT can occur. This is due to the exotic ground-state phase diagram which has either critical curves between two gapped phases (with the same or different Chern numbers) or critical points where four different gapped phases (with Chern numbers and ) terminate. These results reveal that the Kitaev spin model on a triangle-honeycomb lattice exhibits novel topological properties. Moreover, we find that such QPTs result from the singular behaviors of the nonlocal spin-spin correlations at the critical points.

The paper is organized as follows. In Sec. II, we present the solution for the ground state of the Hamiltonian in the uniform-flux sector. Sections III and IV show two typical phase diagrams of the ground-state wavefunctions and study various QPTs in the considered Kitaev spin model. Finally, a brief conclusion is given in Sec. V.

## Ii Ground state in the uniform-flux sector

The Kitaev spin model on a triangle-honeycomb lattice is schematically shown in Fig. 1(a) and the model Hamiltonian is given by

where , with and , are the three Pauli operators at site . Among the six sums in Eq. (LABEL:spinH), four involve interactions within each unit cell (see Fig. 1): (i) The -link couples either spins 2 and 3 or spins 5 and 6, (ii) the -link couples either spins 1 and 3 or spins 4 and 6, (iii) the -link couples either spins 1 and 2 or spins 4 and 5, and (iv) the -link couples spins 3 and 6. Nearest-neighbor unit cells are coupled by the - and -links. As found in Ref. HongYao, , the ground state of Hamiltonian (LABEL:spinH) is at least 8 (6)-fold degenerate for the Abelian (non-Abelian) phase.

For the three types of hermitian plaquette operators defined by [see Fig. 1(b)]

(2) |

each has eigenvalues . These plaquette operators commute with not only each other but also the Hamiltonian (LABEL:spinH). As verified in Ref. HongYao, . the ground state of Hamiltonian (LABEL:spinH) is either in the sector of the Hilbert space in which all plaquette operators in Eq. (2) have eigenvalue one or in the sector where the time-reversal transformation of each plaquette operator in Eq. (2) has eigenvalue one. The former is denoted as the uniform-flux sector.footnote1 ()

By using the Jordan-Wigner transformation, Hamiltonian (LABEL:spinH) can be converted to a form represented by Majorana fermions.HongYao () Performing Fourier transform on the Hamiltonian in the uniform-flux sector, we obtain

(3) |

where denotes the first Brillouin zone. In Eq. (3), are fermionic operators, where is the Fourier transform of the Majorana fermionic operator at site () in a unit cell, and is a function of variable (see Appendix A). One can prove that Hamiltonian (3) breaks time reversal symmetry from the property of Majorana fermions . This is in sharp contrast to the Kitaev model on a honeycomb lattice in which the time reversal symmetry is preserved. This is due to the difference between the bipartite nature of the honeycomb lattice and the non-bipartite nature of the triangle-honeycomb lattice. Hamiltonian (3) has six energy bands: , and , with . In each unit cell, six Majorana fermions are defined, but the number of the corresponding fermions are three. Thus, the lowest three bands should be filled for the ground state: , with the normalization factor which results from the fact that the fermion is constructed from Majorana fermions in space, each of whom can only take as its occupation number. The ground-state energy per site is given by , where is the number of unit cells. The energy-band gap, i.e., the minimal energy to excite a fermion from the ground state is , where denotes the minimal value of a function with variable .

Below we show the ground-state phase diagrams in two different parameter regimes. The first diagram contains critical curves separating two phases of either the same or different Chern numbers. This unveils that QPTs can also occur in the same topological class. The second diagram contains critical points where four different phases with Chern numbers , , and terminate.

## Iii Case A: , , and

We first choose parameters , and to study the ground-state property of the system. In particular, the parameters with and correspond to the case studied in Ref. HongYao, where a topological QPT between an Abelian phase and a non-Abelian phase was found at point . There is also a perturbative studySDusuel () of this model with parameters either or when .

In order to find the ground-state phase diagram of the Kitaev spin model, we should first distinguish the gapless-phase regions from the gapped-phase regions in the phase diagram. Then, we calculate the Chern numberXLQi06 (); TKNN () as a topological index to characterize each gapped-phase region. We note that the six energy bands satisfy the relation , where , which implies that the closing of the energy-band gap corresponds to either or . Combining this condition with the relation , we can derive that if one has , the energy-band gap is zero, i.e., . This gives rise to

(4) |

where () applies when (). In Fig. 2(a), each gapless phase determined by Eq. (4) is schematically shown by a solid curve or line. One can see that the plane is divided into 12 distinct regions by these solid curves or lines. Each of these 12 regions is a gapped topological phase and can be characterized by a Chern number.

We can define the Chern number by using the Berry’s phase for a gapped ground state.TKNN (); XLQi06 () Among the six bands of Hamiltonian (3), the lower three bands with states , where , are occupied. The Berry’s phase gauge field in momentum space is

(5) |

where . This gives

(6) |

where denotes the imaginary part of a complex variable. Numerical results show that each yellow (red) region in Fig. 2(a) corresponds to a gapped non-Abelian phases with Chern number . The other 8 white regions correspond to the Abelian phases with . A remarkable feature is that a critical curve or line (i.e. gapless phase) can separate two gapped phases with Chern numbers (i) 0 and , (ii) 0 and 0, or (iii) and . This reveals that a QPT can occur between two gapped phases belonging to the same topological class.

Below we explicitly display the QPTs (see Fig. 3). As in Ref. Uzunov, , here we classify the QPTs as two types: The first-order QPT where the first derivative of the ground-state energy with respect to the driving parameter is discontinuous at the transition point, and the continuous QPT where a higher-order derivative of is discontinuous and the derivative(s) with order(s) lower is (are) continuous. Figure 3(a) [3(b)] shows that there is a first-order QPT between two phases of the same Chern number at [, when one parameter varies along the horizontal dashed (dotted) line in Fig. 2(a). Using the perturbation method in Kitaev06, , the effective Hamiltonian at can be obtained, up to third order, as (see Appendix B)

where denotes the th unit cell, is a term containing none of , or , and the subscripts , and (, and ) denote the sites linked to plaquette () with -, - and -link, respectively. For the QPT between two phases of the same Chern number, the parameter changes its sign at the transition point . This gives rise to different ’s at the two sides of the transition point . Also, the difference between the two phases of the same Chern number but with positive and negative ’s can be seen from their wavefunctions (see Appendix A).

Using Feynmann theorem,JQYou () it can be derived that

(8) |

where the spin-spin correlation functions are defined by , with and for and . It is clear that the discontinuity of at a QPT point results from the discontinuity of there. Here we find that, at the QPT point, the nonlocal spin-spin correlation function on the -, -, or -link changes its sign, displaying discontinuity there. As shown in Fig. 3(a) [Fig. 3(b)], indeed has a jump at the QPT point [ when changes from positive to negative. It should be noted that only part of the spin-spin couplings in the Hamiltonian (LABEL:spinH) change their sign at these transition points, while the other spin-spin couplings are kept unchanged.

In addition to the first-order QPTs, continuous QPTs can also occur in the Kitaev spin model on a triangle-honeycomb lattice. As shown in Fig. 3 (a), and are continuous while becomes discontinuous at . This corresponds to a topological QPT between Abelian and non-Abelian phases. Figure 3(b) shows that such a continuous QPT can also happen at or . Similar to the first-order QPTs, the continuous QPTs result from the discontinuity of the second derivative of spin-spin correlation functions at the critical point.

Figure 3(c) displays three continuous QPTs. Interestingly, the QPT at , where diverges, occurs between two non-Abelian phases with Chern numbers and , respectively. This is in sharp contrast to the other two QPTs occurring at and , where is discontinuous and each transition is between Abelian and non-Abelian phases, instead of two non-Abelian phases. Also, it can be derived that

(9) |

where . This unveils that the nonanalyticity of results from the nonanalyticity of either or . Indeed, Fig. 3(c) shows that is divergent at . Such a QPT between two phases of Chern numbers can also occur when changing the sign of the magnetic-field-related parameter in the Kitaev spin model on a honeycomb lattice.Kitaev06 ()

## Iv Case B: , where , , and

Here we use parameters , and to characterize the phase diagram. The gapless-phase curves determined by are

(10) | |||||

Here , and . The gapless phase determined by each condition in Eq. (IV) is shown by a solid curve in Fig 2(b). These gapless-phase curves divide the plane into 9 regions: Two phases of Chern number , denoted by yellow (red) regions, and five gapped phases of Chern number , denoted by white regions. In this phase diagram, four gapped phases with and all terminate at a gapless-phase point or . Such a point is analogous to the eutectic point in crystallography. Also, a similar critical point exists in the phase diagram of the Haldane model.Haldane ()

Below we focus on the QPTs at the points where different gapped phases terminate. When varies along the dashed line in Fig. 2(b), a continuous QPT occurs at point , where diverges [see Fig. 4(a)]. Similar to the QPT between two non-Abelian phases belonging to the same topological class, this QPT involves two Abelian phases with the same Chern number . When varies along the dotted line in Fig. 2(b), in addition to the continuous QPTs between Abelian and non-Abelian phases (which occur at and , where are discontinuous), a continuous QPT between two non-Abelian phases happens at point , where diverges. In contrast to the QPT between two non-Abelian phases belonging to the same topological class, this transition involves two non-Abelian phases with . Analogous to the first-order QPTs occurring in the same topological class (Fig. 3), the continuous QPTs here are also due to the singularity of the nonlocal correlation functions at the critical points (see, e.g., the thick solid curves in Fig. 4).

## V Conclusion

We have studied QPTs in the Kitaev spin model on a triangle-honeycomb lattice and revealed the exotic ground-state phase diagram of this model. In addition to the ordinary topological QPTs between Abelian and non-Abelian phases, we find new QPTs that occur between two phases belonging to the same topological class. Moreover, we show that such QPTs are due to the singular behaviors of the nonlocal spin-spin correlation functions at the critical points.

###### Acknowledgements.

This work was supported by the National Basic Research Program of China under Grant No. 2009CB929300, and the National Natural Science Foundation of China under Grant No. 10625416.## Appendix A

### a.1 Derivation of the ground state in the uniform-flux sector

The Kitaev spin model on a triangle-honeycomb lattice is schematically shown in Fig. 5 and the model Hamiltonian is given by

To use the Jordan-Wigner transformation, we need to label the sites by row and column indices. We deform the honeycomb lattice [see Fig 1(a)] into the topologically equivalent one in Fig. 5. Now each site can be labeled by , where () and () are the row and column indices, respectively.

By performing the Jordan-Wigner transformationELieb ()

(12) |

on each spin, and using the Majorana fermions

(13) |

for sites , and

(14) |

for sites in each unit cell [see Fig. 5], the Hamiltonian (LABEL:SspinH) is converted to

(15) | |||||

where , , or , is the global flux operator calculated along a given contour encircling the system in the horizontal direction, denotes the in an up-pointing triangle, and the in a down-pointing triangle.

The global flux operator calculated along a given contour encircling the system in the vertical direction, which does not appear in the Hamiltonian (15), is given by

(16) | |||||

where with , or . These two global flux operators and commute with Hamiltonian (LABEL:SspinH). This leads to a topological degeneracy for the ground state because and can have eigenvalues either or . For the Abelian phase, this degeneracy is 4-fold because can be , , , and , while the degeneracy is 3-fold for the non-Abelian phase since is not allowed.HongYao () In addition to the global flux operators, the three types of local plaquette operators , and also commute with Hamiltonian (LABEL:SspinH). Using the Jordan-Wigner transform, they can be written as

(17) |

where , , and correspond, respectively, to the sites denoted by , , and in each unit cell. As shown in Ref. HongYao, , the ground state lies in the sector of the Hilbert space where either (i) and or (ii) and . In each of these two sectors, the time reversal symmetry of Hamiltonian (LABEL:SspinH) is broken. The former is called as the uniform-flux sector. This means that the ground state has another 2-fold degeneracy due to this broken time reversal symmetry, in addition to the topological degeneracy discussed above. Thus, we have 8-fold (6-fold) degeneracy for the ground state in the Abelian (non-Abelian) phase, corresponding to 8 (6) flux configurations for , and .

Here we focus on the ground state in the uniform-flux sector with , which allow both Abelian and non-Abelian phases.HongYao () Note that only one ground state exists in this sector. Now, Hamiltonian (15) is reduced to

(18) | |||||

By using the Fourier transform

(19) |

which satisfies

where denotes the first Brillouin zone, denote the six sites in each unit cell, and is the position of a unit cell, Eq. (18) becomes

(21) | |||||

where , and the two basis vectors of the unit cell are , and . To obtain the quasi-particle spectrum, we write Eq. (21) as

(22) |

where , and

(23) |

From the eigenvalue equation , one has

(24) |

where

(25) | |||||

with

(26) |

The solution of Eq. (24) reads