# Possible proximity of the Mott insulating Iridate Na2IrO3 to a topological phase: Phase diagram of the Heisenberg-Kitaev model in a magnetic field

## Abstract

Motivated by the recent experimental observation of a Mott insulating state for the layered Iridate NaIrO, we discuss possible ordering states of the effective Iridium moments in the presence of strong spin-orbit coupling and a magnetic field. For a field pointing in the direction – perpendicular to the hexagonal lattice formed by the Iridium moments – we find that a combination of Heisenberg and Kitaev exchange interactions gives rise to a rich phase diagram with both symmetry breaking magnetically ordered phases as well as a topologically ordered phase that is stable over a small range of coupling parameters. Our numerical simulations further indicate two exotic multicritical points at the boundaries between these ordered phases.

###### pacs:

71.20.Be, 75.25.Dk, 75.30.Et, 75.10.JmIn the realm of condensed matter physics, spin-orbit coupling has long been considered a residual, relativistic correction of minor relevance to the macroscopic properties of a material. In recent years this perspective has dramatically changed, especially due to the theoretical prediction and subsequent experimental observation of fundamentally new states of quantum matter, so-called topological insulators (1), that are solely due to the effect of spin-orbit coupling. The topological insulators experimentally realized so far are semiconductors, whose physical properties can be largely captured by band theory of non-interacting electrons. It is an interesting challenge, for both theory and experiment, to identify an even broader class of materials where this physics plays out even in the presence of interactions and strong correlations (2). Good candidate materials for the latter are the Iridates (3); (4). These transition metal oxides are prone to exhibit electronic correlations and form (weak) Mott insulators, while the relatively large mass of the Iridium ions () gives rise to a comparably strong spin-orbit coupling, which has been found to be as large as meV (5). The most common valence of the Iridium ions in these materials is Ir. The -orbitals of this configuration are typically split by the surrounding crystal field, and for the octahedral geometry of the IrO oxygen cage, result in an orbital configuration where five electrons occupy the lowered, threefold degenerate level. Spin-orbit coupling will further lift this degeneracy of the orbitals and for strong coupling the effective orbital angular momentum (6) is combined with the spin degree of freedom carried by the hole of this partially filled orbital configuration. This leaves us with two Kramers doublets of total angular momentum and , of which the former is of lower energy and fully occupied by four electrons, while the partial filling of the latter gives rise to an effective spin-1/2 degree of freedom.

In this manuscript we focus on the Iridate NaIrO, in which NaIrO slabs are stacked along the crystallographic -axis, and the Ir ions in the layers form a hexagonal lattice (4). Recent measurements of the magnetic susceptibility provide evidence of effective spin-1/2 moments and magnetic correlations below K indicating that NaIrO is indeed a Mott insulator (4). Theoretically, it has been argued (7); (8) that the interactions between the effective Iridium moments in the Mott regime are captured by a combination of isotropic and highly anisotropic exchanges, which can be tracked back to the spin and orbital components of the effective momenta. A microscopic Hamiltonian interpolating between these two types of exchanges is given by

(1) |

where the denote the effective spin-1/2 moment of the Ir ions, indicates the three different links of the hexagonal lattice, and parametrizes the relative coupling strength of the isotropic and anisotropic exchange between the moments. For the Hamiltonian reduces to the ordinary Heisenberg model, while in the opposite limit of highly anisotropic exchanges () the system corresponds to the Kitaev model (9). The latter is known to exhibit a gapless spin-liquid ground state (for equal coupling along the links) that can be gapped out into a topological phase with non-Abelian quasiparticle excitations by certain time-reversal symmetry breaking perturbations (9). One such perturbation is a magnetic field pointing in the direction, perpendicular to the honeycomb layer

(2) |

The main result of our manuscript is the rich phase diagram of this model, shown in Fig. 1. Besides two conventional, magnetically ordered phases we find a topologically ordered phase and two multicritical points, which we will discuss in detail in the remainder of the manuscript.

## Numerical simulations.–

We determine the ground-state phase diagram of Hamiltonian (2) by extensive ‘quasi-2D’ density-matrix renormalization group (DMRG) (10) calculations on systems with up to sites. In particular, we consider clusters of size , which are spanned by multiples and of the unit cell vectors and as illustrated in Fig. 2. It should be noted that the numerical analysis of Hamiltonian (2) is a challenging endeavor, since not only the entire Hilbert space needs to be considered (due to the lack of SU(2) invariance), but one also has to work with complex data types (due to the orientation of the magnetic field). Our DMRG calculations keep up to states, which is found to give excellent convergence with typical truncation errors of less than . We further use periodic boundary conditions in both lattice directions, which reduces finite-size effects. We have determined the phase boundaries in Fig. 1 by extensive scans of the ground-state energy, magnetization, and their derivatives in the ()-parameter space (11).

## Magnetically ordered states.–

We start our discussion of the phase diagram shown in Fig. 1 by first recapitulating previous results (8) for the Heisenberg-Kitaev model (1) in the absence of a magnetic field. Interpolating the relative coupling strength between the isotropic Heisenberg limit () and the highly anisotropic Kitaev limit () a sequence of three phases has been observed (8): The Néel ordered state of the Heisenberg limit is stable for , when it gives way to a ‘stripy’ Néel ordered state illustrated in Fig. 2 which covers the coupling regime . In the extended parameter regime the collective ground state is a gapless spin liquid. Near , perturbation theory reveals that the gapless excitations of this phase are emergent Majorana fermions forming two Dirac cones in momentum space.

Including a magnetic field in the direction a rich phase diagram evolves out of this sequence of three phases. For the magnetically ordered states we find that the orientation of the order in the Néel and stripy AFM phase cants along the direction. To further characterize these canted states, it is helpful to analyze the independent symmetries of Hamiltonian (2). Besides the lattice translational symmetry and a reflection symmetry around the centers of the hexagons, there is an additional symmetry, which is a combination of a three-fold rotation around an arbitrary lattice site and a three-fold spin rotation along the spin axis (12). Both canted phases break a subset of these discrete symmetries of the Hamiltonian. The canted Néel order breaks the and the symmetries, which thus leads to a six-fold ground-state degeneracy in this phase. The canted stripy phase breaks both the and translational symmetry (since the ordering pattern doubles the unit cell). As a consequence, we also find a six-fold ground-state degeneracy in this phase.

For sufficiently large magnetic field, the order of both canted phases is destroyed and they give way to a simple polarized state. Our numerical simulations strongly suggest that the transitions between the polarized state and these canted states are continuous, which is in agreement with their spontaneous symmetry breaking. On the other hand, the transition between the two canted states at finite field strength (indicated by the bold line in Fig. 1) is found to be first-order. In our simulations this is indicated by a sharp drop of the first derivative of the energy as a function of the coupling parameter across this transition – as shown in Fig. 3 for increasing strength of the magnetic field . Approaching the endpoint of this first-order line around this drop smoothly vanishes, which indicates that this endpoint possibly is a tricritical point at which the two canted magnetically ordered phases and the polarized phase meet. The existence of such a tricritical point can be understood within a Landau description with two distinct order parameters – corresponding to the discrete symmetry breaking of the two different magnetically ordered phases – opening a gap to magnon excitations.

## Topological phase.–

We now turn to the spin liquid phase found for coupling parameters . For the Kitaev limit () it has previously been argued (9) that an infinitesimal field along the direction will drive the system from the gapless spin liquid into a gapped non-Abelian topologically ordered phase. As we will discuss in the following our numerical simulations allow us to confirm the existence of such a topologically ordered state for small magnetic field strengths not only in the Kitaev limit, but for the full extent of the gapless spin liquid phase, as indicated in the phase diagram of Fig. 1. We will further present an independent and non-perturbative way to determine the topological nature of this phase. This complements the original argument by Kitaev (9), which was primarily based on a perturbation expansion showing that the leading order effect of a small magnetic field is to introduce a topological mass term for the Majorana fermions – however, such a perturbative argument should be carefully tested when applied to a gapless state. For this purpose, we consider an additional three-spin exchange term , indicated by the blue bonds in Fig. 2a), in our Hamiltonian

(3) |

In the Kitaev limit (, ) this Hamiltonian is exactly solvable in the same Majorana fermion representation used in the solution of the unperturbed Kitaev model (9). In particular, one can prove that the three-spin exchange breaks time-reversal symmetry and gaps out the spin liquid phase into a topologically ordered state with non-Abelian excitations, so-called Ising anyons. To demonstrate that a small magnetic field in the direction drives the system into the same phase, we have numerically calculated the phase diagram in the presence of both perturbations as shown in Fig. 4. The phase boundaries were again obtained by scanning the derivatives of ground-state energy and magnetization in the -parameter space. In particular, this phase diagram shows that one can adiabatically connect the phase for large and vanishing magnetic field with the phase for small, non-vanishing magnetic field and , thus proving that the magnetic field gaps out the spin liquid into the same non-Abelian topological phase stabilized by the three-spin exchange. The only feature in the diagram is a single phase transition line which separates the topologically ordered state from the fully polarized state expected for large magnetic field strengths. For the Kitaev limit () this transition occurs for . It is interesting to note that the critical field initially grows with increasing , but then saturates to some finite value around . Physically, this saturation can be understood by the behavior of the gap for the Majorana fermions in the exact solution for . The dispersion of the Majorana fermion is given by . For small , the Majorana fermion has a gap . However, for large the gap of Majorana fermion remains finite and independent from , given by . Since the magnetic field strength required to destroy the topological phase is determined by the Majorana fermion gap at , the critical field thus also increases and then saturates at large .

## Field-driven transition out of the topological phase.–

We now return to the phase diagram of the Heisenberg-Kitaev model in Fig. 1 and focus on the transition between the topologically ordered state and the polarized state for large field strength. For the Kitaev limit () this transition occurs at a critical field strength of and remains almost constant as the coupling parameter is decreased. Interestingly, our numerics suggest that this field-driven phase transition might be continuous or weakly first-order. In particular, we find that the second-derivative of the ground state energy at this transition diverges with increasing system size, while the magnetization does not show any discontinuity, as shown in Fig. 5a) and b), respectively.

While the limited system sizes in our study do not allow to unambiguously determine the continuous nature of this field-driven phase transition, our numerics nevertheless provide some further insights what might cause such a continuous transition (13). To this end, we plot the number of vortices in the ground state as a function of magnetic field, i.e. the number of plaquettes with a non-trivial flux, in Fig. 5c). Below the critical magnetic field, i.e. , there are no vortices indicating a deconfined phase as expected in the presence of a vortex gap. At the phase transition, however, the vortices appear to condense and the number of vortices in the ground state quickly increase above the critical field strength. The nature of the phase transition might thus be framed in terms of a confinement-deconfinement transition of a non-Abelian gauge field, akin to the confinement-deconfinement transition in the Abelian discrete gauge theory (14). An example of the latter is the gauge theory, e.g. the toric code in a magnetic field (15), for which it is well known that flux condensation leads to a confinement transition (16).

## A second multicritical point.–

Finally, we note that there appears to be a second multicritical point in our phase diagram around and , where the stripy AFM phase and the gapless spin liquid meet. We find that in the presence of the magnetic field the transition lines of the field-driven phase transition out of the corresponding canted and topologically ordered states bend in and merge only in the zero-field limit as depicted in the inset of Fig. 1. To show that there is indeed no direct transition between the canted stripy Néel state and the topological phase we have made extensive scans in the coupling parameter in the vicinity of this putative multicritical point for small field strength. As shown in Fig. 6 for , the second derivative of the ground-state energy clearly shows two peaks proliferating with increasing system size indicative of two well separated phase transitions. However, the underlying effective theory for such a multicritical point is not known, and will be left for further study.

## Outlook.–

Having established the rich phase diagram of the Heisenberg-Kitaev model in a magnetic field, it is interesting to speculate where one would place the Iridate NaIrO. While experiments (4) report indications of an AFM ordered ground state below K, the precise nature of the order remains open. Given the considerable suppression of the ordering temperature in comparison with the Curie-Weiss temperature K (4), which is typically interpreted as an indicator of frustration, an alternative explanation would be the proximity to a quantum critical point, such as the multicritical point in the context of our phase diagram. This would bring the material in close proximity to the spin liquid phase for and the topological phase found for a magnetic field pointing in the direction. To further substantiate this possibility, it is desirable to study the finite-temperature phase diagram of our model system and to consider the effects of disorder, such as site mixing between the Ir and Na sites (4). Finally, it would be interesting to bring the Mott physics discussed in this manuscript in competition with the topological insulator phase suggested in Ref. (17).

###### Acknowledgements.

We acknowledge discussions with G. Jackeli, R. Kaul, D. N. Sheng, and R. Thomale. XLQ is supported partly by the Alfred P. Sloan Foundation.### References

- See e.g. J. E. Moore, Nature 464, 194 (2010); M. Z. Hasan and C. L. Kane, Rev. Mod. Phys. 82, 3045 (2010); X.-L. Qi and S. C. Zhang, arXiv:1008.2026.
- S. Raghu et al., Phys. Rev. Lett. 100, 156401 (2008); H. M. Guo and M. Franz, Phys. Rev. Lett. 103, 206805 (2009); D. Pesin and L. Balents, Nat. Phys. 6, 376 (2010); B.-J. Yang and Y. B. Kim, arXiv:1004.4630 (2010); X. Wan et al., arXiv:1007.0016 (2010); M. Dzero et al., Phys. Rev. Lett. 104, 106408 (2010); M. Kargarian et al., arXiv:1101.0007 (2011).
- Y. Okamoto et al., Phys. Rev. Lett. 99, 137207 (2007). B. J. Kim et al., Phys. Rev. Lett. 101, 076402 (2008); B. J. Kim et al., Science 323, 1329 (2009); H. Jin et al., arXiv:0907.0743.
- Y. Singh and P. Gegenwart, Phys. Rev. B 82, 064412 (2010).
- O. F. Schrimer et al., J. Phys. C 17, 1321 (1984).
- A. Abragam and B. Bleaney, Electron Paramagnetic Resonance of Transition Ions (Clarendon Press, Oxford, 1970).
- G. Jackeli and G. Khaliullin, Phys. Rev. Lett. 102, 017205 (2009).
- J. Chaloupka, G. Jackeli, and G. Khaliullin, Phys. Rev. Lett. 105, 027204 (2010).
- A. Kitaev, Ann. Phys. 321, 2 (2006).
- S. R. White, Phys. Rev. Lett. 69, 2863 (1992).
- Some of our calculations were supplemented by TRG calculations developed in H. C. Jiang, Z. Y. Weng, and T. Xiang, Phys. Rev. Lett. 101, 090603 (2008); Z. C. Gu, M. Levin, and X. G. Wen, Phys. Rev. B 78, 205116 (2008).
- The three-fold spin rotation along the spin axis corresponds to the cyclic symmetry .
- See also C. Gils et al., Nat. Phys. 5, 834 (2009).
- Although the field-driven phase transition might share many similarities for topological states with Abelian and non-Abelian topological order, there also exist important differences. For the Abelian gauge theory, it has been shown that the critical field strength is of the same magnitude as the single vortex gap and as a result, the closing of the gap for a single vortex will naturally lead to vortex condensation. In the non-Abelian case, however, the single vortex gap, estimated around , is considerably larger than the critical field strength. There are two potential reasons for this discrepancy in the non-Abelian case: First, the non-Abelian nature of the vortices gives rise to a macroscopic degeneracy which might in turn enhance the low energy quantum fluctuations. Second, in the limit of high vortex density, the vortex core energy can be much smaller than in the single vortex case and thus further enhance the quantum fluctuations.
- S. Trebst et al., Phys. Rev. Lett. 98, 070602 (2007); I. S. Tupitsyn et al., Phys. Rev. B 82, 085114 (2010).
- E. Fradkin and S. Shenker, Phys. Rev. D 19, 3682 (1979).
- A. Shitade et al., Phys. Rev. Lett. 102, 256403 (2009).