# Quantum Anomalous Hall Effect in Magnetic Topological Insulator GdBiTe

## Abstract

The quantum anomalous Hall (QAH) state is a two-dimensional bulk insulator with a non-zero Chern number in absence of external magnetic fields. Protected gapless chiral edge states enable dissipationless current transport in electronic devices. Doping topological insulators with random magnetic impurities could realize the QAH state, but magnetic order is difficult to establish experimentally in the bulk insulating limit. Here we predict that the single quintuple layer of GdBiTe film could be a stoichiometric QAH insulator based on ab-initio calculations, which explicitly demonstrate ferromagnetic order and chiral edge states inside the bulk gap. We further investigate the topological quantum phase transition by tuning the lattice constant and interactions. A simple low-energy effective model is presented to capture the salient physical feature of this topological material.

Department of Physics, McCullough Building, Stanford University, Stanford, CA 94305-4045

Recently time-reversal-invariant topological insulators(TIs) have attracted broad attention in fields of the condensed matter physics, material science and electrical engineering[1, 2, 3, 4]. Quantum spin Hall(QSH) insulators[5, 6] are two dimensions(2D) TIs with a bulk energy gap and gapless helical edge states which are protected by the topological invariant[7]. Both 2D and 3D TIs are natural platforms to realize the QAH when the time-reversal symmetry is spontaneously broken by the ferromagnetic order[8, 9, 10, 11]. The QAH state is the quantum version of the intrinsic anomalous Hall effect [12], and is characterized by a bulk 2D Chern number and gapless chiral edge states without the external magnetic field[13, 8, 14], and is sometimes also referred to as the 2D Chern insulator. The QSH state has two counter-propagating edge states; doping the 2D TIs with magnetic impurities could break the time reversal symmetry, annihilate one edge state and leave the other intact, thus realizing the QAH state with chiral edge state. Alternatively, doping the surface of 3D TIs with magnetic impurities breaks time reversal symmetry, and opens up a gap for the Dirac surface states[11, 15, 16]. Chiral edge states exist on the magnetic domain walls on the surface[11]. In the QAH state, electrons move like cars on a highway, where oppositely moving traffics are spatially separated into opposite lanes[1]. Realizing such a dissipationless transport mechanism without extreme condition could greatly improve the performance of electronic devices.

Normally, ferromagnetic order among magnetic impurities in randomly doped semiconductors are mediated by the free carriers [17], it is difficult to establish magnetic order in the bulk insulating limit. For this reason, we search for stoichiometric ferromagnetic insulators with a nonzero Chern number for the 2D band structure. Rare earth elements have partially filled f electrons, in particular, Gd has exact half filled f electrons. It is possible to realize a stoichiometric QAH insulator by replacing Bi by Gd in the BiTe family of TIs[18, 19, 20, 21]. GdBiTe has a stable structure phase, synthesized thirty years ago[22]. In this work, we predict that stoichiometric QAH insulators could be realized in the single quintuple layer(SQL) of GdBiTe, based on ab-initio calculations.

Crystal structure and band structure

GdBiTe has a rhombohedral crystal structure with the space group . Its crystal consists of close-packed layers stacked along [111] direction with the A-B-C order. The quintuple layer (QL) (Te-X-Te-X-Te) structure is the basic crystal unit, where X presents Bi and Gd with a certain pattern, similar to the case of LaBiTe[21]. It has strong coupling within one QL and weak van der Waals coupling between neighbor QLs, so this material can be grown as two-dimensional (2D) thin film along [111] direction. In addition, though its lattice type and lattice constant have been measured experimentally[22], positions of Bi and Gd have not been clearly resolved yet. Since Bi and Gd have different electronegativity, it is possible to grow the non-mixing structure as Te-Gd-Te-Bi-Te with Molecular beam epitaxy (MBE) technique. This crystal structure of GdBiTe3 is shown in Fig. 1a. It holds the three-fold rotation symmetry with axis as the trigonal axis and the reflection symmetry with axis as its normal axis. Compared with BiTe[18], the inversion symmetry is broken for this structure. Since SQL GdBiTe film is the simplest system to study the QAH effect, in this work we focus on its SQL structure with Te1-Bi-Te2-Gd-Te1, where Te1 and Te1 are marked for the Te closest to Bi and Gd layer separately, and Te2 denotes the Te in the center of this SQL.

All ab-initio calculations are carried out in the framework of density functional theory (DFT) with Perdew-Burke-Ernzerbof-type generalized-gradient approximation[23]. Both BSTATE package[24] with plane-wave pseudo-potential method and the Vienna ab initio simulation package (VASP) with the projected augmented wave method are employed. The k-point grid is taken as , and the kinetic energy cutoff is fixed to 340eV in all self-consistent calculations. A free standing slab model is employed with SQL. Its lattice constant () is taken from experiments[22], and the inner atomic positions are obtained through the ionic relaxation with the force cutoff 0.001eV/Å. The spin-orbit coupling (SOC) is taken into account, because of its importance to realize the QAH effect. GGA+U method[25] is also employed to study the strong correlation effect in GdBiTe because of the existence of narrow occupied f bands of Gd. We find that the ferromagnetic phase is more stable than the non-magnetic and collinear anti-ferromagnetic phases for SQL GdBiTe. In addition, the ferromagnetic phase with magnetic moment along [111] direction has lower energy than that with the magnetic moment along [010] and [110] directions. All GdBiTe calculations are carried out here with the ferromagnetic moments along [111] direction.

The ferromagnetic phase of SGL GdBiTe is an insulator state with exact magnetic moment . Because of the lattice similarity between GdBiTe and GdN(or EuO) with [111] direction, the known magnetic exchange mechanisms in GdN or EuO could provide possible explanation for the ferromagnetic state in SQL GdBiTe. The first mechanism is a third-order perturbation process[26]. A virtual excitation, which takes a 4f to a 5d state, leads to a f-f interaction through the d-f exchange due to the wave-function overlap between neighboring rare-earth atoms. Recently, Mitra and Lambrecht[27], based on ab-initio calculations, presented another magnetic mechanism for the ferromagnetic ground state in GdN. The anti-ferromagnetic ordering, between N p and Gd d small magnetic moments, stabilizes the ferromagnetic structure between nearest neighbor Gd atoms due to the d-f exchange interaction. In addition, our calculations also indicate the similar anti-ferromagnetic ordering between Te p and Gd d small magnetic moments.

BiTe, LaBiTe and GdBiTe have very similar SQL structure. First of all, BiTe have both the inversion symmetry and the time-reversal symmetry. However, inversion symmetry is broken in LaBiTe, and both inversion symmetry and time-reversal symmetry are broken in GdBiTe. The band evolution from BiTe to LaBiTe, and finally to GdBiTe is shown in Fig. 1c-e. The bands of BiTe have double degeneracy because of both inversion and time-reversal symmetries. Its energy gap is calculated to be about , consistent with the experiments[28]. The bottom of the conduction bands at point with a “V” shape originates from the orbitals of Bi, Te1 and Te1. The top of the valence band at the point, which lies below valence bands at other momenta, originates from the orbitals of Te2. For LaBiTe, the double degeneracy of the bands is lifted except at the time-reversal points, due to the lack of the inversion symmetry. Though the bottom of conduction bands still shows the “V” shape, the top of the valence bands moves to , which mainly originates from the orbitals of Te2 and Te1. The schematic evolution of the top of valence bands and the bottom of conduction bands from LaBiTe to GdBiTe is shown in Fig. 2a-b. The Kramers double degeneracy at the point in GdBiTe is broken due to the lack of time-reversal symmetry. The band inversion occurs between the and due to the strong SOC and the large magnetic moment. The existence of this band inversion is the key point to realize the QAH effect in this SQL GdBiTe system.

Fig. 3a shows that the energy gap is quite sensitive with the lattice constant. Smaller lattice constant increases the band inversion. Conversely, increasing the lattice constant would destroy the band inversion. The QAH effect exists for , where is the experimentally determined bulk lattice constant. Because the SQL GdBiTe is very thin, its lattice constant could be controlled by the substrate, giving us a broad tunability range. In addition, due to the narrow f bands, the strong correlation needs to be checked by GGA+U calculations. First of all, the band structure calculations show that the occupied f bands are located at below Fermi level (FL), and the unoccupied f bands are located at above FL. Since both occupied and unoccupied f bands are quite far from FL, correlation effects should not influence bands very close to the Fermi level. Our calculations indicate that this conclusion is true. The dependence of the energy gap on U (from to ) is shown in Fig. 3b. The correlation U lifts up the top spin-down valence band, and pulls down the bottom spin-up valence band. Therefore, the strong correlation U likes to pull GdBiTe from the topologically non-trivial QAH phase to the topologically trivial ferromagnetic insulator phase. But the energy gap is much more sensitive with the lattice constant than with the correlation U. We have shown that the SQL GdBiTe is close to the quantum critical point between the QAH state and topologically trivial magnetic insulator state, and it is possible to realize the QAH state by tuning the lattice constant with proper substrates. Also the 2D Dirac-type band dispersion shows up at the critical point, shown in Fig. 3c. This is very similar to the case of QSH state at the critical point[5].

Topological chiral edge states

The existence of topologically protected chiral edge states is the direct and intrinsic evidence of the QAH phase. It is important to show the explicit features of these topologically protected chiral edge states. In order to calculate the edge states, we employ the tight-binding method based on maximally localized Wannier functions (MLWF), developed by Vanderbilt and his co-workers[29], in the framework of ab-initio calculations. The edge with Te1 and Bi terminated along the [11] direction is chosen to show the edge states, shown in the inset of Fig. 4b. Because the topological nature is completely determined by its bulk electronic structure, here we ignore the edge reconstruction of the atoms on the edge, and also make another approximation that the MLWF hopping parameters close to the edge are the same to ones of the bulk. The bulk MLWF hopping parameters are obtained from the ab-initio calculations of the SQL free-standing GdBiTe slab. Similar to the method described in our previous work[18], we use an iterative method[30] to obtain the edge Green’s function of the semi-infinite system with an edge along [11] direction. The local density of states (LDOS) can be calculated with the imaginary part of these edge Green’s functions, shown in Fig. 4a,b. The LDOS in Fig. 4a shows two edge states for topologically non-trivial GdBiTe with the experimental lattice constant . One topologically trivial edge state only stays within the valence bands, while the topologically non-trivial edge state ties the conduction band with the valence band. This topologically non-trivial edge state is chiral, and its Fermi velocity in the energy gap is (or ). The edge states of topologically trivial ferromagnetic GdBiTe with the lattice constant are shown in Fig. 4b. Both of these two topologically trivial edge states only stay within the valence bands, and do not connect the valence and conduction bands.

Low-energy effective model

As the topological nature is determined by the physics near the point for this material, it is possible to write down a four-band effective Hamiltonian to characterize the low-energy long-wavelength properties of the system. Starting from the four low-lying states , , and at the point, where , , and are constants and are the angular momenta in direction. In addition, the “” subscripts are used to present the component of orbital angular momentum . Such a Hamiltonian can be constructed by the theory of invariants for the finite wave vector k. On the basis of the symmetries of the system, the generic form of the effective Hamiltonian can be constructed up to the order of , and the parameters of the Hamiltonian can be obtained by fitting to ab initio calculations. First we consider the Te-La-Te-Bi-Te system without magnetic order. The important symmetries of this system are (1) time-reversal symmetry , (2) reflection symmetry with axis as the normal axis, and (3) three-fold rotation symmetry along the axis. In the basis of , , and , the representation of the symmetry operations is given by , and , where is the complex conjugation operator, and denote the Pauli matrices in the spin and orbital space, respectively and is the angular momentum in direction. For low energy effective model, we ignore the in-plane anisotropy and impose continuous rotational symmetry . By requiring these three symmetries, we obtain the following generic form of the effective Hamiltonian:

(1) |

with , , . The block diagonal part is just the BHZ model[5], and the off-block diagonal terms breaks the inversion symmetry. Next we add the ferromagnetic exchange term which breaks time reversal symmetry,

(2) |

with and as Land- factors for two different orbitals and the exchange field introduced by the FM ordering of Gd with half-occupied f electrons. Then the full Hamiltonian can be written as . By fitting the energy spectrum of the effective Hamiltonian with that of the ab initio calculations, the parameters in the effective model can be determined. For GdBiTe, our fitting leads to , , , , , , , and , which agree well with the ab initio results. Such an effective model can be used for illustration of the formation of the GdBiTe QAH system. As shown in Fig. 2c-e, when we have no FM ordering, the fact that , suggests it is the topologically trivial insulator as SQL LaBiTe, and the lack of inversion symmetry splits the double degeneracy except at point. As we adiabatically increase the exchange field , the and states start to move towards each other. At the transition point, those two bands form a 2D Dirac cone, and as we further increase the exchange field, those two bands become inverted and the term hybridizes them and creates a band gap again. In this process, a topological phase transition of SQL GdBiTe is shown clearly from a topologically trivial phase to the QAH phase.

Our theoretical calculations show that a SQL of GdBiTe is a 2D stoichiometric magnetic topological insulator, realizing the long-sought-after QAH state. This topological material can be grown by MBE, or by exfoliating the bulk crystal. In addition, YBiTe with the same structure is found to be a normal insulator[21], and could serve as the best substrate for MBE growth. Experimentally, the best way to see the QAH effect is to measure the four-terminal Hall conductance as a function of gate voltage. A quantized plateau in Hall conductance should be observed when the chemical potential is inside the gap. These intrinsic QAH materials could be used for applications with dissipationless electronic transport.

We are indebted to B.H. Yan at University of Bremen, X.L. Qi and Q.F. Zhang at Stanford University for their great help. We would like to thank Y.L. Chen and D.S. Kong for their useful discussion. This work is supported by the Army Research Office (No.W911NF-09-1-0508) and the Keck Foundation.

The authors declare that they have no competing financial interests.

Correspondence and requests for materials should be addressed to Shou-Cheng Zhang (email:sczhang@stanford.edu).

### References

- Qi, X.-L. & Zhang, S.-C. The quantum spin Hall effect and topological insulators. Physics Today 63, 33 (2010).
- Hasan, M. Z. & Kane, C. L. Colloquium: Topological insulators. Rev. Mod. Phys. 82, 3045–3067 (2010).
- Moore, J. E. The birth of topological insulators. Nature 464, 194–198 (2010).
- Qi, X. L. & Zhang, S. C. Topological insulators and superconductors. arXiv: 1008.2026v1 (2010).
- Bernevig,B. A. , Hughes, T. L. & Zhang, S. C. Quantum spin Hall effect and topological phase transition in HgTe quantum wells. Science 314, 1757 (2006).
- König, M. et al. Quantum spin Hall insulator state in HgTe quantum wells. Science 318, 766–770 (2007).
- Kane, C. L. & Mele, E. J. Z topological order and the quantum spin Hall effect. Phys. Rev. Lett. 95, 146802 (2005).
- Qi, X. L., Wu, Y. S. & Zhang,S. C. Topological quantization of the spin Hall effect in two-dimensional paramagnetic semiconductors. Phys. Rev. B 74, 045125 (2006).
- Liu, C.-X., Qi, X.-L., Dai, X., Fang, Z. & Zhang, S.-C. Quantum anomalous Hall effect in HgMnTe quantum wells. Phys. Rev. Lett. 101, 146802 (2008).
- Yu, R. et al. Quantized anomalous Hall effect in magnetic topological insulators. Science 329, 61–64 (2010).
- Qi, X.-L., Hughes, T. L. & Zhang, S.-C. Topological field theory of time-reversal invariant insulators. Phys. Rev. B 78, 195424–43 (2008).
- Nagaosa, N., Sinova, J., Onoda, S., MacDonald, A. H. & Ong, N. P. Anomalous Hall effect. Rev. Mod. Phys. 82, 1539–1592 (2010).
- Haldane, F. D. M. Model for a quantum Hall effect without landau levels: Condensed-matter realization of the ”parity anomaly”. Phys. Rev. Lett. 61, 2015–2018 (1988).
- Onoda, M. & Nagaosa, N. Quantized anomalous Hall effect in two-dimensional ferromagnets: Quantum Hall effect in metals. Phys. Rev. Lett. 90, 206601 (2003).
- Liu, Q., Liu, C.-X., Xu, C., Qi, X.-L. & Zhang, S.-C. Magnetic impurities on the surface of a topological insulator. Phys. Rev. Lett. 102, 156603 (2009).
- Chen, Y. L. et al. Massive dirac fermion on the surface of a magnetically doped topological insulator. Science 329, 659–662 (2010).
- Dietl, T., Ohno, H., Matsukura, F., Cibert, J. & Ferrand, D. Zener model description of ferromagnetism in zinc-blende magnetic semiconductors. Science 287, 1019–1022 (2000).
- Zhang, H. et al. Topological insulators in BiSe, BiTe and SbTe with a single dirac cone on the surface. Nature Physics 5, 438–442 (2009).
- Xia, Y. et al. Observation of a large-gap topological-insulator class with a single dirac cone on the surface. Nature Physics 5, 398–402 (2009).
- Chen, Y. L. et al. Experimental realization of a three-dimensional topological insulator, BiTe. Science 325, 178–181 (2009).
- Yan, B. et al. Theoretical prediction of topological insulator in ternary rare earth chalcogenides. Phys. Rev. B 82, 161108 (2010).
- Madelung, O., Rossler, U. & M., S. GdBiTe crystal structure. Landolt-Bornstein - Group III Condensed Matter - Numerical Data and Functional Relationships in Science and Technology: 41E: Ternary Compounds, Organic Semiconductors .
- Perdew, J. P., Burke, K. & Ernzerhof, M. Generalized gradient approximation made simple. Phys. Rev. Lett. 77, 3865 (1996).
- Fang, Z. & Terakura, K. Structural distortion and magnetism in transition metal oxides: crucial roles of orbital degrees of freedom. Journal of Physics: Condensed Matter 14, 3001–3014 (2002).
- Anisimov, V. I., Solovyev, I. V., Korotin, M. A., Czyżyk, M. T. & Sawatzky, G. A. Density-functional theory and nio photoemission spectra. Phys. Rev. B 48, 16929–16934 (1993).
- Mauger, A. & Godart, C. The magnetic, optical, and transport properties of representatives of a class of magnetic semiconductors: The europium chalcogenides. Physics Reports 141, 51 – 176 (1986).
- Mitra, C. & Lambrecht, W. R. L. Magnetic exchange interactions in the gadolinium pnictides from first principles. Phys. Rev. B 78, 134421 (2008).
- Li, Y.-Y. et al. Intrinsic topological insulator BiTe thin films on Si and their thickness limit. Advanced Materials 22, 4002–4007 (2010).
- Marzari, N. & Vanderbilt, D. Maximally localized generalized wannier functions for composite energy bands. Phys. Rev. B 56, 12847 (1997).
- Sancho, M. P. L., Sancho, J. M. L., Sancho, J. M. L. & Rubio, J. Highly convergent schemes for the calculation of bulk and surface green functions. Journal of Physics F: Metal Physics 15, 851–858 (1985).