# Loss of Hall Conductivity Quantization in a Non-Hermitian Quantum Anomalous Hall Insulator

## Abstract

Recent work has extended topological band theory to open, non-Hermitian Hamiltonians, yet little is understood about how non-Hermiticity alters the topological quantization of associated observables. We address this problem by studying the quantum anomalous Hall effect (QAHE) generated in the Dirac surface states of a 3D time-reversal-invariant topological insulator (TI) that is proximity-coupled to a metallic ferromagnet. By constructing a contact self-energy for the ferromagnet, we show that in addition to generating a mass gap in the surface spectrum, the ferromagnet can introduce a non-Hermitian broadening term, which can obscure the mass gap in the spectral function. We calculate the Hall conductivity for the effective non-Hermitian Hamiltonian describing the heterostructure and show that it is no longer quantized despite being classified as a Chern insulator based on non-Hermitian topological band theory. Our results indicate that the QAHE will be challenging to experimentally observe in ferromagnet-TI heterostructures due to the finite lifetime of quasi-particles at the interface.

Introduction.—The last decade has seen a revolution in the understanding of the electronic structure of solids with the formulation and development of topological band theory, which provides a unified system to classify materials ranging from insulators and semi-metals to superconductors using topological invariants Hasan and Kane (2010); Qi and Zhang (2011); Bernevig (2013). These quantized topological invariants provide a robust classification for materials, as they cannot be changed by adiabatic deformations of the systems. An important consequence of a non-trivial topological classification is that some response of the system to an external stimulus is also quantized proportional to its topological invariant. One well-known example of this quantization is in the integer quantum hall effect (IQHE) or the quantum anomalous Hall effect (QAHE), in which the Hall conductivity is given as

(1) |

where is the electron charge, is Planck’s constant, and is the sum of the TKNN invariants or Chern numbers of occupied bands Klitzing et al. (1980); Haldane (1988). Because of the topological quantization of , the Hall response is remarkably robust to perturbations and the presence of disorder, allowing for experimental measurements of the IQHE accurate to a few parts in of the theoretically-predicted, quantized value Jeckelmann et al. (1997). Despite this success in predicting the quantization of the Hall conductivity, topological band theory is formulated for closed, Hermitian Hamiltonians, and it is, therefore, unclear if and how open systems can be topologically classified.

To address this issue, recent studies have extended topological band theory to characterize non-Hermitian Hamiltonians Lieu (2018); Cerjan et al. (2018); Zyuzin and Zyuzin (2018); Zhou et al. (2018); Shen et al. (2018), which arise in systems that are opened to external reservoirs or interactions with other particles. Notably, non-Hermitian Hamiltonians can host topological phases and invariants that cannot be seen in Hermitian systems, resulting in unusual predictions such as bulk Fermi arcs in 2D systems Papaj et al. (2018); Zhou et al. (2018). Despite this progress in the understanding of non-Hermitian systems and their topological classifications, the effect of non-Hermiticity on the quantization of physical observables is not well-understood.

In this work, we explore the consequences of non-Hermiticity on physical observables by quantitatively studying the QAHE generated in the Dirac surface states of a 3D time-reversal-invariant topological insulator (TI) when proximity-coupled to a metallic ferromagnet. In addition to a time-reversal-breaking mass gap generated in the Dirac surface spectrum by the ferromagnet, we see that the presence of metallic bands at the Dirac point give the surface states a finite lifetime, as electrons can escape into the ferromagnet. This finite lifetime results in broadening of the states that is comparable in magnitude to the mass gap, which in turn, results in a gapless spectral function. To characterize the impact of the this broadening on the QAHE, we calculate the Hall conductivity of this system via the Kubo-Streda formula. Non-Hermitian topological band theory suggests that such a system retains its classification as a Chern insulator, but we find that the Hall conductivity is no longer quantized as in Eq. (1). We compare the proximity-coupled case to one where the mass gap is generated by bulk magnetic dopants and find that the broadening due to magnetic impurity scattering is much smaller than the mass gap, thus allowing for the observation of the QAHE in these systems. Our results show that the non-Hermiticity introduced in open topological systems causes the loss of topological quantization of observables and can severely limit the ability to experimentally observe such responses.

Model Hamiltonian and Ferromagnet Contact Self-Energy.—Figure 1(a) depicts a schematic for the TI-ferromagnet heterostructure we study. The low-energy, effective Hamiltonian for the surface states of a 3D time-reversal-invariant TI is given by the 2D Dirac equation

(2) |

where is the momentum of the electron, , is the Fermi velocity of the surface electrons, and are the spin Pauli matrices. We model the ferromagnet with a tight-binding Hamiltonian with nearest-neighbor hopping in the direction given by

(3) |

where is the creation (annihilation) operator for an electron with in-plane momentum and position , is the on-site term, is the in-plane dispersion of the metallic bands of the ferromagnet, is the spin-splitting energy within the ferromagnet, and is the hopping matrix in the direction. In Supplemental Material Sec. I, we calculate the contact self-energy that fully captures the influence of a semi-infinite ferromagnetic metal coupled to the surface state Hamiltonian in the direction with coupling matrix . The resulting expression for the contact self-energy is

(4) |

where the diagonal components are given as

(5) |

and the upper (lower) sign corresponds to up (down) spin.

To understand the impact of metallic bands at the Dirac point, we focus on the case where the bands of the ferromagnet are centered around such that, at low momenta, . In addition, we impose the constraint to ensure that the spin up and spin down bands do not completely separate in energy to create a ferromagnetic insulator. Within this regime, we can understand the influence of the metallic ferromagnet by studying the low-energy limit of the contact self-energy in Eq. (4)-(5):

(6) |

Utilizing this approximation, we create an effective Hamiltonian that describes the TI surface states in the presence of a proximity-coupled ferromagnet as

(7) |

where and . As expected, the proximity-coupled ferromagnet introduces a time-reversal breaking term proportional to the exchange interaction strength in the ferromagnet. In addition, the presence of the metallic bands from the ferromagnet introduces a non-Hermitian broadening term that gives the surface states a finite-lifetime as surface state electrons can escape into the ferromagnet. Broadening is a common consequence of a integrating out the effect of an interaction or coupling to an external reservoir, but this self-energy is notable in that the broadening, , can exceed the mass gap, , when . This implies that the spectral function, given as , where is the Green function of the system and , can be gapless, as is demonstrated in Fig. 1(a), despite the fact that a mass gap has been generated in the surface spectrum.

When we inspect the the energy eigenvalues, given as and plotted with in Fig. 1(b), we see that the real part is gapped and exactly that of a massive Dirac electron. The non-Hermitian broadening simply shifts these eigenvalues by but does not close the gap in the complex energy spectrum. Since the bands remain separable with non-zero and the eigenvectors are unchanged from the Hermitian Hamiltonian, this non-Hermitian Hamiltonian is adiabatically connected to the Hermitian Chern insulator with Shen et al. (2018). When the Fermi energy is within the mass gap of a Hermitian Chern insulator, we anticipate that the Hall conductivity should be quantized to be for positive values of as in Eq. (1). Since the non-Hermitian broadening introduced by the ferromagnet contact self-energy can be large enough that the gap in the spectral function is closed, it is not immediately obvious if the Hall conductivity continues to be exactly quantized for non-Hermitian Chern insulators.

Hall Conductivity.—To quantify the impact of non-Hermiticity on the quantization of the topological observable in this system, we explicitly compute the DC Hall conductivity of the non-Hermitian Hamiltonian in Eq. (7) using the Kubo-Streda formula Streda (1982), which at zero temperature is given as

(8) |

where

(9) | ||||

(10) | ||||

(11) |

Here is the volume of space, is the velocity operator in the direction, and is the Fermi energy. For the effective Hamiltonian in Eq. (7), the velocity operators are and .

The first two terms in this formulation, and , are Fermi surface contributions and are only non-zero when the Fermi energy crosses an energy band. The first term, , includes the intrinsic Berry phase component of the anomalous Hall conductivity in addition to extrinsic effects due to the presence of scattering mechanisms such as side-jump and skew scattering Sinitsyn et al. (2007); Nunner et al. (2007); Nagaosa et al. (2010). The second term, , is identically zero for the Dirac Hamiltonian (See Supplemental Material Sec. II and Ref. Nunner et al. (2007)). The third term, , corresponds to the contribution to the Hall conductivity that is due to the Fermi sea since the integration over energy can, in principle, include contributions from all occupied states. This contribution is quantized when the Fermi energy is within the mass gap and gives rise to the QAHE when a Chern insulating band is fully occupied and .

When is finite, the Fermi surface and Fermi sea contributions to the Hall conductivity take the closed form:

(12) | ||||

(13) |

(See Supplemental Material Sec. II). Figure 2 shows the Hall conductivity for the effective non-Hermitian Hamiltonian as a function of Fermi level, , at three different values of . When and when the Fermi energy is within the mass gap, the components of the Hall conductivity take the expected form: is identically zero, while is exactly quantized to Sinitsyn et al. (2007); Nunner et al. (2007). When is non-zero, however, both and are non-zero within and above the mass gap. When , despite the the fact that and , the total Hall conductivity, , appears to remain quantized within the mass gap. This behavior can be understood by expanding the expressions in Eqs. (12)-(13) in powers of . In Supplemental Material Sec. III, we show that the correction of to first-order in exactly cancels that of . Therefore, the leading-order correction to the quantized Hall conductivity within the mass gap is cubic in :

(14) |

Thus, when , the total Hall conductivity deviates negligibly from the quantized value.

When is comparable in magnitude to the mass gap, however, higher-order corrections are large enough to significantly decrease the total Hall conductivity from the quantized value. Thus, broadening can generate a distinct non-quantization of the Hall conductivity, in stark contrast to the robustness associated with Hermitian topological systems. In fact, we see from the expressions for the Hall conductivity in Eqs. (12)-(13) that any non-zero value of breaks the quantization of the Hall conductivity.

Discussion.—Our characterization of the impact of non-Hermiticity on the quantization of the Hall conductivity in Chern insulating systems has clear ramifications for experimental observation of such phenomena. Because the non-Hermitian broadening introduced by the ferromagnet contact self-energy in Eqs. (4)-(5) is comparable to the mass gap in the ferromagnet-TI heterostructure, the Hall conductivity deviates significantly from the quantized value predicted from the topological classification of the system. Although our analysis was limited to the specific case of a ferromagnetic metal with bands centered on the Dirac point, we note that large levels of broadening can arise even in heterostructures with ferromagnetic insulators when the Dirac point lies within the band gap of the ferromagnet. Figure 3 depicts the band diagram of a heterostructure of the TI BiSe and the ferromagnetic insulator MnSe Luo and Qi (2013). Due to charge transfer generated by the work function difference between the two materials, a significant amount of band bending occurs at the interface. The Dirac point, indicated by the red circle, remains within the band gap of MnSe, which allows us to neglect broadening caused by the bulk bands of the ferromagnet. The bend bending on the TI side of the interface, however, shifts the Dirac point below the top of the valence band of the TI. As such, the surface state electrons can tunnel through the potential barrier and escape into the bulk valence band of the TI, giving the surface states a finite lifetime similar to what we observe in metallic ferromagnets. Theoretical studies on band bending effects at the surface of TIs have already shown that significant broadening is generated through tunneling into bulk bands Bahramy et al. (2012); Garate and Glazman (2012). Therefore, even when a ferromagnetic insulator is used to generate a mass gap in the TI surface states, the QAHE will be challenging to observe unless the work function difference between the materials is overcome using electrostatic gating.

We contrast our analysis of the mass gap and broadening generated within proximity-coupled, ferromagnet-TI heterostructures with that created in a magnetically-doped TI Yu et al. (2010), a material system in which the QAHE has already been experimentally demonstrated Chang et al. (2013). In addition to the exchange splitting that is generated by the magnetic dopants, scattering of electrons on these impurities can give the states a finite lifetime that could cause a loss in quantization of the Hall conductivity. In Supplemental Material Sec. IV, we characterize the mass gap and broadening generated by this magnetic impurity scattering through a disorder-averaged self-energy. The leading-order contributions to this self-energy take the form

(15) | ||||

(16) |

where is the concentration of magnetic impurities and is the effective exchange interaction introduced by the magnetic dopants. This scattering self-energy is perturbatively expanded in powers of , which is taken to be much less than unity. Similar to the ferromagnet contact self-energy, we see that that the presence of magnetic impurities introduces both a mass gap, , and a non-Hermitian broadening, . These contributions, however, arise from different orders in the perturbation series and, therefore, it is trivial to show that . Therefore, the broadening introduced by magnetic impurities is constrained to be much smaller than the mass gap, resulting in a nearly quantized Hall conductivity based on Eqs. (12)-(13) and Fig. 2. Such analysis excludes vertex corrections that naturally arise in perturbative calculations of conductivity, but the qualitative interpretation remains the same. The small broadening generated by magnetic impurity scattering is much less than that generated by a proximity-coupled ferromagnet and explains why experimental observation of the QAHE is possible in magnetic topological insulators Chang et al. (2013).

In summary, we analytically derive a contact self-energy that characterizes the effect of proximity-coupling a metallic ferromagnet to a TI surface state. We show that when metallic bands from the ferromagnet are present near the Dirac point, a non-Hermitian broadening term is introduced to the Hamiltonian in addition to the anticipated time-reversal-breaking mass gap term. This contact self-energy can introduce large enough broadening that the surface spectrum appears gapless despite the presence of a mass gap. We calculate the Hall conductivity for the effective Hamiltonian describing the heterostructure and show that for any non-zero broadening, the Hall conductivity is no longer quantized. For cases when broadening is on the order of the mass gap, we find that the Hall conductivity can be significantly reduced, making the QAHE extremely challenging to observe in proximity-coupled ferromagnet-TI systems. For comparison, we also calculate the broadening and mass gap expected from scattering off of magnetic impurities in magnetically-doped TIs and find that the broadening is constrained to be much less than the mass gap. As such, the QAHE remains nearly quantized in magnetic topological insulators despite the presence of finite broadening. Therefore, our work demonstrates that although non-Hermitian Hamiltonians can be classified using non-Hermitian topological band theory, observables that are topologically quantized in the Hermitian theory, may no longer retain such quantization.

This work is supported by the National Science Foundation (NSF) under CAREER Award ECCS-1351871. M.J.G acknowledges financial support from the Office of Naval Research (ONR) under grant number N00014-17-1-3012. T.M.P acknowledges fruitful discussions with Y. Kim, M. J. Park, G. A. Hamilton, and B. Basa.

### References

- M. Z. Hasan and C. L. Kane, Rev. Mod. Phys. 82, 3045 (2010).
- X.-L. Qi and S.-C. Zhang, Rev. Mod. Phys. 83, 1057 (2011).
- B. A. Bernevig, Topological Insulators and Topological Superconductors (Princeton University Press, Princeton, New Jersey, 2013).
- K. V. Klitzing, G. Dorda, and M. Pepper, Phys. Rev. Lett. 45, 494 (1980), arXiv:arXiv:1011.1669v3 .
- F. D. M. Haldane, Phys. Rev. Lett. 61, 2015 (1988).
- B. Jeckelmann, B. Jeanneret, and D. Inglis, Phys. Rev. B 55, 13124 (1997).
- S. Lieu, Phys. Rev. B 97, 045106 (2018), arXiv:1709.03788 .
- A. Cerjan, M. Xiao, L. Yuan, and S. Fan, Phys. Rev. B 97, 075128 (2018), arXiv:1712.02444 .
- A. A. Zyuzin and A. Y. Zyuzin, Phys. Rev. B 97, 1 (2018), arXiv:1710.05344 .
- H. Zhou, C. Peng, Y. Yoon, C. W. Hsu, K. A. Nelson, L. Fu, J. D. Joannopoulos, M. Soljaić, and B. Zhen, Science 9859, eaap9859 (2018), arXiv:1709.03044 .
- H. Shen, B. Zhen, and L. Fu, Phys. Rev. Lett. 120, 146402 (2018), arXiv:1706.07435 .
- M. Papaj, H. Isobe, and L. Fu, (2018), arXiv:1802.00443 .
- P. Streda, J. Phys. C Solid State Phys. 15, 717 (1982).
- N. A. Sinitsyn, A. H. MacDonald, T. Jungwirth, V. K. Dugaev, and J. Sinova, Phys. Rev. B 75, 045315 (2007), arXiv:0608682 [cond-mat] .
- T. S. Nunner, N. A. Sinitsyn, M. F. Borunda, V. K. Dugaev, A. A. Kovalev, A. Abanov, C. Timm, T. Jungwirth, J.-i. Inoue, A. H. MacDonald, and J. Sinova, Phys. Rev. B 76, 235312 (2007), arXiv:0502386 [cond-mat] .
- N. Nagaosa, J. Sinova, S. Onoda, A. H. MacDonald, and N. P. Ong, Rev. Mod. Phys. 82, 1539 (2010).
- W. Luo and X.-L. Qi, Phys. Rev. B 87, 085431 (2013), arXiv:1208.4638 .
- M. Bahramy, P. King, A. de la Torre, J. Chang, M. Shi, L. Patthey, G. Balakrishnan, P. Hofmann, R. Arita, N. Nagaosa, and F. Baumberger, Nat. Commun. 3, 1159 (2012), arXiv:1206.0564 .
- I. Garate and L. Glazman, Phys. Rev. B 86, 035422 (2012), arXiv:1206.1239 .
- R. Yu, W. Zhang, H.-J. Zhang, S.-C. Zhang, X. Dai, and Z. Fang, Science 329, 61 (2010).
- C. Chang, J. Zhang, X. Feng, J. Shen, Z. Zhang, M. Guo, K. Li, Y. Ou, P. Wei, L.-L. Wang, Z.-Q. Ji, Y. Feng, S. Ji, X. Chen, J. Jia, X. Dai, Z. Fang, S.-C. Zhang, K. He, Y. Wang, L. Lu, X.-C. Ma, and Q.-K. Xue, Science 340, 167 (2013).