# Stability of the X-Y-phase of the two-dimensional point group insulator

## Abstract

Noninteracting insulating electronic states of matter can be classified according to their symmetries in terms of topological invariants which can be related to effective surface theories. These effective surface theories are in turn topologically protected against the effects of disorder. Topological crystalline insulators are, on the other hand, trivial in the sense of the above classification but still possess surface modes. In this work we consider an extension of the Bernevig-Hughes-Zhang model that describes a point group insulator. We explicitly show that the surface properties of this state can be as robust as in topologically nontrivial insulators, but only if the -component of the spin is conserved. However, in the presence of Rashba spin-orbit coupling this protection vanishes and the surface states localize, even if the crystalline symmetries are intact on average.

## I Introduction

Soon after the prediction kane2005 (); kane2005a (); bernevigzhang2006 () and subsequent discovery of the quantum spin Hall insulator koning2006 () as a novel state of electronic matter with properties protected by topology, it became clear that this state fits into an even grander scheme. moore2007 (); fukane2006 (); fukane2007a (); fukane2007b () By now many more topological phases have been identified both theoretically and experimentally. hsieh2008 (); hsieh2009 (); xia2009 (); zhang2009 (); chen2009 (); hasan-kane-review (); qi-zhang-review () The classification of noninteracting electronic insulating states with topological order was one of the milestones of theoretical condensed matter physics in the last decade. schnyder2008 (); kitaev2009 (); Qi2008 () Within the so-called ’tenfold periodic table’ states of matter with topological order are classified according to the presence or absence of symmetries of the underlying systems, schnyderNJP2010 () such as time-reversal, particle-hole, and chiral.

One of the prominent features of topological electronic systems is the existence of exotic gapless edge or surface states. In particular, these boundary modes can realize one-dimensional chiral or single two-dimensional Dirac fermions, usually ruled out by the fermion doubling theorem (at least if all the symmetries are preserved). Importantly, in the presence of disorder these states can be protected against localization which leads to unusually robust transport properties. An instructive point of view on the existence of these robust boundary states is that the tenfold periodic table does not require spatial symmetries such as translations or rotations to be present, which is particularly apparent in the classification scheme using non-linear models. schnyder2008 () This implies that although the calculation of topological invariants can in general most easily be accomplished in clean band-insulators, in principle there is no need to have a well-defined crystal momentum for doing so. On the other hand, electronic states trivial according to the tenfold classification but still featuring edge or surface modes have been recently identified once crystalline symmetries, such as translations, rotations, reflections and inversions, were taken into account.fu2011 (); slager2013 (); chiu2013 (); morimoto2013 (); shiozaki2014 () These states are conventionally referred to as topological crystalline insulators fu2011 () (TCI). Interestingly, these states, predicted to be realized in Sn- and Pb-based compounds, fuNatComm2012 () have been reported to be observed first in Refs. ando2012, ; xu2012, ; story2012, , and subsequently have been experimentally studied in Refs. tanaka2013, ; okada2014, ; zeljkovic2014, . An urging question is therefore whether the boundary states in topological phases where crystal symmetries allow to define topological invariants can enjoy similar protection against the effects of disorder as they do in tenfold-wise topologically nontrivial insulators.

To answer this question at least for one concrete example, we investigate the transport properties of an extension of the well-known Bernevig-Hughes-Zhang (BHZ) model that was recently introduced. slager2013 (), to which we refer as a point group insulator (PGI) in the following. There, it was identified that a rotational symmetry leads to the existence of a state characterized by a trivial topological invariant in the tenfold sense, but non-trivial in a sense defined in Sec. II. Within this note we make a comparison of this PGI with a standard quantum spin Hall insulator (QSHI), for which the BHZ model was originally formulated.bernevigzhang2006 () For the QSHI we find, as expected and well known, very robust transport properties with a quantized lead-to-lead conductance, both in presence and absence of disorder (as long as the disorder strength is smaller than the bulk-gap scale). Importantly, this property is also robust against breaking of the spin-rotational symmetry, introduced for instance by Rashba spin-orbit coupling, and is a direct consequence of time-reversal symmetry. For the PGI, on the other hand, we find that the conductance is only quantized and robust against disorder if the -component of the spin is conserved. However, in the presence of Rashba spin-orbit coupling disorder localizes, in the Anderson sense, the boundary modes leading to a vanishing conductance, in agreement with the specific PGI state being trivial in the sense of the periodic table. We note here that our results concerning the stability of the edge modes in a PGI without Rashba spin-orbit coupling are in agreement with the findings recently reported in Refs. jiang2014, and ezawa2014, , but also show that the results obtained therein are fine tuned and non generic.

The paper is organized as follows. In Sec. II, we introduce the model and its phase digram, and in Sec. III, we present the results concerning the transport properties in both QSHI and PGI phases. Our conclusions are drawn in Sec. IV.

## Ii Model, Topological Invariants and Phase Diagram

The model we study was recently introduced in Ref. slager2013, and represents an extension of the BHZ Hamiltonian that includes next-nearest neighbor hoppings. Its Bloch Hamiltonian has the generic form

(1) |

where

(2) |

and is the vector composed of the Pauli matrices. The wave function is a four component object, and, motivated by the low-energy band-structure of HgTe/CdTe quantum wells, contains and like orbitals with the respective spins . The parameter is related to the offset of the chemical potential between the and like orbitals, while () is proportional to the (next-)nearest-neighbor hopping between the orbitals of the same type, while both nearest-neighbor and next-nearest-neighbor hoppings between different orbitals are set to one.

We choose a version of Rashba spin-orbit coupling which is not symmetric in the orbitals and rothe2010 () but respects all other symmetries discussed below. The corresponding term in the Hamiltonian (1) reads

(3) |

The Hamiltonian (1) obeys time reversal symmetry, which is implemented via , where acts in spin space and denotes complex conjugation ( is the identity in orbital space). This symmetry will be intact hereafter. Additionally, the clean system has the discrete rotational symmetry about the axis orthogonal to the crystal plane, represented by , with . The invariance of the Hamiltonian (1) under this symmetry operation is shown in the Appendix. However, this symmetry will be broken by disorder and the boundary of the sample.

### ii.1 Topological Characterization of the phases on a square lattice

Before we study the model (1), we discuss a topological characterization of the possible topological phases on the square lattice. In Ref. slager2013, it was shown that in the presence of time-reversal symmetry, the distinct topological phases on a lattice with a given space group symmetry can be characterized in terms of the band inversions at time-reversal invariant (TRI) momenta in the Brillouin zone (BZ). The band inversion is given in terms of the sign of the Pfaffian at TRI momenta. In the BHZ model the sign of the Pfaffian at a TRI momentum is equal to the sign of the mass term [ in Eq. (1)] at that point. slager2013 (); ezawa2014 () When these points are, in addition, related by the rotational symmetry, as is the case with and points on the square lattice, the corresponding band inversions are connected to each other. If there is a band inversion at point, there is one at point as well, and this is enforced precisely by the symmetry,slager2013 () as it is the case in the phase, which is obtained in the model (1), as detailed below. Notice, however, that this phase has a trivial topological invariant, which is also confirmed by our stability analysis. Finally, given the gauge of the Bloch states, the sign of the Pfaffian cannot be changed without closing the band gap, and thus phase is topologically distinct from other possible phases on the square lattice, and phases, which both have nontrivial invariant.slager2013 ()

An alternative way to show that the phase is topologically distinct from the other topological phases on a square lattice was outlined by Ezawa in Ref. ezawa2014, , where he derived the form of the mirror Chern number in terms of the signs of the Pfaffians at TRI points, Eq. (34) therein. Just by interchanging the mirror operator with the operator, which is allowed since the only ingredient necessary in this derivation is the commutation of the Bloch Hamiltonian with the point-group operation, footnote () the topological invariant associated with symmetry is also given in terms of signs of the Pfaffians at TRI points . Therefore, the topological invariant corresponding to symmetry is

(4) |

since due to symmetry. In the phase, , while , so . On the other hand, in the () phase, with (), , and therefore both and phases are topologically distinct from the phase.

In the phase, depending on the boundary, the two resulting edge modes lie at different TRI momenta, and this is important for the resulting physics of the edge states. For instance, when the cut is along one of the two principal crystallographic axes or , the two edge states originate from the Dirac cones located at the TRI momenta and in the boundary BZ. On the other hand, when the cut is along the diagonal, the two Dirac cones are both located at a non-TRI momentum . The types of the edges are completely analogous to the surface states discussed in Ref. fu-surface, in the context of three-dimensional TCIs on the rocksalt lattice with the bulk band inversion at the four points in the BZ. The surface perpendicular to the crystallographic direction in this 3D TCI corresponds to an edge along one of the principal axes in the phase, since the corresponding edge/surface Dirac cones are separated by a finite momentum, and therefore a finite momentum transfer is needed to mix them. On the other hand, the surface in the 3D TCI is analogous to the edge along one of the diagonals in the phase, since in both cases the surface/edge Dirac cones lie at the same momentum, and can therefore hybridize.

These general arguments imply that upon adiabatically switching on the Rashba-coupling in the phase, provided that the bulk gap remains open, the two Dirac points cannot be mixed and consequently edge states are stable, as long as there is no perturbation providing the momentum transfer required to mix the two Dirac cones. As mentioned before, there is one special situation in which this is not true, however, and that is when the two Dirac cones corresponding to the two Kramers pairs of edge states coincide in the edge BZ, corresponding to a diagonal cut, as discussed above. In that situation a gap will open due to Rashba spin-orbit coupling. The most natural candidate to generically mix the Dirac cones in the presence of Rashba spin-orbit coupling is scalar disorder, which we consider in the paper.

### ii.2 Phase diagram

The phase diagram for the model (1) is shown for as a function of and in Fig. 1. The phase diagram for and all values of was introduced recently. slager2013 () If , the Hamiltonian explicitly conserves the -component of the spin and we can deduce a phase diagram by calculating the Chern numbers in the respective spin sectors. As a function of this leads to three phases. For values of there is the -phase which is the standard QSHI from the AII symmetry class with a topological index kane2005 (). This phase can also be understood as a stack of two time-reversed Chern insulators of Chern number tknn (); haldane () , which leads to the pair of helical boundary modes related to each other by time-reversal symmetry. For values we find the X-Y valley phase, indexed as in Ref. slager2013, , which we refer to as PGI. In this parameter regime the time-reversed bands have Chern number . This results in two pairs of helical edge modes, where each of the pairs consists of counterpropagating modes related by time-reversal symmetry. This phase is trivial in the sense of the periodic table of electronic topological states, i.e., the -index is trivial (), but features a non-trivial topological invariant stemming from the symmetry, as shown above. For higher values of the model enters a trivial phase, meaning that both the spin Chern numbers as well as the -index are trivial (, , and ). All phases possess time-reversal symmetry and are connected via quantum phase transitions with quantum critical points that are metallic.

Upon turning on the Rashba spin orbit coupling, , all symmetries of the system are preserved, except for spin-rotational. Therefore, we cannot define the spin Chern numbers any more (the -index is still well defined and calculable, though), but we can still define . At a critical value of the Rashba spin-orbit coupling , the bulk band-gap closes, and all the phases become metallic as is further increased. To show that the clean system still features edge states without conserved spin, we studied the finite size spectrum on a cylinder. In the case of the -phase for finite we find one pair of helical edge states (Fig. 2 (a)) while in the X-Y valley phase we find two pairs (except for diagonal cut), see Fig. 2 (b).

Overall we conclude that finite Rashba coupling does not change the system properties in the clean system in either phase below a respective critical where the insulator is converted into a metal.

## Iii Transport properties

One of the most striking properties of topological insulator states is not only the existence of boundary modes but also their stability with respect to disorder. The most prominent example is the quantum Hall state which has quantized Hall conductivity , with as an integer. The integer can be viewed in two equivalent ways: (i) it is the cumulative Chern number of the bands below the chemical potential or (ii) the number of chiral boundary modes. The conductance then accounts for the number of channels at the boundary. Naively, one would expect disorder to localize these modes, but their chiral nature provides an escape route: disorder cannot localize them since there is no way of converting a left-mover into a right-mover and vice versa by virtue of them being unidirectional.

In the case of a QSHI we do not have chiral channels but instead helical modes. This implies that there are right- and left-movers on either side of the sample which potentially allows for elastic backscattering due to scalar disorder. However, the helical modes are Kramers’ pairs related by time-reversal symmetry, which implies that scalar disorder cannot convert left-movers into right-movers and vice versa due to the orthogonality of Kramers’ pairs. The situation is schematically depicted in Fig. 3 (b). Consequently, edge transport is also ballistic resulting in a quantized conductance. This has also been observed in transport experiments on HgTe/CdTe quantum wells. koning2006 ()

In the case of the PGI we find there are two pairs of boundary modes at each sample edge, see right-hand-side of Fig. 2 (b), but usually two pairs of modes are unstable since any right-mover in general is not mutually orthogonal to both left-movers and they can thus scatter into each other under generic circumstances. So the question we address here is in which sense and under which conditions the valley phase, topologically trivial according to the periodic table, has stable transport properties once disorder is added.

The setup that we probe is shown in Fig. 3 (a) where we connect a central sample region to left and right leads and measure the response coefficient for transport from the left lead to the right lead in the linear response regime (we assume , being small).

In order to study the stability of the edge states with respect to disorder we resort to the well established non-equilibrium Green-function method, which we implement numerically in the linear response regime dattabook ().

In the clean system we can count the boundary modes assuming ballistic transport based on the finite size spectra (see Fig. 2), and find

(5) |

(Note that the result in the phase does not hold if the edge is cut along the diagonal)

We have modelled disorder by scalar disorder, i.e., through local variations in the chemical potential. We have chosen disorder of the box type with a strength , thus the local energy is drawn from . This preserves all the symmetries of the system required by the classification scheme, but it obviously breaks translational and rotational symmetries (locally, but not on average). Subsequently, we have checked the localization tendencies by probing the system (i) with a fixed sample size as a function of increasing disorder strength, and (ii) at fixed disorder strength as a function of increasing sample size.

In order to make better comparisons between the QSHI and PGI , we have chosen parameters such that bulk gaps are approximately the same in both systems. We found that a convenient parameter set is given by , (QSHI) or (PGI) without Rashba coupling, while with Rashba coupling we choose for the PGI (to have comparable gaps).

In the absence of Rashba spin orbit coupling () with conserved spin, we find that both and the X-Y valley phase are equally stable against disorder. This is not very surprising since both in the -phase as well as in the X-Y valley phase we can think of the system as two time-reversed Chern insulators with Chern numbers for the QSHI and for the PGI and the left- and right-moving channels do not mix, see Fig. 3 (b) and (c). In Fig. 4, we display a plot of the conductance of a sample of size with the chemical potential in the bulk gap as a function of disorder strength. Our results show that both the QSHI and the PGI phases are stable against disorder, since the conductance is quantized. In Fig. 5 we plot conductance at fixed disorder strength as a function of the system size (we always study systems of transverse size 80 sites), and find again that both QSHI and PGI are stable.

In the presence of Rashba spin-orbit coupling (), we find that the features of the QSHI phase are unchanged, as expected. However, the protection of the conductance is lost in the case of the PGI. This can explicitly seen in Fig. 4, where the conductance as a function of disorder strength at fixed sample size decreases, as soon as Rashba spin orbit coupling is switched on. Furthermore, in Fig. 5 we observe that the conductance as a function of the system size decreases and eventually would vanish if we made the sample long enough. This signals the localization of the modes in agreement with the absence of topological protection. As discussed before, this can be traced back to the loss of orthogonality of left- and right-movers at the sample edges. We have explicitly checked the loss of orthogonality under scattering from a structurless impurity () of the left- and right-movers in the finite size spectrum (as also discussed in Fig. 3 (c)). We can therefore conclude that the absence of protection against backscattering leads to the localization of the boundary modes.

## Iv Conclusions

In this paper we have studied the stability of the edge states of a topological system outside the tenfold classification of the topological insulator states with respect to disorder. We considered a special instance of a PGI where discrete rotational symmetry guarantees the existence of edge states in a clean system. If the -component of the spin is a conserved quantity in this system we find that the boundary modes are protected against localization due to disorder. This protection against disorder is lost if Rashba spin-orbit coupling is present. Consequently, the system indeed behaves like a trivial insulator in the sense of the topological classification of electronic states. For the future it is an interesting prospect to study the localization properties of other systems with boundary modes which are outside the tenfold classification of topological states, such as the recently observed three-dimensional TCI phase featuring band-inversions at symmetry-related points in the Brillouin zone.ando2012 (); xu2012 (); story2012 ()

## V Acknowledgement

We thank Michael Wimmer and Jason Frank for helpful discussions. We acknowledge funding from the DFG FR 2627/3-1 (C.K. and L.F.). This work is part of the D-ITP consortium, a program of the Netherlands Organisation for Scientific Research (NWO) that is funded by the Dutch Ministry of Education, Culture and Science (OCW). V. J. acknowledges financial support from NWO.

## Appendix A Invariance of the Hamiltonian under rotation

In this Appendix we explicitly show that the Hamiltonian in Eq. (1) in the main text is invariant under rotations. rotation is represented by

(6) |

with

(7) | |||

(8) |

This symmetry operation acts on the components of the momentum as

(9) |

First, the Hamiltonian (1) in the main text without the Rashba term () can be conveniently rewritten as

(10) |

and is clearly invariant under spin rotation. The invariance under rotation in the orbital space is a consequence of the fact that

(11) |

and the transformation of the momentum under given by Eq. (9). Namely, the form of the transformation yields , while is invariant, which together with Eq. (11), implies the invariance of under the rotation.

To show that is a symmetry of the Rashba Hamiltonian, we conveniently rewrite it as

(12) |

The orbital part of this Hamiltonian is invariant under rotation, since the matrices and commute with each other. Under rotation, Pauli matrices transform as

(13) |

Transformation of the momentum under , Eq. (9), implies that under the same operation. This, together with Eq. 13, implies the invariance of the spin part of the Hamiltonian (12), which shows the invariance of the Hamiltonian (12) under transformation.

### References

- C. L. Kane, E. J. Mele, Phys. Rev. Lett. 95, 146802 (2005).
- C. L. Kane, E. J. Mele, Phys. Rev. Lett. 95, 226801 (2005).
- B. A. Bernevig, T.L. Hughes, S.C. Zhang, Science 314, 1757 (2006).
- M. König, et al., Science 318, 766 (2007).
- J.E. Moore, L. Balents, Phys. Rev. B 75, 121306 (2007).
- L. Fu, C. L. Kane, Phys. Rev. B 74, 195312 (2006).
- L. Fu, C. L. Kane, Phys. Rev. Lett. 98, 106803 (2007).
- L. Fu, C. L. Kane, Phys. Rev. B 76, 045302 (2007).
- D. Hsieh, et al., Nature 452, 970 (2008).
- D. Hsieh, et al., Science 323, 919 (2009).
- Y. Xia, et al., Nature Phys. 5, 398 (2009).
- H. Zhang, et al., Nature Phys. 5, 438 (2009).
- Y. L. Chen, et al., Science 325, 178 (2009).
- M. Z. Hasan, C. L. Kane, Rev. Mod. Phys. 82, 3045 (2010).
- X. L. Qi, S. C. Zhang, Rev. Mod. Phys. 83, 1057 (2011).
- A. P. Schnyder, S. Ryu, A. Furusaki, A.W.W. Ludwig, Phys. Rev. B 78, 195125 (2008).
- A. Kitaev, AIP Conf. Proc. 22, 1132 (2009).
- X.-L. Qi, T. Hughes, S.-C. Zhang, Phys. Rev. B 78, 195424 (2008).
- S. Ryu, A. P. Schnyder, A. Furusaki, A. Ludwig, New J. Phys., 12, 065010 (2010).
- L. Fu, Phys. Rev. Lett. 106, 106802 (2011).
- R.-J. Slager, A. Mesaros, V. Juričić, and J. Zaanen, Nature Phys. 9, 98 (2013).
- C.-K. Chiu, H. Yao, and S. Ryu, Phys. Rev. B 88, 075142 (2013).
- T. Morimoto and A. Furusaki, Phys. Rev. B 88, 125129 (2013).
- K. Shiozaki and M. Sato, Phys. Rev. B 90, 165114 (2014).
- T. H. Hsieh, H. Lin, J. Liu, W. Duan, A. Bansil, L. Fu, Nat. Comm. 3, 982 (2012)
- Y. Tanaka, Z. Ren, T. Sato, K. Nakayama, S. Souma, T. Takahashi, K. Segawa, and Y. Ando, Nature Phys. 8, 800 (2012).
- S.-Y. Xu, C. Liu, N. Alidoust, M. Neupane, D. Qian, I. Belopolski, J. D. Denlinger, Y. J. Wang, H. Lin, L. A. Wray, G. Landolt, B. Slomski, J. H. Dil, A. Marcinkova, E. Morosan, Q. Gibson, R. Sankar, F. C. Chou, R. J. Cava, A. Bansil, and M. Z. Hasan, Nature Commun. 3, 1192 (2012).
- P. Dziawa, B. J. Kowalski, K. Dybko, R. Buczko, A. Szczerbakow, M. Szot, E. Lusakowska, T. Balasubramanian, B. M. Wojek, M. H. Berntsen, O. Tjernberg, and T. Story, Nature Mater. 11, 1023 (2012).
- Y. Tanaka, T. Sato, K. Nakayama, S. Souma, T. Takahashi, Z. Ren, M. Novak, K. Segawa, and Y. Ando, Phys. Rev. B 87, 155105 (2013).
- Y. Okada, M. Serbyn, H. Lin, D. Walkup, W. Zhou, C. Dhital, M. Neupane, S. Xu, Y. J. Wang, R. Sankar, F. Chou, A. Bansil, M. Z. Hasan, S. D. Wilson, L. Fu and V. Madhavan, Science 341, 6153 (2014).
- I. Zeljkovic, Y. Okada, C. Y. Huang, R. Sankar, D. Walkup, W. Zhou, M. Serbyn, F. Chou, W. F. Tsai, H. Lin, A. Bansil, L. Fu, M. Z. Hasan, and V. Madhavan, Nature Phys. 10, 572 (2014).
- H. Jiang, H. Liu, J. Feng, Q. Sun, and X. C. Xie, Phys. Rev. Lett. 112, 176601 (2014).
- M. Ezawa, New J. Phys. 16, 065015 (2014).
- D. G. Rothe, R. W. Reinthaler, C.-X. Liu, L. W. Molenkamp, S.-C. Zhang, and E. M. Hankiewicz, New J. Phys. 12, 065012 (2010).
- An analogous derivation can be carried out based on the results in Ref. fang2012, .
- C. Fang, M. J. Gilbert, and B. A. Bernevig, Phys. Rev. B 86, 115112 (2012).
- J. Liu, W. Duan, and L. Fu, Phys. Rev. B 88, 241303(R) (2013).
- D. J. Thouless, M. Kohmoto, M. P. Nightingale, M. den Nijs, Phys. Rev. Lett. 49, 405 (1982).
- F. D. M. Haldane, Phys. Rev. Lett. 61, 2015 (1988).
- S. Datta, Electronic Transport in Mesoscopic Systems, Cambridge University Press (1997).