Emergent topological mirror insulator in t_{2g}-orbital systems

# Emergent topological mirror insulator in t2g-orbital systems

## Abstract

Motivated by the itinerant band structure of high- iron pnictides, which exhibit four Dirac cones in the bulk, we demonstrate the prospect of pnictides with transition elements to be topological insulators in two dimensions. In this report, we explore interaction-induced topological phases, in contrast to the spin-orbit-coupling interaction, as the crucial mechanism for tuning Dirac metals into Z-topological insulators protected by time reversal and mirror symmetries. We find spontaneous orbital currents generated through nearest-neighbor inter-orbital Coulomb interaction in the manifold of the orbitals. When spin degrees of freedom are incorporated, spontaneous orbital currents lead to two stable topological phases of the ground state. The first topological insulator is an anomalous orbital Hall phase, characterized by an even Chern number, while the second topological insulator is realized by protected mirror symmetries with a Z index.

###### pacs:
71.10.Fd,71.10.Pm,73.20.-r

#### Introduction.−

Topological insulators (TIs) are typically characterized by the band topology of their electronic wave function in the bulk, which is connected to protected edge or surface states. In fact, the integer quantum Hall insulator is the first known TI, which Thouless and collaborators (1) characterized by the topological Chern number under the condition of broken time-reversal (TR) symmetry due to an external magnetic field. In the absence of an external magnetic field, the analogs of the quantum Hall effect and topological Chern number were discussed by Haldane (2) for the honey-comb lattice with spontaneous internal magnetic fields between two different sub-lattices, and by Volovik (3) for charged and neutral superfluids with He-like order parameters. By promoting Haldane’s model to a spinful version that respects the TR symmetry in the presence of strong spin-orbit coupling (SOC), the concept of the quantum-spin Hall insulator was proposed (4), which is characterized by a nontrivial topological invariance. The manifestation of TIs with symmetry is accompanied by the opening of a gap in Dirac semi-metals due to the SOC interaction, and the emergence of gapless, symmetry-protected edge (or surface) states in two (or three) dimensions.

Ever since the discovery of TIs, new types and realizations have been extended to new materials (5); (46); (7); (8); (9); (10); (11); (12); (13); (14); (15); (16); (18); (19). In this letter, we show a very different realization of the topological mirror insulator (TMI) (20); (21) in the bands of two-dimensional (2D) insulators with pnictide-like band structure. In contrast to the insulating gap generated by the SOC models for graphene or bismuth, (16); (17) or interaction-driven topological insulators in the presence of strong SOC, we propose a new route to realizing non-trivial, emergent topological phases within the low-energy manifold in transition element materials (42) with the insulating gap opened by purely correlated electron interactions (23), see also Refs. (18); (19) and references therein. Our discussion is based on a realistic, minimal quasi-2D model (44); (45); (26), which proved successful in reproducing the electronic structure and phase diagram of the 122 iron pnictides. In this two orbital model, we consider onsite intra-orbital and nearest-neighbor (NN) inter-orbital Coulomb interaction treated within mean-field theory. With reasonable hopping and Coulomb parameters, we find spontaneous orbital currents in the ground state. These orbital currents generate non-trivial topological phases with two pairs of Dirac cones appearing as edge states. We further show that a non-trivial mirror- phase can be identified for the spinful Hamiltonian. This novel phase involves mirror reflection symmetries in 2D and is robust against weak TR breaking perturbations. To be more specific, this phase can only be destroyed by perturbations that break the mirror symmetry in the Brillouin zone (BZ). Hence the presented scenario is markedly different from previous TIs, which are protected by TR symmetry and exhibit an odd number of Dirac cones.

#### The spinless t2g orbital model.−

We start with a simplified orbital model Hamiltonian, , for spinless fermions to facilitate our symmetry analysis and discussion. Here is the hopping term, is the interaction term, and are perturbation terms:

 H0= ∑IJ,αβ(tαβIJ−μδIJδαβ)c†IαcJβ, (1) HV= iϵλAOH∑IJ,ανα¯αIJc†IαcJ¯α, h0= λ0∑I,α(−1)αc†IαcIα, h1= iϵλ1∑I,α(−1)αc†IαcI¯α,

where are lattice site indices, and are indices for the and orbitals in the manifold. We choose the nonvanishing hopping elements as , , , , , with . The tensor elements describe the direction of the NN inter-orbital currents as shown in Fig. 1(a) with , and . The scalar describes the direction of the orbital current loop or the direction of the arrows in Fig. 1(a). The hopping parameters between different lattice sites and orbitals are given by , and is the chemical potential which includes the mean-field energy shift from the onsite Coulomb interaction. The anomalous orbital Hall (AOH) effect is the multiorbital analog of the anomalous Hall effect and is described by the complex hopping term between different orbitals and different sites and with the spinless coupling constant determined by the current order through the NN inter-orbital Coulomb interaction . The real part of the current order, , can be absorbed into the hopping terms and does not affect our conclusions. In this paper all the parameters are in units of which can be adjusted to fit the band structure from the ARPES experiment or the DFT calcuations.

The onsite orbital energy difference in the term is responsible for the orbital charge polarization, which can be induced by an external electric field perpendicular to the lattice or anisotropic strain from the substrate. On the other side, the coupling constant in is responsible for inter-orbital coherence. Although and may be negligible in real systems, they allow us to perform a stability analysis of the topological phases toward TR symmetry violation.

Due to the translational invariance of the periodic lattice structure, the Hamiltonian can be diagonalized in the momentum space , that is, , with basis functions (where is the transpose operation). We derive the expression , where is the 2-by-2 unit matrix, and are the Pauli matrices. The ancillary functions , , and are given by

 E0= 2t1[cos(kx)+cos(ky)]+2t6[cos(2kx)+cos(2ky)], (2) +2(t2+t3)[cos(kx)cos(ky)]−μ, X= 4t4[cos(kx)cos(ky)]+2(t5+δt)[cos(kx)+cos(ky)], Y= ϵ(2λAOH[cos(kx)−cos(ky)]+λ1), Z= 2(t2−t3)[sin(kx)sin(ky)]+λ0.

The diagonalization of attains the eigenvalues , where . The corresponding eigenvectors are

 |+,k⟩ =(Z+B,X+iY)T/√2B2+2ZB, (3) |−,k⟩ =(−X+iY,Z+B)T/√2B2+2ZB.

We find an even number of (four) Dirac cones in the dispersion of the noninteracting bulk material, i.e., . Their positions are located at the and axes as determined by , see Fig. 1(b)  (27). For any finite orbital current order () a nonzero will be generated, inducing the anomalous orbital Hall effect. Consequently, the Dirac cones in the bulk, which are responsible for nontrivial band topology, become gapped. If we manually turn off in the presence of , the four Dirac cones are pushed toward the center of each quadrant of the BZ. When spin degrees of freedom are considered, the term is also responsible for band topology protected by reflection symmetry, , which is the main focus of this work.

It is worth to note that similar ideas about the importance of the NN-inter-orbital Coulomb interaction and resulting current flux phases have been discussed in the context of bilayer graphene (28); (29); (30) and the cuprates (31); (32); (33); (34). Although the origin of our proposed inter-orbital currents is similar to the loop or circulating current flux phase in the pseudo-gap phase of the cuprates (31); (32); (33); (34), our model does not rely on strong onsite Coulomb interaction and has different orbital degrees of freedom, crystal symmetry, and conduction band topology. Specifically, the single-orbital models of the cuprates with -density-wave order (32); (33) break the 1-atom per unit cell translational invariance and the TR symmetry. The current loop model (34) violates the TR symmetry.

#### Anomalous Orbital Hall Phases.−

The ground state of the spinless Hamiltonian is illustrated in Fig. 1(a) based on standard self-consistent mean-field calculations. In Fig. 1(b) and 1(c) we show the dispersion of the electronic band structure of the noninteracting bulk material at half filling when . For insulators, the nonlocal topology of band can be captured by the Chern number directly through the Berry curvature of the Hamiltonian , which is defined as , where the expression of is defined in the Supplemental Material (SM) (35). The first observation is that for any real but nonzero . The second observation is that the sign of depends on the direction of the orbital current loop through and the band index , which determine the class of the Chern insulator, . Since the topological phase with is robust against weak perturbations by TR symmetry violation, we show its stability region in the - phase diagram in Fig. 1(d), where for illustration purposes we chose the strong coupling limit . Note that the TI phase is induced by interaction and therefore vanishes for . Following the orange line in the phase diagram, we monitor the evolution of the bulk band gap as it closes and reopens with increasing , see Figs. 1(e) to 1(g). This leads to a sequence of phase transitions from topological Chern insulator to metal (around ) and on to trivial band insulator with .

#### The spinfull t2g orbital model.−

In materials with magnetic interactions we need to consider electrons as fermions with spin degrees of freedom. Therefore, we promote the spinless two-band orbital model to the spinful model . Here the sign of the spinful orbital current direction is denoted as for each spin index . A detailed analysis of the Hamiltonian (see the SM) reveals the following invariants of the stable topological phases:
Phase I:    with Chern number ,
Phase II:   with Chern number .
For phase I, we find that the Chern number of the occupied bands is twice that of the spinless case due to the twofold degeneracy of spins, because degenerate spins share the same orbital current direction. For phase II, we find that the Chern number classification scheme is insufficient to capture the topological nontrivial insulator phase, because of .

It is interesting to note that the form of the interaction term in phase II is formally equivalent to an inter-orbital SOC, with . This equivalence shows that a distinction between correlation- and SOC-induced topological states may not be that important after all, and similar analogies for the interaction driven phase to the SOC has already been discussed in other topological systems (36); (37). Here, if we regard this term as an intrnsic SOC and interplay with the NN inter-orbital Coulomb interaction (), this results the imaginary part of the orbital order emerges earlier as increase.

To see whether phase II is protected by band topology, we plot the edge states of the slab geometry in Fig. 2(a). The calculated edge states along the (1,0) direction show two surface Dirac cones at =. Furthermore, these edge states are robust against the TR perturbation up to a critical value of roughly 3, although the position of the surface Dirac cones evolves away from (see also Fig. 2(b) for ). One may tend to claim that phase II of the TMI is a conventional quantum-spin Hall insulator, since the TR symmetry is respected by the mean-field Hamiltonian for phase II with . However, this cannot be reconciled with the fact that in our case the number of pairs of degenerate edge states is even instead of odd, as is the case for the quantum spin-Hall insulator. Consequently, we claim that phase II has topology different from earlier work (4) and is a new type of topological phase in 2D, protected by mirror reflection symmetries (spinful). This is accomplished by TR symmetry (spinful) and reflection symmetry (spinless), as indicated by the even mirror Chern number  (14); (11). is related to the spin Chern number of the occupied band with spin up/down, , and given by , as opposed to the quantum-spin Hall insulator in the Kane-Mele lattice model with odd mirror Chern number . In the next section, we propose a mirror Pfaffian with a invariant to connect the nontrivial topology of the TI protected states with their mirror symmetries in phase II.

#### Mirror-Z2 topological invariant.−

The spinful mean-field Hamiltonian of phase II respects the TR symmetry. However, the number of degenerate Dirac cone pairs at the edges is even instead of odd in addition to an even number of Dirac cones in the bulk BZ. This is in sharp contrast to quantum-spin Hall insulators, which are solely protected by the TR symmetry and other spatial symmetries such as inversion symmetry. A detailed symmetry analysis (see the SM) reveals that the topological phase II of the TMI satisfies the mirror symmetry under the combination of space () and time (spin) operations, , in which the mirror operator is given by . The operator performs the complex conjugation identical to the TR operation for spinless fermions. The generalized parity operator exchanges two orbitals, while is responsible for the spin flip under the TR operation . The overall operation is equivalent to mirror reflection (including spin sectors) with respect to the principle axis or .

Analogous to the analysis in the Kane-Mele model for the quantum-spin Hall insulator (4), we introduce a mirror-invariant Pfaffian for occupied states to quantify the invariant of the “even/odd parity” of the spinful Hamiltonian with the mirror symmetry . Specifically, we define the mirror-invariant Pfaffian to measure the band topology as

 P(k) ≡Pf[⟨um(k)|M|un(k)⟩], (4)

where are two occupied orthogonal eigenstates of the Hamiltonian , e.g., and or vice versa. The commutation relation holds for points belonging to the “even parity” subspace along the boundaries of the four quadrants of the BZ. Therefore, the two occupied eigenstates and are identical states up to a phase factor. As a result, the absolute value of the Pfaffian , with along the and axes is unity, . On the other hand, points belonging to the “odd parity” subspace, given by the roots of the Pfaffian, satisfy the anti-commutation relation . Here the mirror operation turns one occupied state, for example, at into an unoccupied and orthogonal eigenstate at , and vice versa, with vanishing Pfaffian for the occupied states at .

In Fig. 2(d) we show four vortices appearing in the Pfaffian for phase II with opposite vorticity in adjacent quadrants of the BZ. All four vortices are well separated by the “even parity” subspace along the and axes or the boundaries of the BZ quadrants. Note that for the TMI the even parity subspaces are connected lines which is different from the case of the TI with inversion symmetry (38), where the even parity subspaces are separted points in the BZ.

It is an important question to confirm whether the proposed topological phase is protected by mirror symmetry. For that purpose, we examine the effects of a mirror-symmetry breaking perturbation on . To perform a stability analysis, we introduce an onsite SOC interaction, which might be called an onsite Rashba term, , but is different in nature from the usual off-site Rashba term: The corresponding matrix elements, written in matrix notation in momentum space as , break both the TR and space-time mirror symmetry. In other words, together with , does not commute with anywhere in the BZ, . This symmetry breaking field will destroy the mirror topological phase even though the interaction is infinitesimal. As we expect, the four vortices (Dirac cones) disappear for any nonzero onsite Rashba-like SOC interaction as shown in Fig. 2(f). Consequently, an infinitesimal destroys the degeneracy of edge states and the previously gapless (crossing) edge states become gapped, see Fig. 2(c).

A completely different scenario occurs when the local inter-orbital coupling is turned on adiabatically. For this case, the Pfaffian is plotted in Fig. 2(e). As the strength of increases the positions of the pair of vortices in the upper half-plane of the BZ are modified and move toward the pair in the lower half-plane compared to the onsite SOC case in Fig. 2(d). As we already mentioned before, this trend continues until the vortices disappear at a critical strength before entering the even parity subspace protected by the mirror symmetry. Indeed this corresponds to the stability boundary discussed previouly in the phase diagram in Fig. 1(d) of spinless fermions. Furthermore this observation is consistent with the corresponding degeneracies of edge states as displayed in Fig. 2(b). Therefore, according to the mirror symmetry, which maps the entire k space of the BZ onto one quadrant, a new mirror-Z index can be defined to count the number of vorticies of the Pfaffian in one quadrant of the BZ (see the SM).

#### Conclusion.−

Our work shows that unconventional topological insulators can emerge from Coulomb correlations in real materials with non-local crystal symmetry in the absence of spin-orbit coupling. A remarkable result of our quasi-2D model Hamiltonian is the finding of an even number of pairs of Dirac cones at the edges. In the spinless case, the four edge states are described by the topological Chern number . On the other hand, in the spinful case of phase I the Chern number is , while in phase II the Chern number is insufficient to classify the topology. In that case, the Pfaffian enumerates the four edge states and is connected to a invariant. Similar to the previous Z invariant in TIs with inversion symmetry (4); (38), the mirror-Z invariant in TMI is robust against TR breaking perturbations, because the topological state is protected by a mirror reflection symmetry.

Similar ideas of the mirror-Chern number (14); (11) and the mirror reflection symmetry of the group (39) have been discussed before. The novalty here is that we have extended these cases to a spinfull Hamiltonian in the absence of spin-orbit coupling and found a new mirror-Z index in phase II.

Finally, our studies provide a new direction toward the realization of correlation-induced topological phases in -orbital material. We suggest to search for TMIs in the paramagnetic iron-pnictide compounds with crystallographic 11, 111, 122 and 1111 structures near half-filling (40), where the model is expected to be valid. In view of recent interest in superconducting topological phase (41), the interplay of our orbital order proposed here and superconductivity will be of a very interesting topic for future study.

#### Acknowledgements.−

We acknowledge Fan Zhang, J. Ren, G.-W. Chern and Tanmoy Das for sharing their views on topological insulators. The work at Houston was supported in part by the Robert A. Welch Foundation under Grant No. E-1146 and AFOSR under Grant No. FA9550-09-1-0656. The work at Los Alamos was supported by the U.S. DOE Contract No. DE-AC52-06NA25396 through the LDRD program (Y.-Y.T., C.-C.J.W.), the Office of Basic Energy Sciences (BES), Division of Materials Sciences and Engineering (M.J.G., J.-X.Z.). This work was supported in part by the Center for Integrated Nanotechnologies, a DOE BES user facility.

## I Supplemental Material: Emergent topological mirror insulator in t2g-orbital systems

In this Supplemental Material, we provide the additional technical information and details used in the main text of the publication. The following sections contain the in-depth description of the model, supporting numerical calculations, and symmetry analysis:

1. Crystal field splitting of orbitals

2. Mean-field lattice Hamiltonian of correlated electron system

3. Inter-orbital current order of the flux phase

4. Lifting of ground state degeneracy with exchange interaction

5. The Hamiltonian in momentum representation

6. Two types of orbital order with C symmetry

7. Vortices as generators of the Berry flux and Chern numbers

8. Symmetry analysis of the time and mirror invariance

## Ii Crystal field splitting of d orbitals

For a transition-metal (TM) atom situated in crystal fields due to surrounding ions in compounds, the atomic levels align differently. The five orbitals include , and . In a spherical crystal field, the orbitals are all degenerate. For a TM atom caged by four ligand atoms, a tetrahedral crystal field splits the five orbitals into and manifolds, where levels are higher in energy. With a tetragonal crystal field distortion, introduced by anisotropic strains or effective strains with different types of the surrounding ligand atoms, part of the degeneracy of the orbitals is lifted with the level energetically separated from and orbitals, which are the relevant degrees of freedom we are interested in. (42) The level will be higher in energy for uniaxial stretching strain along the axis and lower for compression as shown in Fig. (3).

## Iii Mean-field lattice Hamiltonian of correlated electron system

In this section, we derive the mean-field form of the Coulomb interaction. The Hamiltonian of interest is , where the superscript indicates that we include the spin degrees of freedom. In real-space the lattice Hamiltonian is given by

 H0 =∑IJ,αβ,σ(tαβIJ−μδIJδαβ)c†Iα,σcJβ,σ, (5) HU =U∑I,α,σnIα,σnIα,¯σ, HJh =U′∑IσαnIα,σnI¯α,¯σ+(U′−Jh)∑IσαnIα,σnI¯α,σ, HV =∑I≠J,α,σVIJnIα,σnJ¯α,σ,

where . Here is the kinetic term describing the hopping of electrons in the 1-Fe per unit cell formulation.(44); (45) The “bar” above subscripts stands for the opposite orbital or spin component, i.e., and . In our two-orbital model with the non-zero hopping parameters are chosen to describe the generic pnictide BaFeAs for purpose of illustration,

 t1= tαα±^x=tαα±^y=0.09, (6) t2= t11±(^x−^y)=t22±(^x+^y)=0.08, t3= t11±(^x+^y)=t22±(^x−^y)=1.35, t4= tα¯α±(^x±^y)=−0.12, t5= tα¯α±^x=tα¯α±^y=−1, t6= tαα±2^x=tαα±2^y=0.25.

The interaction part is captured by the terms and , which are the on-site intra-orbital Hubbard interaction and Hund’s coupling, as well as the term , which is the inter-orbital () Coulomb interaction between lattice sites and . Note that we also considered intra-orbital offsite Coulomb interaction (), but found no interesting topological phases. Thus, we will not further consider the intra-orbital interaction in the current work. By investigating and in the mean-field approximation, we can test whether there exist new and anomalous ground states due to the inter-orbital Coulomb interaction, although the incorporation of quantum fluctuations may change the details of such a phase diagram. For the on-site inter-orbital we write in standard mean-field approximation

 HU=U∑Iα,σ≠σ′⟨nIασ⟩nIασ′. (7)

On the other hand, we have at least two possibilities for the inter-orbital to decouple the fermionic operators within mean-field theory, namely , where

 HCDW =∑I≠J,α,σVIJ⟨nIασ⟩nJ¯ασ, (8) HAOH =−∑I≠J,α,σVIJ⟨c†Iα,σcJ¯α,σ⟩c†J¯α,σcIα,σ.

In this work, we simplify the Coulomb coupling and include only the nearest-neighbor (NN) interaction with =. In the 2D-periodic calculation of the bulk material, the CDW term is not a stable ground state and only has a stable solution at a finite value of , see Fig. 4. Unlike the Bardeen-Cooper-Schrieffer (BCS) theory, where any non-zero pairing strength will lead to superconductivity, here a threshold has to be overcome to induce long-range orbital order. Naturally, this makes it more challenging to find materials with orbital-ordered ground states. We also checked numerically for magnetism and found that a Hubbard term with and does not induce long-range magnetic order in our simple, low-energy two-orbital model. While one might expect for real materials that the inter-site Coulomb interaction is less than the onsite interaction, , we consider for illustrational purposes of the anomalous orbital effect the opposite case, when we show results for strong coupling with . This also corresponds to the region of the interaction phase diagram where no magnetic order emerges.

## Iv Inter-orbital current order of the flux phase

Here, let us define the mean-field inter-orbital current order parameter of the flux phase due to the anomalous orbital Hall term ,

 χα,σIJ≡V1⟨c†Iα,σcJ¯α,σ⟩. (9)

The mean-field calculated nearest-neighbor order parameter is a complex number with real and imaginary parts, , where we define,

 δt =−|Reχα¯α,σIJ|, (10) λAOH =|Imχα¯α,σIJ|.

Note that the first term, , is always negative and homogeneous in real space. It will contribute to through its NN inter-orbital hopping term . The second term, , is the generator of the anomalous orbital Hall effect, which is of key interest in this work. Consequently, the term can be written in a more compact form,

 HAOH=iϵσλAOH∑IJ,α,σνα¯αIJc†Iα,σcJ¯α,σ, (11)

where and the elements of the tensor describe the direction of the NN inter-orbital currents as shown in Fig. 1(a) of the main text with , and . The real part of has been absorbed into the NN inter-orbital hopping terms, . Figure 4(b) shows that the hole pockets are shifted downward at the and points, due to the contribution of alone. Once the purely imaginary part is included, the degeneracy of the bulk Dirac cone near the point in the Brillouin zone (BZ) is lifted immediately, as shown by the opening of a gap in Fig. 4(c). The gap increases with increasing value of as further shown in Fig. 4(d).

## V Lifting of ground state degeneracy with exchange interaction

In the main text, we have shown that without introducing other terms, the ground state of leads to two degenerated phases, namely . In order to discuss the possibility of lifting this degeneracy, we introduce a perturbation caused by an additional Hund’s coupling term, to Eq. (5), as discussed by Sano(43), .

 HJh2=−Jh∑Iα(c†I,α,↑cI,α,↓c†I,¯α,↓cI,¯α,↑+h.c). (12)

Then we have the mean-field decoupled Hamiltonian ,

 HJh2=λJh∑Iα(c†I,α,↑cI,α,↓+c†I,¯α,↓cI,¯α,↑+h.c), (13)

where the mean-field order parameter is defined as . Finally, if we insert Eq. (13) into the Hamiltonian and manually assign a real value to , we can confirm that phase II with is the preferred ground state for any small .

## Vi The Hamiltonian in momentum representation

In this section, we Fourier transform into k-space for the 1-Fe per unit cell with the mean-field calculated order parameter and the manually added perturbation terms and . For pedagogical reasons, we focus first on the spin polarized Hamiltonian for the spin component , , where and stand for annihilating electrons on / orbitals. In k-space the Hamiltonian is conveniently written as

 ^H(ϵσ)=E0+→B⋅→τ, (14)

where and is the vector of Pauli matrices spanning the orbital space with In the matrix notation we can write explicitly,

 ^H(ϵa)=(E0+Z,X−iYX+iY,E0−Z). (15)

Note that Eq. (15) has been used in the main text for the calculation of the Chern number.

The ancillary functions , , and are following from the manuscript,

 E0= 2t1[cos(kx)+cos(ky)]+2t6[cos(2kx)+cos(2ky)], (16) +2(t2+t3)[cos(kx)cos(ky)]−μ, X= 4t4[cos(kx)cos(ky)]+2(t5+δt)[cos(kx)+cos(ky)], Y= ϵ(2λAOH[cos(kx)−cos(ky)]+λ1), Z= 2(t2−t3)[sin(kx)sin(ky)]+λ0.

We close this section by expanding the formulation of the spinless Hamiltonian to include spin degrees of freedom. The enlarged spin-orbital space becomes or . In such a notation, an “onsite Rashba” or onsite spin-orbital coupling term enters on the off-diagonal entries of the 44 matrix,

 ^Hs=(^H(ϵ↑=1),iλRσz−iλRσz,^H(ϵ↓=−1)). (17)

Note that Eq. (17) has been used in the main text to calculate the stability of the mirror symmetry of the topological crystalline phase in two dimensions.

## Vii Two types of orbital order locked in a lattice with C4v symmetry

As shown in Fig. 5 the lattice with C symmetry allows two different types of orbital order. The second type can generate Dirac cones along the - direction in the bulk dispersion. In what follows, we focus on the k-dependent inter-orbital hopping energies with C symmetry,

 Hxz,yz=∑k[ϵxz(k)c†xz,kcxz,k+ϵyz(k)c†yz,kcyz,k+⋯, (18)

with

 ϵxz=−2tcos(k⋅→a1), (19) ϵyz=−2tcos(k⋅→a2),

where are orthogonal to each other and indicate the bond direction of the effective hopping term . If we choose and , then this corresponds to type-I order with C symmetry. On the other hand, for and it becomes

 ^Hxz,yz=−2t[cos(kx)cos(ky)+τzsin(kx)sin(ky)], (20)

which gives rise to the term in the spinless Hamiltonian in Eq. (15).

## Viii Vortices as generators of the Berry flux and Chern numbers

The anomalous orbital Hall effect of the spinless Hamiltonian with Chern number is the combined effect of functions , and . The Chern number can be calculated directly through the area integration of the Berry curvature(51)

 Ω(k)±=i⟨±|∂^H(k)∂kx|∓⟩⟨∓|∂^H(k)∂ky|±⟩−(kx↔ky)(E±−E∓)2. (21)

In viewing the symmetry of , we know that the Dirac cones can be re-defined by taking any two components of (). To be specific, let us consider the simplified spinless Hamiltonian in Eq. (2) of the manuscript with interaction renormalized coefficients,

 ~E0= 0, (22) ~X= tx(4t4[cos(kx)cos(ky)]+2t5[cos(kx)+cos(ky)]), ~Y= ty(2λAOH[cos(kx)−cos(ky)]), ~Z= tz(2(t2−t3)[sin(kx)sin(ky)]).

Here can be regarded as an energy shift, because does not enter the wave function it does not contribute to the calculation of the topological invariant. In the main text, four Dirac cones are generated in the bulk bands with renormalized hopping parameters , where the band energy becomes . The four Dirac cones are located around the point of the BZ, satisfying the criterion . A key result of this work is that a non-zero function is a necessary, but not sufficient condition for a topological ground state. It can be viewed as a perturbation, where the massless fermions acquire mass and a gap opens at the Dirac cones. Here, if we choose the parameters , then the dispersion becomes , and the four Dirac cones are located in the BZ at . In addition, when we turn on a small or , then the degeneracy at the Dirac cones is lifted and the calculated Chern number becomes for each band. The corresponding Berry curvature is shown as a color-map image in Fig. 6(a), where four high intensity spots can be found at the positions of the Dirac cones. If we regard these four Dirac cones as topological defects of a generalized magnetic field in the Hamiltonian acting on the pseudo-spin degrees of freedom, that is, , where , then it is rather straightforward to map out the (in-plane) and (out-of-plane) components in Figs. 6(b) and (c). Knowing these two-dimensional (2D) vector maps, one can graphically solve for the Chern number by mapping the vector around each singularity onto the Bloch sphere, following the procedure outlined by Bernevig (46). Thus the -field winding around each topological defect (Dirac cone) contributes the winding number , where the factor one-half stems from the spin . Hence the spinless Hamiltonian has the total Chern number .

## Ix Symmetry analysis of the time and mirror invariance

In this section, we discuss the symmetry classification of phases I and II of the spinful Hamiltonian with spin degrees of freedom. A detailed account of the symmetry operators used in the main text is given. The spinful Hamiltonian in k-space of fermions is defined by , where . We re-write as a direct tensor product of Pauli matrices in the combined orbital pseudo-spin and spin spaces, , where the orbital flux term, , of phases I and II is either or . Here is the unity matrix in spin space.

### ix.1 Intrinsic inversion symmetry and TR symmetry violation of spinless ^H

We start our symmetry analysis by noting that the quasi-2D Hamiltonian of spinless fermions, , in a tetragonal system has intrinsic inversion symmetry . This corresponds to a rotation in the - plane. Moreover, for the spinless Hamiltonian the time-reversal (TR) operator is given by the charge conjugation operator, , and satisfies the relation

 T^H[λAOH](k)T−1= ^H[−λAOH](−k) (23) = ^H[−λAOH](k),

which tells us that the TR symmetry is violated, because it reverses the orbital current direction from . Hence it is not too surprising that the TI is quite robust against perturbations and , which break TR symmetry, too.

### ix.2 Reflection invariance of spinless ^H

In addition to the inversion symmetry, the spinless quasi-2D Hamiltonian is invariant under reflections. The two reflection axes and obey the parity operation

 P^H(kx,ky)P−1=^H(±kx,∓ky), (24)

respectively, with . This statement is universally true for our model Hamiltonians and applies also to phases I and II of the spinful Hamiltonian. Note that in 2D the parity operation is a reflection which only acts on orbital degrees of freedom.