# Unconventional topological phase transition in two-dimensional systems with space-time inversion symmetry

## Abstract

We study a topological phase transition between a normal insulator and a quantum spin Hall insulator in two-dimensional (2D) systems with time-reversal and two-fold rotation symmetries. Contrary to the case of ordinary time-reversal invariant systems where a direct transition between two insulators is generally predicted, we find that the topological phase transition in systems with an additional two-fold rotation symmetry is mediated by an emergent stable two-dimensional Weyl semimetal phase between two insulators. Here the central role is played by the so-called space-time inversion symmetry, the combination of time-reversal and two-fold rotation symmetries, which guarantees the quantization of the Berry phase around a 2D Weyl point even in the presence of strong spin-orbit coupling. Pair-creation/pair-annihilation of Weyl points accompanying partner exchange between different pairs induces a jump of a 2D topological invariant leading to a topological phase transition. According to our theory, the topological phase transition in HgTe/CdTe quantum well structure is mediated by a stable 2D Weyl semimetal phase since the quantum well, lacking inversion symmetry intrinsically, has two-fold rotation about the growth direction. Namely, the HgTe/CdTe quantum well can show 2D Weyl semimetallic behavior within a small but finite interval in the thickness of HgTe layers between a normal insulator and a quantum spin Hall insulator. We also propose that few-layer black phosphorus under perpendicular electric field is another candidate system to observe the unconventional topological phase transition mechanism accompanied by emerging 2D Weyl semimetal phase protected by space-time inversion symmetry.

###### pacs:

Introduction. Symmetry protected topological phases have become a quintessential notion in condensed matter physics, after the discovery of time-reversal invariant topological insulators (1); (2); (3) and topological crystalline insulators (4). Although the importance of symmetry to protect bulk topological properties is widely recognized, relatively little attention has been paid to understand the role of symmetry for the description of topological phase transition (TPT). Early studies on this issue have focused on time-reversal and inversion , and have shown that the nature of TPT in -invariant three-dimensional (3D) systems changes dramatically depending on the presence or absence of symmetry (5). Namely, when the system has both and symmetries, a direct transition between a normal insulator (NI) and a topological insulator is possible when a band inversion happens between two bands with opposite parities. Whereas in noncentrosymmetric systems lacking , the transition between a NI and a topological insulator is generally mediated by a 3D Weyl semimetal (WSM) phase in between. The intermediate stable semimetal phase can appear when the following two conditions are satisfied. Firstly, the codimension analysis for accidental band crossing at a generic momentum should predict a group of gapless solutions. Secondly, a gapless point in the semimetal phase should carry a quantized topological invariant guaranteeing its stability.

Contrary to 3D, in two-dimensions (2D), the codimension analysis (6) predicts that there always is a direct transition between a NI and a quantum spin Hall insulator (QSHI) even in noncentrosymmetric systems (See Fig. 1(a)). The reason is that, in a generic 2D system with a single tuning parameter (representing pressure, doping, etc.), an effective Hamiltonian describing band crossing depends on three independent variables including two momenta and . To achieve a band crossing, however, since the coefficients of three Pauli matrices associated with the effective Hamiltonian should vanish by adjusting three variables, only a single gap-closing solution can be found. This unique gap-closing solution describes the critical point for a direct transition between two gapped insulators. The absence of a stable semimetal phase mediating the transition between two insulators is consistent with the fact that a gap-closing point does not carry a topological invariant in a generic -invariant 2D system in the presence of spin-orbit coupling.

In this Letter, we show that the TPT between a NI and a QSHI is always mediated by an emerging 2D Weyl (7) semimetal (See Fig. 1(b)), when a -invariant noncentrosymmetric 2D system is invariant under two-fold rotation about an axis perpendicular to the 2D plane. The intermediate 2D WSM is stable due to the Berry phase around a Weyl point (WP), which is quantized even in the presence of spin-orbit coupling. Here the central role is played by the so-called space-time inversion which is nothing but the combination of and , i.e., . Since ensures the quantization of the Berry phase around a 2D WP, the transition between an insulator and a WSM is accompanied by pair-creation and pair-annihilation of 2D WPs. Moreover, partner exchange between pairs of WPs can induce the change of the topological invariant, thus the 2D WSM can mediate a TPT. We propose two candidate materials where the unconventional TPT mediated by a 2D WSM can be realized. One is the HgTe/CdTe quantum well where inversion is absent intrinsically (9); (10); (11); (12) and the TPT can be controlled by changing the thickness of the HgTe layer. We expect that there can be a finite thickness window where a stable 2D WSM appears between a NI and a QSHI. Also we propose that the unconventional TPT can be observed in few-layer black phosphorus under vertical electric field.

Band crossing in systems with . transforms a spatial coordinate to . Since the -coordinate is invariant under , for a layered 2D system with a fixed , can be considered as an effective inversion symmetry mapping to . Thus, in 2D systems, transforms a space-time coordinate to . In momentum space, on the other hand, it is a local symmetry since the momentum remains invariant under . As discussed in Ref. (8), has various intriguing properties. For instance, since Berry curvature transforms to under , vanishes locally unless there is a singular gapless point, which guarantees the quantization of Berry phase around a 2D WP. Moreover, since irrespective of the presence/absence of spin-orbit coupling, it does not require Kramers degeneracy at each . This can be contrasted to the case of , satisfying in the presence (absence) of spin-orbit coupling. Especially, when , Kramers theorem requires double degeneracy at each . Since Berry phase is not quantized in this case, a Dirac point is unstable (13).

Here we show that also modifies the gap-closing condition in an essential way, leading to an unconventional TPT. Since each band is nondegenerate at a generic momentum in noncentrosymmetric systems, an accidental band crossing can be described by a matrix Hamiltonian where indicates the two bands touching near the Fermi level and describes a tuning parameter such as electric field, pressure, etc. Since is antiunitary, it can generally be represented by where is a unitary matrix and denotes complex conjugation. By choosing a suitable basis, one can obtain as shown in Supplemental Materials. Then the symmetry requires , which leads to

(1) |

Accidental gap-closing can happen if and only if . Since there are three independent variables whereas there are only two equations to be satisfied, one can expect a line of gapless solutions in space, which predicts an emerging 2D stable semimetal (See Fig. 1(b)). Near the critical point where accidental band crossing happens, the Hamiltonian can generally be written as

(2) |

which describes a gapped insulator (a 2D WSM) when () assuming . Due to symmetry, accidental band crossing happens at two momenta . Since two WPs are created at each band crossing point, the WSM has four WPs in total. Moreover, when becomes larger than , four WPs migrate in momentum space, and eventually, they are annihilated pairwise at . Interestingly, when pair-annihilation/pair-annihilation is accompanied by partner-switching between WP pairs, the two gapped phases mediated by the WSM should have distinct topological property as shown below.

Change of invariant via pair-creation/pair-annihilation of 2D WPs. The invariant of a -invariant 2D system can be written as (14)

(3) |

where is the time-reversal polarization of a -invariant one-dimensional (1D) subsystem connecting two time-reversal invariant momenta (TRIM) with given . For instance, Fig. 2(a) shows -invariant 1D subsystems passing two TRIMs with or , respectively. For such a 1D subsystem, is defined as

(4) |

where is the charge polarization, and and are the partial polarization associated with the wave function and its Kramers partner where labels occupied bands. Namely, with . can also be written as a summation of Wannier function centers such as using the Wannier function (14); (15); (16). In general, can take any real value, modulo an integer, whereas is an integral quantity whose magnitude is gauge dependent. Thus in a generic 1D -invariant system, among , , , none of them can serve as a topological invariant.

However, in the presence of additional symmetry, becomes a topological invariant (17). Namely, becomes quantized and gauge-invariant modulo 2 when exists. Since makes to take a quantized value, either or modulo an integer, naturally becomes a quantity. is also proposed as a topological invariant in a -invariant 1D system with mirror symmetry in Ref. (24).

Now let us explain how the pair-creation/pair-annihilation of 2D WPs can change the invariant given by

(5) |

In an insulating phase, since the Chern number of the whole system is zero, one can choose a continuous gauge in which

(6) |

where EBZ indicates the effective half-Brillouin zone bounded by two -invariant 1D systems defined above, and we have used that Berry curvature due to . Thus the change of is simply given by . Since is a topological invariant, it can be changed only if an accidental gap-closing happens in the relevant 1D -invariant subsystem. In the process shown in Fig. 2(b) where 2D WPs pass through the 1D subsystem with , we have

(7) |

where is a closed loop encircling one WP shown in Fig. 2(c). Since a 2D WP has -Berry phase, () if encloses an odd (even) number of WPs. Hence the invariant can be changed by 1 via partner exchange between two pairs of 2D WPs.

Model Hamiltonian. To demonstrate the unconventional TPT, we construct a simple model Hamiltonian, a variant of Bernevig-Hughes-Zhang (BHZ) model, which is originally proposed to describe QSH effect in a HgTe/CdTe quantum well (25); (26). The BHZ model is defined on a square lattice in which each site has two s-orbitals , and two spin-orbit coupled p-orbitals , . Nearest-neighbor hopping between these four orbitals gives a tight-binding Hamiltonian given by

(8) |

where , , , and the Pauli matrices () denote the orbital (spin) degrees of freedom. describes a QSHI (a NI) when (). The TPT can also be understood from the energy eigenvalues of , . The energy gap closes when three equations are simultaneously satisfied, which uniquely determines the three variable when .

is invariant under inversion , four-fold rotation about axis , two-fold rotation about , axis , , and time-reversal . However, the real HgTe system lacks and while retaining their product symmetry as well as , , , and . In fact, once is absent, a constant term is allowed. Then the resulting Hamiltonian is invariant under , , , , and . The energy eigenvalues of are . Since gap-closing requires only two conditions, and to be satisfied, one can expect a line of gapless solutions in space describing a WSM. Generally, the solution of each gap-closing condition forms a closed loop in momentum space. In Fig. S6, we plot the evolution in the shape of two loops which describe the momenta satisfying and , respectively. Only when these two loops overlap, the band gap closes at the momentum where two loops touch, thus the system becomes a WSM. We find that, when , two loops overlap at eight points indicating an emergent WSM having 8 WP. Here and . On the other hand, when or , two loops do not overlap, thus the sytem is a gapped insulator. At the critical point with , the band gap closes at four points related by , and each splits into two WPs in the WSM phase. This result demonstrates that the TPT in a HgTe/CdTe quantum well can be mediated by an intermediate 2D WSM. The occurrence of a semimetal phase in the HgTe/CdTe quantum well is also proposed in Ref. (11) based on symmetry analyses. Let us note that the number of gap-closing points at the critical point depends on the symmetry of the system. For instance, once symmetry is broken by applying uniaxial strain, one can observe two gap-closing points at the critical point and the intermediate WSM has four WPs. (See Supplemental Materials.) However, irrespective of the number of gap-closing points, the WSM can mediate a TPT as long as the trajectory of WPs forms a single closed loop.

To prove that the 2D WSM mediates a TPT, we should compare the invariant of two insulating phases existing when and , respectively. For this purpose, we first compute the energy spectrum of a strip structure having a finite size along one direction. As shown in Fig. S7 (b), when , one can clearly observe helical edge states localized on the sample boundary, which is absent when . Thus the system is a QSHI (a NI) when (). For further confirmation, we directly compute numerically. Since inversion symmetry is broken, one cannot use parity eigenvalues to evaluate (27). Instead we determine by computing the change of the time-reversal polarization between and by using Eq. (3). Since is given by the difference in the Wannier function centers of Kramers pairs, one can determine by examining how Wannier function centers of Kramers pairs evolve between and (14); (28). As shown in Fig. S7 (c), when (), one can see partner-switching (no partner-switching) of Wannier functions when changes from 0 to . Since the partner-switching (no partner-switching) between Wannier states indicates the change of by 1 (0), one obtains () when ().

Discussion. The unconventional TPT mediated by a WSM can generally occur in any 2D noncentrosymmetric system with symmetry. Similar to the BHZ model including an inversion breaking term, we have found that the Kane-Mele model on the honeycomb lattice including Rashba coupling also undergoes a TPT mediated by a 2D WSM when uniaxial strain is applied (See Supplemental Materials). Among real materials, we propose few-layer black phosphorus as an another candidate system since its band gap can be controlled by electric field breaking inversion (29). Although there are several theoretical proposal for possible TPT in this system (30); (31); (32), the unconventional mechanism we propose has never been discussed. In fact, a recent experiment (33) has shown that the band gap of this system can be controlled by doping potassium on the surface, thus the insulator-semimetal transition can be realized. Also the presence of 2D WPs in the semimetal phase is observed in a first-principles calculation (34). We expect, when stronger electric field is applied to the WSM phase, even a QSHI can be obtained. For confirmation, we have studied a tight-binding model describing black phosphorus under vertical electric field, and have shown that the unconventional TPT can occur. (See Supplemental Materials for details.) Since the presence of with broken inversion is the only requirement to realize the novel TPT, the same phenomenon may occur in various 2D materials with such as the puckered honeycomb structure of arsenene, antimonene, bismuthene (35), the dumbbell structure of germanium-tin, stanene (36), and the bismuth monobromide (37), etc.

We conclude with the discussion about electron correlation and disorder effects on the TPT. Although 2D WSM is perturbatively stable against weak interaction, sufficiently strong interaction can induce nontrivial physical consequences. For instance, a phase breaking symmetry is proposed to appear between a NI and a QSHI due to interaction (38). Especially, at the critical point between the WSM and an insulator, since the energy dispersion becomes anisotropic, i.e., linear in one direction and quadratic in the other direction (See Eq. (2)), the density of states shows with the energy , contrary to in the WSM with linear dispersion in two directions. Such an enhancement of makes the electron correlation and disorder to cause nontrivial physical consequences. For instance, a recent renormalization group study (39) has shown that quantum fluctuation of anisotropic Weyl fermions makes the screened Coulomb interaction to have spatial anisotropy, which eventually leads to marginal Fermi liquid behavior of low energy quasi-particles. Also, in the presence of disorder, a disorder-induced new semimetal phase can appear between the insulator and WSM (40). Understanding the interplay of electron correlation and disorder is an important problem, which we leave for future study.

Acknowledgement. J. Ahn was supported by IBS-R009-D1. B.-J. Y was supported by IBS-R009-D1, Research Resettlement Fund for the new faculty of Seoul National University, and Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (Grant No. 0426-20150011). We thank Keunsu Kim, Akira Furusaki, Takahiro Morimoto for useful discussion.

## Appendix SI 1 Derivation of

In this section, we show that can be expressed as the complex conjugation via a unitary transformation of the basis . Since space-time inversion is antiunitary, in general, it can be written as

(S1) |

where is a unitary operator. From the property we see that is symmetric. Then can be written in the form of

(S2) |

where is real and symmetric. The unitary part of space-time inversion transform as

(S3) |

under the change of basis , which follows from Especially for orthogonal group elements, i.e., ,

(S4) |

Since is a real symmetric matrix, it can be diagonalized by an orthogonal transformation of basis. In general, for bands after the change of basis. The phases can be erased by a phase rotation so that we have . As holds for both spinless and spinful systems, the conclusion of this section does not depend on the existence of spin degrees of freedom.

## Appendix SI 2 Berry Connection, Berry Curvature, and Sewing Matrices

### A Berry Connection and Berry Curvature

The non-abelian Berry connection and Berry curvature are defined by

(S5) |

and

(S6) |

where are the band indices. The matrix elements and are considered three dimensional vectors in this definition of Berry curvature while the momentum lives in a two-dimensional space.

The part of the non-abelian Berry connection, i.e., the abelian Berry connection which is just called the “Berry connection” in many cases, is written in italic letter.

(S7) |

and

(S8) |

Consider a gauge transformation

(S9) |

where . The non-abelian Berry connection and curvature transform under the gauge transformation as

(S10) |

and

(S11) |

which follows from their definition.

The transformation of part is then found after taking trace over the above transformations.

(S12) |

and

(S13) |

### B Berry Connection under , , and Symmetries

Consider the case where both and symmetry is present. Sewing matrices and are defined as

(S14) |

They are unitary matrices because and are (anti)unitary operators which preserves the norm of wavefunctions. We see explicitly that

(S15) |

and

(S16) |

Using and unitarity of and matrices,

(S17) |

Thus we have

(S18) |

and relates the non-abelian Berry connections at time-reversal momenta.

(S19) |

The space-time inversion is the combination of and .

(S20) |

Unitarity and the following relations also follow from that of and .

(S21) |

The above relations can be derived noting that so that . gives a constraint on the non-abelian Berry connection which is a combination of and constraints.

(S22) |

The part of the Berry connection satisfies

(S23) |

and

(S24) |

combining the above two. The Berry connection is locally a pure gauge, and thus the Berry curvature is trivial.

### C Gauge transformation of the sewing matrices , , and

Consider a gauge transformation

(S25) |

Left and right hand side of each of the equation

(S26) |

is then expressed in the new basis as

(S27) |

so that we have

(S28) |

We have found the transformation of and matrices.

(S29) |

From this we find the gauge transformation of matrix.

(S30) |

## Appendix SI 3 Proof of change by a partner exchange of Weyl pairs

### A as a Topological Invariant of the 1D Subsystem with and Symmetries

We show here that is a topological invariant of a 1D subsystem with and symmetries where . First, we show it is quantized to an integer. In a time-reversal invariant one-dimensional system, we can define the time-reversal polarization

(S31) |

which is gauge dependent, but quantized to an integer.(14) In the presence of symmetry, the charge polarization is also quantized.

(S32) |

because at any TRIM due to Kramers pairs having eigenvalue pairs and under .

Thus is quantized to an integer.

(S33) |

Now we show the gauge invariance of . Under the gauge transformation , partial polarization transforms as

(S34) |

where is a partial trace over for all , , and the winding number is an arbitrary integer. Thus is invariant mod 2. Invariance under non-abelian gauge transformation can be seen from the expression

(S35) |

presented by Fu and Kane(14). The abelian Berry connection is manifestly invariant. for . The matrix transformation as under . Using the identity , the Pfaffian part is also invariant.

This invariant was first noted by Lau, van den Brink, and Ortix (24) as a topological invariant of the topological mirror insulator where . Notice that the two-fold rotation reduces to a mirror operation on any time-reversal invariant 1D subsystem.

### B Strategy of the Proof

We calculate the change of invariant due to movements of Weyl points by using the formula

(S36) |

where () is the time-reversal polarization of a TRI 1D system passing through the two TRIM points with (). In an insulating phase, we can have a continuous gauge as Chern number in the whole BZ is zero. In the gauge, we have

(S37) |

where from the spacetime inversion symmetry. EBZ is the effective Brillouin zone of a time-reversal system which is a half Brillouin zone with TRI 1D systems at and as its boundary. Thus the invariant is expressed as a partial polarization pump

(S38) |

in a continuous gauge when the spacetime inversion symmetry is present. The two times partial polarization will change only if the TRI 1D system closes a gap since it is a topological invariant. From now on, we will prove that when two Weyl point whose position in BZ is related by time-reversal cross the TRI 1D system, changes by one so that partner exchange of Weyl points leads to a change of invariant. Such crossing occurs even number of times in a partner preserving process so that the invariant does not change. [See Fig. S1(a,b).]

### C Change of the partial polarization at the gap closing

We take a time-reversal invariant 1D systems avoiding pair creation and annihilation as in Fig. S1. The gauge is chosen so that the wavefunction is discontinuous only at the singularities. It is possible because there is no Berry curvature punching out singularities. In this gauge, we can use the expression

(S39) |

The time-reversal invariant curve can be deformed adiabatically to the line after Weyl points pass through. [See Fig. S1(c).] Thus we calculate the change of (two times) partial polarization by the crossing of Weyl points as the difference of it between the curve and .

(S40) |

where is the part of the curve , we used the single-valuedness of the sewing matrix at and Y points in the fourth line, and used the property of Weyl points at the last line. We have now proven that the invariant changes through a partner exchange.

## Appendix SI 4 Alternative Proof

### A Alternative Proof by 3D Embedding

Here we present an alternative proof of the change of invariant due to the pair change of Weyl points. This proof starts by embedding the 2D BZ of our system into a 3D BZ of a system with the same symmetry, and . After a suitable embedding, the proof goes by following the analysis of Murakami et al. (5). Let plane of the 3D BZ be the 2D BZ of our system. The 2D Weyl points will be lifted into 3D Weyl points rather than a line node in general because there is no protection of Berry flux out of the invariant plane, i.e., out of the or plane. In fact, we can choose an embedding to get 3D Weyl points. Next, consider the deformed time-reversal invariant two-dimensional system illustrated as the orange sheet in Fig. S2. In this construction, a Weyl point passes through the EBZ of the orange sheet. This change the invariant of the orange sheet as claimed by Murakami et al.(5) and it will be proved explicitly below. The sheet can be adiabatically deformed into before/after the creation/annihilation of Weyl points. Thus the change of invariant of the 2D system at is the same as the change of the orange sheet, and the proof ends.

### B Explicit Proof of the Claim by Murakami and Kuga

For completeness, we prove the statement in Phys. Rev. B78, 165313. In the paper, the authors claimed that the invariant of a 2D TRI BZ changes when a Weyl point pass through its EBZ, noting that the Berry flux of the half-BZ determines invariant (up to some extra term) as proved by Moore and Balents (18). The extra term is the Berry flux of the extended part which is added to make EBZ closed, i.e., homotopically equivalent to a sphere. It is not obvious why the extra term does not contribute to the change without having analytic expression of it. An explicit proof has not been presented anywhere while the statement is believed to be valid. On the other hand, Fu and Kane argued in the appendix of Phys. Rev. B74, 195312 that the extra term corresponds to the pump of . In this subsection, we show using the expression of the 2D invariant as a time-reversal polarization pump that the invariant changes when a 3D Weyl points pass through it on a EBZ. This proof will clarify why is it possible to calculate the change of 2D invariant from the Berry flux jump within a EBZ.

Consider the the Brillouin zone of a time-reversal invariant system as in Fig. S3-(a,b). symmetry is not assumed. The Blue balls are Weyl points with positive chirality. The orange sheet is a deformed 2D time-reversal invariant sub-BZ sharing the same EBZ boundary with the plane. The lines are also common. Weyl points did not pass through the plane yet. After Weyl points cross, the orange sheet can be adiabatically deformed to plane. Thus the difference of the