# Model Hamiltonian and Time Reversal Breaking Topological Phases of Anti-ferromagnetic Half-Heusler Materials

## Abstract

In this work, we construct a generalized Kane model with a new coupling term between itinerant electron spins and local magnetic moments of anti-ferromagnetic ordering in order to describe the low energy effective physics in a large family of anti-ferromagnetic half-Heusler materials. Topological properties of this generalized Kane model is studied and a large variety of topological phases, including Dirac semimetal phase, Weyl semimetal phase, nodal line semimetal phase, type-B triple point semimetal phase, topological mirror (or glide) insulating phase and anti-ferromagnetic topological insulating phase, are identified in different parameter regions of our effective models. In particular, we find that the system is always driven into the anti-ferromagnetic topological insulator phase once a bulk band gap is open, irrespective of the magnetic moment direction, thus providing a robust realization of anti-ferromagentic topological insulators. Furthermore, we discuss the possible realization of these topological phases in realistic anti-ferromagnetic half-Heusler materials. Our effective model provides a basis for the future study of physical phenomena in this class of materials.

## I Introduction

The discovery of time reversal invariant topological insulators
(TIs) Qi and Zhang (2011); Hasan and Kane (2010) provides us the first example of a novel topological state
that is protected by certain types of symmetry (time reversal symmetry),
and greatly deepens our understanding of the role of symmetry
and topology in electronic band structures of solid materials.
Soon after this discovery, the idea of symmetry protected topological
states is generalized to other systems,
leading to different topological states, including
topological crystalline insulatorsFu (2011); Lin *et al.* (2010a); Tanaka *et al.* (2012); Xu *et al.* (2012); Hsieh *et al.* (2012) that are protected by
crystalline symmetry, topological superconductorsAlicea (2012); Schnyder *et al.* (2008); Lutchyn *et al.* (2010); Beenakker (2013) which
require particle-hole symmetry, and topolgical semimetals (TSMs)Yan and Felser (2016); Yang *et al.* (2015); Wan *et al.* (2011); Weng *et al.* (2015); Huang *et al.* (2015); Xu *et al.* (2015); Lv *et al.* (2015); Soluyanov *et al.* (2015); Wang *et al.* (2012); Liu *et al.* (2014); Wang *et al.* (2013); Burkov *et al.* (2011); Bradlyn *et al.* (2016).
Most current experimental studies of topological states are focused
on non-magnetic materials that preserve time reversal symmetryYan and Zhang (2012)
and ferromagnetic materials (mainly the quantum anomalous Hall effect)Haldane (1988); Liu *et al.* (2008); Chang *et al.* (2013); Yu *et al.* (2010).
Theoretically, a large variety of topological states can also
exist in materials with other types of magnetic structures, such as
anti-ferromagnetism (AFM) Mong *et al.* (2010); Fang *et al.* (2013); Yoshida *et al.* (2013); Wu *et al.* (2015); Bègue *et al.* (2016); Brzezicki and Cuoco (2016); Young and Wieder (2016).
Nevertheless, material proposals of these topological states are still rare.

Half-Heusler compounds are a large group of materials consisting of three metal elements
and have been widely studied for their flexible electronic properties and functionalities Graf *et al.* (2011).
Around 50 half-Heusler compounds are theoretically predicted to possess inverted band structure
and can be driven into the topological insulating phase by applying strains
Lin *et al.* (2010b); Chadov *et al.* (2010); Xiao *et al.* (2010); Al-Sawai *et al.* (2010); Yan and de Visser (2014).
Unusual surface states were recently observed experimentally in LnPtBi(Ln=Lu, Y) Liu *et al.* (2016) and LuPtSb Logan *et al.* (2016),
and serve as an evidence of non-trivial bulk topology.
Weyl semimetal (WSM) phase was theoretically discussed in several half-Heusler compounds,
including GdPtBi under external magnetic fieldsCano *et al.* (2016)
and LaPtBi with in-plane strainRuan *et al.* (2016),
and the corresponding evidences were found in recent experimentsHirschberger *et al.* (2016); Shekhar *et al.* (2016); Suzuki *et al.* (2016).
Our interest in this work is focused on possible topological states in
half-Heusler materials with AFM at zero external magnetic field.
AFM has been experimentally observed in RPdBi (R Er, Ho, Gd, Dy, Tb, Nd) and GdPtBi
Pan *et al.* (2013); Gofryk *et al.* (2011); Müller *et al.* (2014); Nikitin *et al.* (2015); Nakajima *et al.* (2015); Pavlosiuk *et al.* (2016a, b), with Neel temperature ranging from 1K to 13K.
To describe AFM in half-Heusler compounds, we construct a
generalized six-band Kane model with anti-ferromagnetic coupling terms
based on the symmetry principle and justify this model with the microscopic
tight-binding model. With this model, we predicted a large variety of
time reversal breaking TSM phases, including Dirac semimetal (DSM) phase
(if inversion symmetry breaking is insignificant), WSM phase,
nodal line semimetal (NLSM) phase (if AFM preserves mirror or glide symmetry) and
Type-B triple point semimetal (TPSM) phaseZhu *et al.* (2016) (if AFM preserves and glide symmetries),
and topological insulating phases, including
topological mirror (or glide) insulating (TMI) phase
and anti-ferromagnetic topological insulating (AFMTI) phase that is protected by
the combination of time reversal and half translation,
depending on the material parameters.
We also discuss phase diagrams of this model and identify
the candidate half-Heulser compounds to search for these topological phases experimentally.

This paper is organized as follows. In Sec.II, we first present our generalized six-band Kane model and describe the symmetry aspect of this model. In Sec.III, we focus on the block of this model which is relatively simple and includes all the four bands that are close to Fermi energy. A variety of topological phases, including DSM phase, WSM phase, NLSM phase, type-B TPSM phase and TMI phase, were identified for different aligning direction of magnetic moments of AFM. In Sec.IV, we discuss the limitation of the four-band model and extract the phase diagram of the six-band Kane model. We demonstrate a robust realization of AFMTI phase in half-Heusler materials with AFM once anti-ferromagnetic coupling is strong enough. The conclusion is drawn in Sec.V.

## Ii Model Hamiltonian of Anti-ferromagnetic Half-Heusler Materials

We start with a derivation of the model Hamiltonian for
half-Heusler materials with AFM. The half-Heusler compound is normally labelled
by XYZ Canfield *et al.* (1991),
in which two metal atoms X and Y together play the role of cations
and the metal atom Z is regarded as anions.
The crystal structure of half-Heusler materials is shown in Fig.1a,
in which X and Z atoms form the NaCl-type substructure and
Y and Z atoms form the zinc-blende substructure.
For certain half-Heusler materials with AFM, including GdPtBi, DyPdBi, HoPdBi and TbPdBi
Pavlosiuk *et al.* (2016a); Nakajima *et al.* (2015); Müller *et al.* (2014),
magnetic moments come from the X atom and
align ferromagnetically in one layer perpendicular to (111) direction and anti-ferromagnetically
between two adjacent layers (type-G anti-ferromagnetic ordering), as shown in Fig.1b,
while the exact direction of magnetic moments
is still not clear and may depend on detailed compound composition.

To construct the effective Hamiltonian,
we need to understand the symmetry aspect of this crystal. Without magnetic moments,
the space group of half-Heusler compounds is Fm Canfield *et al.* (1991)
and the corresponding point group is , similar to zinc-blende structure.
As a result, the lattice vectors should be chosen to be those of a face-centered cubic
lattice, labelled as , and in Fig.1b.
The existence of type-G AFM has two main effects to the symmetry of the system:
(1) it doubles the lattice vector and
the unit-cell along the direction,
and the new lattice vector is labeled as ;
(2) in the mean field level, the magnetic moments of AFM give rise to exchange coupling to electron spins.
Due to the doubling of unit-cell, the space group and point group of the crystalline structure are reduced to
Li *et al.* (2015) and , where the latter has two generators: three-fold rotation
along (111) direction
and mirror symmetry with respect to plane.
As a pseudo-vector, magnetic moments of AFM can further lower the symmetry.
Besides the above crystal symmetries,
another essential symmetry due to AFM is the combination
of time reversal and half tranlation ,
denoted as .

Similar to the semiconductors with the zinc-blende structure,
the low energy physics of half-Heusler materials can be described by
two bands with s-orbital nature, labelled by
with z-directional spin component ,
and four bands with p-orbital nature,
labelled by with total angular momentum which is a combination of spin and orbital angular momentum.
Here the notations and refer to the irreducible representations
of the corresponding bands under the group Winkler *et al.* (2003) and we still keep
this notation even though the symmetry is lowered.
The form and symmetry of the basis wave functions are also justified
with microscopic local atomic orbitals, as discussed in details
in Appendix A.
It should be emphasized that due to the doubling of the unit-cell, the
basis wave functions in the new unit-cell is a bonding or anti-bonding
state of the basis functions in the original unit-cell. For the low energy physics,
we find both and are bonding states
in the doubled unit-cell and are still suitable to be bases.
To obtain a matrix expansion of the effective Hamiltonian, we need to classify
all the matrices according to the irreducible representations of the symmetry
group for the above basis.
The Hamiltonian should be invariant
under symmetry operations of the symmetry group, which means it belongs to the identity
representation, and can be constructed by combining
the symmetrized matrices and symmetrized tensor components of physical quantities
(such as the momentum and the order parameters for AFM)
which belong to the same irreducible representation.Winkler *et al.* (2003)
This symmetry
approach allows us to systematically construct the full Hamiltonian, including
the standard six-band Kane model and an additional AFM term.
The standard six-band Kane model is Winkler *et al.* (2003)

(1) |

where

(2) |

(3) |

with to be the identity matrix,

(4) |

with

(5) |

and

(6) |

Here we have , , , , , and . ’s are angular momentum matrices for spin (see Appendix B for details), , , , , , is the effective mass of bands near point, , , and . In this work, , and are always assumed, unless being specified otherwise. Besides the above terms in the standard Kane model, AFM can lead to new terms, which can be constructed in a similar manner with the symmetry group combining point group , and (see Appendix B for more details). For the materials that we are interested in, the main influence of AFM only occurs for the four bands. Thus, we only consider the anti-ferromagnetic terms in the basis of the four bands and the corresponding Hamiltonian is given by

(7) |

where the detailed expressions for () are listed in Tab.1.

Next we discuss the symmetry properties of this Hamiltonian. For the standard Kane model , if the parameter in is zero, the Hamiltonian possess point group. Non-zero term breaks inversion symmetry and lowers the point group from to . The existence of AFM results in the doubling of the unit cell along and reduce the point group of the lattice to . For a fixed non-zero anti-ferromagnetic order , the AFM term will further reduce the symmetry. If is along the (111) direction, symmetry is maintained but all mirror symmetries in are broken, whereas the glide symmetries , and are preserved. If lies in a mirror plane ( or or ) but away from the (111) direction, all symmetries in are broken, whereas the glide symmetry is preserved. If is perpendicular to a mirror plane of the lattice, the only remaining symmetry is that mirror symmetry. For a generic AFM Hamiltonian, will break all the symmetries in group, as well as the combination with . Furthermore, we notice that only quadratic terms of appear in our Hamiltonian while any linear terms vanish. This is because reverses its sign under translation , while translation is just identity matrix for the basis of four bands and thus commutes with any representation matrix (see Appendix B for more details). This suggests that any term with the odd orders of the anti-ferromagnetic order parameter cannot exist.

## Iii Topological Phases in the Four-band Model

Since we are interested in the half-Heusler materials with inverted band structures, only four bands appear near the Fermi energy while the bands are far below the Fermi energy. Thus, we first focus on the four bands with the Hamitonian . For the inverted band structure, the Fermi energy is between the bands, and the two bands with lower energies are valence bands while the other two bands with higher energies are conduction bands. We emphasize that the bands are important for certain types of topological states even though they are away from the Fermi energy, as discussed in details in the next section. However, for the TSM phases discussed in this section, only the bands are essential. Another advantage of the 4-band Hamiltonian is that it can be solved analytically in certain limit, thus providing us valuable insight into the underlying physics. In this section, we first focus on the case without inversion symmetry breaking term (i.e. choosing ) and reveal the occurence of DSM phase due to the coexistence of inversion symmetry and anti-unitary symmetry. In realistic system, inversion symmetry is broken and DSM phase becomes unstable. Nevertheless, DSM phase can be viewed as the “parent” phase to generate other TSM phases after including . We further study the situation with non-zero inversion symmetry breaking term , focusing on the situations with (1) magnetic moments polarized within or perpendicular to the plane and (2) along the (111) direction.

### iii.1 Dirac Semimetal Phase and Topological Mirror Insulating Phase

In this part, is assumed and the total Hamiltonian takes the form with inversion symmetry. In this case, the eigenenergy of Hamiltonian can be solved analytically as

(8) |

Without AFM term, for in Eq.7,
and there are four-fold degeneracy of the bands at the point () due to
the group symmetry. The conduction and valence bands touch each other quadratically at ,
leading to a critical semimetal phase for the four-band Luttinger model .
Early studies have demonstrated various TSM phases induced by strain or external magnetic fields
in this system Shekhar *et al.* (2016); Ruan *et al.* (2016).
The AFM term can lower the symmetry of the system and remove
the four-fold degeneracy at point.
However, since preserves the inversion symmetry and symmetry, all the bands
are still doubly degenerate, similar to the Kramer’s degeneracy due to inversion and time reversal symmetriesMurakami (2007).

Next we will study the influence of AFM term on the energy dispersion. The AFM term can lead to a non-zero gap at point, given by . Thus, the two doubly degenerate bands are split at the point, but they may cross each other at some finite momenta , giving rise to a semimetal phase. The realization of such semimetal phase requires for all and the details are discussed in Appendix C. Here we focus on the cases where magnetic moments of AFM are perpendicular to plane ( and , where are x,y,z components of AFM magnetic moment ) with mirror symmetry , or lie in the plane () with the glide symmetry . According to the expressions of ’s, both conditions imply the same requirement for parameters: and , which is reasonable since and have the same matrix representation for four basis wave functions of bands. For the existence of gapless points, one of the following additional conditions is required for the values of , and (see Appendix C for more details): (i) , ; (ii) , ; (iii) , , . Due to and , we require and from with , indicating two possibilities for the locations of gapless point, either on the plane with the form for the conditions (i) and (ii), or on the axis with the form for the condition (iii). Due to the symmetry, if a gapless point occurs at a finite momentum , there must be another one at , leading to even number of gapless points. The number of gapless points are confirmed to be 2 by solving for positions of gapless points in each case (see Appendix C for more details).

Based on the above conditions for gapless points, we can further extract the phase diagram of this model as a function of . An example of a phase diagram is shown in Fig.2a for the choices of parameters listed in Tab.6 in Appendix F. The blue line in the phase diagram, labelled by II, represents DSM phase. For our choices of the parameters, the condition (i) or (ii) can be satisfied and Dirac points are on the plane. Fig.2c reveals a typical energy dispersion of the semimetal phase at the point B in Fig.2a with and to satisfy the condition (i), where the energy unit is defined as and is a real positive parameter with unit of length. The energy dispersion around the gapless point behaves linearly, thus forming two Dirac cones at , given the double degeneracy for each band. Further theoretical analysis of the effective low-energy Hamiltonian expanded around these two gapless points confirms this DSM phase, as shown in details in Appendix D.

The DSM phase separates
two insulating phases, labeled by I and III in Fig.2a. To identify the nature of these two insulating phases,
we perform a numerical calculation of energy dispersion on the surface of an approximately semi-infinite sample.Sancho *et al.* (1985)
The local density of states at the top surface along axis is shown in Fig.2b for the point A with in the
phase I and Fig.2d for the point C with in the phase III, respectively.
One can see two sets of gapless modes appearing for the phase I
while a full gap existing for the phase III.
Thus, we expect the phase I is topologically non-trivial while the phase III
is trivial.
We also perform a calculation of mirror Chern number (MCN) Teo *et al.* (2008) on the mirror or glide plane
() for this system and find MCN to be 2 for the phase I and 0 for the phase III.
This confirms that two sets of gapless modes in Fig.2b are protected by mirror or glide symmetry and makes phase I to be TMI phase.
Thus, DSM phase can be viewed as the topological phase
transition point between a TMI phase and a trivial insulating phase.

We emphasize that, in realistic half-Heusler materials, inversion symmetry is absent and the Kramer’s degeneracy at a generic is split for both the conduction and valence bands, which means Dirac points are also split. However, DSM phase will evolve into other TSM phases, as discussed in the next section. Therefore, DSM phase can be viewed as the “parent” phase to search for and understand other topological phases.

### iii.2 Weyl Semimetal Phase and Topological Mirror Insulating Phase

In this part, we include the inversion symmetry breaking term into the four-band Hamiltonian and the total Hamiltonian becomes . As a consequence, the Kramer’s degeneracy of each band at a generic momentum is split. We still consider the magnetic moments of anti-ferromagnetic ordering aligning within or perpendicular to the plane to preserve either the glide symmetry or the mirror symmetry , which gives and . Due to the existence of term, the full Hamiltonian cannot be diagonalized analytically and thus numerical methods are adopted to extract phase diagram. All the parameters are the same as previous choices, except the parameter which is chosen as , as listed in Tab.7 in Appendix F.

The phase diagram as a function of and is shown in Fig.3a. The phases I and III in Fig.3a remain robust due to existence of mirror or glide symmetry. A direct calculation of surface local density of states, as well as MCN, shows four surface modes with MCN being 2 (Fig.3c) for point A in Fig.3a ( and ) and a full gap with zero MCN (Fig.3d) for point C in Fig.3a ( and ).

We notice that the phase II is expanded from a line of Dirac semimetal phase in Fig.2a to a region of WSM phase in Fig.3a. The reason is that breaks the inversion symmetry and splits each Dirac cone in Dirac semimetal phase into two Weyl points. Since there are two Dirac points in the DSM phase, the phase II in Fig.3a typically has four Weyl points in the whole momentum space, denoted as (), as shown in Fig.3b and Fig.4a. Different Weyl points can be related to each other by symmetries: and (or and ) are related by or and therefore dubbed a ’ pair’, and (or and ) are related by and called a ’ pair’, and finally and (or and ) are related by and named a ’ pair’. It is known Volovik (2003, 2007, 2013) that a Weyl fermion can carry topological charge or chirality, which can be extracted from Chern number (CN) on a small spherical surface surrounding the Weyl point. Two Weyl points related by mirror symmetry have opposite Chern numbers while time reversal operation leaves Chern number of a Weyl point unchanged. As a result, a pair carries opposite CNs and so does a pair, while a pair has the same CN. Due to the existence of topological charge, a single Weyl point cannot be gaped and it can only move in the momentum space when tuning the parameters until it merges with another Weyl point with opposite CN. We track the motion of Weyl points through the path in the inset of Fig.3a, and find and (or and ) emerge from a point in the mirror (or glide) plane as a pair and move in the momentum space and finally annihilate at another point on the mirror (or glide) plane, as shown in Fig.3b with its inset.

Now we focus on the energy dispersion of the WSM phase by choosing the B point in Fig.3a with and as an example. Four Weyl points are approximately located at

in the momentum space, respectively.
By integrating the Berry curvature on the sphere around each point,
we found CNs of four Weyl points are ,, and (see Fig.4a),
which is consistent with the symmetry analysis above.
The Fermi surface around at the energy of the Weyl point, as shown in Fig.4b,
demonstrates the existence of a type II Weyl point in our system.
Type-II Weyl fermions are topologically non-trivial and can lead to Fermi arc Soluyanov *et al.* (2015)
on the surface.
Thus, we perform a calculation of local density of states at the energy of Weyl points
on the surface of an approximately semi-infinite sampleSancho *et al.* (1985), as shown in Fig.4c,
in which a complex surface Fermi arcs overlaps with bulk bands.
Weyl points are depicted by white points in the plot and there is one Fermi arc
starting from each Weyl point and merging into bulk bands.
Additional surface states with same energy exist and they form a circle surrounding the point .
The surface energy dispersions along the momentum lines and
are shown in Fig.4d and Fig.4e.
We notice one chiral edge mode existing along the momentum line
while a helical edge mode along the line .
We may treat as a parameter and consider
two-dimensional (2D) planes formed by and for different .
According to the bulk-edge correspondence,
the existence of chiral edge mode for at the surface suggests for the
corresponding 2D plane. Similarly, CN for the 2D plane at should be zero.
This is consistent with the fact that two 2D planes at
and enclose one Weyl point whose CN is .
However, the helical edge mode along the line cannot be explained by
CN. The crossing point between two branches of the helical edge mode
is protected by S symmetry, thus the 2D plane at can be viewed as an AFMTI phase.
In addition, since the crossing point at also falls into the mirror (or glide) plane, as shown
by diagonal line in Fig.4c, that crossing point thus can also be protected by MCN, which
is equal to 1 for the phase II.
Therefore, two blue lines in Fig.3a can be viewed
as transition lines between the phase II with and the phase I or III
with MCN being 0 or 2 respectively.

Although we focus on the magnetic moments parallel or perpendicular to the mirror plane in this section, the WSM phase can NOT be destroyed immediately when magnetic moments is tilted away from these directions due to the non-zero CNs carried by Weyl points. The Weyl points can only move in the momentum space and should be robust in certain parameter regimes. On the other hand, for the TMI phase, gapless points of helical surface mode at finite non-zero momenta are solely protected by the mirror or glide symmetry and thus sensitively depend on the direction of magnetic moments.

### iii.3 Nodal Line Semimetal Phase

For magnetic moments of anti-ferromagnetic ordering within or perpendicular to the plane, another possible TSM phase is NLSM phase. An example of NLSM phase is shown in Fig.5, where the parameters are shown in Tab.8 in Appendix F. Due to differnet AFM parameters, the band sequence is different and two crossing bands at the low energy are from two opposite mirror subspaces, in contrast to the band crossing between two bands with the same mirror parity for the phase boundary in Fig.3a. The energy dispersion of NLSM phase is depicted in Fig.5a and the positions of two nodal rings are shown in Fig.5b. The topological stability of each nodal ring can be extracted by the Berry phase along a small circle (red circle in inset of Fig.5b) around the nodal line.

### iii.4 Triple Point Semimetal Phase

In this section, we consider the case with anti-ferromagnetic magnetic moments along the direction, where the system has three-fold rotational symmetry and glide symmetry . Since the matrix representation of is equivalent to the mirror symmetry for the basis of bands, the symmetry group generated by and is isomorphic to the point group . By linearly combining the four basis functions of bands which carry total angular momentum , we can get a pair of states belonging to two-dimensional representation of the double group, and the other two states belonging to two one-dimensional representations and respectively.Burns (2014) The character table of the double group and the linear combinations of bases are shown in Appendix E.

To confirm this symmetry analysis, we consider the total Hamiltonian with magnetic moments along the direction (). This corresponds to the conditions and . Along the direction, we indeed find that four bands are split into one doubly degenerate band, labeled as bands, and another two non-degenerate bands, labeled as and bands respectively. The corresponding energy dispersion can be solved analytically as

where and .
A typical energy dispersion is shown in Fig.6a with the parameters shown in Tab.9 in Appendix F.
Under the condition and ,
the bands cross with the () band at two points
labeled by 1 and 3 (2 and 4) in Fig.6a.
At each crossing point, there is a three-fold degeneracy and the energy dispersion
behave linearly along the axis.
Points 1 and 2 (or 3 and 4) are connected by four nodal lines,
as shown in Fig.6b. Along a circle enclosing any of these four nodal lines, the accumulated Berry phase
is found to be . This type of semi-metal phase is known as type-B TPSM phase Zhu *et al.* (2016).

## Iv Anti-ferromagnetic Topological Insulating Phase in Six-band Kane Model

In the discussion above, we neglected bands and only focused on four bands.
This simplification has irrelevant influence on TSM phases since bands are far away
from Fermi energy.
However, due to the inverted nature between and bands, bands may
play an essential role for topological insulating phases.
It is well known that the band inversion between the
and bands leads to the topological insulating phase in HgTe Bernevig *et al.* (2006),
as well as non-magnetic half-Heusler materials
Lin *et al.* (2010b); Chadov *et al.* (2010); Xiao *et al.* (2010); Al-Sawai *et al.* (2010).
Therefore,we study the influence of bands by considering the full six-band Kane model in this section.
With the anti-ferromagnetic ordering, the full Hamitonian takes the form

(9) |

which can be diagonalized numerically to extract the energy dispersion.

To systematically understand the AFM term in the Kane model, we consider different terms ()
in Eq.7, separately.
We notice that applying three-fold rotational symmetry to is equivalent to
performing the following transformations: and , which means are related with each other by and so do .
Therefore, we study the direct band gap as a function of (i) and for
in which case the effective Hamiltonian in Eq.9 preserves two-fold rotation symmetry along the x,y,z axes,
or (ii) and for in which case the effective model preserves ,
two-fold rotation along z axis and mirror symmetry perpendicular to (110).
The phase diagrams for the case (i) and (ii) are shown in Fig.7a and Fig.7b, respectively,
from which one can find both the gapless phases existing in the blue region (the region I)
and the gaped phases in the three yellow regions (the region II).
Detailed parameters for our calculation can be found in Tab.10 and Tab.11 in Appendix F.
We notice that term takes the same form as the strain term described in Ref.[dccclxxx(46)].
Therefore, on the line in Fig.7a or the line in Fig.7b,
we expect the gapless and gaped phases should be equivalent to the corresponding ones studied
for strained HgTe and half-Heusler materials Ruan *et al.* (2016).
To verify the nature of gapless and insulating phases, we calculate the energy dispersion
for two typical sets of parameters: the point A with
and B with in Fig.7b. A non-zero bulk direct gap
is found for the point A and thus we consider an approximately semi-infinite configuration and plot the
local density of states on (001) surface, as shown in Fig.7c.
A helical surface mode is found in the bulk gap and is protected by the symmetry
instead of the time reversal symmetry due to the anti-ferromagnetic ordering, thus giving
rise to a realization of AFMTI phase.
The gapless phase at point B is found to be WSM phase
and the bulk dispersion is shown Fig.7d with the Weyl points located
at the momenta (or equivalent ).

Below we emphasize that the AFMTI phase is quite robust in this system. The phase diagram for is shown in Fig.8a for . Other parameters are listed in Tab.12 in Appendix F. We notice that after introducing non-zero which breaks all symmetries and their combinations with half translation, the previous gapless phase region I in Fig.7a shrinks to a smaller region I in Fig.8a, while the region II of AFMTI phase is greatly extended. In the region I, Weyl points are found (not exclusively) at for point B of Fig.8a, as shown in Fig.8b. The local density of states calculation on (001) surface for point A in Fig.8a with gives very similar graph as Fig.7c, thus demonstrating the AFMTI phase in the region II. Given the large region in the parameter space for the realization of AFMTI phases, we can conclude that anti-ferromagnetic half-Heusler materials provide a robust material realization of the AFMTI phase. Moreover, if mirror or non-symmorphic symmetry exists, anti-ferromagnetic half-Heusler materials can also provide a robust material realization of anti-ferromagnetic mirror or non-symmorphic topological insulator phase, which has not been demonstrated in experiments.

## V Conclusion

Based on the above studies of the four-band and six-band Kane models, we can summarize the overall physical picture for electronic structures of anti-ferromagnetic half-Heusler compounds in Fig.9. Without AFM, the band ordering of and bands are inverted and the Fermi energy lies at the four fold degenerate point () of bands, leading to a critical semimetal phase shown in Fig.9a. With AFM, four-fold degeneracy of bands at point is removed. The band structure of four bands can be either normal (Fig.9b) or inverted (Fig.9c), depending on the detailed form and parameters of AFM terms. When the band ordering of bands is normal, it is trivial for the four-band model but non-trivial for the six-band Kane model due to the inversion between the and bands, leading to either AFMTI phase or TMI phase. When the band ordering is inverted, the AFM terms can either lead to a full inverted band gap (AFMTI or TMI phase in Fig.9d) or preserve certain gapless points in the momentum space (the TSM phase in Fig.9e). In either situation, we find that anti-ferromagnetic half-Heusler compounds are topologically non-trivial. Thus, our work demonstrates that half-Heusler materials with AFM provide a platform for a robust realization of anti-ferromagnetic topological phases, either WSM phase or AFMTI phase, in a wide parameter regime.

The AFMTI phase was first proposed in Ref.[dccclxxx(30)] based on a four band toy model and our results have shown
this interesting topological phase indeed can exist in anti-ferromagnetic half-Heusler materials.
We notice that the first principles calculation in combining with tight-binding model has been adopted for the
AFM GdPtBiLi *et al.* (2015), in which a semi-metal phase is found.
However, the topological nature of this semi-metal phase
has not been extracted and our results identify the existence of Weyl points in this semimetal phase.
In addition, the authors use the representation of crystal symmetry group to label each band,
aiming in identifying band inversion.
We believe this approach is insufficient for the AFMTI phase
since this topological phase is protected by the symmetry which is not included
in the crystal symmetry group. Thus, the AFMTI phase cannot be identified from the crystal symmetry representation.
Our results suggest that the AFMTI phase can exist in the G-type anti-ferromagnetic system, which is identified to be
topologically trivial in Ref.[dccclxxx(59)].
In the existing experiments, Weyl semi-metal phase has been unveiled in GdPtBi under an external
magentic field through the observation of a large anomalous Hall angle Suzuki *et al.* (2016); Shekhar *et al.* (2016), large negative magnetoresistanceShekhar *et al.* (2016); Hirschberger *et al.* (2016)
and the strong suppression of thermopowerHirschberger *et al.* (2016).
Our results suggest Weyl semi-metal phase may already occur even in absence of external magnetic fields.
In addition, nodal line fermions, type-B triply degenerate fermions and
topological mirror or glide insulators, are also possible in certain parameter regimes when
magnetic moments of anti-ferromagnetic ordering are along some specific directions.
The topological surface states or surface Fermi arcs in these topological phases can be extracted
from angle-resolved photoemission spectroscopy or quasi-particle spectrum from
the scanning tunneling microscopy Qi and Zhang (2011); Hasan and Kane (2010); Yan and Felser (2016).
Our generalized Kane model also provides a basis for the future study of magnetic,
transport or optical phenomena in this class of materials.
Furthermore, we notice that superconductivity can coexist with anti-ferromagnetism in
RPdBi(R=Tb,Ho,Dy,Er)Nakajima *et al.* (2015); Pan *et al.* (2013); Pavlosiuk *et al.* (2016a).
Thus, it is interesting to ask if topological superconductivity can be realized in these materials.

## Vi Acknowledgment

We would like to thank Rui-Xing Zhang, Jian-Xiao Zhang, Qing-Ze Wang, Yang Ge and Di Xiao for helpful discussion. C.-X.L. acknowledges the support from Office of Naval Research (Grant No. N00014-15-1-2675). B.Y. acknowledges support of the Ruth and Herman Albert Scholars Program for New Scientists in Weizmann Institute of Science, Israel.

## Appendix A Tight-Binding Model and its relationship to the Kane model

In this section, we will describe a tight-binding model for our anti-ferromagnetic half-Heusler materials, from which we can justify the extend Kane model that we derived by symmetry principles and used for the low energy physics in Sec.II in the main text. For simplicity, we take the anti-ferromagnetic half-Heusler material ErPdBi as an example and assume it has anti-ferromagnetic structure of GdPtBi. We consider nine orbitals , , , , , , , and to construct the tight-binding model for this material. Due to the doubling of the unit cell along , all the orbitals are labeled as , where stands for atoms and orbitals and labels two sub-lattices that are related by . We only considered the hopping terms between the nearest neighbor atoms, including Bi, Pd and Er atoms Slater and Koster (1954), giving rise to a 18 by 18 Hamiltonian . In order to include AFM term and spin-orbit term, we need to consider spin degree of freedom and the hopping term is block diagonal with each block given by in the spin space. The AFM term is given by with for two sets of cells. Here is the mean field value for anti-ferromagnetic magnetic moments and labels electron spin. The spin-orbit coupling term takes the form , where denotes the angular momentum operator that acts on the oribtal ( orbitals).

The above 36 by 36 tight-binding Hamiltonian is complicated and we are not interested in using this model for realistic calculation here. Instead, we hope to use this model to justify the form of the extended Kane model that we derived from the symmetry principle in the main text. We need to transform the basis wave functions into the ones used for the Kane model, which can be achieved by the following steps. (1) The basis wave functions of the Kane model only include two orbital type of bands and 6 orbital type of bands after taking into account spin degree of freedom. Therefore, we need to first reduce the number of basis wave functions. Since both the and orbitals belong to representation of the group (The representation table of group is shown in Fig.10), the basis wave functions should be a linear combination of the and orbitals, which are given by , where , and . For the s orbital type of bands, the basis wave functions take the form , where and . Here we still keep the sub-lattice index and the number of basis wave functions are reduced to 16. (2) The basis wave functions of the Kane model is chosen to be the eigenstates of total angular momentum operator because of strong spin-orbit coupling. Therefore, we apply a similar transformation to our basis wave functions here, as listed below:

where are sub-lattice indexes. (3) Since the low energy physics only occurs at the point () for the Kane model, we only need to consider the bonding states between two sub-lattices. The 8 bonding basis wave functions are given by , where and ,or including