Anisotropy induces non-Fermi-liquid behavior and nematic magnetic order in three-dimensional Luttinger semimetals

Anisotropy induces non-Fermi-liquid behavior and nematic magnetic order in three-dimensional Luttinger semimetals

Igor Boettcher Department of Physics, Simon Fraser University, Burnaby, British Columbia, Canada    Igor F. Herbut Department of Physics, Simon Fraser University, Burnaby, British Columbia, Canada

We illuminate the intriguing role played by spatial anisotropy in three-dimensional Luttinger semimetals featuring quadratic band touching and long-range Coulomb interactions. We observe the anisotropy to be subject to an exceptionally slow renormalization group (RG) evolution so that it can be considered approximately constant when computing the impact of quantum fluctuations on the remaining couplings of the system. Using perturbative RG we then study the competition of all local short-range interactions that are generated from the long-range interactions for fixed anisotropy. Two main effects come to light for sufficiently strong anisotropy. First, the three-dimensional system features an Abrikosov non-Fermi liquid ground state. Second, there appear qualitatively new fixed points which describe quantum phase transitions into phases with nemagnetic orders – higher-rank tensor orders that break time-reversal symmetry, and thus have both nematic and magnetic character. In real materials these phases may be realized through sufficiently strong microscopic short-range interactions. On the pyrochlore lattice, the anisotropy-induced fixed points determine the onset of all-in-all-out or spin ice ordering of local magnetic moments of the electrons.

I Introduction

Understanding Fermi points at high-symmetry band crossings in semimetallic materials constitutes a promising portal towards designing exotic states of matter Vafek and Vishwanath (2014). The intriguing properties of systems with linear band touching described by a low-energy effective Dirac Hamiltonian have been studied extensively in graphene both theoretically and experimentally Herbut (2006); Castro Neto et al. (2009); Das Sarma et al. (2011). The recent advance in realizing three-dimensional Weyl semimetals and the subsequent experimental verification of their peculiar properties marks another milestone on this road Xu et al. (2015); Lv et al. (2015). A quadratic band touching (QBT) point is realized in bilayer graphene and three-dimensional Luttinger semimetals, the latter being described by the Luttinger Hamiltonian Luttinger (1956), which includes GaAs, HgTe, -Sn, or the recently actively studied class of Pyrochlore Iridates Murakami et al. (2004); Moon et al. (2013); Witczak-Krempa et al. (2014).

Systems with QBT become particularly interesting when strong spin-orbit coupling leads to a band inversion such that both positive and negative energy states meet at the Fermi point. Furthermore, whereas the short-range part of Coulomb interactions is screened in three dimensions, its long-range part substantially influences the many-body electron correlations. It has been pointed out by Abrikosov Abrikosov and Beneslavski (1971); Abrikosov (1974) and reinvestigated more recently Moon et al. (2013) that the interplay between an inverted QBT point and long-range Coulomb repulsion leads to a non-Fermi liquid (NFL) ground state of the system. Both requirements for Abrikosov’s scenario are realized in the Pyrochlore Iridates Nakatsuji et al. (2006); Machida et al. (2010); Kurita et al. (2011); Balicas et al. (2011); Witczak-Krempa et al. (2013); Rhim and Kim (2015); Kondo et al. (2015); Nakayama et al. (2016); Liang et al. (2016), which therefore constitutes an ideal experimental platform to test its predictions. It is found that for most members of the material class the ground state is a magnet with spins ordered in all-in-all-out (AIAO) states on the tetrahedra of the pyrochlore lattice. An exception is given by PrIrO, which remains a bad metal for the lowest temperatures explored in experiment Witczak-Krempa et al. (2014).

Although the density of states is proportional to the square root of energy at a QBT point in three dimensions so that the short-range part of the Coulomb repulsion is technically irrelevant, short-range interactions can still influence the phase structure of the model in two ways. First, if the corresponding coupling constant exceeds a certain critical value, they can lead to qualitatively different ordered states, represented in the renormalization group (RG) by a runaway flow. Second, even if initially absent, short-range interactions are generated during the RG flow. The feedback of those short-range couplings onto each other leads to the appearance of further fixed points, which may collide with Abrikosov’s infrared fixed point and annihilate it. In fact, within an -expansion around four dimensions extrapolated to three dimensions, this does indeed happen for the isotropic system, and the Abrikosov fixed point is removed through precisely this mechanism Herbut and Janssen (2014). The system is left with a runaway flow towards an ordered nematic state Janssen and Herbut (2015, 2016).

In this work we extend the analysis of the influence of short-range interactions on Abrikosov’s scenario by incorporating the effect of spatial anisotropy of the band structure at the QBT point. In fact, most real materials are not fully rotationally symmetric, but rather feature only cubic rotation invariance – even in the low-energy description encoded in the Luttinger Hamiltonian. Whereas Abrikosov points out that a stable fixed point can only be isotropic Abrikosov (1974), Savary, Moon, and Balents Savary et al. (2014) find a stable quantum critical point towards AIAO order at (maximally) strong anisotropy within a controlled -expansion. Here we investigate within the -expansion whether (i) anisotropy can lead to a stable NFL fixed point in three dimension, and (ii) whether it can yield further unstable directions, such as towards the AIAO state. A key finding is that the RG flow of the anisotropy parameter is negligibly slow so that the anisotropy can simply be considered constant for all practical purposes; i. e. an approximately marginal coupling. Within this approximation both (i) and (ii) will be found to be answered in the affirmative.

This work is organized as follows. In Sec. II we introduce the field theoretic framework describing the anisotropic Luttinger semimetal together with the RG flow of the anisotropy parameter and Abrikosov’s NFL scenario. In Sec. III we study the RG fixed point structure and influence of short-range interactions in the anisotropic system. We close with a discussion of our results in Sec. IV. Extensive appendices are devoted to deriving the full set of RG equations (A), constructing irreducible spin tensors (B), Fierz identities (C), and computing the functions used throughout the text (D).

Ii Field theoretic framework

ii.1 Lagrangian

The physics of three-dimensional Luttinger fermions with long-range Coulomb repulsion is captured by the Lagrangian


where is a four-component Grassmann field, is the real electrostatic photon field, denotes imaginary time, is electric charge, and is the Luttinger Hamiltonian Luttinger (1956). The chemical potential is tuned to be at . We assume time-reversal invariance at the single-particle level, no external magnetic field shall be applied. Under these conditions, each of the two bands touching quadratically is doubly degenerate, and Luttinger showed that the most general single-particle Hamiltonian is given by


with Luttinger parameters , effective electron mass , and momentum operator . The matrices represent the spin-3/2 angular momentum operators. We set in the following. We have written the Lagrangian (1) in a manner such that the dynamic critical exponent at the noninteracting Gaussian fixed point. Further, denotes the unit matrix.

The Luttinger Hamiltonian can be written in the computationally more advantageous form Murakami et al. (2004); Moon et al. (2013); Herbut and Janssen (2014); Boettcher and Herbut (2016)


with five Hermitean matrices satisfying the Clifford algebra,


the functions being given by , , , , , and we have and . We define the matrices below in a more general context. The functions constitute the real spherical harmonics on a sphere of radius .

For the field theoretic treatment we rescale the Hamiltonian by , normalize the field such that , and introduce the particle-hole asymmetry and anisotropy parameters, and , via


In the following we set , corresponding to particle-hole symmetry, which will be shown to emerge dynamically during the renormalization group flow. With the normalization in Eq. (5), the parameter lies within the real interval . We eventually arrive at


Squaring the Hamiltonian yields


due to and . The roots of this expression determine the doubly degenerate spectrum of the Hamiltonian. We observe the spectrum to be rotation symmetric (a function of alone) only for .

The latter observation is a key element of this work. Under a spatial rotation of coordinates with , , the operators and transform as vectors, i.e., in the same manner as . Obviously, the first line in Eq. (2) is rotation invariant. The same is true for the first line in the representation of Eq. (3) – although less obviously so at this point. It will become apparent once we define the -matrices as components of the second rank tensor below: the term is then seen to be proportional to , which is clearly rotation invariant.

Rotation invariance of the Hamiltonian is broken for . However, rotations with certain fixed angles still leave the expression invariant, namely those which rotate the individual coordinate axes onto each other. Roughly, those transformations permute the coordinate labels . If this symmetry is present we say that the system has cubic symmetry. The Luttinger Hamiltonian exhausts all cubic invariant terms to order , so that three Luttinger parameters suffice to parametrize a quadratic band touching point. The full rotation and cubic rotation groups are and , respectively.

ii.2 RG flow of the anisotropy

Performing the usual Wilson’s integration of the fermionic modes within the momentum shell and with all frequencies, we derive a flow equation for the anisotropy parameter from the renormalization of the fermion self-energy. The computation is presented in detail in App. A. For this, angular integrations are performed in three dimensions, but the qualitative results remain invariant when performing the angular integration in four dimensions, which constitutes the upper critical dimension.

The RG flow equations presented in this work are valid for arbitrary values of . Except for some special values, such as , the -function can only be determined numerically. To make them more accessible, however, we introduce functions with the following properties: We have for , and for general , is nonzero, positive, and of order unity. In this way, the qualitative and mostly quantitative aspects of the RG flow can be understood by setting


in the -functions. The functions are computed in App. D and shown in Fig. 8.

The RG flow for the anisotropy parameter to leading order in reads


Since , we immediately discern three fixed points at . The coefficient multiplying , however, is exceptionally small. Close to the attractive fixed point , for example, the linearized flow reads


This signals an extremely slow flow towards the fixed point. In fact, the entire prefactor multiplying in Eq. (9) remains below in magnitude for all values of , see Fig. 1. Consequently, the parameter can approximately be considered to be marginal - in contrast to a running coupling.

Figure 1: RG flow of the anisotropy . We observe an attractive fixed point at (negative slope), and two repulsive ones at . This, however, is hardly of practical relevance since the prefactor multiplying in Eq. (9) is exceptionally small so that can effectively be treated as a constant parameter with during the running of couplings. Put differently, the tiny value of the beta function also implies very small gradients and thus an anomalously slow RG flow. As a consequence, is approximately constant for those momentum rescale factors where renormalizations of the charge and of the generated short-range interactions are important.

ii.3 Abrikosov’s NFL fixed point

The loop corrections to the fermion self-energy determine the RG flow of the couplings and , the fermion anomalous dimension , and the dynamic critical exponent . The photon self-energy leads to a renormalization of the charge . The diagrammatic one-loop contributions to the fermion and photon self-energies are displayed in Fig. 2. The flow of has been discussed in the previous section. It is easy to see (App. A) that the coupling , which is marginal at the Gaussian fixed point, only receives corrections according to


At an interacting fixed point with the coupling is then attracted towards . This justifies setting in Eq. (6). In contrast to Eq. (10), is not exceptionally small so that a nonzero diminishes quickly.

The existence of a nontrivial fixed point close to dimensions with anomalous fermion scaling and charge renormalization for a three-dimensional quadratic band touching system has first been pointed out by Abrikosov Abrikosov (1974). This approach to an NFL is ingenious in its simplicity, as it basically relies on the “chirality” of the band dispersion, i.e. the presence of positive and negative eigenenergies (which implies and ), and the frequency independence of the photon propagator (which implies .) This has to be contrasted with an otherwise similar system of ultracold atoms at resonance with , where the lower band is missing, and the dimer propagator is Galilean invariant, implying anomalous boson scaling, but and Diehl and Wetterich (2006); Nikolić and Sachdev (2007); Boettcher et al. (2014). Abrikosov further points out that a stable fixed point of can only be at , and thus sets . As we have seen above, this statement is true, but it is still reasonable to consider the modifications of the Abrikosov NFL fixed point when taking into account a nonzero value of .

Figure 2: One-loop self-energy contributions to the RG flow. Diagrams a) and b) show the fermion and photon self-energy, respectively. A straight line represents a fermion propagator, a wiggly line a photon propagator. Contribution a) generates the anomalous fermion scaling, expressed by and , as well as the exceptionally weak running of the anisotropy . Diagram b) results in the charge renormalization. Both diagrams taken together yield Abrikosov’s NFL fixed point scenario.

The fermion anomalous dimension is given by


Close to it reads


The numerical coefficient should be compared with the one in Eq. (10). Equations (10) and (13) are consistent with the results of Ref. Boettcher and Herbut (2016), which have been obtained in a perturbative expansion in . The comparison is facilitated by setting in the reference, and adjusting some couplings and prefactors, which leads to identical loop contributions to obtain the fermion self-energy. The dynamic critical exponent is given by


The flow equation for the charge is then given by


The function is bounded from below by . We find the Abrikosov fixed point of the charge for small to be


Again, is positive and of order unity for all , with . We thus conclude that the Abrikosov fixed point persists for all values of . However, as , the fixed point becomes weakly coupled. This behavior is visualized in Fig. 3.

Figure 3: Fixed point of the charge. The (effective) attractive infrared fixed point of close to four dimensions, divided here by , survives for all fixed . The solid red line shows the result from Eq. (16) and the dashed blue line is , which amounts to setting and yields an excellent approximation. For strong anisotropy the theory becomes weakly coupled.

The presence of the prefactor in the fixed point charge renders the problem perturbative for strong anisotropy, even when working with . A similar observation has been made in Ref. Savary et al. (2014) at the anisotropic fixed point with . In contrast to the -expansion applied here, the fixed point in the reference is controlled by a -expansion in three dimensions. The findings of the two investigations differ in that we do not find the anisotropic fixed point to be stable. This will be discussed further in Sec. IV.

Iii Short-range interactions

We restrict the discussion of short-range interactions to those that can be expressed in terms of local four-fermion terms. Every such term can be written as a contribution


to the Lagrangian, with coupling and some matrices , where is the set of complex Hermitean matrices. In particular, even the terms of the form can be brought into the form of Eq. (17) by a Fierz transformation Boettcher and Herbut (2016).

In order to cover all possible terms of the form (17), it is sufficient to restrict to contributions , where is an -basis of . Furthermore, rotation or cubic symmetry impose severe constraints on the possible values of . One possible choice of basis consists in with


where , , and we require . (There are ten such matrices.) This choice is particularly convenient due to the computational simplifications arising from the Clifford algebra. However, the matrices do not have definite transformation properties under rotations of the cubic symmetry group. For our purposes it is convenient to introduce a basis which is manifestly cubic invariant. We denote it by


with and . Note that, indeed, both and consist of 16 elements each.

In this section we first construct irreducible spin tensors to derive the basis , then study the fixed point structure of the RG flow including these couplings, and eventually investigate the instabilities associated with these fixed points (expressed through a divergent susceptibility at the transition).

iii.1 Irreducible spin tensors

To obtain the basis (19) we construct all possible operators that transform as tensors under SO(3). For this let be a rotation matrix, . We define a tensor of rank as any object labelled by indices that transforms as


under a change of coordinates . In particular, we will be interested in the case that is a matrix. Recall that the rank of a tensor can be reduced by one or two units, respectively, by contracting it with or according to


If such a contraction yields zero we say that the tensor is irreducible. From Eqs. (21) and (22) it is then clear that irreducible tensors are precisely the symmetric and traceless tensors Hess (2015). Here, we say that a tensor is traceless if all its partial traces yield zero, i.e., it vanishes whenever two indices are contracted.

In order to construct the basis with definite transformation properties under rotations we apply the following recipe: The spin matrices transform as a vector under , and the product transforms as a tensor of rank . The corresponding irreducible tensors of rank can be computed from these products through symmetrization and the subtraction of suitable traces. One may wonder whether this procedure yields irreducible tensors of arbitrary rank. However, by means of the Cayley–Hamilton theorem it is easy to show that there are no irreducible spin tensors with rank , see App. B.3. Hence, in our case, we obtain irreducible tensors of rank , which can be used to build the basis introduced above.

Due to the non-commutativity of the spin matrices, , products of the type are not symmetric. We define the symmetrized rank tensors


Next, irreducible tensors , are constructed from the quantities with overbar by subtracting the partial traces such that . With a suitable ansatz and by making use of we arrive at


These are the irreducible spin tensors of rank for spin .

In App. B we show in detail that the irreducible tensors and can be expressed as


with , , and orthogonal basis tensors and . Since the spin tensors are matrix-valued, the components and are matrices as well. The desired basis in Eq. (19) is now constructed from the 1+3+5+7=16 elements , , , and after a proper normalization to satisfy


We define


In particular, the five components of the second rank tensor are the -matrices introduced in the context of the Luttinger Hamiltonian in Eq. (3). We explicitly have


Note that since is purely imaginary, the matrices are real, whereas are imaginary. The seven components of the third-rank tensor are given by


The operators and are even under time-reversal transformations Boettcher and Herbut (2016), whereas and are odd. All operators feature inversion invariance. A nonzero expectation value of , , and corresponds to a nonzero density, magnetization, and nematic order, respectively. The order parameter constitutes a tensorial magnetization that does not point in a particular direction. We therefore suggest to refer to it as “nemagnetic order”. These observations are summarized in Table 1.

order rank  # cubic case
0 1  +  +
1 3 +
2 5 + +
3 7 +
Table 1: Overview of local fermion bilinears that feature full or cubic rotation invariance. We display the tensor rank with respect to and the number of independents components (#) of these tensors. The scalar, vector, and rank-2 tensor order parameters constitute density, and the usual magnetic and nematic orders. We refer to a nonzero expectation value of as nemagnetic ordering. In particular, is the continuum version of the AIAO order on the pyrochlore lattice. We further indicate the behavior of these orders with respect to inversion () and time-reversal (), where indicates even/odd transformation properties, and how the tensors split up into subgroups upon restricting rotations to the cubic group.

iii.2 Local four-fermion couplings

The most general local four-fermion interaction term in the rotation (), inversion, and time-reversal invariant case is given by


The individual terms transform as scalar, vector, second- and third-rank tensors under , respectively. For the latter two this becomes particularly transparent when writing and . There are two Fierz identities


which reveal that the expression (45) contains a certain degree of redundancy that can be removed by eliminating two of the terms. Due to Eq. (46), one of them needs to be either or . Here we choose to eliminate both of them and thus arrive at the Fierz complete interaction term


Together with the Lagrangian from Eq. (1) this constitutes the field theoretic setup considered in Ref. Herbut and Janssen (2014).

To study the cubic symmetric case we introduce


The most general cubic, inversion, and time-reversal symmetric local four-fermion term is given by






Each of the terms transforms as a singlet under the action of the cubic group. There are also five Fierz identities among the (App. C), allowing us to eliminate five couplings. We are thus left with three independent couplings, which we choose to be . Hence


constitutes a Fierz complete interaction term. For it reduces to the expression in Eq. (47). The short-range interaction terms can also be evoked for the study of other systems with four-component fermions, for instance having linear dispersion, such as Weyl semimetals Maciejko and Nandkishore (2014) or quantum critical antiperovskites Isobe and Fu (2015, 2016).

Apart from the eight fermionic vertices that appear in Eq. (49) one may wonder whether the combinations , , , , and can also be generated during the RG flow. This, however, is forbidden by cubic and time-reversal symmetry (), as can be seen as follows: Since is even under , but , , and are odd, the terms explicitly break . Due to the original Lagrangian in Eq. (1) being time-reversal symmetric, no such term can be generated during the RG flow. The terms are forbidden by cubic symmetry. For this consider a rotation around the z-axis by . The coordinate vector transforms as . In the same way, and transform as and , respectively. Accordingly, the term is both time-reversal and cubic symmetric, and indeed emerges in the present RG analysis. On the other hand, transforms under the same rotation as , so that . Again, since the original Lagrangian is cubic symmetric, the term cannot be generated during the RG evolution.

In the isotropic case, the four fermionic vertices appearing in Eq. (45) are chosen such that they have distinct transformation properties under . Accordingly, no mixing between these terms is possible due to symmetry. This changes in the cubic -invariant case, where we observe the term in Eq. (57) to couple and . In fact, for the tensors and have exactly the same symmetries and are thus physically equivalent. Furthermore, every orthogonal pair of linear combinations of and may also be chosen for a basis. Here we define


In Ref. Isobe and Fu (2016) these tensors are labelled and , respectively. As further pointed out in the latter reference, the satisfy the three-dimensional Clifford algebra


They are also generators of an algebra. Due to this extra symmetry, the linear combination is distinguished for . The second linear combination, , then follows as the orthogonal partner, but does not possess any further symmetries. Note that in terms of the original we have


The pseudospin variable introduced in Eq. (10) of Ref. Savary et al. (2014) coincides with up to a prefactor. Further note that in the parametrization of Ref. Murray et al. (2015), where the magnetic channel is written as , corresponds to , which unfortunately falls outside the range considered in the reference.

The Luttinger semimetal considered in this work may be understood as the low-energy effective field theory describing itinerant electrons on a pyrochlore lattice such as in Pyrochlore Iridates. The nemagnetic orders and can then be associated to particular orders of the local magnetic moments of electrons on the four corners of the tetrahedra forming the pyrochlore lattice. As shown in Ref. Goswami et al. (2017), a nonzero expectation value of or one component of corresponds to an all-in-all-out (AIAO) or spin ice (SI) ordering, respectively. (The reference uses the notation and .) The correspondence is facilitated by observing that


from Eqs. (280) and (211). For this reason we will refer to the tensor orders corresponding to and as AIAO and SI ordering, although, of course, these notions only make sense on the pyrochlore lattice.

iii.3 Renormalization group flow

During the RG flow, short-range interactions are generated from long-range interactions by the diagram to the left in Fig. 4. More explicitly, we have a nonvanishing term proportional to in Eqs. (68) and (69) below for the flow of and . This generates and , and they eventually also generate . Once the are present, they couple via the remaining two diagrams in the figure, and thereby lead to a sufficiently rich fixed point structure of the flow.

Figure 4: Schematic loop contributions to the short-range interactions. The assignment of lines is as in Fig. 2, i.e., a straight (wiggly) line represents a fermion (photon) propagator. The box diagram to the left is proportional to and generates local short-range interactions even if they are initially absent. Once present, they contribute to the flow of the short-range couplings via the diagrams shown in the middle and to the right, which are proportional to and , respectively.

The flow equations for the couplings are given by


with as in Eqs. (12) and (14) and


The anisotropy parameter enters through the functions and . We define the latter by


We have for and


for all . In particular, this implies


in the limits of strong anisotropy, since obviously from the very definition.

The flow of the couplings is supplemented by the flow equation for given in Eq. (15), namely


Since the flow equation for is not altered by the short-range interactions, any possible fixed point in the space of couplings necessarily has either or with from Eq. (16). Note that the -functions (67)-(69) depend on and only through the combination . In App. A we show that this behavior results from the fact that the frequency integral of the squared fermion propagator vanishes.

In the isotropic limit ( and ) we recover the flow equations of Ref. Herbut and Janssen (2014) given by


Here we use a different convention for defining the renormalized couplings and than in the reference, see the comment below Eq. (95) for a mapping. The flow equations (75) only support the Abrikosov fixed point for . In dimensions it annihilates with a quantum critical point, and consequently is absent for lower dimensions.

As the anisotropy is varied within the interval , various new fixed points in the space appear as solutions of the RG flow equations. These fixed points typically have several relevant directions and their impact on the phase structure will be discussed below. In order to uniquely identify the Abrikosov fixed point in this zoo of fixed points we define it as the one having only irrelevant directions. (In the four-dimensional coupling space this corresponds to four negative eigenvalues of the stability matrix at the fixed point.) The fixed point defined in this manner indeed satisfies