# Interplay between orbital-quantization effects and the Fulde-Ferrell-Larkin-Ovchinnikov instability in multiple-band layered superconductors

###### Abstract

We explore superconducting instability for a clean two-band layered superconductor with deep and shallow bands in the magnetic field applied perpendicular to the layers. In the shallow band, the quasiclassical approximation is not applicable, and Landau quantization has to be accounted for exactly. The electronic spectrum of this band in the magnetic field is composed of the one-dimensional Landau-level minibands. With increasing magnetic field the system experiences series of Lifshitz transitions when the chemical potential enters and exits the minibands. These transitions profoundly influence the shape of the upper critical field at low temperatures. In addition, the Zeeman spin splitting may cause the nonuniform state with interlayer modulation of the superconducting order parameter (Fulde-Ferrell-Larkin-Ovchinnikov state). Typically, the quantization effects in the shallow band strongly promote the formation of this state. The uniform state remains favorable only in the exceptional resonance cases when the spin-splitting energy exactly matches the Landau-level spacing. Furthermore, for specific relations between electronic spectrum parameters, the alternating FFLO state may realize, in which the order parameter changes sign between the neighboring layers. For all above cases, the reentrant high-field superconducting states may emerge at low temperatures if the shallow band has significant contribution to the Cooper pairing.

## I Introduction

The nature of superconducting instability in the magnetic field is a long-standing fundamental problem. In clean type-II superconductors, the upper critical field and its temperature dependence are sensitive to the electronic band properties as well as to the gap structure which contains crucial information on how the correlated Cooper pairs are formed. The magnitude of is mostly determined by the suppression of superconductivity due to the quasiparticles orbital motion and the Zeeman spin splitting of the Fermi surface by the magnetic field. The problem of orbital for a single-band system was solved in the seminal papers Helfand and Werthamer (1966); *Werthamer:PRev147.1966 within the quasiclassical approximation which neglects the Landau quantization of the orbital motion. This approach works with high accuracy in common metals with large Fermi energies and describes very well most of known conventional superconductors. Nevertheless, the exact Landau quantization calculation was discussed in Refs. Gunther and Gruenberg, 1966; *Gruenberg:PRev176.1968; Rajagopal and Vasudevan, 1966 shortly after the quasiclassical work and the topic was further elaborated in great details laterTešanović and Rasolt (1989); *Tesanovic:PRB43.1991; Rieck et al. (1990); MacDonald et al. (1992); Norman et al. (1992); Maniv et al. (1992); *Maniv:RMP73.2001.

Landau-quantization effects are most relevant when the Fermi energy is comparable to the cyclotron frequency so that only a few lowest Landau levels are occupied. Such extreme quantum limit has not been viable in most of known superconducting materials. The situation changed with the recent discovery of the iron-based superconductors (FeSC). It is very common in FeSCs that some of the bands have very small Fermi energies and they can be driven through the Lifshitz transition by chemical dopingsLiu et al. (2010); Xu et al. (2013); Miao et al. (2015); Shi et al. (2017) and, importantly, these materials usually remain superconducting after one of the bands is completely depleted. Even though these shallow bands cannot be treated within the conventional BCS approach, they may strongly influence the superconducting pairingInnocenti et al. (2010); Chen et al. (2015); Valentinis et al. (2016); Chubukov et al. (2016). In addition, FeSCs are characterized by the very high upper critical fields, up to 100 tesla.

As the orbital upper critical field is inversely proportional to the Fermi energy, it is strongly affected by the shallow bands. Moreover, in the vicinity of the Lifshitz transition, the extreme quantum limit can be reached for these bands near meaning that the quasiclassical consideration cannot be used. It has been indeed demonstrated that the Landau quantization has a dramatic effect on superconducting instability in the magnetic field for two-dimensional materials near the Lifshitz transitionSong and Koshelev (2017). The most spectacular prediction is the emergence of the pronounced reentrant states at the magnetic field corresponding to the matching of the Landau levels with the chemical potential.

In addition to the orbital effect, shallow bands also promote the Zeeman spin splitting suppression of superconductivity. The relative role of the spin and orbital mechanisms is usually characterized by the Maki parameter defined as , where and are the orbital and spin upper critical fields respectively. In a clean isotropic single-band material with the free-electron -factor, the Maki parameter is proportional to the ratio of the superconducting gap and the Fermi energy , . In this case, unless the band is very shallow, is very small meaning that the orbital effect strongly dominates. The Maki parameter is greatly enhanced in special cases of weak orbital effect such as quasi-one-dimensional materials and layered superconductors for magnetic field directed along the layers.

An important consequence of the strong Zeeman effect (i.e., large ) is the emergence of the nonuniform Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) state in very clean materials Fulde and Ferrell (1964); Larkin and Ovchinnikov (1964). In this state, the order parameter is periodically modulated which allows for a gain in the Zeeman energy which may exceed the kinetic-energy loss due to the nonzero center-of-mass momentum pairing. Rich physics of the FFLO state has been extensively investigated in many subsequent theoretical studies within quasiclassical approximation Gruenberg and Gunther (1966); Takada and Izuyama (1969); Machida and Nakanishi (1984a); Burkhardt and Rainer (1994); Shimahara (1994); *Shimahara:JPSJ66.1997; *Shimahara:PRB80.2009; Houzet and Buzdin (2000); Buzdin et al. (2005); Dao et al. (2013); Croitoru and Buzdin (2014). The FFLO instability may only realize in very pure materials with weak scattering of quasiparticles. That is why, even though the FFLO state was predicted more than half century ago, the experimental data consistent with this state have been reported only relatively recently in organic superconductors Beyer and Wosnitza (2013); Agosta et al. (2017) and, less convincingly, in the heavy-fermion compounds Kumagai et al. (2006); Matsuda and Shimahara (2007). The conditions for the FFLO instability with modulation along the magnetic field in isotropic material were studied by Gruenberg and GuntherGruenberg and Gunther (1966) within the quasiclassical framework. They found that the nonuniform state appears only for very large Maki parameters, , which is unlikely to realize in any single-band isotropic material. That is why most FFLO studies have been focused on quasi-low-dimensional materials with strongly reduced orbital effects.

The FFLO state may realize in the iron-based superconductors due to the presence of shallow bands and huge upper critical fields which are likely to be limited by the Zeeman effect. This motivated recent investigations of the conditions for the emergence of this state in multiple-band materials within the quasiclassical approach in different situations Gurevich (2010); Mizushima et al. (2014); *Takahashi:PRB89.2014; Adachi and Ikeda (2015). However, in the presence of very shallow bands, this approach may be insufficient and quantization effects have to be accounted for.

In this paper, we investigate the superconducting instability for a two-band layered material in the magnetic field applied perpendicular to the layers. We consider the case of the material near the Lifshitz transition when one of the bands is very shallow. In this case, the Landau-quantization effects strongly influence the formation of the superconducting state. Even though our consideration is motivated by physics of iron-based superconductors, it is very general, and our goal is not to describe any particular compound but, instead, to develop a general understanding of how the orbital-quantization effects influence the onset of superconductivity in clean two-band layered materials. The major new feature in comparison with a pure two-dimensional case Song and Koshelev (2017) is that, due to the large Zeeman effect in the shallow band, this system is prone to the formation of the FFLO state with interlayer modulation of the order parameter. The quantum effects have a profound influence on this FFLO instability.

The interlayer tunneling lifts all the degenerate Landau levels to dispersive minibands along the out-of-plane momentum direction. As a consequence, the system experiences series of Lifshitz transitions with increasing the magnetic field corresponding to crossing of the chemical potential with the miniband edges. Every Landau-level miniband has two van Hove singularities at the reduced z-axis momentums and , at which the density of states (DoS) is enhanced. In special situations, when two such singular points for spin-up and spin-down bands simultaneously match the Fermi level, the pairing strongly enhanced. Two distinct cases of such resonance matching are possible. The first well-known case is realized when spin-splitting energy equal to the Landau-level spacing Maniv et al. (1992); Norman et al. (1992). In this case, the van Hove points of the same kind (either or ) may match leading to the standard uniform superconducting state. The second case corresponds to matching of the opposite van Hove points, e. g., spin-down/ and spin-up/ points, which may occur only for a certain relation between the electronic band parameter. In this situation, the alternating FFLO state may emerge, in which the superconducting order parameter changes sign between the neighboring layers. These matching effects may generate the high-field reentrant superconducting states which are somewhat less pronounced than for two-dimensional caseSong and Koshelev (2017) due to the DoS spreading by the interlayer tunneling.

On the other hand, the Landau-level spreading somewhat mitigates the Zeeman pair-breaking effect, since the dispersive spin-up and spin-down minibands can cross the Fermi level simultaneously within a finite energy range for arbitrary spin splitting. In such generic situation, the shallow band favors the formation of the FFLO state, in which the optimal modulation wave vector equals to the difference between the spin-up and spin-down Fermi momenta. Such FFLO instability leads to different kind of reentrant states with the field-dependent modulation wave vectors. In contrast to the reentrant states caused by the matching of the van Hove singularities, the latter states can extend over a broad magnetic-field range.

This paper is organized as follows. In section II, we introduce the microscopic Hamiltonian describing two-band layered superconductor in magnetic field and derive the corresponding linearized gap equations. In section III, we briefly discuss these equations for the transition temperature in zero magnetic field providing the reference for the further investigation of instability in finite magnetic field. In section IV, we derive the equations for the upper critical field and evaluate the pairing kernels in these equations. In section V, we discuss the dependences of the quantum pairing kernel on relevant parameters. In section VI, we present the typical magnetic field versus temperature phase diagrams. We conclude the paper by section VII.

## Ii The model of a two-band layered superconductor

We will investigate the shallow band effects in layered superconductors using the simple tight-binding Hamiltonian with only the nearest-neighbor interlayer hopping term,

(1) |

where is the in-plane coordinate, is the layer index, represents spin, and () represents the -band (-band). Furthermore, is the (nearest) interlayer hopping energy, and are intralayer energy dispersions with the band masses and and the momentum operator . In the model, we also consider the Zeeman spin splitting and assuming the band electron’s magnetic moment, , be the same in each band for simplicity. In the second line of Eq. (1), is the line integration along the out-of-plane direction and is the interlayer spacing (set to unity in the later calculation) and is the -component of the vector potential. In zero magnetic field, the three-dimensional energy dispersions are , see Fig. 1(a).

In this paper, we limit ourselves to the case of the magnetic field applied in the direction. In this case, with standard gauge choices and the magnetic phase factor in Eq. (1) tight-binding terms drops out. For definiteness, we consider the case when the hole band is shallow meaning that the chemical potential is located near its edge . We note that whether the shallow band is hole-like or electron-like does not have any influence on the results of this paper. The two Lifshitz transitions in this model occur at [see Fig. 1(b)] Lifshitz (1960). At the neck near interrupts and the hole band got completely depleted.

To study the superconducting instabilities in the model Eq. (1), we follow the standard approach and write the linearized gap equation as

(2) |

where the gap function is defined as and we used the notation , are the Matsubara frequencies, and the kernel is

(3) |

where is the one-particle Green’s function in the normal phase and the subscripts or describe spin orientation. Without magnetic field, the normal Green’s functions can be expanded into the plane-wave basis

(4) |

where and . In the presence of out-of-plane magnetic fields, in the symmetric gauge the one-particle Green’s function can be represented as

(5) |

where is the magnetic length and . We use the quasiclassical approximation for the Green’s function of the deep -band

(6) |

and expand the Green’s function of the shallow -band over the exact Landau-level basis Rajagopal and Ryan (1991); Maniv et al. (1992)

(7) |

where , , is the cyclotron frequency, and are the Laguerre polynomials.

## Iii Transition temperature in zero magnetic field ()

For zero magnetic field, the superconducting order parameter is homogeneous . This gives the linearized gap equation, which we represent as

(8) |

where we have introduced the notations for the coupling matrix with being the 2D densities of states, and

(9) |

The kernels are determined by the zero-field Green’s function in Eq. (4) giving

see Appendix A.1. We further integrate out the wave vectors in by using for -band ( for -band) and . Here, is the energy cutoff and we have assumed that . We therefore obtain the following gap equation near

(10) |

where with

(11a) | ||||

(11b) |

, and is the EulerâMascheroni constant. Here the parameter is the value of the temperature-dependent function

(12) |

at . This parameter appears due to the cut off at the band edge for the shallow band. We can carry out -integration and the Matsubara-frequency sum in using the relation . This gives

(13) |

where is the Bessel’s function. In the limit of low temperatures, , this function diverges logarithmically,

The transition temperature is determined by the condition of degeneracy of the matrix , i. e., . This equation determines as

(14) |

with and . This parameter directly determines by Eq. (11a). In the following consideration, we will use the zero-field equation, Eq. (10), to eliminate the logarithmic divergencies from the gap equation at finite magnetic field.

## Iv Superconducting instability in magnetic field

In the presence of magnetic field, the superconducting order parameter is nonuniform. Near the onset of the superconducting instabilities in the magnetic field, the solutions for the linear problem in Eq. (2) are given by the Landau-level eigenfunctions. Typically, the lowest Landau-level eigenfunction yields the optimal , where the superconducting instability develops first
^{1}^{1}1For strong Zeeman spin-splitting, the largest may be realized for higher Landau-level eigenfunction Buzdin and Brison (1996a); *Buzdin:PLettA218.1996.
. This solution has the following shapeGruenberg and Gunther (1966)

(15) |

We have assumed the possibility of the layer-to-layer modulation in the order parameters with the wave vector . ^{2}^{2}2To be accurate, the complex order parameter in Eq. (15) was suggested by Fulde and Ferrel Fulde and Ferrell (1964). As demonstrated by Larkin and OvchinnikovLarkin and Ovchinnikov (1964), the ground state below the transition is actually given by the order parameter with the amplitude modulation .
We only investigate the instability location here, which is the same for both states.
Such modulation is the realization of the nonuniform FFLO state Fulde and Ferrell (1964); Larkin and Ovchinnikov (1964).
Superconducting instability is determined by the condition that Eq. (15) provides a solution of the gap equation at least for one value of and then this optimal modulation wave vector realizes in the emerging superconducting state.
The ansatz in Eq. (15) is the kernel eigenfunction in the magnetic field,

(16) |

with

(17) |

Here and below we use notation for the averaging with respect to . We omit the dependence of on , , and the electronic parameters. As follows from the definitions of the Green’s functions, Eqs. (6) and (7), . Next, we first discuss the kernel eigenvalue for the deep -band and then for the shallow -band.

For the deep -band, the Landau quantization does not play a role. Using Eqs. (6) and (17), linearizing the band dispersion at Fermi level [, where with the Fermi wave vector ], we obtain the following quasiclassical result for the kernel eigenvalue

(18) |

where . The derivation details are described in the Appendix A.2.1.

For the shallow -band, substituting the Green’s function, Eq. (7), into the general presentation in Eq. (17), and using relation , we derive

(19) | ||||

where we introduced the following notations

(20a) | ||||

(20b) |

Here all “barred” normalized quantities are defined as (with ). Furthermore, is the reduced spin-splitting parameter, where is the free-electron mass and is the band-electrons -factor.

Therefore, the gap equation, Eq. (2), in the presence of the magnetic field becomes

(21) |

The Matsubara-frequency sums, , are logarithmically-divergent and have to be cut at . Similarly to the two-dimensional case Song and Koshelev (2017), we can regularize the gap equation using its zero-field counterpart, Eq. (10). Namely, we decompose as with being the only log-diverging term. This leads to

(22) |

Here are the temperature-dependent parts of the pairing eigenvalues, and , where the function is defined in Eq. (13). The field-dependent parts, , are

(23a) | |||

(23b) |

Note that, by definition, and . Therefore, the UV cutoffs are explicitly removed and the Matsubara-frequency sums in the right-hand side in Eqs. (23a) and (23b) converge now in the limit of . We can represent the functions in this limit as

(24a) | ||||

(24b) |

where we introduced notations , . Furthermore, is the -band cyclotron frequency, , , , and . We remind that the parameters and also depend on , see Eqs. (20a) and (20b). We describe derivation of in the Appendix A.2.2. The double sum in Eq. (24a) collects the contributions to pairing coming from the quasiparticles located at the Landau levels and . The quantum kernel eigenvalue depends on five independent dimensionless parameters: the reduced magnetic field , the reduced temperature , the modulation wave vector , the ratio , and the spin-splitting factor .

The problem is reduced to the solution of equation

(25) |

for a given and , and we need to find the optimized in for which the instability develops first. As the matrix is degenerate, this equation can be rewritten as

(26) |

This is our main equation for determination of the upper critical field in two-band layered superconductors. All information about the coupling matrix is contained in the two parameters, and . The analytical expressions for these parameters can be derived from Eq. (10), see also Ref. Song and Koshelev (2017),

(27a) | ||||

(27b) |

with , , and

(28) |

As follows from Eq. (26) the weights with which the bands contribute to the pairing near the upper critical field scale as .

The behavior of the upper critical field is mostly determined by the shape of the kernel eigenvalues . In general, larger values of correspond to stronger pairing strength. In the next section, we will explore in detail the quantum kernel eigenvalue .

## V Behavior of the quantum pairing kernel eigenvalue

The shape of magnetic field-temperature phase diagrams is determined by the behavior of the pairing kernel eigenvalues . While the quasiclassical kernel eigenvalue is well studied and has monotonic dependence on the magnetic field for all temperatures, the quantum kernel eigenvalue has rather complicated nonmonotonic dependence on the magnetic field which is very sensitive to the electronic-spectrum parameters and as well as the spin-splitting factor . The electronic spectrum of the shallow band in the magnetic field is composed of the Landau-level minibands with width (see Fig. 2a). The system has series of Lifshitz transitions with increasing magnetic field when the chemical potential enters or exits a particular miniband. At low temperatures, the magnetic-field dependences of have features at these transitions whenever the chemical potential crosses the van Hove points at the miniband edges. There are two such points for every miniband corresponding to two values of , and . In addition, there are two minibands for every Landau level for two spin orientations. This gives four miniband-edge magnetic fields per Landau level, , corresponding to cyclotron frequencies

(29) |

where (/) describes spin orientation and and corresponds to and respectively. In addition, we have to consider the behavior of for different modulation wave vectors . In general, as the larger corresponds to stronger pairing strength, the shallow band favors the states which maximize .

Before proceeding to the investigation of the magnetic field-temperature phase diagrams for two-band systems, it is very instructive to study the analytical properties of the function , particularly, in the low-temperature limit. We first focus on its singularities when the magnetic field crosses the typical values in Eq. (29) and on identifying the possible divergencies for , since these features have important implications on the superconducting instabilities. Next, we consider the magnetic field dependences of for representative cases. Furthermore, by studying the dependences of for different Zeeman spin-splitting parameters, we identify possible parameter ranges for FFLO instabilities.

### v.1 The low-temperature limit and its divergencies

In this subsection, we investigate the leading divergencies of the quantum kernel eigenvalues as for different cases. We review first the behavior of the quasiclassical kernel eigenvalue, , Eq. (24b). For all magnetic fields, has the same logarithmic divergency as so that approaches a finite value. In the typical situation of a moderate spin-splitting factor, , it can be treated perturbatively. For the uniform case, , we derive from Eq. (24b) the result for the zero-temperature limit of the full quasiclassical kernel eigenvalue

(30) | |||

The parameter is just the ratio , where is the upper critical field of the deep band at zero temperature. In the case of finite the parameter has to be replaced by the function . The closed analytical result for is not available even for . One can only derive an approximate result in the limits, and

(31) |

Generally, the quasiclassical kernel eigenvalue is a monotonically decreasing function of at all temperatures and spin-splitting parameters meaning that the magnetic field always suppresses superconductivity. As expected, it has maximum at in the limit meaning that the deep band favors the uniform state.

The quantum kernel eigenvalue typically behaves similarly to , i. e., it has the same logarithmic divergency as , so that the total kernel eigenvalue approaches a finite value in the zero-temperature limit. This zero temperature value, however, has singular contributions when the magnetic field crosses the typical values given by Eq. (29), corresponding to Lifshitz transitions for the Landau-level minibands. We discuss these singularities in the next subsection. In several exceptional resonant cases, when two miniband-edge fields with opposite spin orientations are identical, the Landau quantization leads to faster divergency . In the case when these two fields originate from the van-Hove singular points of the same type (either or ) the divergency occurs in the uniform channel . On the other hand, if the two fields correspond to the opposite van-Hove points, the divergency takes place in the alternating channel . We discuss both these cases below. Another divergency appears within the field ranges where the chemical potential simultaneously crosses two minibands with opposite spin orientations. In this case, the total kernel eigenvalue diverges logarithmically for at the optimal wave vector connecting the minibands’ Fermi momenta, see Fig. 2(b). All these low- divergencies may lead to the high-field superconducting states.

#### v.1.1 Square-root singularity of the pairing kernel at the miniband-edge fields

As discussed above, the system has series of the miniband Lifshitz transitions at the magnetic fields given by Eq. (29). In this subsection we discuss singularity of the pairing kernel eigenvalue at these transition, when the cyclotron frequency crosses the miniband-edge value . We consider here only a general nondegenerate situation when the corresponding transition magnetic field, , is separated from other typical fields. The derivation in Appendix B.1 gives the result for the singular contribution at zero temperature

(32) |

with being the step function and

We see that the square-root singularity appears near the transition point when the chemical potential is inside the Landau-level miniband, i.e., or . It reflects the pairing enhancement caused by the square-root divergency of the density of state at the edge of one-dimensional miniband. Finite temperature smears this singularity.

#### v.1.2 Resonant cases for the uniform state ()

In the uniform state, , the resonant condition corresponds to the matching of the Zeeman spin-splitting energy, , and the Landau-level energy spacing, , giving with integer . The divergency occurs when the Fermi level matches the van Hove singular point at or corresponding to the magnetic field

(33) |

Fig. 2(a) illustrates the simplest case with . In this case we derive in Appendix B.2 the following asymptotic behavior for