# Detecting spin fractionalization in a spinon Fermi surface spin liquid

###### Abstract

Motivated by the recent proposal of the spinon Fermi surface spin liquids for several candidate materials such as YbMgGaO, we explore the experimental consequences of the external magnetic fields on this exotic state. Specifically, we focus on the weak field regime where the spin liquid state is well preserved and the spinon remain to be a good description of the magnetic excitations. From the spin-1/2 nature of the spinon excitation, we predict the unique features of the spinon continuum when the weak magnetic field is applied to the system. Due to the small energy scale of the exchange interactions between the local moments in the spin liquid candidate like YbMgGaO, our proposal for the spectral weight shifts and spectral crossing in the magnetic fields can be immediately tested by inelastic neutron scattering experiments. Several other experimental aspects about the spinon Fermi surface and the spinon excitations are discussed and proposed. Our work provides an experimental scheme to examine the fractionalized spinon excitation and the candidate spin liquid states in YbMgGaO, the 6H-B phase of BaNiSbO and other relevant materials.

## I Introduction

A quantum spin liquid (QSL) is an exotic quantum phase of matter that carries long-range quantum entanglements in the modern terms. The association with the long-range quantum entanglements has not yet led to the observable effects for QSLs. A more common description of the QSLs often involves certain emergent gauge structure and the fractionalized spinon excitation Lee (2008); Balents (2010); Savary and Balents (2017). The experimental search of QSLs has lasted for forty years since the original proposal by Anderson in 1973 Anderson (1973, 1987). The absence of magnetic order is often used as the first diagnosis of QSLs in experiments. Many QSL candidate materials have been proposed so far, but the direct confirmation of QSLs has not been achieved in any of these materials. More recently, several materials including YbMgGaO and 6H-B phase of BaNiSbO, have been proposed to realize a spinon Fermi surface spin liquid Shen et al. (2016); Li et al. (2016a, b); Fåk et al. (2017); Serbyn et al. (2011), an exotic state that was originally proposed for the triangular lattice organic materials -(ET)Cu(CN) and EtMeSb[Pd(dmit)] Yamashita et al. (2008); Itou et al. (2008); Shimizu et al. (2003); Itou et al. (2007); Lee and Lee (2005); Motrunich (2005).

The well-known QSL candidates -(ET)Cu(CN) and EtMeSb[Pd(dmit)] are proximate to the Mott transition from the superconductor or metal state to the Mott insulating state Yamashita et al. (2008); Itou et al. (2008); Shimizu et al. (2003); Itou et al. (2007); Kurosaki et al. (2005); Furukawa et al. (2017). Due to the proximity to the Mott transition, the charge gap is small and the charge fluctuation is strong, which induces a sizable four-spin ring exchange interaction Motrunich (2005). This ring exchange interaction competes the pair-wise Heisenberg interaction and frustrates the 120-degree magnetic order. These competing interactions were suggested to be the driving force to enhance the quantum fluctuation of the spin degrees of freedom and help stabilize the QSL ground state in the organic materials Lee and Lee (2005); Motrunich (2005, 2006). Based on the proximity to the Mott transition, the spinon Fermi surface U(1) QSL and their instabilities Lee and Lee (2005); Motrunich (2005, 2006); Lee et al. (2007); Nave et al. (2007); Tang et al. (2013); Ribeiro and Lee (2011); Zhou and Lee (2011); Grover et al. (2010); Katsura et al. (2010); Metlitski et al. (2015); Senthil (2008); Mross and Senthil (2011a); Zou and Senthil (2016) were suggested to provide a resonable and compelling description and/or theoretical prediction of various experimental results such as the thermodynamic and thermal transport properties Itou et al. (2010); Yamashita et al. (2011, 2008); Itou et al. (2008); Shimizu et al. (2003); Itou et al. (2007); Kurosaki et al. (2005); Furukawa et al. (2017). Despite being promising QSL candidates, these organic materials have a rather low spin concentration that prevents the data-rich inelastic neutron scattering measurement at the current laboratory setting. Thus, the full momentum and energy resolved spectroscopic information of the magnetic excitations in these materials is lacking.

In contrast to the weak Mott insulating phase of the organics, both the rare-earth triangular lattice antiferromagnet YbMgGaO and the spin-1 antiferromagnet 6H-B BaNiSbO are in the strong Mott regime where the charge fluctuation is weak. The underlying QSL physics, if there is, should be fundamentally different from the organics. Let us begin with YbMgGaO. This material was first discovered in the powder form and proposed as a gapless QSL Li et al. (2015a), and was later suggested as the first QSL candidate in the strong spin-orbit-coupled Mott insulator with odd electron fillings Li et al. (2015b, 2016c); Shen et al. (2016); Li et al. (2016d, b). The later proposal is compatible with the more fundamental view based on the time reversal symmetry and quantum entanglements Watanabea et al. (2015); Li et al. (2015b, 2016c); Shen et al. (2016); Li et al. (2016b) that states the ground state of strong spin-orbit-coupled Mott insulator with odd electron fillings must be exotic provided the presence of time reversal symmetry. Here, the Yb local moments in YbMgGaO remain disordered down to the lowest measured temperature at which the magnetic entropy is almost exhausted Li et al. (2015a, 2016a); Shen et al. (2016); Paddison et al. (2016); Xu et al. (2016). The low-temperature heat capacity with and the dispersion of the dynamic spin structure from the inelastic neutron scattering on single-crystal samples found a reasonable agreement with the theoretical prediction for the spinon Fermi surface state Shen et al. (2016); Li et al. (2016b); Lee and Nagaosa (1992); Lee and Lee (2005); Motrunich (2005). Unlike YbMgGaO, the 6H-B phase of BaNiSbO has spin-1 local moments Cheng et al. (2011); Quilliam et al. (2016); Fåk et al. (2017); Chen et al. (2012); Bieri et al. (2012); Kaul (2012); Xu et al. (2012); Serbyn et al. (2011). Nevertheless, both earlier theoretical suggestion and the recent inelastic neutron scattering experiments propose that the spinon Fermi surface QSL is realized in this material Serbyn et al. (2011); Fåk et al. (2017).

In general, there are two major questions concerning the candidate QSL ground states of YbMgGaO and 6H-B BaNiSbO. The first and probably the most crucial one is whether the excitation continuum in the inelastic neutron scattering is truly a spinon continuum and represents the spin quantum number fractionalization. The second question is the microscopic mechanism for the QSL behavior in these materials. For YbMgGaO, it was suggested that the anisotropic interaction of the local moments, due to the spin-orbit entanglement, could enhance the quantum fluctuation and destabilize the ordered phases Li et al. (2015b, 2016c, 2016d). These two questions have been partially addressed by the mean-field theory Shen et al. (2016) and the later projective symmetry group analysis Shen et al. (2016); Li et al. (2016b) that identify the spinon Fermi surface U(1) QSL as the candidate ground state for YbMgGaO. For 6H-B BaNiSbO, the phenomenological approach based on fermionic partons with competing exchange interactions also suggests a spinon Fermi surface state Fåk et al. (2017); Serbyn et al. (2011).

Ideally, it would be nice to directly solve the relevant microscopic spin model and see if one can obtain any QSL ground state in the phase diagram, then both questions may be completely resolved. Due to the complication of the models, this is difficult even numerically Li et al. (2016c, b); Zhu et al. (2017); Luo et al. (2017). In this work, instead of directly tackling the microscopic spin model Li et al. (2016c, d); Liu et al. (2016), we work on the spinon mean-field Hamiltonian Shen et al. (2016); Li et al. (2016b) that has provided a reasonable description of the inelastic neutron scattering results for YbMgGaO. To ensure the nature of the spinon continuum in the inelastic neutron scattering results, we propose a simple experimental scheme to test the spin quantum number fractionalization and confirm the spinon excitations. We suggest to apply a weak external magnetic field and study the spectral weight shifts of the dynamic spin structure factor. The splitting of the degenerate spinon bands by the magnetic field is directly revealed by the spinon particle-hole continuum that is detected by the dynamic spin structure factor. We show that the persistence of the spinon continuum, the spectral weight shifts and the spectral crossing around the point, the existence of the upper and lower excitation edges under the weak magnetic field represent unique properties of the spinon excitation for the spinon Fermi surface state, and thus provide a visible experimental prediction for the identification of the spinon excitation with respect to the spinon Fermi surface. In the bulk of the paper, we illustrate this idea with an effective phenomenological spinon model for YbMgGaO, and the approach can be adjusted to other systems with little modification.

The remaining part of the paper is organized as follows. In Sec. II, we explain our view on the magnetic excitation continuum and the weak spectral weight in the inelastic neutron scattering results on YbMgGaO and motivate our approach in this paper. In Sec. III, we justify the mean-field Hamiltonian in the magnetic field. In Sec. IV, we obtain the dynamic spin structure factor from the free spinon theory in the magnetic field and explain the spectral weight shifts. In Sec. V, we include the spinon interactions with a random phase approximation in our results. Finally in Sec. VI, we conclude with a discussion about various future experimental direction for the spinon Fermi surface state.

## Ii The spinon Fermi surface state

We start with the fermionic parton representation for the spin operator such that

(1) |

where () creates (annihilates) one spinon with spin () at the lattice site and is a vector of Pauli matrices. This construction is further supplemented by a Hilbert space constraint . At the mean-field level, the following spinon Hamiltonian,

(2) | |||||

was proposed for YbMgGaO and gives a large spinon Fermi surface Shen et al. (2016); Li et al. (2016b). Here we introduce the second neighbor spinon hopping in addition to the first neighbor spinon hopping on the lattice, although the first neighbor spinon hopping is sufficient to give the spinon Fermi surface state. Adding the second neighbor spinon hopping could give a mean field state with a better variational energy. Moreover, the spin rotational symmetry for the first and the second neighbor spinon hoppings in Eq. (2) is required by the lattice symmetry that is realized projectively for the spinons Li et al. (2016b). Finally, the chemical potential is introduced to impose the Hilbert space constraint.

It was found that the spinon particle-hole excitation of the simple spinon Fermi surface ground state of Eq. (2) provides a consistent magnetic excitation continuum with the inelastic neutron scattering experiments. Moreover, the anisotropic spin interaction, that is included into the the spinon mean-field theory by a random phase approximation (RPA), gives a weak spectral peak at the M points, which is consistent with the experimental observation Shen et al. (2016); Paddison et al. (2016).

The spinon continuum itself is certainly more important than the weak spectral peak at low energies. The weak spectral peak at certain momenta merely represents some collective mode of the spinons that is enhanced by the residual and short-range interaction between the fermionic spinons, and is quite common for example in the Fermi liquids of electrons as an analogy. Nevertheless, the spectral peak does provide hints about the form of the microscopic interactions. In contrast, the spinon continuum is a consequence of the spin quantum number fractionalization that reveals the defining nature of QSLs.

Since we think the spinon continuum is more important and the spinon continuum is already obtained by the free-spinon theory of , our approach will mostly rely on the free-spinon mean-field theory and focus on the spinon continuum rather than the weak spectral peak. The (short-range) anisotropic spin interaction will be included into the free-spinon theory in the later parts of the paper. The coupling to the gapless U(1) gauge photon is not included throughout this paper. Nevertheless, the spinon-gauge coupling should have an important effect on the low-energy properties of the system Lee and Nagaosa (1992); Lee and Lee (2005).

## Iii Coupling to the magnetic field

Unlike the electron, the fermionic spinon is a charge neutral object and does not couple to the external magnetic field via the conventional Lorentz coupling. Here, we point out that the prior theory on the organic spin liquid material Shimizu et al. (2003) -(ET)Cu(CN) has actually invoked the interesting Lorentz coupling of the spinons to the external magnetic field indirectly through the internal U(1) gauge flux Motrunich (2006). This is because the organic material -(ET)Cu(CN) is in the weak Mott regime where the charge gap is small and the four-spin ring exchange interaction can be significant Motrunich (2005). It is the four-spin ring exchange that connects and transfers the external magnetic flux to the internal emergent U(1) gauge flux Motrunich (2006). In contrast, the electrons of the Yb ions are in the strong Mott regime and is very localized. As we have explained, the effective spin arises from the strong spin-orbit coupling (SOC) and crystal electric field splitting, and the four-spin ring exchange is strongly suppressed due to the very large on-site interaction of the electrons. Therefore, the orbital coupling to the magnetic field of the spinons in the organic spin liquid does not apply to YbMgGaO, and there will not be the spinon Landau level nor Hofstadter band structures. Although a strong magnetic field would polarize the Yb local moments along the field direction and thus destabilizes the spin liquid state, in the weak field regime, the field does not change the spin liquid ground state and the spinon remains to be a valid description of the magnetic excitation. From the above argument, if YbMgGaO ground state is a spinon Fermi surface QSL, the appropriate spinon mean-field Hamiltonian for YbMgGaO in a weak external magnetic field should be

(3) | |||||

where only Zeeman coupling is needed, and is the Land factors for the field normal to the triangular plane of the Yb atoms, respectively. The mean-field Hamiltonian in Eq. (3) will be the basis of the analysis below.

## Iv Spectral weight shifts from the free-spinon mean-field theory

As the magnetic field is varied, the microscopic spin Hamiltonian that includes the Zeeman coupling is modified. Thus, for each magnetic field, the spinon hopping and the chemical potential in Eq. (3) need to be adjusted to optimize the variational energy of the microscopic spin Hamiltonian that is

(4) | |||||

Here ’s are the bond-dependent phase variables that arises from the spin-orbit coupling of the Yb electrons Li et al. (2015b, 2016c, 2016d, 2016b), and for along the , , bond, respectively. The recent polarized neutron scattering measurement has provided a clear support for the above nearest-neighbor anisotropic spin Hamiltonian for YbMgGaO Toth et al. (2017). Moreover, the experiment did not find strong signature of the exchange disorder Toth et al. (2017); Li et al. (2017). In the following calculation of this paper, we set Li et al. (2015b, 2016c). The -direction magnetic field shifts the chemical potential for the spin- and spin- spinons up and down such that the spinon excitations are given by

(5) | |||||

(6) |

where is the dispersion that is obtained from the first line of Eq. (3). In Fig. 1, we plot the mean-field dispersions of the spinons in the magnetic field, where the spin up and spin down spinons have different Fermi surfaces. Therefore, in the weak field regime, the system remains gapless.

In the inelastic neutron scattering measurement, the neutron spin flip excites the spinon particle-hole pairs across the spinon Fermi surface. The spin-flip event corresponds to the inter-band particle-hole excitation, and we will focus on this process in the following discussion. In the free-spinon mean-field theory, the energy and momentum change of the neutron, and , is shared by the one spinon particle-hole pair, and we have

(7) | |||||

(8) |

In the mean-field theory, the field essentially breaks the degenerate spinon bands by separating the dispersions of spin- and spin- spinon bands in energy with a Zeeman splitting. Thus, there exists a large density of particle-hole excitations at the zero momentum transfer with the energy (see the illustration in Fig. 1b). We thus expect a spectral peak at the point with a finite energy transfer . In the absence of the magnetic field, we have shown in Refs. Shen et al., 2016; Li et al., 2016b that the spectral weight at the finite energy is suppressed at the point.

In Fig. 2, we calculate and plot the dynamic spin structure factor for various values of the magnetic field on a lattice of size . Here selects the spin-flipping events and is given by

(9) |

where is the system size and the expectation is taken with respect to the spinon Fermi surface ground state of the mean-field Hamiltonian. In the above equation, the time integration yields a delta function for the energy conservation, and we replace the delta function by with in the calculation. First we observe that the spectrum remains gapless at the weak field regime. Second, we indeed find that the spectral weight at the point occurs at the energy transfer due to the splitted structure of the spinon bands that were discussed previously.

Apart from the enhancement of the spectral intensity at the point and the Zeeman splitting energy, there is an interesting spectral crossing near the Zeeman splitting energy around the point. This is the unqiue consequence of the external magnetic field on the spinon continuum. To understand this phenomenon, we start from the vertical particle-hole transition in Fig. 1b and slightly tilt the transition such that the momentum transfer of the neutron is finite but small (see Fig. 2h). Depending on the tilted direction of the momentum, the energy transfer can be greater or smaller than and take the value , where we have linearized the dispersion and in the weak field limit. In principle, the velocity would depend on the momenta of the spinon particle and spinon hole, but for the convenience of presentation, such dependence is not indicated. The neutron energy transfer is thus located within the energy range , and this explains the upper and lower excitation edges near the point.

The above behaviors of spinon continuum in the weak magnetic field are qualitatively different from what one would expect for the magnon-like excitation. For the magnons that are integer-spin excitations, the magnetic field directly couples to the magnon, most often shifts the whole magnon band by gapping out the low-energy modes. The magnon lifetime becomes longer, the magnon quasi-particle become sharper, and the magnon band would be narrowed. In contrast, for the spinons that are spin-1/2 excitations, the magnetic field shfits the spin-up spinon and the spin-down spinon bands oppositely, and the spin-up and spin-down spinons are combined together to give the inter-band particle-hole contribution in the dynamic spin structure factor. The spinon continuum in the magnetic field is sensitive to the dispersions of both spinon bands and thus reflects the fractionalized nature of the magnetic excitations. It is hard to imagine the broad excitation continuum, the spectral crossing at and , and the upper and lower excitation edges near the point in Fig. 2 can be obtained from the magnon-like excitation under the magnetic field.

Another feature of the spinon continuum in the weak magnetic field is the suppression of the overall intensity. Originally at the zero field, the spectral weight is suppressed above an upper excitation edge (see Fig. 2a). The weak magnetic fields create spectral weights near the point at finite energies, i.e. in the regions where the spectral weights are suppressed at the zero field. Therefore, the overall intensity of the continuum is suppressed at the small magnetic field.

To further manifest two excitation edges near the point and , we depict the dynamic spin structure factor in the Brillouin zone for different neutron energy transfers in Fig. 3. The intensity distribution within the Brillouin zone further reflects the variation of the spinon band structure under the magnetic field. For the energy and the momentum below the lower excitation edge in Fig. 2, the spectral weight is strongly suppressed, leading to a reduced spectral intensity around the points (see Fig. 3a). Likewise, for the energy above the upper excitation edge in Fig. 2, the spectral intensity around the points is similarly suppressed (see Fig. 3b). Nevertheless, for energy right at , the spectral intensity around the points is not suppressed due to the reason that was explained previously.

The energy distribution curve (EDC) of at certain momenta is a common measurement with inelastic neutron scattering that could reveal important information of the ground state. We depict the EDC in Fig. 4. The peak at the point and the shift of the peak position under the magnetic field are the most salient experimental features, and can be readily probed using inelastic neutron scattering and/or optical measurements. Off the point, the spectral peak in the EDC becomes broad since the finite momentum transfer allows a range of energies for the spinon particle-hole continuum, and the energy range of the peak is bounded by the upper and lower excitation edges.

Finally, we comment on the caveat of the mean-field theory. In the mean-field theory for the spinons, we have ignored the spinon-gauge coupling. The gapless U(1) gauge photon is expected to play an important role at low energies. For example, the Yukawa coupling between the fermionic spinons and the gapless U(1) gauge photon would give rise a self-energy correction to the spinon Green’s function and thus enhance the low-energy density of states Lee and Nagaosa (1992); Lee and Lee (2005). Therefore, the inelastic neutron scattering process that excites the spinon particle-hole pair, would have an enhanced spectral weight at the low energies. This property is not captured in the spinon mean-field theory. We thus expect the very low energy spectral weights in Figs. 2,4 and also in Fig. 5 to be enhanced when the gauge flucutation is included. Moreover, the slight enhancement of the overall bandwidth of the spinon continuum in the field is probably a mean-field artifact as well because the spinon bandwidth is expected to be set by the exchange interaction of the system.

## V The RPA correction from the anisotropic interaction

As we have proposed in Ref. Li et al., 2016c, the anisotropic spin exchange terms and from the strong SOC in Eq. (4) is likely to play an important role in stabilizing the QSL ground state. The SOC is further suggested to be responsible for the weak spectral peak at the M point Shen et al. (2016); Li et al. (2016d, b). Here we consider the effect of the anisotropic spin interaction on the dynamic spin structure factors following a phenomenological approach introduced in Refs. Li et al., 2016b; Brinckmann and Lee, 1999. Starting from the free-spinon theory and the corresponding susceptibility , we treat the anisotropic interaction , that includes the and exchange interactions, as perturbations. The resulting magnetic susceptibility is calculated in the random phase approximation (RPA) Brinckmann and Lee (1999),

(10) |

where is the exchange matrix from ,

(11) |

with , , and .

The RPA corrected dynamic spin structure factor is related by the equation . The renormalized dynamic spin structure factor is shown in the Fig. 5, where we choose the parameters to be , . From the results we conclude that the anisotropic exchange terms merely redistribute the spectral weight within the Brillouin zone and leave the qualitative features in the vicinity of the point mentioned in previous sections unaffected.

## Vi Discussion

Our prediction of the behaviors of the spinon continuum in the magnetic field relies on the spinon mean-field theory and the proposed spinon Fermi surface QSL state. In the calculation, we have applied the field along the direction (i.e. normal to the triangular plane). Due to the spin rotation symmetry of the mean-field spinon Hamiltonian in Eq. (3), we expect the orientation of the magnetic field to lead to qualitatively similar effects on the inelastic neutron spectrum. The microscopic spin model did not play a significant role in our experimental prediction, but the feasibility of the experiments in the magnetic field strongly relies on the fact that the microscopic interactions between the local moments are of the order of 110 Kelvins Li et al. (2015b). A couple Tesla magnetic field applied to the material could readily lead to a visible effect on the spinon continuum. In contrast, the exchange couplings of other spin liquid candidate materials such as the triangular lattice organic spin liquids Shimizu et al. (2003); Itou et al. (2008, 2007) and herbertsmithite Han et al. (2012); Mendels et al. (2007); Helton et al. (2007) are of the order of 100K, and a much larger field is required there.

The previous works on YbMgGaO Paddison et al. (2016); Li et al. (2017) (including two of our earlier theoretical work Li et al. (2016c, d)) have explored the strong field regime, where the magnetic excitations are simply the gapped magnons with respect to an almost fully polarized state along the field directions. These works may provide useful information about the spin interaction, but do not give information about the spin quantum number fractionalization since the ground state is no longer a spin liquid state. In the zero field case Shen et al. (2016); Li et al. (2016b) where the QSL physics was proposed Li et al. (2015a, b), we have shown that the spinon continuum develops a broad continuum in the momentum and energy domain. In particular, due to the particle-hole excitation near the Fermi surface, there exists a “V-shaped” excitation edge at low energies near the point (see Fig. 2a).

In the weak magnetic field regime, the spinon bands are splitted.
The spin-flipping particle-hole process is no longer degenerate
with the spin-preserving particle-hole process. The inelastic
neutron scattering measurement, that detects the
correlations of the spin components perpendicular to the momentum,
includes both the spin-flipping and the spin-preserving processes.
The spin-preserving process corresponds to the intra-band process,
and we would expect a “V-shaped” excitation edge at low energies
near the point that is analogous to the zero-field results.
In contrast, the spin-flipping process corresponds to the inter-band
particle-hole excitation and is studied in this paper. For the inter-band
process, our theoretical predictions of

1) the spectral weight shifts,

2) the spectral crossing at the point at the Zeeman splitting
energy,

3) the upper and lower excitation edges near the point,

directly reveal the fractionalized spinon excitations.
If these predictions for the weak field regime are confirmed experimentally,
it will provide further support for the spinon Fermi surface QSL ground
state in YbMgGaO.

Apart from the spin quantum number fractionalization, the emergence of the Fermi statistics for the spinons is a rather unusual phenomenon. Although the particular spectral structure of the spinon continuum is a direct consequence of the spinon Fermi surface and the spinon Fermi statistics, directly confirming the Fermi statistics is certainly desirable. The temperature dependence of the dynamic spin structure factor could provide hints for the Fermi statistics. Moreover, the spin-orbital-entangled nature of the Yb local moments may provide a route to visualize the spinon Fermi surface. Clearly, the orbital degrees of freedom are sensistive to the ion position. The Yb local moment, that results from the spin-orbital entanglement, may be more susceptible to the lattice degrees of freedom than the conventional spin-only moment. Therefore, like the electron-phonon coupling in Fermi liquids, one may expect a similar “” Kohn anomaly Mross and Senthil (2011b) in the phonon spectrum that arises from the spinon-phonon coupling in YbMgGaO and use the Kohn anomaly to construct the spinon Fermi surface.

Recently, there is an experimental work Yuesheng et al. (2017) that proposes a scenario of nearest-neighbor resonant valence bond state for YbMgGaO and claims “the excitation continuum bears no obvious relation to spinons”. In fact, the nearest-neighbor resonant valence bond state on a frustrated lattice like the triangular lattice is precisely a spin liquid with gapped spinons and visons Moessner (2001); Moessner and Sondhi (2001); Balents et al. (2002). Thus, the magnetic excitation revealed by the inelastic neutron scattering for such a spin liquid scenario has to be spinons. If such a senario is correct, doping this material will lead to superconductivity since the doped resonant valence bond state was suggested to one origin for the high temprature superconductivity in cuprates Anderson (1987); Lee et al. (2006) However, it is hard to reconcile the gapped spinon continuum with the particular dispersive continuum along high symmetry momenta in Ref. Shen et al., 2016. Moreover, such a scenario does not seem to be compatible with the heat capacity, magnetic susceptibility and SR results on this material Li et al. (2015a, 2016a); Paddison et al. (2016). On the theoretical side, the scenario of disorder was recently proposed Zhu et al. (2017). It might be an interesting possibility. The spectroscopic property about the magnetic excitation from this disorder scenario has not yet been given. The direct comparison and prediction with the inelastic neutron scattering results would certainly be desirable.

While the main focus of this paper is on YbMgGaO, our results for the spinon continuum may generally be applicable to other candidate materials with spinon Fermi surfaces. For the 6H-B phase of BaNiSbO, the Curie-Weiss temperature is about K and sets the bandwidth of the spinons Cheng et al. (2011). The spin-1 nature of the local moment enhances the Zeeman coupling, so a 10T magnetic field would have a visible effect on the spinon dispersion. This may be useful to differentiate other theoretical proposals on this material Chen et al. (2012); Bieri et al. (2012); Xu et al. (2012). If the ground state of this material is a spinon Fermi surface QSL, we expect similar effect of the spectral weight shift and spectral crossing may occur.

## Vii Acknowledgements

We acknowledge useful discussion with Jun Zhao and Patrick Lee, and constructive comments from an anonymous referee. This work is supported by the Ministry of Science and Technology of China with the Grant No.2016YFA0301001, the Start-Up Funds and the Program of First-Class University Construction of Fudan University, and the Thousand-Youth-Talent Program of China. G.C. thanks the hospitality of Prof Ying Ran at Boston College during the visit in January 2017 when this work is finalized.

## References

- Lee (2008) Patrick A. Lee, “An end to the drought of quantum spin liquids,” Science 321, 1306–1307 (2008).
- Balents (2010) Leon Balents, “Spin liquids in frustrated magnets,” Nature 464, 199–208 (2010).
- Savary and Balents (2017) Lucile Savary and Leon Balents, “Quantum spin liquids: a review,” Reports on Progress in Physics 80, 016502 (2017).
- Anderson (1973) P.W. Anderson, “Resonating valence bonds: A new kind of insulator?” Materials Research Bulletin 8, 153–160 (1973).
- Anderson (1987) P.W. Anderson, “The Resonating Valence Bond State in LaCuO and Superconductivity,” Science 235, 1196–1198 (1987).
- Shen et al. (2016) Yao Shen, Yao-Dong Li, Hongliang Wo, Yuesheng Li, Shoudong Shen, Bingying Pan, Qisi Wang, H. C. Walker, P. Steffens, M Boehm, Yiqing Hao, D. L. Quintero-Castro, L. W. Harriger, Lijie Hao, Siqin Meng, Qingming Zhang, Gang Chen, and Jun Zhao, “Evidence for a spinon Fermi surface in a triangular-lattice quantum-spin-liquid candidate,” Nature 540, 559â562 (2016).
- Li et al. (2016a) Yuesheng Li, Devashibhai Adroja, Pabitra K. Biswas, Peter J. Baker, Qian Zhang, Juanjuan Liu, Alexander A. Tsirlin, Philipp Gegenwart, and Qingming Zhang, “Muon Spin Relaxation Evidence for the U(1) Quantum Spin-Liquid Ground State in the Triangular Antiferromagnet ,” Phys. Rev. Lett. 117, 097201 (2016a).
- Li et al. (2016b) Yao-Dong Li, Yuan-Ming Lu, and Gang Chen, “The Spinon Fermi Surface U(1) Spin Liquid in a Spin-Orbit-Coupled Triangular Lattice Mott Insulator YbMgGaO,” arXiv preprint 1612.03447 (2016b).
- Fåk et al. (2017) B. Fåk, S. Bieri, E. Canévet, L. Messio, C. Payen, M. Viaud, C. Guillot-Deudon, C. Darie, J. Ollivier, and P. Mendels, ‘‘Evidence for a spinon Fermi surface in the triangular quantum spin liquid ,” Phys. Rev. B 95, 060402 (2017).
- Serbyn et al. (2011) Maksym Serbyn, T. Senthil, and Patrick A. Lee, “Exotic spin-liquid state with fermionic excitations on the triangular lattice,” Phys. Rev. B 84, 180403 (2011).
- Yamashita et al. (2008) Satoshi Yamashita, Yasuhiro Nakazawa, Masaharu Oguni, Yugo Oshima, Hiroyuki Nojiri, Yasuhiro Shimizu, Kazuya Miyagawa, and Kazushi Kanoda, “Thermodynamic properties of a spin-1/2 spin-liquid state in a kappa-type organic salt,” Nature Physics 4, 459–462 (2008).
- Itou et al. (2008) T. Itou, A. Oyamada, S. Maegawa, M. Tamura, and R. Kato, “Quantum spin liquid in the spin- triangular antiferromagnet ,” Phys. Rev. B 77, 104413 (2008).
- Shimizu et al. (2003) Y. Shimizu, K. Miyagawa, K. Kanoda, M. Maesato, and G. Saito, “Spin liquid state in an organic mott insulator with a triangular lattice,” Phys. Rev. Lett. 91, 107001 (2003).
- Itou et al. (2007) T Itou, A Oyamada, S Maegawa, M Tamura, and R Kato, “Spin-liquid state in an organic spin-1/2 system on a triangular lattice, EtMeSb[Pd(dmit)],” Journal of Physics: Condensed Matter 19, 145247 (2007).
- Lee and Lee (2005) Sung-Sik Lee and Patrick A. Lee, “U(1) Gauge Theory of the Hubbard Model: Spin Liquid States and Possible Application to ,” Phys. Rev. Lett. 95, 036403 (2005).
- Motrunich (2005) Olexei I. Motrunich, “Variational study of triangular lattice spin- model with ring exchanges and spin liquid state in ,” Phys. Rev. B 72, 045105 (2005).
- Kurosaki et al. (2005) Y. Kurosaki, Y. Shimizu, K. Miyagawa, K. Kanoda, and G. Saito, “Mott transition from a spin liquid to a fermi liquid in the spin-frustrated organic conductor ,” Phys. Rev. Lett. 95, 177001 (2005).
- Furukawa et al. (2017) Tetsuya Furukawa, Kazuhiko Kobashi, Yosuke Kurosaki, Kazuya Miyagawa, and Kazushi Kanoda, “Quasi-continuous transition from a fermi liquid to a spin liquid,” arXiv preprint 1707.05586 (2017).
- Motrunich (2006) Olexei I. Motrunich, “Orbital magnetic field effects in spin liquid with spinon fermi sea: Possible application to ,” Phys. Rev. B 73, 155115 (2006).
- Lee et al. (2007) Sung-Sik Lee, Patrick A. Lee, and T. Senthil, “Amperean Pairing Instability in the U(1) Spin Liquid State with Fermi Surface and Application to ,” Phys. Rev. Lett. 98, 067006 (2007).
- Nave et al. (2007) Cody P. Nave, Sung-Sik Lee, and Patrick A. Lee, “Susceptibility of a spinon fermi surface coupled to a u(1) gauge field,” Phys. Rev. B 76, 165104 (2007).
- Tang et al. (2013) Evelyn Tang, Matthew P. A. Fisher, and Patrick A. Lee, “Low-energy behavior of spin-liquid electron spectral functions,” Phys. Rev. B 87, 045119 (2013).
- Ribeiro and Lee (2011) P. Ribeiro and P. A. Lee, “Magnetic impurity in a spin liquid with a spinon fermi surface,” Phys. Rev. B 83, 235119 (2011).
- Zhou and Lee (2011) Yi Zhou and Patrick A. Lee, “Spinon phonon interaction and ultrasonic attenuation in quantum spin liquids,” Phys. Rev. Lett. 106, 056402 (2011).
- Grover et al. (2010) Tarun Grover, N. Trivedi, T. Senthil, and Patrick A. Lee, ‘‘Weak mott insulators on the triangular lattice: Possibility of a gapless nematic quantum spin liquid,” Phys. Rev. B 81, 245121 (2010).
- Katsura et al. (2010) Hosho Katsura, Naoto Nagaosa, and Patrick A. Lee, “Theory of the thermal hall effect in quantum magnets,” Phys. Rev. Lett. 104, 066403 (2010).
- Metlitski et al. (2015) Max A. Metlitski, David F. Mross, Subir Sachdev, and T. Senthil, ‘‘Cooper pairing in non-fermi liquids,” Phys. Rev. B 91, 115111 (2015).
- Senthil (2008) T. Senthil, “Theory of a continuous mott transition in two dimensions,” Phys. Rev. B 78, 045109 (2008).
- Mross and Senthil (2011a) David F. Mross and T. Senthil, “Decohering the fermi liquid: A dual approach to the mott transition,” Phys. Rev. B 84, 165126 (2011a).
- Zou and Senthil (2016) Liujun Zou and T. Senthil, “Dimensional decoupling at continuous quantum critical mott transitions,” Phys. Rev. B 94, 115113 (2016).
- Itou et al. (2010) T. Itou, A. Oyamada, S. Maegawa, and R. Kato, “Instability of a quantum spin liquid in an organic triangular-lattice antiferromagnet,” Nature Physics 6, 673–676 (2010).
- Yamashita et al. (2011) Satoshi Yamashita, Takashi Yamamoto, Yasuhiro Nakazawa, Masafumi Tamura, and Reizo Kato, “Gapless spin liquid of an organic triangular compound evidenced by thermodynamic measurements,” Nature Communications 2, 275 (2011).
- Li et al. (2015a) Yuesheng Li, Haijun Liao, Zhen Zhang, Shiyan Li, Feng Jin, Langsheng Ling, Lei Zhang, Youming Zou, Li Pi, Zhaorong Yang, Junfeng Wang, Zhonghua Wu, and Qingming Zhang, “Gapless quantum spin liquid ground state in the two-dimensional spin-1/2 triangular antiferromagnet YbMgGaO,” Scientific Reports 5, 16419 (2015a).
- Li et al. (2015b) Yuesheng Li, Gang Chen, Wei Tong, Li Pi, Juanjuan Liu, Zhaorong Yang, Xiaoqun Wang, and Qingming Zhang, “Rare-Earth Triangular Lattice Spin Liquid: A Single-Crystal Study of ,” Phys. Rev. Lett. 115, 167203 (2015b).
- Li et al. (2016c) Yao-Dong Li, Xiaoqun Wang, and Gang Chen, “Anisotropic spin model of strong spin-orbit-coupled triangular antiferromagnets,” Phys. Rev. B 94, 035107 (2016c).
- Li et al. (2016d) Yao-Dong Li, Yao Shen, Yuesheng Li, Jun Zhao, and Gang Chen, “The effect of spin-orbit coupling on the effective-spin correlation in YbMgGaO,” arXiv preprint 1608.06445 (2016d).
- Watanabea et al. (2015) Haruki Watanabea, Hoi Chun Po, Ashvin Vishwanath, and Michael Zaletel, “Filling constraints for spin-orbit coupled insulators in symmorphic and nonsymmorphic crystals,” PNAS 112, 14551–14556 (2015).
- Paddison et al. (2016) Joseph A. M. Paddison, Zhiling Dun, Georg Ehlers, Yaohua Liu, Matthew B. Stone, Haidong Zhou, and Martin Mourigal, “Continuous excitations of the triangular-lattice quantum spin liquid YbMgGaO,” Nature Physics 13, 117–122 (2016).
- Xu et al. (2016) Y. Xu, J. Zhang, Y. S. Li, Y. J. Yu, X. C. Hong, Q. M. Zhang, and S. Y. Li, “Absence of Magnetic Thermal Conductivity in the Quantum Spin-Liquid Candidate ,” Phys. Rev. Lett. 117, 267202 (2016).
- Lee and Nagaosa (1992) Patrick A. Lee and Naoto Nagaosa, “Gauge theory of the normal state of high- superconductors,” Phys. Rev. B 46, 5621–5639 (1992).
- Cheng et al. (2011) J. G. Cheng, G. Li, L. Balicas, J. S. Zhou, J. B. Goodenough, Cenke Xu, and H. D. Zhou, “High-Pressure Sequence of Structural Phases: New Quantum Spin Liquids Based on ,” Phys. Rev. Lett. 107, 197204 (2011).
- Quilliam et al. (2016) J. A. Quilliam, F. Bert, A. Manseau, C. Darie, C. Guillot-Deudon, C. Payen, C. Baines, A. Amato, and P. Mendels, “Gapless quantum spin liquid ground state in the spin-1 antiferromagnet 6HB-,” Phys. Rev. B 93, 214432 (2016).
- Chen et al. (2012) G. Chen, M. Hermele, and L. Radzihovsky, “Frustrated Quantum Critical Theory of Putative Spin-Liquid Phenomenology in ,” Phys. Rev. Lett. 109, 016402 (2012).
- Bieri et al. (2012) Samuel Bieri, Maksym Serbyn, T. Senthil, and Patrick A. Lee, “Paired chiral spin liquid with a Fermi surface in model on the triangular lattice,” Phys. Rev. B 86, 224409 (2012).
- Kaul (2012) Ribhu K. Kaul, “Spin nematic ground state of the triangular lattice biquadratic model,” Phys. Rev. B 86, 104411 (2012).
- Xu et al. (2012) Cenke Xu, Fa Wang, Yang Qi, Leon Balents, and Matthew P. A. Fisher, “Spin Liquid Phases for Spin-1 Systems on the Triangular Lattice,” Phys. Rev. Lett. 108, 087204 (2012).
- Zhu et al. (2017) Z. Zhu, P.A. Maksimov, S.R. White, and A.L. Chernyshev, “Disorder-induced Mimicry of a Spin Liquid in YbMgGaO,” arXiv preprint arXiv:1703.02971 (2017).
- Luo et al. (2017) Qiang Luo, Shijie Hu, Bin Xi, Jize Zhao, and Xiaoqun Wang, “Ground-state phase diagram of an anisotropic spin- model on the triangular lattice,” Phys. Rev. B 95, 165110 (2017).
- Liu et al. (2016) Changle Liu, Rong Yu, and Xiaoqun Wang, “Semiclassical ground-state phase diagram and phase of a spin-orbit-coupled model on triangular lattice,” Phys. Rev. B 94, 174424 (2016).
- Toth et al. (2017) Sandor Toth, Katharina Rolfs, Andrew R. Wildes, and Christian Ruegg, “Strong exchange anisotropy in ybmggao from polarized neutron diffraction,” arXiv preprint 1705.05699 (2017).
- Li et al. (2017) Yuesheng Li, Devashibhai Adroja, Robert I. Bewley, David Voneshen, Alexander A. Tsirlin, Philipp Gegenwart, and Qingming Zhang, “Crystalline Electric-Field Randomness in the Triangular Lattice Spin-Liquid ,” Phys. Rev. Lett. 118, 107202 (2017).
- Brinckmann and Lee (1999) Jan Brinckmann and Patrick A. Lee, “Slave Boson Approach to Neutron Scattering in Superconductors,” Phys. Rev. Lett. 82, 2915–2918 (1999).
- Han et al. (2012) Tian-Heng Han, Joel S Helton, Shaoyan Chu, Daniel G Nocera, Jose A Rodriguez-Rivera, Collin Broholm, and Young S Lee, “Fractionalized excitations in the spin-liquid state of a kagome-lattice antiferromagnet,” Nature 492, 406–410 (2012).
- Mendels et al. (2007) P. Mendels, F. Bert, M. A. de Vries, A. Olariu, A. Harrison, F. Duc, J. C. Trombe, J. S. Lord, A. Amato, and C. Baines, “Quantum magnetism in the paratacamite family: Towards an ideal kagomé lattice,” Phys. Rev. Lett. 98, 077204 (2007).
- Helton et al. (2007) J. S. Helton, K. Matan, M. P. Shores, E. A. Nytko, B. M. Bartlett, Y. Yoshida, Y. Takano, A. Suslov, Y. Qiu, J.-H. Chung, D. G. Nocera, and Y. S. Lee, “Spin dynamics of the spin- kagome lattice antiferromagnet ,” Phys. Rev. Lett. 98, 107204 (2007).
- Mross and Senthil (2011b) David F. Mross and T. Senthil, “Charge Friedel oscillations in a Mott insulator,” Phys. Rev. B 84, 041102 (2011b).
- Yuesheng et al. (2017) Li Yuesheng, Devashibhai Adroja, David Voneshen, Robert I. Bewley, Qingming Zhang, Alexander A. Tsirlin, and Philipp Gegenwart, “Nearest-neighbor resonating valence bonds in YbMgGaO,” Nature Communications, arXiv:1704.06468 8, 15814 (2017).
- Moessner (2001) R Moessner, “Magnets with strong geometric frustration,” Canadian Journal of Physics 79, 1283–1294 (2001).
- Moessner and Sondhi (2001) R. Moessner and S. L. Sondhi, “Resonating valence bond phase in the triangular lattice quantum dimer model,” Phys. Rev. Lett. 86, 1881–1884 (2001).
- Balents et al. (2002) L. Balents, M. P. A. Fisher, and S. M. Girvin, “Fractionalization in an easy-axis kagome antiferromagnet,” Phys. Rev. B 65, 224412 (2002).
- Lee et al. (2006) Patrick A. Lee, Naoto Nagaosa, and Xiao-Gang Wen, “Doping a mott insulator: Physics of high-temperature superconductivity,” Rev. Mod. Phys. 78, 17–85 (2006).