FeshbachStabilized Insulator of Bosons in Optical Lattices
Abstract
Feshbach resonances  namely resonances between an unbound twobody state (atomic state) and a bound (molecular) state, differing in magnetic moment  are a unique tool to tune the interaction properties of ultracold atoms. Here we show that the spinchanging interactions, coherently coupling the atomic and molecular state, can act as a novel mechanism to stabilize an insulating phase  the Feshbach insulator  for bosons in an optical lattice close to a narrow Feshbach resonance. Making use of quantum Monte Carlo simulations and meanfield theory, we show that the Feshbach insulator appears around the resonance, preventing the system from collapsing when the effective atomic scattering length becomes negative. On the atomic side of the resonance, the transition from condensate to Feshbach insulator has a characteristic firstorder nature, due to the simultaneous loss of coherence in the atomic and molecular components. These features appear clearly in the groundstate phase diagram of e.g. Rb around its 414 G resonance, and they are therefore directly amenable to experimental observation.
pacs:
05.30.Jp, 03.75.Hh, 67.85.Hj, 03.75.MnIntroduction. Feshbach resonances offer an invaluable tuning knob to control quantum manybody phenomena in ultracold atoms Chinetal2010 . An (unbound) state of two interacting atoms and a bound state  hereafter called molecule  are brought into resonance by the application of a magnetic field thanks to the different magnetic moments of the two states. As the two states are coupled by spinchanging interactions, their resonance allows to control the effective scattering length of unbound atoms both in magnitude and sign. This latter aspect has been widely used experimentally to explore quantum manybody phases Greineretal2003 ; KetterleZ2008 ; RanderiaT2014 ; Winkleretal2006 ; Chinetal2010 and to tune transitions between them Jordensetal08 ; Deissleretal2010 . On the other hand the coherent coupling between atoms and molecules has been exploited e.g. to observe atommolecule Rabi oscillations Syassenetal2007 ; Olsenetal2009 ; at the theory level, this coupling is at the basis of the prediction of a quantum phase transition betwen mixed atommolecule and purely molecular condensates Radzihovskyetal2004 ; Romansetal2004 ; SenguptaD2005 ; Radzihovskyetal2008 ; Ejima_2011 ; Bhaseenetal2012 .
A fundamental trait of the atommolecule coupling is its nonlinear nature: only pairs of atoms can be converted to molecules, leading to a strong density dependence of atommolecule conversion (AMC). Here we shall show that, in the case of bosons in optical lattices, this aspect alone can drive the system towards an insulating phase. Indeed the AMC can pin the particle density to two particles per site, opening a particlehole (ph) gap in the spectrum and leading to insulating behavior. We dub this state of matter a Feshbach insulator (FI). It appears at a Feshbach resonance sufficiently narrow for the repulsive interaction to prevent the collapse of the atomic cloud. AMC can be visualized as the coherent pair hopping of atoms into a fictitious, secondary lattice hosting the molecules (see Fig. 1). In this picture the mechanism stabilizing the FI is the appearance of strong entanglement between neighboring sites in the fictitious extra dimension, and the consequent suppression of entanglement in the realspace dimensions, which is instead at the heart of condensation.
A conservative definition of a FI is that of an insulating phase whose ph gap vanishes when the AMC is suppressed; as we shall later discuss, the AMC rate is in fact tunable in the experiment, so that this characterization has a direct operational meaning. Yet the same AMCdriven mechanism, which opens a ph gap in the FI from scratch, can be found to enhance the preexisting gap of a molecular or atomic Mott insulator (MI); in this respect, the FI can be continuously connected to a (Feshbachenhanced) MI. This picture is to be contrasted with the conventional one in which the AMC perturbatively renormalizes the atomatom or moleculemolecule interactions via dressing the atoms (molecules) with virtual molecules (atoms): in this case ph gaps are reduced by the AMC. We show that all three regimes of AMC (FI, Feshbachenhanced MI and Feshbachsuppressed MI) are remarkably exhibited in the theoretical phase diagram of Rb close to its (narrow) 414 G resonance; moreover we discuss the condensateFI transition, which in dimensions is found to have a strong firstorder nature.
AtomMolecule Hamiltonian. We model spinless bosons in an optical lattice close to a narrow Feshbach resonance via a singleband BoseHubbard model footnote:H with atomic and molecular bosons, coherently coupled via spinchanging atomatom interactions Koehleretal2006 . The Hamiltonian of the system reads , where
(1)  
(3) 
The operator corresponds to the kinetic energy for hopping between nearest neighboring sites defined on a dimensional hypercubic lattice with periodic boundary conditions. The and ( and ) operators are bosonic creation and annihilation operators of atoms (molecules) on site . and are the corresponding number operators. The operator contains the intraspecies and interspecies interactions, as well as the chemical potential term; in particular it contains the detuning term (controlled experimentally by a magnetic field Chinetal2010 ), which brings the state of two atoms and a molecule in and out of resonance on each site, () corresponding to the molecular (atomic) side of the resonance. Finally the operator is the hyperfine coupling converting two atoms into a molecule and viceversa.
The above atommolecule Hamiltonian on a lattice has been mainly studied in RousseauD2008 ; EckholtR2010 ; Ejima_2011 ; Bhaseenetal2012 for some peculiar choices of the numerous Hamiltonian parameters. Here we rather focus on the case of and 3 footnote:oneD , which we investigate numerically by a combined strategy based on Gutzwiller meanfield theory (MFT) in and 3, supplemented in with numerically exact quantum Monte Carlo (QMC) simulations based on the Stochastic Green Function algorithm SGF . MFT is found to correctly capture the succession of phases in the system, and it allows for the rapid reconstruction of phase diagrams. The exact nature of the phase transitions encountered with MFT has also been systematically investigated with QMC, and will be the subject of a future publication. The discussion of our results is structured as follows. First, employing an idealized choice of Hamiltonian parameters, we show how the AMC term can open a ph gap in the spectrum, giving rise to a Feshbach insulator. We then move on to investigating the occurrence of the FI regime for realistic parameters related to Rb in an optical lattice.
The Feshbach Insulator in a 2D system. We begin our discussion of the groundstate physics of a square lattice by imposing and . This establishes an (artificial) symmetry between the physics of atoms and molecules, which greatly simplifies the behavior of the system and reduces the number of parameters to four only: and .
Fig. 2(a) shows the phase diagram with fixed ratio as a function of the detuning and of the chemical potential , as obtained via a MF calculation. In the absence of coupling between atoms and molecules (inset of Fig. 2(a)) the only phases appearing in the system are an atomic BoseEinstein condensate (BECa), a molecular BEC (BECm) and the vacuum – our specific choice of the ratio does not allow for the appearance of finitedensity (Mott) insulating phases in this case (see inset of Fig. 2(a)). On the other hand, when ( in Fig. 2(a)) the phase diagram changes dramatically. The most striking feature of the phase diagram at is the occurrence of a broad FI insulating region at fixed total density footnote:resonance . Its incompressible nature (, see Fig. 2(a)) reveals the existence of a ph gap which can be readily estimated from the chemicalpotential width of the FI region, and which stems from the nonlinear dependence of the conversion term on the occupancy footnote:gap .
An important ingredient for the stabilization of a homogeneous FI is the choice of a narrow Feshbach resonance. Indeed, given that the potential energy scales as and the conversion energy scales as , one finds that the two compensate each other at a density . To achieve a homogeneous state, one needs therefore ( for the example in question); if this is not the case, atoms and molecules tend to gradually cluster on single sites, with a cluster density . This clustering effect is found to initially lead to the destruction of the insulating phase and to the appearance of a simultaneous atomic/molecular BEC (BECam – see below); but for a very large it ultimately leads to the collapse of the gas (see Supplementary Material (SM) Supplementary for a detailed discussion).
A direct experimental consequence of the ph gap induced by the AMC in the FI is a finite coherence length. Its inverse is related to the full width at half maximum (FWHM) of the peak in the molecular momentum distribution , as well as in the atomic one. To correctly estimate the FWHM we make use of QMC: as shown in Fig. 2(c) (calculated for a detuning ), the FWHM of the molecular momentum distribution as a function of exhibits the quantum phase transition (QPT) from BECm to FI. Further increasing leads to a second QPT from FI to BECam (or possibly to a very thin BECm phase prior to entering in the BECam), when the conversion energy overcomes the interaction energy leading to clustering as discussed above.
When lowering the hopping to , the phase diagram with (inset of Fig. 2(b)) features a molecular MI phase. As shown in Fig. 2(b), introducing a has the effect of enhancing the ph gap (namely the width of the insulating phase), and consequently of increasing the FWHM, as shown in Fig. 2(d), giving rise to a Feshbachstabilized MI of molecules and atoms (MIam). Direct inspection into the structure of the ph excitations Supplementary shows that the mechanism of stabilization of the MI is analogous to that opening the gap in the FI.
Phase diagram of Rb in a lattice. We now turn our attention to a realistic implementation of the atommolecule coherence model with a narrow Feshbach resonance, featuring most of the salient features observed in the symmetric atommolecule mixture. To make contact with common experimental setups, we consider this time a 3D cubic optical lattice, and estimate the parameters and from atomic and molecular maximally localized Wannier functions Supplementary . The atommolecule coherence is estimated starting from the solution of the scattering problem for two particles in a harmonic potential Buschetal1998 ; Syassenetal2007 ; Supplementary . We have focused on the two narrow resonances at G in Rb, and at G for Na, extensively investigated experimentally Syassenetal2007 ; Inouyeetal1998 ; Stengeretal1999 ; footnote:narrowFeshbach . The Hamiltonian parameters around these resonances have been estimated for a lattice depth where is the recoil energy footnote:recoil . We then scan the phase diagram as a function of chemical potential , controlled by the trapping potential, and detuning , controlled by the applied magnetic field, making use of MFT (expected to predict phase boundaries with an accuracy of in 3D systems). As the two resonances in Rb and Na feature a similar phase diagram, in the following we shall focus on the case of Rb only. For the microscopic parameters take values , , , and . With this choice of the lattice depth, the atommolecule conversion term is quite sizable (), yet a Mott insulator of atoms cannot be stabilized at any filling (neither at the meanfield level nor at the exact level Capogrossoetal2007 ): therefore insulating behavior at finite density on the atomic side of the resonance is necessarily induced by the conversion term, namely it has the nature of a FI.
Fig. 3(a) shows the meanfield phase diagram for Rb at the above cited resonance. In the absence of atommolecule conversion (inset) the system features a BECa phase and a molecular MI (MIm) phase with . When , on the other hand, the BEC phase acquires a BECam nature, and similarly the MIm acquires an atomic component (MIam). The ph gap of the insulating region is found to shrink substantially on the molecular () side: this is the wellknown effect of Feshbach resonances, renormalising in this case the moleculemolecule interaction – and hence the ph gap — in the molecular MI via the conversion to virtual atom pairs (see footnote:nonmonotonic for the complementary effect on the atomic side ). Nonetheless, for the ph gap is instead found to open because of the conversion term: it vanishes in the limit Supplementary , and it is still found to grow with the conversion rate around the value of Rb. As a consequence an insulating phase is found to persist up to on the atomic side of the resonance. Hence, even if the insulating phase on the atomic side is continuously connected with that on the molecular side, it is clear that the conversion term has an opposite role on the atomic side, opening a ph gap instead of shrinking it. The insulating region whose ph gap grows with has therefore the nature of a FI. Its onset is found to correspond to a change in sign of the derivative , to which one can associate a change in sign of the derivative of the FWHM for the atomic momentum distribution (see Fig. 3(b)). Both aspects can be used as a direct experimental signature – see below.
As already seen in the previous example, the transition from the BECam to all the insulating phases is found to be strongly firstorder. Remarkably, the unconventional nature of the BECinsulator transition in this system can be directly observed in the experiments using stateoftheart diagnostics. Fig. 4 shows the evolution of the coherence peak across the BECamFI transition for a trapped system with trapping frequency Hz, footnote:normalization . The data for the local mean fields are obtained from singlesite MFT via a local density approximation. We follow a trajectory in the plane  thin dashed magenta line in Fig. 3(a)  along which the density in trap center is fixed at in the BECam phase – the density jumps to when entering in the FI. Along this realistic trajectory in parameter space we observe a very sharp jump of both the atomic and the molecular coherent peak as the trap center crosses the BECinsulator transition footnote:nonmonotonic . A further accessible experimental evidence of the firstorder nature of the BECinsulator transition comes from the density profile. As shown in the SM Supplementary , when the trap center is in the FI regime, the density jumps twice upon moving towards the trap wings: once when going from FI to BECam, and a second time when going from BECam to vacuum.
Conclusions. We have shown that coherent atommolecule coupling at a narrow Feshbach resonance offers a novel mechanism stabilizing an insulating phase for bosons in an optical lattice – the Feshbach insulator (FI). The appearance of a thermodynamically stable FI shows that an optical lattice can actively protect the bosonic cloud against the two main enemies of ultracold resonant bosons in continuum space, namely 1) collapse on the attractive side of the resonance; 2) rapid threebody recombination on the repulsive side Remetal2013 ; Fletcheretal2013 ; Makotynetal2014 – as triple occupancy of a site is largely suppressed in the FI. How to distinguish experimentally a FI from a conventional Mott insulator? As discussed above, FI behavior manifests itself for lattice depths at which atoms (and possibly even molecules) are far from a MI phase at all fillings: in this case, loss of atomic (and molecular) coherence upon tuning the system towards resonance is a clear manifestation of the appearance of a FI regime. Unlike conventional MI’s, a FI consists of an almost equal mixture of atoms and molecules, – an aspect directly accessible to experiments via SternGerlach separation during the cloud expansion Herbigetal2003 or speciesselective imaging Syassenetal2007 . Atommolecule coherence can in principle be measured on small samples via the analysis of momentumnoise correlations between atoms and molecules Supplementary . But the most direct probe of the nature of the FI comes from the evolution of its spectral properties and coherence length upon tuning the atommolecule coherence , as shown in Fig. 2(c) and Fig. 3(b). The parameter can indeed be tuned below its intrinsic value via a periodic modulation of the magnetic field (already used in the experiments Thompsonetal2005 to resonantly associate molecules), which in turn drives a periodic modulation of the detuning . As shown in the SM Supplementary , for (namely far from the molecular binding energy) this leads to an effective renormalisation of to , where is the zeroth order Bessel function. This aspect paves the way for an unambiguous detection of Feshbachstabilized insulating phases in current experimental setups.
Acknowledgements. We thank S. Dürr for useful discussions, and M. EckholtPerotti for her contributions at the early stages of this project. All calculations were performed on PSMN cluster (ENS Lyon). This work is supported by the ANRJCJC programme (ArtiQ project).
References
 (1) C. Chin, R. Grimm, P. Julienne, and E. Tiesinga, Rev. Mod. Phys. 82, 1225 (2010).
 (2) M. Greiner, C. A. Regal, and D. S. Jin, Nature (London) 426, 537 (2003).
 (3) M. Randeria and E. Taylor, Annu. Rev. Condens. Matter Phys. 5, 209 (2014).
 (4) W. Ketterle and M. W. Zwierlein, in Ultracold Fermi Gases, Proceedings of the International School of Physics ”Enrico Fermi”, Course CLXIV, edited by M. Inguscio, W. Ketterle, and C. Salomon (IOS Press, Amsterdam, 2008).
 (5) K. Winkler, G. Thalhammer, F. Lang, R. Grimm, J. Hecker Denschlag, A. J. Daley, A. Kantian, H. P. B üchler, and P. Zoller, Nature (London) 441, 853 (2006).
 (6) R. Jördens, N. Strohmaier, K. Günter, H. Moritz, and T. Esslinger, Nature (London) 455, 204 (2008).
 (7) B. Deissler, M. Zaccanti, G. Roati, C. D’Errico, M. Fattori, M. Modugno, G. Modugno, and M. Inguscio, Nat. Phys. 6, 354 (2010).
 (8) N. Syassen, D. M. Bauer, M. Lettner, D. Dietze, T. Volz, S. Dürr, and G. Rempe, Phys. Rev. Lett. 99, 033201 (2007).
 (9) M. L. Olsen, J. D. Perreault, T. D. Cumby, and D. S. Jin, Phys. Rev. A 80, 030701(R) (2009).
 (10) L. Radzihovsky, J. Park, and P. B. Weichman, Phys. Rev. Lett. 92, 160402 (2004).
 (11) M. W. J. Romans, R. A. Duine, S. Sachdev, and H. T. C. Stoof, Phys. Rev. Lett. 93, 020405 (2004).
 (12) K. Sengupta and N. Dupuis, Europhys. Lett. 70, 586 (2005).
 (13) L. Radzihovsky, P. B. Weichman, and J. I. Park, Ann. Phys. (Amsterdam) 323, 2376 (2008).
 (14) S. Ejima, M. J. Bhaseen, M. Hohenadler, F. H. L. Essler, H. Fehske, and B. D. Simons, Phys. Rev. Lett. 106, 015303 (2011).
 (15) M. J. Bhaseen, S. Ejima, F. H. L. Essler, H. Fehske, M. Hohenadler, and B. D. Simons, Phys. Rev. A 85, 033636 (2012).
 (16) The restriction to a single band is justified as long as the atommolecule conversion term is significantly smaller than the band gap. This is verified in all our calculations.
 (17) T. Köhler, K. Góral, and P. S. Julienne, Rev. Mod. Phys. 78, 1311 (2006).
 (18) V. G. Rousseau and P. J. H. Denteneer, Phys. Rev. A 77, 013609 (2008); Phys. Rev. Lett. 102, 015301 (2009).
 (19) M. Eckholt and T. Roscilde, Phys. Rev. Lett. 105, 199603 (2010).
 (20) The FI phase exists in as well, but it has not been discussed in the previous literature.
 (21) V. G. Rousseau, Phys. Rev. E 77, 056705 (2008); Phys. Rev. E 78, 056707 (2008).
 (22) The FI appears for , namely on the molecular side of the (onsite) resonance. This is easily understood by looking at the case : in the system at hand the relevant states coupled by the terms are a BECa and a BECm, whose degeneracy is not reached for but for ( would be the resonance condition between atomic and molecular MI). The shift of the resonance to negative stems from the mismatch in kinetic energy between atoms and molecules (in the case at hand, even if the molecules are  at most  half the number the atoms); to stabilize a molecular state, the increase in kinetic energy has then to be compensated by a negative detuning.
 (23) Indeed, let us consider a ph excitation from a twosite state to a state (where the states are expressed in the basis). Here we consider the excitation transfering a molecule between two sites, as appropriate for the insulating phases at in the phase diagram of Fig. 2 – see SM Supplementary . The conversion energy increases from to , whence the ph gap. If one considers instead an atomic ph excitation, with excited state (as appropriate for the atomic side of the phase diagram in Fig. 3), the conversion energy increases from to .
 (24) See Supplementary Material, which includes Refs. WannierFunction ; Bakretal2009 ; Follingetal2005 , for: 1) details of the methods; 2) further characterization of the meanfield phase diagram (Fig. 2(a)); 3) a discussion of the gradual passage from Feshbach insulator(s) to collapse; 4) a discussion of the ab initio calculation of the Hamiltonian parameters; 5) the density profiles of Rb in a trap; 6) the role of the atommolecule conversion in controlling the particlehole gap; 7) the extraction of atommolecule coherence from momentumnoise correlations; 8) the control of the atommolecule conversion via a periodic modulation of the magnetic field.
 (25) W. Kohn, Phys. Rev. 115, 809 (1959); S. Kivelson, Phys. Rev. B 26, 4269 (1982).
 (26) W. S. Bakr, J. I. Gillen, A. Peng, S. Fölling, and M. Greiner, Nature (London) 462, 74 (2009).
 (27) S. Fölling, F. Gerbier, A. Widera, O. Mandel, T. Gericke, and I. Bloch, Nature (London) 434, 481 (2005).
 (28) Th. Busch, B.G. Englert, K. Rzazewski, and M. Wilkens, Found. Phys. 28, 549 (1998).
 (29) S. Inouye, M. R. Andrews, J. Stenger, H.J. Miesner, D. M. StamperKurn, and W. Ketterle, Nature (London) 392, 151 (1998).
 (30) J. Stenger, S. Inouye, M. R. Andrews, H.J. Miesner, D. M. StamperKurn, and W. Ketterle, Phys. Rev. Lett. 82, 2422 (1999).
 (31) We have considered several other narrow Feshbach resonances for various atomic species, namely: Rb (resonances at 9 G, and 1007 G), Na (907 G), Li (543 G) Chinetal2010 . Yet only the above cited ones turn out to be sufficiently narrow for the appearance of a stable FI phase with . See Ref. Supplementary for further discussion.
 (32) The recoil energy is defined as for particles of mass in a laser wave of wavelength ; we chose nm.
 (33) B. CapogrossoSansone, N. V. Prokof’ev, and B. V. Svistunov, Phys. Rev. B 75, 134302, (2007).
 (34) In Fig. 4 it is interesting to observe the nonmonotonic behavior of the atomic coherence as is reduced towards the resonance: the coherence peak is enhanced first – as expected from the reduction of the effective scattering length – but then it is suppressed, owing to the localizing effect of the resonant atommolecule conversion term. Moreover, due to the firstorder nature of the transition, an adiabatic sweep in should lead to hysteresis, namely to a historydependent form of the momentum distribution depending on whether the detuning is swept across the transition coming from the molecular or the atomic side.
 (35) The normalization of the momentum distribution is such that gives the number of particles in the infinitesimal volume of momentum space.
 (36) B. S. Rem et al., Phys. Rev. Lett. 110, 163202 (2013).
 (37) R. J. Fletcher, A. L. Gaunt, N. Navon, R. P. Smith, and Z. Hadzibabic, Phys. Rev. Lett. 111, 125303 (2013).
 (38) P. Makotyn, C. E. Klauss, D. L. Goldberger, E. A. Cornell, and D. S. Jin, Nature Phys. 10, 116 (2014).
 (39) J. Herbig, T. Kraemer, M. Mark, T. Weber, C. Chin, H.C. Nägerl, and R. Grimm, Science 301, 1510 (2003).
 (40) S. T. Thompson, E. Hodby, and C. E. Wieman, Phys. Rev. Lett. 95, 190404 (2005).
I Supplementary Material: FeshbachStabilized Insulator of Bosons in Optical Lattices
Ii 1. Methods used
In and 3 we supplement the QMC calculations with Gutzwiller meanfield theory (MFT), consisting in solving the problem of a single site coupled to the selfconsistent atomic and molecular mean fields, and , respectively. Moreover in , we make use of numerically exact quantum Monte Carlo (QMC) simulations based on the Stochastic Green Function (SGF) algorithm with directed updates SM_SGF , both in a canonical and grandcanonical setting. We treat lattices with sizes up to . An inverse temperature of (where ) allows to eliminate thermal effects from the QMC results.
When comparison between MFT and QMC is made, MFT is found to correctly capture the succession of phases in the system upon varying the Hamiltonian parameters, although the agreement between the two approaches is typically semiquantitative, as it will be further discussed below.
Iii 2. Additional information on the meanfield phase diagram
In this section, we provide additional elements concerning the meanfield phase diagram plot in Fig. 2(a) of the main text. The phase diagram is obtained by selfconsistent minimisation of the following singlesite Hamiltonian
(4)  
where and are evaluated on the ground state of ; is the coordination number.
Fig. 5 shows the modulus of the atomic and molecular coherences ( and ), exhibiting the presence of two distinct BoseEinstein condensate (BEC) phase: atommolecule BEC (BECam) and molecular BEC (BECm), separated by a continuous transition. Moreover two insulating phases are present: the vacuum phase and the Feshbach insulator (FI). Vertical cuts through this diagram, obtained by increasing for two different values of , are shown in Fig. 6. The cut at shows the continuous transitions between BECm and FI both for the atomic/molecular coherences as well as for the densities. On the other hand, the cut at shows that the BECamFI transition has a firstorder nature, with clear jumps in both coherences and densities.
The firstorder transition implies a coexistence between different phases. This can be clearly singled out at the meanfield level by setting and in the meanfield Hamiltonian Eq. (4) as parameters, and reconstructing the groundstate meanfield energy function (both and are assumed to be real without loss of generality). Fig. 7 shows the meanfield groundstate energy as is increased across the BECamFI transition at and . For (Fig. 7(a)) the energy landscape clearly shows three minima: two stables ones with and (corresponding to a BECam ground state), and a metastable one with and (corresponding to a metastable BECm). The three minima exhibit the strong asymmetry present between atoms and molecules due to the atommolecule conversion term: when the phase of the molecular field is fixed (to zero in this case), the meanfield energy remains symmetric under the transformation , manifesting a residual symmetry in the choice of the phase of the atomic field. As increases, the metastable BECm minimum is found to jump to the origin , acquiring the nature of a FI (e.g. Fig. 7(c) and (d)), and to become degenerate with the BECam minima for . The presence of multiple minima at the transition point is characteristic of a firstorder transition.
Iv 3. From Feshbach insulator to collapse for broad resonances
As mentioned in the main text, increasing the strength of the atommolecule coupling – controlled by the width of the Feshbach resonance – one expects to gradually destabilize the system towards collapse. In particular, a simple scaling argument predicts that the total density increases as when increases. In the case of the symmetric atommolecule mixture (, ) one can conduct a more refined analysis based on an approximate meanfield energy function, uniquely dependent upon the atomic and molecular densities and , and taking the form
(5)  
Here we have neglected the kinetic energy altogether  this is justified in the limit in which and are the largely dominant energy scales in the system. Moreover, in the conversion term we have assumed a phase difference of between the atom pairs and the molecules, minimizing the conversion energy. In the case , minimization of the above density function with respect to and leads to a rather simple result for the equilibrium value of the total number density
(6)  
exhibiting again the anticipated scaling. As shown in Fig. 8 the above result reproduces very well the full solution of the selfconsistent minimization of the meanfield energy in the case and .
Fig. 9 shows the meanfield phase diagram of the symmetric atommolecule mixture with at fixed and variable chemical potential and atommolecule coupling, reconstructed via the total coherent fraction and the total meanfield number density . The FI for is found to persist in the phase diagram only for moderate values of , beyond which it leaves space to an atomicmolecular condensate phase with increasing density. Interestingly, another incompressible insulating phase is found at (mass) density for the particular value of the detuning chosen here  suggesting that in fact a whole family of FIs of increasing density might be stabilized in the system  yet a full characterization of the phase diagram upon varying the detuning goes beyond our current scopes.
V 4. Hamiltonian parameters for a atommolecule resonant mixture in a lattice
The Hamiltonian parameters for Rb atoms and Rb Feshbach molecules in a cubic optical lattice, with wavevector and depth , are estimated making use of the numerically calculated band structure and maximally localized Wannier states SM_WannierFunction . The atom and molecule tunnelling matrix element and are directly extracted from the width of the lowest energy band. The molecules see an optical lattice of effective depth , due to the fact that the dipole matrix element, whose square enters , takes contribution from both atoms of the molecule. This leads to (by orders of magnitude), and not to , as often assumed in the literature.
The onsite atomatom, moleculemolecule and atommolecule interactions matrix element, respectively , and , are obtained as
(7)  
(8)  
(9) 
with being the background scattering length of the atoms, the atomic mass and the atomic (molecular) Wannier function.
The conversion rate between atoms and molecules is obtained via the solution of the scattering problem for two atoms in a parabolic potential SM_Buschetal1998 . Following SM_Syassenetal2007 , the parameter is given by
(10) 
with the difference of magnetic moments, the width of the Feshbach resonance, the lattice depth seen by an atom and the atomic recoil energy. Here is expressed in units of , namely the atomatom repulsion within the harmonic approximation for the Wannier function, . For the lattice depth we considered () overestimates by some 20%. Yet this choice of normalization for is justified by the fact that is also calculated by assuming a single harmonic potential, and hence can be expected to be similarly overestimated with respect to the actual value.
Making contact with the previous section, it is interesting to summarize here our estimates for the value of the ratio associated with some relevant narrow Feshbach resonances already investigated experimentally SM_Chinetal2010 . Considering optical lattices of depth varying between 4 and , we find

Rb, G: ;

Na, G: ;

Rb, G: ;

Li, G: .

Rb, G: ;

Na, G: ;
(Here indicates the position of the resonance in magnetic field). As already mentioned in the main text, apart from the first two resonances, we cannot find a stable, incompressible FI at density . This is consistent with the results of the previous section, which, albeit focusing on an artificially symmetric atommolecule mixture, were also suggesting that a FI is only expected for .
Vi 5. Density profiles for nearresonant Rb in a trap
Fig. 10 shows the density profile of Rb in a cubic optical lattice of depth plus a harmonic trap, and in a magnetic field close to its 414G Feshbach resonance. The chemical potential and detuning take values and , which set the trap center into a gapped FI phase with total density . The resulting density profile is obtained using standard localdensity approximation. Moving towards the trap wings, we observe a density jump in both the atomic and molecular components when the system goes locally from FI to BECam, and a further jump when going from BECam to vacuum. Such jumps are observed as well for square lattices, and they are therefore directly amenable to experimental observation using e.g quantum gas microscopes SM_Bakretal2009 .
Vii 6. Controlling the particlehole gap with the atommolecule conversion
In this section we discuss how the role of the atommolecule conversion rate in opening, stabilizing or suppressing the particlehole (ph) gap in the system can be fundamentally understood from the structure of the particle and hole excitations around a given filling, whose wave function can be extracted from MFT. This analysis allows to contrast the various cases of FI, Feshbachenhanced MI or Feshbachsuppressed MI encountered in the main text.
We first focus on the symmetric atommolecule mixture (Fig. 2 of the main text). In this case, we show that the atommolecule conversion may open a ph gap, driving a quantum phase transition from a BEC to a FI; or enhance the ph gap of a preexisting MI for sufficiently small . In both cases, for larger values of the atommolecule conversion has instead the opposite role of suppressing completely the ph gap. Then, we shall focus our attention on the case of Rb: for this system the distinction between Feshbach and (molecular) Mott insulator is particularly important as the two phases are continuously connected upon varying the detuning (Fig. 3(a) of the main text). The analysis of the nature of the particlehole gap provides further details concerning the crossover between the two regimes – beyond the sign change in the derivative of the ph gap with respect to , discussed in the main text (Fig. 3(b)).
vii.1 Symmetric atommolecule mixture
vii.1.1 Dependence of the particlehole gap on conversion rate
Focusing on the symmetric case ( and ), we discuss the ph gap evolution versus for a fixed detunning and for the two values of the hopping, and , discussed in the main text. As shown in Fig. 2 of the main text, at this value of the detuning for a Mott phase (of molecules, MIm) exists only for , whereas the system is in a molecular BEC phase (BECm) for .
The ph gap, , is given by the chemical potential width of the insulating phase at ,
(11) 
where () is the critical chemical potential to add a particle (hole) to the incompressible phase. Fig. 11 shows the evolution of when grows. In the case (Fig. 11(a)) one observes that a quantum phase transition is driven by the atommolecule conversion, which opens a gap for a critical value () giving rise to a FI. On the other hand, in the case (Fig. 11(b)), starting in the MIm phase at , the already existing gap at is enhanced by the atommolecule conversion up to , characterising a Feshbachstabilized MI of molecules and atoms (MIam).
For both cases, a larger value of has the opposite effect of suppressing the ph gap, leading to a further quantum phase transition to a molecular BEC (BECm) first, and to an atommolecule condensate (BECam) for a slightly larger value of . We observe that this nonlinear gap evolution is in a very good qualitative agreement with the QMC results Fig. 2(c,d) of the main text concerning the full width at half maximum of the momentum distribution. (On the basis of our QMC data we cannot definitely conclude about the presence or absence of a narrow BECm region for the same choice of parameters as in the MFT calculation; nonetheless QMC confirms the existence of a tricritical point at which the three transition lines FIBECm, FIBECam and BECmBECam meet).
vii.1.2 Wavefunction of the particle and hole excitations
To gain a detailed microscopic understanding of the role of in opening the ph gap of the FI, it is useful to analyze the nature of the particle and hole excitations which are admixed to the ground state at the critical chemical potentials and respectively. Within meanfield theory, the nature of the excitations can be directly read out of the singlesite groundstate wave function
(12) 
Indeed, slightly above the chemical potential, namely for , one has
(13) 
where is the ground state of the insulating phase, is the wave function of the particle excitation, and ; similarly for one has
(14) 
where again and is the wave function for the hole excitation.
In the presence of an atommolecule conversion, the insulating ground state with has the form
(15) 
where and depend on the conversion as shown in Figs. 12(a), 13(a).
When , the particle excitation
(16) 
is admixed with the insulating ground state, while for the hole excitation (or the vacuum) is admixed to the ground state, as shown in Figs. 12(bc), 13(bc). This result shows clearly how atommolecule conversion establishes a fundamental asymmetry between the nature of the particle excitation and that of the hole excitation: while the particle excitation is a coherent superposition between an “atompairlike” excitation ( and ) and a “moleculelike” excitation (), the hole excitation cannot have such a coherent nature. Due to the nonlinear dependence of the conversion energy on the particle density, the latter enters in the energy balance of particle hole excitations (where is the energy cost of the particle excitations and that of the hole excitation). The hole excitation makes the conversion energy vanish, and hence it has a conversion energy cost of starting from the ground state with . The particle excitation varies instead the conversion energy by a term . Hence the conversion energy contributes to the ph excitation energy with a term
(17)  
vii.1.3 Conversion energy contribution to the particlehole gap
Fig. 12(d) shows that, for , marks the BECmFI quantum phase transition, and for sufficiently small its magnitude is consistent with the gap in Fig. 11(a), namely the conversion energy contribution dominates the particlehole gap. Similarly, for Fig. 13(d) shows that is of the order of the gap increment in Fig. 11(b) for sufficiently small . For both cases, turns negative at larger , showing that the atommolecule conversion plays an opposite, gapsuppressing role, in agreement with the overall dependence of the gap.
What is the origin of this complex, dual role of the atommolecule conversion (gapopening or gapstabilizing for small , gapsuppressing for large )? It can be again read out of the structure of the particle and hole excitations. For small , is negligible, and : the atommolecule admixture in the ground state is more significant than in the particle excitation state. This latter aspect is generic, because the overall atommolecule detuning (combining the explicit detuning as well as the interaction and kinetic energy difference) is more significant at higher fillings, namely it is stronger in the particleexcitation state than in the ground state. Hence for small the atommolecule conversion is more efficient in the ground state than in the particleexcitation state, and therefore it drives the mechanism behind the gap opening/gap enhancement.
Nonetheless an increase in bridges this difference between insulating ground state and particle excitation. Indeed the atommolecule conversion is all the more effective the larger the filling (due to the bosonenhancement factor), and when one reaches the condition at a lower then that required for the condition (not shown in the figures). As a consequence, grows faster than ; moreover the appearance of a nonnegligible introduces a further reduction term in , and these two aspects together drive the ph towards zero. The lowering of the energy cost of the particle excitation comes from the fact that the atommolecule conversion favors clustering of atoms and molecules: this is again due to the boson enhancement factor, which introduces an explicit density dependence of the conversion energy. As a result, within MFT in the grandcanonical ensemble a large is found to drive a quantum phase transition from an insulating phase with to a BECm with .
vii.2 Rb close to its 414G Feshbach resonance
vii.2.1 Dependence of the particlehole gap on conversion rate and detuning
We now focus on the phase diagram of Rb, Fig. 3 of the main text. To understand how the atommolecule conversion acts in opening a ph gap on the atomic side of the resonance, it is instructive to monitor the evolution of the insulating phase in the phase diagram of Fig. 3(a) of the main text upon changing . As discussed in the main text and later in Sec. IX, a continuous tuning of between zero and its intrinsic value for the chosen Feshbach resonance is achievable in the experiment via periodic modulation of the atommolecule detuning, controlled by the magnetic field. Fig. 14(a) shows the evolution of the insulating lobe at when grows. The evolution with of the gap , Eq. (11), for different values of is further plotted in Fig. 14(b), while the dependence for different values of is shown in Fig. 15. One can clearly distinguish two separate regimes in the dependence of the gap: I) for sufficiently negative the gap decreases monotonically with , a behavior well understood in terms of the renormalisation effect that (virtual) atoms have on moleculemolecule interactions. At the lowest order in perturbation theory, the moleculemolecule interaction is decreased from by a term of order ; II) on the other hand, for the gap is monotonically increasing with , starting from zero at . This latter behavior stems from the nonperturbative role of the coherent atommolecule conversion, and we identify it as the distinctive feature of a FI. In particular, for the case of Rb ( = 1.23) one observes that the gap changes from being a decreasing function of to being an increasing function of for , and that this value of marks the separatrix between curves which vanish when and those who do not. Hence this value marks the crossover from Mott insulating to FI behavior, as indicated in the phase diagram of Fig. 3(a) of the main text.
vii.2.2 Wavefunction of the particle and hole excitations
Similarly to the symmetric atommolecule mixture, the nature of the particle and hole excitations give a microscopic understanding of the role of in opening the ph gap of the FI. In the insulating phases (MIam and FI), the insulating ground state has the form , where and depend on the detuning , and in particular for , as shown in Fig. 16(a). As shown in Fig. 16(bc), when the particle excitation
(18) 
is admixed with the insulating ground state, while for the hole excitation is admixed to the ground state, and in particular it is predominantly a single hole, , for , while it is predominantly a double hole (or the vacuum) for . Similarly to the symmetric case, the atommolecule conversion establishes a fundamental asymmetry between the nature of the particle excitation and that of the hole excitation: while the particle excitation is a coherent superposition between an ”atomlike” excitation () and a ”moleculelike” excitation (), the hole excitation does not possess such a coherent nature and, for instance, it is predominantly atomlike () for . The hole excitation makes the conversion energy has again a conversion energy cost of starting from the ground state with . The conversion energy variation due to the particle excitation is instead . As a result the conversion energy contribution to the ph excitation energy reads
(19)  
Fig 16(d) shows that this quantity is positive and almost constant all over the FI regime . Therefore it remains positive even on the BECam side – at least close to the crossover to FI – marking a regime of Feshbachenhanced MI. On the other hand, becomes negative sufficiently far from the resonance on the molecular side, marking instead the conventional regime of Feshbachsuppressed MI.
vii.2.3 Opening of the particlehole gap in the Feshbach insulator
The above data clearly explain the mechanism for the opening of a ph gap in the FI regime. When coming from the BECam side at , the conversion term introduces a positive energy cost in the ph excitation, Eq. (19), which depends on the atommolecule coherences and . The latter are controlled by the detuning , and become maximal around resonance. As the prefactor of in Eq. (19) is upperbounded, it is necessary that be sufficiently large (namely, the Feshbach resonance be sufficiently broad) for the conversion energy cost to outgrow the kineticenergy gain of a ph excitation and to open a gap. Figs. 14 and 15 show indeed that for , namely on the atomic side of the resonance, needs to exceed a dependent critical value for a ph gap to open. is actually nonzero for any . This means that a FI phase can be realized by Rb at the 414 G resonance because the width of the resonance not only is sufficiently narrow to avoid the collapse of the system, but it is also sufficiently large, namely . The interparticle repulsion, which drives the opening of a ph gap for , has mostly a “spectator” role for the FI, stabilizing the density of the system around the value , and hence preventing the system from collapsing.
Viii 7. Atommolecule coherence from momentumnoise correlations
The FI is characterized by a strong atommolecule coherence . Can one measure the coherence in an experiment?
A possibility comes from the analysis of the correlations between the noise of atomic and molecular momentum distributions. The momentum distribution of atoms and molecules can be measured simultaneously using SternGerlach separation during time of flight SM_Herbigetal2003 . This leads us to consider the correlation function
(20) 
where and are the momentum distribution of atoms and molecules respectively.
We assume for simplicity that the state of the system factorises between sites, namely
(21) 
where the state is expressed in the basis.
Then we readily obtain that
(22) 
where , are the average atomic and molecular densities, is a reciprocal lattice vector, and .
Introducing then the function
(23) 
we find that, for the factorized state of Eq. (21)
(24) 
namely this function exhibits a series of bunching peaks describing the reciprocal lattice, and whose height is proportional to the square of the atommolecule coherence. Hence the noise correlations in the fluctuations of the atomic and molecular momentum distributions reveal the presence of atommolecule coherence. Yet an important limitation of this approach is that the bunching peaks are suppressed like . This means that the bunching signal can only be seen on relatively small samples – as it is a factor of weaker than the typical bunching signal of a Mott insulator in a very deep optical lattice, as detected e.g. in SM_Follingetal2005 .
Ix 8. Control of the atommolecule conversion rate via modulation of the magnetic field
In this section we show how to continuously reduce the value of the atommolecule conversion from its intrinsic value associated with the chosen Feshbach resonance down to zero, making use of the periodic modulation of the magnetic field, which controls the atommolecule detuning .
We start our discussion by considering a simple singlesite problem, in which the atommolecule detuning is driven periodically at a frequency :
(25) 
The stand for terms diagonal in the atomic and molecular densities, which will not be affected by the subsequent manipulations.
Consider then the solution to the timedependent Schrödinger equation governed by the Hamiltonian . Introducing the unitary operator
(26) 
the state evolves according to the effective Hamiltonian which takes the form
(27) 
where
Assuming that is not resonant with any transition in the system, and in particular , the oscillating part of the coupling constant has little effect on the dynamics, so that it is justified to approximate with its time average:
where is the zeroth Bessel function of the first kind. This function goes from 1 to 0 when its argument goes from 0 to . Therefore a weak periodic modulation with amplitude going from to zero to a value allows to continuously reduce the strength of the atommolecule coupling from its bare value down to zero. Notice that past experiments investigating the periodic modulation of the magnetic field SM_Thompsonetal2005 have rather explored the resonant regime to drive the association of molecules away from the resonance, but this implies that the regime of weak driving at low frequency is equally accessible.
The previous discussion generalizes immediately to the full Hamiltonian of the problem, Eq.(13) of the main text. The unitary transformation generalizes to
(28) 
and it commutes with all the terms of Hamiltonian (conserving the number of molecules) except the atommolecule conversion one. Hence the continuous control on the coupling is generally possible in the full manybody setting.
References
 (1) V. G. Rousseau, Phys. Rev. E 77, 056705 (2008); V. G. Rousseau, Phys. Rev. E 78, 056707 (2008).
 (2) W. Kohn, Phys. Rev. 115, 809 (1959); S. Kivelson, Phys. Rev. B 26, 4269 (1982).
 (3) Th. Busch, B.G. Englert, K. Rzazewski, and M. Wilkens, Found. Phys. 28, 549 (1998).
 (4) N. Syassen, D. M. Bauer, M. Lettner, D. Dietze, T. Volz, S. Dürr, and G. Rempe, Phys. Rev. Lett. 99, 033201 (2007).
 (5) C. Chin, R. Grimm, P. Julienne, and E. Tiesinga, Rev. Mod. Phys. 82, 1225 (2010).
 (6) W. S. Bakr, J. I. Gillen, A. Peng, S. Fölling, and M. Greiner, Nature 462, 74 (2009).
 (7) J. Herbig, T. Kraemer, M. Mark, T. Weber, C. Chin, H.C. Nägerl, and R. Grimm, Science 301, 1510 (2003).
 (8) S. Fölling, F. Gerbier, A. Widera, O. Mandel, T. Gericke and I. Bloch, Nature 434, 491 (2005).
 (9) S. T. Thompson, E. Hodby, and C. E. Wieman, Phys. Rev. Lett. 95, 190404 (2005).