# Three-dimensional light-matter interface for collective spin squeezing in atomic ensembles

## Abstract

We study the three-dimensional nature of the quantum interface between an ensemble of cold, trapped atomic spins and a paraxial laser beam, coupled through a dispersive interaction. To achieve strong entanglement between the collective atomic spin and the photons, one must match the spatial mode of the collective radiation of the ensemble with the mode of the laser beam while minimizing the effects of decoherence due to optical pumping. For ensembles coupling to a probe field that varies over the extent of the cloud, the set of atoms that indistinguishably radiates into a desired mode of the field defines an inhomogeneous spin wave. Strong coupling of a spin wave to the probe mode is not characterized by a single parameter, the optical density, but by a collection of different effective atom numbers that characterize the coherence and decoherence of the system. To model the dynamics of the system, we develop a full stochastic master equation, including coherent collective scattering into paraxial modes, decoherence by local inhomogeneous diffuse scattering, and backaction due to continuous measurement of the light entangled with the spin waves. This formalism is used to study the squeezing of a spin wave via continuous quantum nondemolition (QND) measurement. We find that the greatest squeezing occurs in parameter regimes where spatial inhomogeneities are significant, far from the limit in which the interface is well approximated by a one-dimensional, homogeneous model.

###### pacs:

## I Introduction

Cold atomic ensembles interacting with electromagnetic fields are powerful tools in quantum information science with applications that include quantum memory Fleischhauer and Lukin (2002); Julsgaard *et al.* (2004); Choi *et al.* (2008), quantum communication Duan *et al.* (2001); Matsukevich and Kuzmich (2004), continuous variable quantum computing Braunstein and van
Loock (2005), and metrology Appel *et al.* (2009); Leroux *et al.* (2010). At the heart of these protocols is the strong coupling between a quantum mode of the field and an effective collective spin of the ensemble. This coupling can generate entanglement between atoms and photons, such that measurement of the light yields strong quantum backaction on the atoms. Photons can also enable a quantum data bus for entangling atoms with one another. Enhancing the atom-light interface is thus essential for improving the performance of quantum technologies and for reaching new regimes where a quantum advantage becomes manifest. This can be achieved through confined modes such as in optical cavities Miller *et al.* (2005); Leroux *et al.* (2010); Chen *et al.* (2011) or waveguides in optical nanostructures Vetsch *et al.* (2010); Bose *et al.* (2012); Hung *et al.* (2013).

Strong atom-photon coupling can also occur in free space in the interaction between light and an extended ensemble of atoms. This occurs when photons are scattered collectively by the ensemble, and interference enhances the radiation into the probe mode relative to diffuse scattering into steradians Tanji-Suzuki *et al.* (2011); Bienaimé *et al.* (2013). Early experiments demonstrated such strong coupling and entanglement in high pressure vapor cells where a one-dimensional description of plane wave modes and uniform atomic density is applicable Kuzmich *et al.* (1999). This theory accurately describes a variety of experiments including the entanglement of macroscopic ensembles in remote vapor cells Julsgaard *et al.* (2001) and quantum memory for continuous variables Julsgaard *et al.* (2004).

More recently, experiments have employed ensembles of ultracold atoms in pencil-shaped dipole traps probed by focused laser beams Kaminski *et al.* (2012); Koschorreck *et al.* (2010a). When the radiation pattern of the atomic ensemble is effectively matched with the paraxial mode of the probe, the atomic dipoles are indistinguishable and the scattering is cooperative. Such geometries have the potential to strongly enhance the atom-photon quantum interface, but their description is more complex, requiring a full treatment of diffraction, inhomogeneous coupling, radiation patterns, and decoherence. Harnessing the advantages of these atomic ensembles thus requires a three-dimensional quantum theory of the underlying interaction, including both coherent coupling and quantum noise.

In the last decade there has been significant progress in developing a three-dimensional quantum description of the atom-light interface. A rigorous field-theoretic treatment separates the mean-field classical effects from the quantum fluctuations and noise, including the spatial inhomogeneities of atoms and light modes Sørensen and Sørensen (2008). Models that include spatial modes have been developed in a variety of contexts Kuzmich and Kennedy (2004); Windpassinger *et al.* (2008); Koschorreck and Mitchell (2009); Sau *et al.* (2010). Applications include remote entanglement via collective Raman scattering in a DLCZ-type protocol Duan *et al.* (2002); Sørensen and Sørensen (2009) and for multimode quantum memories Zeuthen *et al.* (2011). From such studies, it is clear that one-dimensional models not only fail to describe the relevant coherent and incoherent effects, but they also do not take advantage of the resources associated with spatial modes Grodecka-Grad *et al.* (2012); Higginbottom *et al.* (2012).

In this paper we revisit the three-dimensional atom-light interface with particular emphasis on spin squeezing through QND measurement of the collective spin via the Faraday effect Kuzmich *et al.* (2000); Koschorreck *et al.* (2010b); Takano *et al.* (2009), shown schematically in Fig. 1. In this protocol, the key interaction is the off-resonant scattering of horizontally polarized photons into vertical polarization. Measurement in a balanced polarimeter corresponds to a homodyne measurement of these scattered photons. The degree of scattering into the local oscillator, defined by the paraxial laser mode, determines the measurement strength and the resulting backaction that generates squeezing.

Central to this problem are the spatial modes of the light and the collective spin waves of the atomic ensemble. In the one-dimensional model, one collective parameter defines the strong-coupling regime of the atom-photon interface, the optical density on resonance, OD , where is the atomic density, is the resonant scattering cross section, is the length of the vapor cell, and is the number of atoms in the volume for a uniform beam of area . In contrast, in a fully three-dimensional model, where the atomic density, , and paraxial beam intensity distribution, , are not uniform, there is a collection of parameters that dictate the strong-coupling regime. Different effective atom numbers, , govern different physical effects. For example, determines the mean Faraday signal in the polarimeter, while determines the size of the measurement uncertainty from spin projection noise.

The entangling strength of the atom-light interface is determined by the size of the spin projection uncertainty compared to the quantum uncertainty in the measured light quadratures (shot noise). This collective interaction is proportional to an effective optical density, . In contrast, decoherence acts locally on the atoms in a noncollective manner, and the noise injected into the system due to optical pumping and spin flips is governed by other parameters. A proper accounting of the balance between the coherent coupling and decoherence is especially challenging given the tensor nature of the atom-photon interaction of real alkali atoms. Previous treatments of quantum noise in a multimode Faraday-based atom-light interface have been carried out in a one-dimensional model Kupriyanov *et al.* (2005); Vasilyev *et al.* (2012). Our goal is to extend this to the three-dimensional case.

In this work, we derive a stochastic master equation describing the dynamics of the collective atomic state conditioned on balanced polarimetry measurements, including the effects of measurement backaction, collective decoherence from unmeasured paraxial light, and local decoherence from diffuse photon scattering that gives rise to optical pumping. While we apply this to study conditional spin squeezing generated by a QND measurement, the formalism we develop is broadly applicable to other protocols where a strong, free-space atom-light interface is essential, and where measurement backaction may be a tool for induced atom-atom interactions.

The remainder of the article is organized as follows. We lay out the physical model for an ensemble of alkali atoms dispersively interacting with a coherent probe laser in Sec. II. We begin with a semiclassical model that can be used to describe the scattered paraxial fields and to identify the collective spin wave that is coupled to the laser mode. To understand the entangling Faraday interaction in a multimode geometry, we then present a fully quantum mechanical model. This serves as the cornerstone for a complete description of QND squeezing and allows us to account for the damaging effects of decoherence. When the output light is measured continuously, the quantum dynamics, including the combined effects of measurement backaction and decoherence, are described by a stochastic master equation. We use this fully quantum mechanical atom-light description to study effects of spatial modes on the squeezing of spin waves in Sec. III. In particular we use the multimode description to model the dynamics of spin squeezing and to analyze the dependence of peak squeezing on cloud and beam geometry. We use numerical simulations to help build physical intuition about the three-dimensional atom-light interface and to investigate how the model can be used to optimize an experimental design. Finally, we summarize our results and present future directions for this work in Sec. IV.

## Ii Paraxial Atom-Light Interface

When driven by an off-resonant laser field such that the excited state probability is small, atoms elastically scatter electromagnetic waves in a manner equivalent to a set of linearly polarizable particles. Thus, a great deal of qualitative and quantitative information can be obtained from classical radiation theory. In a rigorous field-theoretic analysis, Srensen and Srensen showed that the mean-field effect of the light interacting with an atomic ensemble gives rise to an index of refraction of the gas, while fluctuations are due to the random positions of the atoms and the vacuum noise of the light Sørensen and Sørensen (2008). In particular, the index of refraction is due to the spatially-averaged local density of the atoms, while the diffuse scattering into 4 arises from the random positions of the point atomic scatterers and is equivalent to decoherence by local spontaneous emission. This diffuse scattering, which leads to attenuation of the incident wave and optical pumping of the atomic state, is accounted for by an imaginary part of the polarizability according to the optical theorem.

We can thus break up the problem into two pieces. First, the mean field effect is described by classical scattering of a laser beam incident on a linearly polarizable dielectric whose shape is determined by the atomic density distribution. For a paraxial probe beam and an extended cloud, the scattered field is also paraxial, and the solution is easily found by Fraunhofer scattering theory Newton (1982). As we are interested in the Faraday effect, we include the tensor nature of the atomic polarizability. Scattering of an incident horizontal polarization to an orthogonal vertical polarization is the key effect that we seek to measure in the polarimeter. Second, to properly account for quantum backaction on the atoms resulting from measurement and to describe the decoherence due to diffuse scattering and optical pumping, we turn to the fully quantum theory.

### ii.1 Semiclassical theory

Consider the scattering of an incident paraxial laser beam with frequency and complex amplitude, , by a particle located at a position with dynamical tensor electric-dipole polarizability The field envelope has the standard form , where is the laser polarization and is chosen to be the Gaussian TEM mode given by

(1) |

The -dependent beam waist, the radius of curvature of the phase fronts, and the Guoy phase are given by

(2a) | ||||

(2b) | ||||

(2c) |

respectively, with beam waist and Rayleigh range . In the first Born approximation, the scattered field amplitude is that radiated by the induced dipole,

(3) |

where the subscript denotes the component of the dipole transverse to the direction of observation. The last approximation is valid for paraxial points of observation, . Gaussian-cgs units for the electromagnetic field equations are used throughout.

Because the dipole radiation is not mode-matched with the Gaussian laser beam, the light is scattered into all paraxial modes as well as off-axis nonparaxial modes. In the far field, , the total field takes the form,

(4) |

where is the scattered field into all spatial modes other than the probe mode, and as shown in Appendix A, Eq. (82),

(5) |

is the field amplitude “forward scattered” into the laser mode. is the effective beam area.

The key physical effects are seen in these equations. The component of the radiated field vector along the laser polarization gives rise to the scalar index of refraction and attenuation. The component of orthogonal to gives rise to a rotation of the polarization on the Poincaré sphere – Faraday rotation and birefringence. For example, suppose the laser is linearly polarized along (. The total field thus can be written,

(6) | |||

where,

are respectively: is the index of refraction phase shift, is the Beer’s law attenuation coefficient, is the rotation angle of the Stokes vector corresponding to the Faraday effect, and is the corresponding angle for birefringence, with the polarizability matrix elements denoted as in the - basis.

The above description of the atom-field coupling is most easily generalized using the theory of scattering of paraxial waves Müller *et al.* (2005); details can be found in Appendix A. The mean field is described by the electric field envelope , where is the temporal pulse envelope evaluated at the retarded time, and is the spatial envelope satisfying the paraxial wave equation,

(7) |

with spatially averaged dielectric susceptibility . The scattering solution to this equation is well known Newton (1982). In the first Born approximation, i.e. for dilute samples where multiple scattering is negligible, given an incident field , the total field is

(8) | |||

where is the paraxial propagator. This solution is the superposition of incident and reradiated dipole fields. The solution for a paraxial field scattered from a point dipole at position , Eq. (II.1), is recovered by setting .

The diagonal matrix elements of the susceptibility give rise to the index of refraction and a slight distortion of the wavefront of the beam. We can neglect this effect for dilute gases, though it is easily accounted for. The Faraday effect arises from the scattering of initially -polarized light into orthogonal -polarization as discussed above, governed by the off-diagonal element of the dielectric susceptibility matrix, . To measure Faraday rotation, one employs a balanced polarimeter at , so that the signal is proportional to , integrated across the detector surface at position in the far field,

(9) |

The result is an effective homodyne detector for , with playing the role of the local oscillator. Using the solution for , Eq. (8), and the properties of the propagator, Eq. (78),

(10) | |||||

The measured signal is thus proportional to the local value of the susceptibility component integrated over the dielectric, weighted by the local field intensity .

For an ensemble of dilute cold atoms at fixed positions , the dielectric susceptibility of the gas is

(11) |

where is the the dynamic polarizability tensor operator for the atom. We consider here atoms restricted to a subspace defined by a total (hyperfine) angular momentum . In terms of the hyperfine spin operator , the polarizability operator can be decomposed into irreducible tensor components Deutsch and Jessen (2009),

(12) |

where is the characteristic polarizability and is the coefficient of the irreducible rank- tensor component. The rank-0 component is a scalar, which does not influence spin and polarization dynamics. The vector (rank-1) component is responsible for the Faraday effect, while the tensor (rank-2) component induces birefringence. For alkali atoms driven on a fine-structure multiplet, and the coefficients are given in Deutsch and Jessen (2009).

The effect of the tensor component complicates both the collective coupling of the atoms to the probe as well as the internal spin dynamics. In special cases, the deleterious effects of the rank-2 component of the tensor polarizability can be removed via dynamical decoupling Koschorreck *et al.* (2010c). More generally, a large bias field removes the rank-2 component of the interaction that couples the collective spin to the polarization of the probe Norris *et al.* (2012), leaving only internal spin dynamics that can be compensated.
We thus retain only the vector component of the off-diagonal element of the dielectric susceptibility, , which describes a pure Faraday interaction. Substituting into Eq. (10) yields

(13) |

Equation (13) is the central result of the semiclassical model. In a plane wave, homogeneous, one-dimensional description, the measured observable is , the symmetric collective spin of the ensemble. For paraxial beams, the polarimeter measures an effective spin wave determined by the inhomogeneous weighting of the atomic spin operators by the local intensity of the beam. The spin wave is stationary because it is coupled to the forward-scattered light, where the absorbed and emitted modes are the same. Physically, it is this collective observable that radiates indistinguishably into the probe mode and is effectively selected by the homodyne measurement of the polarimeter.

Further intuition can be gained from the semiclassical model. We recover symmetric atom-light coupling when the field intensity is constant over the atomic ensemble. Geometrically, this is achieved when the beam waist, , is much larger than the transverse extent of the cloud and the length of the cloud is short compared to twice the Rayleigh range, . The mean-field radiation pattern of such a cloud described by Eq. (8), however, has poor overlap with the probe as depicted in Fig. 2(a). The end result is that the polarimeter detects only a small fraction of the signal photons. On the other hand, perfect mode matching is achieved for atoms confined as a uniform dielectric sheet at a fixed -plane as seen in Fig 2(b). However, for a finite number of atoms, the realizable OD is low in this configuration. Indeed, a uniform dielectric slab of extent much larger than the beam waist achieves perfect mode matching, but one cannot achieve such an dielectric distribution with high OD using cold atomic gases. An intermediate “pencil”-shaped geometry is more realistic, allowing for reasonable mode matching while maintaining a high OD, as in Figs. 2(c-d).

In addition to maximizing the signal, we must minimize the sources of noise. There are two fundamental effects: (i) the polarimeter has a finite shot noise resolution; (ii) atoms scatter photons diffusely into all directions (spontaneous emission). The latter is accompanied by optical pumping that can both depolarize the spins and inject noise into the measured spin wave. In order to address these effects, we must turn to the fully quantum theory.

### ii.2 Quantum theory

Following Refs. Hammerer (2006); Vasilyev *et al.* (2012), we partition the quantized electric field into paraxial modes and nonparaxial, diffuse modes,

(14) |

This decomposition is motivated by the geometry we consider – photon scattering of a paraxial laser beam by an extended atomic ensemble. The mean-field, spatially averaged atomic density, which plays the role of the index of refraction in the classical theory, appears as coherent radiation by a collective atomic observable in the quantum theory. The coupling of collective atomic observables to paraxial modes thus describes the coherent atom-photon light-shift interaction, mediated by the Hermitian part of the atomic polarizability operator.

The diffuse modes, in contrast, couple to the density fluctuations in the ensemble due to the discrete atomic positions and thus act locally on each atom Sørensen and Sørensen (2008). In the usual Born-Markov approximation, tracing over these modes leads to decoherence and is described by the anti-Hermitian part of the atomic polarizability Deutsch and Jessen (2009).

In this section we first derive a multimode generalization of the Faraday interaction that coherently entangles the atomic ensemble and the paraxial quantum field. Then, we employ a master equation to account for the effects of local decoherence (optical pumping) driven by diffuse scattering. Finally, we present the stochastic master equation describing the conditional collective atomic state given polarimetry measurements of the paraxial field, which we use to analyze spin squeezing in Sec. III.

#### Paraxial multimode Faraday interaction

Quantization of paraxial electromagnetic fields was discussed in Deutsch and Garrison (1991); relevant extensions to the current problem are summarized in Appendix B. We decompose the paraxial field operator into an orthogonal set of tranverse spatial modes, here the Laguerre-Gauss modes , given in Eq. (90), which are convenient for cylindrical symmetry. The positive-frequency component of the electric field restricted to the paraxial subspace is,

(15) |

where the quantization area is chosen as the natural scale of the Gaussian beam, . The traveling wave creation/annihilation operators for each transverse mode freely propagate according the Hamiltonian

(16) |

with solution, and free-field commutation relations,

(17) |

We have normalized so that is the local photon flux in transverse mode with polarization .

For weak excitation (linear atomic response), the interaction Hamiltonian governing the coupling of the quantized paraxial modes is

(18) |

As before, the index is summed over atoms in the ensemble at respective positions . Upon substituting the decomposition of into its irreducible components given in Eq. (12), we find scalar (rank-0), vector (rank-1), and tensor (rank-2) contributions to the interaction. We retain only the vector contribution that leads to the Faraday effect, as the scalar contribution does not entangle photons with the atoms and the tensor contribution can in principle be removed Koschorreck *et al.* (2010a). The Faraday interaction is then,

(19) |

where

(20) |

is the Faraday rotation angle, is the resonant cross-section for unit oscillator strength, is the atomic linewidth, and is the detuning from resonance. For an transition with much larger than the excited state hyperfine splitting, , where is the Landé g-factor. We can interpret Eq. (II.2.1) as a scattering process, whereby an -polarized photon in a given transverse mode, , is absorbed and a -polarized photon in the mode is emitted, and vice versa, as mediated by the collective atomic spin wave.

Here, we consider an initial macroscopic occupation in the laser probe, again taken to be the fundamental Gaussian TEM mode with -polarization. In that case the interaction can be linearized by substituting , where is the photon flux of the laser with peak intensity . The quantum fluctuations in the field of interest are then represented by the -polarized mode, , and the Faraday interaction then takes the form

(21) |

where the local amplitude for scattering from the fundamental (laser) mode into mode is given by

(22) |

The interaction has been written in terms of the “measurement strength” per atom,

(23) |

which characterizes the rate at which photons are scattered into the paraxial modes, where is the unit-oscillator-strength photon scattering rate at the peak intensity.

The Heisenberg equation of motion for a -polarized traveling wave mode interacting with the atomic media in the presence of the probe field is

(24) |

whose solution is

(25) | ||||

where is the Heaviside step function. Neglecting the time it takes light to propagate across the sample, the mode amplitude at the detector plane, , in the far field is

(26) |

a form familiar from input-output theory Gardiner and Zoller (2004). The collective atomic spin wave that couples to this paraxial mode is

(27) |

In the balanced polarimeter, the probe mode acts as a local oscillator so that one measures the Stokes vector component associated with the fundamental spatial mode defined by the laser beam, , where is the mode quadrature. The measured quadrature at the detector plane, , is thus

(28) |

Thus, the total polarimeter signal, integrated over a time , is determined by the output operator

(29) |

where is the fundamental spin wave found in the semiclassical calculation, Eq. (13). The fully quantum theory explicitly includes the additional vacuum noise entering the polarimeter, , that leads to a shot-noise (SN) variance of the polarimeter signal, in Eq. (29), .

Of particular interest here is the application to spin squeezing via QND measurement. In this case, the signal we seek to measure arises from different spin-projections associated with the eigenstates of . Whereas in magnetometry these shot-to-shot variations are known as “projection noise” (PN), in the context of creating a spin squeezed state, these variations from the mean value represent the “signal” one seeks to resolve over the laser shot noise. For the fundamental spin wave measured in the polarimeter, the projection noise variance is

(30) | ||||

Given an initial spin coherent state polarized orthogonal to , , and thus,

(31) |

Here we define a set of effective atom numbers

(32) | ||||

(33) |

where the sum becomes an integral in the continuum limit. The atomic density distribution , is normalized so that , the total atom number. The effective atom number determines the projection noise contribution to Eq. (29), .

The coupling strength that sets the degree of entanglement one can attain between the atoms and photons is the ratio of the projection noise variance to the shot noise resolution Deutsch and Jessen (2009). Using Eqs. (23) and (31) we find

(34) |

where we have defined the effective optical density for the laser mode probing the spin wave on a unit-oscillator-strength transition,

(35) |

The key to achieving a large OD is choosing an atomic and beam geometry that addresses a large number of atoms and maximizes while keeping the mode area small. It should be noted that whereas in the one-dimensional case the optical density is associated with both the coupling strength and the Beer’s law attenuation of the probe, in the three-dimensional case different parameters are associated with each of these effects. Because the attenuation coefficient in Eq. (6) is proportional to the local intensity of the field, the total attenuation depends upon the effective atom number .

While Eq. (34) implies an ever increasing coupling strength with integration time , we have neglected so far the decoherence that limits the total useful integration time and the strength of the atom-light interface. In the following section we treat these effects from a first-principles master equation, including spatial variations in the scattering rate which drives local decoherence.

#### Local decoherence and optical pumping

The discrete random atomic positions are associated with the density fluctuations that give rise to diffuse scattering into 4 steradians Sørensen and Sørensen (2008). We consider light far detuned from any atomic resonance in a highly transparent regime, and thus we can safely neglect the small attenuation of the laser probe associated with this absorption. The scattering processes, however, cause decoherence of the spin wave due to optical pumping. This local decoherence breaks the collective symmetry of the problem and adds additional noise, which is detected in the polarimeter and competes with squeezing.

To treat the decoherence due to diffuse scattering, we employ a master equation,

(36) |

where is the multimode Faraday interaction given in Eq. (21). The key feature of this equation is that the paraxial modes couple to collective spin waves, while the diffuse scattering couples to localized atoms and induces optical pumping according to

(37) |

The map acts on the atom, proportional to the local scattering rate,

(38) |

Here is the local intensity at the position of the atom and is the peak scattering rate. We consider here a probe driving an transition in an alkali atom, with a detuning that is small compared to the ground state hyperfine splitting but large compared to any hyperfine splitting in the excited state. In this case, the light coherently couples substantially only to atoms in a given ground-electronic hyperfine manifold and the master equation is restricted to this subspace. As shown in Appendix C, with an -polarized probe and applying a large bias magnetic field along the -direction, the local decoherence in the master equation due to optical pumping is given by the map

(39) |

The first term on the right-hand side of Eq. (39) describes the decay of correlations due to optical pumping, while the second term represents a feeding due to “transfer of coherences” that can reduce this decay rate Cohen-Tannoudji (1977). Note that for , this master equation is not trace preserving, since atoms can be optically pumped to the other ground hyperfine manifold where they are lost to any further measurement.

Given the master equation, we can find the effect of diffuse scattering on atomic correlations. Consider a inhomogeneous collective operator of the form . Because is a weighted sum over single atom operators, the equation of motion for its expectation value depends upon the evolution of the single atom density operator, . By summing over a single index in Eq. (37) we obtain

(40) |

from which the evolution of is given by

(41) |

For inhomogeneous collective operators that depend on pairs of atoms,

(42) |

we require the joint density operator of the and atoms, , with equation of motion

(43) |

The evolution of due to diffuse scattering is then

(44) |

The degree of squeezing that one can ultimately produce is determined by a balance between QND measurement backaction on the spin wave mediated by the collective radiation and the damage to that observable caused by diffuse scattering. To properly treat this we must include the effects of measurement on the atoms, as discussed in the next section.

#### The conditional stochastic master equation

The Faraday Hamiltonian, Eq. (21), is an entangling interaction between the atomic spin waves and the paraxial modes of the field. When the light is measured in the polarimeter, quantum backaction leads to stochastic evolution of the atomic state, conditioned on the measurement result. A complete description of the dynamics is then described by a stochastic master equation (SME), with decoherence from unmeasured light and squeezing due to information gained from the continuous measurement record. In a balanced polarimeter, the measurement signal is proportional to the interference of the probe and scattered fields integrated over the detector faces, as in Eq. (9). Due to the orthogonality of the spatial modes, Eq. (91), such a measurement selects only paraxial light that is scattered into the mode of the probe, . The result is a continuous measurement of the quadrature .

We derive the SME for the atoms following Jacobs and Steck (2006); Wiseman and Milburn (2010), with details presented in Appendix D. Measurement of the quadrature by the homodyne polarimeter generates a differential stochastic measurement record

(45) |

where is a Weiner interval in the Itō calculus and is the measurement strength given in Eq. (23). Assuming unit measurement efficiency, the evolution of the ensemble conditioned upon the measurement record is given by

(46) |

The effects of measurement backaction on the ensemble are taken into account by the superoperator , where

(47) |

The Lindblad dissipator,

(48) |

describes the effect on the atomic ensemble arising from collective radiation into all paraxial modes of the field.

Including local decoherence from diffuse scattering, Eq. (39), the full stochastic master equation for homodyne polarimetry measurements of the -mode is

(49) | ||||

This SME is a complete description of the evolution of the collective atomic state, accounting for the three-dimensional nature of the atom-photon modes, decoherence, and measurement backaction. We see that through its interaction with the probe, the atomic ensemble undergoes an additional form of *collective* decoherence, Eq. (48), corresponding to light radiated into paraxial modes that ultimately goes unmeasured. Thus we have arrived at the same conclusion as in Ref. Duan *et al.* (2002). That is, decoherence arises through two distinct processes - first, the inherent mode-mismatch that gives rise to collectively scattered light in spatial modes other than the probe mode and second, the diffuse scattering of photons that acts locally on atoms in the ensemble.

## Iii QND squeezing of spin waves

### iii.1 Quantifying squeezing of the spin waves

One typically quantifies the amount of squeezing created in a QND measurement according to the Wineland squeezing parameter Wineland *et al.* (1992),

(50) |

where is the projection-noise limited resolution when measuring an angle of rotation for a generic spin of the given input state, and is the corresponding resolution when the input is a spin coherent state (SCS). For a mean value , and variance orthogonal to the mean, the projection-noise resolution is . With , the Wineland squeezing parameter is then

(51) |

For the spin waves of the inhomogeneous ensemble under consideration here, we must tie the squeezing parameter directly to the measured quantities. For an initial mean spin polarization along and a small rotation around the polarimeter signal will be determined by the mean spin wave component , and the projection noise contribution to the resolution of the measurement will be given by , defined in Eq. (30). The projection-noise limited resolution of this rotation is therefore . Furthermore, given a SCS initially polarized along x, the mean spin of interest is where the effective atom number contributing to this signal is given in Eq. (32), while the projection noise in the spin coherent state is . The projection noise limited resolution for a SCS preparation is thus, , and will depend on the shape of the atomic cloud and beam geometry. Putting this together, we define the squeezing parameter for the measured spin wave to be

(52) |

This parameter quantifies the degree of “quantum backaction,” on a spin coherent state, accounting for the change in projection noise due to QND measurement as well as the damage done to both the mean spin polarization and variance due to optical pumping.

In a real-world metrological application such as an optically probed atomic magnetometer Budker and Romalis (2007); Sewell *et al.* (2012), spin rotations are measured by passing the probe through the atom sample and measuring the resulting Faraday rotation in a polarimeter. In addition to spin projection noise, the measurement resolution is then subject also to “technical noise,” including probe shot noise, detector electronic noise, and atom number fluctuations. Under those circumstances, optimizing the squeezing parameter as defined in Eq. (52) is distinct from optimizing the magnetometer sensitivity.

### iii.2 The dynamical evolution of squeezing

To determine the squeezing as function of time, we employ the SME in Eq. (49) to track and . For ensembles with large numbers of atoms, we can work in the central-limit approximation where fluctuations in the spin waves are treated as Gaussian random variables Hammerer *et al.* (2010); Vasilyev *et al.* (2012). Following Jacobs and Steck (2006), the SME then couples solely means and covariances. The moments of the fundamental spin wave that characterize the spin squeezing parameter then evolve according to

(53a) | ||||

(53b) | ||||

Because we assume the fundamental mode is measured with unit efficiency, diffuse scattering by local spontaneous emission is the only process contributing to the decoherence of the variance . Collective radiation into other transverse modes commutes with and does not contribute to any decay or noise injection into the fundamental variance. In contrast, the mean spin decoheres due to both diffuse scattering and collective scattering into other unmeasured paraxial modes. It also evolves stochastically due to the continuous measurement of . However, the contributions to the dynamics from both collective scattering and continuous measurement are small in comparison to diffuse scattering and can be neglected when the radiation pattern of the cloud is well matched to that of the probe.

We consider the moment evolution, Eq. (53), with the initial condition that the ensemble is in a SCS polarized along . The initial mean spin and variance are and