# The impact of correlated noise on galaxy shape estimation for weak lensing

###### Abstract

The robust estimation of the tiny distortions (shears) of galaxy shapes caused by weak gravitational lensing in the presence of much larger shape distortions due to the point-spread function (PSF) has been widely investigated. One major problem is that most galaxy shape measurement methods are subject to bias due to pixel noise in the images (“noise bias”). Noise bias is usually characterized using uncorrelated noise fields; however, real images typically have low-level noise correlations due to galaxies below the detection threshold, and some types of image processing can induce further noise correlations. We investigate the effective detection significance and its impact on noise bias in the presence of correlated noise for one method of galaxy shape estimation. For a fixed noise variance, the biases in galaxy shape estimates can differ substantially for uncorrelated versus correlated noise. However, use of an estimate of detection significance that accounts for the noise correlations can almost entirely remove these differences, leading to consistent values of noise bias as a function of detection significance for correlated and uncorrelated noise. We confirm the robustness of this finding to properties of the galaxy, the PSF, and the noise field, and quantify the impact of anisotropy in the noise correlations. Our results highlight the importance of understanding the pixel noise model and its impact on detection significances when correcting for noise bias on weak lensing.

###### keywords:

data analysis – techniques: image processing – gravitational lensing: weak## 1 Introduction

Gravitational lensing, the deflection of light by mass, has been used for many different applications due to its sensitivity to all gravitating mass, including dark matter. Weak gravitational lensing (for a review, see Bartelmann & Schneider, 2001; Refregier, 2003; Schneider, 2006; Hoekstra & Jain, 2008; Massey et al., 2010a; Weinberg et al., 2013), which causes tiny but coherent distortions in galaxy shapes, can reveal the dark matter halos in which galaxies live (e.g., Leauthaud et al., 2012; Tinker et al., 2013; Velander et al., 2014; Coupon et al., 2015; Han et al., 2015; Hudson et al., 2015; Zu & Mandelbaum, 2015), can constrain the amplitude of matter fluctuations and reveal the impact of dark energy on structure growth (e.g., Heymans et al., 2013; Jee et al., 2013; Mandelbaum et al., 2013), and teach us about the theory of gravity on cosmological scales (e.g., Reyes et al., 2010; Simpson et al., 2013; Pullen et al., 2015).

As lensing surveys grow and their statistical errors decrease in size, systematic errors become more important. Among the main systematic errors in weak lensing measurements are those that arise when trying to accurately infer the shears (shape distortions) based on galaxy shape measurements (e.g., Heymans et al., 2006; Massey et al., 2007; Bridle et al., 2010; Kitching et al., 2012; Mandelbaum et al., 2015). An important component of this problem is the substantial bias that arises due to pixel noise (“noise bias”) (Hirata et al., 2004; Kacprzak et al., 2012; Melchior & Viola, 2012; Refregier et al., 2012), which is typically larger than the statistical uncertainties expected from weak lensing datasets in the next few years.

The noise model in astronomical images is typically a combination of Poisson noise on the pixel counts plus other sources of noise that are Gaussian (e.g., read noise). The noise can commonly be modeled as stationary on the scale of galaxy images, at least for the faint galaxies that dominate weak lensing measurements. In this regime, the variance is the same in each pixel, which is true when the Poisson noise on the sky level dominates and when the sky level does not vary much across each galaxy. In an unprocessed image, the noise is largely uncorrelated between pixels, though the light profiles of galaxies that are just below the detection threshold do induce some noise correlations. Even more significant correlations between the noise in different pixels can be induced via processes such as correction for charge transfer inefficiency (Massey et al., 2010b) and image resampling (Lauer, 1999; Fruchter & Hook, 2002; Fruchter, 2011; Rowe et al., 2011) as part of the image combination process, particularly if the images are resampled to smaller pixels than those on the original detector. In combined images from the HST (Hubble Space Telescope) used for weak lensing, this effect can be quite significant (e.g., Koekemoer et al., 2007), enough so that naive estimates of signal-to-noise ratios are wrong by factors of more than two if the correlations are ignored (Casertano et al., 2000).

In this paper, we therefore consider the issue of noise bias in the context of correlated noise. There are several (essentially interchangeable) ways to think about noise bias. In general, it can be thought of as a change in shape of the likelihood surface due to noise (Refregier et al., 2012), such that a maximum likelihood estimator of the per-object shape is a biased estimator. Alternatively, for the case of moments-based estimators, the bias arises because PSF correction involves the division of two noisy quantities, which has an expectation value that is not mathematically equivalent to the ratio of their (noiseless) expectation values. The size of the bias can be written as a Taylor expansion in the inverse detection significance (e.g., Hirata et al., 2004).

In light of this background, the following questions are of
particular interest. First, does noise bias have any fundamentally
different characteristics in the presence of correlated noise? Given the mathematical
origin of the effect, it seems that the noise bias should be described the same way for
uncorrelated and correlated noise, provided that the detection significance is properly quantified.
Given this requirement, the second question is whether one can use simulations with
uncorrelated noise to calibrate noise bias even for data with correlated noise
after accounting for the way correlated noise modifies the signal-to-noise ratio (SNR) of the object
detections? Third, should such corrections use the SNR of the galaxy size, flux, or shape?
Finally, if correlated noise has some directionality, how large is the expected bias
(Mandelbaum et al., 2012) in shear estimates? We address these questions using
GalSim (Rowe
et al., 2015), an
open-source^{1}^{1}1https://github.com/GalSim-developers/GalSim image simulation
package that was designed for tests of weak lensing measurement algorithms. It includes routines to
simulate data with uncorrelated or correlated noise with a particular correlation function, making
it ideal for this application.

In case the answers to these questions depend on the galaxy shape measurement method, we use two quite different methods. The first is a moment-based routines (re-Gaussianization; Hirata & Seljak 2003) used for the majority of this work, but a subset of our measurements use a forward model-fitting method, im3shape (Zuntz et al., 2013, 2014). A comparison of results with these methods may give some insight into the general applicability of our results.

## 2 Technical Background

In this section, we define the quantities that are commonly measured from galaxy images used for weak lensing, along with the uncertainty on those quantities.

### 2.1 Galaxy properties

#### 2.1.1 Galaxy shape

The majority of the methods of estimating weak lensing shear distortions from imaging data involve a two-step
process (see, e.g., Mandelbaum
et al., 2015). The first step is to estimate a per-galaxy shape
(defined in one of several ways), and the second step is to combine all the per-galaxy shapes to
estimate the cosmological shear or its correlation function for the ensemble. The prevalence of the
first step in methods that have been used for realistic weak lensing analysis^{2}^{2}2For
alternatives that are under development, see, e.g., Bernstein &
Armstrong (2014) and Zhang
et al. (2015). motivates our
focus on per-galaxy shape estimates as an intermediate step towards the end goal of inferring
lensing shear.

Weak gravitational lensing can be described as a linear transformation between unlensed and lensed coordinates, as encoded in the two components of the complex-valued lensing shear and the lensing convergence . The shear describes the stretching of galaxy images due to lensing, while the convergence describes a change in apparent size and flux for lensed objects at fixed surface brightness. In practice, the observed quantity is the reduced shear, .

Estimation of shear typically starts with some measure of the galaxy shape. Despite the fact that galaxies do not in general have elliptical isophotes, galaxy light profiles are typically modelled as having a well-defined ellipticity, , with magnitude , where is the semi-minor to semi-major axis ratio, and orientation angle of the major axis with respect to some fixed coordinate system. The two shape components can be defined as and , or combined as a complex ellipticity . For a randomly-oriented population of source ellipticities, the ensemble average ellipticity after lensing is an unbiased estimate of the reduced shear: .

Another common choice of shape parametrization is based on second moments of the galaxy image,

(1) |

where the coordinates and correspond to the and directions (respectively), denotes the galaxy image light profile, is a weighting function (see Schneider, 2006), and the coordinate origin is at the galaxy image center (the centroid). A second definition of ellipticity, sometimes referred to as the distortion, can be written as

(2) |

For many weight functions , an image with elliptical isophotes of axis ratio has

(3) |

For a randomly-oriented population of source distortions, the ensemble average after lensing gives an unbiased estimate of approximately twice the shear that depends on the population root mean square (RMS) ellipticity, .

Finally, in the Gaussian approximation (Bernstein & Jarvis, 2002; Refregier et al., 2012), the statistical uncertainty on and are and . Here is a total detection significance. Using simulations with many noise realizations to estimate the uncertainty on allows for the definition of a shape-based detection-significance . In practice, we use the approximate relations above (, without the factor of ) since per-object scatter in would result in very uncertain estimates; however, when using the true value of , the conclusions of the paper are unchanged.

#### 2.1.2 Galaxy shape estimation methods

Our work relies on two methods of galaxy shape estimation, re-Gaussianization (Hirata & Seljak, 2003) and im3shape (Zuntz et al., 2013, 2014), that use a distortion and ellipticity (respectively) to characterize shapes.

The re-Gaussianization method is a modified version of ones that use “adaptive moments” (which are equivalent to fitting the light intensity profile to an elliptical Gaussian). The re-Gaussianization method involves determining shapes of the PSF-convolved galaxy image based on adaptive moments and then correcting the resulting shapes based on adaptive moments of the PSF. There are also additional steps to correct for non-Gaussianity of both the PSF and the galaxy surface brightness profiles (Hirata & Seljak, 2003).

im3shape is a maximum-likelihood fitting method. It iteratively fits the selected type of galaxy model to the observed galaxy image, using an oversampled image of the PSF. Our adopted settings for all im3shape calculations in this work are given in Appendix A.

#### 2.1.3 Galaxy size

Many forward-modeling methods of estimating galaxy shapes also include an estimate of the intrinsic galaxy size (before convolution with the PSF; see, e.g., Zuntz et al., 2013). Estimates of galaxy intrinsic size are also useful for estimates of lensing magnification that take advantage of size information (Schmidt et al., 2012; Huff & Graves, 2014). With im3shape, the natural size estimate to use is the intrinsic half-light radius, .

While measurements of the observed (PSF-convolved) size from the elliptical Gaussian-weighted adaptive moments (Hirata & Seljak, 2003) do not say much about the real galaxy light profile, they can be useful for quantifying the detection significance:

(4) |

#### 2.1.4 Flux

Another type of detection significance is derived from the flux from the adaptive moments, and its uncertainty. Note that the adaptive moments flux may differ from the total flux due to PSF-convolved galaxies not having Gaussian profiles. Nonetheless, it is a fast and efficient way to estimate another form of detection significance:

(5) |

### 2.2 SNR estimators

Estimates of noise bias using simulations are typically done as a function of the SNR of the object detection. Several weak lensing community challenges (Bridle et al., 2009; Mandelbaum et al., 2014) have used an optimal SNR estimator that uses the object light profile itself as a weight function, which is only possible if the true noise-free light profile is known. This optimal estimator, , is defined as follows. The signal can be defined using a weighted sum over the pixels in the image,

(6) |

and its variance is

(7) |

In the limit that the sky background dominates, is a constant, denoted (the pixel noise variance). Adopting a matched filter for , i.e., and putting our assumptions into Eqs. (6) and (7) gives

(8) |

However, with noisy images, is not a measurable quantity, forcing us to use realistic estimators of SNR that can differ from by as much as a factor of two (see, e.g., Mandelbaum et al., 2015). As noted in Sec. 2.1, there are several possible empirical estimates of detection significance: , , and . These can be estimated using analytic formulae in some cases, or using measurements of multiple noise realizations for each object. One of our goals is to compare the behavior of these different estimators of detection significance and relate them to .

### 2.3 Pixel noise models

The noise fields used in this work are stationary (noise variance constant across the image) and Gaussian-distributed. This approximation is appropriate in the limit that the sky background dominates and is large enough that a Poisson distribution is nearly Gaussian, which describes most galaxies used for weak lensing measurements from the ground and a smaller but non-negligible fraction from space.

Our uncorrelated noise fields are simply drawn from the appropriate Gaussian distribution, with a new random number drawn for each pixel. The correlated noise fields include the full scale-dependent and direction-dependent correlation between noises in adjacent pixels in the images used for weak lensing science in the COSMOS survey (Koekemoer et al., 2007; Scoville et al., 2007a, b). These correlations largely arise due to choices in how the image resampling and combination process is done. Figure 2 in Rowe et al. (2015) illustrates the noise correlation function for these noise fields; as shown, when moving one pixel in the horizontal or vertical direction, this correlation function is to , while moving two pixels away results in it decreasing by another factor of 10.

This specific noise correlation function is adopted as a practical example, but we caution that the details of our results (for example, detection significances without vs. with correlated noise) depend on the adopted level of correlations, which will in general differ for different datasets and analysis methods. We explore variations in our default level of noise correlation in Section 4.5 and find that our overall conclusions are valid for significantly stronger and weaker noise correlations, as well. Thus, while coadded ground-based data will typically have weaker correlation patterns than our default (since the images are typically resampled to a pixel scale comparable to the native one, rather than a significantly smaller one), our overall conclusions should apply to typical ground-based data.

Finally, a subset of our simulations include anisotropies in the correlated noise. Correlated noise is in general anisotropic, due (for example) to direction-dependent corrections for charge transfer inefficiency and camera distortion. Our tests reveal the degree to which these low-level anisotropies in the correlated noise field can contaminate per-galaxy shear estimates.

## 3 Simulation and analysis methods

Image generation (Sec. 3.1) and the measurement process (Sec. 3.2) is performed in a single seamless pipeline using GalSim. Galaxy and PSF profiles are convolved and rendered using Fourier-based rendering methods (Rowe et al., 2015). Many independent noise realizations (up to ) are generated for each value of SNR, and analyzed using the shape measurement methods in Sec. 2.1.2.

### 3.1 Simulation generation

#### 3.1.1 Galaxy Profile

For this work, galaxy light profiles are represented using the Sérsic profile (Sérsic, 1963), a family of light profiles that can describe a wide range of galaxies. The surface brightness of a Sérsic profile varies as

(9) |

where is the total flux, is the half-light radius, and and are known functions with numerical solutions. Flux normalization determines , while is set by requiring that the half-light radius enclose half of the flux.

Our tests use sixty possible Sérsic galaxy profiles with one of five values of Sérsic index, two values for each shape component, and three values of size. Each galaxy consists of a single Sérsic component. Table 1 lists the values used for these parameters.

, | |

, | |

PSF | Kolmogorov (FWHM: 0.7″) |

Pixel Scale | 0.2″/ pixel |

#### 3.1.2 PSF and pixel response

The simulated PSF is a circular Kolmogorov profile with a full width at half maximum (FHWM) of 0.7″. This choice is representative of long-exposure times from ground-based telescopes at sites with good imaging conditions. Our simulated ground-based images have a pixel scale of ″ per pixel.

#### 3.1.3 Image rendering

The galaxy and PSF (including the Kolmogorov profile and pixel response) are convolved using Fourier-space rendering in GalSim, and the light profiles are sampled onto a pixel image using the chosen pixel scale.

#### 3.1.4 Noise field

For each value of , Eq. (8) was used to calculate a noise variance to be added to the initially noise-free postage stamp image. Many noise realizations with that variance are then generated, using uncorrelated and correlated noise as described in Sec. 2.3. Example images are shown in Fig. 1, with the sheared correlated noise case only used in Sec. 4.5.2.

### 3.2 Measurement process

Measurements of flux, size (), and shape () were taken on as many as different realizations of correlated and uncorrelated noise for the same galaxy image. In addition to quantifying biases in the above quantities as a function of the idealized detection significance , the many noise realizations are also used to estimate the effective detection significances defined in Sec. 2, namely , , . These are strictly below (which assumes perfect information about the light profile) and lower for correlated noise than for uncorrelated noise.

Flux, size, and shape measurements from a single galaxy image were considered failures and discarded if the code raised a fatal error, or if one of the measurement of flux or size fell outside a generalized range centered at the mean. This range is determined by independently constructing the probability distribution function (PDF) of the values of both log(size) and log(flux), and then estimating and as the difference between the 50th and 16th percentile, and the 84th and 50th percentile, respectively for each PDF. Measured values that are less than below the mean, or more than above the mean, are excluded as failures.

Any dataset consisting of the measurements of flux, size, and shape for all noise realizations of a particular galaxy model and noise variance with a failure rate above 20 per cent is discarded, out of a concern that the remaining 80 per cent of successful measurements might be a non-representative sample. Failure rates this high only occur for the case of correlated noise with and ; the failures arise primarily due to a failure of the adaptive moment routine to converge in a reasonable number of iterations at low effective SNR. As a result, there are two fewer points for correlated noise results than for uncorrelated noise on every plot in Section 4. Failure rates as a function of are shown for one particular galaxy model in Fig. 2, which shows a rapid convergence to a very low failure rate at for both correlated and uncorrelated noise. In principle, a dependence of the failure rate on galaxy position angle and/or shape could result in a form of shear selection bias that depends on the type of noise (correlated vs. uncorrelated). We will explore this possibility in Sec. 4.4.2.

## 4 Results

Our results are organized as follows. Sec. 4.1 shows the biases in galaxy shapes for a single galaxy model as a function of the ideal detection significance for correlated and uncorrelated noise. Sec. 4.2 shows our results for the effective detection significance defined in various ways, again for both correlated and uncorrelated noise.

An obvious question to ask is that since uncorrelated and correlated noise with different variance (zero-lag value of the noise correlation function) results in different effective detection significance but the same , are the shear biases determined in Sec. 4.1 better quantified in terms of the effective detection significance? And, if defined that way, do the results with uncorrelated and correlated noise follow identical trends? We address this point in Sec. 4.3. The results up to that point are for a specific galaxy model and to a noise correlation structure similar to that seen in COSMOS weak lensing images. The dependence of these results on galaxy properties such as size, Sérsic index, and shape (Sec. 4.4), and on PSF and noise field properties (Sec. 4.5) are shown next. Finally, the extension of our results to another galaxy shape measurement code, im3shape, is shown in Sec. 4.6.

### 4.1 Shape bias

Unless otherwise specified– i.e., in Sec. 4.4 and 4.5 – the plots and data presented in this and subsequent subsections are for a galaxy with , , and . These midrange values were chosen to represent the overall conclusions drawn from analysis of all combinations of galaxy parameters in Table 1.

Fig. 3 shows the fractional bias in galaxy shape measurements^{3}^{3}3For a weak
lensing measurement, what actually matters is the bias in the ensemble average shear
rather than the per-galaxy shapes. The bias in that ensemble shear estimate is a weighted
average bias of the per-galaxy shape biases, which will in general depend on SNR, resolution,
morphology, and other quantities. Nonetheless, for this study
we focus on galaxy shape biases, as a simple way to gain some initial insight into the impact of
correlated noise. as
a function of the ideal detection significance, , for this galaxy model. The
results are quite similar for both components, so future plots of
shear biases are restricted to a single component. There
is clear convergence to a bias of zero at for
both the correlated and uncorrelated cases. At lower detection
significance, the bias becomes negative, indicating an overly round
shape. The sign of this effect is consistent with simulation-based
investigations of the shear biases of the re-Gaussianization method
(Mandelbaum et al., 2012; Mandelbaum
et al., 2015).

At a fixed value of , the bias is stronger for correlated noise than for uncorrelated noise, but the curves have approximately the same shape. This trend suggests an interpretation in terms of some effective, empirically-determined detection significance, an idea that will be explored further in Sec. 4.3.

### 4.2 Effective detection significance

This section contains empirical estimates of the effective detection significance (, , and ), using the same galaxy model as in the previous subsection. The results shown here use noise realizations in order to robustly measure uncertainties in measured quantities.

Measurements for a range of noise variance values spanning over two orders of magnitude (), shown in Fig. 4, reveal that for the majority of that range, the ratio of effective to ideal detection significance is consistent with a single constant value that is significantly below , as in Mandelbaum et al. (2015). That ratio takes on different values in the case of uncorrelated noise and correlated noise, with the former being higher (as expected) by roughly a factor of 2. For , this simple relation is violated, with falling below the linear relation.

For fixed , the values satisfy . This hierarchy is not necessarily consistent for galaxy models with a different size or shape, or for different correlated noise fields, as will be shown in subsequent sections.

### 4.3 Remapping shape biases with

The results from Sec. 4.2 can be used to reinterpret
those from Sec. 4.1 in terms of effective detection
significances . Starting with the results in Fig. 4, we
can scale the values of for each dataset by
to plot galaxy shape biases as a function of
. This ratio, , is taken to be a single
number that can be read off from Fig. 4 for each type of detection significance
and noise^{4}^{4}4One might ask why we do not simply
use the measured values of directly, instead of applying
the simple linear scaling . We adopt
this simpler procedure because in a study aimed at calibrating shear
biases, it seems more likely that one would determine roughly how
the detection significance scales with the square root of the noise
variance, and apply an overall correction factor, just as we are
doing here.. The
key step is to do this rescaling separately for uncorrelated and correlated noise. Our goal is to
determine whether, after doing so, the highly offset curves in
Fig. 3 for correlated vs. uncorrelated noise lie on top
of each other. We
focus on the region where the shape bias is significantly nonzero,
, since that is the regime where noise bias
must be calibrated using simulations.

Fig. 5 shows the relative bias in one shape component as a function of the three values and , for both uncorrelated and correlated noise. In Fig. 3, differences in the shear calibration bias for uncorrelated noise vs. correlated noise at fixed were typically factors of . It is clear that the use of any of our options in Fig. 5 significantly reduces this difference to at most 20 per cent. However, of the three options, most reduces the difference between the shear biases with correlated and uncorrelated noise. When plotting the bias in galaxy shape as a function of , the curves for uncorrelated and correlated noise lie nearly on top of each other. is the next best option, and is, interestingly, the worst option of the three.

The results of this subsection suggest that when applying simulation-based noise bias corrections to galaxy shape estimates, results from noise correlation-free simulations could be used to correct the results in data that includes noise correlations, as long as the impact of noise correlations on the detection significance are accounted for. To quantify what error might be induced by doing so, we take the points in Fig. 5 for uncorrelated noise, and find a three-parameter fitting function that describes the bias as a function of , using to define the effective detection significance. This function is

In reality the expansion is more likely a constant plus plus higher order terms as shown in Hirata et al. (2004), but we used a single term with varying power-law index to effectively include some higher order behavior without introducing many degrees of freedom into the fit. We evaluate this function at the values of in Fig. 5, and calculate the residual bias for the correlated noise case, defining the residual as the actual bias on Fig. 5 minus the bias from the fit. The RMS value of the residual bias over all points on the curve is 0.3%, more than an order of magnitude below the typical biases for .

### 4.4 Dependence on galaxy properties

This section illustrates the dependence of our results on the properties of the simulated galaxies, going beyond the single galaxy model used in Sec. 4.1. We test how the ratio inferred from Fig. 4, and the efficacy of the remapping described in Sec. 4.3, varies with different properties described below.

#### 4.4.1 Sérsic index and size

The same process as in previous sections was carried out for fifteen different combinations of parameters (five values of Sérsic index, three values of and a single value of ) originally listed in Table 1.

Carrying out the exercise in Fig. 4 to find the relationship between for various choices of reveals variations in those ratios that are as large as 40 per cent across the entire range in Sérsic index or . For a fixed value of (i.e., noise variance) and galaxy flux, tends to be largest for small galaxies (low ) and low Sérsic index. This makes sense, since the light profile covers fewer pixels and thus has effectively less noise in that case.

Tests of the impact of correlated vs. uncorrelated noise use the ratio , as shown in Fig. 6. The left column shows that at fixed Sérsic index, this ratio is a weakly increasing function of , for all three types of . The strength of the trend is larger at low fixed , but still less than 10 per cent in the most extreme case (and therefore weaker than the individual trends in or with ). The right column shows that at fixed size, this ratio is a weakly decreasing function of Sérsic , again reaching a maximum variation of 10 per cent across the entire range of galaxy parameters and being strongest for the largest galaxies considered.

This stability in is fairly remarkable given that the range of galaxy sizes is a factor of 2.5 and the range of Sérsic goes from to . The stability derives from the fact that and both have stronger trends with galaxy properties as mentioned above, but the trends are sufficiently similar that they largely cancel out in the ratio . Any scheme that aims to correct for calibration biases in shape measurements using a combination of simulations with uncorrelated and correlated noise would need to account properly for the differences between and .

#### 4.4.2 Shape Dependence

The next test was for the dependence of these results on the galaxy shape using galaxies with Sérsic and , for and . Since our results in prior sections were very similar for the two shape components, only was varied in this section.

The ratio varies very little ( per cent) over the range of shapes considered here. Moreover, Fig. 8 demonstrates that the remapping successfully reduces the differences between the shape bias for correlated and uncorrelated noise for the three values of considered. This result demonstrates that our results for can be considered as a general result, independent of shape.

As noted in Sec. 3.2, selection effects that correlate with shape can cause a bias in ensemble shear estimation. If this bias differs for correlated and uncorrelated noise, then that would also be a potential issue with remapping shear biases for the two noise types. However, for the simulated galaxy and PSF profiles in this work, and the range of explored in this section, we find that the failure rate is totally independent of for both correlated and uncorrelated noise. It is also independent of the galaxy position angle for fixed . Note that in a more complex situation with anisotropic PSFs, it is possible that this would no longer be the case.

### 4.5 Dependence on image properties

In this section, we explore the dependence of our results on the imaging properties, including the nature of the noise correlations and the type of PSF.

#### 4.5.1 Different scale-length of noise correlations

Our first tests are of the dependence of the results on the scale-length of the noise correlations. While our original noise correlation function corresponded to that in the COSMOS weak lensing science images, GalSim permits users to “magnify” the noise correlation function. This operation corresponds to preserving the variance (zero-lag correlation function) while changing the scale length of correlations. Our tests involve linearly expanding the scale-length of correlations by factors of , for a galaxy with Sérsic , shape , and =0.7″.

The results for are shown in Fig. 9. Since only the correlated noise field is being modified, these should be interpreted as modifications in with remaining fixed. The ratio increases linearly with , implying that the detection significance with correlated noise, , is smaller for large . The sign of this trend makes sense: for a fixed point variance, larger implies that the noise correlations extend over larger spatial scales, which should decrease the effective detection significance for extended objects.

Fig. 10 shows that remapping the shape biases based on the successfully matches up the shape biases for uncorrelated and correlated noise for all three values of . Comparison of the panels indicates that this is a non-trivial statement, since the difference between the results for uncorrelated and correlated noise as a function of the point variance can be quite large for large values of . Hence our results seem fairly robust to the scale-length of the noise correlations.

It is possible that if the scale-length of the correlations became comparable to the galaxy size itself, then additional effects would come into play. The cases considered here are not in this regime (which should not be relevant for too many galaxies that will be used for weak lensing), and we defer this question to future work.

#### 4.5.2 Anisotropy in noise correlations

The correlated noise field used throughout this work has a mild level of anisotropy in
the noise correlations (a few per cent). Generally, unless explicitly
accounted for^{5}^{5}5Most shear estimation routines do not have a
means to account for correlated noise fields, regardless of their
anisotropy level. these anisotropies are expected to be imprinted
at some level in the measurements of galaxy shape, particularly at low
detection significance. This effect is explored in this section, with a
galaxy with Sérsic , =0.7″, and , using our original
correlated noise field as well as versions that have an additional shear applied to the noise correlation
function. To make the trends visually apparent, the noise correlations are sheared by an
exaggerated amount,
(see right-most panel of
Fig. 1 for an example image).

In this case, what matters are the results with shear applied to the correlated noise compared to the results with the original correlated noise. More specifically, when measuring for shape component and the original noise field, and with the sheared noise field, for the case where the expected shape is , the following differences are calculated and plotted:

The results are shown in Fig. 11. When shearing the correlated noise in one component, the dominant effects on the shape bias are in that component, but they are not completely zero for the other component. The sign of the effect is as expected, with positive noise shear in one component translating into a positive shape bias. For the lowest value considered (), the magnitude of the additional shape bias is typically of order 5 per cent. We expect this to scale linearly with the noise shear, so for more realistic values like , the maximum shape bias would be of order 1 per cent for the lowest SNR galaxies in our simulated sample. This effect is therefore large enough to matter for upcoming surveys that require understanding of shear calibration biases at the sub-percent level.

An additional source of bias not considered here is selection bias. If there is anisotropic correlated noise, and if galaxies are more or less likely to be selected for measurement if their shape aligns in some particular way with respect to that noise field, then this will cause an additional bias in ensemble shear estimates that is not captured by our tests.

#### 4.5.3 Ground-like versus space-like data

To test the impact of the PSF type and sampling, simulated space-based data was also generated, with parameters in Table 3, and used for the same analysis as was already presented for the ground-like simulations. For a galaxy model with , , and Sérsic , the ratio of is qualitatively similar to that in Fig. 4, in the sense that there are flat curves for . The actual ratios differ, with values of and for uncorrelated and correlated noise, respectively; and ; and and . Thus, in all three cases, the ratio of for correlated vs. for uncorrelated noise is . However, the hierarchy from Fig. 4 () is not respected here, implying that the hierarchy depends on the details of the type of data. Note that the galaxy model here is smaller than for ground-like data, with instead of . This choice, which is meant to reflect the fact that the typical galaxy that is analyzed in space-based datasets is smaller than can be easily resolved in ground-based datasets, may in part be responsible for the change.

Despite this difference in the error hierarchy, Fig. 12 clearly shows that our conclusion about remapping shape biases using to account for the differences between uncorrelated and correlated noise is also valid for space-like data. In this case, the best remapping is provided by , while the worst comes from .

, | |

, | |

PSF | Airy (: .041253″) |

Pixel Scale | 0.03″/ pixel |

### 4.6 Generalizing to other methods

An obvious question about the results earlier in this section is how specific are they to re-Gaussianization, versus applying more generally. For one galaxy model, the same set of tests were carried out using im3shape, a maximum-likelihood forward model-fitting approach. The configuration settings used to run im3shape for this test are given in Appendix A. The galaxy model used for this test has the following parameters: , , ″, using our standard PSF and pixel scale for ground-based simulations from Table 1.

First, the shape bias as a function of is shown in Fig. 13. The results with uncorrelated noise can be compared against those from the upper left panel of figure 2 in Kacprzak et al. (2012), which investigated noise bias in the context of im3shape. While the shape of our curve appears different from the one in Kacprzak et al. (2012), it is important to keep in mind that our results go to lower signal-to-noise ratio. For the range that the plots have in common, the shape of the curve and magnitude of the bias at low detection significance is actually fairly consistent (with some mild differences that could arise due to the galaxy models not being identical). The curve with correlated noise does not seem to go to positive values, unlike the curve with uncorrelated noise; however, the curves have in common the behavior at the lowest detection significance, where they both go to negative values.

Next, we investigated the for shape, size, and flux from im3shape (Fig. 14). It is important to bear in mind that for im3shape is intrinsically different from from adaptive moments, since the former is intrinsic size and the latter is PSF-convolved. In general, from im3shape is therefore expected to be lower than that from adaptive moments. Indeed, Fig. 14 reveals quite low values of . It is less clear why is so low, implying that detections that are visually quite apparent in images have a flux-based detection significance around 1. Also, it is generally the case that (for all three types of ) are not consistent with being flat, unlike the results in Fig. 4 from adaptive moments, though is closer to flat than . Nonetheless, our final step is to check the remapping of the galaxy shape biases from Fig. 13 using in Fig. 15. Clearly the remapping brings the curves significantly closer together, though their behavior is not identical due to the lack of a small positive bias for some range of detection significance when using correlated noise. Thus, while the results for remapping from re-Gaussianization do not completely carry over to im3shape, the remapping is still somewhat beneficial in reducing differences in shape biases for uncorrelated and correlated noise.

## 5 Conclusions

In this work, we have investigated the impact of correlated noise fields on galaxy shape estimation using the re-Gaussianization galaxy shape estimation method. Most galaxy shape estimation methods ignore correlated noise, which may be present due to galaxies below the detection threshold, or image resampling and other elements of the image processing. Ignoring correlated noise leads to overestimation of the detection significance, and could in principle lead to incorrect conclusions about biases in galaxy shape measurement.

Our results suggest that correlated noise tends to change the effective detection significance in ways that depend only mildly on galaxy properties, and are easily characterized. Moreover, while the galaxy shape biases as a function of the variance of the noise field can differ dramatically for uncorrelated and correlated noise, using an empirical detection significance estimator that takes into account the impact of noise correlations can account for essentially all of this difference. This statement is robust to significant changes in the parameters of the galaxies, PSFs, and correlated noise fields in the images. While this result is perhaps unsurprising given the underlying cause of noise bias (as discussed in Sec. 1), a direct validation using simulations is nonetheless valuable in confirming our understanding of noise bias.

From a theoretical perspective, this result suggests that the correlated noise is not doing anything untoward to the images, and the primary reason they change the results is due to the degradation in detection significance for extended objects. As a practical matter, this is useful; depending on the required level of accuracy, it may not be necessary to run suites of simulations to calibrate noise bias for many different correlated noise fields, but rather that simulations with uncorrelated noise (or a single level of correlated noise) can be used to calibrate noise bias even when the level of noise correlations is not completely uniform across the dataset. This result rests on using a reliable estimator of detection significance that includes the effect of noise correlations. The development of such a calibration scheme that properly takes all factors into account is beyond the scope of this work. In addition, the impact of anisotropy in the correlated noise may cause shear biases that are significant enough to matter for large upcoming weak lensing surveys when using galaxies near the detection limit.

Defining the detection significance can be non-trivial, and if it is done in a way that correlates with shape, this can result in a selection bias with respect to the lensing shear (e.g. Jarvis et al., 2015). While we have verified that shape measurement failure rates do not tend to depend on the magnitude of the galaxy ellipticity for either type of noise, it is unclear whether the correlated noise could interact in some way with the definition of detection significance in a way that modifies selection biases. That issue is beyond the scope of this paper.

Our results about the impact of correlated noise may explain some of the findings of Hoekstra et al. (2015), who note that the estimated shear calibration bias for galaxies limited to -band magnitude depends on the limiting magnitude of the simulated galaxy sample, only converging at a limiting magnitude fainter than 26.5. The galaxies below the detection limit, while not directly used for shear estimation, affect the results for the brighter galaxies that are used, presumably due to the fact that they constitute a source of correlated noise. If this is the primary reason for the need to include galaxies below the detection limit in the simulations in order to get the shear biases to converge, it is possible that the simulated galaxy sample could actually be truncated slightly below (rather than well below) the detection limit if the noise field is generated with a level of noise correlations consistent with the inclusion of fainter galaxies. Including the deblending issues that arise from objects right below the detection threshold will require the simulated sample to go at least magnitudes fainter, but it is worth investigating whether the rest of the effect could be incorporated using correlated noise. This simplification would make the simulation process less expensive, since the steepness of the galaxy number counts means that the sample of galaxies that must be simulated is enlarged by a substantial factor when going magnitudes fainter.

The generality of our result that shape biases can be remapped from uncorrelated to correlated noise is unclear, though our preliminary results suggest that it is beneficial for one other method of galaxy shape estimation and mathematically there are reasons to expect it to be more widely valid. More generally, our results highlight the complexity of noise bias in the case of correlated noise, and the need to account for its impact on effective detection significances when calibrating noise bias in weak lensing measurements.

## 6 Acknowledgements

We thank the anonymous referee for many useful suggestions that improved this paper. We thank Barney Rowe and Joe Zuntz for their assistance in using im3shape for this work and for useful conversations about the results, and Mike Jarvis for helpful comments on the paper itself. RM acknowledges the support of HST-AR-12857.01-A, provided by NASA through a grant from the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Incorporated, under NASA contract NAS5-26555.

## References

- Bartelmann & Schneider (2001) Bartelmann M., Schneider P., 2001, Phys. Rep., 340, 291
- Bernstein & Armstrong (2014) Bernstein G. M., Armstrong R., 2014, MNRAS, 438, 1880
- Bernstein & Jarvis (2002) Bernstein G. M., Jarvis M., 2002, AJ, 123, 583
- Bridle et al. (2009) Bridle S., et al., 2009, Annals of Applied Statistics, 3, 6
- Bridle et al. (2010) Bridle S., et al., 2010, MNRAS, 405, 2044
- Casertano et al. (2000) Casertano S., et al., 2000, AJ, 120, 2747
- Coupon et al. (2015) Coupon J., et al., 2015, MNRAS, 449, 1352
- Fruchter (2011) Fruchter A. S., 2011, PASP, 123, 497
- Fruchter & Hook (2002) Fruchter A. S., Hook R. N., 2002, PASP, 114, 144
- Han et al. (2015) Han J., et al., 2015, MNRAS, 446, 1356
- Heymans et al. (2006) Heymans C., Van Waerbeke L., Bacon D., Berge J., Bernstein G., Bertin E., Bridle S., et al. 2006, MNRAS, 368, 1323
- Heymans et al. (2013) Heymans C., et al., 2013, MNRAS, 432, 2433
- Hirata & Seljak (2003) Hirata C., Seljak U., 2003, MNRAS, 343, 459
- Hirata et al. (2004) Hirata C. M., et al., 2004, MNRAS, 353, 529
- Hoekstra & Jain (2008) Hoekstra H., Jain B., 2008, Annual Review of Nuclear and Particle Science, 58, 99
- Hoekstra et al. (2015) Hoekstra H., Herbonnet R., Muzzin A., Babul A., Mahdavi A., Viola M., Cacciato M., 2015, MNRAS, 449, 685
- Hudson et al. (2015) Hudson M. J., et al., 2015, MNRAS, 447, 298
- Huff & Graves (2014) Huff E. M., Graves G. J., 2014, ApJ, 780, L16
- Jarvis et al. (2015) Jarvis M., et al., 2015, preprint, (arXiv:1507.05603)
- Jee et al. (2013) Jee M. J., Tyson J. A., Schneider M. D., Wittman D., Schmidt S., Hilbert S., 2013, ApJ, 765, 74
- Kacprzak et al. (2012) Kacprzak T., Zuntz J., Rowe B., Bridle S., Refregier A., Amara A., Voigt L., Hirsch M., 2012, MNRAS, 427, 2711
- Kitching et al. (2012) Kitching T. D., et al., 2012, MNRAS, 423, 3163
- Koekemoer et al. (2007) Koekemoer A. M., et al., 2007, ApJS, 172, 196
- Lauer (1999) Lauer T. R., 1999, PASP, 111, 227
- Leauthaud et al. (2012) Leauthaud A., et al., 2012, ApJ, 744, 159
- Mandelbaum et al. (2012) Mandelbaum R., Hirata C. M., Leauthaud A., Massey R. J., Rhodes J., 2012, MNRAS, 420, 1518
- Mandelbaum et al. (2013) Mandelbaum R., Slosar A., Baldauf T., Seljak U., Hirata C. M., Nakajima R., Reyes R., Smith R. E., 2013, MNRAS, 432, 1544
- Mandelbaum et al. (2014) Mandelbaum R., et al., 2014, ApJS, 212, 5
- Mandelbaum et al. (2015) Mandelbaum R., et al., 2015, MNRAS, 450, 2963
- Massey et al. (2007) Massey R., et al., 2007, MNRAS, 376, 13
- Massey et al. (2010a) Massey R., Kitching T., Richard J., 2010a, Reports on Progress in Physics, 73, 086901
- Massey et al. (2010b) Massey R., Stoughton C., Leauthaud A., Rhodes J., Koekemoer A., Ellis R., Shaghoulian E., 2010b, MNRAS, 401, 371
- Melchior & Viola (2012) Melchior P., Viola M., 2012, MNRAS, 424, 2757
- Pullen et al. (2015) Pullen A. R., Alam S., Ho S., 2015, MNRAS, 449, 4326
- Refregier (2003) Refregier A., 2003, ARA&A, 41, 645
- Refregier et al. (2012) Refregier A., Kacprzak T., Amara A., Bridle S., Rowe B., 2012, MNRAS, 425, 1951
- Reyes et al. (2010) Reyes R., Mandelbaum R., Seljak U., Baldauf T., Gunn J. E., Lombriser L., Smith R. E., 2010, Nature, 464, 256
- Rowe et al. (2011) Rowe B., Hirata C., Rhodes J., 2011, ApJ, 741, 46
- Rowe et al. (2015) Rowe B. T. P., et al., 2015, Astronomy and Computing, 10, 121
- Schmidt et al. (2012) Schmidt F., Leauthaud A., Massey R., Rhodes J., George M. R., Koekemoer A. M., Finoguenov A., Tanaka M., 2012, ApJ, 744, L22
- Schneider (2006) Schneider P., 2006, in Meylan G., Jetzer P., North P., Schneider P., Kochanek C. S., Wambsganss J., eds, Saas-Fee Advanced Course 33: Gravitational Lensing: Strong, Weak and Micro. pp 269–451
- Scoville et al. (2007a) Scoville N., et al., 2007a, ApJS, 172, 1
- Scoville et al. (2007b) Scoville N., et al., 2007b, ApJS, 172, 38
- Sérsic (1963) Sérsic J. L., 1963, Boletin de la Asociacion Argentina de Astronomia La Plata Argentina, 6, 41
- Simpson et al. (2013) Simpson F., et al., 2013, MNRAS, 429, 2249
- Tinker et al. (2013) Tinker J. L., Leauthaud A., Bundy K., George M. R., Behroozi P., Massey R., Rhodes J., Wechsler R. H., 2013, ApJ, 778, 93
- Velander et al. (2014) Velander M., et al., 2014, MNRAS, 437, 2111
- Weinberg et al. (2013) Weinberg D. H., Mortonson M. J., Eisenstein D. J., Hirata C., Riess A. G., Rozo E., 2013, Phys. Rep., 530, 87
- Zhang et al. (2015) Zhang J., Luo W., Foucaud S., 2015, J. Cosmology Astropart. Phys., 1, 24
- Zu & Mandelbaum (2015) Zu Y., Mandelbaum R., 2015, MNRAS, 454, 1161
- Zuntz et al. (2013) Zuntz J., Kacprzak T., Voigt L., Hirsch M., Rowe B., Bridle S., 2013, MNRAS, 434, 1604
- Zuntz et al. (2014) Zuntz J., Kacprzak T., Voigt L., Hirsch M., Rowe B., Bridle S., 2014, IM3SHAPE: Maximum likelihood galaxy shear measurement code for cosmic gravitational lensing, Astrophysics Source Code Library (ascl:1409.013)

## Appendix A im3shape settings

Table 4 shows the parameters used for all im3shape plots used in this work.

Parameter | Value |
---|---|

`model_name` |
`sersics` |

`perform_pixel_integration` |
`N` |

`verbosity` |
`-1` |

`minimizer_verbosity` |
`-1` |

`noise_sigma` |
`1.0` |

`background_subtract` |
`N` |

`upsampling` |
`7` |

`n_central_pixel_upsampling` |
`0` |

`n_central_pixels_to_upsample` |
`0` |

`padding` |
`0` |

`stamp_size` |
`50` |

`rescale_stamp` |
`Y` |

`minimizer_max_iterations` |
`150` |

`levmar_eps1` |
`1e-40` |

`levmar_eps2` |
`1e-40` |

`levmar_eps3` |
`1e-40` |

`levmar_eps4` |
`-1` |

`levmar_eps5` |
`1e-05` |

`levmar_tau` |
`1e-5` |

`levmar_LM_INIT_MU` |
`1e-20` |

`minimizer_loops` |
`1` |

`sersics_bulge_A_min` |
`-50` |

`sersics_radius_start` |
`10` |

`psf_input` |
`psf_image_single` |