Catastrophic photometric redshift errors:
weak lensing survey requirements
We study the sensitivity of weak lensing surveys to the effects of catastrophic redshift errors — cases where the true redshift is misestimated by a significant amount. To compute the biases in cosmological parameters, we adopt an efficient linearized analysis where the redshift errors are directly related to shifts in the weak lensing convergence power spectra. We estimate the number of unbiased spectroscopic redshifts needed to determine the catastrophic error rate well enough that biases in cosmological parameters are below statistical errors of weak lensing tomography. While the straightforward estimate of is , we find that using only the photometric redshifts with leads to a drastic reduction in to while negligibly increasing statistical errors in dark energy parameters. Therefore, the size of spectroscopic survey needed to control catastrophic errors is similar to that previously deemed necessary to constrain the core of the distribution. We also study the efficacy of the recent proposal to measure redshift errors by cross-correlation between the photo-z and spectroscopic samples. We find that this method requires a priori knowledge of the bias and stochasticity of the outlier population, and is also easily confounded by lensing magnification bias. The cross-correlation method is therefore unlikely to supplant the need for a complete spectroscopic redshift survey of the source population.
Weak gravitational lensing is a very promising cosmological probe that has potential to accurately map the distribution of dark matter and measure the properties of dark energy and the neutrino masses (for reviews, see Bartelmann and Schneider (2001); Huterer (2002); Munshi et al. (2008); Hoekstra and Jain (2008)). It is well understood, however, that systematic errors may stand in the way of weak lensing reaching its full potential — that is, achieve the statistical errors predicted for future ground and space based surveys such as the Dark Energy Survey (DES), Large Synoptic Survey Telescope (LSST), and the Joint Dark Energy Mission (JDEM). Controlling the systematic errors is a primary concern in these and other surveys so that a variety of dark energy tests (recently proposed and reviewed by the JDEM Figure of Merit Science Working Group Albrecht et al. (2009)) can be performed to a desired high accuracy.
Several important sources of systematic errors in weak lensing surveys have already been studied. Chief among them is the redshift accuracy—approximate, photometric redshifts are necessary because it is infeasible to obtain optical spectroscopic redshifts for the huge number (-) of galaxies that future surveys will utilize as lensing sources. It is therefore imperative to ensure that statistical errors and systematic biases in the relation between photometric and spectroscopic redshifts (recently studied in depth with real data Oyaizu et al. (2007); Banerji et al. (2007); Lima et al. (2008); Cunha et al. (2008); Abdalla et al. (2008)) do not lead to appreciable biases in cosmological parameters.
The relation between the photometric and spectroscopic redshift has been previously modeled as a Gaussian with redshift-dependent bias and scatter. It is found that both the bias and scatter (that is, quantities and in each bin of ), need to be controlled to about or better in order to lead to less than degradation in cosmological parameter accuracies Ma et al. (2005); Huterer et al. (2006); Kitching et al. (2008a). These constraints are a bit more stringent in the most general case when the redshift error cannot be described as a Gaussian Ma and Bernstein (2008). These requirements imply that spectra are required in order to calibrate the photometric redshifts to the desired accuracy. Gathering such relatively large spectroscopic sample will be a challenge, setting a limit to the useful depth of weak lensing surveys. [While the redshift errors have been well studied, other systematics are also important, especially theoretical errors in modeling clustering of galaxies at large and small scales, intrinsic shape alignments, and various systematic biases that take place during observations Huterer and Takada (2005); White (2004); Hagan et al. (2005); Zhan and Knox (2004); Huterer and White (2005); Zentner et al. (2008); Rudd et al. (2007); Shapiro and Cooray (2006); Shapiro (2008); Heitmann et al. (2005, 2008); Takada and Bridle (2007); Takada and Jain (2008); Kitching et al. (2008b); Joachimi and Schneider (2008); Bernstein (2008); Jarvis and Jain (2004); Ma et al. (2008); Guzik and Bernstein (2005); Stabenau et al. (2007); Heymans et al. (2006); Massey et al. (2007); Paulin-Henriksson et al. (2007); Amara and Refregier (2007); Heavens et al. (2000); Croft and Metzler (2000); Crittenden et al. (2001); Mackey et al. (2002); Jing (2002); Heymans and Heavens (2003); King and Schneider (2003); Hirata et al. (2004); Hirata and Seljak (2004); Mandelbaum et al. (2006); Bridle and Abdalla (2007); Bridle and King (2007).]
All of the aforementioned photo-z requirement studies (e.g. Huterer et al. (2006); Ma et al. (2005)), however, have modeled the errors as a perturbation around relation. While this perturbation was allowed to be large and to have a nonzero scatter and even skewness (e.g. Ma et al. (2005)), it did not subsume a general, multimodal error in redshift.
In this paper we would like to remedy this omission by estimating the effect of catastrophic redshift errors. Catastrophic errors are loosely defined as cases when the photometric redshift is grossly misestimated, i.e. when , and are represented by arbitrary “islands” in the plane. We develop a formalism that treats these islands as small “leakages” (or “contaminations”) and directly estimates their effect on bias in cosmological parameters. We then invert the problem by estimating how many spectroscopic redshifts are required to control catastrophic errors at a level that makes them harmless for cosmology.
The paper is organized as follows. In §II we derive the relevant equations for the bias in cosmological parameters induced by misestimated catastrophic redshift errors in a tomographic weak lensing survey. In §III we apply these methods to a canonical ambitious weak-lensing cosmology project. In §IV we ask: how large must a complete spectroscopic redshift survey be in order that the catastrophic photo-z error rates be measured sufficiently well that remnant cosmological biases are well below the statistical uncertainties? Newman Newman (2008) has suggested an alternative mode of measuring the photo-z error distribution, namely the angular cross-correlation of the photometric galaxy sample nominally at with a spectroscopic sample at ; in §V we investigate whether systematic errors in the photo-z outlier rates derived from this cross-correlation technique will be small enough to render cosmological biases insignificant. The final section discusses the scaling of these results with critical survey parameters, the ramifications for survey design, and areas of potential future investigation.
In this section we establish the formalism that takes us from “islands” in the plane to biases in cosmological parameters. First, however, we define the basic observable quantity, the convergence power spectrum, and its corresponding Fisher information matrix.
ii.1 Basic observables and the Fisher matrix
The convergence power spectrum of weak lensing at a fixed multipole and for the th and th tomographic bin is given by
where is the comoving distance, is the Hubble parameter, and is the matter power spectrum. The weights are given, for a flat Universe, by where , is the comoving distance and is the comoving density of galaxies if falls in the distance range bounded by the th redshift bin and zero otherwise. We employ the redshift distribution of galaxies of the form that peaks at .
The observed convergence power spectrum is
where is the rms intrinsic shear in each component which we assume to be equal to , and is the average number of galaxies in the th redshift bin per steradian. The cosmological constraints can then be computed from the Fisher matrix
where is the inverse of the covariance matrix between the observed power spectra. For a Gaussian convergence field, its elements are given by
where is fraction of the sky observed and is the binning of the convergence power spectra in multipole.
Our fiducial SNAP survey described below, without any theoretical systematics, determines and to accuracies of and (corresponding to the pivot value determined to ).
ii.2 Biases in the Gaussian limit
Consider the general problem of constraining a vector of cosmological parameters based on an observed data vector . If the observable quantities are distributed as Gaussians with covariance matrix , then the first-order formula for bias in the th parameter, , induced by a bias in the data is (e.g. Knox et al. (1998); Huterer and Turner (2001))
Here is the Fisher matrix for the cosmological parameters, and is defined as the second derivative of log likelihood () with respect to the parameters. The bias above can be more concisely expressed as
where we have defined the matrix and vector as
The induced parameter bias is considered unimportant if it is small compared to the expected statistical variation in the cosmological parameters. In the case where the likelihood in the parameter space is Gaussian, the likelihood of the bias being exceeded by a statistical fluctuation is determined by
In the Appendices we prove two useful theorems about :
The bias significance always decreases or stays fixed when we augment the likelihood with (unbiased) prior information, e.g. data from a non-lensing technique;
always decreases or stays fixed when we marginalize over one or more dimensions of the parameter space. In the Gaussian limit, the bias is unaffected by marginalization over other parameters.
We will use these results later to argue that our requirements on the control of catastrophic redshift errors are conservative, in the sense that adding other cosmological data or considering individual cosmological parameter biases will only weaken the requirements.
ii.3 The case of catastrophic photo-z errors
For weak-lensing tomography, the data elements are the convergence (or shear) cross-power spectrum elements between photo-z bins and at multipole . Let us examine how these will be biased by photo-z outliers. [The data covariance matrix of §II.2 is the matrix of Eq. (4).]
We assume a survey with the (true) distribution of source galaxies in redshift , divided into some number of bins in redshift. Let us define the following terms
Leakage: fraction of objects from a given spectroscopic bin that are placed into an incorrect (non-corresponding) photometric bin.
Contamination: fraction of galaxies in a given photometric bin that come from a non-corresponding spectroscopic bin.
One could estimate either of these quantities — after all, when specified for each bin, they contain the same information. Let leakage fraction of galaxies in some spectroscopic-redshift bin (the “source” of leakage) end up in some photo-z bin (the “target” of leakage), so that is the fractional perturbation in the source bin. Note that, since bins and may not have the same number of galaxies, the fractional perturbation in the target bin is not the same. The contamination of the target bin , , is related to the source bin leakage via
where and are the absolute galaxy numbers in the source and target bin respectively.
The redshift distribution of galaxies (normalized to unity) in the source bin, , does not change, since a fraction of galaxies is lost — but the redshift distribution is normalized to unit integral; see Fig. 1. Conversely, things are perturbed in the target bin, since it now contains two populations of galaxies, the original one with fraction , and the contamination at incorrect (source bin) redshift with fraction ; again this is clearly shown in Fig. 1. Therefore
and only the target bin is affected (i.e. biased) by photo-z catastrophic errors.
The effect on the cross power spectra is now easy to write down. Clearly, only the cross-spectra where one of the bins is the target bin — or — will be affected
We have checked that ignoring the quadratic terms in leads to no observable effects to the results (for contamination). The biases can now be computed as the right hand side minus the left hand side in the formulae above. We replace the single index for data elements in Eq. (5) with the triplet so that and we reserve the symbol for the covariance of the data elements. The bias in data induced by the catastrophic errors is
If we make the further assumption that the convergence is a Gaussian random field, then we have
As a reminder, the Fisher matrix in the case of a zero-mean Gaussian variable is Tegmark et al. (1997)
In a cosmological application we will marginalize over all parameters except a subset of interest . In the Fisher approximation bias is simply projected onto the subset: , where is the projection matrix (see Appendix B). If is the marginalized Fisher matrix, then the of the bias after marginalization is
Iii Application to canonical surveys
For first study we examine a weak lensing survey similar to that proposed for SNAP Aldering et al. (2004), but with the source-galaxy selection restricted to incur minimal catastrophic error rate. Evaluation of other potential surveys could be performed following the same model.
We take the eight-parameter cosmological model considered by the Dark Energy Task Force (DETF; (Albrecht et al., 2006)): dark energy physical density , and equation of state parameters and ; normalization of the primordial power spectrum and spectral index ; and matter, baryon, and curvature physical densities and . The fiducial values of these parameters are taken from the 5-year WMAP data Komatsu et al. (2008). We will assume a Planck CMB prior as specified by the DETF report. Recall that application of further priors can only weaken the requirements on photo-z outliers (see Appendix A).
We assume shear tomography with 20 bins linearly spaced over with ; we have checked that the results are stable with . The redshift distribution and fiducial are taken from a simulation of the photo-z performance of SNAP as described in Jouvel et al. (2009). The procedure is to (1) create a simulated catalog of galaxies; (2) calculate their noise-free apparent magnitudes in the SNAP photometric bands spanning the visible and NIR to 1.6 m; (3) add the anticipated observational noise to each magnitude; (4) determine a best-fit galaxy type and redshift with the template-fitting program Le Phare111www.oamp.fr/people/arnouts/LE PHARE.html; (5) examine the 95% confidence region determined by Le Phare and retain only those galaxies satisfying (Dahlen et al., 2008)
This strict cut results in a catalog of galaxies per arcmin, with a median redshift of . The WL survey is assumed to cover of the full sky, with shape noise of per galaxy. We consider only shear tomography at the 2-point level, as this will likely maximize the bias imparted by redshift errors. We also ignore systematic errors other than redshift outliers, which will likely maximize the statistical significance of the outlier bias.
We will henceforth in this paper define a redshift outlier to satisfy
In the simulated photo-z catalog, 2.2% of the source galaxies are outliers by this criterion. In our analyses below we will only consider biases from photo-z errors meeting this outlier criterion, i.e. we assume the “core” of the photo-z distribution is well determined.
Figure 2 illustrates the canonical model and the sensitivity to redshift outliers in this model. The left-hand panel shows the quantity vs and . [We scale the contamination by to produce a quantity that is independent of the choice of bin size .] The highest contaminations are in two “islands”: one at , is probably due to confusion between high- Lyman breaks and low- 400-nm breaks. Because true galaxies are relatively rare, a small leakage rate from can produce a high contamination fraction. The Le Phare code run for this simulation does not incorporate a magnitude prior for the photo-; doing so might reduce the size of this island.
A second high-contamination region is , . Again the contamination rate is high because the target-bin density is much lower than the source-bin density.
The right-hand panel in Figure 2 shows evaluated using Eq. (21) for this case of catastrophic errors. We calculate the significance of the bias after marginalization of the cosmology onto the plane. We simplify by considering the bias arising from contamination in a single bin. This is
Again the inclusion of the factor defines a that is invariant under rebinning. The interpretation of is as follows: if there is an “island” of outliers that spans a range of photo-z bins, and contains a fraction of the galaxies in these photo-z bins, then the 2d significance of the resultant bias will be
Figure 2 has been scaled by 1000, so that it indicates the bias significance of a contamination rate . We desire to keep the bias well within the 68% confidence contour. The most severe constraint on would be to take the peak value and a very wide island, , in which case the criterion for small bias becomes
The contamination rate into any island of outliers must be known to 0.0015 or better to avoid significant cosmological bias. This conclusion is independent of the nominal outlier rate. The tolerance on outlier rate will scale with sky coverage as .
Iv Constraint via spectroscopic sampling
The most obvious way to determine the contamination rate is to conduct a complete spectroscopic redshift survey of galaxies in photo-z bin . It is of course essential that the spectra be of sufficient quality to determine redshifts even for the outliers in the sample.
Let us now estimate the total number of spectra required in order to keep the total bias below some desired threshold. We will assume that each redshift drawn from the spectroscopic survey is statistically independent. In this case the distribution of , the number of galaxies in photo-z bin that have spectro-z in bin , will be described by a multinomial distribution. When the outlier rates are small, the number of spectra in each outlier bin tend toward independent Poisson distributions.
We would like to relate the contamination uncertainties to the required number of galaxy spectra. Let be the number of spectra drawn from the photometric redshift bin so that . In this case and the variance of the contamination estimate is
Since the Poisson errors between different outlier bins are uncorrelated, the expected bias significance becomes
We would like to quote a total number of spectroscopic redshifts rather than the number per photo-z slice ( above) in order to make our findings more transparent. We consider two cases: first, a slitless or untargeted spectroscopic survey will obtain redshifts in proportion to the number density of source galaxies in each redshift bin: . Then we will consider a targeted survey, in which the number can be chosen bin-by-bin to produce the minimal bias for given total .
iv.1 Untargeted spectroscopic survey
In the untargeted case, and the condition becomes (from Eq. (29))
Figure 3 plots the summand of this expression in the plane. The required is hence the sum over values in this plane (note that we omit the bins near the diagonal that do not meet the “outlier” definition). We find that requires , and in the full 8-D parameter space (that is, considering for the 8-dimensional parameter ellipsoid), we need .
These requirements are daunting, potentially larger than the that are needed to constrain the core of the photo-z distribution as determined by Ma and Bernstein (2008). Note however that the requirement is strongly driven by the region , . This is because contamination rates are high here in the nominal photo-z distribution (Figure 2), and these bins are sparsely populated (small ), meaning that many spectra must be taken in order to accumulate a strong enough constraint on these contamination coefficients.
This suggests a strategy of omitting the galaxies from the tomography analysis entirely. Omitting from the sum (30) produces a much relaxed requirement: for , we need , a 30-fold reduction. This strategy would eliminate of the source galaxies in the SNAP model, and reduce the dark-energy constraint power by 18%, as mesured by the DETF figure of merit. This would be an acceptable strategy to reduce the outlier bias if one were unable to eliminate the high- island of outliers by refinements to the photo-z methodology.
iv.2 Targeted spectroscopic survey
If we wish to minimize in Eq. (29) for a given total , a simple optimization yields
Optimized targeting reduces the requirement for to be . Eliminating the sources reduces the requirement sixfold, .
Note that the targeted redshift requires times fewer calibration redshifts than the untargeted survey, if we are using the full source redshift range, but only times smaller if we restrict in the lensing analysis.
iv.3 Scaling and Robustness
The required to reduce outlier-rate biases to insignificance scales with the sky coverage and the mean outlier rate as
when most of the information is coming from shear tomography, and the depth of the survey is held fixed. We have verified that varies little as the number of tomography bins grows () and the information content of the tomography saturates. The two bias theorems imply that the required will drop if we add additional unbiased prior information, or if we marginalize down to a single dark-energy parameter.
More precisely, we find that the ratio (featuring the well known “figure of merit” (Huterer and Turner, 2001; Albrecht et al., 2006)) is roughly the same with several alternative survey specifications we consider (and is exactly the same if only the sky coverage is varied).
We have used two independent codes to verify the robustness of results to the myriad of assumptions made and check for the presence of unwanted numerical artifacts. The two codes agree to roughly a factor of two in , which is satisfactory given the differences between the implementations, e.g. whether curvature and/or neutrino masses are free to vary, and whether the fiducial redshift distribution is smoothed over the cosmic variance in the simulated catalog.
iv.4 Correlated outlier errors and incompleteness
So far we have considered the case where bin-to-bin fluctuations in contamination are uncorrelated. Contamination-rate errors that are correlated from bin to bin might arise if the spectroscopic survey systematically misses outliers in certain redshifts islands (or if the spectroscopy is not done at all!). We can set a specification on the maximum allowable systematic contamination rate error in an island of photo-z width using Eq. (26). Our previous results for the canonical survey show that the contamination rate in the island should be known to . In other words a spectroscopic-redshift failure rate of only 0.15% in some range of can cause a significant cosmology bias if all of these missed redshift are outliers in a particular island. A 99.9% success rate has rarely if ever been achieved in a spectroscopic redshift survey.
V Constraining outlier rates using galaxy correlations
The above requirements on and completeness may be too expensive to accomplish, particularly for fainter galaxies. We now examine the possibility that one could make use of a spectroscopic galaxy sample that does not fairly sample the photo-z galaxies Newman (2008). The idea is to cross-correlate a photo-z sample at nominal redshift with a spectroscopic sample known to be confined to a distinct bin . The amplitude of this cross-correlation will tell us something about the contamination rate , since there is no intrinsic correlation between the galaxy densities at the two disparate redshifts.
Newman Newman (2008) calculates the errors in this estimate that would be induced by shot noise in the sample (for a somewhat related work, see Schneider et al. (2006)). Here we assume that statistical errors will be negligible and attend to two systematic errors that will arise.
v.1 Outlier bias and correlation coefficients
First, we define , to be the fluctuations in sky density of the projected distributions of the spectroscopic-survey galaxies in bin and the photometric-redshift galaxies in bin . We set to be the fractional fluctuations in the projected mass density in redshift bin . The bias is defined by . The amplitude of the measured cross-correlation between the angular distributions and of the spectroscopic and photometric samples can be written as
where is the bias of the outlier population, and is the correlation coefficient between the density fluctuations of the spectroscopic and the outlier populations. The outlier population are those minority of sources in photo-z bin that have spectroscopic redshifts in bin .
In a large survey, shot noise in and in might become small, and could be determined to good accuracy through study of the redshift-space power spectrum of the spectroscopic targets. It will be difficult, however, to discern because the angular correlation signal of the outlier population is overwhelmed by the correlations of the galaxies in the “core” of the photo-z error distribution (i.e. photo-zs which have not been catastrophically misestimated). The correlation coefficient also has no alternative observable signature we have identified. Hence there will be a systematic-error floor on arising from the finite a priori knowledge of the product .
v.2 Lensing Magnification
The second complication to the cross-correlation method is that gravitational lensing magnification bias will induce a correlation between the spectroscopic and photometric samples even if there is no contamination. Let us assume that the spectroscopic sample is in the foreground of the photo-z “core”; a similar analysis can be done when using cross-correlation to search for contamination by background galaxies. There are two types of magnification-induced correlations. Following the notation of Hirata and Seljak (2004), there is a “GG” correlation, in which both the spectro and photo galaxy samples are lensed by mass fluctuations in the foreground of both. Then there is a “GI” effect, in which the mass associated with the fluctuations in the foreground sample induces magnification bias on the background sample.
The GI correlation is as follows: Let the spectroscopic bin span a range in comoving radial distance. The matter fluctuations induce a lensing convergence on the photo-z bin at of
where is the comoving matter density, and are the comoving angular diameter distances to and , and we have assumed a flat Universe.
The lensing magnification will induce apparent density fluctuations in the background sample as
where is a magnification bias factor for the galaxies in the photo-z bin. For instance if the selection criteria for the bin were a simple flux limit, and the intrinsic flux distribution were a power law , then we would have . In general will be of order unity.
The foreground galaxy distribution has a correlation coefficient with the mass , hence a covariance between populations results:
(we have ignored shot noise in the galaxy auto-correlation). This lensing contamination will have to be subtracted from in order to extract the information on contamination . Even if all the cosmological factors are well determined, the magnification coefficient will have to be empirically estimated. Finite accuracy in this estimate will increase .
The GG correlation scales as follows: let be the convergence induced on the foreground (spectroscopic) sample by mass at . This produces density fluctuations . This mass induces convergence on the background (photo-z) source population, where is an integral involving the distributions of foreground mass which must satisfy . Not concerning ourselves with details, we take The induced angular correlation will be
Typical RMS values are 0.01–0.02 at cosmological distances. The GG lensing correlation must be removed from the signal to retrieve the contamination fraction, and again even if there is no shot noise and all distances and lensing amplitudes are known perfectly, the values of and will only be known to finite precision.
v.3 Estimate of systematic errors
Summing the GG, GI, and intrinsic contributions, the cross-correlation between spectroscopic and photometric samples is
All of the right-hand quantities are potentially well measured from the survey data itself or from other cosmological probes, except the outlier covariance factor and the magnification coefficients and . Uncertainties in the a priori assumed values of these factors will propagate into the contamination coefficient as
Here we assume , .
Earlier we showed that contamination into an outlier “island” should be known to to avoid significant parameter bias. Can such a small contamination be measured using the cross-correlation technique?
If the nominal outlier rate is , then we require a prior estimate of outlier bias/covariance accurate to . Little will be known about the outlier population besides its luminosity range, and the outliers may tend to be active galaxies or those with unusual spectra whose clustering properties might be deviant as well. We would consider a 10% prior knowledge on outlier bias to be optimistic but perhaps attainable.
For the second (GI) term, if we take the distance factors to be , and the outlier population to span a range , then the magnification bias coefficient must be known to an accuracy of . This accuracy in will be challenging to achieve. If the galaxy selection is by simple magnitude cut, then the slope of the counts yields and potentially could be measured to high precision. Weak lensing samples typically have more complex cuts and weightings, however, than a simple flux cutoff. Surface brightness, photo-z accuracy, and ellipticity errors are involved, making estimation of more difficult.
The third (GG) term places constraints on and that will generally be weaker than those from the GI term.
If the cross-correlation technique is to determine outlier contamination fractions to an accuracy that renders them harmless, then we will need to know the product of the outlier population to 0.1 or better, and also must know the magnification-bias coefficients of our populations to accuracy. This is true regardless of sample size, and these tolerances will scale as . The demands on also becomes more stringent linearly with the photo-z outlier rate.
We have not considered the possibility of extraneous angular correlations induced by dust correlated with the foreground galaxy sample, or by dust in front of both samples (B. Menard, private communication). This signal will be present to some degree, though may perhaps be diagnosed with color information.
In summary, while we have shown here that the cross-correlation technique proposed by Newman (2008) is sensitive to catastrophic redshift errors, we found that prospects of measuring these errors (that is, the contamination coefficients ) will be difficult using this technique alone.
Vi Discussion and Conclusion
In this paper we have considered the effects of the previously ignored catastrophic redshift errors — cases when the photometric redshift is grossly misestimated, i.e. when , and are represented by arbitrary “islands” in the plane. We developed a formalism, captured by Eqs. (13), that treats these islands as small “leakages” (or “contaminations”) and directly estimates their effect on bias in cosmological parameters. We then inverted the problem by estimating how many spectroscopic redshifts are required to control catastrophic errors at a level that makes them harmless for cosmology. In the process, we have proven two general-purpose theorems (in the Appendices): that the bias due to systematics always decreases or stays fixed if 1) (unbiased) prior information is added to the fiducial survey, or 2) we marginalize over one or more dimensions of the parameter space.
We found that, at face value, of order million redshifts are required in order not to bias the dark energy parameter measurements (that is, in order to lead to in the plane). However, the requirement becomes significantly (30 times) less stringent if we restrict the survey to redshift ; in that case, is only of order a few tens of thousands. Essentially, leakage of galaxies from lower redshift to is damaging since there are few galaxies at such high redshift and the relative bias in galaxy number is large. Therefore, using only galaxies with helps dramatically by lowering the required while degrading the dark energy (figure-of-merit) constraints by mere %.
We have studied two approaches for a spectroscopic survey: the untargeted one where the number of spectra at each redshift is proportional to the number of photometric galaxies (§IV.1) and the targeted one where the number of spectra is optimized to be minimal for a given degradation in cosmological parameters (§IV.2). For the case where galaxies with are dropped, the targeted survey gave only a modestly (%) smaller required .
We do not imply that these requirements to apply to all proposed surveys to high accuracy, although the required knowledge on catastrophe rates is robust. The calculation should be repeated with the fiducial photo-z outlier distribution, survey characteristics, and cosmological parameters of interest to a particular experiment.
Our work demonstrates for example that efforts to reduce the “island” of catastrophic mis-assignment from to , such as magnitude priors, could greatly reduce the required . Since (with being the mean rate of catastrophic contamination), it is clear that a photo-z survey with improved and wavelength coverage to reduce the total catastrophe rate will also require lower to calibrate these rates.
Another practical implication of these results is that the spectroscopic redshift surveys must be of very high completeness—99.9% if there is a possibility that all failures could be in an outlier island, but less if some fraction of the failures are known to be in the core of the error distribution.
If an outlier island is known to exist at a particular location, it may be possible to include the contamination as a free parameter in the data analysis and marginalize over its value. It is possible that self-calibration may reduce the bias in cosmological parameters. It is likely infeasible, however, to leave the values over the full plane as free parameters. We leave self-calibration of outlier rates for future work.
We have also studied whether the technique proposed by Newman Newman (2008), which correlates a photometric sample with a spectroscopic one, can be used to measure, and thus correct for, catastrophic redshift errors. The advantage of this approach is that the spectroscopic survey need not be a representative sampling of the photometric catalog. While we found that the cross-correlation technique is sensitive to catastrophic errors (specifically, the contamination coefficients ), the contamination coefficient is degenerate with the value of the bias and stochasticity of the outlier population. Furthermore there is a correlation induced by lensing magnification bias that spoofs the contamination signal. It will therefore be difficult to use the cross-correlation technique to constrain outlier rates to the requisite accuracy.
Overall, we are very optimistic that the catastrophic redshift errors can be controlled to the desired accuracy. We have identified a simple strategy that requires only of order 30,000 spectra out to for the calibration to be successful for a SNAP-type survey. Incidentally, this number of spectra required for the catastrophic errors is of the same order of magnitude as that required for the non-catastrophic, “core” errors Ma et al. (2005); Huterer et al. (2006); Ma and Bernstein (2008). Total spectroscopic requirements of a survey will be based on the greater of requirements of these two error regimes.
GB is supported by grant AST-0607667 from the NSF, DOE grant DE-FG02-95ER40893, and NASA grant BEFS-04-0014-0018. DH is supported by the DOE OJI grant under contract DE-FG02-95ER40899, NSF under contract AST-0807564, and NASA under contract NNX09AC89G.
Appendix A Effect of unbiased priors on bias significance
Will a bias get worse or better (more or less significant) when additional unbiased prior information is added to the likelihood? It is intuitive that biases should decrease when unbiased information is added. However for some new non-negative-definite prior Fisher matrix , meaning that the statistical errors also shrink. So which effect wins out? We prove here that addition of unbiased prior information cannot increase the significance of parameter bias.
The proof is straightforward: Eq. (9) gives the significance of a bias in terms of the original positive-definite Fisher matrix and the vector . If is a non-negative-definite prior, then the change significance of the bias is
This quantity cannot be positive. If it were, then there would some and a positive-definite matrix such that
If is non-negative definite, this situation cannot occur. We hence conclude that the of some bias is always reduced (or stays the same) by addition of an unbiased prior.
Appendix B Effect of marginalization on bias significance
Second we can ask: If we calculate the significance of a bias induced over a parameter space, then marginalize away parameter vector to leave parameter vector , how might the significance differ in the smaller space? We show that in the Gaussian limit, marginalization always reduces (or leaves unchanged) the assigned to the bias, although the per DOF may increase. To see this: first we note that marginalization over does not change the biases in the parameters if the distribution is Gaussian. So the bias in is simply a projection matrix times : . The after marginalization down to the space is determined by the marginalized Fisher matrix, . So we have
The equivalence in (45) can be derived from manipulation of the common expression for the inverse of a matrix decomposed into an array of submatrices. Because and its inverse must be non-negative-definite, the last term is negative, so we are assured that . Equality is, however, easily obtained, for example if there is no bias in the parameters. We thus know that can potentially increase.
- Bartelmann and Schneider (2001) M. Bartelmann and P. Schneider, Phys. Rept. 340, 291 (2001), eprint astro-ph/9912508.
- Huterer (2002) D. Huterer, Phys. Rev. D65, 063001 (2002), eprint astro-ph/0106399.
- Munshi et al. (2008) D. Munshi, P. Valageas, L. Van Waerbeke, and A. Heavens, Phys. Rept. 462, 67 (2008), eprint astro-ph/0612667.
- Hoekstra and Jain (2008) H. Hoekstra and B. Jain (2008), eprint arXiv:0805.0139.
- Albrecht et al. (2009) A. Albrecht et al. (2009), eprint arXiv:0901.0721.
- Oyaizu et al. (2007) H. Oyaizu, M. Lima, C. E. Cunha, H. Lin, and J. Frieman (2007), eprint arXiv:0711.0962.
- Banerji et al. (2007) M. Banerji, F. B. Abdalla, O. Lahav, and H. Lin (2007), eprint arXiv:0711.1059.
- Lima et al. (2008) M. Lima et al., Mon. Not. Roy. Astron. Soc. 390, 118 (2008), eprint arXiv:0801.3822.
- Cunha et al. (2008) C. E. Cunha, M. Lima, H. Oyaizu, J. Frieman, and H. Lin (2008), eprint arXiv:0810.2991.
- Abdalla et al. (2008) F. B. Abdalla, M. Banerji, O. Lahav, and V. Rashkov (2008), eprint arXiv:0812.3831.
- Ma et al. (2005) Z.-M. Ma, W. Hu, and D. Huterer, Astrophys. J. 636, 21 (2005), eprint astro-ph/0506614.
- Huterer et al. (2006) D. Huterer, M. Takada, G. Bernstein, and B. Jain, Mon. Not. Roy. Astron. Soc. 366, 101 (2006), eprint astro-ph/0506030.
- Kitching et al. (2008a) T. D. Kitching, A. N. Taylor, and A. F. Heavens (2008a), eprint arXiv:801.3270.
- Ma and Bernstein (2008) Z. Ma and G. Bernstein, Astrophys. J. 682, 39 (2008), eprint arXiv:0712.1562.
- Huterer and Takada (2005) D. Huterer and M. Takada, Astropart. Phys. 23, 369 (2005), eprint astro-ph/0412142.
- Heitmann et al. (2005) K. Heitmann, P. M. Ricker, M. S. Warren, and S. Habib, Astrophys. J. Suppl. 160, 28 (2005), eprint astro-ph/0411795.
- Heitmann et al. (2008) K. Heitmann, M. White, C. Wagner, S. Habib, and D. Higdon (2008), eprint arXiv:0812.1052.
- Zentner et al. (2008) A. R. Zentner, D. H. Rudd, and W. Hu, Phys. Rev. D77, 043507 (2008), eprint arXiv:0709.4029.
- Takada and Jain (2008) M. Takada and B. Jain (2008), eprint arXiv:0810.4170.
- White (2004) M. J. White, Astropart. Phys. 22, 211 (2004), eprint astro-ph/0405593.
- Hagan et al. (2005) B. Hagan, C.-P. Ma, and A. V. Kravtsov, Astrophys. J. 633, 537 (2005), eprint astro-ph/0504557.
- Zhan and Knox (2004) H. Zhan and L. Knox, Astrophys. J. 616, L75 (2004), eprint astro-ph/0409198.
- Huterer and White (2005) D. Huterer and M. J. White, Phys. Rev. D72, 043002 (2005), eprint astro-ph/0501451.
- Shapiro and Cooray (2006) C. Shapiro and A. Cooray, JCAP 0603, 007 (2006), eprint astro-ph/0601226.
- Shapiro (2008) C. Shapiro (2008), eprint arXiv:0812.0769.
- Rudd et al. (2007) D. H. Rudd, A. R. Zentner, and A. V. Kravtsov (2007), eprint astro-ph/0703741.
- Kitching et al. (2008b) T. D. Kitching, A. Amara, F. B. Abdalla, B. Joachimi, and A. Refregier (2008b), eprint arXiv:0812.1966.
- Joachimi and Schneider (2008) B. Joachimi and P. Schneider (2008), eprint arXiv:0804.2292.
- Bernstein (2008) G. M. Bernstein (2008), eprint arXiv:0808.3400.
- Guzik and Bernstein (2005) J. Guzik and G. Bernstein, Phys. Rev. D72, 043503 (2005), eprint astro-ph/0507546.
- Heymans et al. (2006) C. Heymans et al., Mon. Not. Roy. Astron. Soc. 368, 1323 (2006), eprint astro-ph/0506112.
- Massey et al. (2007) R. Massey et al., Mon. Not. Roy. Astron. Soc. 376, 13 (2007), eprint astro-ph/0608643.
- Paulin-Henriksson et al. (2007) S. Paulin-Henriksson, A. Amara, L. Voigt, A. Refregier, and S. L. Bridle (2007), eprint arXiv:0711.4886.
- Amara and Refregier (2007) A. Amara and A. Refregier (2007), eprint arXiv:0710.5171.
- Stabenau et al. (2007) H. F. Stabenau, B. Jain, G. Bernstein, and M. Lampton (2007), eprint arXiv:0710.3355.
- Jarvis and Jain (2004) M. Jarvis and B. Jain (2004), eprint astro-ph/0412234.
- Takada and Bridle (2007) M. Takada and S. Bridle, New J. Phys. 9, 446 (2007), eprint arXiv:705.0163.
- Ma et al. (2008) Z. Ma, G. Bernstein, A. Weinstein, and M. Sholl (2008), eprint arXiv:809.2954.
- Heavens et al. (2000) A. Heavens, A. Refregier, and C. Heymans, Mon. Not. Roy. Astron. Soc. 319, 649 (2000), eprint astro-ph/0005269.
- Croft and Metzler (2000) R. A. C. Croft and C. A. Metzler (2000), eprint astro-ph/0005384.
- Crittenden et al. (2001) R. G. Crittenden, P. Natarajan, U.-L. Pen, and T. Theuns, Astrophys. J. 559, 552 (2001), eprint astro-ph/0009052.
- Mackey et al. (2002) J. Mackey, M. J. White, and M. Kamionkowski, Mon. Not. Roy. Astron. Soc. 332, 788 (2002), eprint astro-ph/0106364.
- Jing (2002) Y. P. Jing, Mon. Not. Roy. Astron. Soc. 335, L89 (2002), eprint astro-ph/0206098.
- Heymans and Heavens (2003) C. Heymans and A. Heavens, Mon. Not. Roy. Astron. Soc. 339, 711 (2003), eprint astro-ph/0208220.
- King and Schneider (2003) L. J. King and P. Schneider, Astron. Astrophys. 398, 23 (2003), eprint astro-ph/0209474.
- Hirata et al. (2004) C. M. Hirata et al. (SDSS), Mon. Not. Roy. Astron. Soc. 353, 529 (2004), eprint astro-ph/0403255.
- Mandelbaum et al. (2006) R. Mandelbaum, C. M. Hirata, M. Ishak, U. Seljak, and J. Brinkmann, Mon. Not. Roy. Astron. Soc. 367, 611 (2006), eprint astro-ph/0509026.
- Bridle and Abdalla (2007) S. Bridle and F. B. Abdalla, Astrophys. J. 655, L1 (2007), eprint astro-ph/0608002.
- Bridle and King (2007) S. Bridle and L. King, New J. Phys. 9, 444 (2007), eprint arXiv:705.0166.
- Hirata and Seljak (2004) C. M. Hirata and U. Seljak, Phys. Rev. D70, 063526 (2004), eprint astro-ph/0406275.
- Newman (2008) J. A. Newman (2008), eprint arXiv:0805.1409.
- Knox et al. (1998) L. Knox, R. Scoccimarro, and S. Dodelson, Phys. Rev. Lett. 81, 2004 (1998), eprint astro-ph/9805012.
- Huterer and Turner (2001) D. Huterer and M. S. Turner, Phys. Rev. D64, 123527 (2001), eprint astro-ph/0012510.
- Tegmark et al. (1997) M. Tegmark, A. Taylor, and A. Heavens, Astrophys. J. 480, 22 (1997), eprint astro-ph/9603021.
- Aldering et al. (2004) G. Aldering et al. (SNAP) (2004), eprint astro-ph/0405232.
- Albrecht et al. (2006) A. Albrecht, G. Bernstein, R. Cahn, W. L. Freedman, J. Hewitt, W. Hu, J. Huth, M. Kamionkowski, E. W. Kolb, L. Knox, et al. (2006), eprint astro-ph/0609591.
- Komatsu et al. (2008) E. Komatsu et al. (WMAP) (2008), eprint arXiv:0803.0547.
- Jouvel et al. (2009) S. Jouvel et al. (2009), eprint arXiv:0902.0625.
- Dahlen et al. (2008) T. Dahlen, B. Mobasher, S. Jouvel, J.-P. Kneib, O. Ilbert, S. Arnouts, G. Bernstein, and J. Rhodes, Astrophys. J. 136, 1361 (2008), eprint 0710.5532.
- Schneider et al. (2006) M. Schneider, L. Knox, H. Zhan, and A. Connolly, Astrophys. J. 651, 14 (2006), eprint astro-ph/0606098.