Guided patch-wise nonlocal SAR despeckling
We propose a new method for SAR image despeckling which leverages information drawn from co-registered optical imagery. Filtering is performed by plain patch-wise nonlocal means, operating exclusively on SAR data. However, the filtering weights are computed by taking into account also the optical guide, which is much cleaner than the SAR data, and hence more discriminative. To avoid injecting optical-domain information into the filtered image, a SAR-domain statistical test is preliminarily performed to reject right away any risky predictor. Experiments on two SAR-optical datasets prove the proposed method to suppress very effectively the speckle, preserving structural details, and without introducing visible filtering artifacts. Overall, the proposed method compares favourably with all state-of-the-art despeckling filters, and also with our own previous optical-guided filter.
Remote sensing imagery represents nowadays an invaluable source of information for the analysis of the Earth’s state. Among the many types of sensors, synthetic aperture radar (SAR) systems are especially precious, since they observe features that are not captured by optical sensors, and acquire data on the target scene independently of the weather and lighting conditions. SAR images are routinely exploited in many key applicative fields, like, for example, the analysis of the environment [1, 2] or urban planning [3, 4]. Unfortunately, they are severely impaired by the presence of speckle, caused by the coherent nature of the scattering phenomena. Speckle noise strongly degrades the quality of SAR images, thereby affecting the performance of subsequent automated tasks, like segmentation [5, 6] or classification [7, 8], and causing problems even to human interpreters.
To tackle this problem, the scientific community has produced a major research effort on SAR despeckling in the last decades . A large number of methods have been proposed, working in the spatial domain [10, 11, 12], using wavelet transform [13, 14, 15], sparse representations [16, 17, 18], variational approaches [19, 20, 21] and, very recently, deep learning [22, 23]. As of today, however, the most successful approach to SAR despeckling appears to be the nonlocal paradigm [24, 25], which has produced powerful and widespread methods such as PPB  and SAR-BM3D . Key to this success is the ability to recognize “similar” pixels, that is, pixels characterized by the same underlying signal. This allows, for each target pixel, to single out its best predictors in the whole image, and use them to perform reliable estimation. Therefore, in nonlocal filtering the main issue is how to find such good predictors. This problem is usually addressed by using suitable patch-based similarity measures , leveraging the contextual information conveyed by patch-wise analysis. However, speckle impacts also this process, reducing the ability to find good predictors and, eventually, impairing the filtering performance.
We highlight the limits of the nonlocal approach for SAR despeckling with the help of Fig.1 which shows a single-look SAR image (top-left) together with an optical image of the same scene co-registered with it (top-right). The SAR image carries precious information on the scene which is not available in the optical bands. Nevertheless the signal of interest is overwhelmed by speckle noise: the scene structure is hardly visible, and the boundaries between different land covers can be barely detected. This impacts heavily on nonlocal filtering, preventing the correct localization of the best predictors. As an example, for the target patch marked by a green box in the figure, the selected predictors (red boxes) are dominated by speckle and hence spread all around the target. With such a poor selection of predictors, plain nonlocal filtering can only provide unsatisfactory results (bottom-left). On the optical image, however, which is virtually noiseless, finding good predictors is very easy. Now the selected patches (red boxes) exhibit clearly a signal content similar to the target, and are very likely the best possible predictors. Based on these observations, we decided to leverage optical data to improve nonlocal SAR despeckling, obtaining promising results, as shown in the figure (bottom-right).
Of course, data fusion is nothing new in remote sensing. The large abundance of imagery from sensors of different types offers a wealth of opportunities [29, 30, 31, 32] that can be exploited for many remote sensing applications [33, 34, 35, 36].
However, using optical data to support SAR despeckling requires great care. The example of Fig.2 provides some insight into this point. In fact, despite the careful co-registration, and the obvious correspondence of the observed scene, important differences exist between optical and SAR images. These regard not only the signal amplitudes, which show no obvious relationship due to the completely different imaging mechanisms, but also the signal structure, especially in the presence of man-made objects and regions characterized by a significant orography (not present in this example). Therefore, while optical data can be certainly helpful to guide the despeckling process, there is the risk to inject alien information into the filtered SAR image, generating annoying artifacts.
Based on these concepts, in  we proposed a nonlocal despeckling technique for SAR images, driven by co-registered optical images. Within the frame of a generalized bilateral filtering, optical data were used to properly weight predictor pixels for the current target. To prevent the injection of alien optical structures, the SAR image was preliminary classified, and the optical guide was used only in low-activity areas, switching to a full SAR-domain technique in high-activity areas.
In this work (a preliminary version of which was presented in ) we keep pursuing the same general approach but propose a much more effective and simple optical-guided SAR despeckling method. We replace the pixel-wise bilateral filter of  with patch-wise nonlocal means. Moreover, to avoid optical-related artifacts, we use a simple statistical test which discards unreliable predictors on the fly, during the filtering process. Extensive experiments on real-world imagery prove the potential of the proposed method, also in comparison with state-of-the-art reference methods and with our own previous proposal. In addition, by avoiding the preliminary classification phase and the external complementary filter, the method is much faster and easy to use than . Note that, with respect to our conference paper , we introduce here a reliability test which allows us to despeckle effectively also high-activity areas, keeping all available information and removing only bad predictors. Moreover, we perform a theoretical analysis of the proposed test, and carry out a much deeper experimental analysis of performance.
In the rest of the paper after recalling previous work (Section 2), we describe the proposed solution (Section 3), study the effect of key parameters on performance (Section 4), discuss experimental results (Section 5), and eventually draw conclusions (Section 6). The SAR-domain distance used in the reliability test is analyzed in Appendix A.
Ii Previous work
The optical-guided despeckling paradigm was first proposed in . It was observed, there, that a virtually noiseless optical image, co-registered with the target SAR image, can provide precious information to support the speckle removal process. Although SAR and optical images refer to completely different imaging mechanisms, and hence there is no relationship between their signal amplitudes, they share important structural information. A boundary between two fields, for example, keeps the same geometry in both the optical and the SAR image. This structural information can be exploited by means of guided filtering. However, one must also be aware that such a structural similarity does not always hold. This is the case of man-made areas, for example, characterized by intense double reflection lines in SAR images that have no correspondence in optical images. Therefore, care must be taken not to generate filtering artifacts due to the optical guide.
In  the problem was solved by introducing a preliminary soft classification phase. A high-level scheme of the method is shown in Fig.3. The single-look input SAR image is filtered twice: by means of a guided despeckling tool, leveraging the co-regirested optical image, and by means of a conventional SAR-domain despeckling filter. A soft classifier  distinguishes between low-activity and high-activity regions, the latter possibly related to man-made areas where optical and SAR geometries differ. The output image is then obtained as a linear combination of the two filtered images, with weights given by the continuous-valued classifier. In low-activity areas, the guided filter prevails, while the opposite happens in high-activity areas.
For the despeckling of high-activity areas we used SAR-BM3D [27, 40], which is known  to guarantee a good preservation of fine details and man-made structures. The guided filter for low-activity areas, instead, is a generalization of the bilateral filter . Assuming the usual multiplicative model, , with and the true and observed intensity values at pixel , and a Gamma random variable (RV) modeling the speckle, the estimated intensity is given by the weighted average of predictors drawn from a small neighborhood of :
The weight associated with the predictor at location is computed as
where indicates the vector-valued optical data, and is a normalizing constant. The weights depend both on spatial distance, , like in a ordinary Gaussian filter, and amplitude distances in the SAR domain, , and in the optical domain, . A simple Euclidean norm is used for the optical-domain distance, while in the SAR-domain we use the dissimilarity measure (loosely referred to as distance in the following) proposed in  for multiplicative noise. The weights and are set by preliminary experiments on training data. When a simple bilateral filter in the SAR domain is obtained. On the contrary, if , the weights depend only on the optical-domain distances. However, it is worth underlining that in the filtering procedure there is no leakage of optical data in the filtered output. The filtered value in Eq.(1) is a linear combination of exclusively SAR-domain original values, and optical data impact only on their weighting. Likewise, the soft classifiers uses only SAR data as input. Hence, the optical guide only helps locating predictors that are most similar to the target or, under a different perspective, de-emphasizing the contributes of predictors that are not really similar to the target despite their low spatial and SAR-domain distances.
Iii Proposed method
This work introduces two major improvements with respect to , consisting in
replacing the pixel-wise generalized bilateral filter with patch-wise nonlocal means;
using a reliability test to reject poor predictors on the fly and prevent structural leakages from optical data.
The resulting filter, besides providing a much better performance, is much simpler and easy to use, since we remove altogether the activity-based classifier, and do not need external filters to manage high-activity areas.
Iii-a Going patch-wise
In recent years, there has been a steady trend towards patch-based processing for SAR imagery . Patch-wise nonlocal means, in particular, is well known to significantly outperform the pixel-wise version. The key idea is to compute a large number of estimates of the same pixel, which are eventually aggregated to improve accuracy. This is obtained by applying the nonlocal weighted average to all pixels of a patch, not just its center.
Let us consider a target patch, }, where is an anchor pixel (for example, the patch center), and indicates the set of spatial offsets with respect to . Then, an estimate of the clean patch is obtained through a patch-wise nonlocal average
where the weights depend on a suitable patch-wise similarity measure. This is the same expression as in Eq.(1), except that it now involves all pixels in the target patch, namely
Since a pixel belongs to multiple target patches, it will be estimated several times, allowing for the eventual aggregation of all estimates.
The weights are now computed as
where and are suitable SAR-domain and optical-domain distances, is a parameter that balances their contribution, and is a parameter which determines how fast weights decay as a function of the distance, and hence impacts on the sharpness/smoothness of the filtered image. Note that, unlike in , the weights depend only on signal amplitudes, both SAR and optical, not on spatial distances.
For the SAR domain, we use a normalized and slightly modified version of the distance proposed in 
where is the patch size, and (described in more detail in Subsection III.C and in the Appendix) is the mean of the single-pixel distance under same-signal hypothesis. The normalization ensures that, in strictly homogeneous regions, predictor patches have unitary average distance from the target, say , with standard deviation . For the optical domain, instead, we consider the normalized Euclidean distance
with the number of bands.
Iii-B Discarding unreliable predictors
Patch-level processing allows for a much stronger noise suppression than pixel-level processing . Nonetheless, for our application, patch-level operations entail also some risks. Consider, for example, a low-activity target patch with some high-activity patches in its neighborhood, , maybe patches with double reflection lines. Since all patches in are averaged to estimate the clean target, the structures observed in the predictors will be reproduced in the estimate, with an attenuation that depends on optical-domain and SAR-domain distances. Hence, insufficient attenuation of high-activity patches will generate visible artifacts. This can easily happen when the SAR-domain distance is not too large (e.g. just a few double-reflection pixels in the predictor) and the optical-domain distance is not particularly discriminative. Fig.4 shows a clear example of this behavior. A few high-intensity pixels, included in the estimation of surrounding patches, produce disturbing artifacts in a large area of the filtered image.
Of course, to reduce such artifacts, one can modify the filter’s parameters, increasing the relative weight of SAR-domain vs. optical-domain distance. However, this would reduce the benefit of the optical guide in other areas where such artifacts do not occur. To cope with this problem, in  we decided to avoid patch-level processing altogether and to treat differently low-activity and high-activity areas. Here, we use a more effective solution. We keep using patch-wise processing but, for each target patch, carry out a preliminary test to single out unreliable predictors and exclude them altogether from the nonlocal average. That is, we perform the nonlocal average of Eq.(3) replacing with a new set such that
With this solution, we are free to select the filter parameters that optimize performance in low-activity areas. Moreover, by removing problematic patches beforehand, we can keep using patch-wise averages also in high-activity areas, with clear benefits in terms of speckle suppression. Only in the extreme case in which no predictor is reliable, maybe due to the presence of corner reflectors, the target patch is not filtered at all, which makes sense in this condition. However, since each pixel belongs to many patches, it is still likely that many individual pixels will be filtered anyway.
Of course, the success of this strategy depends on the discriminative power of the SAR-domain distance, and on a suitable selection of the threshold. These aspects are analyzed in the following subsection. In the example of Fig.4, however, it is already clear that this simple test impacts heavily on the quality of filtered image.
A further problem, besides filtering artifacts, is image oversmoothing, and the consequent loss of resolution. This effect is especially visible at the boundary between fields, where signal differences are small both in the optical and SAR domains. In this situation, the weights depend only mildly on the signal, and more strongly on the intense SAR-domain speckle, causing an incoherent averaging of patches and ultimately the loss of details. In Fig.4, for example, a small road between two fields goes completely lost. These losses can be reduced by limiting the maximum number of predictors to patches, with the search area size, choosing those with smallest optical-domain distance. Therefore, the new set , has cardinality
Thanks to this limitation, in homogeneous areas of the image many irrelevant patches are excluded from the average, emphasizing fine details that would be lost otherwise. Instead, in high-activity areas this limitation has no effect, since most predictors are already discarded by the SAR-domain test. Back to Fig.4, we see that this further limitation allows recovering the road as well as many other details, with no big loss in terms of speckle rejection. The final result of filtering shows both strong speckle suppression and good detail preservation, comparing favourably with state-of-the-art filters.
Iii-C SAR-domain distance and reliability test
Our filtering strategy founds heavily on the reliability test’s capacity of rejecting bad predictors. Therefore, it is worth investigating in more depth the SAR-domain distance, also to gain sensitivity on how to set the decision threshold .
Let us focus, for the time being, on the pixel-wise SAR-domain distance
This is not exactly the distance proposed in  because of the extra 2 at the denominator, which we include to ensure zero distance for .
Based on measured distances, we would like to identify pixels that have a similar signal component as the target, that is, . Of course, since the observed intensity, , depends also on the speckle, the distance depends on the speckle too. Let us consider two limiting cases, (i) speckle-free data, and (ii) homogeneous signal.
In the first case, , the distance depends only on the ratio of signal intensities , that is
In Fig.5(left) we plot as a function of the signal intensity ratio. For close to 1, the distance is rather flat around the minimum, zero, and begins growing linearly (in semilog axes) only for much larger/smaller values. Therefore, it is not much discriminative for samples with relatively close intensity.
On the other hand, with homogeneous signal, , the distance is the random variable
with and independent Gamma distributed RV’s, with unit mean, and shape parameter equal to the number of looks of the image, . In Appendix A we compute the probability density function (pdf) of as a function of . Fig.5(right) shows two such pdf’s, for =1 and =16. In the relatively uninteresting case of =16 (low noise) the pdf is highly peaked around 0, that is, homogeneous pixels do have small SAR-domain distances. However, in the more interesting and relevant case of (single-look images) the pdf is much flatter, and has a long non-negligible tail.
The plots of Fig.5 can be used to gain insight into the discriminative power of the SAR-domain distance. As an example, for , the speckle-free distance is about 0.2. This should allow one to recognize that these pixels are non-homogeneous. However, for homogeneous pixels, , the single-look distance exceeds 0.2 with probability 0.4. This means that a 0.2 distance provides little or no information on the quality of the predictor.
For a more precise analysis, the pdf of the distance for arbitrary signal intensity ratio is necessary. Lacking the closed-form pdf, we resort to MonteCarlo simulation. Fig.6(left) shows the empirical pdf’s for (solid red) and (dashed blue). As expected, they largely overlap, indicating that no reliable discrimination is possible, and justifying, in hindsight, the classification-based solution proposed in . In this work, however, we use patch-wise distances obtained by summing pixel-wise distances over many samples. Assuming, for the sake of simplicity, two patches with constant signal intensity ratio, that is
the patch-wise distance becomes the sum of independent identically distributed RV’s, well approximated by a Gaussian law. Fig.6(right) shows again the estimated pdf’s for and when 1010-pixel patches are considered. As expected, same-signal and different-signal patches have now well separated pdf’s, suggesting that a test based on patch-wise distances can provide reliable indications.
Note that, for good predictors, that is, patches similar to the target, the constant-ratio hypothesis with is quite reasonable. Under this hypothesis, mean and variance of the pixel-wise distance, and , are computed in Appendix A, and hence the approximating Gaussian curve is perfectly known. Therefore, we can set the threshold test with a Neyman-Pearson criterion, deciding in advance which fraction of the good predictors can be lost to ensure rejection of virtually all the bad ones.
Iv Exploring key parameters
Like all numerical algorithms, the proposed method depends on several key parameters which impact significantly on performance. Some of them are related to nonlocal means and are set based on literature results, like search area, 3939, and patch size, 88. Others are set based on preliminary experiments to meet all contrasting quality requirements. Of these latter, decay and balance parameters, and , have a rather obvious meaning and need no special analysis. Instead, it is worth gaining more insight into how the test threshold, , and the maximum number of predictors, , impact on performance. To this end we carry out a visual analysis on the T1 clip, shown in Fig.10, which displays both agricultural and urban areas, and hence allows for a study of all features of interest.
Fig.7 refers to a 512192 strip of the T1 image, thin enough to allow for a simple visual comparison of results. From left to right we show the original SAR strip, and 5 filtered versions obtained with threshold in , where is the standard deviation of the normalized SAR distance for homogeneous signal. With the test does not operate, and bad predictors contribute to the estimate, causing a severe impairment of the filtered image. Major artifacts are visible in urban areas, due to strong reflectors, but problems arise also in other areas, for example the dark road in the top almost vanishes with filtering. The test on SAR distances solves most of these problems. Even a large threshold, , which excludes only a tiny fraction of good predictors, removes most bad ones. Lowering the threshold to , a more selective test is obtained, and a further quality improvement is observed. With further smaller values, however, a large part of good predictors is rejected together with the bad ones, reducing the speckle rejection ability of the filter in homogeneous areas, and causing the appearance of residual noise.
Fig.8 provides further insight into how the test impacts on the number of predictors. Besides the original T1 clip, on the left, we show a false-color map of the number of patches that pass the test with the selected threshold , going from 1 (dark blue), to the maximum =3939 (intense red). It clearly appears that in urban areas only a few patches survive the test, those that are structurally similar to the target, thus avoiding filtering artifacts. Instead, a very large number of patches pass the test in homogeneous regions, ensuring good speckle rejection.
Iv-B Maximum number of predictors
To visualize the impact of the maximum number of predictors, , on filtering quality, we use a thin horizontal strip of the T1 image which contains mostly a mosaic of homogeneous regions and some roads. In fact, in urban areas, the number of predictors is already limited by the SAR-domain test and no further constraint is needed. Fig.9 shows, from top to bottom, the optical guide, the original SAR data, and the output of the filter with =1521 (all the patches), 256, and 64. In the first case, the filtered image appears oversmoothed. For example, the thin white road on the right is lost, and the boundary between the fields on the left is much smeared. These structures and others are recovered in the second case, =256. On the down side, some textures appear in the fields which cannot be spotted in the original SAR image, and may raise the suspect of incorrect behavior. However, one must remember that only original SAR data are averaged. The optical guide can only give priority to some patches over others and, so to speak, “combing” the data in a certain direction, but the data are all SAR. This is confirmed by the thin white road on the right. A careful inspection reveals clear traces of the road in the original data, which are reinforced by the guided filtering. With these considerations, also the third case, with =64, is probably acceptable, however we prefer to ensure a stronger speckle rejection and a smoother output and use =256 as default parameter.
Based on these experiments, we select eventually a configuration with parameters and , ensuring sharp details and good speckle rejection. However, we also consider a more conservative configuration, with and , which outputs somewhat smoother images. Of course, for other datasets, these parameters may require some fine tuning, also due to different dynamics. With our COSMO dataset, for example, we only needed to multiply all ’s by a factor 4.
V Experimental analysis
In this Section, we discuss the results of several experiments on two real-world datasets, where the proposed method is compared with several state-of-the-art references. In the following, we describe datasets, quality assessment criteria, reference methods and, finally, numerical and visual results.
We designed two SAR/optical datasets, called for short T-SAR and COSMO, in the following.
The T-SAR dataset includes four 512512-pixel clips extracted from a large single-look TerraSAR-X image (courtesy of ©Infoterra GmbH) acquired over Rosenheim (D) in spotlight mode with single HH polarization on January 27th, 2008. For this image, we do not have an optical reference of comparably high quality, therefore we resort to the freely available RGB optical images provided by Google Earth Pro. This ”barebone” setting is of particular interest for us, since we want the proposed method to be adopted with minimal effort also by users with limited budget. In the Google Earth repository, we found, as of october 2018, several images of the Rosenheim region, spanning from 2002 to 2017, with the closest one acquired on December 31st, 2009, about two years after the target.
In Fig.10 we show the four SAR-optical pairs, called T1, T2, T3, and T4, from now on. Each optical image was co-registered with the corresponding SAR image, used as master. The available geocoding information was used for a first raw alignment, refined manually based on a few prominent keypoints. On the average, the co-registration process took about 5 minutes per image. Despite the large temporal gap, all pairs match quite well. On the other hand, some mismatches in the test set are welcome, because good optical references may be unavailable for several reasons, like the presence of clouds, and the proposed method must provide sensible results also in the presence of mismatches or missing data.
The four SAR-optical pairs of the T-SAR dataset are available online, at www.grip.unina.it, to allow other researchers to experiment with the very same data. Moreover, to ensure the reproducibility of our research, the executable code of the proposed method is also available online at the same address.
The COSMO dataset includes four 512512-pixel clips extracted from a large COSMO-SkyMed image acquired over the province of Caserta (I) on July 18th, 2011. In this case, as optical guide we can rely on a GeoEye-1 multispectral image, acquired over the same region on July 26th, 2010 (both images curtesy of the Italian Aerospace Research Center). The same co-registration process as for T-SAR was used. In Fig.11 we show the four SAR-optical pairs, called C1, C2, C3, and C4, from now on, with a suitable RGB rendering of the 4-band Geoeye-1 images.
V-B Quality assessment criteria
Despite intense research on SAR despeckling, quality assessment is still an open problem . A good filter should guarantee both effective suppression of speckle and faithful preservation of informative image details, such as region boundaries or man-made structures. These are contrasting requirements, as intense image smoothing, which ensures speckle removal, tends to smear all relevant high-frequency details. Therefore, these two aspects should be analyzed individually.
To this end, we consider here two objective indicators, i) the equivalent number of looks (ENL), and ii) the ratio image structuredness (RIS). In any case, we leave the last word to the visual inspection of filtered and ratio images.
The ENL is the squared ratio between the mean and the standard deviation of the signal computed over a homogeneous region of the image. In Fig.10 and Fig.11 the regions used to compute the ENL are shown enclosed in a white box. Before filtering, the ENL approximates the number of looks of the SAR image. After despeckling, instead, it should be as large as possible, as the filtered image is supposed to approach a constant.
The RIS is computed on the ratio image, that is, the ratio between original and filtered images. With ideal filtering and fully developed speckle, the ratio image becomes a field of i.i.d. speckle samples. Imperfect filtering, instead, causes the leakage of image structures in the ratio, which looses its i.i.d. nature. Hence, neighboring pixels tend to be more similar to one another. We measure this tendency through a suitable function of their joint pdf
and compare it with the reference value, , computed on the product of the marginals, , obtaining eventually the RIS index defined as
V-C Reference methods
We compare the proposed method with a few selected reference methods, chosen because of their diffusion in the community and good performance. More precisely, we include
PPB : a patch-based iterative nonlocal filter, where the output is given by a weighted maximum likelihood estimator with data-driven weights;
FANS : a faster and spatially-adaptive version of SAR-BM3D, which ensures a better speckle rejection in homogeneous areas;
G-BF : our previous optical-guided pixel-wise despeckling method, based on generalized bilateral filter.
PPB and SAR-BM3D, in particular, represent sort of two limiting cases, with PPB ensuring very strong speckle rejection in homogeneous areas at the cost of some smearing of high-frequency details, and SAR-BM3D much better at preserving details but less effective otherwise.
For all methods, we selected parameters as suggested in the original papers or, lacking clear indications, such to optimize filtering quality. For the proposed method, we consider the two configurations described at the end of Section IV, resulting in two versions, a first one (sharp) which makes a more aggressive use of the optical guide, and a second one (smooth) more conservative.
Tab.I and Tab.II report the ENL results for all T-SAR and COSMO clips, respectively, with the average values in the last row. Despite the obvious variations from clip to clip, clear indications emerge from these data, well summarized by the average values, which are almost identical for the two datasets. The proposed method ensures a very strong rejection of speckle, with average ENL beyond 100 for the first version and over 600 for the second one. Among reference methods, only PPB provides comparable results, with ENL around 250, while enhanced Lee and BM3D even remain below 10. Our previous optical-guided filter also provides a good ENL, about 100 on the average.
|optical guide||single-look SAR data||FANS|
|G-BF||G-NLM sharp||G-NLM smooth|
As already said, however, these results may be misleading if not accompanied by visual analysis. Therefore, we now show and comment the output of all filters for some relevant details selected from our datasets. To allow the reader to conduct a more thorough inspection, we publish online, at www.grip.unina.it, the results of all methods under comparison on all clips of our datasets. Fig.12 shows a 256256 section of the C2 clip. Except for some buildings in the left part, the scene includes only fields and some thin roads; the region used to compute the ENL is in the upper-right corner. The visual inspection reveals a number of phenomena hardly captured by the ENL or other numerical measures. It confirms the limited speckle suppression of enhanced Lee and SAR-BM3D, but also the well-known detail preservation ability of the latter. FANS produces a smoother output, but many wavelet-related artifacts appear, which impair significantly the perceived quality. Likewise, PPB is very effective in removing speckle, but introduces “brushstroke” patterns and, what is worse, smears edges and buildings. Also our previous optical-guided filter generates some artifacts in smooth areas, and produces annoying halos of residual speckle in high-activity areas. The proposed optical-guided nonlocal means, in both versions, produces images of much better quality. While man-made structures are faithfully preserved (compare with the original SAR image) speckle is largely rejected in all homogeneous areas. In the “smooth” version, these areas become basically flat, while in the “sharp” version some subtle patterns emerge as a result of the SAR data “combing”. It remains to understand whether these are real structures, hidden in the SAR data and recovered through filtering, or else they come from the alien optical guide. Lacking a clean reference, no ultimate answer can be given. Nonetheless, some of these structures can be clearly spotted in the original SAR image, though overwhelmed by speckle. For example, the diagonal dark strip in the center, or the parallel thin strips above the central bright fork, which in fact are both captured also by enhanced Lee and SAR-BM3D. Therefore, a reasonable guideline could be to use the sharp version if the optical guide is temporally close to the SAR image, and the smooth version otherwise. In any case, it is clear that both versions of G-NLM work much better than G-BF, based itself on the use of an optical guide, since they ensure a much better suppression of speckle and do not introduce filtering artifacts.
|optical guide||single-look SAR data|
|G-NLM sharp||G-NLM smooth|
All the above consideration are reinforced by further visual analyses. Here we only show another detail, in Fig.13, a 192384 section from the bottom of the T2 clip. Again, the proposed method ensures a better speckle rejection (especially the smooth version) and detail preservation (especially the sharp version) than all references, with an excellent overall quality, considering also the very noisy single-look original data. Obviously, the comparison with conventional methods is not fair, since G-NLM relies on precious auxiliary data to improve performance. On the other hand, our first aim is exactly to show that optical-guided despeckling is a simple and safe solution towards obtaining high-quality despeckled SAR data.
However, to fully support this claim, we must consider more challenging scenes, man-made structures, roads, and sharp boundaries between regions of different nature. To this end, in Fig.14 we show a 256256 detail of the C4 clip. The fields show the phenomena already described before, just note that both enhanced Lee and PPB suggest the presence of diagonal strips, further enhanced in G-NLM sharp. As for the buildings, enhanced Lee and PPB smear and sometimes lose details; SAR-BM3D and related FANS and G-BF preserve all structures very well, with an accuracy comparable to that of G-NLM. The latter, however, succeed in removing speckle even inside the man-made area on in its near proximity, providing a sharper result and contributing to a better perceived quality.
|optical guide||single-look SAR data||FANS|
|G-BF||G-NLM sharp||G-NLM smooth|
All this said, if we consider our numerical measure of structuredness, the RIS, we obtain quite different indications. In fact, the results reported in Tab.III and Tab.IV for the two datasets show SAR-BM3D to have by far the lowest RIS, around 2%, followed by FANS and G-BF (which inherit some good features of SAR-BM3D), and by enhanced Lee, while both versions of G-NLM come last, together with PPB, with almost 7%.
|optical guide||single-look SAR data||FANS|
|G-BF||G-NLM sharp||G-NLM smooth|
To explain such conflicting results, in Fig.15 we show the ratio images themselves. The visual inspection provides clear answers. Basically, none of the ratio images show significant leakages from the original image, some image structures are visible only in the PPB ratio and just barely in the enhanced Lee and G-NLM ratios. In these conditions, the RIS measures mostly the grain of the ratio image, which overwhelms truly structural dependencies. Under this point of view, the SAR-BM3D ratio image is clearly preferable, as it resembles very closely a white noise field. All other methods introduce some weak correlation which, however, does not seem to impact on image quality. In summary, RIS provides some valuable information (the ratio image grain) but correlates poorly with image quality. Unfortunately, this holds for all other measures we tested, which leaves visual inspection as the most reliable form of quality assessment.
Therefore, we conclude our analysis by studying, with Fig.16, a last detail, the central part of clip T1, comprising mostly urban areas. Again, G-NLM preserves faithfully all features related to man-made objects and guarantees a strong rejection of speckle in homogeneous areas, even amidst buildings. All reference methods, instead, present some shortcomings, like limited speckle rejection, loss of resolution, or the introduction of filtering artifacts.
|optical guide||single-look SAR data||FANS|
|G-BF||G-NLM sharp||G-NLM smooth|
So, the analysis of this image does not seem to add new information. However, by looking at the whole set of Google Earth images of this area, we discovered some significant temporal changes, which allow us to study the robustness of the proposed method to mismatches between SAR and optical data. In particular, as shown in Fig.17, a group a buildings was leveled between 30-10-2002, date of the previous available optical image, and 31-12-2009, date of our guide. In the test SAR image, dated 27-01-2008, the buildings were still standing, as testified by several double reflection lines. Therefore, there is strong mismatch between SAR data and optical guide. Nonetheless, this does not seem to affect the filtered image, where the building-related structures are clearly visible, and look very similar to those provided by other filters, e.g., SAR-BM3D. As a further proof, we applied the proposed filter using the 2002 image as optical guide, and obtaining similar results.
|2002 optical guide||2008 single-look SAR||2009 optical guide|
|2002-G-NLM sharp||SAR-BM3D||2009-G-NLM sharp|
In this paper, we proposed a nonlocal SAR despeckling filter which makes use of available optical imagery to improve performance. Experiments on two real-world datasets show the proposed method to provide filtered images of excellent quality, arguably out of the reach of purely SAR-domain methods. The performance is also much better than that of our own previous optical-guided filter.
It is not surprising that information provided by optical imagery may help improving SAR despeckling. Patch-wise nonlocal filtering allows us to exploit this information in a seamless way, avoiding any optical-induced artifacts. However, better solutions are certainly possible, and we hope to witness increasing activity on this line of research. A crucial point along this path is user-friendliness. End users willing to obtain high-quality despeckling images look for simple and efficient plug-and-play tools, which do not require much direct involvement.
The proposed method moves a step in this direction. It provides high-quality and stable results based on the freely available Google Earth images, even in the presence of temporal changes. However, the user is still required to manually co-register the optical images with their SAR data. Our future work aims at improving the automation of this latter phase.
In this appendix we compute distribution, mean and variance of the random variable
where and are i.i.d. RV’s, with unit-mean Gamma distribution,
which model speckle samples in a -look SAR image.
Since and are non negative, their geometric and arithmetic means, and , are well defined and non negative, and their ratio does not exceed 1, . Then, we consider the RV transformation
with inverse transformation
and Jacobian with determinant
by which we obtain
having defined .
By rearranging the terms of the above expression we see that and are independent random variables
where is a unit-mean Gamma RV with parameter
while the ratio has pdf
A further RV transformation provides the desired pdf
To obtain mean and variance of we compute its moment generating function
expressed in terms of the -order polygamma functions
-  M. Latifur Rahman Sarker, J. Nichol, H. Baki Iz, B. Bin Ahmad, and A. Abdul Rahman, “Forest Biomass Estimation Using Texture Measurements of High-Resolution Dual-Polarization C-Band SAR Data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 51, no. 6, pp. 3371–3384, June 2013.
-  K. Irwin, A. Braun, G. Fotopoulos, A. Roth, and B. Wessel, “Assessing Single-Polarization and Dual-Polarization TerraSAR-X Data for Surface Water Monitoring,” Remote Sensing, vol. 10, no. 6, 2018.
-  G. He, G.-S. Xia, and H. Sun, “An Adaptive and Iterative Method of Urban Area Extraction From SAR Images,” IEEE Geoscience and Remote Sensing Letters, vol. 3, no. 4, pp. 504–507, Oct 2006.
-  P. Gamba, M. Aldrighi, and M. Stasolla, “Robust Extraction of Urban Area Extents in HR and VHR SAR Images,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 4, no. 1, pp. 27–34, March 2011.
-  H. Deng and D. Clausi, “Unsupervised segmentation of synthetic aperture Radar sea ice imagery using a novel Markov random field model,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 3, pp. 528–538, Mar 2005.
-  F. Wang, Y. Wu, Q. Zhang, W. Zhao, M. Li, and G. Liao, “Unsupervised SAR Image Segmentation Using Higher Order Neighborhood-Based Triplet Markov Fields Model,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 8, pp. 5193–5205, Nov 2014.
-  R. Dekker, “Texture analysis and classification of ERS SAR images for map updating of urban areas in The Netherlands,” IEEE Transactions on Geoscience and Remote Sensing, vol. 41, no. 9, pp. 1950–1958, Sep 2003.
-  A. Popescu, D. Faur, C. Vaduva, and M. Datcu, “Enhanced classification of land cover through joint analysis of Sentinel-1 and Sentinel-2 data,” in ESA Living Planet Symposium, May 2016, pp. 9–13.
-  F. Argenti, A. Lapini, T. Bianchi, and L. Alparone, “A tutorial on speckle reduction in synthetic aperture radar images,” IEEE Geoscience and Remote Sensing Magazine, vol. 1, no. 3, pp. 6–35, Sept 2013.
-  J.-S. Lee, “Digital image smoothing and the sigma filter,” Computer Vision, Graphics, and Image Processing, vol. 24, no. 2, pp. 255 – 269, 1983.
-  A. Lopes, R. Touzi, and E. Nezry, “Adaptive speckle filters and scene heterogeneity,” IEEE Transactions on Geoscience and Remote Sensing, vol. 28, no. 6, pp. 992–1000, Nov 1990.
-  J. S. Lee, J. H. Wen, T. L. Ainsworth, K. S. Chen, and A. J. Chen, “Improved Sigma Filter for Speckle Filtering of SAR Imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 1, pp. 202–213, Jan 2009.
-  H. Guo, J. Odegard, M. Lang, R. Gopinath, L. Selesnick, and C. Burrus, “Wavelet based speckle reduction with application to SAR based ATD/R,” in IEEE International Conference on Image Processingn, 1994, pp. 75–79.
-  A. Achim, P. Tsakalides, and A. Bezarianos, “SAR image denoising via Bayesian wavelet shrinkage based on heavy-tailed modeling,” IEEE Transactions on Geoscience and Remote Sensing, vol. 41, no. 8, pp. 1773–1784, Aug 2003.
-  F. Argenti, T. Bianchi, and A. Alparone, “Segmentation-based MAP despeckling of SAR images in the undecimated wavelet domain,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 9, pp. 2728–2742, Sep 2008.
-  S. Foucher, “SAR image filtering via learned dictionaries and sparse representations,” in IEEE International Geoscience and Remote Sensing Symposium, July 2008, pp. 229–232.
-  C. Ozcan, B. Sen, and F. Nar, “Sparsity-Driven Despeckling for SAR Images,” IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 1, pp. 115–119, 2016.
-  S. Tabti, L. Verdoliva, and G. Poggi, “Sparse-coding adapted to SAR images with an application to despeckling,” in IEEE International Geoscience and Remote Sensing Symposium, July 2018.
-  L. Denis, F. Tupin, J. Darbon, and M. Sigelle, “SAR image regularization with fast approximate discrete minimization,” IEEE Transactions on Image Processing, vol. 18, no. 7, pp. 1588–1600, 2009.
-  W. Feng, H. Lei, and Y. Gao, “Speckle Reduction via Higher Order Total Variation Approach,” IEEE Transactions on Image Processing, vol. 23, no. 4, pp. 1831–1843, Apr 2014.
-  Y. Zhao, J. Liu, B. Zhang, W. Hong, and Y.-R. Wu, “Adaptive Total Variation Regularization Based SAR Image Despeckling and Despeckling Evaluation Index,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2756–2774, May 2015.
-  G. Chierchia, D. Cozzolino, G.Poggi, and L.Verdoliva, “SAR image despeckling through Convolutional Neural Networks,” in IEEE International Geoscience and Remote Sensing Symposium, July 2017.
-  P. Wang, H. Zhang, and V. Patel, “SAR Image Despeckling Using a Convolutional Neural Network,” IEEE Signal Processing Letters, vol. 24, no. 12, pp. 1763–1767, 2017.
-  C. A. Deledalle, L. Denis, G. Poggi, F. Tupin, and L. Verdoliva, “Exploiting Patch Similarity for SAR Image Processing: The nonlocal paradigm,” IEEE Signal Processing Magazine, vol. 31, no. 4, pp. 69–78, July 2014.
-  G. Di Martino, M. Poderico, G. Poggi, D. Riccio, and L. Verdoliva, “Benchmarking Framework for SAR Despeckling,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 3, pp. 1596–1615, 2014.
-  C. A. Deledalle, L. Denis, and F. Tupin, “Iterative weighted maximum likelihood denoising with probabilistic patch-based weights,” IEEE Transactions on Image Processing, vol. 18, no. 12, pp. 2661–2672, Dec 2009.
-  S. Parrilli, M. Poderico, C. V. Angelino, and L. Verdoliva, “A Nonlocal SAR Image Denoising Algorithm Based on LLMMSE Wavelet Shrinkage,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 2, pp. 606–616, Feb 2012.
-  C. Deledalle, L. Denis, and F. Tupin, “How to compare noisy patches? Patch similarity beyond gaussian noise,” International Journal of Computer Vision, vol. 99, pp. 86–102, June 2012.
-  M. Schmitt and X. X. Zhu, “Data fusion and remote sensing: An ever-growing relationship,” IEEE Geoscience and Remote Sensing Magazine, vol. 4, no. 4, pp. 6–23, Dec 2016.
-  N. Joshi, M. Baumann, A. Ehammer, R. Fensholt, K. Grogan, P. Hostert, M. Jepsen, T. Kuemmerle, P. Meyfroidt, E. Mitchard, J. Reiche, C. Ryan, and B. Waske, “A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring,” Remote Sensing, vol. 8, no. 1, p. 70, 2016.
-  M. Schmitt, F. Tupin, and X. Zhu, “Fusion of SAR and optical remote sensing data - challenges and recent trends,” in IEEE International Geoscience and Remote Sensing Symposium, July 2017.
-  R. Bahmanyar, D. Espinoza-Molina, and M. Datcu, “Multisensor Earth Observation Image Classification Based on a Multimodal Latent Dirichlet Allocation Model,” IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 3, pp. 459–463, Mar 2018.
-  C. Corbane, J.-F. Faure, N. Baghdadi, N. Villeneuve, and M. Petit, “Rapid Urban Mapping Using SAR/Optical Imagery Synergy,” Sensors, vol. 8, no. 11, pp. 7125–7143, 2008.
-  G. Laurin, V. Liesenberg, Q. Chen, L. Guerriero, F. D. Frate, A. Bartolini, D. Coomes, B. Wilebore, J. Lindsell, and R. Valentini, “Optical and SAR sensor synergies for forest and land cover mapping in a tropical site in West Africa,” International Journal of Applied Earth Observation and Geoinformation, vol. 21, pp. 7–16, Apr 2013.
-  A.Errico, C.V.Angelino, L.Cicala, G.Persechino, C.Ferrara, M.Lega, A.Vallario, C.Parente, G.Masi, R.Gaetano, G.Scarpa, D.Amitrano, G.Ruello, L.Verdoliva, and G.Poggi, “Detection of environmental hazards through the feature-based fusion of optical and SAR data: A case study in southern Italy,” International Journal of Remote Sensing, vol. 36, pp. 3345–3367, July 2015.
-  T. Idol, B. Haack, and R. Mahabir, “Comparison and integration of spaceborne optical and radar data for mapping in Sudan,” International Journal of Remote Sensing, pp. 1551–1569, 2015.
-  L. Verdoliva, R. Gaetano, G. Ruello, and G. Poggi, “Optical-Driven Nonlocal SAR Despeckling,” IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 2, pp. 314–318, Feb 2015.
-  R. Gaetano, D. Cozzolino, L. D’Amiano, L. Verdoliva, and G. Poggi, “Fusion of SAR-optical data for land cover monitoring,” in 2017 IEEE Geoscience and Remote Sensing Symposium, 2017.
-  D. Gragnaniello, G. Poggi, G. Scarpa, and L. Verdoliva, “SAR Image Despeckling by Soft Classification,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 9, no. 6, pp. 2118–2130, June 2016.
-  D. Cozzolino, S. Parrilli, G. Scarpa, G. Poggi, and L. Verdoliva, “Fast Adaptive Nonlocal SAR Despeckling,” IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 2, pp. 524–528, Feb 2014.
-  C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in International Conference on Computer Vision, Jan 1998, pp. 839–846.
-  L. Gomez, R. Ospina, and A. C. Frery, “Unassisted quantitative evaluation of despeckling filters,” Remote Sensing, vol. 9, no. 4, p. 389, 2017.
-  R. Haralick, K. Shanmugam, and I. Dinstein, “Textural features for image classification,” IEEE Transactions on Systems, Man and Cybernetics, vol. 3, no. 6, pp. 610–621, 1973.
-  J. Lee, “Digital image enhancement and noise filtering by use of local statistics,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-2, no. 2, pp. 165–168, March 1980.
-  K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, Aug 2007.