Self-synchronizing scheme for high speed computational ghost imaging
Computational ghost imaging needs to acquire a large number of correlated measurements between reference patterns and the scene for reconstruction, so extremely high acquisition speed is crucial for fast ghost imaging. With the development of technologies, high frequency illumination and detectors are both available, but their synchronization needs technique demanding customization and lacks flexibility for different setup configurations. This letter proposes a self-synchronization scheme that can eliminate this difficulty by introducing a high precision synchronization technique and corresponding algorithm. We physically implement the proposed scheme using a 20kHz spatial light modulator to generate random binary patterns together with a 100 times faster photodiode for high speed ghost imaging, and the acquisition frequency is around 14 times faster than that of state-of-the-arts.
Department of Automation, Tsinghua University, Beijing 100084, China
Graduate School at Shenzhen, Tsinghua University, Shenzhen 518055, China
Corresponding author: firstname.lastname@example.org
110.0110, 110.1758, 110.3010.
Ghost imaging that takes root from quantum optics and then extends to classical light field , can recover the scene information using its correlated measurements with a reference beam. Computational imaging  is later proposed to reconstruct the scene using a computer controlled pattern sequence instead of physical reference arm. The relationship and comparison among above three variants—quantum, classical and computational—are comprehensively reviewed by Erkmen and Shapiro in . Such an imaging technique without a spatially resolvable sensor holds great potential for building compact optical harsh systems and makes good use the high performance of the single-pixel bucket photo-detectors. Large progresses in ghost imaging have been made in the past years. Recently, Sun el al. combine ghost imaging with computer vision algorithms (photometric stereo) and successfully conduct 3D reconstruction using multiple single pixel detectors and one projector. Later, Welsh et al. extend the gray scale computational ghost image to full-color using three detectors that response to different wavelengths.
Among all of the above state-of-the-arts, each computational ghosting imaging approach needs to collect a large number of single-pixel measurements for the final imaging , usually by a pair of illumination and detector synchronized either manually or automatically. Although one can utilize the redundancy in nature images  or a coarse-to-fine scheme  to reduce the number of measurements, the required number of measurement is still quite large, especially in the real experiments where orders of magnitude times more measurements are needed to compensate the system noise and influences from external factors. As far as we know, by utilizing the mechanisms of DLP smartly  state-of-the-art fast computational ghost imaging is able to capture no higher than 1.5K correlated measurements per second. However, it is still quite slow since hundreds of thousands of measurements are necessary for visually pleasant reconstruction. Therefore, ghost imaging is really time consuming and this drawback largely limits its real applications.
Raising frequency of illumination and detector can increase the acquisition frequency naturally, but the synchronization becomes more and more technically demanding with the increase of this frequency, because hardware expertise (in both illumination and detector) is necessary for the synchronization. In addition, the close bounding between the synchronization and hardware makes the system less flexible. Therefore, we attempt to propose a general and flexible fast ghost imaging scheme being able to readily incorporating currently available high speed illuminations and detectors, which both work at constant frequencies and are synchronized computationally, as illustrated in Fig. 1.
This letter draws some inspirations from the high-speed hyper-spectral imaging method proposed by Han et al., who use a ”white board” with known reflectance behaviours to synchronize illumination and camera working at constant frequencies and capture the scene’s reflectance fast. However, introducing such an auxiliary object is inapplicable for our task since its information is coupled with that of scene. To address this problem, we introduce designed patterns with the corresponding measurements largely different from that of the random patterns, to synchronize the illumination and detector. Then, we can exclude these synchronization patterns and reconstruct the scene using off-the-shelf ghost imaging algorithms.
Currently, we can use a programmable spatial light modulator (SLM), e.g. digital micro-mirror device (DMD), to shed binary coded illuminations fast with a small elapse during the 0/1 transition, which is determined by the transition and stabilization velocity of SLM units. This elapse is quite small but non-ignorable for high speed illumination of thousands Hz. Therefore, we can regard the latent measurement for ghost imaging as a close-to-ideal square wave, as illustrated in the top two rows of Fig. 2. Note that here we assume a gradual transition between adjacent illumination patterns in general, there may be some other transition ways. For example, there is a reset operation of micro mirror position in DMD modulation, so a zero measurement will occur in the transition slot. On the other hand, photodiodes with a much higher speed (up to GHz) are widely available and can be sampled at lower frequencies using an acquisition card. Therefore, we can conduct data acquisition multiple or even orders of magnitudes faster than illumination patterns and combine with an algorithm to avoid the influences from inter-pattern transition and analog-to-digital noise, as illustrated in the third row of Fig. 2.
Without loss of generality, we assume both the SLM and detector work at constant frequencies denoted as and respectively, and the latter is integral times of the former (e.g., ). Mathematically self-synchronization turns into computing the optimum sampling points, as illustrated by the red points in the bottom row of Fig. 2, and their corresponding illumination patterns. To synchronize these two modules, we keep one working repeatedly and start the other from an arbitrary time instant , then the misalignment can be described by two parameters: the integral misalignment with being the total number of illumination patterns, i.e., index of the starting illumination pattern with corresponding measurement captured; the fractional misalignment described as a reliable index set , whose size is determined by the time span of SLM’s stabilized stage.
The misalignment parameters and between illumination patterns and the measurements can be estimated easily by prefixing some illumination patterns whose measurements are largely different from those of random patterns, as shown in Fig. 3. The adopted alternate all-0 and all-1 synchronization patterns are somewhat similar to but of a different usage from that in . Assuming groups of synchronization patterns are prefixed, the optimum misalignment parameters are estimated by minimizing the measurement difference within the all-0 or all-1 pattern set (numerator term in Eq. 1) and maximizing that between measurement sets corresponding to all-0 and all-1 illumination (denominator term in Eq. 1)
Here represents a set of measurements in the stabilized stage of an illumination pattern, denotes the remainder operator, and denotes average operation along temporal dimension. For fast computation, we can firstly locate the coarse position of the synchronization sequence according to their quite large or small measurements, and then optimize the parameters via traversing searching locally. The candidate parameter is no larger than 2, so the algorithm is quite efficient. Experimentally, only 2-3 such alternative pattern groups are sufficient for synchronization.
After estimating and , we then cycle shift the pattern sequence to get
and subsample the measurements to get corresponding responses
Since the error caused by slight unsynchronization grows with the frequency increasing, so fractional synchronization is another advantage over the simple synchronization in .
At the final step, we can just exclude the synchronization patterns and retrieve the 2D scene from the synchronized measurements and illumination patterns. As for ghost imaging algorithms, there are mainly three variants—TGI, DGI and NGI—and each can be implemented in either iterative or batch manner. Among the three algorithms, we choose normalized ghost imaging due to its high robustness to external disturbances. To make full use of the high speed illumination, here we adopt the latter implementation by processing the entire data set in batch manner.
in which computes the average of the elements in a pattern and denotes temporal average.
Here we conduct a quantitative experiment on synthetic data to check the advantages of the proposed fractional self-synchronization strategy. We generate 10,000 10080 random binary patterns and calculate their inner products with a sharp ’text’ image of the same resolution to get the latent correlated measurements. Then we denote the period of illumination to be and apply a left-half gaussian convolution with standard deviation being 0.24 and 2.8 successively to simulate the elapse in SLM and detector. As for system noise, gaussian white noise with standard deviation being 0.1 of the latent signal is added to obtain the final measurements from the detector. To compare the results with and without fractional synchronization, we simulate the sampling process of acquisition card by sampling at 1 (non-fractional) and 100 (fractional) respectively.
Here we adopt root mean square error (RMSE) to compare the reconstruction qualities of two synchronizations at varying fractional misalignments using the data synthesized at different beginning instants =. The apparent smaller reconstruction error and better visual performance of fractional alignment in Fig. 4 clearly display its consistent performance advantage compared to the non-fractional synchronization. As the starting instant varies, we can see that the fractionally synchronized reconstruction keeps at a high performance, while the result with only 1 sampling degenerates apparently in the transitional stage. The reconstruction error for is obviously larger than that of , and the reconstruction almost fails when . The performance degeneration in the transitional stage of SLM is mainly due to the inaccurate match between illumination and measurement. There also exists some performance superiority of fractional alignment at the stabilized stage, and this advantage mainly comes from the noise suppression by averaging multiple valid (fine aligned) measurements. Therefore, we can maximize the sampling rate within the capability of detector and acquisition card to raise the final reconstruction quality.
To validate the proposed scheme, we build a prototype system exhibited in Fig. 5, whose light path is displayed in Fig. 1. The system consists mainly two modules: the programmable illuminator is generated by a DMD (Texas Instrument DLP Discovery 4100, .7XGA) that can project binary patterns at 20kHz (the maximum 32kHz is so far inaccessible to our own SDK) with 12s 0/1 transition time, i.e., 24% of the slot for each illumination pattern; a high speed single pixel detector is implemented by Thorlabs DET36A Silicon photodiode (350-1100 nm) with rising time being 14. Specifically, the illuminator is implemented by hacking an off-the-shelf DLP projector via replacing its built in DMD with our DLP Discovery. The measurements encoding the scene information are collected by sampling the outputs of the detector using a 14bit acquisition board ART PCI8514, which digitalizes the analog signals and transports them to the computer for follow-up reconstruction. We keep the cooling system and power module of the original projector. In order to ensure high acquisition accuracy, we set the acquisition frequency at 100 times of the illumination, i.e., 2mHz, and each correlated measurement is averaged over 76 samples, which lie in the stabilized stage.
We use 200200 pixels out of the whole 1024768 pixels in the DMD. The synchronization patterns and random patterns are focused onto a 34mm34mm film, and the photons travel through the film are collected onto the detector by a converging lens (=50.8mm, =50.8mm). During capturing, we keep the detector working all through, and then project the preloaded binary patterns onto the film using DMD from an arbitrary instant. In sum, we need only a slot longer than 0.5 seconds to acquire 10,000 measurements and the imaging speed is remarkably higher than state-of-the-arts, which at most take around 1.4k measurements per second. The speed can be further raised by making full use the the 32kHz frequency of the DMD. One result reconstructed from 10,000 measurements acquired by our system is shown in Fig. 5. Recall that the maximum number of projected binary patterns is currently limited by the internal memory size of our DMD controller board. The memory can be increased to shed a longer pattern sequence and improve the performance further but with only a linearly increasing acquisition time. To validate this potential performance of our approach, we load and project four more groups of random patterns (i.e., 50,000 in total) and concatenate the measurements for final reconstruction. One can see that the result is improved further, as displayed in Fig. 5.
In summary, this letter proposes a self-synchronization scheme to perform fast ghost imaging using a pair of un-synchronized detector and illumination pattern, both of which work at constant frequencies. The proposed approach largely addresses the synchronization challenge for fast ghost imaging, takes good advantage of the high frequency illumination/detector, and is flexible for replacing either illumination or detector module easily.
-  T. Pittman, Y. Shih, D. Strekalov, and A. Sergienko, Phys. Rev. A 52, R3429 (1995).
-  D. Strekalov, A. Sergienko, D. Klyshko, and Y. Shih, Phys. Rev. Lett. 74, 3600–3603 (1995).
-  R. S. Bennink, S. J. Bentley, and R. W. Boyd, Phys. Rev. Lett. 89, 113601 (2002).
-  F. Ferri, D. Magatti, A. Gatti, M. Bache, E. Brambilla, and L. Lugiato, Phys. Rev. Lett. 94, 183602 (2005).
-  J. H. Shapiro, Phys. Rev. A 78, 061802 (2008).
-  B. I. Erkmen and J. H. Shapiro, Advances in Optics and Photonics 2, 405–450 (2010).
-  B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, and M. Padgett, Science 340, 844–847 (2013).
-  S. S. Welsh, M. P. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. J. Padgett, Opt. Express 21, 23068–23074 (2013).
-  B. I. Erkmen and J. H. Shapiro, Phys. Rev. A 79, 023833 (2009).
-  R. Meyers, K. S. Deacon, and Y. Shih, Phys. Rev. A 77, 041801 (2008).
-  O. Katz, Y. Bromberg, and Y. Silberberg, Appl. Phys. Lett. 95, 131110 (2009).
-  M. Amann and M. Bayer, Scientific Reports 3, 1–5 (2013).
-  M. Chen, E. Li, and S. Han, Appl. Opt. 53, 2924–2928 (2014).
-  W. Yu, M. Li, X. Yao, X. Liu, L. Wu, and G. Zhai, Opt. Express 22, 7133–7144 (2014).
-  D. Faccio and J. Leach, Science 340, 821–822 (2013).
-  S. Han, I. Sato, T. Okabe, and Y. Sato, in “Asian Conference on Computer Vision,” (Springer, 2011), pp. 323–335.
-  Y. Bromberg, O. Katz, and Y. Silberberg, Phys. Rev. A 79, 053840 (2009).
-  F. Ferri, D. Magatti, L. Lugiato, and A. Gatti, Phys. Rev. Lett. 104, 253603 (2010).
-  B. Sun, S. S. Welsh, M. P. Edgar, J. H. Shapiro, and M. J. Padgett, Opt. Express 20, 16892–16901 (2012).
Informational Fifth Page
-  T. Pittman, Y. Shih, D. Strekalov, and A. Sergienko, “Optical imaging by means of two-photon quantum entanglement,” Phys. Rev. A 52, R3429 (1995).
-  D. Strekalov, A. Sergienko, D. Klyshko, and Y. Shih, “Observation of two-photon ”ghost” interference and diffraction,” Phys. Rev. Lett. 74, 3600–3603 (1995).
-  R. S. Bennink, S. J. Bentley, and R. W. Boyd, “’Two-photon’ coincidence imaging with a classical source,” Phys. Rev. Lett. 89, 113601 (2002).
-  F. Ferri, D. Magatti, A. Gatti, M. Bache, E. Brambilla, and L. Lugiato, “High-resolution ghost image and ghost diffraction experiments with thermal light,” Phys. Rev. Lett. 94, 183602 (2005).
-  J. H. Shapiro, “Computational ghost imaging,” Phys. Rev. A 78, 061802 (2008).
-  B. I. Erkmen and J. H. Shapiro, “Ghost imaging: from quantum to classical to computational,” Advances in Optics and Photonics 2, 405–450 (2010).
-  B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, and M. Padgett, “3D computational imaging with single-pixel detectors,” Science 340, 844–847 (2013).
-  S. S. Welsh, M. P. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. J. Padgett, “Fast full-color computational imaging with single-pixel detectors,” Opt. Express 21, 23068–23074 (2013).
-  B. I. Erkmen and J. H. Shapiro, “Signal-to-noise ratio of gaussian-state ghost imaging,” Phys. Rev. A 79, 023833 (2009).
-  R. Meyers, K. S. Deacon, and Y. Shih, “Ghost-imaging experiment by measuring reflected photons,” Phys. Rev. A 77, 041801 (2008).
-  O. Katz, Y. Bromberg, and Y. Silberberg, “Compressive ghost imaging,” Appl. Phys. Lett. 95, 131110 (2009).
-  M. Amann and M. Bayer, “Compressive adaptive computational ghost imaging,” Scientific Reports 3, 1–5 (2013).
-  M. Chen, E. Li, and S. Han, “Application of multi-correlation-scale measurement matrices in ghost imaging via sparsity constraints,” Appl. Opt. 53, 2924–2928 (2014).
-  W. Yu, M. Li, X. Yao, X. Liu, L. Wu, and G. Zhai, “Adaptive compressive ghost imaging based on wavelet trees and sparse representation,” Opt. Express 22, 7133–7144 (2014).
-  D. Faccio and J. Leach, “Ghost imaging in three dimensions,” Science 340, 821–822 (2013).
-  S. Han, I. Sato, T. Okabe, and Y. Sato, “Fast spectral reflectance recovery using DLP projector,” in “Asian Conference on Computer Vision,” (Springer, 2011), pp. 323–335.
-  Y. Bromberg, O. Katz, and Y. Silberberg, “Ghost imaging with a single detector,” Phys. Rev. A 79, 053840 (2009).
-  F. Ferri, D. Magatti, L. Lugiato, and A. Gatti, “Differential ghost imaging,” Phys. Rev. Lett. 104, 253603 (2010).
-  B. Sun, S. S. Welsh, M. P. Edgar, J. H. Shapiro, and M. J. Padgett, “Normalized ghost imaging,” Opt. Express 20, 16892–16901 (2012).