Laser light-field fusion for wide-field lensfree on-chip phase contrast nanoscopy
Wide-field lensfree on-chip microscopy, which leverages holography principles to capture interferometric light-field encodings without lenses, is an emerging imaging modality with widespread interest given the large field-of-view compared to lens-based techniques. In this study, we introduce the idea of laser light-field fusion for lensfree on-chip phase contrast nanoscopy, where interferometric laser light-field encodings acquired using an on-chip setup with laser pulsations at different wavelengths are fused to produce marker-free phase contrast images of superior quality with resolving power more than five times below the pixel pitch of the sensor array and more than 40% beyond the diffraction limit. As a proof of concept, we demonstrate, for the first time, a wide-field lensfree on-chip instrument successfully detecting 300 nm particles, resulting in a numerical aperture of 1.1, across a large field-of-view of 30 mm without any specialized or intricate sample preparation, or the use of synthetic aperture- or shift-based techniques.
Phase contrast microscopy was introduced as a means of observing objects with light transparent properties . Bright-field microscopy of such objects tend to produce low contrast images since the probing light and the sample being imaged do not undergo a very strong interaction. Phase contrast microscopy, however, allows for visualization of the optical path length differences that the probing light would experience as the result of the interaction with the sample, thereby producing high contrast images of such samples. Despite the aforementioned benefit of phase contrast microscopy, it nevertheless suffers from the same shortcomings of bright-field microscopy such as limited field-of-view (FOV), limited resolution capability, and high complexity of the design and operation of the instrument.
Wide-field lensfree on-chip microscopy, where holography principles are used to capture interferometric light-field encodings without lenses, has become an interesting pathway toward addressing the shortcomings of lens-based techniques [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 12, 15, 16, 17, 18, 19, 20, 21]. At its core, these lensfree on-chip instruments are conceptually simple and offer extremely large FOV in comparison to lens-based instruments. The resolution of such instruments is inherently limited by the pixel pitch of the detectors used, which is currently limited by the fabrication technologies to 1 m. The quest for higher resolving power has led to numerous synthetic aperture- or lateral shift-based techniques [7, 11, 8, 22, 23, 9] for increasing resolving power beyond pixel pitch, though such techniques tend to increase the hardware complexity and decrease performance tolerance of the instrument. More recently, multi-wavelength illumination-based techniques [17, 20, 21, 24] showed considerable promise for increasing the resolving power beyond pixel pitch without the level of complexity of synthetic aperture- or lateral shift-based techniques.
A particular area of interest for employing wide-field lensfree on-chip microscopy is for nano-scale imaging. Super-resolution microscopy [25, 26], atomic force microscopy , and electron microscopy  have been the mainstays for nano-scale imaging but are limited by very high instrumentation cost and complexity, narrow FOV, and low imaging throughput due to the complexity of the imaging process. The ability to harness the principles behind wide-field lensfree on-chip microscopy for nano-scale imaging, which we will refer to as wide-field lensfree on-chip nanoscopy, holds considerable promise for greatly reducing imaging cost and complexity while improving image throughput and FOV, thus enabling comprehensive, large scale studies to be more readily achieved. Recent results are quite promising , demonstrating the ability to distinguish nanoparticles of various sizes. However, such techniques require specialized and intricate sample preparation (e.g.,  requires the use of biocompatible wetting films to self-assemble aspheric liquid nanolenses around individual nanoparticles) which significantly increases imaging complexity. As such, a method to achieve wide-field lensfree on-chip nanoscopy to facilitate for the detection of objects at the nanometer-scale without the need for additional specialized sample preparation, or the use of synthetic aperture- or lateral shift-based techniques is highly desired.
In this study, we introduce the idea of laser light-field fusion phase contrast nanoscopy. Inspired by our earlier, preliminary exploration into spectral light-field fusion [20, 21], laser light-field fusion phase contrast nanoscopy involves the acquisition and fusion of interferometric laser light-field encodings using a lensfree, on-chip setup with laser pulsations at different wavelengths to produce marker-free phase contrast images, as shown in Fig. 1. The proposed instrument allows us to achieve a resolving power below the pixel pitch of the sensor array as well as the wavelength of the probing light source, beyond the diffraction limit. As proof of concept, we demonstrate, for the first time, a lensfree on-chip instrument capable of detecting 300 nm particles across a large field-of-view of 30 mm without utilization of specialized or intricate sample preparation, or the use of synthetic aperture- or lateral shift-based techniques.
The capability of the proposed instrument in resolving objects at the nanometer scale is first demonstrated in Fig. 2, where a phase contrast image of a sample consisting of polystyrene nanospheres (Fluoresbrite, Polysciences, Inc., USA) is shown in Fig. 2a. Upon closer examination, through two zoom levels, Fig. 2b and c, a phase contrast image of an isolated cluster of five 500 nm nanospheres are discovered, arranged in a ‘U’ formation. The light-field encoding captured at nm and nm are shown in Fig. 2d and e, respectively.
Scanning electron microscopy (SEM) (MERLIN, Carl Zeiss AG, Germany) was used to verify the imaging results captured by our instrument, as shown in Fig. 3. The corresponding location of the ‘U’ formation of the nanospheres are shown using a reference lensfree on-chip instrument capturing interferometric light-field encodings at nm (see Fig. 3a), the proposed laser light-field fusion phase contrast nanoscopy instrument (see Fig. 3b), and SEM (see Fig. 3c). The sizes of the nanospheres are noted on the SEM image which are within the manufacturer’s fabrication tolerance of 5003% nm . A number of observations can be made from Fig. 3 with regards to the reference lensfree on-chip instrument and the proposed laser light-field fusion phase contrast nanoscopy instrument. It can be observed that the same nanospheres observed by the SEM cannot be detected using the reference lensfree on-chip instrument capturing interferometric light-field encodings at nm. On the other hand, it can be observed that the same nanospheres observed by the SEM have been detected with the proposed laser light-field fusion phase contrast nanoscopy instrument, thus demonstrating its ability to detect particles that are 495 nm in size as well as achieve superior performance to the reference lensfree on-chip instrument. Furthermore, it can be observed that differentiating the size variability of these nanospheres is beyond the capability of the proposed instrument as such variability is well beyond its resolving power.
It can also be observed that the image quality achieved using the proposed laser light-field fusion phase contrast nanoscopy instrument is high at a detection signal-to-noise ratio (SNR) of 33.35 dB, achieved by only capturing a single interferometric light-field encoding at a given wavelength in the current setup. Fig. 3d shows the cross-sectional profile of the nanoparticle highlighted by the red circle in Fig. 3b as well as the SEM confirmed profile. It can be observed that the discrepancy between the measurements made by the SEM and the proposed instrument is negligible, thus demonstrating the fidelity and performance capabilities of the proposed instrument.
The capability of the proposed instrument was further demonstrated using a sample consisting of a mixture of particles of different sizes that was prepared for imaging. A zoomed-in region of this sample is shown in Fig. 4a and b imaged by the proposed instrument and an SEM, respectively. There are six different sizes of particles in this region varying from 300 nm to 1 m. The size of these particles are verified using the SEM and the corresponding particles are observed using the proposed instrument. A number of observations can be made from Fig. 4. It can be observed that the smallest particle, which is 300 nm in size, is detected using both the SEM and the proposed laser light-field fusion phase contrast nanoscopy instrument, with Fig. 4c and d showing this particle in isolation for the proposed instrument and SEM, respectively. It can also be observed from the cross-sectional profile and the intensity surface of the particle (see inset of Fig 4c) that a positive detection of the 300 nm particle is demonstrated, with a detection SNR of 29.61 dB. The imaging NA as defined by this detection is 1.1, and the improved imaging resolving power of the proposed instrument is more than five times smaller than the pixel pitch of the detector used. Furthermore, it can be observed that the size variation of different sized particles are explicitly reflected in the phase contrast image, as their imaged size tends to reflect the actual size. These results demonstrate as a proof-of-concept the capabilities of the proposed instrument, which is beyond existing lensfree on-chip instruments that do not require any specialized or intricate sample preparation, or the use of synthetic aperture- or shift-based techniques.
We introduced an wide-field lensfree on-chip phase contrast nanoscopy instrument with resolving power beyond the diffraction limit, below the wavelength of the probing illumination source. As proof-of-concept, the capability of the proposed instrument to surpass the pixel pitch resolution limit of the detector is demonstrated by imaging particles which are 300 nm in size, which is more than five times smaller than the pixel pitch of the detector used (1.67 m). These experimental results demonstrate that the unique diffraction behaviour captured in the acquired interferometric laser light-field encodings at different wavelengths can be leveraged through a fusion process to achieve improved image quality and resolving power in a lensfree on-chip instrument beyond that can be achieved using a single wavelength.
It is important to note that the proposed wide-field lensfree on-chip instrument does not require specialized sample preparation, or the use of synthetic aperture- or lateral shift-based techniques to accomplish nanometer-scale resolving power, with the current setup achieving an NA of 1.1. What this implies is that the proposed instrument has low instrumentation complexity and cost, and is easy to operate and maintain, thus allowing for democratization and proliferation of such instruments in healthcare, industry, education, and research. The interferometric light-field encoding acquisitions made by the proposed instrument take less than 2ms (equivalent to > 500 frames per second), thus enabling observations of highly time-resolved dynamic systems or transient phenomena at the nanometer scale, such as for the study of motion dynamics of colloidal nanoparticles.
A limitation with the setup of the proposed instrument used in this current study is that the number of wavelengths used is restricted to two different laser wavelengths, which can be addressed in future studies through the incorporation of a tunable laser to potentially enable further improvements in image quality and resolving power.
Materials and Methods
Imaging apparatus. The proposed wide-field lensfree on-chip phase contrast nanoscopy instrument used for this study (shown in Fig. 1) can be described as follows. A two-channel pulsed laser light source is used, with central wavelengths at nm and nm, and the spectral bandwidth being 0.5 nm. The pulsed laser light source was programmed to pulse with an alternating wavelength pulse sequence, where the duration of the pulsations are configured such that the signal observed on the detector is maximized while mitigating pixel saturation. The pulsed laser light source is coupled to a single-mode fiber optic cable to illuminate the sample. The detector readout is synchronized to the pulse sequence for rapid and seamless interferometric light-field encoding acquisitions at the two wavelengths of the laser light source. The exposure time was < 1 ms for each wavelength, resulting in interferometric light-field encoding acquisitions using the proposed instrument taking less than 2 ms (equivalent to > 500 frames per second)
The sample being imaged is placed on a #1 microscope cover slip with a thickness of 145 m, which sits directly on the detector. Interferometric light-field encoding acquisitions of the sample being imaged at different wavelengths are made by the detector using a 38402748 pixel CMOS sensor array with a pixel pitch of 1.67 m. The FOV of the device is determined based the active sensor size and is 30 mm. The captured interferometric light-field encoding, denoted by with denoting wavelength, is then sent to the digital signal processing unit, where numerical laser light-field fusion is performed to reconstruct a fused phase contrast image .
In this study, the measurement instrument was characterized based on a number of acquisitions of point source targets to determine the aberration transfer function at each wavelength (denoted by ) of the pulsed laser light source to account for differences at different wavelengths.
For comparison purposes, a reference lensfree on-chip instrument capturing interferometric light-field encodings at nm using the aforementioned imaging apparatus (with a single-channel laser light source is used instead) was also evaluated in this study.
Image Reconstruction. The numerical laser light-field fusion performed on the digital signal processing unit to reconstruct fused phase contrast images from measurements made by the proposed instrument can be described as follows. The captured interferometric laser light-field encoding encapsulates unique diffraction behaviour at different wavelengths , which we will leverage to achieved improved image quality beyond that can be achieved using a single wavelength. Let us formulate a fused laser object light-field as a subspace projection of the laser object light-field ,
where denotes the -associated coefficient of the largest eigenvector of the correlation matrix of , thus taking into account the correlation structure across . Given that the instrument described captures , not , one must devise a mechanism to numerically estimate given .
Let us now model the laser object light-field and the interferometric laser light-field encoding as probability distributions. To estimate , we now wish to compute a subspace projection of the most probable laser object light-field given , with a priori knowledge related to and the aberration transfer function (), as well as the Rayleigh-Sommerfeld diffraction transfer function (),
where denotes the likelihood of given and denotes the prior of . According to quantum photon emission statistics, one can express as
where and denotes the forward and inverse Fourier transform, respectively. Modeling as a nonstationary process, one can express as
where denotes the nonstationary expectation and denotes the variance. To reconstruct the fused phase contrast image , the computed is phase-shifted by 90 at the zeroth frequency, and the amplitude of this phase-shifted is then taken as .
An expectation maximization for MAP estimation is used to solve Eq. 2, and is performed until convergence.
The authors would like to thank Dr. Chao Jin and Prof. Monica Emelko from department of Civil and Environmental Engineering at University of Waterloo for providing the samples used in this study.
This work was supported by the Natural Sciences and Engineering Research Council of Canada, Canada Research Chairs Program, and the Ontario Ministry of Research and Innovation.
F.K. and A.W. contributed equally to this study.
The authors declare that they have no competing interests.
Data and Material Availability
All data related to this paper may be requested from the authors by emailing either firstname.lastname@example.org or email@example.com
- F. Zernike, Phase contrast, a new method for the microscopic observation of transparent objects, Physica 9, 686-698 (1942).
- Z. Frentz, S. Kuehn, D. Hekstra, S. Leibler, Microbial population dynamics by digital in-line holographic microscopy, Rev. Sci. Instrum. 81 084301 (2010).
- T. Su, A. Ozcan, On-Chip Holographic Microscopy and its Application for Automated Semen Analysis, Biomed. Opt. Phase Micro. and Nanoscopy, 153-171 (2013).
- O. Mudanyali, C. Oztoprak, D. Tseng, A. Erlinger, A. Ozcan, Detection of waterborne parasites using field-portable and cost-effective lensfree microscopy, Lab Chip 10, 2419-2423 (2010).
- J. Sheng, E. Malkiel, J. Katz, Digital Holographic Microscope for Measuring Three-dimensional Particle Distributions and Motions, App. Opt. 45, 3893-3901 (2006).
- M.Z. Kiss, B.J. Nagy, P. Lakatos, Z. Gorocs, S. Tokes, B. Wittner, L. Orzo, Special Multicolor Illumination and Numerical Tilt Correction in Volumetric Digital Holographic Microscopy, Opt. Exp. 21, 12469-12483 (2013).
- V. Mico, Z. Zalevsky, P. Garcia-Martinez, J. Garcia, Synthetic Aperture Superresolution with Multiple Off-axis holograms, J. Opt. Soc. Am. A 23, 3162-3170 (2006).
- V. Mico, Z. Zalevsky, Superresolved Digital In-line Holographic Microscopy for High-resolution Lensless Biological Imaging, J. Biomed. Opt. 15 046027 (2010).
- L. Granero, V. Mico, Z. Zalevsky, J. Garcia, Synthetic Aperture Superresolved Microscopy in Digital Lensless Fourier Holography by Time and Angular Multiplexing of the Object Information, App. Opt. 49, 845-857 (2010).
- L. Martinez-Leon, B. Javidi, Synthetic Aperture Single-exposure On-axis Digital Holography, Opt. Exp. 16, 161-169 (2008).
- W. Luo1,.A. Greenbaum, Y. Zhang, A. Ozcan, Synthetic aperture-based on-chip microscopy, Light: Sci. & App. 4, e261 (2015).
- S.O. Isikman, A. Greenbaum, W. Luo, A.F. Coskun, A. Ozcan, Giga-Pixel Lensfree Holographic Microscopy and Tomography Using Color Image Sensors PLoS ONE 7, e45044 (2012).
- W. Bishara, U. Sikora, O. Mudanyali, S. Ting-Wei, O. Yaglidere, S. Luckhart, A. Ozcan, Holographic Pixel Super-resolution in Portable Lensless On-chip Microscopy Using a Fiber-optic Array, Lab Chip 11, 1276-1279 (2011).
- A. Greenbaum, W. Luo, B. Khademhosseinieh, T. Su, A. Coskun, A. Ozcan, Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy, Sci. Rep. 3, 1717 (2013).
- S.O. Isikman, I. Sencan, O. Mudanyali, W. Bishara, C. Oztopraka, A. Ozcan, Color and Monochrome Lensless On-chip Imaging of Caenorhabditis Elegans Over a Wide Field-of-view, Lab Chip 10, 1109-1112 (2010).
- A. Greenbaum, A. Feizi, N. Akbari, A. Ozcan, Wide-field Computational Color Imaging Using Pixel Super-resolved On-chip Microscopy, Opt. Exp. 21 12469-12483 (2013).
- D. Noom, K. Eikema, S. Witte, Lensless phase contrast microscopy based on multiwavlength Fresnel diffraction, Opt. Lett. 39 193-196 (2014).
- D. Noom, D. Flaes, E. Labordus, K. Eikema, S. Witte, High-speed multi-wavelength Fresnel diffraction imaging, Opt. Exp. 22 30504-30511 (2014).
- A. Wong, F. Kazemzadeh, C. Jin, X.Y. Wang, Bayesian-based Aberration Correction and Numerical Diffraction for Improved Lensfree On-chip Microscopy of Biological Specimens, Opt. Lett. 40, 2233-2236 (2015).
- F. Kazemzadeh, C. Jin, S. Molladavoodi, Y. Mei, M. B. Emelko, M. B. Gorbet, A Wong, Lens-free spectral light-field fusion microscopy for contrast- and resolution-enhanced imaging of biological specimens, Opt. Lett. 40, 3862-3865 (2015).
- F. Kazemzadeh, A. Wong, Lens-free Multi-Laser Spectral Light-Field Fusion Microscopy, Vision Lett. 1, VL102 (2015).
- A. Pelagotti, M. Paturzo, M. Locatelli, A. Geltrude, R. Meucci, A. Finizio, P. Ferraro, An Automatic Method for Assembling a Large Synthetic Aperture Digital Hologram, Opt. Exp. 20, 4830–4839 (2012).
- T. S. Ralston, D. I. Marks, P.S. Carney, S.A. Boppart, Interferometric Synthetic Aperture Microscopy, Nat. Phys. 3, 129-134 (2007).
- W. Luo, Y. Zhang, A. Feizi, Z. Göröcs, A. Ozcan, Pixel super-resolution using wavelength scanning, Light: Sci. & App. 5, e16060 (2016).
- S. W. Hell, J. Wichmann, Breaking the diffraction resolution limit by stimulated emission: Stimulated-emission-depletion fluorescence microscopy, Optics Letters 19 780-782 (1994).
- E. Betzig, J. K. Trautman, Near-Field Optics: Microscopy, Spectroscopy, and Surface Modification Beyond the Diffraction Limit, Science 257 189-195 (1992).
- G. Binnig, C. F. Quate, C. Gerber, Atomic force microscope, Phys. Rev. Lett. 56 930-933 (1986).
- M. Azubel, J. Koivisto, S. Malola, D. Bushnell, G. L. Hura, A. L. Koh, H. Tsunoyama, T. Tsukuda, M. Pettersson, H. Häkkinen, R. D. Kornberg, Electron microscopy of gold nanoparticles at atomic resolution, Science 345 909-912 (2014).
- O. Mudanyali, E. McLeod, W. Luo, A. Greenbaum, A. F. Coskun, Y. Hennequin, C. F. Allier, A. Ozcan, Wide-field optical detection of nanoparticles using on-chip microscopy and self-assembled nanolenses, Nat. Phot. 7 247-254 (2013).
- Polysciences Inc. Fluoresbrite YG Carboxylate Microspheres 0.50m. Retrieved from: http://www.polysciences.com/default/fluoresbrite-yg-carboxylate-microspheres-050m (2016).