Effective Use of CMOS Pixels

Smaller pixels provide equivalent spatial resolution at lower magnifications, where spatial resolution describes the ability of a camera to distinguish small specimen features. Because some scientific-grade CMOS cameras for microscopy have smaller pixels than typical CCD cameras, they can adequately sample an image at lower magnifications (that is, achieve adequate spatial resolution to properly resolve sample features) and thus take advantage of lower magnification objectives whereas many CCDs cannot. This provides three main advantages.

The first advantage is that the use of smaller magnification allowed with smaller pixels increases the amount of light falling onto each pixel because more of the illuminated width of the specimen is transferred to the image sensor. At a lower magnification, an image representing a larger width of the illuminated specimen falls on the sensor. At a higher magnification with the same illumination, the light from a smaller region of the illuminated specimen is spread over the same width of sensor, so less light hits each pixel. More light increases signal and SNR for improved image quality. The second advantage is a larger field of view. With high magnifications, only a small portion of the sample can be fit onto the camera sensor's field of view because the sample image is magnified, or spread out, across the sensor. Using a smaller magnification means more of the sample area can fit onto the camera. Third, because CMOS cameras do not need to read out pixels one at a time, they can read out somewhat more quickly than CCDs. For example, using the appropriate magnifications with similar fields of view, a typical microscope CCD camera may read out full frames at 10 frames per second, compared to a scientific-grade CMOS camera that can read out 30 frames per second. This is a significant advantage for many applications as more and more research focuses on live cell studies at video frame rates.

Image Quality: Signal-to-Noise Ratio

The SNR is an indicator of image quality because it ratios the signal of interest from the sample to the uncertainty in that signal, the noise. Noise is defined as the uncertainty in a measurement and typically adds random variations to an image. For CCD and CMOS cameras, the noise sources are essentially the same, including read noise, dark noise, and photon noise [Reference Janesick1]. Read noise is the uncertainty in reading out an electronic signal as photoelectrons generated by light hitting the sensor are converted into a voltage and a grayscale value. It is determined by such factors as the readout electronics, sensor technology, and readout speed and is composed of multiple individual electronic components that sum together and that differ for different sensor technologies. For short, low-light exposures, this is typically the major noise source that limits image quality. Dark noise is random variation in the camera dark current, which is caused by thermal, rather than light-induced, generation of signal electrons. It increases with time and temperature so it becomes a limiting noise source at longer exposure times or when the camera is operated at higher temperatures. Dark noise can be reduced through cooling and careful readout electronics design. Photon noise is the variation in signal due to the quantized nature of the signal itself—individual photons and photoelectrons. Because it is signal-dependent, photon noise does not depend on the camera but rather on the magnitude of the signal, and it increases with the detected signal. At high light levels, the photon noise is the dominant noise source.

These various noise sources combine in quadrature to form the SNR equation given in Equation 1.

(1)

Here, flux is the amount of light hitting a sensor pixel in photons per second, and QE is the quantum efficiency, which indicates the percentage of incident photons that are converted into signal electrons. The SNR equation is essentially the same for both CCD and CMOS cameras with the exception that scientific-grade CMOS sensors have random telegraph noise (salt-and-pepper type speckling), which is incorporated into the read noise. Additionally, some scientific-grade CMOS sensors have no noticeable dark current because of their advanced readout circuit design. Furthermore, another type of noise called fixed pattern noise exists in both CCD and CMOS cameras as a variation in intensity across an image rather than random fluctuations in each pixel. In some scientific CMOS sensors, it can still appear as a vertical “bar code” pattern as a result of differences in response among the column readout amplifiers across the chip. In such CMOS cameras, this behavior can be clearly visualized by saturating the image sensor with light, and then the variations in the maximal column gray levels become quite apparent. It should be noted that the QImaging Rolera Bolt scientific CMOS camera used for the data collected in this paper does not exhibit such behavior and has much more uniform pixel response. The non-uniform response seen in other CMOS cameras is mainly an issue in low-light, high-end imaging, so for these situations it may be more advantageous to use a CCD or EMCCD [Reference Joubert and Sharma2]. It should also be noted that fixed pattern noise can be removed through data post-processing by subtracting the pattern.

The SNR is a useful figure of merit to compare standard CCD cameras with up-and-coming scientific-grade CMOS cameras. Two general approaches can be taken in calculating SNR. One approach uses parameters obtained from datasheets provided by the more reputable scientific imaging camera companies and known user parameters, such as magnification, numerical aperture (NA), pixel size, and specimen dye label concentration. Thus, the signal can be calculated along with the SNR using a somewhat complicated method outlined elsewhere [Reference Sabharwal3]. The second approach uses a more empirical method by extracting the noise and the signal from the acquired sample and bias images (these are images acquired with the camera in the absence of light). This is the method used in this paper and laid out in Figure 1. Based on the flow diagram in Figure 1 and images captured on a system with both a CCD and a scientific-grade CMOS attached, SNR can be measured as a function of exposure time, a common parameter in light microscopy for varying captured signal levels.

Figure 1: Outline of the process used for measuring SNR for an image with a given camera under a given set of conditions.

In Figure 2, comparison images of fluorescently labeled bovine pulmonary artery cells are shown, along with their respective SNR values, at various exposure times. The epifluorescence images were taken using a Photometrics DC2 dual camera system with a 50/50 beam-splitter cube to simultaneously split the light equally between the two cameras to be compared [Reference Shroff, Galbraith, Galbraith and Betzig4]. The images in the top row of Figure 3 were acquired with a standard front-illuminated CCD microscope camera using a 60× magnification, 1.35 NA, oil immersion objective such that its 6.45-μm pixel size is optimized for this magnification. The bottom row of Figure 2 shows images acquired with the Rolera Bolt, a new QImaging scientific-grade front-illuminated CMOS camera using a 1.35 NA, 40× magnification oil immersion objective with smaller 3.63-μm pixels optimized for this lower magnification. A stack of 10 images was acquired for each camera and exposure time. The standard deviation and mean of each pixel in the stack was found. The average bias value was subtracted from the mean image to obtain a signal at each pixel. This image was then divided by the standard deviation image, or noise, to obtain the SNR at each pixel. The mean and standard deviation of a background area was measured. A threshold was set at two of these background standard deviations above the background mean. In the final step, the mean value of the SNR in the pixels above this background threshold was measured and shown in Figure 2. The SNR values demonstrate that the 3.63-μm pixel scientific-grade CMOS at 40× has similar SNR performance at short exposures compared to the 6.45-μm pixel CCD at 60×. Again, both cameras are able to appropriately sample under these conditions. It could be proposed that the 6.45-μm-pixel CCD could also be used with a 40× objective or with a 60× objective and a 0.5× coupler, but at these smaller magnifications sensors with larger pixels (>3.8 microns) would no longer adequately sample, leading to lost resolution and aliasing and thus should not be used.

Figure 2: Images acquired with a CCD and a CMOS at 60× and 40×, respectively, over three different exposure times. Image quality clearly improves with exposure time, and the two technologies produce images of comparable quality under these conditions.

Figure 3: Bar graphs showing the increase in Figure 2 SNR values with exposure time. SNR values with error bars appear statistically indistinguishable for scientific grade CMOS and CCD images under these conditions.

Performance comparisons using SNR are useful because they provide quantitative numbers for camera evaluation under various realistic lighting conditions. However, the SNR needed to produce the image quality necessary for further data analysis or publication may not be known ahead of time. As a result, it is advantageous to compare additional complementary information, such as the quality of side-by-side images. Image quality is improved when the SNR is high, and the side-by-side images of CMOS and CCD cameras under these conditions demonstrate this correlation in Figure 2.

It is apparent in Figure 2 that the image quality improves with longer exposure times because the number of photons of light increases with collection time, producing higher signal and thus higher SNR. In terms of image appearance, it should also be noted that the cells in the 40× CMOS images appear slightly larger than in the 60× CCD images. When both sensors are used at the different magnifications where they still adequately sample, because the CMOS is a smaller pixel sensor that incorporates more pixels per square micron and both images have been scaled to approximately the same size, the images appear as displayed. Additionally, it is noticeable that the image quality is similar for the two image sets for the two camera technologies, in spite of several factors. One factor is that to achieve faster frame rates, CMOS pixels employ additional electronics to each pixel, which one might expect to reduce light collection efficiency. In both front-illuminated CCDs and some CMOS sensors, the fill factor (percentage of total pixel area that is light-sensitive) of each pixel is often improved through the addition of microlenses, micron-sized lenses on each pixel of the chip that collect light that would otherwise be blocked by electronic components and redirect it to the photosensitive pixel area. Although additional electronics are added to CMOS pixels, the implementation of increased pixel aperture and optimization of microlens design can compensate for what would be perceived as a potential loss in fill factor. Also, the CMOS has a significantly smaller pixel size. However, these factors are offset with the lower magnification and lower read noise of the Rolera Bolt scientific-grade CMOS (~3 electrons) to provide comparable image quality to the CCD under these conditions. The trend becomes more apparent when SNR values from the two technologies are graphed side-by-side with error bars corresponding to the standard deviation of the SNR across the thresholded cell images, as shown in Figure 3. The differences in SNR between images from the two sensor types fall within the variation in SNR across each image, indicating almost indistinguishable SNR performance under these conditions.

Another important parameter for scientific imaging is the dynamic range. This defines the ability of a camera to quantitatively image dim and bright signals in a single image. It is a function of a pixel's ability to respond to incoming light prior to saturation and its read noise. Many commonly used scientific CCD cameras (based on Sony CCD sensors) have dynamic range of approximately 2000:1 (the full well capacity divided by the read noise, for example, 16000/8). It should be noted that the new Rolera Bolt camera has a dynamic range of approximately 4500:1 (~16000/3.5). Thus due to clever pixel design and low noise electronics, new scientific-grade CMOS sensors compete very well with standard CCD devices in terms of dynamic range.

CMOS Random Telegraph Noise

What is also noticeable in the short exposure images in Figures 2 and 4 is the difference in the noise in the CMOS versus the CCD. The noise in the CMOS images is salt-and-pepper in nature with both bright and dark speckles. These speckles result from random telegraph noise, a noise unique to CMOS sensors and included in the read noise. Telegraph noise results from certain pixels on the CMOS sensor that are noisier than average as their signal fluctuates high and low (salt and pepper, respectively) around the average signal [Reference Martin-Gonthier, Havard and Magnan5, Reference Wang, Rao, Mierop and Theuwissen6]. Because of this noise, a number of pixels have noise levels that fall outside the expected Gaussian distribution seen in scientific CCDs. This difference in noise type can affect the behavior of the noise when operations such as averaging are applied.

Figure 4: Zoomed-in expansion of CMOS image, highlighting random telegraph noise apparent at short exposures.

When multiple experimental images are averaged, often to reduce the noise to make the image clearer or reduce the error bars on a measurement, the noise is expected to decrease by the square root of the number of images averaged for random Gaussian noise. However, as Table 1 demonstrates, the telegraph noise causes the averaging of CMOS data to be less effective than for a scientific CCD with Gaussian read noise.

Table 1: Comparison of the number of experimental images that must be captured and averaged with CMOS and CCD cameras to achieve the same noise reductions.

The noise reductions in Table 1 were obtained as follows. Two stacks of the same number of bias images were obtained, and each stack was averaged. The two average images were then subtracted from each other so that only camera read noise remained as the dominant noise source. The standard deviation, that is, read noise, across this subtracted average image was measured and divided into the standard deviation from a single subtracted image to calculate noise reduction relative to a single image (images not shown). In each row it is apparent that the number of CMOS sample images needed to reduce the noise by the given amount is more than the CCD. Although it is not realistic in many situations to acquire more than 25 images of the same cell due to photobleaching and dynamic cellular changes, CMOS cameras' non-Gaussian read noise would also add variation to the intensity measured even for a single acquisition on a single cell. Therefore, when averaging the intensities from several cells (as is often done in many scientific cell imaging studies) to reduce this variation (and its associated error bars) to distinguish, for example, an experimental sample from a control sample, the noise reduction for a CMOS would still be less effective than a CCD. Simply put, when trying to distinguish similar signals by reducing noise through averaging many experimental samples, a few more images and/or experiments may be needed when using a CMOS camera. This would only be an issue when the research outcomes require differentiation of very small differences in signal. In such scenarios it may be of value to continue to use CCD technology.