Introduction
Ever since Sir Isaac Newton’s observations on light and color [Reference Newton1], scientists have attempted to create systems to objectively classify colors and their reproduction. In the paint and color reproduction industries, color reproduction is well quantified and managed [Reference Varley and Harpur2]. Unfortunately, the use and display of color in brightfield microscopy is not standardized. In particular, several studies have pointed out the lack of a standardized system of color management in medical and histological imaging [Reference Badano3–Reference Penczek9]. A parametric study by Penczek et al. reported the occurrence of condition-dependent color errors in digital imaging systems employed for telemedicine [Reference Penczek9].
The problem faced by microscopy is management of color among different microscopes, sources of illumination, image sensors, displays, and display technologies. Ideally, brightfield micrographs taken of the same sample, but on different instruments, at different times, by different operators, should consistently display the colors and color variations of the sample. This problem is made more serious by the desire to quantify color and brightness [Reference Skandarajah10]. Standardization and calibration of microscopes and the associated imaging chain is the solution to producing consistency and color constancy in micrographic imaging [Reference Hirschorn11].
In order to create a robust color management system, one must consider each step of the imaging process: image capture, processing, and display. Color variations in micrographs are usually due to changes in specimen thickness, differences in staining, variations in image acquisition systems, and post-image processing and display [Reference Yagi4]. While great care may be taken to minimize artifacts generated by sectioning and staining samples, typically little regard is given to the management of color information during image acquisition and display. An attempt to have pathologists agree on a standardized color management system failed in part because of disagreement over what the ideal color for a stain should be [Reference Badano3]. Pathologists and microscopists use standardized protocols for fixation and staining of tissues in order to ensure the consistent production of high-quality tissue sections for light microscopy. Several groups have made the case that there is a need to standardize the imaging side of data collection from microscopes [Reference Yagi and Gilberson5–Reference Kayser and Kayser8].
Adequate color management begins with the realization that a microscope is a dynamic system that needs be calibrated and standardized in the same way as any other instrument in the laboratory. An integrated color management system should provide a record of the daily changes in the microscope, illumination, camera, and image display that affect the rendition of color in the final micrograph. A typical imaging chain for brightfield microscopy (Figure 1) can be described as being comprised of three image-handling operations: image acquisition, image processing, and image display.
Figure 1 Typical imaging chain of brightfield microscopy. The imaging chain consists of three image-handling operations: image acquisition, image processing, and image display.
Image acquisition
During the acquisition phase of the imaging chain, color reproduction is affected by the light source, voltage, optical flare inherent in the microscope optics, ambient illumination, filters and other light modifiers, and the image sensor. These variables can be managed through the use of a standardized slide. The standardized slide consists of a micrographic slide with a range of color and intensity targets whose values have been empirically measured. The values from the standardized slide obtained through the imaging system can be compared with the known values of the target patches. The differences in hue, saturation, and intensity between the values obtained with a particular imaging system and known standardized values can be used to correct images collected from that system. An image of a standardized slide serves as a record of the sum of variations in illumination, microscope, and image sensor. The standardized slide also serves as an indicator of linearity within the imaging system.
Image processing
The image processing part of the imaging chain is the least transparent aspect of digital imaging. Several post-capture factors, from operator to hardware, can impinge on image quality and color fidelity. Certain scientific imaging software systems apply hidden proprietary algorithms to the image captured by the software. In addition, microscope operators may attempt to tune contrast and tonal range in order to improve image display and analysis. An integrated color management system consists of a set of hardware and software controls to monitor and correct variations in the final image due to changes in illumination, camera, software, and display. Integrated color management of the imaging chain provides a mechanism to calibrate color rendering so that colors and color relationships from the original subject are maintained. An integrated color management system also provides the capacity to detect hidden color artifacts generated by image adjustment algorithms concealed within the imaging capture software.
Image display
Display and analysis of most micrographic data is performed on computer monitors, yet Krupinski et al. found a lack of standardized methods for characterization, calibration, and profiling computer displays [Reference Krupinski7]. In keeping with the goal of producing a micrograph that faithfully reproduces morphological data and color data with minimal artifacts, it is imperative that display systems are tuned and calibrated for color constancy [Reference Skandarajah10].
Materials and Methods
A commercially prepared hematoxylin and eosin (H&E) stained histologic section of a mammalian ovary (Ward’s Scientific catalog number 96 W 5532) was used as a test specimen in order to assess the capabilities of a commercially available integrated color management system provided by Datacolor, Inc. [Reference Foster and Sedgewick12]. Within the ovary, a readily identified antral follicle was selected as a marker in order to image the same area in each microscope system. The test specimen and a Datacolor ChromaCal calibration slide, Datacolor catalog number 10-1020 (serial number 1000162), were imaged using three microscope/camera/software combinations.
The first microscope system consisted of an Olympus BX41 microscope equipped with an Olympus UPLAN APO 10× objective, with a numerical aperture of 0.4 and a 6V/30W halogen light source. The BX41 microscope was equipped with a Lumenera Infinity 1 Camera controlled by Infinity Capture software version 6.4.0. The camera was coupled to the microscope using a 0.5× relay lens. Images were captured after adjusting the brightfield illumination to Köhler alignment. No other filtration, light modifier, or alignment system was used in conjunction with the BX41 microscope.
The second microscope/camera combination consisted of an Olympus CH30 microscope equipped with an Olympus EA10× objective, with a numerical aperture of 0.25 and a 6V/20W halogen light source. The CH30 microscope was equipped with a PointGrey Flea USB 3.0 camera controlled by Fly Capture 2 software version 2.6.3.4. The camera was coupled to the microscope by an Olympus NFK 2.5× LD relay lens. Images were captured after adjusting the brightfield illumination to Köhler alignment. No other filtration, light modifier, or alignment system was used in conjunction with the CH30 microscope.
The third microscope/camera combination consisted of a Nikon Alphaphot-2 microscope equipped with a Nikon E 10× objective, with a numerical aperture of 0.25 and a 100V/30W tungsten light source. The Alphaphot-2 was equipped with a Jenoptik ProgRes C3 camera controlled by ProgRes Mac Capture Pro 2013 software version 2.7.6. The camera was coupled directly to the trinocular microscope head by a phototube with no intervening relay lens. Images were captured after adjusting the brightfield illumination to Köhler alignment. An ND blue filter was used in conjunction with the Nikon microscope; the blue filter along with a gamma adjustment from 1.0 to 1.2 was required in order to achieve acceptable color linearity.
System linearity and color reproduction were analyzed using Datacolor ChromaCal Image Calibration software version 2.1 in conjunction with a ChromaCal color calibration slide. Image analysis was performed on a Dell Optiplex 7010 core i7 computer running a Windows 7 Enterprise 64-bit operating system. All computer monitors were calibrated for ambient illumination and color gamut using a ChromaCal monitor calibration sensor (colorimeter) and ChromaCal monitor calibration software.
Results
Figure 2 illustrates the linearity of three different microscope systems as calculated by the ChromaCal software. Linear response of the imaging system is a requirement for the subsequent quantitative colorimetric evaluation of biological specimens [Reference Garcia13, Reference Linden14]. All three microscopes were operating within normal limits for linearity. However, it should be noted that the Nikon/Jenoptik combination required adjustment of gamma from 1.0 to 1.2 and the addition of an ND blue filter in order to reach acceptable linearity limits for the ChromaCal software.
Figure 2 Linearity reports generated by ChromaCal software for each microscope/camera combination. The graphs illustrate the near-linear correlation of calibration slide colors, as plotted in XYZ tri-stimulus color space, against the sRGB color values recorded by the microscope/camera combinations. (a) Olympus BX41 microscope equipped with a Lumenera Infinity camera. (b) Olympus CH30 microscope equipped with a PointGrey Flea 3.0 USB camera. (c) Nikon Alphaphot-2 microscope equipped with a Jenoptik ProgRes C3 camera.
After optical alignment of the microscopes to Köhler illumination, an image was made of the same antral follicle followed by an image of the ChromaCal calibration slide. The calibration slide image was acquired using identical exposure parameters to the antral follicle image. Figure 3 compares the images of the antral follicle, as they were collected from the microscope software (column A), and after post-color standardization by ChromaCal software (column B). The improvement of color consistency and image contrast achieved through the use of the ChromaCal software is readily apparent between pre- and post-calibration. The most dramatic results were evident in the pre- and post-calibration images taken with the Nikon/Jenoptik/ProgRes imaging system.
Figure 3 Images of the same H&E stained, mammalian ovary, antral follicle. (a) Uncorrected image from the Olympus BX41/Lumenera microscope/camera combination. Image width =1206 µm. (b) Same image after color correction, gamma adjustment, and calibration. (c) Uncorrected image from the Olympus CH30/PointGrey microscope/camera combination. Image width =683 µm. (d) Same image after color correction, gamma adjustment, and calibration. (e) Uncorrected image from the Nikon/Jenoptik microscope/camera combination. Image width = 759 µm. (f) Same image after color correction, gamma adjustment, and a calibration. Color correction and gamma adjustments were performed using ChromaCal software.
Discussion
Prior to the advent of digital microscopy, color management consisted of ensuring the color temperature of the illuminant matched the color temperature balance of the photographic film. Color micrographs were captured using color transparency film or color negative film. The assumption was that the film processing chemistry was maintained within acceptable standards of color balance. The advent of digital imaging improved imaging workflow and allowed for near instantaneous distribution of color micrographs. Digital acquisition of micrographs required fundamental changes in workflow and calibration. With film-based imaging, manufacturers ensured consistent color and intensity response from the sensor (film). While there was variation between film brand, the reproduction of color was generally acceptable. Digital image acquisition requires active management of color value, gamma, and saturation. Digital color management needs to address the variation in sensors, light sources, and display media. Tani et al. reported that variation in color reproduction could affect the interpretation of H&E stained images [Reference Tani15]. The variability of digital imaging and capture was effectively demonstrated by Liron Pantanowitz in a study where six practicing pathologists were asked to photograph the same region on a glass slide using similar microscopes and the same attached camera systems, at the same magnification, resulting in six very different micrographs [Reference Pantanowitz16]. Kldiashvili noted that there are no established and accepted methods of accessing image quality and accuracy, including true color reproduction [Reference Kldiashvili17]. This finding was corroborated by Quigley et al. in an extensive literature search of over 7,371 unique publications spanning from 1985 to 2013 [Reference Quigley18]. Bautista et al. noted, in reference to whole slide scanners, that color consistency is an issue between different imaging systems [Reference Bautista19]. Several groups have attempted to address the digital calibration problem by using homemade calibration slides, image correction algorithms, International Color Consortium (ICC) profiles, 18% gray targets, gray scales, and color patches [Reference Benjnordi20–Reference Revie26]. None of the previous studies resulted in a widely adopted and verifiable system. It is past time to adopt a tenable system of color control across imaging modalities that affect clinical and scientific endeavors.
Mobile devices are now being deployed as primary microscopy imaging systems in education and the developing world [Reference Smith27, Reference Akst28]. The standardization of color in micrographs and scientific photographs will facilitate improved outcomes in education, communication, and the efficiency of medical diagnosis [Reference Lee and Plataniotis29]. Two studies that examined the influence of color management and display calibration showed that there was a significant improvement to the workflow of diagnostic pathology when color management systems were in place [Reference Krupinski7, Reference Bautista19].
In this study, calibration and standardization of three different microscope systems allowed for the consistent display of color and density information. The ChromaCal system provides the color framework for high-level image fidelity as called for by Badano et al. [Reference Badano3]. Under this framework, a calibration slide was imaged under the same parameters as the sample. Both the color and tonal range of images were corrected based on the calibration slide inputs. Finally, images were displayed on monitors calibrated using a colorimeter and software provided by Datacolor as part of the color management system.
The minor color and density differences in the color-corrected images are due to multiple factors. Some of the color variations can be attributed to differences in the color gamut of the cameras used to image the samples. Some variation was due to the microscopes being housed in different environments, ranging from a dedicated laboratory to a teaching laboratory, consequently flare from extraneous light sources affected the final images. As can be seen in Figure 2, the images produced on the Olympus BX41 microscope produced fairly parallel linearity plots, however, even that camera/microscope system suffered from some non-linearity in color reproduction. The ChromaCal software was able to correct the images to a reasonable facsimile of the original color and contrast of the specimen. The aim of color calibration is not to produce a perfect color match between the specimen and the display; rather, the function of calibration is to bring the imaging chain into controlled and acceptable limits of color reproduction. Every imaging and display system is limited by the color gamut it can record or display. This project showed that a commercially available, standardized slide can be used to standardize the color output of imaging chains from capture to display of the final images. As the use of digital capture grows in science, medicine, and education, the need for color standardization among imaging chains becomes vital.
The calibration slide image provides the means to monitor linearity of acquisition and manage color throughout the imaging chain. Monitor calibration ensures that images displayed on different monitors or different computers remain within acceptable color tolerances with little variation between displays.
Fixation, sectioning, and staining protocols have been standardized in order to improve diagnostic throughput and to minimize artifacts. It is time to apply the same logic to the capture, processing, and display of micrographs. Much like temperature, color data should be managed as a standard variable. As Figure 3 shows, implementation of a commercial color management system, given three very different inputs, allowed for a consistent and realistic depiction of the tissue section within acceptable tolerances. The system provided color management throughout the imaging chain from image acquisition to image display.
Future research should include an examination of mobile devices and field microscopes used in clinical and scientific applications. Mobile devices offer relatively high image quality, convenience, and low cost. As consumer devices continue to develop, improvements are being made in sensors, algorithms, and display of images. The use of mobile devices in science and medicine will only continue to spread. However, because there are no studies evaluating linearity and color constancy in mobile devices, future work should be directed toward determining their utility in scientific and medical imaging.
Conclusion
Implementation of a color management system does not eliminate color artifacts; it serves as a means to manage and reduce the effect of artifacts throughout the imaging chain. Imaging of a calibration slide during sample acquisition serves as a control for color reproduction and increases the information content of the image. Color management serves as means of normalization among microscopes, microscopists, and imaging systems. A critical aspect of a system for describing and standardizing color is the ability to produce a stable output when presented with varying specimen illumination, microscope optics, camera characteristics, and primary capture software. An integrated color management system provides the necessary control and normalization of data so that color can be viewed as either a standardized variable or perhaps a quantitative dependent variable in brightfield microscopy.
Acknowledgements
Datacolor, Inc. provided a ChromaCal color management system consisting of a calibration slide, monitor colorimeter, image calibration software, and monitor calibration software for this project.
