Perception Metrology

Part III - Perception Metrology

Published online by Cambridge University Press: 20 December 2018

Edited by

Ehsan Samei and

Elizabeth A. Krupinski

Show author details

Ehsan Samei: Affiliation:
Duke University Medical Center, Durham
Elizabeth A. Krupinski: Affiliation:
Emory University, Atlanta

Book contents

Get access

Summary

A summary is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.

Image of the first page of this content. For PDF version, please use the ‘Save PDF’ preceeding this image.'

Type: Chapter
Information: The Handbook of Medical Image Perception and Techniques , pp. 213 - 304

DOI: https://doi.org/10.1017/9781108163781 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abadi, E., Sturgeon, G.M., Agasthya, G., Harrawood, B., Kapadia, A., Segars, W.P., Samei, E. (2017). Airways, vasculature, and interstitial tissue: anatomically-informed computational modeling of human lungs for virtual clinical trials. Proc SPIE Medical Imag, 10132.Google Scholar

Bamber, D. (1975). Area above ordinal dominance graph and area below receiver operating characteristic graph. J Math Psych, 12, 387–415.CrossRef Google Scholar

Boedeker, K.L., McNitt-Gray, M.F. (2007). Application of the noise power spectrum in modern diagnostic MDCT: part II. Noise power spectra and signal to noise Phys Med Biol, 52, 4047–4061.Google Scholar

Burgess, A.E. (1995). Comparison of receiver operating characteristic and forced choice observer performance measurement methods. Med Phys, 22, 643–655.Google Scholar

Burgess, A.E. (2011). Visual perception studies and observer models in medical imaging. Semin Nucl Med, 41, 419–436.Google Scholar

Chawla, A.S., Samei, E. (2007). Ambient illumination revisited: a new adaptation-based approach for optimizing medical imaging reading environments. Med Phys, 34, 81–90.Google Scholar

Chawla, A.S., Samei, E., Saunders, R.S., Lo, J.Y., Baker, J.A. (2008). A mathematical model platform for optimizing a multiprojection breast imaging system. Med Phys, 35, 1337–1345.Google Scholar

Christianson, O., Chen, J., Yang, Z., Saiprasad, G., Dima, A., Filliben, J., Peskin, A., Trimble, C., Siegel, E., Samei, E. (2015). An improved index of image quality for task-based performance of CT iterative reconstruction across three commercial implementations. Radiology, 275, 725–734.Google Scholar

Dobbins, J.T., McAdams, H.P., Song, J.W., Li, C.M., Godfrey, D.J., DeLong, D.M., Paik, S.H., Martinez-Jimenez, S. (2008). Digital tomosynthesis of the chest for lung nodule detection: interim sensitivity results from an ongoing NIH-sponsored trial. Med Phys, 35, 2554–2557.CrossRef Google Scholar

Dorfman, D.D., Berbaum, K.S., Metz, C.E. (1992 ). Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. Invest Radiol, 27, 723–731.Google Scholar

Egglin, T.K., Feinstein, A.R. (1996). Context bias. A problem in diagnostic radiology. JAMA, 276, 1752–1755.Google Scholar

Green, D.M., Swets, J.A. (1966). Signal Detection Theory and Psychophysics. New York, NY: Wiley.Google Scholar

Hanley, J.A., McNeil, B.J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143, 29–36.Google Scholar

Hillis, S., Berbaum, K.S. (2006). MRMC sample size program. http://perception.radiology.uiowa.edu (accessed January 9, 2018).Google Scholar

Hoe, C.L., Samei, E., Frush, D.P., Delong, D.M. (2006). Simulation of liver lesions for pediatric CT. Radiology, 238, 699–705.Google Scholar

Ko, J.P., Rusinek, H., Jacobs, E.L., Babb, J.S., Betke, M., McGuinness, G., Naidich, D P. (2003). Small pulmonary nodules: volume measurement at chest CT – phantom study. Radiology, 228, 864–870.Google Scholar

Metz, C. (2000). Fundamental ROC analysis. In: Beutel, J., Kundel, H.L., Van Metter, R.L. (eds.) Handbook of Medical Imaging. Vol. 1, Physics and Psychophysics. Bellingham, WA: SPIE Press, pp. 751–769.Google Scholar

Obuchowski, N.A. (2000). Sample size tables for receiver operating characteristic studies. AJR Am J Roentgenol, 175, 603–608.Google Scholar

Obuchowski, N.A. (2004). How many observers are needed in clinical studies of medical imaging? AJR Am J Roentgenol, 182, 867–869.Google Scholar

Pokrajac, D.D., Maidment, A.D.A., Bakic, P.R. (2012). Optimized generation of high resolution breast anthropomorphic software phantoms. Med Phys, 39, 2290–2302.Google Scholar

Pollard, B.J., Chawla, A.S., Delong, D.M., Hashimoto, N., Samei, E. (2008). Object detectability at increased ambient lighting conditions. Med Phys, 35, 2204–2213.Google Scholar

Samei, E., Flynn, M.J., Eyler, W.R. (1997). Simulation of subtle lung nodules in projection chest radiography. Radiology, 202, 117–124.Google Scholar

Samei, E., Ranger, N.T., Delong, D.M. (2008). A comparative contrast-detail study of five medical displays. Med Phys, 35, 1358–1364.CrossRef Google Scholar PubMed

Samei, E., Lin, Y., Choudhury, K.R., McAdams, H.P. (2014). Automated characterization of perceptual quality of clinical chest radiographs: validation and calibration to observer preference. Med Phys, 41, 111918.Google Scholar

Saunders, R.S., Samei, E., Hoeschen, C. (2004). Impact of resolution and noise characteristics of digital radiographic detectors on the detectability of lung nodules. Med Phys, 31, 1603–1613.Google Scholar

Saunders, R., Samei, E., Baker, J., Delong, D. (2006). Simulation of mammographic lesions. Acad Radiol, 13, 860–870.Google Scholar

Schindera, S.T., Nelson, R.C., Mukundan, S., Paulson, E.K., Jaffe, T.A., Miller, C.M., DeLong, D.M., Kawaji, K., Yoshizumi, T.T., Samei, E. (2008). Hypervascular liver tumors: low tube voltage, high tube current multi-detector row CT for enhanced detection – phantom study. Radiology, 246, 125–132.Google Scholar

Segars, W.P., Mahesh, M., Beck, T.J., Frey, E.C., Tsui, B.M. (2008). Realistic CT simulation using the 4D XCAT phantom. Med Phys, 35, 3800–3808.CrossRef Google Scholar PubMed

Solomon, J., Samei, E. (2014). A generic framework to simulate realistic lung, liver and renal pathologies in CT imaging. Phys Med Biol, 59, 6637–6657.Google Scholar

Solomon, J., Samei, E. (2016). Correlation between human detection accuracy and observer model-based image quality metrics in CT. J Med Imag, 3, 035506.Google Scholar

Swets, J.A., Pickett, R.M. (1982). Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York, NY: Academic Press.Google Scholar

Thornbury, J.R., Fryback, D.G., Turski, P.A., Javid, M.J., McDonald, J.V., Beinlich, B.R., Gentry, L.R., Sackett, J.F., Dasbach, E.J., Martin, P.A. (1993). Disk-caused nerve compression in patients with acute low-back pain: diagnosis with MR, CT myelography, and plain CT. Radiology, 186, 731–738.Google Scholar

Venjakoba, A.C., Mello-Thoms, C.R. (2016). Review of prospects and challenges of eye tracking in volumetric imaging. J Med Imag, 3, 011002.Google Scholar

Zhou, X.-H., Obuchowski, N.A., McClish, D.K. (2002). Statistical Methods in Diagnostic Medicine. New York, NY: Wiley-Interscience.Google Scholar

References

Aoki, K., Misumi, J., Kimura, T., Zhao, W., Xie, T. (1997). Evaluation of cutoff levels for screening of gastric cancer using serum pepsinogens and distributions of levels of serum pepsinogen I, Ii and of Pg I/Pg Ii ratios in a gastric cancer case-control study. J Epidemiol, 7, 143–151.Google Scholar

Bamber, D. (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J Math Psychol, 12, 387–415.Google Scholar

Begg, C.B., Greenes, R.A. (1983). Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics, 39, 207–215.Google Scholar

Beiden, S.V., Campbell, G., Meier, K.L., Wagner, R.F. (2000a). The problem of ROC analysis without truth: the EM algorithm and the information matrix. Proc SPIE, 3981, 126–134.CrossRef Google Scholar

Beiden, S.V., Wagner, R.F., Campbell, G. (2000b). Components-of-variance models and multiple-bootstrap experiments: an alternative method for random effects, receiver operating characteristic analysis. Acad Radiol, 7, 341–349.Google Scholar

Cortes, C., Mohri, M. (2003). AUC optimization vs. error rate. In: Advances in Neural Information Processing Systems 16: Proceedings of the 2003 Conference. Cambridge, MA: MIT Press.Google Scholar

Delong, E.R., Delong, D.M., Clarke-Pearson, D.L. (1988). Comparing the areas under two or more correlated receiver operating characteristics curves: a non-parametric approach. Biometrics, 44, 837–845.Google Scholar

Deneef, P., Kent, D.L. (1993). Using treatment-tradeoff preferences to select diagnostic strategies. Med Decis Making, 13, 126–132.Google Scholar

Dorfman, D.D., Alf, E. (1968). Maximum likelihood estimation of parameters of signal detection theory: a direct solution. Psychometrika, 33, 117–124.Google Scholar

Dorfman, D.D., Alf, E. (1969). Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals – rating method data. J Math Psychol, 6, 487–496.Google Scholar

Dorfman, D.D., Berbaum, K.S., Metz, C.E. (1992). Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Invest Radiol, 27, 723–731.Google Scholar

Dorfman, D.D., Berbaum, K.S., Metz, C.E., Lenth, R.V., Hanley, J.A., Abu Dagga, H. (1997). Proper receiver operating characteristic analysis: the bigamma model. Acta Radiol, 4, 138–149.Google Scholar

Dwyer, A.J. (1997). In pursuit of a piece of the ROC. Radiology, 202, 621–625.Google Scholar

Efron, B., Tibshirani, R.J. (1993). An Introduction to the Bootstrap. New York, NY: Chapman and Hall.Google Scholar

Faraggi, D., Reiser, B. (2002). Estimation of the area under the ROC curve. Statistics Med, 21, 3093–3106.Google Scholar

Goddard, M.J., Hinberg, I. (1990). Receiver operating characteristic (ROC) curves and non-normal data: an empirical study. Statistics Med, 9, 325–337.Google Scholar

Greiner, M., Sohr, D., Gobel, P. (1995). A modified ROC analysis for the selection of cut-off values and the definition of intermediate results for serodiagnostic tests. J Immunol Methods, 185, 123–132.Google Scholar

Grmec, I., Kupnik, D. (2004). Does the Mainz emergency evaluation scoring (MEES) in combination with capnometry (MEESC) help in the prognosis of outcome from cardiopulmonary resuscitation in a prehospital setting? Resuscitation, 58, 89–96.CrossRef Google Scholar

Hajian-Tilaki, K.O., Hanley, J.A., Joseph, L., Collet, J.P. (1997). A comparison of parametric and nonparametric approaches to ROC analysis of quantitative diagnostic tests. Med Decis Making, 17, 94–102.Google Scholar

Halpern, E.J., Albert, M., Krieger, A.M., Metz, C.E., Maidment, A.D. (1996). Comparison of receiver operating characteristic curves on the basis of optimal operating points. Acad Radiol, 3, 245–253.Google Scholar

Hand, D.J., Till, R.J. (2001). A simple generalization of the area under the ROC curve to multiple class classification problems. Machine Learn, 45, 171–186.Google Scholar

Hanley, J.A. (1988). The robustness of the “binormal” assumptions used in fitting ROC curves. Med Decis Making, 8, 197–203.Google Scholar

Hanley, J.A. (1996). The use of the ‘‘binormal’’ model for parametric ROC analysis of quantitative diagnostic tests. Statistics Med, 15, 1575–1585.Google Scholar

Hanley, J.A., Hajian-Tilaki, K.O. (1997). Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: an update. Acad Radiol, 4, 49–58.Google Scholar

Hanley, J.A., McNeil, B.J. (1982). The meaning and use of the area under a receiver operating characteristic curve. Radiology, 143, 29–36.Google Scholar

Hanley, J.A., McNeil, B.J. (1983). A method for comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology, 148, 839–843.Google Scholar

Harrell, F.E., Jr., Califf, R.M., Pryor, D.B., Lee, K.L., Rosati, R.A. (1982). Evaluating the yield of medical tests. JAMA, 247, 2543–2546.Google Scholar

Henkelman, R.M., Kay, I., Bronskill, M.J. (1990). Receiver operator characteristic (ROC) analysis without truth. Med Decis Making, 10, 24–29.Google Scholar

Ikeda, M., Ishigaki, T., Yamauch, K. (2003). How to establish equivalence between two treatments in ROC analysis. Proc SPIE, 5034, 383–392.Google Scholar

Jiang, Y., Metz, C.E., Nishikawa, R.M. (1996). A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology, 201, 745–750.Google Scholar

Johnson, W.O., Gastwirth, J.L., Pearson, L.M. (2001). Screening without a “gold standard”: the Hui-Walter paradigm revisited. Am J Epidemiol, 153, 921–924.Google Scholar

Kairisto, V., Poola, A. (1995). Software for illustrative presentation of basic clinical characteristics of laboratory tests – Graphroc for Windows. Scand J Clin Lab Invest, 55, 43–60.Google Scholar

Kijewski, M.F., Swennson, R.G., Judy, P.F. (1989). Analysis of rating data from multiple-alternative tasks. J Math Psychol, 33, 1–23.Google Scholar

Lee, W.C. (1999). Probabilistic analysis of global performances of diagnostic tests: interpreting the Lorenz curve-based summary measures. Statistics Med, 18, 455–471.Google Scholar

Lee, W.C., Hsiao, C.K. (1996). Alternative summary indices for the receiver operating characteristic curve. Epidemiology, 7, 605–611.Google Scholar

Li, C.R., Liao, C.-T., Liu, J.-P. (2008). A non-inferiority test for diagnostic accuracy based on the paired partial areas under ROC curves. Statistics Med, 27, 1762–1776.Google Scholar

Liu, J.-P., Ma, M.-C., Wu, C.-Y., Tai, J.-Y. (2006). Tests of equivalence and non-inferiority for diagnostic accuracy based on the paired areas under ROC curves. Statistics Med, 25, 1219–1238.Google Scholar

Lusted, L.B. (1960). Logical analysis in roentgen diagnosis. Radiology, 74, 78–93.Google Scholar

Lusted, L.B. (1961). Signal detectability and medical decision making. Science, 171, 1217–1219.Google Scholar

McClish, D.K. (1989). Analyzing a portion of the ROC curve. Med Decis Making, 9, 190–195.Google Scholar

Metz, C.E. (1978). Basic principles of ROC analysis. Semin Nucl Med, 8, 283–298.Google Scholar

Metz, C.E. (1986a). Statistical analysis of ROC data in evaluating diagnostic performance. In: Herbert, D., Myers, R. (eds.) Multiple Regression Analysis: Applications in the Health Sciences. New York, NY: American Institute of Physics, pp. 365–384.Google Scholar

Metz, C.E. (1986b). ROC methodology in radiologic imaging. Invest Radiol, 21, 720–733.Google Scholar

Metz, C.E., Kronman, H.B. (1980). Statistical significance tests for binormal ROC curves. J Math Psychol, 22, 218–243.Google Scholar

Metz, C.E., Pan, X. (1999). “Proper” binormal ROC curves: theory and maximum-likelihood estimation. J Math Psychol, 43, 1–33.Google Scholar

Metz, C.E., Wang, P.-L., Kronman, H.B. (1984). A new approach for testing the significance of differences between ROC curves measured from correlated data. In: Deconinck, F. (ed.) Information Processing in Medical Imaging. The Hague: Nijhoff, pp. 432–445.Google Scholar

Metz, C.E., Herman, B.A., Shen, J.-H. (1998). Maximum-likelihood estimation of ROC curves from continuously-distributed data. Statistics Med, 17, 1033–1053.Google Scholar

Miller, D.P., O’Shaughnessy, K.F., Wood, S.A., Castellino, R.A. (2004). Gold standards and expert panels: a pulmonary nodule case study with challenges and solutions. Proc SPIE, 5372, 173.CrossRef Google Scholar

Mossman, D. (1999). Three-way ROCs. Med Decis Making, 19, 78–89.Google Scholar

Obuchowski, N.A. (1994). Sample size for receiver operating characteristic studies. Invest Radiol, 29, 238–243.Google Scholar

Obuchowski, N.A. (1997). Testing for equivalence of diagnostic tests. Am J Radiol, 168, 13–17.Google Scholar

Obuchowski, N.A. (2000). Sample size tables for receiver operating characteristic studies. Am J Roentgenol, 175, 603–608.Google Scholar

Obuchowski, N.A. (2005). Multi-reader multi-modality ROC studies: hypothesis testing and sample size estimation using an ANOVA approach with dependent observations. Acad Radiol, 2, 522–529.Google Scholar

Obuchowski, N.A. (2006). An ROC-type measure of diagnostic accuracy when the gold standard is continuous-scale. Statistics Med, 25, 481–493.Google Scholar

Obuchowski, N.A., Liebler, M.L. (1998). Confidence intervals for the receiver operating characteristic area in studies with small samples. Acad Radiol, 5, 561–571.Google Scholar

Obuchowski, N.A., Goske, M.J., Applegate, K.E. (2001). Assessing physicians’ accuracy in diagnosing pediatric patients with acute abdominal pain: measuring accuracy for multiple diseases. Statistics Med, 20, 3261–3278.Google Scholar

Pepe, M.S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford, UK: Oxford University Press.Google Scholar

Petrick, N., Gallas, B.D., Samuelson, F.W., Wagner, R.F., Myers, K.J. (2005). Influence of panel size and expert skill on truth panel performance when combining expert ratings. Proc SPIE, 5749, 49.Google Scholar

Phelps, C.E., Hutson, A. (1995). Estimating diagnostic test accuracy using a “fuzzy gold standard.” Med Decis Making, 15, 44–57.Google Scholar

Schafer, H. (1989). Constructing a cut-off point for a quantitative diagnostic test. Statistics Med, 8, 1381–1391.Google Scholar

Schoonjans, F., Zalata, A., Depuydt, C.E., Comhaire, F.H. (1995). Medcalc: a new computer program for medical statistics. Comput Methods Programs Biomed, 48, 257–262.Google Scholar

Schisterman, E.F., Perkins, N.J., Aiyi, L., Bondell, H. (2005). Optimal cut-point and its corresponding Youden index to discriminate individuals using pooled blood samples. Epidemiology, 16, 73–81.Google Scholar

Schuirmann, D.U.I. (1987). A comparison of the two 1-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. J Pharmacokinet Pharmacodyn, 15, 657–680.Google Scholar

Stephan, C., Wesseling, S., Schink, T., Jung, K. (2003). Comparison of eight computer programs for receiver-operating characteristic analysis. Clin Chem, 49, 433–439.Google Scholar

Swets, J.A. (1979). ROC analysis applied to the evaluation of medical imaging techniques. Invest Radiol, 14, 109–121.Google Scholar

Swets, J.A. (1986). Empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. Psychol Bull, 99, 181–198.Google Scholar

Swets, J.A. (1988). Measuring the accuracy of diagnostic systems. Science, 240, 1285–1293.Google Scholar

Swets, J.A. (1992). The science of choosing the right decision threshold in high-stakes diagnostics. Am Psychol, 47, 522–532.Google Scholar

Toledano, A.Y., Gatsonis, C. (1999). Generalized estimating equations for ordinal categorical data: arbitrary patterns of missing responses and missingness in a key covariate. Biometrics, 55, 488–496.Google Scholar

Vergara, I.A., Norambuena, T., Ferrada, E., Slater, A.W., Melo, F. (2008). StAR: a simple tool for the statistical comparison of ROC curves. BMC Bioinformatics, 9, 265.Google Scholar

Wagner, R.F., Beiden, C.V., Metz, C.E., Campbell, G. (2001). Continuous versus categorical data for ROC analysis: some quantitative considerations. Acad Radiol, 8, 328–334.Google Scholar

Wagner, R.F., Metz, C.E., Campbell, G. (2007). Assessment of medical imaging systems and computer aids: a tutorial review. Acad Radiol, 14, 723–748.Google Scholar

Walsh, S.J. (1999). Goodness-of-fit issues in ROC curve estimation. Med Decis Making, 19, 193–201.Google Scholar

Wieand, S., Gail, M.H., James, B.R., James, K.L. (1989). A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data. Biometrika, 76, 585–592.Google Scholar

Youden, W.J. (1950). Index for rating diagnostic tests. Cancer, 3, 32–35.3.0.CO;2-3>CrossRef Google Scholar PubMed

Zhang, D.D., Zhou, X.-H., Freeman, D.H., Jr., Freeman, J.L. (2002). A non-parametric method for the comparison of partial areas under ROC curves and its application to large health care data sets. Statistics Med, 21, 701–15.Google Scholar

Zhou, X.-H., Higgs, R.E. (2000). Assessing the relative accuracies of two screening tests in the presence of verification bias. Statistics Med, 19, 1697–1705.Google Scholar

Zhou, X.-H., Obuchowski, N.A., McClish, D.K. (2002). Statistical Methods in Diagnostic Medicine. New York, NY: Wiley.Google Scholar

Zou, K.H. (2001). Comparison of correlated receiver operating characteristic curves derived from repeated diagnostic test data. Acad Radiol, 8, 225–233.Google Scholar

Zou, K.H., Hall, W.J., Shapiro, D.E. (1997). Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests. Statistics Med, 16, 2143–2156.Google Scholar

Zou, K.H., Tempany, C.M., Fielding, J.R., Silverman, S.G. (1998). Original smooth receiver operating characteristic curve estimation from continuous data: statistical methods for analyzing the predictive value of spiral CT of ureteral stones. Acad Radiol, 5, 680–687.Google Scholar

Zou, K.H., Resnic, F.S., Talos, I.F., et al. (2005). A global goodness-of-fit test for receiver operating characteristic curve analysis via the bootstrap method. J Biomed Informatics, 38, 395–403.Google Scholar

References

Abbey, C.K., Samuelson, F.W., Gallas, B.D. (2013). Statistical power considerations for a utility endpoint in observer performance studies. Acad Radiol, 20, 798–806.Google Scholar

Abbey, C.K., Gallas, B.D., Boone, J.M., Niklason, L.T., Hadjiiski, L.M., Sahiner, B., Samuelson, F.W. (2014). Comparative statistical properties of expected utility and area under the ROC curve for laboratory studies of observer performance in screening mammography. Acad Radiol, 21, 481–490.Google Scholar

Bamber, D. (1975). Area above ordinal dominance graph and area below receiver operating characteristic graph. J Math Psychol, 12, 387–415.Google Scholar

DeLong, E.R., DeLong, D.M., Clarke-Pearson, D.L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44, 837–845.Google Scholar

Dorfman, D.D., Berbaum, K.S. (2000a). A contaminated binormal model for ROC data – part III. Initial evaluation with detection ROC data. Acad Radiol, 7, 438–447.Google Scholar

Dorfman, D.D., Berbaum, K.S. (2000b). A contaminated binormal model for ROC data – part II. A formal model. Acad Radiol, 7, 427–437.Google Scholar

Dorfman, D.D., Berbaum, K.S., Lenth, R.V., Chen, Y.F. (1999). Monte Carlo validation of a multireader method for receiver operating characteristic discrete rating data: split plot experimental design. SPIE Proc Ser, 3663, 91–99.Google Scholar

Dorfman, D.D., Berbaum, K.S., Brandser, E.A. (2000). A contaminated binormal model for ROC data – part I. Some interesting examples of binormal degeneracy. Acad Radiol, 7, 420–426.Google Scholar

Gallas, B.D. (2006). One-shot estimate of MRMC variance: AUC. Acad Radiol, 13, 353–362.Google Scholar

Gallas, B.D. (2017). iMRMC v3p3 Application for Analyzing and Sizing MRMC Reader Studies (computer software). Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, Silver Spring, MD. Available for download from https://github.com/DIDSR/iMRMC/releases.Google Scholar

Gallas, B.D., Bandos, A., Samuelson, F.W., Wagner, R.F. (2009). A framework for random-effects ROC analysis: biases with the bootstrap and other variance estimators. Commun Statistics – Theory Methods, 38, 2586–2603.Google Scholar

Hanley, J.A., McNeil, B.J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143, 29–36.Google Scholar

Hillis, S.L. (2007). A comparison of denominator degrees of freedom methods for multiple observer ROC analysis. Statistics Med, 26(3), 596–619.Google Scholar

Hillis, S.L. (2014). A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data. Statistics Med, 33, 330–360.Google Scholar

Hillis, S.L., Berbaum, K.S. (2005). Monte Carlo validation of the Dorfman-Berbaum-Metz method using normalized pseudovalues and less data-based model simplification. Acad Radiol, 12, 1534–1542.Google Scholar

Hillis, S.L., Schartz, K.M. (2015). Demonstration of multi- and single-reader sample size program for diagnostic studies software. In: Mello-Thoms, C.R., Kupinski, M.A. (eds.) Medical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment, Proc SPIE 9416 94160E.Google Scholar

Hillis, S.L., Obuchowski, N.A., Schartz, K.M., Berbaum, K.S. (2005). A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Statistics Med, 24, 1579–1607.Google Scholar

Hillis, S.L., Schartz, K.M., Berbaum, K.S. (2007). DBM MRMC 3.0 for SAS (computer software). Available at: http://perception.radiology.uiowa.edu.Google Scholar

Hillis, S.L., Berbaum, K.S., Metz, C.E. (2008). Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Acad Radiol, 15, 647–661.Google Scholar

Hillis, S.L., Obuchowski, N.A., Berbaum, K.S. (2011). Power estimation for multireader ROC methods: an updated and unified approach. Acad Radiol, 18, 129–142.Google Scholar

Jiang, Y.L., Metz, C.E., Nishikawa, R.M. (1996). A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology, 201, 745–750.Google Scholar

McClish, D.K. (1989). Analyzing a portion of the ROC curve. Med Decis Making, 9, 190–195.Google Scholar

Metz, C.E., Pan, X.C. (1999). “Proper” binormal ROC curves: theory and maximum-likelihood estimation. J Math Psychol, 43, 1–33.Google Scholar

Obuchowski, N.A. (1995). Multireader receiver operating characteristic studies: a comparison of study designs. Acad Radiol, 2, 709–716.Google Scholar

Obuchowski, N.A. (2009). Reducing the number of reader interpretations in MRMC studies. Acad Radiol, 16, 209–217.Google Scholar

Obuchowski, N.A., Rockette, H.E. (1995). Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an ANOVA approach with dependent observations. Commun Statistics – Simul Comput, 24, 285–308.Google Scholar

Obuchowski, N.A., Gallas, B.D., Hillis, S.L. (2012). Multi-reader ROC studies with split-plot designs: a comparison of statistical methods. Acad Radiol, 19, 1508–1517.Google Scholar

Pan, X.C., Metz, C.E. (1997). The ‘‘proper’’ binormal model: parametric receiver operating characteristic curve estimation with degenerate data. Acad Radiol, 4, 380–389.Google Scholar

Schartz, K.M., Hillis, S.L. (2018). Multi-reader sample size program for diagnostic studies (computer software). Available for download from http://perception.radiology.uiowa.edu.Google Scholar

Schartz, K.M., Hillis, S.L., Pesce, L.L., Berbaum, K.S. (2018). OR-DBM MRMC (Version 2.51) (computer software). Available for download from http://perception.radiology.uiowa.edu.Google Scholar

Shao, J., Dongshen, T. (1995). The Jackknife and Bootstrap. New York, NY: Springer-VerlagGoogle Scholar

Van Dyke, C.W., White, R.D., Obuchowski, N.A., Geisinger, M.A., Lorig, R.J., Meziane, M.A. (1993). Cine MRI in the diagnosis of thoracic aortic dissection. 79th RSNA Meetings, Chicago, IL, November 28–December 3.Google Scholar

Zeger, S.L., Liang, K.Y. (1986). Longitudinal data analysis for discrete and continuous outcomes. Biometrics, 42, 121–130.Google Scholar

References

Allard, F., Graham, S., Paarsalu, M.E., et al. (1980). Perception in sport: basketball. J Sport Psychol, 2(1), 14–21.Google Scholar

Alloway, T.P., Alloway, R.G. (2010). Investigating the predictive roles of working memory and IQ in academic attainment. J Exp Child Psychol, 106, 20–29.Google Scholar

Anderson, M.C. (1994). Remembering can cause forgetting: retrieval dynamics in long-term memory. J Exp Psychol Learn, 20(5), 1063.Google Scholar

Baddeley, A.D. (1986). Working Memory. Oxford, UK: Clarendon Press.Google Scholar

Baddeley, A.D. (2007). Working Memory, Thought, and Action (Vol. 45). Oxford, UK: OUP.Google Scholar

Baddeley, A.D., Hitch, G.J. (2000). Development of working memory: should the Pascual-Leone and the Baddeley and Hitch models be merged? J Exp Child Psychol, 77(2), 128–137.Google Scholar

Baddeley, A.D., Hitch, G.J. (2017). Is the levels of processing effect language-limited? J Mem Lang, 92, 1–13.Google Scholar

Balassy, C. (2005). Flat-panel display (LCD) versus high-resolution gray-scale display (CRT) for chest radiography: an observer preference study. Am J Roentgenol, 184(3), 752–756.Google Scholar

Brady, T.F., Konkle, T., Alvarez, G.A. (2008). Visual long-term memory has a massive storage capacity for object details. Proc Natl Acad Sci U S A, 105, 14325–14329.Google Scholar

Brady, T.F., Konkle, T., Alvarez, G.A., et al. (2011). A review of visual memory capacity: beyond individual items and toward structured representations. J Vision, 11(5), 4, 1–34.Google Scholar

Chase, W.G., Ericsson, K.A. (1982). Skill and working memory. Psychol Learn Motiv, 16, 1–58.Google Scholar

Chase, W.G., Simon, H.A. (1973). Perception in chess. Cogn Psychol, 4(1), 55–81.Google Scholar

Cohen, M.A., Evans, K.K., Horowitz, T.S., et al. (2011). Auditory and visual memory in musicians and nonmusicians. Psychonom B Rev, 18(3), 586–591.Google Scholar

Daneman, M., Carpenter, P.A. (1980). Individual differences in working memory and reading. J Verb Learn Verb B, 19, 450–466.Google Scholar

de Hoop, B., De Boo, D.W., Gieterna, H.A., et al. (2010). Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology, 257(2), 532–540.Google Scholar

de Smet, A.A., Norris, M.A., Yandow, D.R., et al. (1993). Diagnosis of meniscal tears of the knee with MR imaging: effect of observer variation and sample size on sensitivity and specificity. Am J Roentgenol, 160, 555–559.Google Scholar

de Vries, A.H., Venema, H.W., Florie, J., et al. (2008). Influence of tagged fecal material on detectability of colorectal polyps at CT: phantom study. Am J Roentgenol, 2008, W181–W189.Google Scholar

Evans, K.K., Cohen, M.A., Tambouret, R., et al. (2011). Does visual expertise improve visual recognition memory? Atten Percept Psychophys, 73(1), 30–35.Google Scholar

Evans, K.K., Haygood, T.M., Cooper, J., et al. (2016a). A half-second glimpse often lets radiologists identify breast cancer cases even when viewing the mammogram of the opposite breast. Proc Natl Acad Sci U S A, 113(37), 10292–10297.Google Scholar

Evans, K.K., Marom, E.M., Godoy, M.C.B., et al. (2016b). Radiologists remember mountains better than radiographs, or do they? J Med Imag, 3(1), 011005.Google Scholar

Frey, P.W., Adesman, P. (1976). Recall memory for visually presented chess positions. Mem Cognit, 4(5), 541–547.Google Scholar

Fuhrman, C.R., Britton, C.A., Bender, T., et al. (2002). Observer performance studies: detection of single versus multiple abnormalities of the chest. Am J Roentgenol, 179, 1551–1553.Google Scholar

Fukuda, K., Vogel, E.K., Mayr, U., et al. (2010). Quantity not quality: the relationship between fluid intelligence and working memory capacity. Psychonom B Rev, 17, 673–679.Google Scholar

Gardiner, J.M. (2001). Episodic memory and autonoetic consciousness: a first–person approach. Philos T R Soc B, 356(1413), 1351–1361.Google Scholar

Good, W.F., Sumkin, J.H., Dash, N., et al. (1999). Observer sensitivity to small differences: a multipoint rank-order experiment. Am J Roentgenol, 175, 275–278.Google Scholar

Hardesty, L.A., Ganott, M.A., Hakim, C.M., et al. (2005). “Memory effect” in observer performance studies of mammograms. Acad Radiol, 12(3), 286–290.Google Scholar

Haygood, T.M., Ryan, J., Liu, M.A.Q., et al. (2012). Image recognition and consistency of response. Proc SPIE 8318, Medical Imaging 2012: Image perception, observer performance, and technology assessment.Google Scholar

Haygood, T.M., Liu, M.A.Q., Galvan, E.M., et al. (2013). Memory for previously viewed radiographs and the effect of prior knowledge of memory task. Acad Radiol, 10, 1598–1603.Google Scholar

Henderson, J.M. (2003). Human gaze control during real-world scene perception. Trends Cogn Sci, 7(11), 498–504.Google Scholar

Hillard, A., Myles-Worsley, M., Johnston, W., et al. (1985). The development of radiologic schemata through training and experience: a preliminary communication. Invest Radiol, 18, 422–425.Google Scholar

Hollingworth, A. (2005). The relationship between online visual representation of a scene and long-term scene memory. J Exp Psychol Learn, 31, 396–411.Google Scholar

Hollingworth, A., Henderson, J.M. (2003). Testing a conceptual locus for the inconsistent object change detection advantage in real-world scenes. Mem Cognit, 31, 930–940.Google Scholar

Isola, P., Xiao, J., Parikh, D., et al (2014). What makes a photograph memorable? IEEE T Pattern Anal, 36(7), 1469–1482.Google Scholar

Jolicoeur, P. (1985). The time to name disoriented natural objects. Mem Cognit, 13(4), 289–303.Google Scholar

Kallergi, M., Pianou, N., Georgakopoulos, A., et al (2012). Quantitative evaluation of the memory bias effect in ROC studies with PET/CT. Proc SPIE, 8318, 83180D1-8.Google Scholar

Kandel, E.R., Schwartz, J.H., Jessell, T.M., et al. (2000). Principles of Neural Science, Vol. 4. New York: McGraw-Hill, pp. 1227–1246.Google Scholar

Kane, M.J., Bleckley, M.K., Conway, A.R.A., et al. (2001). A controlled-attention view of working-memory capacity. J Exp Psychol Gen, 130, 169–183.Google Scholar

Kim, A.Y., Cho, K.S., Song, K., et al. (2001). Urinary calculi on computed radiography: comparison of observer performance with hard-copy versus soft-copy images on different viewer systems. Am J Roentgenol, 177(2), 331–335.Google Scholar

Konkle, T., Brady, T.F., Alvarez, G.A., et al. (2010). Scene memory is more detailed than you think: the role of categories in visual long-term memory. Psychol Sci, 21, 1551–1556.Google Scholar

Koutstaal, W. (2003). Older adults encode – but do not always use – perceptual details: intentional versus unintentional effects of detail on memory judgments. Psychol Sci, 14(2), 189–193.Google Scholar

Landauer, T.K. (1986). How much do people remember? Some estimates of the quantity of learned information in long-term memory. Cogn Sci, 10(4), 477–493.Google Scholar

Lewandowsky, S. (2014). Implicit Memory: Theoretical Issues. New York, NY: Psychology Press.Google Scholar

Meisamy, S., Bolan, P.J., Baker, E.H., et al. (2005). Adding in vivo quantitative ¹H MR spectroscopy to improve diagnostic accuracy of breast MR imaging: preliminary results of observer performance study at 4.0 T. Radiology, 236, 465–467.Google Scholar

Mervis, C.B., Rosch, E. (1981). Categorization of natural objects. Annu Rev Psychol, 32(1), 89–115.Google Scholar

Metz, C.E. (1989). Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol, 24, 234–245.Google Scholar

Reiner, B.I., Krupinski, E. (2012). The insidious problem of fatigue in medical imaging practice. J Digit Imag, 25(1), 3–6.Google Scholar

Ryan, J.T., Haygood, T.M., Yamal, J., et al. (2011). The “memory effect” for repeated radiologic observations. Am J Roentgenol, 197, W985–W991.Google Scholar

Schacter, D.L. (1993). Implicit memory: a selective review. Annu Rev Neurosci, 16(1), 159–182.Google Scholar

Schacter, D.L., Tulving, E. (1994). What are the memory systems of 1994? In: Schacter, D.L., Tulving, E. (eds.) Memory Systems. Cambridge, MA: MIT Press.Google Scholar

Shepard, R.N. (1967). Recognition memory for words, sentences, and pictures. J Verb Learn Verb B, 6, 156–163.Google Scholar

Shiffrin, R.M., Steyvers, M. (1997). A model for recognition memory: REM – retrieving effectively from memory. Psychonom B Rev, 4(2), 145–166.Google Scholar

Sperling, G. (1963). A model for visual memory tasks. Hum Factors, 5(1), 19–31.Google Scholar

Squire, L.R. (1989). On the course of forgetting in very long-term memory. J Exp Psychol Learn, 15(2), 241.Google Scholar

Standing, L. (1973). Learning 10,000 pictures. Q J Exp Psychol, 25, 207–222.Google Scholar

Taylor-Phillips, S., Elze, M.D., Krupinski, E.A., et al. (2015). Retrospective review of the drop in observer detection performance over time in lesion-enriched experimental studies. J Digit Imag, 28(1), 32–40.Google Scholar

Tchou, P.M., Haygood, T.M., Atkinson, E.N., et al. (2010). Interpretation time of computer-aided detection at screening mammography. Radiology, 257, 40–46.Google Scholar

Tulving, E. (2000). Concepts of memory. In: The Oxford Handbook of Memory. Oxford: Oxford University Press, pp. 33–43.Google Scholar

Tulving, E. (2002). Episodic memory: from mind to brain. Annu Rev Psychol, 53(1), 1–25.Google Scholar

Vogt, S., Magnussen, S. (2007). Expertise in pictorial perception: eye-movement patterns and visual memory in artists and laymen. Perception, 36(1), 91–100.Google Scholar

Voss, J.F., Vesonder, G.T., Spilich, G.J., et al. (1980). Text generation and recall by high-knowledge and low-knowledge individuals. J Verb Learn Verb B, 19(6), 651–667.Google Scholar

Wiseman, S., Neisser, U. (1974). Perceptual organization as a determinant of visual recognition memory. Am J Psychol, 87, 675–681.Google Scholar

References

Abbey, C.K., Barrett, H.H. (2001). Human and model-observer performance in ramp-spectrum noise: effects of regularization and object variability. J Opt Soc Am A, 18, 473–488.Google Scholar

Abbey, C.K., Bochud, F.O. (2000). Modeling visual detection tasks in correlated image noise with linear model observers. In: Beutel, J., Kundel, H.L., Van Metter, R.L. (eds.). The Handbook of Medical Imaging: Volume 1, Progress in Medical Physics and Psychophysics, Bellingham, WA: SPIE Press, pp. 629–654.Google Scholar

Abbey, C.K., Eckstein, M.P. (2007). Classification images for simple detection and discrimination tasks in correlated noise. J Opt Soc Am A, 24, B110–B124.Google Scholar

Ahumada, A.J., Jr. (1987). Putting the visual system noise back in the picture. J Opt Soc Am A, 4, 2372–2378.Google Scholar

Aufrichtig, R., Xue, P. (2000). Dose efficiency and low-contrast detectability of an amorphous silicon X-ray detector for digital radiography. Phys Med Biol, 45, 2653–2669.Google Scholar

Barrett, H.H., Yao, J., Rolland, J.P., Myers, K.J. (1993). Observer models for assessment of image quality. Proc Natl Acad Sci USA, 90, 9758–9765.Google Scholar

Barrett, H.H., Denny, J.L., Wagner, R.F., Myers, K.J. (1995). Objective assessment of image quality. II. Fisher information, Fourier crosstalk, and figures of merit for task performance. J Opt Soc Am A, 12, 834–852.Google Scholar

Barten, P.G.J. (1987). The SQRI method: a new method for the evaluation of visible resolution on a display. Proc Soc Inf Disp, 28: 253–262.Google Scholar

Barten, P.G.J. (1999). Contrast Sensitivity of the Human Eye and its Effect on Image Quality. Bellingham, WA: SPIE Press.Google Scholar

Burgess, A.E. (1994). Statistically defined backgrounds: performance of a modified nonprewhitening observer model. J Opt Soc Am A, 11, 1237–1242.Google Scholar

Burgess, A.E. (1995). Comparison of receiver operating characteristic and forced choice observer performance measurement methods. Med Phys, 22, 643–655.Google Scholar

Burgess, A.E. (1999). The Rose model revisited. J Opt Soc Am A, 16, 633–646.Google Scholar

Burgess, A.E., Judy, P.F. (2007). Signal detection in power-law noise: effect of spectrum exponents. J Opt Soc Am A, 24, B52– B60.Google Scholar

Burgess, A.E., Wagner, R.F., Jennings, R.J., Barlow, H.B. (1981). Efficiency of human visual signal discrimination. Science, 214, 93–94.Google Scholar

Burgess, A.E., Li, X., Abbey, C.K. (1997). Visual signal detectability with two noise components: anomalous masking effects. J Opt Soc Am A, 14, 2420–2442.Google Scholar

Diaz, I., Abbey, C.K., Timberg, P.A.S., Eckstein, M.P., Verdun, F.R., Castella, C., Bochud, F.O. (2015). Derivation of an observer model adapted to irregular signals based on convolution channels. IEEE Trans Med Imag, 34, 1428–1435.Google Scholar

Eckstein, M.P., Whiting, J.S. (1995). Lesion detection in structured noise. Acad Radiol, 2, 249–253.Google Scholar

Fukunaga, K. (1972). Introduction to Statistical Pattern Recognition. New York, NY: Academic Press.Google Scholar

Gang, G.J., Stayman, J.W., Zbijewski, W., Siewerdsen, J.H. (2014). Task-based detectability in CT image reconstruction by filtered backprojection and penalized likelihood estimation. Med Phys, 41, 081902 1–19.Google Scholar

Gifford, H.C., Pretorius, P.H., King, M.A. (2003). Comparison of human- and model-observer LROC studies. Proc SPIE Med Imag, 5034, 112–122.Google Scholar

Gifford, H.C., King, M.A., Pretorius, P.H., Wells, R.G. (2005). A comparison of human and model observers in multislice LROC studies. IEEE Trans Med Imag, 24, 160–169.Google Scholar

Gifford, H.C., Kinahan, P.E., Lartizien, C., King, M.A. (2007). Evaluation of multiclass model observers in PET LROC studies. IEEE Trans Nucl Sci, 54, 116–123.Google Scholar

Green, D.M., Swets, J.A. (1966). Signal Detection Theory and Psychophysics. New York, NY: Wiley.Google Scholar

Gur, D., Sumkin, J.H., Rockette, H.E., et al. (2004). Changes in breast cancer detection and mammography recall rates after the introduction of a computer-aided detection system. J Natl Cancer Inst, 96, 185–190.Google Scholar

Hanson, K.M. (1979). Detectability in computed tomographic images. Med Phys, 6, 441–451.Google Scholar

Ikejimba, L.C., Kiarashi, N., Ghate, S.V., Samei, E., Lo, J.Y. (2014). Task-based strategy for optimized contrast enhanced breast imaging: analysis of six imaging techniques for mammography and tomosynthesis. Med Phys, 41,061908 1-14.Google Scholar

Judy, P.F., Swensson, R.G. (1981). Lesion detection and signal-to-noise ratio in CT images. Med Phys, 8, 13–23.Google Scholar

Judy, P.F., Swensson, R.G. (1987). Display thresholding of images and observer detection performance. J Opt Soc Am A, 4, 954–965.Google Scholar

Kelly, D.H. (1975). Spatial frequency selectivity in the retina. Vision Res, 15, 665–672.Google Scholar

Kersten, D. (1984). Spatial summation in visual noise. Vision Res, 24, 1977–1990.Google Scholar

Mahalanobis, P. (1936). On the generalized distance in statistics. Proc Natl Inst Sci India (Calcutta), 2, 49–55.Google Scholar

Metz, C.E. (1986). ROC methodology in radiologic imaging. Invest Radiol, 21, 720–733.Google Scholar

Myers, K.J., Barrett, H.H. (1987). Addition of a channel mechanism to the ideal-observer model. J Opt Soc Am A, 4, 2447–2457.Google Scholar

Myers, K.J., Barrett, H.H., Borgstrom, M.C., Patton, D.D., Seeley, G.W. (1985). Effect of noise correlation on detectability of disk signals in medical imaging. J Opt Soc Am A, 2, 1752–1759.Google Scholar

Obuchowski, N.A., Schoenhagen, P., Modic, M.T., Meziane, M., Budd, G.T. (2007). Incidence of advanced symptomatic disease as primary endpoint in screening and prevention trials. Am J Roentgenol, 189, 19–23.Google Scholar

Pisano, E.D., Gatsonis, C., Hendrick, E., et al. (2005). Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med, 353, 1773–1783.Google Scholar

Reiser, I., Nishikawa, R.M. (2006). Identification of simulated microcalcifications in white noise and mammographic backgrounds. Med Phys, 33, 2905–2911.Google Scholar

Richard, S., Siewerdsen, J.H. (2007). Optimization of dual-energy imaging systems using generalized NEQ and imaging task. Med Phys, 34, 127–139.Google Scholar

Richard, S., Siewerdsen, J.H. (2008). Comparison of model and human observer performance for detection and discrimination tasks using dual-energy X-ray images. Med Phys, 35, 5043–5053.Google Scholar

Rolland, J.P., Barrett, H.H. (1992). Effect of random background inhomogeneity on observer detection performance. J Opt Soc Am A, 9, 649–658.Google Scholar

Rolland, J.P., Barrett, H.H., Seeley, G.W. (1991). Ideal versus human observer for long-tailed point spread functions: does deconvolution help? Phys Med Biol, 36, 1091–1109.Google Scholar

Rose, A. (1953). Quantum and noise limitations of the visual process. J Opt Soc Am, 43, 715–716.Google Scholar

Segui, J.A., Zhao, W. (2006). Amorphous selenium flat panel detectors for digital mammography: validation of a NPWE model observer with CDMAM observer performance experiments. Med Phys, 33, 3711–3722.Google Scholar

Swets, J.A., Pickett, R.M. (1982). Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York, NY: Academic Press.Google Scholar

Tapiovaara, M.J., Wagner, R.F. (1993). SNR and noise measurements for medical imaging. I. A practical approach based on statistical decision theory. Phys Med Biol, 38, 71–92.Google Scholar

Wagner, R.F., Brown, D.G. (1985). Unified SNR analysis of medical imaging systems. Phys Med Biol, 30, 489–518.Google Scholar

Watson, A.B. (1987a). Efficiency of a model human image code. J Opt Soc Am A, 4, 2401–2417.Google Scholar

Watson, A.B. (1987b). The cortex transform: rapid computation of simulated neural images. Comput Vision, Graphics Image Proc, 39, 311–327.Google Scholar

Webster, M.A., De Valois, R.L. (1985). Relationship between spatial-frequency and orientation tuning of striate-cortex cells. J Opt Soc Am A, 2, 1124–1132.Google Scholar

Yao, J., Barrett, H.H. (1992). Predicting human performance by a channelized Hotelling observer model. Proc SPIE Med Imag, 1768, 161–168.Google Scholar

Zhang, Y., Pham, B.T., Eckstein, M.P. (2004). Automated optimization of JPEG 2000 encoder options based on observer model performance for detecting variable signals in X-ray coronary angiograms. IEEE Trans Med Imag, 23, 459–474.Google Scholar

Zhang, Y., Pham, B.T., Eckstein, M.P. (2007). Evaluation of internal noise methods for Hotelling observer models. Med Phys, 34, 3312–3322.Google Scholar

References

Abbey, C.K., Barrett, H.H. (2001). Human- and model-observer performance in ramp-spectrum noise: effects of regularization and object variability. J Opt Soc Am A, 18, 473–488.Google Scholar

Barrett, H.H. (1990). Objective assessment of image quality: effects of quantum noise and object variability. J. Opt Soc Am A, 7(7), 1266–1278.Google Scholar

Barrett, H.H., Myers, K.M. (2004). Foundations of Image Science. Hoboken, NJ: John Wiley.Google Scholar

Barrett, H.H., Wilson, D.W., Tsui, B.M.W. (1994). Noise properties of the EM algorithm I: theory. Phys Med Biol, 39, 833–846.Google Scholar

Barrett, H.H., Abbey, C., Gallas, B., Eckstein, M. (1998). Stabilized estimates of Hotelling-observer detection performance in patient-structured noise. Proc SPIE Med Imag, 3340, 27–43.Google Scholar

Barrett, H.H., Myers, K.J., Gallas, B.D., Clarkson, E., Zhang, H. (2001) Megalopinakophobia: its symptoms and cures. Proc SPIE Med Imag, 4320, 299–307.Google Scholar

Barrett, H.H., Myers, K.J., Devaney, N., Dainty, C. (2006). Objective assessment of image quality. IV. Application to adaptive optics. J Opt Soc Am A, 23(12), 3080–3105.Google Scholar

Brankov, J.G. (2013). Evaluation of the channelized Hotelling observer with an internal-noise model in a train-test paradigm for cardiac SPECT defect detection. Phys Med Biol, 58(20), 7159.Google Scholar

Chan, H.-P., Sahiner, B., Wagner, R.F., Petrick, N. (1999). Classifier design for computer-aided diagnosis: effects of finite sample size on the measured performance of classical and neural-network classifiers, Med Phys, 26(12), 2654–2669.Google Scholar

Clarkson, E. (2007). The estimation receiver operating characteristic curve and ideal observers for combined detection/estimation tasks, J Opt Soc Am A, 24(12), B91–B98.Google Scholar

Ferrero, A., Favazza, C.P., Yu, L., Leng, S., McCollough, C.H. (2017). Practical implementation of channelized Hotelling observers: effect of ROI size. Proc SPIE Med Imag, 10132, 101320G.Google Scholar

Fukunaga, K. (1990). Statistical Pattern Recognition. San Diego, CA: Academic Press.Google Scholar

Gifford, H.C. (2014). Efficient visual-search model observers for PET. Br J Radiol, 87(1039), 20140017.Google Scholar

Gifford, H.C., King, M.A., Wells, R.G. (2000). Single-photon emission computed tomography: LROC analysis of detector-response compensation in SPECT. IEEE Trans Med Imag, 19, 463–473.Google Scholar

Gilks, W.R., Richardson, S., Spiegelhalter, D.J. (1996). Markov Chain Monte Carlo in Practice. Boca Raton, FL: Chapman and Hall/CRC.Google Scholar

Gross, K., Kupinski, M.A., Peterson, T., Clarkson, E. (2003). Optimizing a multiple-pinhole SPECT system using the ideal observer. Proc SPIE Med Imag, 5034, 314–322.Google Scholar

He, X., Caffo, B.S., Frey, E.C. (2008). Toward realistic and practical ideal observer estimation for the optimization of medical imaging systems. IEEE Trans Med Imag, 27(10), 1535–1543.Google Scholar

Henkelman, R.M., Kay, I., Bronskill, M.J. (1990). Receiver operator characteristic (ROC) analysis without truth. Med Decis Making, 10, 24–29.Google Scholar

Hernandez-Giron, I., Calzado, A., Geleijns, J., Joemai, R.M.S., Veldkamp, W.J.H. (2011) Automated assessment of low contrast sensitivity for CT systems using a model observer. Med Phys, 38, S1.Google Scholar

Hoppin, J.W., Kupinski, M.A., Kastis, G., Clarkson, E., Barrett, H.H. (2002). Objective comparison of quantitative imaging modalities without the use of a gold standard. IEEE Trans Med Imag, 21, 441–449.Google Scholar

Khurd, P., Gindi, G. (2005). Decision strategies maximizing the area under the LROC curve SPIE Proc Med Imag, 5749, 150–161.Google Scholar

Kupinski, M.A., Hoppin, J.W., Clarkson, E., Barrett, H.H. (2002). Estimation in medical imaging without a gold standard. Acad Radiol, 9, 290–297.Google Scholar

Kupinski, M.A., Hoppin, J.W., Clarkson, E., Barrett, H.H. (2003). Ideal-observer computation in medical imaging with use of Markov-chain Monte Carlo techniques. J Opt Soc Am A, 20(3), 430–438.Google Scholar

Kupinski, M.A., Clarkson, E., Hesterman, J.Y. (2007). Bias in Hotelling observer performance computed from finite data. Proc SPIE Med Imag, 6515, 65150S-1-7.Google Scholar

Lehovich, A., Gifford, H.C., King, M.A. (2008). Model observer to predict human performance in LROC studies of SPECT reconstruction using anatomical priors. Proc SPIE Med Imag, 6917, 67170R-1–7.Google Scholar

McCollough, C.H. (2012). Achieving routine submillisievert CT scanning: report from the summit on management of radiation dose in CT. Radiology, 264(2), 567–580.Google Scholar

Park, S., Kupinski, M.A., Clarkson, E. (2003). Ideal-observer performance under signal and background uncertainty. Proc Inform Process Med Imag, 18, 342–353.Google Scholar

Park, S., Clarkson, E., Kupinski, M.A., Barrett, H.H. (2005). Efficiency of the human observer detection random signal in random backgrounds J Opt Soc Am A, 22(1), 3–16.Google Scholar

Platiša, L., Goossens, B., Vansteenkiste, E., Park, S., Gallas, B.D., Badano, A., Philips, W. (2011). Channelized Hotelling observers for the assessment of volumetric imaging data sets. J Opt Soc Am A 28(6) 1145–1163.Google Scholar

Rusinek, H., Naidich, D.P., McGuinness, G., Leitman, B.S., McCauley, D.I., Krinsky, G.A., Clayton, K., Cohen, H. (1998). Pulmonary nodule detection: low-dose versus conventional CT. Radiology, 209(1), 243–249.Google Scholar

Schindera, S.T., Oddra, D., Raza, S.A., Kim, T.K., Jang, H.J., Szucs-Farkas, Z., Rogalla, P. (2013). Iterative reconstruction algorithm for CT: can radiation dose be decreased while low-contrast detectability is preserved? Radiology, 269(2), 511–518.Google Scholar

Sen, A., Gifford, H. C. (2016). Accounting for anatomical noise in search-capable model observers for planar nuclear imaging. J Med Imag, 3(1), 015502.Google Scholar

Tseng, H.-W., Fan, J., Kupinski, M.A., Sainath, P., Hsieh, J. (2014). Assessing image quality and dose reduction of a new X-ray computed tomography iterative reconstruction algorithm using model observers. Med Phys, 41(7), 071910.Google Scholar

Tseng, H.-W., Fan, J., Kupinski, M.A. (2015). Combination of detection and estimation tasks using channelized scanning linear observer for CT imaging systems. Proc SPIE Med Imag, 9416, 94160H.Google Scholar

Whitaker, M.K., Clarkson, E., Barrett, H.H. (2008). Estimating random signal parameters from noisy images with nuisance parameters: linear and scanning-linear methods. Optics Express, 16(11), 8150–8173.Google Scholar

Wilson, D.W., Tsui, B.M., Barrett, H.H. (1994). Noise properties of the EM algorithm: II. Monte Carlo simulations, Phys Med Biol, 39(5), 847–871.Google Scholar

Woodbury, M.A. (1950). Inverting modified matrices. In: Statistical Research Group. Princeton, NJ: Princeton University Press.Google Scholar

Wunderlich, A., Goossens, B. (2014). Nonparametric estimation receiver operating characteristic analysis for performance evaluation on combined detection and estimation tasks. J Med Imag, 1(3), 031002.Google Scholar

References

Adler, R.J. (2000). On excursion sets, tube formulas and maxima of random fields. Ann Appl Probab, 10(1), 1–74.Google Scholar

Barrett, H.H., Yao, J., Rolland, J.P., Myers, K.J. (1993). Observer models for assessment of image quality. Proc Natl Acad Sci USA, 90(21), 9758–9765.Google Scholar

Bochud, F.O., Valley, J.-F., Verdun, F.R., Hessler, C., Schnyder, P. (1999). Estimation of the noisy component of anatomical backgrounds. Med Phys, 26(7), 1365–1370.Google Scholar

Chakraborty, D.P. (2006). A search model and figure of merit for observer data acquired according to the free-response paradigm. Phys Med Biol, 51, 3449–3462.Google Scholar

Chakraborty, D.P. (2013). A brief history of free-response receiver operating characteristic paradigm data analysis. Acad Radiol, 20(7), 915–919.Google Scholar

Gifford, H.C. (2014). Efficient visual-search observer models for pet. Br J Radiol, 87(1039), 20140017.Google Scholar

Hand, D.J. (1996). Statistics and the theory of measurement. J R Statist Soc A, 159, 445–492.Google Scholar

Hoeschen, C., Tischenko, O., Buhr, E., Illers, H. (2005). Comparison of technical and anatomical noise in digital thorax X-ray images. Radiat Protect Dosim, 114(1–3), 75–80.Google Scholar

Kohli, M., Prevedello, L.M., Filice, R.W., Geis, J.R. (2017). Implementing machine learning in radiology practice and research. Am J Roentgenol, 208(4), 754–760.Google Scholar

Kundel, H.L., Nodine, C.F., Krupinski, E.A. (1989). Searching for lung nodules: visual dwell indicates location of false-positive and false negative decisions. Invest Radiol, 4, 472–478.Google Scholar

Myers, K.J., Barrett, H.H. (1987). Addition of a channel mechanism to the ideal-observer model. J Opt Soc Am A, 4(12), 2447–2457.Google Scholar

Popescu, L.M. (2008). Model for the detection of signals in images with multiple suspicious locations. Med Phys, 35(12), 5565–5574.Google Scholar

Popescu, L.M. (2011). Nonparametric signal detectability evaluation using an exponential transformation of FROC curve. Med Phys, 38(10), 5690–5702.Google Scholar

Popescu, L.M., Myers, K.J. (2013). CT image assessment by low contrast signal detectability evaluation with unknown signal location. Med Phys, 40(11), 111908–111910.Google Scholar

Rolland, J.P., Barrett, H.H. (1992). Effect of random background inhomogeneity on observer detection performance. J Opt Soc Am A, 9(5), 649–658.Google Scholar

Sharp, P., Barber, D.C., Brown, D.G., Burgess, A.E., Metz, C.E., Myers, K.J., Taylor, C.J., Wagner, R.F., Brooks, R., Hill, C.R., Kuhl, D.E., Smith, M.A., Wells, P., Worthington, B. (1996). Medical Imaging: The Assessment of Image Quality. ICRU Report 54. Bethesda, MD: International Commission on Radiological Units and Measurements.Google Scholar

Suppes, P., Zinnes, J.L. (1963). Basic measurement theory. In: Luce, R.L., Bush, R.R., Galanter, E. (eds.) Handbook of Mathematical Psychology, Vol. 1. New York, NY: Wiley, pp. 1–76.Google Scholar

Swets, J.A. (1973). The relative operating characteristic in psychology. Science, 183, 990–1000.Google Scholar

Wagner, R.F., Brown, D.G. (1985). Unified SNR analysis of medical imaging systems. Phys Med Biol, 30(6), 489–518.Google Scholar

Wang, S., Summers, R.M. (2012). Machine learning and radiology. Med Image Analysis, 16(5), 933–951.Google Scholar

Book contents

Part III - Perception Metrology

Summary

Access options

References

References

References

References

References

References

References

References

Save book to Kindle

Save book to Dropbox

Save book to Google Drive