Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-23T07:54:25.450Z Has data issue: false hasContentIssue false

Empirical Performance of Optimal Bayesian Adaptive Estimation

Published online by Cambridge University Press:  10 January 2013

Miguel Ángel García-Pérez
Affiliation:
Universidad Complutense (Spain)
Rocío Alcalá-Quintana*
Affiliation:
Universidad Complutense (Spain)
*
Correspondence concerning this article should be addressed to Rocío Alcalá-Quintana, Departamento de Metodología, Universidad Complutense de Madrid, Campus de Somosaguas, 28223 Madrid (Spain). Phone: 34-913-943061. Fax: 34-913-943189. E-mail: [email protected]

Abstract

Simulation studies have shown how Bayesian adaptive estimation methods should be set up for optimal performance. We assessed the extent to which these results hold up for human observers, who are more subject to failure than simulation subjects. Discrimination and detection experiments with two-alternative forced-choice (2AFC) tasks were used for that purpose. Forty estimates of the point of subjective equality (PSE, or the 50% correct point on the psychometric function for discrimination) and 32 estimates of detection threshold (the 80% correct point on the psychometric function for detection) were taken for each of four observers with the optimal Bayesian method, while data for fitting the psychometric function Ψ were gathered concurrently with an adaptive method of constant stimuli governed by fixed-step-size staircases. The estimated parameters of the psychometric function served as a criterion for comparison. In the discrimination task, PSEs for each observer were distributed around the independently estimated 50% correct point on Ψ and their variability was occasionally minimally larger than simulation results indicated it should be. In the detection task, the distribution of threshold estimates was consistently above the independently estimated 80% correct point on Ψ and their variability was as expected from simulations. A close analysis of these results suggests that the optimal Bayesian method is affected by growing inattention or fatigue in detection tasks (factors that are not considered in simulations), and limits the practical applicability of Bayesian estimation of detection thresholds.

Los métodos bayesianos de estimación adaptativa han sido optimizados en varios estudios de simulación. En este trabajo evaluamos hasta qué punto los resultados obtenidos en las simulaciones son aplicables a observadores humanos. Para ello se sometió a cuatro observadores a dos tipos de experimento (discriminación y detección) con la tarea de elección forzada entre dos alternativas (2AFC). La configuración óptima del método bayesiano sirvió para obtener, por cada observador, 40 estimaciones del punto de igualdad subjetiva (PSE, que es el punto de la función psicométrica que lleva aparejado un porcentaje de éxito del 50% en un experimento de discriminación) y 32 estimaciones del umbral de detección, definido como el punto de la función psicométrica cuyo porcentaje de éxito asociado es el 80%. Simultáneamente, se utilizó el método adaptativo de los estímulos constantes para obtener una estimación independiente de los parámetros la función psicométrica Ψ de cada observador que sirviera como criterio de comparación. En la tarea de discriminación, y para todos los observadores, las distribuciones de los PSE se situaron en torno a los puntos del 50% de Ψ estimados de manera independiente y la variabilidad fue sólo ligeramente superior a la esperada a partir de las simulaciones. Por el contrario, en la tarea de detección, las distribuciones de estimaciones del umbral se situaron consistentemente por encima de los puntos del 80% de Ψ, aunque su variabilidad fue similar a la registrada en las simulaciones. Un análisis minucioso de estos resultados sugiere que el método bayesiano óptimo se ve muy afectado por la creciente falta de atención y la fatiga en las tareas de detección (factores que no fueron contemplados en las simulaciones), lo que limita la aplicabilidad de los métodos bayesianos en la estimación práctica de umbrales de detección.

Type
Research Article
Copyright
Copyright © Cambridge University Press 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alcalá-Quintana, R. & García-Pérez, M. A. (2004a). The role of parametric assumptions in adaptive Bayesian estimation. Psychological Methods, 9, 250271.CrossRefGoogle ScholarPubMed
Alcalá-Quintana, R. & García-Pérez, M. A. (2004b). Empirical performance of optimal Bayesian adaptive psychophysical methods. Perception, Suppl., 33, 178.Google Scholar
Alcalá-Quintana, R. & García-Pérez, M. A. (2005). Stopping rules in Bayesian adaptive threshold estimation. Spatial Vision, 18, 347374.CrossRefGoogle ScholarPubMed
Alcalá-Quintana, R. & García-Pérez, M. A. (2007). A comparison of fixed-step-size and Bayesian staircases for sensory threshold estimation. Spatial Vision, 20, 197218.CrossRefGoogle ScholarPubMed
García-Pérez, M. A. (2000). Optimal setups for forced-choice staircases with fixed step sizes. Spatial Vision, 13, 431448.CrossRefGoogle ScholarPubMed
García-Pérez, M. A. & Alcalá-Quintana, R. (2005). Sampling plans for fitting the psychometric function. Spanish Journal of Psychology, 8, 256289.CrossRefGoogle ScholarPubMed
García-Pérez, M. A. & Alcalá-Quintana, R. (2007). Bayesian adaptive estimation of arbitrary points on a psychometric function. British Journal of Mathematical and Statistical Psychology, 60, 147174.CrossRefGoogle ScholarPubMed
Green, D. G. (1990). Stimulus selection in adaptive psychophysical procedures. Journal of the Acoustical Society of America, 87, 26622674.CrossRefGoogle ScholarPubMed
Green, D. G. (1993). A maximum-likelihood method for estimating thresholds in a yes-no task. Journal of the Acoustical Society of America, 93, 20962105.Google Scholar
Kollmeier, B., Gilkey, R. H., & Sieben, U. K. (1988). Adaptive staircase techniques in psychoacoustics: A comparison of human data and a mathematical model. Journal of the Acoustical Society of America, 83, 18521862.CrossRefGoogle ScholarPubMed
Laming, D. & Marsh, D. (1988). Some performance tests of QUEST on measurements of vibrotactile thresholds. Perception & Psychophysics, 44, 99107.CrossRefGoogle ScholarPubMed
Lesmes, L. A., Jeon, S.-T., Lu, Z.-L., & Dosher, B. A. (2006). Bayesian adaptive estimation of threshold versus contrast external noise functions: The quick TvC method. Vision Research, 46, 31603176.CrossRefGoogle ScholarPubMed
Madigan, R. & Williams, D. (1987). Maximum-likelihood psychometric procedures in two-alternative forced-choice: Evaluation and recommendations. Perception & Psychophysics, 42, 240249.Google Scholar
Meese, T. S. (1995). Using the standard staircase to measure the point of subjective equality: A guide based on computer simulations. Perception & Psychophysics, 57, 267281.CrossRefGoogle ScholarPubMed
Numerical Algorithms Group (1999). NAG Fortran Library Manual, Mark 19. Oxford: Author.Google Scholar
Peli, E. & García-Pérez, M. A. (1997). Contrast sensitivity in dyslexia: Deficit or artifact?. Optometry and Vision Science, 74, 986988.CrossRefGoogle ScholarPubMed
Pentland, A. (1980). Maximum likelihood estimation: The best PEST. Perception & Psychophysics, 28, 377379.CrossRefGoogle ScholarPubMed
Schlauch, R. S. & Rose, R. M. (1990). Two-, three-, and four-interval forced-choice staircase procedures: Estimator bias and efficiency. Journal of the Acoustical Society of America, 88, 732740.CrossRefGoogle ScholarPubMed
Simpson, W. A. (1989). The step method: A new adaptive psychophysical procedure. Perception & Psychophysics, 45, 572576.CrossRefGoogle Scholar
Stillman, J. A. (1989). A comparison of three adaptive psychophysical procedures using inexperienced listeners. Perception & Psychophysics, 46, 345350.CrossRefGoogle ScholarPubMed
Stuart, G. W., McAnally, K. I., & Castles, A. (2001). Can contrast sensitivity functions in dyslexia be explained by inattention rather than a magnocellular deficit? Vision Research, 41, 32053211.CrossRefGoogle ScholarPubMed
Swift, D., Panish, S., & Hippensteel, B. (1997). The use of VisionWorks TM in visual psychophysics research. Spatial Vision, 10, 471477.CrossRefGoogle ScholarPubMed
Wichmann, F. A. & Hill, N. J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 12931313.CrossRefGoogle ScholarPubMed