Skip to main content Accessibility help
×
×
Home

Sampling Plans for Fitting the Psychometric Function

  • Miguel A. García-Pérez (a1) and Rocío Alcalá-Quintana (a1)
Abstract

Research on estimation of a psychometric function Ψ has usually focused on comparing alternative algorithms to apply to the data, rarely addressing how best to gather the data themselves (i.e., what sampling plan best deploys the affordable number of trials). Simulation methods were used here to assess the performance of several sampling plans in yes–no and forced-choice tasks, including the QUEST method and several variants of up–down staircases and of the method of constant stimuli (MOCS). We also assessed the efficacy of four parameter estimation methods. Performance comparisons were based on analyses of usability (i.e., the percentage of times that a plan yields usable data for the estimation of all the parameters of Ψ) and of the resultant distributions of parameter estimates. Maximum likelihood turned out to be the best parameter estimation method. As for sampling plans, QUEST never exceeded 80% usability even when 1000 trials were administered and rendered accurate estimates of threshold but misestimated the remaining parameters. MOCS and up–down staircases yielded similar and acceptable usability (above 95% with 400–500 trials) and, although neither type of plan allowed estimating all parameters with optimal precision, each type appeared well suited to estimating a distinct subset of parameters. An analysis of the causes of this differential suitability allowed designing alternative sampling plans (all based on up–down staircases) for yes–no and forced-choice tasks. These alternative plans rendered near optimal distributions of estimates for all parameters. The results just described apply when the fitted Ψ has the same mathematical form as the actual Ψ generating the data; in case of form mismatch, all parameters except threshold were generally misestimated but the relative performance of all the sampling plans remained identical. Detailed practical recommendations are given.

Los estudios sobre estimación de la función psicométrica Ψ se han centrado tradicionalmente en comparar los algoritmos que se pueden aplicar a los datos, dejando al margen el problema de cómo recoger los propios datos (es decir, qué esquema de muestreo despliega de mejor forma los ensayos disponibles). Aquí se utilizan técnicas de simulación para evaluar el rendimiento de varios esquemas de muestreo en tareas de sí–no y de elección forzada, incluyendo QUEST y distintas variantes de escaleras de paso fijo y del método de los estímulos constantes. También se evalúa la eficacia de cuatro métodos de estimación de parámetros. Las comparaciones se basan en análisis de usabilidad (es decir, del porcentaje de veces que un esquema proporciona datos válidos para estimar todos los parámetros de Ψ) y de las distribuciones de las estimaciones. El mejor método de estimación resultó ser el de máxima verosimilitud. En cuanto a esquemas de muestreo, QUEST no llegó a rendir una usabilidad del 80% ni siquiera cuando se administraron 1000 ensayos y, aunque proporcionó buenas estimaciones del umbral, estimó erróneamente el resto de los parámetros. El método de los estímulos constantes y las escaleras de paso fijo rindieron una usabilidad similar (superior al 95% con 400–500 ensayos) y, aunque ninguno de estos esquemas permitió estimar con precisión óptima todos los parámetros, cada tipo de esquema se mostró adecuado para estimar un subconjunto distinto de parámetros. El análisis de las causas de estas diferencias permitió diseñar esquemas alternativos (todos ellos basados en escaleras de paso fijo) para tareas de sí–no y de elección forzada. Estos esquemas alternativos proporcionaron estimaciones con distribuciones casi óptimas. Los resultados descritos son válidos cuando la función cuyos parámetros se estiman tiene la misma forma analítica que la función psicométrica que ha generado los datos; cuando esas funciones difieren en forma, todos los parámetros excepto el umbral resultan estimados erróneamente, aunque la eficacia relativa de los distintos esquemas de muestreo no varía. Se ofrecen recomendaciones prácticas basadas en estos resultados.

Copyright
Corresponding author
Address correspondence to: Miguel A. García-Pérez, Departamento de Metodología, Facultad de Psicología, Universidad Complutense, Campus de Somosaguas, 28223 Madrid (Spain). Phone: +34 913 943 061. Fax: +34 913 943 189. E-mail: miguel@psi.ucm.es
References
Hide All
Alcalá-Quintana, R., & García-Pérez, M.A. (2002, August). Bias and standard errors in forced-choice Bayesian staircases. Paper presented at the 33rd European Mathematical Psychology Group Meeting, Bremen, Germany.
Alcalá-Quintana, R., & García-Pérez, M.A. (2004a). The role of parametric assumptions in adaptive Bayesian estimation. Psychological Methods, 9, 250271.
Alcalá-Quintana, R., & García-Pérez, M.A. (2004b). Empirical performance of optimal Bayesian adaptive psychophysical methods. Perception (Suppl.), 33, 178.
Balakrishnan, N. (1992). Handbook of the logistic distribution. New York: Marcel Dekker.
Berkson, J. (1955). Maximum likelihood and minimum χ 2 estimates of the logistic function. Journal of the American Statistical Association, 50, 130162.
Brand, T., & Kollmeier, B. (2002). Efficient adaptive procedures for threshold and concurrent slope estimates for psychophysics and speech intelligibility tests. Journal of the Acoustical Society of America, 111, 28012810.
Brown, L.G. (1996). Additional rules for the transformed up-down method in psychophysics. Perception & Psychophysics, 58, 959962.
Dixon, W.J., & Mood, A.M. (1948). A method for obtaining and analyzing sensitivity data. Journal of the American Statistical Association, 43, 109126.
Evans, M., Hastings, N., & Peacock, B. (1993). Statistical distributions (2nd ed.). New York: Wiley.
Foster, D.H., & Bischof, W.F. (1991). Thresholds from psychometric functions: Superiority of bootstrap to incremental and probit variance estimators. Psychological Bulletin, 109, 152159.
Freeman, P.R. (1970). Optimal Bayesian sequential estimation of the median effective dose. Biometrika, 57, 7989.
García-Pérez, M.A. (1998). Forced-choice staircases with fixed step sizes: Asymptotic and small-sample properties. Vision Research, 38, 18611881.
García-Pérez, M.A. (2001). Yes-no staircases with fixed step sizes: Psychometric properties and optimal setup. Optometry & Vision Science, 78, 5664.
García-Pérez, M.A., Giorgi, R., Woods, R.L., & Peli, E. (2005). Thresholds vary between spatial and temporal forced-choice paradigms: The case of lateral interactions in peripheral vision. Spatial Vision, 18, 99127.
Hall, J.L. (1981). Hybrid adaptive procedure for estimation of psychometric functions. Journal of the Acoustical Society of America, 69, 17631769.
Harvey, L.O. Jr., (1997). Efficient estimation of sensory thresholds with ML-PEST. Spatial Vision, 11, 121128.
Kaernbach, C. (1991). Simple adaptive testing with the weighted up—down method. Perception & Psychophysics, 49, 227229.
Kaernbach, C. (2001). Slope bias of psychometric functions derived from adaptive data. Perception & Psychophysics, 63, 13891398.
Kershaw, C.D. (1985). Statistical properties of staircase estimates from two interval forced choice experiments. British Journal of Mathematical & Statistical Psychology, 38, 3543.
King-Smith, P.E., & Rose, D. (1997). Principles of an adaptive method for measuring the slope of a psychometric function. Vision Research, 37, 15951604.
King-Smith, P.E., Grigsby, S.S., Vingrys, A.J., Benes, S.C., & Supowit, A. (1994). Efficient and unbiased modifications of the QUEST threshold method: Theory, simulations, experimental evaluation and practical implementation. Vision Research, 34, 885912.
Klein, S.A. (2001). Measuring, estimating, and understanding the psychometric function: A commentary. Perception & Psychophysics, 63, 14211455.
Kontsevich, L.L., & Tyler, C.W. (1999). Bayesian adaptive estimation of psychometric slope and threshold. Vision Research, 39, 27292737.
Lam, C.F., Mills, J.H., & Dubno, J.R. (1996). Placement of observations for the efficient estimation of a psychometric function. Journal of the Acoustical Society of America, 99, 36893693.
Leek, M.R., Hanna, T.E., & Marshall, L. (1992). Estimation of psychometric functions from adaptive tracking procedures. Perception & Psychophysics, 51, 247256.
Maloney, L.T. (1990). Confidence intervals for the parameters of psychometric functions. Perception & Psychophysics, 47, 127134.
Marks, B.L. (1962). Some optimal sequential schemes for estimating the mean of a cumulative normal quantal response curve. Journal of the Royal Statistical Society, Series B, 24, 393400.
McKee, S.P., Klein, S.A., & Teller, D.Y. (1985). Statistical properties of forced-choice psychometric functions: Implications of probit analysis. Perception & Psychophysics, 37, 286298.
Miller, J., & Ulrich, R. (2001). On the analysis of psychometric functions: The Spearman—Kärber method. Perception & Psychophysics, 63, 13991420.
Müller, H.G., & Schmitt, T. (1990). Choice of number of doses for maximum likelihood estimation of the ED50 for quantal dose-response data. Biometrics, 46, 117129.
Myers, R.H. (1990). Classical and modern regression with applications. Boston, MA: PWS-KENT.
Numerical Algorithms Group (1999). NAG Fortran library manual, Mark 19. Oxford: Numerical Algorithms Group.
O'Regan, J.K., & Humbert, R. (1989). Estimating psychometric functions in forced-choice situations: Significant biases found in threshold and slope estimations when small samples are used. Perception & Psychophysics, 46, 434442.
Owen, R.J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351356.
Ramsey, F.L. (1972). A Bayesian approach to bioassay. Biometrics, 28, 841858.
Robbins, H., & Monro, S. (1951). A stochastic approximation method. Annals of Mathematical Statistics, 22, 400407.
Santoro, L., Burr, D., & Morrone, M. C. (2002). Saccadic compression can improve detection of Glass patterns. Vision Research, 42, 13611366.
Serrano-Pedraza, I., & Sierra-Vázquez, V. (2003, August). Comparison between two methods for estimation of the parameters of a psychometric function: Effect of initial guess. Poster presented at the 34th European Mathematical Psychology Group Meeting, Madrid, Spain.
Simmers, A.J., Bex, P.J., Smith, F.K.H., & Wilkins, A.J. (2001). Spatiotemporal visual function in tinted lens wearers. Investigative Ophthalmology & Visual Science, 42, 879884.
Snoeren, P.R., & Puts, M.J.H. (1997). Multiple parameter estimation in an adaptive psychometric method: MUEST, an extension of the QUEST method. Journal of Mathematical Psychology, 41, 431439.
Snowden, R.J., & Hammett, S.T. (1998). The effects of surround contrast on contrast thresholds, perceived contrast and contrast discrimination. Vision Research, 38, 19351945.
Solomon, J.A., & Morgan, M.J. (2000). Facilitation of collinear flanks is cancelled by non-collinear flanks. Vision Research, 40, 279286.
Strasburger, H. (2001). Invariance of the psychometric function for character recognition across the visual field. Perception & Psychophysics, 63, 13561376.
Swanson, W.H., & Birch, E.E. (1992). Extracting thresholds from noisy psychophysical data. Perception & Psychophysics, 51, 409422.
Treutwein, B. (1997). YAAP: Yet another adaptive procedure. Spatial Vision, 11, 129134.
Treutwein, B., & Strasburger, H. (1999). Fitting the psychometric function. Perception & Psychophysics, 61, 87106.
Watson, A.B., & Pelli, D.G. (1983). QUEST: A Bayesian adaptive psychometric method. Perception & Psychophysics, 33, 113120.
Watson, A.B., & Turano, K. (1995). The optimal motion stimulus. Vision Research, 35, 325336.
Werkhoven, P., & Snippe, H.P. (1996). An efficient adaptive procedure for psychophysical discrimination experiments. Behavior Research Methods, Instruments, & Computers, 28, 556562.
Wetherill, G.B., & Levitt, H. (1965). Sequential estimation of points on a psychometric function. British Journal of Mathematical & Statistical Psychology, 18, 110.
Wichmann, F.A., & Hill, N.J. (2001a). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 12931313.
Wichmann, F.A., & Hill, N.J. (2001b). The psychometric function: II. Bootstrap-based confidence intervals and sampling. Perception & Psychophysics, 63, 13141329.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

The Spanish Journal of Psychology
  • ISSN: 1138-7416
  • EISSN: 1988-2904
  • URL: /core/journals/spanish-journal-of-psychology
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed