Undecided cases in dual-presentation methods

Fechner (Reference Fechner1860/1966, p. 77) reported that the experiments he started in 1855 were designed to be a more exact check of Weber’s law but he soon realized that a thorough investigation of the implications of procedural details should be undertaken first. One of his concerns was whether observers should be forced to respond in each trial that the second weight was either perceptually heavier or lighter than the first or, instead, they should be allowed to express indecision and report the two weights to be perceptually equally heavy whenever needed. He sentenced that “in the beginning I always used the first procedure, but later I discarded all experiments done in that way and used the second course exclusively, having become convinced of its much greater advantages” (p. 78).

The undecided response had been allowed earlier by Hegelmaier (1852; translated into English by Laming & Laming, Reference Laming and Laming1992) in a study on the perceived length of lines, a work that Fechner credited. Despite some opposition (see Fernberger, Reference Fernberger1930), the undecided response continued to be allowed until signal detection theory took over and denied undecided cases entirely, although there were attempts to reinstate it that did not seem to stick (e.g., Greenberg, Reference Greenberg1965; Olson & Ogilvie, Reference Olson and Ogilvie1972; Treisman & Watts, Reference Treisman and Watts1966; Vickers, Reference Vickers, Rabbitt and Dornic1975, Reference Vickers1979; Watson, Kellogg, Kawanishi, & Lucas, Reference Watson, Kellogg, Kawanishi and Lucas1973). The influence of signal detection theory is perhaps the reason that the conventional framework invariably assumes a decision rule by which observers are never undecided and always respond according to whether the decision variable is above or below some criterion β. Ironically, and despite interpreting the data as if observers could never be undecided, undecided cases are explicitly acknowledged in practice and observers are instructed to guess in such cases. Undoubtedly, undecided cases are impossible under the conventional framework and they ask for an amendment that can capture them adequately to allow proper inferences from observed data. In fact, the consequences of ignoring them are serious, as will be seen below. It is worth saying in anticipation that neither the admonition to guess when undecided nor Fechner’s strategy of counting undecided responses as half right and half wrong are advisable: They both distort the psychometric function in ways that it no longer expresses the true characteristics of the psychophysical function and, thus, neither the location nor the slope of the psychometric function reflect the underlying reality that researchers aim at unveiling.

A closely related issue that calls for an analogous amendment is that many studies use dual-presentation methods under the same–different task (see Macmillan & Creelman, Reference Macmillan and Creelman2005, pp. 214–228; see also Christensen & Brockhoff, Reference Christensen and Brockhoff2009), sometimes referred to as the equality task to distinguish it from the previously discussed comparative task where observers report which stimulus is stronger in some respect. The equality task also implies a binary response but now observers indicate only whether the two stimuli appear to be equal or different along the dimension of comparison. Leaving aside the sterile discussion as to whether responding “same” in the equality task is tantamount to being undecided in the comparative task, it is nevertheless clear that the same–different task is incompatible with the conventional framework: With a single boundary (at β = 0 as in Figure 3c or elsewhere), observers should invariably respond “different” in all trials. It certainly does not seem sensible to amend the decision space only for the same–different task, because it would then imply that whether an observer can or cannot judge equality depends on the question asked at the end of the trial, which can indeed be withheld so the observer gets to know it only after the stimuli are extinguished (see the control experiment reported by Schneider & Komlos, Reference Schneider and Komlos2008). The conventional framework thus also needs an amendment to accommodate the empirical fact that observers can perform the same–different task if so requested, and this must be done in a way that is consistent with the fact that observers often guess in the comparative task.

Order or position effects in dual-presentation methods. Part I: The constant error

Presentation of two magnitudes cannot occur co-spatially and simultaneously. Then, standard and test have to be presented in some temporal order or in some spatial arrangement in each trial. Fechner (Reference Fechner1860/1966, pp. 75–77) reported that the temporal order in which the two weights were lifted greatly affected the results in a way that the ratio of right to wrong cases turned out quite different and beyond what could be expected by chance even with long series of trials. This was regarded as evidence of a so-called constant error, which is puzzling because it implies that standard and test stimuli of the exact same magnitude and also identical in all other respects are systematically perceived to be different according to the order or position in which they are presented. Fechner also reported that constant errors varied in size across sessions for the same observer and that the cause of their “totally unexpected general occurrence” (p. 76) was unclear, but he found solace by accepting them as a constituent part of psychophysical measurements. He also devised a method to average out their influence so that unique estimates of differential sensitivity could still be obtained. Unbeknownst to Fechner, such unique estimates are essentially flawed, as will be seen below.

Fechner was mainly interested in investigating Weber’s law, which requires that standard and test stimuli differ exclusively in their magnitude along the dimension of comparison. The ubiquitous presence of constant errors becomes an obstacle to this endeavor because it implies that standard and test stimuli differ also on something else by an unknown mechanism related to procedural factors. Measures of differential sensitivity reflecting how much heavier the test must be for observers to notice a difference in heaviness cannot be obtained when the perceived heaviness of the standard differs due to such procedural factors. Although the notion of a psychometric function was not available to Fechner, in present-day terms his studies sought to measure the DL as defined earlier. But this, of course, requires that a test stimulus with the exact same weight as the standard stimulus is actually perceived to be as heavy as the standard, a pre-condition that the presence of constant errors violates.

Figure 4 illustrates what the psychometric function should be like under the conventional framework when test and standard are identical except in magnitude along the dimension of comparison (visual luminance contrast in this case), which in comparison to Figure 3 only differs in that test and standard stimuli have the same spatial frequency and, thus, μ_t = μ_s. The PSE is predicted to be at the standard contrast and the DL as defined earlier can be computed from the slope of the function. But constant errors defy this framework, because they imply a lateral shift of the psychometric function such that the 50% point is no longer at the standard and the 75% point is similarly shifted. Despite the shift, differential sensitivity might still be assessed if it is computed relative to the (shifted) location of the 50% point rather than relative to the actual location of the standard. If the unknown mechanism only shifts the psychometric function laterally without affecting its slope, constant errors will not be a problem provided that the necessary precautions are taken to compute sensitivity measures adequately. We will show later that this is not the case, but something must still be missing in the conventional framework when it cannot account for the occurrence of constant errors.

Figure 4.

Psychometric Function Arising from the Conventional Framework when Test and Standard Stimuli Differ only along the Dimension of Comparison so that μ_t = μ_s.

The 50% point is then at the standard level and, thus, at the perceptual PSE. Layout and graphical conventions as in Figure 3.

Constant errors have been around since Fechner but their cause was not investigated by the early psychophysicists. Interest soon focused on methodological research to establish optimal conditions for preparation and administration of stimuli and for the collection of data with the ternary response format. In this context, figuring out how to eliminate constant errors was not as much a priority as was keeping their influence constant. Thus, for the sake of experimental control, Urban (Reference Urban1908) always used sequential presentations in which the standard weight was lifted before the test weight in each trial (a strategy that was later referred to as the reminder paradigm; see Macmillan & Creelman, Reference Macmillan and Creelman2005, pp. 180–182). The data that Urban reported in tabulated or graphical form invariably reflected constant errors in the form of psychometric functions shifted laterally, suggesting that the PSE was not at the standard magnitude but somewhat below it. Fernberger (Reference Fernberger1913, Reference Fernberger1914a, Reference Fernberger1914b, Reference Fernberger1916, Reference Fernberger1920, Reference Fernberger1921) also used the reminder paradigm in analogous studies aimed at perfecting the experimental procedure and investigating additional issues so that, again, the constant error was regarded as an extraneous factor simply kept constant in the interest of experimental control.

We are not aware of any early study in which both orders of presentation (i.e., standard first and standard second) had been used in separate series in a within-subjects design to ascertain how psychometric functions under the ternary response format used at that time differ across presentation orders. Some studies were conducted along these lines, but with undecided responses already transformed into half right and half wrong (e.g., Woodrow, Reference Woodrow1935; Woodrow & Stott, Reference Woodrow and Stott1936). Some research conducted during the second half of the 20th century used the binary format asking observers to guess when undecided, but these studies only measured constant errors at isolated pairs of magnitudes with no possibility of looking at psychometric functions (e.g., Allan, Reference Allan1977; Jamieson & Petrusic, Reference Jamieson and Petrusic1975). Ross and Gregory (Reference Ross and Gregory1964) appeared to be the first to conduct a study of constant errors under each of the possible presentation orders, but they also forced observers to guess when undecided. In other studies, psychometric functions were also measured but only with the reminder paradigm and using the binary response format with guesses (e.g., Levison & Restle, Reference Levison and Restle1968).

It should be noted that the PSE empirically determined from the 50% point on an observed psychometric function is only a misnomer in the presence of constant errors. In fact, when standard and test are identical in all respects, a PSE declared to be away from the standard would literally mean that a stimulus has to be different from itself for an observer to perceive them both as equal. This nagging incongruence was handled in early psychophysics by defining the constant error as the difference between the observed PSE and the standard, on the implicit assumption that the true PSE was indeed located at the standard and that the observed deviation was only disposable error of an unknown origin. Unfortunately, there is a large set of situations in which one cannot reasonably assume that the true PSE is at the standard, as in studies aimed at investigating whether and by how much the psychophysical function for some stimulus dimension varies across extra dimensions, where one has reasons to believe that μ_t ≠ μ_s as illustrated in Figure 3. All studies on perceptual illusions also fall in this class (e.g., whether and by how much the perceived length of a line varies when it terminates in arrowheads vs. arrow tails in the Müller–Lyer figure, or whether and by how much the perceived luminance of a uniform gray field varies with the luminance of the background it is embedded in), and other types of studies also fall in this category (e.g., whether and by how much does perceived contrast vary with spatial frequency, or whether and by how much the perceived duration of an empty interval differs from that of a filled interval). In such cases, constant errors make it impossible to determine whether a 50% point found to be away from the standard reflects different psychophysical functions for test and standard. Then, an amendment to the conventional framework seems necessary to incorporate a mechanism that produces constant errors so as to be able to extract the characteristics of the underlying psychophysical functions and, thus, to establish with certainty whether a 50% point that is found to be away from the standard magnitude is only disposable error or a true indication that test and standard stimuli are perceived differently.

Order or position effects in dual-presentation methods. Part II: Interval bias

Although constant errors continued to be a topic of research (for recent papers and references to earlier studies, see Hellström & Rammsayer, Reference Hellström and Rammsayer2015; Patching, Englund, & Hellström, Reference Patching, Englund and Hellström2012), they have been ignored in research estimating psychometric functions with dual-presentation methods. It is not easy to trace back the time at which (and to dig out the reasons why) this happened, but around the 1980’s it became customary to intersperse at random trials presenting the standard first or second and to bin together all the responses given at each test magnitude irrespective of presentation order to fit a single psychometric function to the aggregate. The ternary response format had also long been abandoned and observers were routinely forced to guess when undecided. The notion of “constant error” then resurfaced under the name “interval bias” to refer to differences in the observed proportion of correct responses contingent on the order of presentation of test and standard, although the term had been used earlier by Green and Swets (1966, pp. 408–411) in a context not involving estimation of psychometric functions, and only at that point within their entire book.

Nachmias (Reference Nachmias2006) seems to have been the first to fit and plot separate psychometric functions for the subsets of trials in which the test was first or second, although he appeared to have used for this purpose aggregated data across all available observers and experimental conditions. His results revealed that interval bias (constant error) was also present when trials with both orders of presentation were randomly interwoven in a session and, thus, not only when separate sessions were conducted for each order as Ross and Gregory (Reference Ross and Gregory1964) had done earlier. In line with Hellström (Reference Hellström1977), this suggested that the constant error is perceptual in nature and not due to a response bias developed over the course of a session with trials of the same type. Again, this result questions the validity of the conventional framework, which assumes that μ_t = μ_s when test and standard are identical except along the dimension of comparison and invariant with presentation order.^{Footnote 4} An interval bias of perceptual origin would imply that the psychophysical functions differ when the stimulus (test or standard) is presented first or second, even when such stimuli are identical. There are good reasons to think that this might indeed occur with intensive magnitudes due to desensitization or analogous influences from the first presentation on the second (see Alcalá-Quintana & García-Pérez, Reference Alcalá-Quintana and García-Pérez2011), but this is instead impossible with the extensive magnitudes for which interval bias and constant errors have been described more often.

Interval bias has been found in data sets that were originally treated conventionally (i.e., by aggregating responses across presentation orders) but for which the order of presentation in each trial had been recorded and a re-analysis was possible (García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcalá-Quintana2011a). Interval bias was also found in new studies intentionally conducted to investigate it (e.g., Alcalá-Quintana & García-Pérez, 2011; Bausenhart, Dyjas, & Ulrich, Reference Bausenhart, Dyjas and Ulrich2015; Bruno, Ayhan, & Johnston, Reference Bruno, Ayhan and Johnston2012; Cai & Eagleman, Reference Cai and Eagleman2015; Dyjas, Bausenhart, & Ulrich, Reference Dyjas, Bausenhart and Ulrich2012; Dyjas & Ulrich, Reference Dyjas and Ulrich2014; Ellinghaus, Ulrich, & Bausenhart, Reference Ellinghaus, Ulrich and Bausenhart2018; García-Pérez & Alcalá-Quintana, 2010a; Ulrich & Vorberg, Reference Ulrich and Vorberg2009). Many of these studies have documented order effects in two different forms that Ulrich and Vorberg called “Type A” and “Type B”. The former is the lateral shift that we have described thus far; the latter is a difference in the slope of the psychometric functions for each presentation order, which can be observed with or without lateral shifts. Whichever its origin may be, an amendment to the conventional framework is necessary to account for the empirical fact that psychometric functions vary with the order or position of presentation of test and standard.

Effects of response format in single-presentation methods

Presentation of a single magnitude per trial certainly removes any potential complication arising from order effects, but there are still aspects of single-presentation data that are incompatible with the conventional framework. One feature on which available single-presentation methods differ from one another is the response format with which judgments are collected. And one form in which the conventional framework fails here is by being unable to account for differences in the psychometric functions or their characteristics across response formats, which should imply the same psychophysical functions because the operation of sensory systems cannot be retroactively affected by how observers report their judgments. We will only discuss two of the various lines of evidence in this direction.

One of them arises in the context of research on perception of simultaneity. These studies rest on earlier procedures developed to determine the smallest inter-stimulus temporal gap that can be experienced (see James, Reference James1886) and they were originally designed by Lyon and Eno (Reference Lyon and Eno1914) to investigate the transmission speed of neural signals with the method of constant stimuli. Trials consisted of the administration of punctate tactual or electrical stimuli on different parts of the arm with a variety of temporal delays. The rationale is that transmission speed can be estimated by measuring how much earlier stimulation at a distal location must be administered for an observer to perceive it synchronous with stimulation at a proximal location. Two stimuli are obviously used here but a single value of the dimension under investigation (namely, temporal delay) is presented in each trial, so this is one of the single-presentation methods in which two identifiable stimuli are required to administer a single magnitude along a dimension unrelated to any of the physical characteristics of the stimuli themselves. Contrary to the spirit of their time, Lyon and Eno asked observers to report which stimulus had been felt first, disallowing “synchronous” responses and forcing observers to guess. The exact same strategy is still used today under the name temporal-order judgment (TOJ) task, but variants thereof include the binary synchrony judgment (SJ2) task in which observers simply report whether or not the presentations were perceptually synchronous, and the ternary synchrony judgment (SJ3) task which is a TOJ task with allowance to report synchrony (Ulrich, Reference Ulrich1987). Usually, the purpose of these studies is to determine the point of subjective simultaneity (PSS), defined as the delay with which one of the stimulus must be presented for observers to perceive synchronicity. One of the most puzzlingly replicated outcomes of this research is that PSS estimates meaningfully vary across tasks within observers (see, e.g., van Eijk, Kohlrausch, Juola, & van de Par, Reference van Eijk, Kohlrausch, Juola and van de Par2008), with differences such that the delay at which “synchronous” responses are maximally prevalent in SJ2 or SJ3 tasks is also one of those at which asynchronous responses of one or the other type prevail in the TOJ task. The typical ad hoc argument to explain away these contradictory outcomes is to assume that each task calls for different cognitive processes, which goes hand in hand with the far-fetched assumption that the psychophysical function mapping physical asynchrony onto perceived asynchrony varies with the response format.

The second line of evidence arises in research on perception of duration and, specifically, on the ability to bisect a temporal interval, which is generally investigated with either of two alternative single-presentation methods. In the temporal bisection task (Gibbon, Reference Gibbon1981), observers report during an experimental phase whether intervals of variable duration are each closer in length to a “short” or to a “long” interval repeatedly presented earlier during a training phase; in the temporal generalization task (Church & Gibbon, Reference Church and Gibbon1982), observers report during the experimental phase whether or not the temporal duration of each of a set of individually presented intervals is the same as that of a “sample” interval repeatedly presented during the training phase. Not unexpectedly in the context of this discussion, the temporal bisection point has been found to vary meaningfully with the variant used to collect data (e.g., Gil & Droit-Volet, Reference Gil and Droit-Volet2011), which again seems to suggest that the psychophysical function mapping chronometric duration onto perceived duration varies with the response format.

Note that the various response formats in each of these research areas (TOJ vs. SJ2 or SJ3 in perception of simultaneity and bisection vs. generalization in perception of duration) resemble those discussed earlier for dual-presentation methods (comparative vs. equality). Just as a single-boundary decision space could never give rise to “same” responses in a dual-presentation method, never would it give rise to “synchronous” or “same” responses in the single-presentation methods just discussed. It may well be that these apparent failures of the conventional framework are another manifestation of the consequences of forcing (in TOJ or bisection tasks) or not forcing (in SJ2, SJ3, or generalization tasks) observers to give responses that do not reflect their actual judgment of synchrony in onset or equality in duration. Then, this is another area where an alternative framework that reconciles these conflicting results is needed.

Effects of response strategy in single-presentation methods

Another line of evidence against the conventional framework for single-presentation methods comes from studies in which data are collected under different conditions or instructions aimed at manipulating the observers’ decisional or response strategies. Admittedly, the effect of some of these manipulations are still compatible with the conventional framework as long as they can be interpreted as altering the location of criterion β (see, e.g., Allan, Reference Allan2002; Ellis, Klaus, & Mast, Reference Ellis, Klaus and Mast2017; Raslear, Reference Raslear1985; Wearden & Grindrod, Reference Wearden and Grindrod2003). Even with such allowance, this reveals an inescapable problem that will be discussed later, namely, that single-presentation methods confound sensory and decisional components of performance (see García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcalá-Quintana2013) and leave researchers unable to figure out if they are observing a relevant result indicative of the characteristics of sensory processing (a feature of the psychophysical function) or an outcome reflecting only the effect of non-sensory factors (an influence of the location of the response criterion). Yet, the interest in investigating some instructional manipulations and the fact that they happen to produce effects are inherently incongruent with the conventional framework, which ignores all of these influences entirely. For instance, Capstick (2012, exp. 2) reported the results of a within-subjects study that used two variants of the single-presentation TOJ task respectively differing in that observers were asked to always give one or the other allowed responses when uncertain. The 50% point on the psychometric function for each variant of the TOJ task shifted in opposite directions away from physical simultaneity, which is inexplicable under the conventional framework with single decision boundary. In line with this result, Morgan Dillenburger, Raphael, and Solomon (2012) reported the same effect on the psychometric function in a spatial bisection task under the binary left–right format: It shifted laterally according to instructions as to how to respond when uncertain so that the 50% point on the psychometric function came out on the far left of the physical midpoint in one case and on the far right in the other. Which of these two, if any, is the PSS or the perceptual midpoint? The dissonance is further apparent because observers can never be uncertain under the conventional decision space (see Figure 2) and, hence, instructions as to what to do in those (inexisting) cases should be inconsequential. Of course, the fact that such instructions produce effects on the location of the psychometric function implies again that something is missing in the conventional framework, further stressing that the observed location of a psychometric function is not necessarily indicative of the characteristics of the psychophysical function mapping a physical continuum maps onto its perceptual counterpart.

The indecision model in single-presentation methods

The simple decision space of the conventional framework in the sample application of the single-presentation method in Figure 2 might actually involve as many regions along the dimension of perceived position as positions an observer is capable of discerning (e.g., slightly on the right, far on the right, etc.). Qualitative and ill-defined categories of this type were indeed used in the method of absolute judgment by the early psychophysicists. For instance, Fernberger (Reference Fernberger1931) had observers classify each individually lifted weight into the three categories of light, intermediate, or heavy. Analogously, in a study on taste, Pfaffmann (Reference Pfaffmann1935) asked observers to rate the intensity of each tastant on a six-point scale. A potential gradation of the perceptual continuum with multiple boundaries is accepted even by signal detection theorists, as it is the basis for classification, identification, and rating designs (Green & Swets, Reference Green and Swets1966, pp. 40–43; Macmillan, Kaplan, & Creelman, Reference Macmillan, Kaplan and Creelman1977; Wickelgren, Reference Wickelgren1968). A single-presentation method with multiple ordered response categories that capitalize on this gradation is a quantized psychophysical scaling method without the burden of arbitrary and unnatural numerical scales (Galanter & Messick, Reference Galanter and Messick1961). Arguably, these regions might be the basis by which observers express confidence when so requested, although we will not discuss this here (but see Clark, Yi, Galvan-Garza, Bermúdez Rey, & Merfeld, Reference Clark, Yi, Galvan-Garza, Bermúdez Rey and Merfeld2017; Lim, Wang, & Merfeld, Reference Lim, Wang and Merfeld2017; Yi & Merfeld, Reference Yi and Merfeld2016).

Without loss of generality, we will restrict ourselves here to cases in which the partition can be anchored relative to a well-defined internal reference such as the perceptual midpoint, the subjective vertical, subjective synchrony, perceptual straight ahead, etc. A defining assumption of the indecision model is that the smallest possible number of partitions after any necessary aggregation is three and that the central region stretches around the location of the internal reference (the null value in perceptual space in all of the sample cases just mentioned) but not necessarily centered with it, as seen in Figure 5. This minimum number of three regions does not deny subdivisions that might manifest if the response format asked observers to use them (e.g., by expanding the response set to include, say, slightly on the right, far on the right, etc.). What the indecision model posits is that there are three qualitatively different regions, even if the response format only asks for a binary response. In other words, the boundaries δ₁ and δ₂ (with δ₁ < δ₂) of the central region in Figure 5 will not come together by simply forcing observers to give a binary left–right response. Or, yet in other words, judgments of “center” (under any suitable designation) are qualitatively different from, and impossible to collapse with, judgments of “left” or “right”, whereas judgments of degrees of “left” (or “right”) to several discernible extents can and will be collapsed into a qualitative judgment of “left” (or “right”) if no gradation is required by the response format. This central region can be regarded as the interval of perceptual uncertainty to stress that it is defined in perceptual space and, thus, not to be confused with the interval of uncertainty defined by Fechner (Reference Fechner1860/1966, p. 63) and further elaborated by Urban (1908, Ch. V) to represent the range of stimulus magnitudes where undecided responses prevail.

Figure 5.

Alternative Framework of the Indecision Model in the Single-Presentation Example of Figure 2 with a Partition of Perceptual Space into Three Regions Demarcated by Boundaries δ₁ and δ₂ that Can Be Symmetrically Placed with Respect to the Perceptual Midpoint (a) or Displaced (b).

In either case, when the spatial bisection task is administered with the response options “left”, “right”, and “center”, the psychometric functions come out as shown in the back projection planes. Although the same psychophysical function mapping the physical midpoint onto the perceptual midpoint is used in both cases, the peak of the psychometric function for “center” responses does not occur at the physical midpoint in (b) due to the displacement of the region for “center” judgments.

With allowance for “center” responses so that there is a one-to-one mapping between (qualitatively different) judgments and responses, the psychometric functions come out as illustrated in the back projection planes in Figure 5 and are given by

(5a)$$P\left( {{\rm{''left'';}}\,x} \right) &#x003D; P\left( {S &#x0026;lt; {{\rm{\delta }}_1}{\rm{;}}\,x} \right) &#x003D; P\left( {Z &#x0026;lt; {{{{\rm{\delta }}_1} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right) &#x003D; \Phi \left( {{{{{\rm{\delta }}_1} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right)$$,

(5b)$$P\left( {{\rm{''right'';}}\,x} \right) &#x003D; P\left( {S &#x0026;gt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right) &#x003D; P\left( {Z &#x0026;gt; {{{{\rm{\delta }}_2} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right) &#x003D; \Phi \left( {{{{\rm{\mu }}\left( x \right) - {{\rm{\delta }}_2}} \over {{\rm{\sigma }}\left( x \right)}}} \right)$$,

(5c)$$P\left( {{\rm{''center'';}}\,x} \right) &#x003D; P\left( {{{\rm{\delta }}_1} &#x0026;lt; S &#x0026;lt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right) &#x003D; \Phi \left( {{{{{\rm{\delta }}_2} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right) - \Phi \left( {{{{{\rm{\delta }}_1} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right)$$,

with the probabilities at each position x indicated by the areas colored in red, gray, and blue, respectively, under the Gaussian distributions of perceived position in Figure 5. In Figure 5a, the peak of the psychometric function for center responses occurs at the physical midpoint, reflecting that the psychophysical function in this example is given by Eq. 1 with a = 0. Then, the location of the psychometric functions reveals no distortion of perceptual space. Yet, this is only because the boundaries δ₁ and δ₂ are also symmetrically placed with respect to the perceptual midpoint (i.e., δ₁ = −δ₂). If these boundaries were displaced (i.e., δ₁ ≠ −δ₂; see Figure 5b), the psychometric functions also get displaced, producing the appearance of a perceptual distortion of space despite the fact that the underlying psychophysical function with a = 0 still maps the physical midpoint onto the perceptual midpoint in this example. In general, the empirical finding of displaced psychometric functions is not necessarily indicative of a true perceptual distortion. In the case of Figure 5b, the displacement is only caused by what we call decisional bias: An off-center location of the interval of perceptual uncertainty by which the amount of evidence needed for a “left” judgment differs from that needed for a “right” judgment. (We will comment on empirical evidence to this effect in the Discussion.) The question is whether one could also tell in empirical practice whether or not δ₁ = −δ₂ so as to elucidate the cause of an observed shift of the psychometric function.

Unfortunately, the short answer is no. Single-presentation methods cannot distinguish decisional bias from perceptual effects and, thus, they are unsuitable for investigating sensory aspects of performance (see Allan, Reference Allan2002; García-Pérez, Reference García-Pérez2014a; García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcalá-Quintana2013, Reference García-Pérez, Alcalá-Quintana, Vatakis, Balcı, Di Luca and Correa2018; García-Pérez & Peli, Reference García-Pérez and Peli2014; Raslear, Reference Raslear1985; Schneider & Bavelier, Reference Schneider and Bavelier2003). Without loss of generality, this is easily understood by simply noting that, with µ(x) = a + bx and σ(x) constant, the psychometric function for “center” responses (Eq. 5c) peaks at $x &#x003D; {{{{\rm{\delta }}_1} &#x002B; {{\rm{\delta }}_2} - 2a} \over {2b}}$ so that perceptual factors (i.e., parameters a and b in the psychophysical function) and decisional factors (i.e., parameters δ₁ and δ₂ in decision space) jointly determine the location of the psychometric function with no chance of separating out their individual influences. This inherent incapability of single-presentation methods has strong implications on research practices that will be summarized later, but it does not prevent an analysis of how the indecision model explains the empirical anomalies and inconsistencies described earlier.

Our first illustration involves the lateral shifts reported by Capstick (Reference Capstick2012) or Morgan et al. (Reference Morgan, Dillenburger, Raphael and Solomon2012) but only in the latter case, that is, when a single-presentation spatial bisection task is administered with the binary left–right response format under alternative instructions as to what to do when uncertain. Under the indecision model, observers are uncertain when their actual judgment was “center” and the problem arises because they are forced to misreport it as a “left” or a “right” response. If they always respond “right” when uncertain, they will misreport “center” judgments as “right” responses and the probability of a “right” response will then be the sum of Eqs. 5b and 5c, namely,

(6)$$P\left( {{\rm{''right'';}}\,x} \right) &#x003D; P\left( {S &#x0026;gt; {{\rm{\delta }}_1}{\rm{;}}\,x} \right) &#x003D; P\left( {Z &#x0026;gt; {{{{\rm{\delta }}_1} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right) &#x003D; \Phi \left( {{{{\rm{\mu }}\left( x \right) - {{\rm{\delta }}_1}} \over {{\rm{\sigma }}\left( x \right)}}} \right)$$.

If, instead, they always respond “left” when uncertain, they will misreport “center” judgments as “left” responses and the probability of a “right” response will then be simply Eq. 5b. The shift arises because observers functionally operate only with the boundary δ₁ or only with δ₂ when they misreport “center” judgments as “left” or “right” responses, respectively (see Figs. 6a and 6b). It is also interesting to note that observers who are instead requested to guess with equiprobability when uncertain will misreport “center” judgments as “right” responses on only half of the trials at each physical position so that the observed psychometric function for “right” responses would be the sum of Eq. 5b plus half of Eq. 5c, or

(7)$$P\left( {{\rm{''right'';}}\,x} \right) &#x003D; P\left( {S &#x0026;gt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right) &#x002B; {{P\left( {{{\rm{\delta }}_1} &#x0026;lt; S &#x0026;lt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right)} \over 2} &#x003D; 1 - {1 \over 2}\Phi \left( {{{{{\rm{\delta }}_2} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right) - {1 \over 2}\Phi \left( {{{{{\rm{\delta }}_1} - {\rm{\mu }}\left( x \right)} \over {{\rm{\sigma }}\left( x \right)}}} \right)$$.

Figure 6.

Psychometric Function for “Right” Responses Arising from the Indecision Model under Different Response Strategies when Observers are Forced to Give “Left” or “Right” Responses.

(a) “Center” judgments are always misreported as “right” responses. (b) “Center” judgments are always misreported as “left” responses. (c) “Center” judgments are misreported as “left” or “right” responses with equiprobability. In all cases the psychophysical function maps the physical midpoint onto the perceptual midpoint and the interval of perceptual uncertainty is centered (i.e., δ₁ = −δ₂). However, the psychometric function is displaced leftward in (a), rightward in (b) and not displaced in (c) although its slope is shallower in the latter case.

As seen in Figure 6c, equiprobable guessing does not shift the psychometric function relative to the true perceptual midpoint when δ₁ = −δ₂ as in this example (but a shift will still occur if δ₁ ≠ −δ₂). But guessing distorts the slope of the psychometric function, which is now much shallower. It should also be noted that Eq. 7 and the resultant shape of the psychometric function in Figure 6c do not arise from mere instructions to guess evenly. It is necessary that the observer responds “right” on exactly half of the trials that rendered “center” judgments at each position x. This will never occur in practice because observers do not know which position was presented in each trial and do not keep track of how many guesses of each type they have already given. Yet, this can be strictly achieved by allowing observers to give “center” responses and then splitting them as half “left” and half “right” in analogy to how Fechner treated such cases. Although we will discuss this strategy in the original context of dual-presentation methods below, this illustration already makes clear that such a strategy only distorts the slope of the psychometric function.

Our second illustration involves the conflicting outcomes arising in within-subjects studies that use TOJ and SJ2 (or SJ3) tasks for estimating the PSS. This illustration uses empirical data collected by van Eijk et al. (Reference van Eijk, Kohlrausch, Juola and van de Par2008) in a within-subjects study that used SJ2, SJ3, and TOJ tasks with the exact same stimuli, consisting of an auditory click and a flashed visual shape presented with onset asynchronies that varied from negative (auditory first) to positive (visual first) through zero (synchronous). The illustration shows the results of a reanalysis presented in García-Pérez and Alcalá-Quintana (Reference García-Pérez and Alcalá-Quintana2012a) under the indecision model, which revealed that the seemingly discrepant outcomes could be neatly accounted for on the assumption that the underlying sensory processes are identical in all cases and only decisional and response processes vary across tasks. In contrast to the cases described thus far, in this case perceived asynchrony does not have a Gaussian distribution but an asymmetric Laplace distribution (for justification and further details, see García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcalá-Quintana2012a, Reference García-Pérez and Alcalá-Quintana2015a, Reference García-Pérez and Alcalá-Quintana2015b, Reference García-Pérez, Alcalá-Quintana, Vatakis, Balcı, Di Luca and Correa2018). Sensory (timing) parameters of the model are those describing these distributions and were found to be identical for all tasks whereas the decisional parameters (boundaries in decision space) varied across tasks, plus the need for a specific response parameter describing how “synchronous” judgments are misreported in the TOJ task.

Figure 7 shows data and fitted psychometric functions for one of the observers in each task, along with the (common) estimated distributions of perceived asynchrony and the different decision boundaries in each task. Note that the apparent PSS in SJ2 and SJ3 tasks (i.e., the location of the peak of the psychometric function for “synchronous” responses in Figs. 7a and 7b) is virtually at physical synchrony whereas the apparent PSS in the TOJ task (i.e., the 50% point on the psychometric function for “visual first” responses in Figure 7c) is at a meaningfully larger positive asynchrony. Literally interpreted, this says that only in the TOJ task the click has to be presented with some delay for the observer to perceive it simultaneous with the flash. The discrepancy can yet be perfectly accounted for by identical distributions of perceived asynchrony on all tasks and, then, by a single underlying sensory reality. The discrepant result in the TOJ task occurs only because of the manner in which the observer handles the misreporting of “synchronous” judgments, which here causes the same spurious lateral shifts illustrated earlier in Figure 6 (see also figure 1 in Diederich & Colonius, Reference Diederich and Colonius2015).

Figure 7.

Differences in Performance across Single-Presentation Methods for Timing Tasks.

The red, black, and blue psychometric functions are for “audio first”, “synchronous”, and “visual first” responses, respectively. (a) Binary synchrony judgment, SJ2 task. (b) Ternary synchrony judgment, SJ3 task. (c) Temporal-order judgment, TOJ task. Distributions of perceived asynchrony are identical in all tasks but the location of the boundaries δ₁ and δ₂ vary across them. Estimated parameters indicate that the observer responded “visual first” with probability .2 upon “synchronous” judgments in the TOJ task.

The indecision model in dual-presentation methods

Application of the model to dual-presentation tasks is analogous. The interval of perceptual uncertainty now resides on the dimension of perceived differences and replaces the single boundary at β = 0 in Figs. 3c or 4c with two boundaries at δ₁ and δ₂ (again with δ₁ < δ₂). This brings up an issue that was immaterial before but requires explicit treatment now, namely, how the difference D is computed. It is not realistic to model it via D = S _t − S _s as in the conventional framework, both because an observer never knows which presentation displayed the standard and which one the test (and they may even be entirely unaware that this distinction exists) and because computation of the difference in that way does not take into account the order of presentation of test and standard (which is declared irrelevant under the conventional framework). Then, the perceived difference is defined realistically as D = S ₂ – S ₁, where subscripts denote the sequential order (or spatial arrangement) of presentations. It should be easy to see that defining it in reverse would not make any difference. Note, then, that D = S _t − S _s when the test is presented second and D = S _s − S _t when it is presented first.

For illustration, Figure 8 shows the predicted psychometric functions for each presentation order in the dual-presentation contrast discrimination task of Figure 4, where the same psychophysical function holds for test and standard and the true PSE lies at the standard magnitude. Observers are asked to report in which presentation the stimulus had a higher contrast and they are allowed to respond “first”, “second”, or “none”, then translated by the experimenter into “test lower”, “test higher”, or “equal” responses. Note that the boundaries δ₁ and δ₂ are placed asymmetrically in this example. This decisional bias is the reason that psychometric functions are shifted away from the true PSE in opposite directions but by the same amount for each order of presentation. The true PSE remains hidden from view because both psychometric functions are displaced, but it is still identifiable as discussed later.

Figure 8.

Psychometric Functions arising from the Indecision Model in a Dual-Presentation Contrast Discrimination Task with the Same Psychophysical Function for Test and Standard Stimuli, separately for Cases in which the Test Stimulus is always Presented First (a) or Second (b) in Each Trial.

The red, black, and blue psychometric functions are for “test lower”, “equal”, and “test higher” responses. Note that the boundaries δ₁ and δ₂ of the interval of perceptual uncertainty are not symmetrically placed with respect to zero; this is responsible for the rightward shift in (a) and the leftward shift in (b) such that the peak of the psychometric function for “equal” responses does not occur at the standard magnitude.

The ternary response format that allows “equal” responses permits a one-to-one mapping between judgments and responses. In the general case in which psychophysical functions may differ for test and standard, the psychometric functions for “test higher”, “test lower”, and “equal” responses when the test is presented first are given by

(8a)$${P_1}\left( {{\rm{''lower'';}}\,x} \right) &#x003D; P\left( {D &#x0026;gt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right) &#x003D; P\left( {Z &#x0026;gt; {{{{\rm{\delta }}_2} - \left( {{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) &#x003D; \Phi \left( {{{{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\delta }}_2}} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$,

(8b)$${P_1}\left( {{\rm{''higher'';}}\,x} \right) &#x003D; P\left( {D &#x0026;lt; {{\rm{\delta }}_1}{\rm{;}}\,x} \right) &#x003D; P\left( {Z &#x0026;lt; {{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) &#x003D; \Phi \left( {{{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$,

(8c)$${P_1}\left( {{\rm{''equal'';}}\,x} \right) &#x003D; P\left( {{{\rm{\delta }}_1} &#x0026;lt; D &#x0026;lt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right) &#x003D; \Phi \left( {{{{{\rm{\delta }}_2} - \left( {{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) - \Phi \left( {{{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$.

When the test is presented second, they are instead given by

(9a)$${P_2}\left( {{\rm{''lower'';}}\,x} \right) &#x003D; P\left( {D &#x0026;lt; {{\rm{\delta }}_1}{\rm{;}}\,x} \right) &#x003D; P\left( {Z &#x0026;lt; {{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) &#x003D; \Phi \left( {{{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$,

(9b)$${P_2}\left( {{\rm{''higher'';}}\,x} \right) &#x003D; P\left( {D &#x0026;gt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right) &#x003D; P\left( {Z &#x0026;gt; {{{{\rm{\delta }}_2} - \left( {{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) &#x003D; \Phi \left( {{{{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\delta }}_2}} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$,

(9c)$${P_2}\left( {{\rm{''equal'';}}\,x} \right) &#x003D; P\left( {{{\rm{\delta }}_1} &#x0026;lt; D &#x0026;lt; {{\rm{\delta }}_2}{\rm{;}}\,x} \right) &#x003D; \Phi \left( {{{{{\rm{\delta }}_2} - \left( {{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) - \Phi \left( {{{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$.

Unlike single-presentation methods, dual-presentation methods with a ternary response format supply all the data needed to tell apart the contributions of sensory and decisional processes to the observed shifts of the psychometric functions and, thus, to resolve the ambiguity as to whether these shifts have a true perceptual origin or a spurious decisional origin. Although this is not immediately obvious by inspection of the psychometric functions in Eqs. 8 and 9, the true PSE is located at the point relative to which the psychometric functions for each presentation order are pushed apart in opposite directions and by the same amount, whereas decisional bias is the reason that the psychometric functions are pushed apart in the first place (i.e., they are instead superimposed when δ₁ = −δ₂). For a more detailed description of these distinct signatures of decisional bias and perceptual effects, see the discussion surrounding Figure 3 in García-Pérez and Alcalá-Quintana (Reference García-Pérez and Alcalá-Quintana2013), Figures 7 and 8 in García-Pérez (Reference García-Pérez2014a), or Figure 2 in García-Pérez and Peli (Reference García-Pérez and Peli2014). In contrast, single-presentation methods only allow observing one set of psychometric functions (for the inherent lack of a companion) without the benefit of observing what the displacement (if any) would have been in an inexistent companion.

It is also interesting to see how the indecision model accounts for the anomalies described earlier, which involve the practices of using a single presentation order, forcing observers to guess when undecided, or aggregating responses across presentation orders. We will leave aside the now obvious origin of “same” responses in same–different tasks, which is the direct reporting of “equal” judgments. With these illustrations in mind, we chose the example in Figure 8 for immediate connection with a result mentioned above, namely, that data collected in the early studies with lifted weights, a ternary response format, and presentation of the standard before the test in each trial (i.e., as in Figure 8b except for weight comparisons rather than contrast comparisons) showed a leftward shift of the psychometric functions and led to declaring the “totally unexpected” displacement as constant error. The indecision model with a displaced interval of perceptual uncertainty accounts for those ubiquitous results, as seen in Figure 8b. No study appears to have been reported during those early years or soon afterwards that used the reverse order of presentation to allow us to check out whether an opposite displacement occurs (as seen in Figure 8a relative to Figure 8b), but recent studies using the ternary response format indeed demonstrated opposite lateral displacements of psychometric functions for each presentation order or position (for graphical illustrations, see García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcala-Quintana2011b; García-Pérez & Peli, Reference García-Pérez and Peli2014, Reference García-Pérez and Peli2015, Reference García-Pérez and Peli2019; Self et al., Reference Self, Mookhoek, Tjalma and Roelfsema2015).

To illustrate, Figure 9 shows results to this effect from two observers in one of the conditions of the study reported by García-Pérez and Peli (Reference García-Pérez and Peli2015). The study addressed the measurement of aniseikonia, a condition by which perceived size differs between the eyes. Observers compared the perceived size of a standard semicircle haploscopically presented to one of the eyes with that of a spatially adjacent test semicircle presented to the other eye. Presentations were simultaneous and position effects (the spatial analogue of order effects) were handled by presenting the test on the left or on the right in interwoven series of trials. The ternary response format asked observers to indicate which semicircle was larger or else that they appeared equal. Obviously, the goal of the study required uncontaminated estimation of the PSE: On checking for aniseikonia, one needs to be sure that a potential displacement of the observed PSE reflects a true sensory effect (i.e., that the psychophysical function for perceived size differs between the eyes) and it is not instead a spurious consequence of decisional or response factors. By application of the indecision model to the analysis of data, neither of the observers in Figure 9 shows any meaningful displacement of the PSE away from the size of the standard semicircle (i.e., no aniseikonia) although they differ in a decisional bias that would have contaminated the results in a conventional analysis of data aggregated across presentation positions and collected with a response format that forces observers to guess when they perceive both semicircles as having the same size.

Figure 9.

Data and Fitted Psychometric Functions under the Indecision Model for Observers 2 (Left Column) and 8 (Right Column) in One of the Conditions of the Study Reported by García-Pérez and Peli (Reference García-Pérez and Peli2015) when the Test Semicircle Was Presented on the Left (Top Row) and on the Right (Bottom Row).

The estimated PSEs (solid vertical line in the back projection planes) do not differ meaningfully from the size of the standard (dashed vertical line in the projection planes). Observer 2 does not show any sign of decisional bias (psychometric functions for both presentation positions are superimposed and δ₁ ≈ −δ₂) whereas observer 8 shows a relatively strong decisional bias (psychometric functions for each presentation position are shifted laterally and δ₁ and δ₂ are asymmetrically placed).

Ulrich and Vorberg’s (Reference Ulrich and Vorberg2009) Type A order effects are precisely what Figs. 8 and 9b illustrate, in the form of lateral displacements of the psychometric functions for each presentation order or position even under the ternary format that allows a one-to-one mapping of judgments onto responses. Leaving Type B order effects for the Discussion, there are still two aspects that we should address. The first one relates to the consequences of using a response format that forces observers to guess when undecided. The rule by which “equal” judgments are misreported in these cases cannot realistically favor the standard or the test, since observers are unaware of which is which. Misreports then take the form of “first” or “second” responses at random in some form, from strong bias towards guessing “first” through equiprobability to strong bias towards guessing “second”. Calling ξ the probability of responding “first” when guessing, the resultant psychometric functions for “test higher” responses when the test is presented first or second are, respectively,

(10a)$${P_1}\left( {{\rm{''higher'';}}\,x} \right) &#x003D; \left( {1 - {\rm{\xi }}} \right)\Phi \left( {{{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) &#x002B; {\rm{\xi }}\,\Phi \left( {{{{{\rm{\delta }}_2} - \left( {{{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right) - {{\rm{\mu }}_{\rm{t}}}\left( x \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$,

(10b)$${P_2}\left( {{\rm{''higher'';}}\,x} \right) &#x003D; 1 - \left( {1 - {\rm{\xi }}} \right)\Phi \left( {{{{{\rm{\delta }}_1} - \left( {{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right) - {\rm{\xi }}\,\Phi \left( {{{{{\rm{\delta }}_2} - \left( {{{\rm{\mu }}_{\rm{t}}}\left( x \right) - {{\rm{\mu }}_{\rm{s}}}\left( {{x_{\rm{s}}}} \right)} \right)} \over {{{\rm{\sigma }}_{\rm{D}}}\left( x \right)}}} \right)$$.

Equation 10a is Eq. 8b plus ξ times Eq. 8c whereas Eq. 10b is Eq. 9b plus 1 – ξ times Eq. 9c. Figure 10 shows these psychometric functions for ξ ∈ {0, .5, 1} in the conditions of Figure 8 except that the interval of perceptual uncertainty is now centered (i.e., δ₁ = −δ₂). Some of the characteristics discussed next differ when δ₁ ≠ −δ₂ and will be described later. When ξ = 1 (Figure 10a), all “equal” judgments are misreported as “first” responses that contribute to the final count of “test higher” responses when the test was presented first but not when it was presented second. As a result, the psychometric function for “test higher” responses when the test was first shifts to the left of the true PSE whereas it shifts to the right when the test was second. The opposite holds when ξ = 0 (Figure 10b) because “equal” judgments are now misreported as “second” responses. In contrast, splitting guesses evenly via ξ = .5 renders psychometric functions that do not differ across presentation orders (Figure 10c) and have their 50% point at the true PSE, but their slope is greatly reduced. Recall that such an even split is what Fechner’s strategy of recording undecided cases and counting them as half right and half wrong attains, which has serious consequences on measures of differential sensitivity from the slope of the observed psychometric function. In fact, as ξ varies from 0 to 1, the observed psychometric function for each presentation order exhibits a gradual transition from the pattern in Figure 10b (steep and located on one side) to the pattern in Figure 10a (steep and located on the other side) through that in Figure 10c (shallow and centered). These features are summarized by the red and blue curves in the top row of Figure 11.

Figure 10.

Psychometric Functions for “Test Higher” Responses under the Binary Response Format (i.e., Guessing when Uncertain) on Trials in which the Test Stimulus is Presented First (Top Row) or Second (Bottom Row), for Sample Values of the Probability ξ of Responding “First” when Guessing (Columns).

Areas marked in dark color indicate the probability of authentic “test higher” judgments; areas marked in light color indicate the probability of “equal” judgments that will or will not contribute to “test higher” responses according to the guessing strategy. (a) ξ = 1 so that all guesses result in “first” responses translating into “test higher” when the test is presented first and “test lower” when it is second. (b) ξ = 0 so that all guesses result in “second” responses translating into “test lower” when the test is first and “test higher” when it is second. (c) ξ = 0.5 so that half of the guesses result in “first” responses and the other half in “second” responses.

Figure 11.

Psychometric Functions for “Test Higher” Responses under the Binary Response Format with Guessing on Trials in which the Test Stimulus is Presented First (Red Curve) or Second (Blue Curve) and also for Aggregated Data (Black Curve).

The three functions are identical and superimposed in the top-right panel. Functions are plotted for sample values of the probability ξ of responding “first” when guessing (columns) in three cases (rows). Top row: Centered interval of perceptual uncertainty and equal psychophysical functions for test and standard (i.e., the conditions in Figure 10). Center row: Displaced interval of perceptual uncertainty (in the form illustrated in Figure 8) still with identical psychophysical functions for test and standard. Bottom row: Displaced interval of perceptual uncertainty with unequal psychophysical functions for test and standard that shift the PSE (green vertical line) away from the standard magnitude (dashed vertical line).

This pattern is slightly different when the interval of perceptual uncertainty is not centered (i.e., when δ₁ ≠ −δ₂; see the red and blue curves in the center row in Figure 11) under otherwise identical conditions. Displacements across presentation orders still occur relative to the true PSE but now psychometric functions differ across presentation orders even when ξ = .5 (center panel in Figure 11c). They were identical in Figure 10c (and in the top panel of Figure 11c) because Eqs. 10a and 10b are identical when ξ = .5 and δ₁ = −δ₂. In general, psychometric functions for each presentation order are always displaced in opposite directions with respect to the true PSE by a distance that is null only when ξ = .5 and δ₁ = −δ₂. In passing, note that there is evidence of Ulrich and Vorberg’s (Reference Ulrich and Vorberg2009) Type B order effects in the center and bottom panels of Figure 11b, although we will defer commentary on this to the Discussion.

The second aspect that needs consideration is the consequences of aggregating data across presentation orders. To our knowledge, this was never done with ternary data except for summary purposes after psychometric functions for each presentation order had already been estimated (e.g., García-Pérez & Peli, Reference García-Pérez and Peli2014; Self et al., Reference Self, Mookhoek, Tjalma and Roelfsema2015). Then, here we will only illustrate the consequences of aggregating binary data collected with instructions to guess. Aggregation has indeed been the universal first step toward fitting a psychometric function, even years after the unacceptable consequences of this practice had been demonstrated by Ulrich and Vorberg (2009; see also García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcala-Quintana2011b) and despite the availability of user-friendly freeware to fit psychometric functions properly to data separated by presentation order (Bausenhart, Dyjas, Vorberg, & Ulrich, Reference Bausenhart, Dyjas, Vorberg and Ulrich2012; García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcalá-Quintana2017). It is clear that, at any test magnitude, the overall proportion of “test higher” responses equals the average of the proportions separately computed at each presentation order when the number of trials was the same under both orders. Then, averaging the psychometric functions in each column of Figure 10 renders the same result regardless of ξ (note that the black curves in all the panels in the top row of Figure 11 are identical). Yet, aggregation across presentation orders does not really remove the effects of ξ; it only standardizes the data to a putative ξ = .5 (as if guesses had been evenly split into “first” and “second” responses via Fechner’s strategy) but, again, only if δ₁ = −δ₂ as in Figure 10 or in the top row in Figure 11. As seen in Figure 11, the psychometric function for aggregated data (black curves) is not generally invariant with ξ and it also differs from the psychometric functions for each presentation order. The only invariant feature is that the psychometric function for aggregated data always has its 50% point at the true PSE, which may (top and center rows in Figure 11) or may not (bottom row in Figure 11) coincide with the standard magnitude contingent on whether or not µ_t = µ_s.

The picture just described is more complex when the number of trials administered at each individual test magnitude is not the same for both presentation orders (see Figure 13 in García-Pérez & Alcalá-Quintana, Reference García-Pérez and Alcalá-Quintana2011a), an unjustifiable practice when responses will be aggregated (which they should never be in the first place). In any case, note that aggregation does not misestimate the PSE although the artifactually reduced slope of the psychometric function misrepresents the true sensitivity of the observer. This explains why Lapid, Ulrich, and Rammsayer (2008, 2009; see also Rammsayer & Ulrich, Reference Rammsayer and Ulrich2012) found that the DL is generally smaller under the reminder task (i.e., all trials with test-second presentations) than it is under a procedure that they called 2AFC and involved a random mixture of trials with test-first and test-second presentations followed by aggregation of data to fit a single psychometric function. But this artifact does not mean that the observer’s sensitivity differs across tasks by sensory factors (i.e., due to the underlying psychophysical functions). Far from it, the sensory components of performance (including sensitivity) are identical and it is only the responses that are contaminated by irrelevant non-sensory factors that broaden the observed psychometric function for aggregated data. For further analyses and discussion of this issue, see García-Pérez and Alcalá-Quintana (Reference García-Pérez and Alcalá-Quintana2010b).