Motivation and background

Nevertheless, it is very important to emphasise that this otherwise rather open characterisation of the notion of datum is by no means unconstrained. Thus, we do not consider any piece of information to be a datum. The plausibility of data is initially determined by the reliability of their source. The more reliable the source is on the basis of which a plausibility value is assigned to a datum, the higher initial plausibility value one can associate it with. A datum must possess a certain degree of plausibility when it comes into contact with other hypotheses of the theory. On this, we agree with Rescher completely:

In the sense of (47), hypotheses obtained not by inference but originating from some linguistic paper count as data as well. The source of such data is the reference to the linguistic paper at issue. At the same time, if one gives a detailed account of a psycholinguistic experiment instead of only summarising its outcome, then one is not endowed with data (receiving their plausibility value from some direct source). Instead, statements summarising the results of the experiment obtain their plausibility value from indirect sources, namely, by plausible inferences presented in the experimental report. In such cases, those statements count as data in the sense of (47) which serve as the starting points of the presented argumentation (systems of inductive generalisations, analogical inferences) such as authenticated and interpreted perceptual data etc. For more on this, see Chapter 16 and Section 19.6.

Examples 30(1)--(4)

1. (1) Statements capturing introspective judgements based on the linguistic intuition of native speakers are data. What counts as a datum in the sense of (47) is not the linguistic expression or utterance whose grammaticality is judged, but the statement expressing the judgement about the grammaticality of this expression or utterance. Accordingly, the statement

   1. (a) The sentence Wer_i meint Lydia, liebt t_i Jakob? is correct.

   or the statement

   1. (b) The sentence Wer_i meint Lydia, liebt t_i Jakob? is more correct than the sentence Wer_i meint Lydia, dass t_i Jakob liebt?.

   are data if the judgements are based on the linguistic intuition (introspection) of the linguist X or on an experiment carried out on a totality of 100 informants. Nevertheless, neither the sentence

   1. (c) Wer_i meint Lydia, liebt t_i Jakob?

   can be considered to be a datum, nor do (a) or (b), if the latter's plausibility stems from an indirect source, that is, if they are inferred from hypotheses of the theory.¹

   We interpret judgements as plausible statements. This means that we do not deem them true with certainty. This decision is motivated by the problems raised in Schütze (Reference Schütze1996) which centres around the complexity and the resulting uncertainty of judgements. The reason for this is, among other things, that non-linguist native speakers do not judge grammaticality directly, but rather they provide acceptability judgements. Acceptability judgements, however, are products of linguistic performance (see Schütze Reference Schütze1996: 24). Therefore, even if one evaluates the statement “Sentence X is unacceptable” to be true with certainty on the basis of the judgement of person Y, it cannot be stated that the statement “Sentence X is grammatically not correct” is also true with certainty, because the unacceptability may result not only from the contribution of linguistic competence but may be traced back to some other factor.

   Linguists often evaluate the grammaticality of sentences in a direct way when providing ‘introspective data’ for their theory. The term ‘(un)grammatical’, however, refers not only to their intuitive judgement about the correctness of the given sentence but inevitably presupposes some theory of grammar, too, which serves as a reference point against which violations of syntactic rules can be evaluated (and other kinds of violations of the rules of the language can be excluded). Thus, such statements are data, too, but the source on the basis of which they receive their plausibility value is more complex and cannot be reduced to the linguist's linguistic intuition alone.

2. (2) An example from Featherston (Reference Featherston2007: 275ff.) illustrates impressively the uncertainty of grammaticality judgements and the problems which arise from ignoring this uncertainty. Several syntacticians agreed on the grammaticality judgement according to which the sentences (a)--(c) are not better than (d):

   1. (a) Wen_i meint Lydia, liebt Jakob t_i?

      whom thinks Lydia loves Jakob t_i

      ‘Who does Lydia think Jakob loves?’

   2. (b) Wer_i meint Lydia, liebt t_i Jakob?

      who thinks Lydia loves t_i Jakob

      ‘Who does Lydia think loves Jakob?’

   3. (c) Wen_i meint Lydia, dass Jakob t_i liebt?

      whom thinks Lydia that Jakob t_i loves

      ‘Who does Lydia think that Jakob loves?’

   4. (d) Wer_i meint Lydia, dass t_i Jakob liebt?

      who thinks Lydia that t_i Jakob loves

      ‘Who does Lydia think that loves Jakob?’

   As Featherston remarks, Haider, Grewendorf, Stechow & Sternefeld, Bayer and Lutz presuppose that native speakers judge (a)--(d) similarly. Therefore, on the basis of their linguistic intuition, they regarded the statement

   1. (A) The sentences (a)--(d) are equally grammatical.

   as a certainly true datum. From this they concluded that A clearly and perfectly refutes the following hypothesis:

   1. (T) standard German, there is a that-trace effect. That is, subjects and objects behave differently in complement clauses with a complementiser insofar as only the latter can be extracted.

   Featherston refers to a paper by Grewendorf (see Grewendorf Reference Grewendorf, Jakobs, von Stechow, Sternefeld and Vennemann1995) where he writes some years later that in standard German, A is a datum that is true with certainty, while this is not the case in North German dialects. Conversely, Fanselow (Reference Fanselow1987) holds that A cannot be treated as a datum in South German.

   Featherston's experiments (see Featherston Reference Featherston2007: 276f.), however, yielded the result that A is not true with certainty – what is more, it is not even a plausible statement. Therefore, it cannot be treated as a datum. The participants of the experiments judged (a) and (b) to be better than (c) or (d). Consequently, on the basis of this experiment as a source one has to judge A to be implausible. Therefore, A cannot support the non-existence of the that-trace effect as part of an indirect source. Quite the contrary seems to be the case: Featherston's experiments serve with data in favour of T, since they can be interpreted as sources making A′ plausible:

   1. (A′) Sentences (a)--(b) are better than (c)--(d).

3. (3) Corpus data are statements capturing characteristics of utterances in some corpus; thus, they do not necessarily have the same structure. They may, for example, have the form ‘The utterance U containing the linguistic phenomenon P can be found in corpus C.’, or ‘The structure S can be identified in corpus C and it had the characteristics X’, etc. They often have the form ‘The sentence in Example x is contained in Corpus C’ and occur together with further data such as ‘The sentence in Example x contains the occurrence of the construction S’. They have to be interpreted as data in the above sense, that is, as plausible statements originating from a direct source. This source, however, is not the corpus alone. Rather, it is a compound of (sub-)sources such as the historical document itself, the linguist's linguistic intuition and his/her skills in the application of some linguistic theory, etc. The reliability of each of these sub-sources influences the plausibility of the datum. Therefore, corpus data usually cannot be considered to be certainly true statements. It is not only the case that factors deciding over the genuineness of the historical document are relevant; difficulties or uncertainties in the identification and interpretation of the given linguistic phenomenon must be taken into account as well. There are several reasons for this:

   1. (a) Both the occurrence of the investigated structure in the corpus (‘positive evidence’) and the lack of its occurrence (‘negative evidence’)² raise serious problems, which are unsolvable or at least difficult to handle with the methodologies at the researchers’ disposal:

      1. It is often difficult to decide how to differentiate relevant occurrences from insignificant, isolated, unreliable, dubious information.

      2. Linguists inevitably have to fall back on their own linguistic intuition when processing corpora. The interpretation of the utterances in the corpus, the identification of utterances containing the given linguistic phenomenon (syntactic structure, semantic or pragmatic phenomenon etc.) and the selection of the reliable examples require the active involvement of linguistic intuition; therefore, these operations contain subjective elements as well. For example, if a cognitive linguist searches for metaphors in a corpus, then he/she cannot rely on observable, ‘objective’ criteria. Therefore, he/she has to decide which expressions he/she is willing to consider – at least partly – metaphorical on the basis of his/her own linguistic intuition.

      3. The absence of a certain form or construction does not necessarily mean that this form/construction is faulty. It may happen that the phenomenon at issue appears rarely, and it can be found in a larger corpus or in a corpus covering another linguistic register.

   2. (b) ‘Quantitative data’ indicate that a given phenomenon's occurrence is statistically significant. Of course, the identification of the occurrences raises the problems mentioned in (a), too. The reliability of such examples also depends on the range of data which their relative frequency is compared to, as well as on the statistical method applied. Thus, for example, the representativeness of the corpus is a factor that has to be checked in order to obtain reliable results.

   3. (c) In connection with questionnaires, manifestations of the ‘observer's paradox’ (see Section 4.2) lead to uncertainty as well.

Meurers (Reference Meurers2007) enumerates several factors evincing that the use of corpus data presupposes the involvement of the linguist's linguistic intuition. Therefore, such data cannot be deemed certainly true statements:

1. The linguist has to decide which sentences of the corpus are relevant to a particular research problem. He/she has to select and reconstruct data. This requires the analysis of the sentences in the corpus with the help of the given theory and the identification of the syntactical, pragmatic etc. parameters.

2. One needs to decide whether a piece of information from the corpus is part of the language one is studying. This means that the normal, grammatical sentences have to be identified.

3. Corpora are finite representations of language use. Therefore, it is possible that they do not contain information relevant to the actual research problem.

1. (4) Adherents of the standard view of linguistic data and evidence (SVLD) (see Section 3.3) – followers of both the introspective and the corpus-based tradition – do not use statements related to ungrammatical (deviant) sentences produced during spontaneous language use as data. They simply presuppose that such sentences can be easily identified and do not contain any relevant information; therefore, they are useless and have to be dismissed from linguistic research. In contrast, Foster (Reference Foster2007) regards them as valuable and reliable (although not certainly true) data. She argues that

She points out that in order to create computer parsers that are capable of parsing ‘real’ sentences stemming from language production (and can eventually create these and similar sentences), one has to make a distinction not only between grammatical and ungrammatical sentences but between sentences which contain frequently and routinely committed, ‘realistic’ errors and which can be easily understood on the one hand, and sentences which contain invented, unrealistic mistakes on the other hand. Foster is of the opinion that this dual task can be accomplished only by relying on one's linguistic intuition. From this it follows that one has to deal with data which are not true with certainty but are inevitably affected by subjective factors, that is, with uncertain data:

We may conclude from Foster's argumentation that the standard view of linguistic data and evidence (SVLD) is built on a paradox: corpora can be regarded as perfectly reliable, objectively controllable sources only after their unreliable, faulty elements have been eliminated – and this can be done only by relying on our subjective opinion.

Examples 31(1)--(2)

1. (1) The p-context-dependence (theory-dependence) of data and the well-motivatedness of interpreting data as plausible statements rather than as instances or products of linguistic behaviour can be nicely illustrated with the help of the analysis of different interpretations of the annotations and degrees of ‘badness’ as summarised by Schütze (Reference Schütze1996: 44ff.):

   1. (a) In the generative literature, ungrammatical sentences are usually marked by asterisks. This, however, leaves open several possibilities of interpretation. According to Householder (Reference Householder1973: 370ff.), for example, ‘*X’ may be interpreted as

      1. ‘I would never say X’;

      2. ‘I have never seen or heard a sentence of the type of X and hereby wager you can't find an example’;

      3. ‘This is quite comprehensible, and I have heard people say it, but they were all K’s [foreigners, Southerners, etc.]; in my dialect we would say Y instead.’

   2. (b) The use of the question mark is ambiguous, too, since ‘?X’ may indicate that

      1. X shows interspeaker variation, i.e. X is good for some people but bad for others;

      2. most speakers rate X marginal;

      3. X has a very literary ring, but it is not unacceptable.

      Schütze cites Andrews (Reference Andrews, Maling and Zaenen1990: 203) who developed a system of symbols in order to make the meaning of the markings used explicit. This system involves a six-point scale:

      ✓:

      completely acceptable and natural;

      ?:

      acceptable, but perhaps somewhat unnatural;

      ??:

      doubtful, but perhaps acceptable;

      ?*:

      worse, but not totally unacceptable;

      *:

      thoroughly unacceptable;

      **:

      horrible.

   The application of this system means that it is not the linguistic examples themselves that one treats as data but rather statements about their acceptability such as ‘Sentence X is doubtful, but perhaps acceptable’ on the basis of the linguistic intuition of speaker Y as a direct source. Since the linguistic intuition of (single) native speakers cannot be regarded as a completely reliable source, one obtains plausible statements which may serve as data in the theory at issue. These data are theory-dependent, because they involve elements of the above system, which is, of course, only one of the possible scales which have been elaborated and can be applied in collecting and evaluating judgements from informants. In this way, acceptability judgements become integrated into the process of linguistic theorising, and they may function as starting points for the estimation of the plausibility value of hypotheses about the grammaticalness of the given sentence or structure. Namely, the degree of acceptability of a sentence cannot be equated with the degree of its grammaticalness, because the former is also determined by semantic, pragmatic, and general cognitive factors. Consequently, the plausibility of ‘X is ungrammatical’ is lower than the plausibility of ‘X is unacceptable’ on the basis of the linguistic intuition of a native speaker, and the plausibility of the statement ‘Structure Y is ungrammatical/bad’ is even lower. In such cases, the source on the basis of which the statements ‘X is ungrammatical’ and ‘Structure Y is ungrammatical/bad’ are evaluated is not direct, because these statements are inferred from acceptance judgement data and other information about other possible reasons of badness; therefore, in such p-contexts they are not data.

2. (2) Another aspect of the theory-dependence (p-context dependence) of data is that it may happen that a datum is evaluated to be true with certainty in one theory (p-context) but it may be deemed to be implausible (and, of course, not regarded as a datum) in another. Schütze (Reference Schütze1996: 38f.) mentions several such cases in order to illustrate the unreliability of unsystematically collected judgement data.

   One of his examples is the case of the alleged that-trace effect of adjunct wh-words. Lasnik & Saito (Reference Lasnik and Saito1984) deemed sentences like Why do you think that he left? ambiguous because in their opinion, why may relate to the reason for the thinking or to the reason for the leaving. In the terminology of the p-model this means that in this p-context (theory), the statement ‘The sentence Why do you think that he left? is ambiguous’ is a datum which is true with certainty on the basis of the two linguists’ linguistic intuition as a direct source. In the p-context (theory) of Aoun et al. (Reference Aoun, Hornstein, Lightfoot and Weinberg1987), however, similar statements are held to be implausible and not treated as data on the basis of two indirect sources. The first indirect source (system of inferences) refers to the uncertainty and inaccuracy of the judgement of informants who claim that they have two readings of such sentences. The second indirect source (system of inferences) bases the refusal to accept the existence of a second reading on the argument that if such sentences were grammatical, then certain similar sentences such as Who remembers what we bought why? should be acceptable, too, which is, however, not the case.

   Since the sources used by the two theories do not judge the plausibility/implausibility of the statements at issue unequivocally, they cannot be regarded as a set of sources on the basis of which a plausibility value could be assigned to the statements at issue. Schütze (Reference Schütze1996: 39) tries to re-evaluate the reliability of the sources mentioned. He comes to the conclusion that all sources used by the authors are unreliable. Since the judgements provided by the speakers he has surveyed are strongly divided, he concludes that without carefully conducted experiments it is not possible to make a decision. Therefore, the statement ‘The sentence Why do you think that he left? is ambiguous’ loses its data status because it has lost its plausibility value.

Examples 32(1)--(2)

1. (1) Wurzel (Reference Wurzel and Heidolph1981) aims to decide between the following rival hypotheses concerning the structure of German affricates:

   1. (h₁) The labial affricate is biphonemic, that is, its phoneme structure is /pf/.

   2. (h₂) The labial affricate is monophonemic, that is, its phoneme structure is /p^f/.

   The datum e₁ is weak evidence for hypothesis h₁:

   1. (e₁) Pfropfen is an existing German word,

   since with the help of e₁ as well as the following two plausible statements an indirect source can be created that makes h₁ plausible:

   1. (p₁) If a vowel /V/ follows a formative‐initial /C₃C₂C₁/ consonant cluster, then the consonant cluster /C₂C₁/ and /C₁/ also occur in the same position.

   2. (p₂) There exist German formatives with the structure /frɔ_ / and /rɔ_ /.

   With the help of the statements mentioned, we gain the following plausible inferences:

   
   It is easy to see that, on similar grounds, e₁ is weak evidence for h₁’s rival h₂, too.

2. (2) As we have already seen in Example 30(4), adherents of the standard view of linguistic data and evidence (SVLD) do not consider statements about ungrammatical sentences which have been obtained from spontaneous language use as relevant data. The main reason for this lies in their conviction that sentences of this kind result from the linguistic performance of the native speaker; therefore, it is not possible to infer properties of linguistic competence from them. As opposed to this, Arppe & Järvikivi (Reference Arppe and Järvikivi2007) – similarly to Foster (Reference Foster2007) – argue that data of this type, too, are relevant insofar as they may serve as (weak) evidence for the grammaticality of linguistic constructions:

   Slips of the Tongue, i.e., output errors that are produced by normal native speakers unintentionally and spontaneously (and sometimes unconsciously) are another well-documented case in point. These errors are not random, but instead follow the way the language system is organised…On many occasions, the speaker in fact corrects the error – therefore even the corrections can tell us quite a lot about how our linguistic system is organised…(Arppe & Järvikivi Reference Arppe and Järvikivi2007: 101f.; emphasis as in the original)

Example 33

In Example 32(1), e₁ is relative evidence for hypothesis h₂, since with the help of hypothesis p₃ a further indirect source can be obtained which makes h₂ plausible, while in the case of h₁, this is not the case:

1. (p₃) If the phoneme cluster /C₁C₂_/ occurs formative‐initially, then there exists a formative in which the phoneme cluster /_C₂C₁/ occurs formative‐finally.

2. (p₄) There exist German formatives with the structure /_rpf/.

With the help of these statements, the following plausible inferences can be formulated:

Here there is only one indirect source supporting h₁, while there are two for h₂. Moreover, the latter provide h₂ with a higher plausibility value than the first source in Example 32(1) does with h₁.

Examples 34(1)--(2)

1. (1) Bowdle & Gentner (Reference Bowdle and Gentner1999) mention the following rival hypotheses in cognitive metaphor research:

   1. (h₁) Metaphors are based on the comparison of two conceptual domains and on the matching of their related properties.

   2. (h₂) Metaphors are stable, unidirectional mappings between two conceptual domains.

   3. (h₃) Metaphors are special cases of categorisation.

   4. (h₄) Conventional metaphors are special cases of categorisation, while novel metaphors are special cases of analogy.

   According to the authors, results of psycholinguistic experiments comparing the interpretation and processing of conventional and novel metaphors make h₁–h₃ equally implausible because the latter describe all metaphors with the same processing model. They support only h₄ and work therefore as strong evidence for h₄ in the above sense.

2. (2) Hoffmann (Reference Hoffmann2007) presents a case study illustrating that it may happen that a problem cannot be solved solely by relying on corpus data since they serve as weak evidence for both the given hypothesis and its rival; experimental data, however, may function as strong evidence. He characterises the problematic nature of the starting p-context as follows:

   the corpus results allow for two competing hypotheses: on the one hand it is possible that sequences such as the way which the satire is achieved in…is just an accidental gap in the data that speakers would use just as they might the world which I was working in…On the other hand, it might be a construction that is not provided by the grammar, i.e., a systematic gap on par with pied piping with that/ø-relativisers…, which is generally considered ungrammatical…(Hoffmann Reference Hoffmann2007: 94; emphasis as in the original)

   That is, the statement

   1. (e₁) The corpus contains neither manner adjunct PPs with the structure which + P or P + that/ø, nor locative PPs with this structure.

   is interpreted by Hoffmann as weak evidence for hypotheses h₁ and h₂ alike:

   1. (h₁) Manner adjunct PPs and locative adjunct PPs of the structure which + P, P + that/ø are grammatical but rare.

   2. (h₂) Manner adjunct PPs and locative adjunct PPs of the structure which + P, P + that/ø are ungrammatical.

   The reason for this decision is that both hypotheses can be supported by plausible inferences which make use of e₁ as a premise:
   
   

   In order to take a well-founded decision between h₁ and h₂, Hoffmann extends the p-context with new data:

   1. (e₂) Participants of an introspection experiment judged the structures in the following order acceptable:

      V + P > which + P (locative adjunct) > which + P (manner adjunct) >>>> P + that/ø = word order violations = agreement errors etc.

   Referring to Sorace & Keller (Reference Sorace and Keller2005), Hoffmann makes use of e₂ as follows:

   
   To sum up, e₂ is strong evidence for hypothesis h₁ and against h₂ in the sense of (51), since inferences were set up that made h₁ plausible and h₂ implausible.

Motivation and background

It is easy to see that none of the notions of ‘evidence’ gained by the p-model satisfies the criteria laid down by the standard view of the analytical philosophy of science (SVAPS). The former do not provide criteria whose fulfilment could ensure that the datum in question perfectly supports or refutes the given hypothesis. First, since the connection between the datum and the hypothesis is in most cases established by plausible inferences relying on plausible premises, they cannot guarantee the truth or falsity of the hypothesis. Second, nothing excludes the possibility of gaining evidence against the given hypothesis on the basis of other data.

(49)--(51) are related to situations in which the datum does not guarantee but only supports or questions the given hypothesis to a certain extent. However, as a limiting case (51)(a) also covers situations in which it is guaranteed that the hypothesis is true with certainty. This is the case if the evidence e is certainly true and the p-context contains statements with the help of which e can be extended into a demonstrative inference the conclusion of which is the hypothesis h. These requirements are, of course, extremely strict stipulations.