We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Educational assessment applications, as well as other natural-language interfaces, need some mechanism for validating user responses. If the input provided to the system is infelicitous or uncooperative, the proper response may be to simply reject it, to route it to a bin for special processing, or to ask the user to modify the input. If problematic user input is instead handled as if it were the system's normal input, this may degrade users' confidence in the software, or suggest ways in which they might try to “game” the system. Our specific task in this domain is the identification of student essays which are “off-topic”, or not written to the test question topic. Identification of off-topic essays is of great importance for the commercial essay evaluation system CriterionSM. The previous methods used for this task required 200–300 human scored essays for training purposes. However, there are situations in which no essays are available for training, such as when users (teachers) wish to spontaneously write a new topic for their students. For these kinds of cases, we need a system that works reliably without training data. This paper describes an algorithm that detects when a student's essay is off-topic without requiring a set of topic-specific essays for training. This new system is comparable in performance to previous models which require topic-specific essays for training, and provides more detailed information about the way in which an essay diverges from the requested essay topic.
The preceding treatment of reasoning indicates how we can interpret psychological rationality in terms of mechanical processes. Let us now look at the ways in which mechanical concepts enter into characterizing forms of economic rationality.
Limits on rationality
The difficulty and slowness with which real agents change their mental state constitutes one of the most evident limitations on rationality. As noted earlier, we can see reflections of the mechanical connection between momentum and force in “the more you need to change, the more you have to force yourself,” “the more you know, the harder it is to change your mind,” and other truisms of popular psychology. We can read the first of these truisms as stating a monotonicity relation between the size of changes and the size of the required forces and work done, and the second as stating a monotonicity relation between the mass and the force required for given changes. Notions of monotonicity and proportionality among the numerical magnitudes of momentum and force are familiar in traditional mechanics, but how do these apply in the discrete mechanical setting?
A mechanical interpretation of thinking also naturally relates slowness of change to inertia. From the same perspective, the unreality of ideal rationality appears because when we determine actions by finding the maxima of an expected utility function generated by instantaneous beliefs and desires, large changes can come from small impulses.
Space and Time! now I see it is true, what I guess'd at,
What I guess'd when I loafed on the grass,
What I guess'd while I lay alone on my bed,
And again as I walk'd alone the beach under the paling stars of the morning.
(Walt Whitman, Song of Myself)
The preceding chapters presented the beginnings of a mathematical and mechanical theory of mind.
We began by examining the curious divorce between mechanical understandings of mind and nature that occurred when natural philosophy developed mathematical techniques useful in characterizing physical mechanics but inapplicable to mental mechanics. The mathematical study of mental materials developed separately, but with the key mathematical theories of logical and economic rationality lacking any connection to mechanics. The mechanical reconciliation of mind and nature began to take shape only when the development of artificial computers enabled construction of artificial minds precise and concrete enough to relate to a new rational mechanics broad enough to encompass mental as well as physical materials. The reconciliation promises not only to open traditional philosophical questions to new forms of technical analysis, but also to provide a new formal vocabulary for describing agents of limited rationality and for engineering computational and social systems based on such agents.
We then examined two sides of the reconciliation of physical and mental mechanics. On the physical side, we recast the axioms of modern rational mechanics so as to cover discrete mechanical systems and their hybrids with physical mechanical systems.
The preceding development of mental mechanics does not require determinism of mechanical systems. It instead requires only that motion satisfy mechanical relationships independent of determinism requirements.
The preceding chapters also illustrated several sources of possible indeterminacy. Reasoning, whether habitual or deliberate, can produce indeterminism when several reasons apply at the same instant, requiring serialization or conflict resolution. In addition, rational deliberation can result in several possible self-constructions from reasoning rules; conservative update in response to reasoned changes can follow multiple resolutions; and volition can encompass multiple choices of action on the basis of the same desires and intentions. These sources of mechanical indeterminism complement the forms of indeterminism acknowledged in traditional mechanics, including situations of indeterministic collapse and bifurcation considered in continuum mechanics and the pervasive indeterminacy of quantum physics. All of these forms of indeterminacy represent theoretical allowances of multiple possibilities that stand separate from uncertainties arising from the practicalities of measurement connected with repeatability and resolution of measuring apparati.
From the viewpoint of psychology, mechanical indeterminism generates what one can call a “kinematical” notion of uncertainty, in which one seeks to measure the amount of indeterminism, or degree of uncertainty about predictions introduced by indeterminism. In the simplest terms, qualities of motion shared by all possible histories represent certain predictions about motion, while qualities exhibited by some histories but not by others represent uncertain predictions about motion. The kinematic conception of uncertainty provides means for comparing these degrees of certainty and uncertainty in quantitative terms.
Understanding psychology and economics in mechanical terms requires looking at specific concepts of psychology and economics from the mechanical point of view. If we look to the literature, however, we find that the cognitive sciences study a wide range of possible or hypothesized psychological organizations as explanations of human thought. For example, the ideally rational agents of economics have one kind of mind, a kind very different from almost all known human minds. But even among humans, individual minds have very different characters, exhibiting different levels of intelligence at different tasks, different temperaments, different degrees of adaptability, and so on. The well-known Myers–Briggs test (Myers & Myers 1980), to give another example, sorts minds into sixteen well-populated classes. These classes correspond to recognizable and common types of personal character, types that give some insight and enable reasonable, though not perfect, predictions of individual behavior.
It does not take deep reflection to realize that if we are already on page 225 and just starting the mechanical examination of psychology and economics, we cannot hope to examine all the concepts of all hypothesized mental organizations in this book, no matter how long, without exhausting all patience. I therefore undertake to examine the structure and mechanical nature of some special kinds of minds that serve to illustrate the mechanical nature of thinking, in part to open the special classes to mechanical investigation, and in part to suggest ways of understanding other kinds of minds in mechanical terms.
This paper describes a novel computer-aided procedure for generating multiple-choice test items from electronic documents. In addition to employing various Natural Language Processing techniques, including shallow parsing, automatic term extraction, sentence transformation and computing of semantic distance, the system makes use of language resources such as corpora and ontologies. It identifies important concepts in the text and generates questions about these concepts as well as multiple-choice distractors, offering the user the option to post-edit the test items by means of a user-friendly interface. In assisting test developers to produce items in a fast and expedient manner without compromising quality, the tool saves both time and production costs.
Many theories of psychological organization posit both long-term and short-term memories. The long-term memories serve as persistent (but not necessarily perfect) repositories of knowledge, skills, and other elements of human capital; the short-term memories serve to store the fleeting facts of present experience, which then either are discarded or incorporated into long-term memory.
The notion of memory in these theories concerns the function of memory structures in thinking, but this function has mainly to do with issues of persistence, not with the content of memory. In common theories, memory content is assumed to contain elements of what we can call the outlook, point of view, or attitudes of the agent, as well as habits, skills, and other aspects of mind.
This chapter examines the notions of memory and outlook from the mechanical point of view, without adopting a position on the exact set of mental elements that define outlook. The fundamental identifications explored take mental outlook to constitute mental position, and memory to consist of both mental mass and persistent aspects of internal configuration reflected in the position. Thinking of memory as mass and configuration fits well with everyday usage. Mass persists across motion, and this also holds for long-term memory; some aspects of configuration, such as the support one belief has in others, also persist and can be used in explaining behavior. Thinking of mental attitudes as positions also finds a good home in everyday usage.
One of the most difficult challenges faced by non-native speakers of English is mastering the system of English articles. We trained a maximum entropy classifier to select among a/an, the, or zero article for noun phrases (NPs), based on a set of features extracted from the local context of each. When the classifier was trained on 6 million NPs, its performance on published text was about 83% correct. We then used the classifier to detect article errors in the TOEFL essays of native speakers of Chinese, Japanese, and Russian. These writers made such errors in about one out of every eight NPs, or almost once in every three sentences. The classifier's agreement with human annotators was 85% (kappa = 0.48) when it selected among a/an, the, or zero article. Agreement was 89% (kappa = 0.56) when it made a binary (yes/no) decision about whether the NP should have an article. Even with these levels of overall agreement, precision and recall in error detection were only 0.52 and 0.80, respectively. However, when the classifier was allowed to skip cases where its confidence was low, precision rose to 0.90, with 0.40 recall. Additional improvements in performance may require features that reflect general knowledge to handle phenomena such as indirect prior reference. In August 2005, the classifier was deployed as a component of Educational Testing Service's Criterion$^{SM}$ Online Writing Evaluation Service.
The common picture of mechanics embodies many unfortunate misconceptions about the nature, scope, and structure of mechanics, with many people having the idea that mechanics consists of applying to physical systems the three axioms stated by Newton. Applying mechanics to psychology and economics requires a firmer theoretical basis than that provided by popular misconceptions. To proceed, we thus must confront and set aside mechanical misconceptions, lest the misconceptions prevent proper appreciation of the contribution mechanics makes to understanding the world. Accordingly, the present chapter examines the nature of mechanics at a high level, reconsidering the content and form of mechanical theories in light of the history of mechanical concepts and mathematical formalisms. This examination highlights the common misconceptions and how they divert one from the proper understanding needed for the following development.
Readers wishing to skip this somewhat philosophical discussion in favor of the development of the mechanical axioms themselves might proceed directly to Chapters 5 and 6, which review the structure and content of the axioms of modern rational mechanics. The modern axioms have enjoyed widespread use for decades among mathematicians studying mechanics and among mechanical engineers, although not in beginning physics textbooks. In contrast to the postulates of popular legend, the modern axioms provide a formal characterization of the notion of force, and reveal the true generality of mechanics in ways that usual textbook presentations do not.
We examine correlations between dialogue behaviors and learning in tutoring, using two corpora of spoken tutoring dialogues: a human-human corpus and a human-computer corpus. To formalize the notion of dialogue behavior, we manually annotate our data using a tagset of student and tutor dialogue acts relative to the tutoring domain. A unigram analysis of our annotated data shows that student learning correlates both with the tutor's dialogue acts and with the student's dialogue acts. A bigram analysis shows that student learning also correlates with joint patterns of tutor and student dialogue acts. In particular, our human-computer results show that the presence of student utterances that display reasoning (whether correct or incorrect), as well as the presence of reasoning questions asked by the computer tutor, both positively correlate with learning. Our human-human results show that student introductions of a new concept into the dialogue positively correlates with learning, but student attempts at deeper reasoning (particularly when incorrect), and the human tutor's attempts to direct the dialogue, both negatively correlate with learning. These results suggest that while the use of dialogue act n-grams is a promising method for examining correlations between dialogue behavior and learning, specific findings can differ in human versus computer tutoring, with the latter better motivating adaptive strategies for implementation.
This paper evaluates four of the most commonly used, freely available, state-of-the-art parsers on a standard benchmark as well as with respect to a set of data relevant for measuring text cohesion, as one example of a learning technology application that requires fast and accurate syntactic parsing. We outline advantages and disadvantages of existing technologies and make recommendations. Our performance report uses traditional measures based on a gold standard as well as novel dimensions for parsing evaluation. To our knowledge, this is the first attempt to evaluate parsers across genres and grade levels for the implementation in learning technology using both gold standard and directed evaluation methods.