Artificial Intelligence and Natural Language Processing

How do infants learn a language? Why and how do languages evolve? How do we understand a sentence? This book explores these questions using recent computational models that shed new light on issues related to language and cognition. The chapters in this collection propose original analyses of specific problems and develop computational models that have been tested and evaluated on real data. Featuring contributions from a diverse group of experts, this interdisciplinary book bridges the gap between natural language processing and cognitive sciences. It is divided into three sections, focusing respectively on models of neural and cognitive processing, data driven methods, and social issues in language evolution. This book will be useful to any researcher and advanced student interested in the analysis of the links between the brain and the language faculty.

A diacritic is a mark placed near or through a character to alter its original phonetic or orthographic value. Many languages around the world use diacritics in their orthography, whatever the writing system the orthography is based on. In many languages, diacritics are ignored either by convention or as a matter of convenience. For users who are not familiar with the text domain, the absence of diacritics within text has been known to cause mild to serious readability and comprehension problems. However, the absence of diacritics in text causes near-intractable problems for natural language processing systems. This situation has led to extensive research on diacritization. Several techniques have been applied to address diacritic restoration (or diacritization) but the existing surveys of techniques have been restricted to some languages and hence left gaps for practitioners to fill. Our survey examined diacritization from the angle of resources deployed and various formulation employed for diacritization. It was concluded by recommending that (a) any proposed technique for diacritization should consider the language features and the purpose served by diacritics, (b) that evaluation metrics needed to be more rigorously defined for easy comparison of performance of models.

The semantic information of documents needs to be represented because it is the basis for many applications, such as document summarization, web search, and text analysis. Although many studies have explored this problem by enriching document vectors with the relatedness of the words involved, the performance remains far from satisfactory because the physical boundaries of documents hinder the evaluation of the relatedness between words. To address this problem, we propose an effective approach to further infer the implicit relatedness between words via their common related words. To avoid overestimation of the implicit relatedness, we restrict the inference in terms of the marginal probabilities of the words based on the law of total probability. The proposed method measures the relatedness between words, which is confirmed theoretically and experimentally. Thorough evaluation on real datasets illustrates that significant improvement on document clustering has been achieved with the proposed method compared with state-of-the-art methods.

It is a challenging task to discover information from a large amount of data in an open domain.1 In this paper, an event network framework is proposed to address this challenge. It is in fact an empirical construct for exploring open information, composed of three steps: document event detection, event network construction and event network analysis. First, documents are clustered into document events for reducing the impact of noisy and heterogeneous resources. Secondly, linguistic units (e.g., named entities or entity relations) are extracted from each document event and combined into an event network, which enables content-oriented retrieval. Then, in the final step, techniques such as social network or complex network can be applied to analyze the event network for exploring open information. In the implementation section, we provide examples of exploring open information via event network.

Charles Wayne restarted funding in speech and language in the mid-1980s after a funding winter brought on by Pierce’s glamour-and-deceit criticisms in the ALPAC report and ‘Whither Speech Recognition’. Wayne introduced a new glamour-and-deceit-proof idea, an emphasis on evaluation. No other sort of program could have been funded at the time, at least in America. One could argue that Wayne has been so successful that the program no longer needs him to continue on. These days, shared tasks and leaderboards have become common place in speech and language (and vision and machine learning) research. That said, I am concerned that the community may not appreciate what it has got until it’s gone. Wayne has been doing much more than merely running competitions, but he did what he did in such a subtle Columbo-like way. Going forward, government funding is being eclipsed by consumer markets. Those of us with research to sell need to find more and more ways to be relevant to potential sponsors given this new world order.

This paper addresses the feasibility of cross-lingual parsing with Universal Dependencies (UD) between Romance languages, analyzing its performance when compared to the use of manually annotated resources of the target languages. Several experiments take into account factors such as the lexical distance between the source and target varieties, the impact of delexicalization, the combination of different source treebanks or the adaptation of resources to the target language, among others. The results of these evaluations show that the direct application of a parser from one Romance language to another reaches similar labeled attachment score (LAS) values to those obtained with a manual annotation of about 3,000 tokens in the target language, and unlabeled attachment score (UAS) results equivalent to the use of around 7,000 tokens, depending on the case. These numbers can noticeably increase by performing a focused selection of the source treebanks. Furthermore, the removal of the words in the training corpus (delexicalization) is not useful in most cases of cross-lingual parsing of Romance languages. The lessons learned with the performed experiments were used to build a new UD treebank for Galician, with 1,000 sentences manually corrected after an automatic cross-lingual annotation. Several evaluations in this new resource show that a cross-lingual parser built with the best combination and adaptation of the source treebanks performs better (77 percent LAS and 82 percent UAS) than using more than 16,000 (for LAS results) and more than 20,000 (UAS) manually labeled tokens of Galician.

Automatic term extraction has become a priority area of research within corpus processing. Despite the extensive literature in this field, there are still some outstanding issues that should be dealt with during the construction of term extractors, particularly those oriented to support research in terminology and terminography. In this regard, this article describes the design and development of DEXTER, an online workbench for the extraction of simple and complex terms from domain-specific corpora in English, French, Italian and Spanish. In this framework, three issues contribute to placing the most important terms in the foreground. First, unlike the elaborate morphosyntactic patterns proposed by most previous research, shallow lexical filters have been constructed to discard term candidates. Second, a large number of common stopwords are automatically detected by means of a method that relies on the IATE database together with the frequency distribution of the domain-specific corpus and a general corpus. Third, the term-ranking metric, which is grounded on the notions of salience, relevance and cohesion, is guided by the IATE database to display an adequate distribution of terms.

Summary

Imagine a personal software agent engaging in electronic commerce on your behalf. Say the task of this agent is to track goods available for sale in various online venues over time, and to purchase some of them on your behalf for an attractive price. In order to be successful, your agent will need to embody your preferences for products, your budget, and in general your knowledge about the environment in which it will operate. Moreover, the agent will need to embody your knowledge of other similar agents with which it will interact (e.g., agents who might compete with it in an auction, or agents representing store owners) – including their own preferences and knowledge. A collection of such agents forms a multiagent system.

– Yoav Shoham and Kevin Leyton-Brown [2008, page xvii]

What should an agent do when there are other agents, with their own goals and preferences, who are also reasoning about what to do? An intelligent agent should not ignore other agents or treat them as noise in the environment. This chapter considers the problems of determining what an agent should do in an environment that includes other agents who have their own values.

Multiagent Framework

This chapter considers environments that contain multiple agents, with the following assumptions:

Summary

There is a real world with real structure. The program of mind has been trained on vast interaction with this world and so contains code that reflects the structure of the world and knows how to exploit it. This code contains representations of real objects in the world and represents the interactions of real objects. The code is mostly modular…, with modules for dealing with different kinds of objects and modules generalizing across many kinds of objects…. The modules interact in ways that mirror the real world and make accurate predictions of how the world evolves….

You exploit the structure of the world to make decisions and take actions. Where you draw the line on categories, what constitutes a single object or a single class of objects for you, is determined by the program of your mind, which does the classification. This classification is not random but reflects a compact description of the world, and in particular a description useful for exploiting the structure of the world.

– Eric B. Baum [2004, pages 169–170]

This chapter is about how to represent individuals (things, objects) and relationships among them. As Baum suggests in the quote above, the real world contains objects and we want compact representations of those objects. Such representations can be much more compact than representations in terms of features alone. This chapter considers logical representations and gives detailed examples of how to use such representations for natural language interfaces to databases. Later chapters address ontologies and the meaning of symbols, implementing knowledge-based systems, relational planning, relational learning, and probabilistic relational models.

Exploiting Relational Structure

One of the main lessons of AI is that successful agents exploit the structure of the world. Previous chapters showed how states can be represented in terms of features. Representing domains using features can be much more compact than representing them using states explicitly, and algorithms can exploit this compactness. There is, however, usually much more structure that can be exploited for representation and inference.

Summary

By a hierarchic system, or hierarchy, I mean a system that is composed of interrelated subsystems, each of the latter being in turn hierarchic in structure until we reach some lowest level of elementary subsystem. In most systems of nature it is somewhat arbitrary as to where we leave off the partitioning and what subsystems we take as elementary. Physics makes much use of the concept of “elementary particle,” although the particles have a disconcerting tendency not to remain elementary very long …

Empirically a large proportion of the complex systems we observe in nature exhibit hierarchic structure. On theoretical grounds we would expect complex systems to be hierarchies in a world in which complexity had to evolve from simplicity.

– Herbert A. Simon [1996]

This chapter shows how an intelligent agent can perceive, reason, and act over time in an environment. In particular, it considers the internal structure of an agent. As Simon points out in the quote above, hierarchical decomposition is an important part of the design of complex systems such as intelligent agents. This chapter presents ways to design agents in terms of hierarchical decompositions and ways that agents can be built, taking into account the knowledge that an agent needs to act intelligently.

Agents

An agent is something that acts in an environment. An agent can, for example, be a person, a robot, a dog, a worm, a lamp, a computer program that buys and sells, or a corporation.

Agents interact with the environment with a body. An embodied agent has a physical body. A robot is an artificial purposive (page 13) embodied agent. Sometimes agents that act only in an information space are called robots or bots, but we just refer to those as agents.

Agents receive information through their sensors. An agent's actions depend on the information it receives from its sensors. These sensors may, or may not, reflect what is true in the world. Sensors can be noisy, unreliable, or broken, and even when sensors are reliable there still may be ambiguity about the world given the sensor readings. An agent must act on the information it has available. Often this information is very weak, for example, “sensor s appears to be producing value v.”

Summary

Computation is the fire in our modern-day caves. By 2056, the computational revolution will be recognised as a transformation as significant as the industrial revolution. The evolution and widespread diffusion of computation and its analytical fruits will have major impacts on socioeconomics, science and culture.

– Eric Horvitz [2006]

In this chapter, we stand back and give a big-picture view of artificial intelligence in terms of the agent design space (page 21), and the future of AI. By placing many of the representation schemes in the agent design space, the relationships among the representations become more apparent. This allows us to see where the frontier of AI research now lies and to get a sense of the evolution of the field. We also consider some of the many social and ethical consequences that have arisen from the development and application of intelligent computational agents. As Horvitz points out in the quote above, computation is changing the world; we must be aware of its positive and negative impacts.

Dimensions of Complexity Revisited

What has AI research achieved? Where do the current frontier issues lie? To get a systematic sense of the big picture, the agent design space (page 21) for AI systems was described in terms of ten dimensions. It is instructive to see how representations presented in the book can be positioned in that space.

Figure 16.1 (on the next page) reviews the dimensions of complexity and classifies, in terms of the values for each dimension, some of the representations we have covered.

Agent Models

Hierarchical control (page 56) allows for hierarchical reasoning. As presented, it did not involve planning or goals, but it can be combined with other techniques. For example, it is possible to do reinforcement learning at multiple levels of a hierarchy, or even to learn the hierarchy.

State-space search, as presented in Chapter 3, allows for an indefinite horizon but otherwise gives the simplest value in all the other dimensions. Regression planning (page 249), using either the STRIPS representation or the feature-based representation extends state-space search to reason in terms of features. Constraint satisfaction problem (CSP) planning (page 252) allows for pruning the search space based on both the initial situation and the goal, but at the cost of planning for only a finite stage.

Summary

He who every morning plans the transaction of the day and follows out that plan, carries a thread that will guide him through the maze of the most busy life. But where no plan is laid, where the disposal of time is surrendered merely to the chance of incidence, chaos will soon reign.

– Victor Hugo (1802–1885)

Planning is the process of finding a sequence of actions to achieve a goal. Because an agent does not usually achieve its goals in one step, what it should do at any time depends on what it will do in the future. What it will do in the future depends on the state it is in, which, in turn, depends on what it has done in the past. This chapter presents three representation of actions and their effects, and four offline algorithms for an agent to find a plan to achieve its goals from a given state.

This chapter makes the following simplifying assumptions:

Summary

Every task involves constraint,

Solve the thing without complaint;

There are magic links and chains

Forged to loose our rigid brains.

Structures, strictures, though they bind,

Strangely liberate the mind.

– James Falen

Instead of reasoning explicitly in terms of states, it is typically better to describe states in terms of features and to reason in terms of these features. Features are described using variables. Often features are not independent and there are hard constraints that specify legal combinations of assignments of values to variables. As Falen's elegant poem emphasizes, the mind discovers and exploits constraints to solve tasks. In planning and scheduling an agent assigns a time for each action. These assignments must satisfy constraints on the order actions can be carried out and constraints specifying that the actions achieve a goal. Preferences over assignments are specified in terms of soft constraints. This chapter shows how to generate assignments that satisfy hard constraints and optimize soft constraints.

PossibleWorlds, Variables, and Constraints

Variables and Worlds

Constraint satisfaction problems are described in terms of variables and possible worlds. A possible world is a possible way the world (the real world or some imaginary world) could be.

Possible worlds are described by algebraic variables. An algebraic variable is a symbol used to denote features of possible worlds. For this chapter, we refer to an algebraic variable simply as a variable. Algebraic variables are written starting with an upper-case letter. Each algebraic variable X has an associated domain, dom(X), which is the set of values the variable can take.

A discrete variable is one whose domain is finite or countably infinite. A binary variable is a discrete variable with two values in its domain. One particular case of a binary variable is a Boolean variable, which is a variable with domain ﹛true, false﹜. We can also have variables that are not discrete; for example, a variable whose domain corresponds to the real line or an interval of the real line is a continuous variable.

Summary

A plan is like the scaffolding around a building. When you're putting up the exterior shell, the scaffolding is vital. But once the shell is in place and you start to work on the interior, the scaffolding disappears. That's how I think of planning. It has to be sufficiently thoughtful and solid to get the work up and standing straight, but it cannot take over as you toil away on the interior guts of a piece. Transforming your ideas rarely goes according to plan.

– Twyla Tharp [2003]

In the quote above, Tharp is referring to dance, but the same idea holds for any agent when there is uncertainty. An agent cannot just plan a fixed sequence of steps; the result of planning needs to be more sophisticated. Planning must take into account the fact that an agent in the real world does not know what will actually happen when it acts. An agent should plan to react to its environment.

What an agent should do at any time depends on what it will do in the future. When an agent cannot precisely predict the effects of its actions, what it will do in the future depends on what it does now and what it will observe before it acts.

With uncertainty, an agent typically cannot guarantee to satisfy its goals, and even trying to maximize the probability of achieving a goal may not be sensible. For example, an agent whose goal is to minimize the probability of injury in a car accident would not get into a car or walk down a sidewalk or even go to the ground floor of a building, each of which increases the probability of being injured in a car accident, however slightly. An agent that does not guarantee to satisfy a goal can fail in many ways, some of which may be much worse than others.

This chapter is about how to take planning, reacting, observing, succeeding and failing into account simultaneously.

An agent's decision on what to do depends on three things:

Artificial Intelligence and Natural Language Processing

Refine search

Refine search

Actions for selected content:

3242 results in Artificial Intelligence and Natural Language Processing

Language, Cognition, and Computational Models

A survey of diacritic restoration in abjad and alphabet writing systems

Formalizing Natural Languages: The NooJ Approach, by Max Silberztein . London, Cognitive science series, Wiley-ISTE, UK, 2016. ISBN 9781848219021

Unsupervised learning of semantic representation for documents with the law of total probability

Exploring open information via event network †

NLE volume 23 issue 6 Cover and Front matter

NLE volume 23 issue 6 Cover and Back matter

Emerging trends: A tribute to Charles Wayne

New treebank or repurposed? On the feasibility of cross-lingual parsing of Romance languages with Universal Dependencies †

DEXTER: A workbench for automatic term extraction with specialized corpora †

11 - Multiagent Systems

Summary

13 - Individuals and Relations

Summary

2 - Agent Architectures and Hierarchical Control

Summary

Part II - Reasoning, Planning and Learning with Certainty

16 - Retrospect and Prospect

Summary

6 - Planning with Certainty

Summary

4 - Reasoning with Constraints

Summary

9 - Planning with Uncertainty

Summary

Part IV - Reasoning, Learning and Acting with Individuals and Relations

Contents

Artificial Intelligence and Natural Language Processing

Refine search

Refine search

Actions for selected content:

Save Search

3242 results in Artificial Intelligence and Natural Language Processing

Language, Cognition, and Computational Models

Summary

Summary

Summary

Summary

Summary

Summary

Summary