So far we have used our ASP knowledge bases to get information about the truth or falsity of some statements or to find objects satisfying some simple properties. These types of tasks are normally performed by database systems. Even though the language's ability to express recursive definitions and the methodology of representing defaults and various forms of incomplete information gave us additional power and allowed us to construct rich and elaboration-tolerant knowledge bases, the types of queries essentially remained the same as in databases.

In this chapter we illustrate how significantly different computational problems can be reduced to finding answer sets of logic programs. The method of solving computational problems by reducing them to finding the answer sets of ASP programs is often called the answer-set programming (ASP) paradigm. It has been used for finding solutions to a variety of programming tasks, ranging from building decision support systems for the Space Shuttle and computer system configuration to solving problems arising in bio-informatics, zoology, and linguistics. In principle, any NP-complete problem can be solved in this way using programs without disjunction. Even more complex problems can be solved if disjunctive programs are used. In this chapter we illustrate the ASP paradigm by several simple examples. More advanced examples involving larger knowledge representation components are discussed in later chapters.

There are currently several ASP inference engines called ASP solvers capable of computing answer sets of programs with millions of ground rules.

In a stochastic control problem one observes and then seeks to favorably influence the behavior of some stochastic system. Such problems involve dynamic optimization, meaning that observations and actions are spread out in time. In this chapter several simple but fundamental stochastic control problems will be solved directly from first principles, with heavy reliance on the ubiquitous Ito formula. For each of the closely related problems to be considered, the optimal policy involves the imposition of control barriers; these may be either jump barriers or reflecting barriers, depending on the cost structure assumed.

Our first problem can be informally described as follows. Consider a controller who continuously monitors the contents of a storage system, such as an inventory or a bank account. In the absence of control, the contents process Z = {Zt, t ≥ 0} fluctuates as a (μ,σ) Brownian motion. The controller can at any time increase or decrease the contents of the system by any amount desired, but is obliged to keep Zt ≥ 0, and there are three types of costs to be considered. First, to increase the contents from x to x + δ, the controller must pay a fixed charge K plus a proportional charge kδ. Similarly, it costs L + lδ to decrease the contents from x to x – δ. Finally, inventory holding costs are continuously incurred at rate kZt.

The initial sections of this chapter are devoted to the definition of Brownian motion (the mathematical object, not the physical phenomenon) and a compilation of its basic properties. The properties in question are quite deep, and readers will be referred elsewhere for proofs. Later sections are devoted to the derivation of further properties and to calculation of several interesting distributions associated with Brownian motion.

Before proceeding, readers are advised to at least look through Appendices A and B, which enunciate some standing assumptions (in particular, joint measurability and right-continuity of stochastic processes) and explain several important conventions regarding notation and terminology. As noted there, and in the Guide to Notation and Terminology, the value of a stochastic process X at time t may be written either as Xt or as X(t), depending on convenience. The former notation is generally preferred, but the latter is used when necessary to avoid clumsy typography like subscripts on subscripts.

Wiener's theorem

A stochastic process X is said to have independent increments if the random variables X(t0), X(t1) – X(t0),…, X(tn) – X(tn-1) are independent for any n ≥ 1 and 0 ≤ t0 < … < tn < ∞. It is said to have stationary independent increments if moreover the distribution of X(t) – X(s) depends only on t – s. Finally, we write Z ∼ N(μ,σ2) to mean that the random variable Z has the normal distribution with mean μ and variance σ2.

The first three sections of this appendix are concerned with notation and terminology. Readers should particularly note the standing assumptions such as joint measurability of stochastic processes. The last two sections are brief, stating without proof a basic result from martingale theory and a useful version of Fubini's theorem.

A filtered probability space

In the mathematical theory of probability, one begins with an abstract space Ω, a σ-algebra ℱ on Ω, and a probability measure P on (Ω, ℱ). The pair (Ω, ℱ) is called a measurable space and the triple (Ω, ℱ, P) is called a probability space. Individual points ω ∈ Ω represent possible outcomes for some experiment (broadly defined) in which we are interested. Identifying an appropriate outcome space Ω is always the first step in probabilistic modeling. Then F specifies the set of all events(subsets of Ω) to which we are prepared to assign probability numbers. Finally, the probability numbers P(·) reflect the relative likelihood of various events, whatever that may be interpreted to mean, and their specification is the second major step in probabilistic modeling. In economics one frequently interprets the probability measure P as a quantification of the subjective uncertainty experienced by some rational economic agent. Most physical scientists feel the need for a stronger, objective interpretation related to physical frequency.

A commonly used model of “sequential learning” is the following. A decision maker is initially uncertain about a parameter θ, and in each period t = 1,2,… he or she observes a random variable Xt = θ + ∈t, where {∈t} is a sequence of independent “noise” terms, each distributed N(0,1). Hereafter the random variable Xt will be referred to as a sample, and the process of observing successive Xt values will be called sampling. The strong law of large numbers says that (X1 + … + Xt)/t → θ almost surely as t → ∞, so the value of θ will eventually be revealed, but estimates of θ based on limited sampling are often important. For example, the decision maker may be considering an investment whose expected return depends on θ, and delaying the decision may have negative consequences because of discounting, or because there are direct costs associated with continued sampling. In such problems one is led to ask after each sample whether current knowledge of θ is “good enough” to justify an immediate acceptance or rejection of the investment opportunity, as opposed to continued sampling.

In the first five sections of this chapter we consider a continuous-time version of the model described above, in which the decision maker observes Yt = θt, + Wt, t ≥ 0, where W is a standard Brownian motion. We adopt a Bayesian framework, so the decision maker's initial information is expressed in the form of a prior distribution for θ, and the dynamic inference problem is to determine the posterior distribution of θ given {Ys, 0 ≤ s ≤ t}.

This is an expanded and updated version of a book that I published in 1985 with John Wiley and Sons, titled Brownian Motion and Stochastic Flow Systems. Like the original, it fits comfortably under the heading of “applied probability,” its primary subjects being (i) stochastic system models based on Brownian motion, and (ii) associated methods of stochastic analysis.

Here the word “system” is used in the engineer's or economist's sense, referring to equipment, people, and operating procedures that work together with some economic purpose, broadly construed. Examples include telephone call centers, manufacturing networks, cash management operations, and data storage centers. This book emphasizes dynamic stochastic models of such man-made systems, that is, models in which system status evolves over time, subject to unpredictable factors like weather, demand shocks, or mechanical failures. Some of the models considered in the book are purely descriptive in nature, aimed at estimating performance characteristics like long-run average inventory or expected discounted cost, given a fixed set of system characteristics. Other models are explicitly aimed at optimizing some measure of system performance, especially through the exercise of a dynamic control capability.

Brownian models of system evolution and dynamic control are increasingly popular in economics and engineering, and in allied business fields like finance and operations. The reason for that popularity is mathematical tractability: In one instance after another, researchers working with Brownian models have been able to derive explicit solutions and clear-cut insights that were unobtainable using conventional models. In the ways that really matter, then, Brownian models provide the simplest possible representations of dynamic, stochastic phenomena.