Concrete data consists of constructions that can be inspected, taken apart, or joined to form larger constructions. Lists are an example of concrete data. We can test whether or not a list is empty, and divide a non-empty list into its head and tail. New elements can be joined to a list. This chapter introduces several other forms of concrete data, including trees and logical propositions.

The ml datatype declaration defines a new type along with its constructors. In an expression, constructors create values of a datatype; in patterns, constructions describe how to take such values apart. A datatype can represent a class consisting of distinct subclasses — like Pascal's variant records, but without their complications and insecurities. A recursive datatype typically represents a tree. Functions on datatypes are declared by pattern-matching.

The special datatype exn is the type of exceptions, which stand for error conditions. Errors can be signalled and trapped. An exception handler tests for particular errors by pattern-matching.

Chapter outline

This chapter describes datatypes, pattern-matching, exception handling and trees. It contains the following sections:

The datatype declaration. Datatypes, constructors and pattern-matching are illustrated through examples. To represent the King and his subjects, a single type person comprises four classes of individual and associates appropriate information with each.

Exceptions. These represent a class of error values. Exceptions can be declared for each possible error.

With each reprinting of this book, a dozen minor errors have silently disappeared. But a reprinting is no occasion for making improvements, however valuable, that would affect the page numbering: we should then have several slightly different, incompatible editions. An accumulation of major changes (and the Editor's urgings) have prompted this second edition.

As luck would have it, changes to ml have come about at the same time. ml has a new standard library and the language itself has been revised. It is worth stressing that the changes do not compromise ml's essential stability. Some obscure technical points have been simplified. Anomalies in the original definition have been corrected. Existing programs will run with few or no changes. The most visible changes are the new character type and a new set of top level library functions.

The new edition brings the book up to date and greatly improves the presentation. Modules are now introduced early — in Chapter 2 instead of Chapter 7 — and used throughout. This effects a change of emphasis, from data structures (say, binary search trees) to abstract types (say, dictionaries). A typical section introduces an abstract type and presents its ml signature. Then it explains the ideas underlying the implementation, and finally presents the code as an ml structure. Though reviewers have been kind to the first edition, many readers have requested such a restructuring.

Most programmers know how hard it is to make a program work. In the 1970s, it became apparent that programmers could no longer cope with software projects that were growing ever more complex. Systems were delayed and cancelled; costs escalated. In response to this software crisis, several new methodologies have arisen — each an attempt to master the complexity of large systems.

Structured programming seeks to organize programs into simple parts with simple interfaces. An abstract data type lets the programmer view a data structure, with its operations, as a mathematical object. The next chapter, on modules, will say more about these topics.

Functional programming and logic programming aim to express computations directly in mathematics. The complicated machine state is made invisible; the programmer has to understand only one expression at a time.

Program correctness proofs are introduced in this chapter. Like the other responses to the software crisis, formal methods aim to increase our understanding. The first lesson is that a program only ‘works’ if it is correct with respect to its specification. Our minds cannot cope with the billions of steps in an execution. If the program is expressed in a mathematical form, however, then each stage of the computation can be described by a formula. Programs can be verified — proved correct — or derived from a specification. Most of the early work on program verification focused on Pascal and similar languages; functional programs are easier to reason about because they involve no machine state.

In a public lecture, C. A. R. Hoare (1989a) described his algorithm for finding the ith smallest integer in a collection. This algorithm is subtle, but Hoare described it with admirable clarity as a game of solitaire. Each playing card carried an integer. Moving cards from pile to pile by simple rules, the required integer could quickly be found.

Then Hoare changed the rules of the game. Each card occupied a fixed position, and could only be moved if exchanged with another card. This described the algorithm in terms of arrays. Arrays have great efficiency, but they also have a cost. They probably defeated much of the audience, as they defeat experienced programmers. Mills and Linger (1986) claim that programmers become more productive when arrays are restricted to stacks, queues, etc., without subscripting.

Functional programmers often process collections of items using lists. Like Hoare's stacks of cards, lists allow items to be dealt with one at a time, with great clarity. Lists are easy to understand mathematically, and turn out to be more efficient than commonly thought.

Chapter outline

This chapter describes how to program with lists in Standard ml. It presents several examples that would normally involve arrays, such as matrix operations and sorting.

The chapter contains the following sections:

Introduction to lists. The notion of list is introduced. Standard ml operates on lists using pattern-matching.

Introduction

In this chapter we study the worst case setting. We shall present results already known as well as showing some new results. As already mentioned in the Overview, precise information about what is known and what is new can be found in the Notes and Remarks.

Our major goal is to obtain tight complexity bounds for the approximate solution of linear continuous problems that are defined on infinite dimensional spaces. We first explain what is to be approximated and how an approximation is obtained. Thus we carefully introduce the fundamental concepts of solution operator, noisy information and algorithm. Special attention will be devoted to information, which is most important in our analysis. Information is, roughly speaking, what we know about the problem to be solved. A crucial assumption is that information is noisy, i.e., it is given not exactly, but with some error.

Since information is usually partial (i.e., many elements share the same information) and noisy, it is impossible to solve the problem exactly. We have to be satisfied with only approximate solutions. They are obtained by algorithms that use information as data. In the worst case setting, the error of an algorithm is given by its worst performance over all problem elements and possible information. A sharp lower bound on the error is given by a quantity called radius of information. We are obviously interested in algorithms with the minimal error.

In the process of doing scientific computations we always rely on some information. In practice, this information is typically noisy, i.e., contaminated by error. Sources of noise include

• previous computations,

• inexact measurements,

• transmission errors,

• arithmetic limitations,

• an adversary's lies.

Problems with noisy information have always attracted considerable attention from researchers in many different scientific fields, e.g., statisticians, engineers, control theorists, economists, applied mathematicians. There is also a vast literature, especially in statistics, where noisy information is analyzed from different perspectives.

In this monograph, noisy information is studied in the context of the computational complexity of solving mathematical problems.

Computational complexity focuses on the intrinsic difficulty of problems as measured by the minimal amount of time, memory, or elementary operations necessary to solve them. Information-based complexity (IBC) is a branch of computational complexity that deals with problems for which the available information is

• partial,

• noisy,

• priced.

Information being partial means that the problem is not uniquely determined by the given information. Information is noisy since it may be contaminated by error. Information is priced since we must pay for getting it. These assumptions distinguish IBC from combinatorial complexity, where information is complete, exact, and free.

Since information about the problem is partial and noisy, only approximate solutions are possible. Approximations are obtained by algorithms that use this information.

Introduction

This chapter deals with the average case setting. In this setting, we are interested in the average error and cost of algorithms. The structure of this chapter is similar to that of the previous chapter. That is, we first deal with optimal algorithms, then we analyze the optimal information, and finally, we present some complexity results.

To study the average error and/or cost, we have to replace the deterministic assumptions of the worst case setting by stochastic assumptions. That is, we assume some probability distribution µ on the space F of problem elements as well as some distribution of the information noise. The latter means that information is corrupted by random noise. Basically, we consider Gaussian distributions (measures) which seem to be most natural and are most often used in modeling.

In Section 3.2, we give a general formulation of the average case setting. We also introduce the concept of the (average) radius of information which, as in the worst case, provides a sharp lower bound on the (average) error of algorithms.

Then we pass to linear problems with Gaussian measures. These are problems where the solution operator is linear, µ is a Gaussian measure, and information is linear with Gaussian noise. In Section 3.3, we recall the definition of a Gaussian measure on a Banach space, listing some important properties. In Sections 3.4 to 3.6 we study optimal algorithms.

In the modern world, the importance of information can hardly be overestimated. Information also plays a prominent role in scientific computations. A branch of computational complexity which deals with problems for which information is partial, noisy and priced is called informationbased complexity.

In a number of information-based complexity books, the emphasis was on partial and exact information. In the present book, the emphasis is on noisy information. We consider deterministic and random noise. The analysis of noisy information leads to a variety of interesting new algorithms and complexity results.

The book presents a theory of computational complexity of continuous problems with noisy information. A number of applications is also given. It is based on results of many researchers in this area (including the results of the author) as well as new results not published elsewhere.

This work would not have been completed if I had not received support from many people. My special thanks go to H. Woźniakowski who encouraged me to write such a book and was always ready to offer his help. I appreciate the considerable help of J.F. Traub. I would also like to thank M. Kon, A. Werschulz, E. Novak, K. Ritter and other colleagues for their valuable comments on various portions of the manuscript.

I wish to express my thanks to the Institute of Applied Mathematics and Mechanics at the University of Warsaw, where the book was almost entirely written.

Introduction

In Chapters 2 to 5, we fixed the set of problem elements and were interested in rinding single information and algorithm which minimize an error or cost of approximation. Depending on the deterministic or stochastic assumptions on the problem elements and information noise, we studied the four different settings: worst, average, worst-average, and average-worst case settings.

In this chapter, we study the asymptotic setting in which a problem element f is fixed and we wish to analyze asymptotic behavior of algorithms. The aim is to construct a sequence of information and algorithms such that the error of successive approximations vanishes as fast as possible, as the number of observations increases to infinity.

The asymptotic setting is often studied in computational practice. We mention only the Romberg algorithm for computing integrals, and finite element methods (FEM) for solving partial differential equations with the meshsize tending to zero. When dealing with these and other numerical algorithms, we are interested in how fast they converge to the solution.

One might hope that it will be possible to construct a sequence φn(yn) of approximations such that for the element f the error ∥S(f) − φn(yn)∥ vanishes much faster than the error over the whole set of problem elements (or, equivalently, faster than the corresponding radius of information). It turns out, however, that in many cases any attempts to construct such algorithms would fail. We show this by establishing relations between the asymptotic and other settings.