Programs have to handle highly varied objects organized into categories, such as numbers, text, images, formulas, as well as other programs. These objects are for the most part foreign to the world of programming and involve quite diverse formal systems.

For each such category of objects, the programmer must define a representation in the objects of the programming language that he or she is using. This representation must satisfy criteria such as efficiency (by exploiting what the programmer has learned about algorithms) and clarity; that is, the way the data is structured should reflect the structure of the objects being represented. Representation of objects from the exterior world should not be some obscure encoding; rather, the conceptual, external structure should remain apparent.

In a typed language like Caml, the nature of an object lies in its type. It is thus natural to ask that the external structure of the objects being handled should be reflected in the types of their internal representation. For that purpose, the programmer must have the means to define a great variety of types.

In this chapter, we present the two principal type constructions that Caml offers: records and sums with constructors. You might think of a record as the product of named components, and a sum as the union of named components.

Record or Named Cartesian Products

Let's assume that we want to write programs working on complex numbers. We can imagine representing complex numbers by pairs (r,i) of type (float * float) giving the real and imaginary parts of such numbers.

This chapter is devoted to functionally programming geometric drawings. We use types and graphic functions from the library MLgraph, and we briefly describe them before we use them.

The graphic approach of MLgraph is the same as PostScript, a language used particularly for programming laser printers. This approach is oriented toward programming drawings meant to illustrate documents, rather than toward the display of drawings on screen, though there is software to display PostScript drawings as well. Such software provides a reasonable approximation of the drawings, but the quality is limited by the actual screen resolution.

PostScript is an imperative language, indeed, a low-level imperative language, which was not meant to be used directly but rather to be generated by software to produce drawings or to type-set text. The approach we present here uses the graphic model of PostScript, but replaces that language by Caml. This enables us to provide much higher level construction functions for drawing.

The graphic model of PostScript freely uses an infinite Cartesian plane with no limit on the coordinates. The basic objects are initially sketched on this plane in absolute coordinates, but a user can then freely apply transformations to modify their size and position. Such transformations rapidly get out of the constraints about positioning and let us construct complicated drawings by functionally combining simpler ones.

At first glance, producing drawings by programming seems less pleasant than using software to draw interactively, especially if we want to produce a particular design rapidly.

Here we begin the second part of this book; it is dedicated to writing various applications in Caml. Its chapters are largely independent of one another, so you can read them separately, choosing them according to your own interests or according to the needs of a course.

Chapter 5 covers symbolic computations over formal terms. In particular, it defines algorithms for pattern matching and unification used in many applications of symbolic computations, such as automatic proof, logic programming, or type synthesis.

Chapter 6 shows how to use balanced trees to represent large bodies of information. It uses the ideas of binary search trees and AVL trees as introduced in every course about algorithms.

Chapter 7 goes deeply into the methods of exploring graphs. It uses techniques of set representation introduced in Chapter 6 and presents applications from the domain of games such as the red donkey or solitaire.

Chapter 8 takes up the writing of lexical and syntactic analyzers. It uses the idea of a Caml stream.

Chapter 9 shows how to program drawing and designs. It first succinctly describes and then exploits the MLgraph library to draw trees and to tile planes and surfaces.

Chapter 10 deals with exact arithmetic, taking into account integers and rational numbers of arbitrary size. It does not attempt to describe a complete and efficient implementation of a library for large numbers, but rather, it tries to convince you that such a project is feasible.

Most programs, especially those that expect a character string as input, begin by recognizing such strings in order to determine whether or not one happens to represent valid input. Next—in fact, at the same time—such a program transforms the string into internal data that is easier to handle. The preceding chapters assumed the existence of such features and functions. This chapter indicates how to program them.

A command to an operating system (such as Is on Unix, etc.) is a very simple example of a program that takes character strings as input and then decides to accept or reject them. A compiler is a more complicated program to take character strings as input; it accepts the concrete syntax of the program to compile. A compiler decides to accept or reject the program to compile; if it accepts, it transforms the program into more easily managed data, such as abstract syntax trees.

In each of these cases, we say that the command shell or the compiler accepts phrases belonging to a certain language; in the case of the command shell, we mean the options and arguments valid for a given command; for the compiler, the programming language to compile. In both cases, they reject any other phrases. In this chapter, we will call a language the set (possibly infinite) of acceptable character strings, and the chief problem that we take up is that of recognizing the phrases belonging to a given language.

This chapter defines a group of functions to manage large-scale data sets. These functions use trees as data structures because they are so well adapted to representing data sets that evolve dynamically, that is, data sets where the size can grow or shrink during computations.

The specific data structure we use is a binary tree. The algorithms we propose make it possible to keep these binary trees balanced, and this property ensures the efficiency of operations we want to carry out.

The main operations for managing a data set are these:

• searching for an element in the data set;

• adding an element to the data set;

• removing an element from the data set.

To search naively for an element in a data set, we can simply sweep sequentially through all the elements of the set, but that takes time proportional to the size of the data set. Its complexity is thus O(n). This technique is the only one we can use when we are managing data sets generically because we do not know any particular properties of the set.

In order to search more efficiently, we have to make hypotheses about the structure of the data set under consideration. For example, if we can assume that the data are ordered, then we can use a binary search as we did in Section 4.4.1 to represent sets by sorted arrays.

In this chapter, we describe a technique to compile a functional language like Caml into machine code. This compilation technique is strongly connected to the evaluation technique developed in the preceding chapter. In particular, the idea of an environment again plays a central role here, and we keep the idea of closure to represent functional values. However, you will see that these two ideas correspond to slightly different objects in a compiler.

To keep the compiler at a sufficiently conceptual level of description, we have opted to use machine code made up of instructions specially adapted to our compilation scheme. These instructions produce operations more complicated than the “real” instructions of an assembly language. Even so, it is clear that these instructions can be expanded into a list of machine instructions producing the same effect. The compilation scheme we present thus leads directly to a real compiler.

To execute the code produced by our compiler, we use a stack to store the intermediate values needed in computations; we also store the addresses of subprograms there, a conventional and time-honored practice. What is special, though, about compiling functional languages are the instructions to allocate memory needed to build structured values, environments, and closures. To explain these instructions and their role in the compilation of functional languages, we must first explain how we use computer memory to represent complex objects.

In Chapter 3, we presented a model for evaluation of expressions; it was based on the idea of rewrites. This model defined a semantics to which every implementation had to refer. However, an implementation that actually used textual rewrites would be highly inefficient. To achieve reasonable efficiency, we must replace textual rewrites by some other mechanism that simulates it. One possibility is to introduce the idea of an environment.

In this chapter, we present evaluation based on environments. We will write an evaluation function that takes an environment and a Caml expression as its arguments and returns the value of this expression as its result. The ideas for implementing this evaluation function will be refined in Chapter 12 to produce a compiler.

Section 11.1 describes evaluation of expressions with the help of environments. As in Chapter 3, it uses inference rules corresponding to each construction in the language to do so. These inference rules define evaluation by value, the evaluation strategy adopted by Caml. Section 11.2 then shows how to translate these rules into an evaluator written in Caml. Finally, Section 11.3 shows how to modify that evaluator to take into account delayed evaluation rather than evaluation by value.

Evaluation with the Help of Environments

An environment may be regarded as a look-up table in which certain variables are bound to values.

In this chapter, we introduce continuations into the setting of a functional language using eager evaluation. We begin by defining a continuation semantics for the language of the previous chapter. Next, we extend this language by introducing continuations as values. Finally, we derive a first-order semantics that reveals how one might implement the extended language interpretively.

Continuation Semantics

When the evaluation of an expression gives an error or fails to terminate, this “computational effect” is the final result of the entire program. In the direct denotational semantics of Section 11.6, this behavior is expressed by passing err, tyerr, or ⊥ through a chain of *-extended functions that preserve these results.

An alternative approach is to use a continuation semantics, similar to the one discussed in Section 5.7, in which the semantic function, when applied to an expression e, an environment η, and a new argument K called a continuation, produces the final result of the entire program in which e is embedded. If the evaluation of e gives a value, then the final result is obtained by applying K to this value; but if the evaluation gives an error stop or fails to terminate (or executes a throw operation, which we will discuss in the next section), this result is produced directly without using the continuation. This makes it immediately evident that the “rest of the computation”, which is represented by the continuation, has no effect on the final result. (We will use the metavariable K, with occasional primes and subscripts, for continuations.)

Here we describe a variety of general mathematical concepts and notations that are used in this book. (Concepts that are more advanced, or are particular to programming-language theory, are explained in the main text when they are first used.) Even though most of these concepts may be familiar, readers are advised to scan this material, especially since some of the notations are novel.

In fact, modern mathematics gives the writer a surprising degree of freedom in defining its fundamental concepts, as long as the definitions are consistent and lead to entities with appropriate properties. In this book, I have made the following choices:

• Pairs are taken to be a primitive notion rather than being defined in terms of sets.

• Relations are defined as sets of pairs, functions as a special kind of relation, sequences as a special kind of function, and pairs as a special kind of sequence. Since a pair, however, cannot be a set containing itself, we break the inconsistent circle by distinguishing between the pairs 〈x,y) that are sequences and the primitive pairs [x,y] that are members of relations.

• We define relations (including functions) to be sets of primitive argument-result pairs, rather than calling such a set the “graph” of the relation and defining the relation itself to consist of a domain, a codomain, and a graph. Thus, for example, the function on the integers that increases its argument by one is both a function from integers to integers and a function from integers to reals, and also a partial function from reals to reals. (This view of functions is taken by Gleason [1966, Sections 3–4.1 to 3–4.11]. It is a special case of the view that, in category theory, the morphism sets of a category need not be disjoint.)

• […]

Peter Landin remarked long ago that the goal of his research was “to tell beautiful stories about computation”. Since then many researchers have told many such stories. This book is a collection my favorites in the area of languages for programming and program specification.

In 1992, the Computer Science Department of Carnegie Mellon University replaced the preliminary examinations for its doctoral students by a set of required courses, including CS711, simply titled “Programming Languages”, which was intended to be a unified treatment of the basic principles of the subject. Previously, such material had been divided between the programming systems examination and the theory examination, and many important topics had fallen through the cracks between the syllabi. (For example, students were expected to know enough of the theory of program proving to understand Cook completeness, yet they never saw the proof of an actual program beyond a trivial one that computed a square root by iterating over successive natural numbers and squaring them.)

As the most vociferous exponent of such a course, I was put in charge of teaching it, and I soon discovered that there was no suitable textbook. Serious texts in areas such as semantics or verification invariably stressed a particular approach and neglected topics that did not fit well with their point of view. At the opposite extreme, surveys of programming languages usually emphasized superficial differences between languages while slighting more fundamental issues. In effect, what was available were profound novels and journalistic popularizations, but what was needed was a collection of short stories sharing some common characters. Thus I produced extensive class notes, which in a few years grew into this book.

In recent years, the dramatic drop in the cost of computing hardware has made it practical, in an ever-increasing variety of contexts, to execute different parts of a program simultaneously. Many authors use the terms “concurrent” or “parallel” indifferently to describe such computations. Following increasingly common usage, however, we will use concurrent to describe computations where the simultaneously executing processes can interact with one another, and we will reserve parallel to describe computations where the behavior of each process is unaffected by the behavior of the others.

Our present concern is concurrency. In this chapter, we consider an approach where processes communicate through shared variables. This approach mirrors the situation where a common memory, typically containing a shared database, is accessed by several physical processors – or perhaps by a single processor that is time-shared among several logical processes. (An alternative approach to concurrency, where processes communicate by passing messages, will be described in Chapter 9.)

Except in trivial cases, the interaction between concurrent processes will depend on the relative speed of the physical processors or on the decisions by a scheduler of when to switch a physical processor from one logical process to another. Thus concurrent programs are usually nondeterminate. This nondeterminacy makes concurrent programs especially difficult to reason about, since one must consider all possible orders of execution. Moreover, the efficacy of debugging is undermined by the irreproducibility of executions and by the fact that the actual occurrences of obscure and possibly erroneous orders of execution may be very infrequent.