Summary

This thesis has developed a coherent framework for analysing image sequences based on the affine camera, and has demonstrated the practical feasibility of recovering 3D structure and motion in a bottom–up fashion, using “corner” features. New algorithms have been proposed to compute affine structure, and these have then been applied to the problems of clustering and view transfer. The theory of affine epipolar geometry has been derived and applied to outlier rejection and rigid motion estimation. Due consideration has been paid to error and noise models, with a χ2 test serving as a termination criterion for cluster growth and outlier detection, and confidence limits in the motion parameters facilitating Kalman filtering.

On a practical level, all the algorithms have been implemented and tested on a wide range of sequences. The use of n points and m frames has lead to enhanced noise immunity and has also simplified the algorithms in important ways, e.g. local coordinate frames are no longer needed to compute affine structure or rigid motion parameters. Finally, the use of 3D information without explicit depth has been illustrated in a working system (e.g. for transfer).

In summary, the affine camera has been shown to provide a solid foundation both for understanding structure and motion under parallel projection, and for devising reliable algorithms.

Future work

There are many interesting problems for future work to address. First, the CI space interpretation of the motion segmentation problem is that each independently moving object contributes a different 3D linear subspace.

Introduction

The first competence required of a motion analysis system is the accurate and robust measurement of image motion. This chapter addresses the problem of tracking independently–moving (and possibly non–rigid) objects in a long, monocular image sequence. “Corner features” are automatically identified in the images and tracked through successive frames, generating image trajectories. This system forms the low–level front–end of our architecture (cf. Figure 1.1), making reliable trajectory computation of the utmost importance, for these trajectories underpin all subsequent segmentation and motion estimation processes.

We build largely on the work of Wang and Brady [156, 157], and extend their successful corner–based stereo algorithm to the motion domain. Their key idea was to base correspondence on both similarity of local image structure and geometric proximity. There are, however, several ways in which motion correspondence is more complex than stereo correspondence [90]. For one thing, objects can change between temporal viewpoints in ways that they cannot between spatial viewpoints, e.g. their shape and reflectance can alter. For another, the epipolar constraint is no longer hard–wired by once–off calibration of a stereo–rig; motion induces variable epipolar geometry which has to be continuously updated (if the constraint is to be used). Furthermore, motion leads to arbitrarily long image sequences (instead of frame–pairs), which requires additional tracking machinery. The benefits are that temporal integration facilitates noise resistance, resolves ambiguities over time, and speeds up matching (via prediction).

Our framework has two parts: the matcher performs two–frame correspondence while the tracker maintains the multi-frame trajectories. Each corner is treated as an independent feature at this level (i.e. assigned an individual tracker as in [26]), and is tracked purely within the image plane. Section 2.2 justifies this feature–based approach and establishes the utility of corners as correspondence tokens.

Introduction

This chapter tackles the motion estimation problem, using affine epipolar geometry as the tool. Given m distinct views of n points located on a rigid object, the task is to compute its 3D motion without any prior 3D knowledge. There are several reasons why many existing point–based motion algorithms are of limited practical use: the inevitable presence of noise is often ignored; unreasonable demands are often made on prior processing (e.g. a suitable perceptual frame must first be selected, the features must appear in every frame, etc.); algorithms often only work in special cases (e.g. rotation about a fixed axis); and some algorithms require batch processing, rather than more natural sequential processing.

Although the epipolar constraint has been widely used in perspective and projective motion applications [43, 57, 87] (e.g. to aid correspondence, recover the translation direction and compute rigid motion parameters), it has seldom been used under affine viewing conditions (though see [66, 79]). This chapter therefore makes the following contributions:

• Affine epipolar geometry is related to the rigid motion parameters, and Koenderink and van Doom's novel motion representation is formalised [79]. The scale, cyclotorsion angle and projected axis of rotation are then computed directly from the epipolar geometry (i.e. using two views). The only camera calibration parameter needed here is aspect ratio. A suitable error model is also derived.

• Images are processed in successive pairs of frames, facilitating extension to the m-view case in a sequential (rather than batch) processing mode.

• […]

Introduction

Once the corner tracker has generated a set of image trajectories, the next task is to group these points into putative objects. The practice of classifying objects into sensible groupings is termed “clustering”, and is fundamental to many scientific disciplines. This chapter presents a novel clustering technique that groups points together on the basis of their affine structure and motion. The system copes with sparse, noisy and partially incorrect input data, and with scenes containing multiple, independently moving objects undergoing general 3D motion. The key contributions are as follows:

• A graph theory framework is employed (in the spirit of [119]) using maximum affinity spanning trees (MAST's), and the clusters are computed by a local, parallel network, with each unit performing simple operations. The use of such networks has long been championed by Ullman, who has used them to fill in subjective contours [150], compute apparent motion [151] and detect salient curves [132].

• Clustering occurs over multiple frames, unlike the more familiar two–frame formulations (e.g. [3, 80, 134]).

• A graduated motion analysis scheme extends the much–used simplistic image motion models, e.g. grouping on the basis of parallel and equal image velocity vectors (as in [80, 134]) is only valid for a fronto–parallel plane translating parallel to the image. The layered complexity of our models utilises full 3D information where available, but doesn't use a more complex model than is required.

• The termination criteria (to control cluster growth) are based on sound statistical noise models, in contrast to many heuristic measures and thresholds (e.g. [119, 134]).

Motivation

Sight is the sense that provides the highest information content – in engineering terms, the highest bandwidth – to the human brain. A computer vision system, essentially a “TV camera connected to a computer”, aims to perform on a machine the tasks which our own visual system seems to perform so effortlessly. Since the world is constantly in motion, it comes as no surprise that time–varying imagery reveals valuable information about the environment. Indeed, some information is easier to obtain from a image sequence than from a single image [62]. Thus, as noted by Murray and Buxton, “understanding motion is a principal requirement for a machine or organism to interact meaningfully with its environment” [100] (page 1). For this reason, the analysis of image sequences to extract 3D motion and structure has been at the heart of computer vision research for the past decade [172].

The problem involves two key difficulties. First, the useful content of an image sequence is intricately coded and implicit in an enormous volume of sensory data. Making this information explicit entails significant data reduction, to decode the spatio–temporal correlations of the intensity values and eliminate redundancy. Second, information is lost in projecting the three spatial dimensions of the world onto the two dimensions of the image. Assumptions about the camera model and imaging geometry are therefore required.

This thesis develops new algorithms to interpret visual motion using a single camera, and demonstrates the practical feasibility of recovering scene structure and motion in a data-driven (or “bottom-up”) fashion. Section 1.2 outlines the basic themes and describes the system architecture.

THE SYNTAX OF LINEAR LOGIC

The connectives of linear logic

Linear logic is not an alternative logic ; it should rather be seen as an extension of usual logic. Since there is no hope to modify the extant classical or intuitionistic connectives, linear logic introduces new connectives.

Exponentials : actions vs situations

Classical and intuitionistic logics deal with stable truths:

if A and A ⇒ B, then B, but A still holds.

This is perfect in mathematics, but wrong in real life, since real implication is causal. A causal implication cannot be iterated since the conditions are modified after its use ; this process of modification of the premises (conditions) is known in physics as reaction. For instance, if A is to spend $1 on a pack of cigarettes and B is to get them, you lose $1 in this process, and you cannot do it a second time. The reaction here was that $1 went out of your pocket. The first objection to that view is that there are in mathematics, in real life, cases where reaction does not exist or can be neglected : think of a lemma which is forever true, or of a Mr. Soros, who has almost an infinite amount of dollars. Such cases are situations in the sense of stable truths. Our logical refinements should not prevent us to cope with situations, and there will be a specific kind of connectives (exponentials, “!” and “?”) which shall express the iterability of an action, i.e. the absence of any reaction ; typically!A means to spend as many dollars as one needs.

Abstract

The paper studies the properties of the subnets of proof-nets. Very simple proofs are obtained of known results on proof-nets for MLL-, Multiplicative Linear Logic without propositional constants.

Preface

The theory of proof-nets for MLL-, multiplicative linear logic without the propositional constants 1 and ⊥, has been extensively studied since Girard's fundamental paper [5]. The improved presentation of the subject given by Danos and Regnier [3] for propositional MLL- and by Girard [7] for the first-order case has become canonical: the notions are defined of an arbitrary proof-structure and of a ‘contex-forgetting’ map (·)- from sequent derivations to proof-structures which preserves cut-elimination; correctness conditions are given that characterize proof-nets, the proof-structures R such that R = (D)-, for some sequent calculus derivation D. Although Girard's original correctness condition is of an exponential computational complexity over the size of the proof-structure, other correctness conditions are known of quadratic computational complexity.

A further simplification of the canonical theory of proof-nets has been obtained by a more general classification of the subnet of a proof-net. Given a proof-net R and a formula A in R, consider the set of subnets that have A among their conclusions, in particular the largest and the smallest subnet in this set, called the empire and the kingdom of A, respectively. One must give a construction proving that such a set is not empty: in Girard's fundamental paper a construction of the empires is given which is linear in the size of the proof-net.

Introduction

The aim of this paper is to give a purely graph-theoretical definition of noncommutative proof nets, i.e. graphs coming from proofs in MNLL (multiplicative noncommutative linear logic, the (⊗, ℘)-fragment of the one-sided sequent calculus for classical noncommutative linear logic, introduced in [Abr91]). Analogously, one of the aims of [Gir87] was to give a purely graph-theoretical definition of proof nets, i.e. graphs coming from the proofs in MLL (multiplicative linear logic, the (⊗, ℘)-fragment of the one-sided sequent calculus for classical linear logic - better, for classical commutative linear logic). - The relevance of the purely graph-theoretical definition of proof nets for the development of commutative linear logic is well-know; thus we hope the results of this paper will be useful for a similar development of noncommutative linear logic.

The language for MNLL is an extension of the language for MLL, obtained simply adding, as atomic formulas, propositional letters with an arbitrary finite number of negations written after the propositional letter (linear post-negation) or before the propositional letter (linear retronegation). Every formula A of MNLL may be translated into a formula Tv(A) of MLL (simply by replacing each propositional letter with an even number of negations by the propositional letter without negations, and each propositional letter with an odd number of negations by the propositional letter with only one negation after the propositional letter).

Abstract

The syntactic calculus, a fragment of noncommutative linear logic, was introduced in 1958 because of its hoped for linguistic application. Working with a Gentzen style presentation, one was led to the problem of finding all derivations f : A1 … An → B in the free syntactic calculus generated by a context free grammar g (with arrows reversed) and to the problem of determining all equations f = g between two such derivations. The first problem was solved by showing that f is equal to a derivation in normal form, whose construction involves no identity arrows and no cuts (except those in g) and the second problem is solved by reducing both f and g to normal form.

The original motivation for the syntactic calculus came from multilinear algebra and a categorical semantics was given by the calculus of bimodules. Bimodules RFS may be viewed as additive functors R → Mod S, where R and S are rings (of several objects). It is now clear that Lawvere's generalized bimodules will also provide a semantics for what may be called labeled bilinear logic.

Introduction.

I was asked to talk about one precursor of linear logic that I happened to be involved in, even though it anticipated only a small fraction of what goes on in the linear logic enterprise. I would now call this system “bilinear logic”, meaning “non-commutative linear logic” or “logic without Gentzen's three structural rules”.

Abstract

We present stochastic interactive semantics for propositional linear logic without modalities. The framework is based on interactive protocols considered in computational complexity theory, in which a prover with unlimited power interacts with a verifier that can only toss fair coins or perform simple tasks when presented with the given formula or with subsequent messages from the prover. The additive conjunction &, is described as random choice, which reflects the intuitive idea that the verifier can perform only “random spot checks”. This stochastic interactive semantic framework is shown to be sound and complete. Furthermore, the prover's winning strategies are basically proofs of the given formula. In this framework the multiplicative and additive connectives of linear logic are described by means of probabilistic operators, giving a new basis for intuitive reasoning about linear logic and a potential new tool in automated deduction.

Introduction

Linear logic arose from the semantic study of the structure of proofs in intuitionistic logic. Girard presented the coherence space and phase space semantics of linear logic in his original work on linear logic [Gir87]. While these models provide mathematical tools for the study of several aspects of linear logic, they do not oifer a simple intuitive way of reasoning about linear logic. More recently, Blass [Bla92], Abramsky and Jagadeesan [AJ94], Lamarche, and Hyland and Ong have developed semantics of linear logic by means of games and interaction. These new approaches have already proven fruitful in providing an evocative semantic paradigm for linear logic and have found a striking application to programming language theory in the work of Abramsky, Jagadeesan, and Malacaria [AJM93] and in the work of Hyland and Ong [HO93].

Abstract

The problems of inheritance reasoning in taxonomical networks are crucial in object-oriented languages and in artificial intelligence. A taxonomical network is a graph that enables knowledge to be represented. This paper focuses on the means linear logic offers to represent these networks and is a follow-up to the note on exceptions by Girard [Gir92a]. It is first proved that all compatible nodes of a taxonomical network can be deduced in the taxonomical linear theory associated to the network. Moreover, this theory can be integrated in the Unified Logic LU [Gir92b] and so taxonomical and classical reasoning can be combined.

Introduction

The problems of inheritance reasoning in taxonomical networks are crucial in object-oriented languages and in artificial intelligence. A taxonomical network is a graph that enables knowledge to be represented. The nodes represent concepts or properties of a set of individuals whereas the edges represent relations between concepts. The network can be viewed as a hierarchy of concepts according to levels of generality. A more specific concept is said to inherit informations from its subsumers. There are two kinds of edges: default and exception. A default edge between A and B means that A is generally a B or A has generally the property B. An exception edge between A and B means that there is an exception between A and B, namely A is not a B or A has not the property B. Nonmonotonic systems were developed in the last decade in order to attempt to represent defaults and exceptions in a logical way: the set of inferred grounded facts is the set of properties inherited by concepts.