Chapter Overview

Every engineered communication system, ranging from satellite communications to hard disk drives to Ethernet must operate under noisy conditions. The key to reliable communication and storage is to add an appropriate amount of redundancy to make the system reliable. The field of channel coding is concerned with constructing channel codes and their decoding algorithms: controlled redundancy is introduced into a message prior to its transmission over a noisy channel (the encoding step), and this redundancy is removed from the received noisy string to unveil the intended message (the decoding step). The encoded message is referred to as the codeword. The collection of all codewords is a channel code. Assuming all the messages have the same length, and all the codewords have the same length, the ratio of message length to codeword length is the code rate. To make coding systems implementable in practice, channel codes must provide the best possible protection to noise while their decoding algorithms must be of acceptable complexity.

There is a clear tension with this dual goal: if a channel code protects a fairly long encoded message with relatively few but carefully derived redundancy bits (necessary for high performance), the optimal, maximum likelihood decoding algorithm has exponential complexity.

Introduction

Boolean Satisfiability (SAT) is an NP-complete decision problem [14]. SAT was the first problem to be shown NP-complete. There are no known polynomial time algorithms for SAT. Moreover, it is believed that any algorithm that solves SAT is exponential in the number of variables, in the worst-case.

Although SAT is in theory an NP-complete problem, in practice it can be seen as a success story of Computer Science. There have been remarkable improvements since the mid 90s, namely clause learning and unique implication points (UIPs) [43], search restarts [15, 26], lazy data structures [48], adaptive branching heuristics [48], clause minimization [59] and preprocessing [18].

Introduction

Computational problems from many different areas involve finding values for variables that satisfy certain specified restrictions and optimise certain specified criteria.

In this chapter, we will show that it is useful to abstract the general form of such problems to obtain a single generic framework. Bringing all such problems into a common framework draws attention to common aspects that they all share, and allows very general analytical approaches to be developed. We will survey some of these approaches, and the results that have been obtained by using them.

The generic framework we shall use is the valued constraint satisfaction problem (VCSP), defined formally in Section 4.3. We will show that many combinatorial optimisation problems can be conveniently expressed in this framework, and we will focus on finding restrictions to the general problem which are sufficient to ensure tractability.

An important and well-studied special case of the VCSP is the constraint satisfaction problem (CSP), which deals with combinatorial search problems which have no optimisation criteria. We give a brief introduction to the CSP in Section 4.2, before defining the more general VCSP framework in Section 4.3. Section 4.4 then presents a number of examples of problems that can be seen as special cases of the VCSP.

The remainder of the chapter discusses what happens to the complexity of the valued constraint satisfaction problem when we restrict it in various ways.

Introduction

Our starting point is research and experiences in the context of the state-of-the art SMT solver Z3 [13], developed by the authors at Microsoft Research. We first cover a selection of the main challenges and techniques for making SMT solving practical, integrating algorithms for tractable subproblems, and pragmatics and heuristics used in practice. We then take a proof-theoretical perspective on the power and scope of the engines used by SMT solvers. Most modern SMT solvers are built around a tight integration with efficient SAT solving. The framework is commonly referred to as DPLL(T), where T refers to a theory or a combination of theories. The theoretical result we present compares DPLL(T) with unrestricted resolution. A straightforward adaption of DPLL(T) provides a weaker proof system than unrestricted resolution, and we investigate an extension we call Conflict Directed Theory Resolution as a candidate method for bridging this gap. Our results apply to the case where T is the theory of equality.

Labeling Problems in Computer Vision

Many problems in computer vision and scene understanding can be formulated in terms of finding the most probable values of certain hidden or unobserved variables. These variables encode some property of the scene and can be continuous or discrete. These problems are commonly referred to as labelling problems as they involve assigning a label to the hidden variables. Labelling problems occur in many forms, from lattice based problems of dense stereo and image segmentation discussed in [6, 40] to the use of pictorial structures for object recognition as done by [10]. Some examples of problems which can be formulated in this manner are shown in Figure 10.1.

Introduction

Many areas of computer science have shown a great interest in tractable and canonical representations of propositional knowledge bases (aka, Boolean functions). The ordered binary decision diagram (OBDD) is one such representation that received much attention and proved quite influential in a variety of areas [13]. Within AI, the study of tractable representations has also had a long tradition (e.g., [61, 30, 31, 49, 62, 14, 28, 19, 13, 52, 66, 50]). This area of research, which is also known as knowledge compilation, has become more systematic since [28], which showed that many known and useful representations are subsets of negation normal form (NNF) and correspond to imposing specific properties on NNF. The most fundamental of these properties turned out to be decomposability and determinism, giving rise to the corresponding language of DNNF and its subset, d-DNNF. This chapter is dedicated to DNNF and its subsets, which also include the influential language of OBDDs, and the more recently introduced sentential decision diagrams (SDDs).

Introduction

Preprocessing (data reduction or kernelization) as a strategy for coping with hard problems is used in many situations. The history of this approach can be traced back to the 1950s [34], where truth functions were simplified using reduction rules. A natural question arises: how can we measure the quality of preprocessing rules proposed for a specific problem? For a long time the mathematical analysis of polynomial time preprocessing algorithms was neglected. The basic reason for this oversight was the following impossibility result: if, starting with an instance I of an NP-hard problem, we could compute in polynomial time an instance I′ equivalent to I and with |I′| < |I|, then it would follow that P=NP, thereby contradicting classical complexity assumptions.

In mathematics and computer science, optimization is the process of finding the best solution from a set of alternatives that satisfy some constraints. Many applications in allied fields of computer science like machine learning, computer vision, bioinformatics, involve the solution of an optimization problem. For instance, optimization is used to schedule trains and airplanes, allocate the advertisements we see on television or in connection with internet search results, ind the optimal placement of sensors to detect and neutralize security threats, or even to make decisions on what is the best way to perform medical surgery on a patient.

Optimization problems are generally hard to solve – their solution may involve exhaustively searching over a set of solutions whose size could increase exponentially with the number of variables whose values we may want to infer. That said, in practice, many of these problems can often be solved with remarkable efficiency. This is usually done by dedicated techniques, developed in each and every application domain, that exploit the “properties” of the problems encountered in practice.

Over the last few decades, researchers working in a number of different disciplines have tried to solve optimization problems that are encountered in their respective ields by exploiting some structure or properties inherent in the problems. In some cases, they have been able to isolate classes of optimization problems that can be solved optimally in time polynomial in the number of variables, while in other cases, they have been able to develop efficient algorithms that can produce solutions that, although not optimal, are good enough.