We consider the problem of fitting a Gaussian autoregressive model to a time series, subject to conditional independence constraints. This is an extension of the classical covariance selection problem to time series. The conditional independence constraints impose a sparsity pattern on the inverse of the spectral density matrix, and result in nonconvex quadratic equality constraints in the maximum likelihood formulation of the model estimation problem. We present a semidefinite relaxation, and prove that the relaxation is exact when the sample covariance matrix is block-Toeplitz. We also give experimental results suggesting that the relaxation is often exact when the sample covariance matrix is not block-Toeplitz. In combination with model selection criteria the estimation method can be used for topology selection. Experiments with randomly generated and several real data sets are also included.

Introduction

Graphical models give a graph representation of relations between random variables. The simplest example is a Gaussian graphical model, in which an undirected graph with n nodes is used to describe conditional independence relations between the components of an n-dimensional random variable x ~ N(0, ∑). The absence of an edge between two nodes of the graph indicates that the corresponding components of x are independent, conditional on the other components. Other common examples of graphical models include contingency tables, which describe conditional independence relations in multinomial distributions, and Bayesian networks, which use directed acyclic graphs to represent causal or temporal relations.

This chapter concerns the use of convex optimization in real-time embedded systems, in areas such as signal processing, automatic control, real-time estimation, real-time resource allocation and decision making, and fast automated trading. By “embedded” we mean that the optimization algorithm is part of a larger, fully automated system, that executes automatically with newly arriving data or changing conditions, and without any human intervention or action. By “real-time” we mean that the optimization algorithm executes much faster than a typical or generic method with a human in the loop, in times measured in milliseconds or microseconds for small and medium size problems, and (a few) seconds for larger problems. In real-time embedded convex optimization the same optimization problem is solved many times, with different data, often with a hard real-time deadline. In this chapter we propose an automatic code generation system for real-time embedded convex optimization. Such a system scans a description of the problem family, and performs much of the analysis and optimization of the algorithm, such as choosing variable orderings used with sparse factorizations and determining storage structures, at code generation time. Compiling the generated source code yields an extremely efficient custom solver for the problem family. We describe a preliminary implementation, built on the Python-based modeling framework CVXMOD, and give some timing results for several examples.

Introduction

Advisory optimization

Mathematical optimization is traditionally thought of as an aid to human decision making.

Non-cooperative game theory is a branch of game theory for the resolution of conflicts among players (or economic agents), each behaving selfishly to optimize their own well-being subject to resource limitations and other constraints that may depend on the rivals' actions. While many telecommunication problems have traditionally been approached by using optimization, game models are being increasingly used; they seem to provide meaningful models for many applications where the interaction among several agents is by no means negligible, for example, the choice of power allocations, routing strategies, and prices. Furthermore, the deregulation of telecommunication markets and the explosive growth of the Internet pose many new problems that can be effectively tackled with game-theoretic tools. In this chapter, we present a comprehensive treatment of Nash equilibria based on the variational inequality and complementarity approach, covering the topics of existence of equilibria using degree theory, global uniqueness of an equilibrium using the P-property, local-sensitivity analysis using degree theory, iterative algorithms using fixed-point iterations, and a descent approach for computing variational equilibria based on the regularized Nikaido–Isoda function. We illustrate the existence theory using a communication game with QoS constraints. The results can be used for the further study of conflict resolution of selfish agents in telecommunication.

Introduction

The literature on non-cooperative games is vast. Rather than reviewing this extensive literature, we refer the readers to the recent survey [20], which we will use as the starting point of this chapter.

This chapter presents, in a self-contained manner, recent advances in the design and analysis of gradient-based schemes for specially structured, smooth and nonsmooth minimization problems. We focus on the mathematical elements and ideas for building fast gradient-based methods and derive their complexity bounds. Throughout the chapter, the resulting schemes and results are illustrated and applied on a variety of problems arising in several specific key applications such as sparse approximation of signals, total variation-based image-processing problems, and sensor-location problems.

Introduction

The gradient method is probably one of the oldest optimization algorithms going back as early as 1847 with the initial work of Cauchy. Nowadays, gradient-based methods have attracted a revived and intensive interest among researches both in theoretical optimization, and in scientific applications. Indeed, the very large-scale nature of problems arising in many scientific applications, combined with an increase in the power of computer technology have motivated a “return” to the “old and simple” methods that can overcome the curse of dimensionality; a task which is usually out of reach for the current more sophisticated algorithms.

One of the main drawbacks of gradient-based methods is their speed of convergence, which is known to be slow. However, with proper modeling of the problem at hand, combined with some key ideas, it turns out that it is possible to build fast gradient schemes for various classes of problems arising in applications and, in particular, signal-recovery problems.

The past two decades have witnessed the onset of a surge of research in optimization. This includes theoretical aspects, as well as algorithmic developments such as generalizations of interior-point methods to a rich class of convex-optimization problems. The development of general-purpose software tools together with insight generated by the underlying theory have substantially enlarged the set of engineering-design problems that can be reliably solved in an efficient manner. The engineering community has greatly benefited from these recent advances to the point where convex optimization has now emerged as a major signal-processing technique. On the other hand, innovative applications of convex optimization in signal processing combined with the need for robust and efficient methods that can operate in real time have motivated the optimization community to develop additional needed results and methods. The combined efforts in both the optimization and signal-processing communities have led to technical breakthroughs in a wide variety of topics due to the use of convex optimization. This includes solutions to numerous problems previously considered intractable; recognizing and solving convexoptimization problems that arise in applications of interest; utilizing the theory of convex optimization to characterize and gain insight into the optimal-solution structure and to derive performance bounds; formulating convex relaxations of difficult problems; and developing general purpose or application-driven specific algorithms, including those that enable large-scale optimization by exploiting the problem structure.

In recent years, there has been a growing interest in blind separation of non-negative sources, known as simply non-negative blind source separation (nBSS). Potential applications of nBSS include biomedical imaging, multi/hyper-spectral imaging, and analytical chemistry. In this chapter, we describe a rather new endeavor of nBSS, where convex geometry is utilized to analyze the nBSS problem. Called convex analysis of mixtures of non-negative sources (CAMNS), the framework described here makes use of a very special assumption called local dominance, which is a reasonable assumption for source signals exhibiting sparsity or high contrast. Under the locally dominant and some usual nBSS assumptions, we show that the source signals can be perfectly identified by finding the extreme points of an observation-constructed polyhedral set. Two methods for practically locating the extreme points are also derived. One is analysis-based with some appealing theoretical guarantees, while the other is heuristic in comparison, but is intuitively expected to provide better robustness against model mismatches. Both are based on linear programming and thus can be effectively implemented. Simulation results on several data sets are presented to demonstrate the efficacy of the CAMNS-based methods over several other reported nBSS methods.

Introduction

Blind source separation (BSS) is a signal-processing technique, the purpose of which is to separate source signals from observations, without information of how the source signals are mixed in the observations. BSS presents a technically very challenging topic to the signal processing community, but it has stimulated significant interest for many years due to its relevance to a wide variety of applications.

In this chapter, we study specific rank-1 decomposition techniques for Hermitian positive semidefinite matrices. Based on the semidefinite programming relaxation method and the decomposition techniques, we identify several classes of quadratically constrained quadratic programming problems that are polynomially solvable. Typically, such problems do not have too many constraints. As an example, we demonstrate how to apply the new techniques to solve an optimal code design problem arising from radar signal processing.

Introduction and notation

Semidefinite programming (SDP) is a relatively new subject of research in optimization. Its success has caused major excitement in the field. One is referred to Boyd and Vandenberghe [11] for an excellent introduction to SDP and its applications. In this chapter, we shall elaborate on a special application of SDP for solving quadratically constrained quadratic programming (QCQP) problems. The techniques we shall introduce are related to how a positive semidefinite matrix can be decomposed into a sum of rank-1 positive semidefinite matrices, in a specific way that helps to solve nonconvex quadratic optimization with quadratic constraints. The advantage of the method is that the convexity of the original quadratic optimization problem becomes irrelevant; only the number of constraints is important for the method to be effective. We further present a study on how this method helps to solve a radar code design problem. Through this investigation, we aim to make a case that solving nonconvex quadratic optimization by SDP is a viable approach.

Mixed logical dynamical systems and linear complementarity systems are representations of switched systems, which under the conditions described here are equivalent to the model used in Chapter 4. They are particularly useful for model-predictive control. The equivalences of several hybrid system models show that different models, which are suitable for specific analysis and design problems and have been investigated in detail, cover the same class of hybrid systems. The analysis of the well-posedness of the models leads to conditions on the model equations under which a unique solution exists.

Model-predictive control of hybrid systems

Model-predictive control (MPC) is a widely used technology in industry for control design of highly complex multivariable processes. The idea behind MPC is to start with a model of the open-loop process that explains the dynamical relations among system's variables (command inputs, internal states, and measured outputs). Then, constraint specifications on system variables are added, such as input limitations (typically due to actuator saturation) and desired ranges where states and outputs should remain. Desired performance specifications complete the control problem setup and are expressed through different weights on tracking errors and actuator efforts (as in classical linear quadratic regulation). At each sampling time, an open-loop optimal control problem based on the given model, constraints, weights, and with initial condition set at the current (measured or estimated) state, is repeatedly solved through numerical optimization.

By considering a solar air conditioning plant, the typical steps for analyzing and controlling hybrid systems under practical circumstances are described in this chapter.

Plant description

The present chapter describes the application and the implementation of a hybrid control scheme of a solar air conditioning plant. The conditioning plant considered is a hybrid system characterized by a variable configuration, with discrete and continuous variables, and components that change their dynamics according to the conditions under which the plant operates. Section 17.1 describes the solar air conditioning plant. Section 17.2 shows briefly the hybrid modeling of the plant. Section 17.3.1 describes the control requirements to operate the plant. Section 17.3.2 develops a hybrid control strategies for the operation of the plant. Section 17.3.3 shows the experimental results and discusses them.

Main components

The solar air conditioning plant considered in this chapter is located in Seville (Spain) and is used to cool down the laboratories of the Department of System Engineering and Automation of the University of Seville. It consists of a solar field producing hot water that feeds into an absorption machine, generating chilled water and injecting it into the air distribution system, which has a cooling power of 35 kW.

Figure 17.1 offers a general scheme of the plant, and shows its main components: the solar subsystem, composed of a set of flat solar collectors; the accumulation subsystem, composed of two tanks storing hot water; and the cooling machine.

Hybrid systems are dynamical systems that consist of components with continuous and discrete behavior. Modeling, analysis, and design of such systems raise severe methodological questions, because they necessitate the combination of continuous variable system descriptions like differential and difference equations with discrete-event models like automata or Petri nets. Consequently, hybrid systems methodology is based on the principles and results of the theories of continuous and discrete systems, which, until recently, have been elaborated separately, with contributions coming from different disciplines, such as control theory, computer science, and mathematics.

This handbook reviews the new phenomena and theoretical problems brought about by the combination of continuous and discrete dynamics and surveys the main approaches, methods, and results that have been obtained during the last decade of research in this field. It is structured into three main parts:

• Part I: Modeling, analysis, and control design methods: The first part gives a thorough introduction to hybrid systems theory. The material is classified by the modeling approaches used to represent hybrid systems in a form that is convenient for analysis and control design. Hybrid automata and switched systems are well-studied system classes, which are extensively described, but other approaches like mixed logical dynamical systems, complementarity systems, quantized systems, and stochastic hybrid systems are also explained.

• Part II: Tools: The second part is concerned with computer-aided systems analysis, control design, and verification. After a survey of the variety of relevant tools, selected tools are described in more detail. […]

An overview of various modeling frameworks for hybrid systems is given followed by a comparison of the modeling power and the model complexity, which can serve as a guideline for choosing the right model for a given analysis or control problem with hybrid dynamics. Then, the main analysis and design tasks for hybrid systems are surveyed together with the methods for their solution, which will be discussed in more detail in subsequent chapters.

Models for hybrid systems

Overview

As models are the ultimate tools for obtaining and dealing with knowledge, not only in engineering, but also in philosophy, biology, sociology, and economics, a search has been undertaken for appropriate mathematical models for hybrid systems. This section gives an overview of the modeling formalisms that have been elaborated in hybrid systems theory in the past.

Structure of hybrid systems Many different models have been proposed in literature, as will be seen in following chapters. These models can be distinguished with respect to the phenomena that they are able to represent in an explicit form. Consequently, these models have different fields of applications. The main idea of these models is described by the block diagram shown in Fig. 2.1, which is often used in literature as a starting point of hybrid systems modeling and analysis, although not all models use this structure in a direct way.

Control loops which are closed over a digital communication network became a topic of intensive research in the recent years. This chapter surveys the main problems to be solved and show how hybrid systems theory can help to solve them. The chapter ends with a case study that was inspired by a practical application of hybrid systems methods in an ore mine.

Introduction to distributed control applications and networked control systems

In this chapter, we deal with control over networks, i.e. with control implementations where the control actions and decisions are taken based on measurement, decisions, and actuations that take place in a distributed environment. The control agents may rely upon a centralized facility that coordinates and optimizes the overall control strategy and on a shared communication resource like a bus (distributed control) or may be acting only on local information and on data exchanged with neighboring nodes (decentralized control). There are obviously pros and cons for each strategy. Decentralized control systems have the following characteristics:

• There is no central control node.

• There is no common communication facility; communication is point-to-point.

• The global network topology is unknown to the nodes, which are only aware of their neighborhood.

These features yield interesting properties:

• The system is scalable: there are no limits imposed by centralized computing power or global communication bandwidth.

• The system is robust and fault tolerant, because it supports dynamical changes of the network topology and losses of nodes. […]

Hybrid automata is a modeling formalism for hybrid systems that results from an extension of finite-state machines by associating with each discrete state a continuous-state model. Conditions on the continuous evolution of the system invoke discrete state transitions. A broad set of analysis methods is available for hybrid automata including methods for the reachability analysis, stability analysis, and optimal control.

Definition

A hybrid automaton is a transition system that is extended with continuous dynamics. It consists of locations, transitions, invariants, guards, n-dimensional continuous functions, jump functions, and synchronization labels. Various definitions exist in the literature which differ only in details. The following definition covers all of the aspects needed for the purpose of this handbook. A hybrid automaton consists of:

• a finite set of locations Q, q ∈ Q;

• a finite transition relation ⊖ ⊆ Q × Q

for the specification of the discrete dynamics. The locations can be seen as discrete states (also called control modes), in other words as the discrete part of the hybrid state space ℋ. Transitions from one control mode to the next are often called control switches. The continuous dynamics is described by:

• a finite and indexed set of continuous variables V = {x1, x2, …, xn}, often written as a vector x = (x1, …, xn);

• a real-valued activity function f : Q × ℝn → ℝn, often defined by a continuous differential equation ẋ = dx/dt = f(q, x).