We have presented a mathematical framework that results in the specification of the broadest possible class of linear stochastic processes. The remarkable aspect is that these continuous-domain processes are either Gaussian or sparse, as a direct consequence of the theory. While the formulation relies on advanced mathematical concepts (distribution theory, functional analysis), the underlying principles are simple and very much in line with the traditional methods of statistical signal processing. The main point is that one can achieve a whole variety of sparsity patterns by combining relatively simple building blocks: non-Gaussian white-noise excitations and suitable integral operators. This results in non-Gaussian processes whose properties are compatible with the modern sparsity-based paradigm for signal processing. Yet, the proposed class of models is also backward-compatible with the linear theory of signal processing (LMMSE = linear minimum mean square estimation) since the correlation structure of the processes remains the same as in the traditional Gaussian case – the processes are ruled by the same stochastic differential equations and it is only the driving terms (innovations) that differ in their level of sparsity. On the theoretical front, we have highlighted the crucial role of the generalized B-spline function βL – in one-to-one relation with the whitening operator L – that provides the functional link between the continuous-domain specification of the stochastic model and the discrete-domain handling of the sample values. We have also shown that these processes admit a sparse wavelet decomposition whenever the wavelet is matched to the whitening operator.

A simple and surprisingly effective approach for removing noise in images is to expand the signal in an orthogonal wavelet basis, apply a soft-threshold to the wavelet coefficients, and reconstruct the “denoised” image by inverse wavelet transformation. The classical justification for the algorithm is that i.i.d. noise is spread out uniformly in the wavelet domain while the signal gets concentrated in a few significant coefficients (sparsity property) so that the smaller values can be primarily attributed to noise and easily suppressed.

In this chapter, we take advantage of our statistical framework to revisit such wavelet-based reconstruction methods. Our first objective is to present some alternative dictionary-based techniques for the resolution of general inverse problems based on the same stochastic models as in Chapter 10. Our second goal is to take advantage of the orthogonality of wavelets to get a deeper understanding of the effect of proximal operators while investigating the possibility of optimizing shrinkage/thresholding functions for better performance. Finally, we shall attempt to bridge the gap between operator-based regularization, as discussed in Sections 10.2–10.3, and the imposition of sparsity constraints in the wavelet domain. Fundamentally, this relates to the dichotomy between an analysis point of view of the problem (typically in the form of the minimization of an energy functional with a regularization term) vs. a synthesis point of view, where a signal is represented as a sum of elementary constituents (wavelets.)

The chapter is composed of two main parts. The first is devoted to inverse problems in general.

As we saw in Chapter 8, we have at our disposal two primary tools to analyze/characterize sparse signals: (1) the construction of the generalized-increment process by application of the discrete version of the whitening operator (localization), and (2) a wavelet analysis using operator-like basis functions. The goal shared by the two approaches is to approximately recover the innovations by numerical inversion of the signal-generation model. While the localization option appears to give the finest level of control, the wavelet option is interesting as well because of its greater robustness to modeling errors. Indeed, performing a wavelet transform (especially if it is orthogonal) is a much stabler operation than taking a discrete derivative, which tends to amplify high-frequency perturbations.

In this chapter, we take advantage of our functional framework to gain a better understanding of the true nature of the transform-domain statistics. Our investigation revolves around the fact that the marginal pdfs are infinitely divisible (id) and that they can be determined explicitly via the analysis of a common white noise (innovation).We make use of this result to examine a number of relevant properties of the underlying id distributions and to show that they are, for the most part, invariant to the actual shape of the analysis window. This implies, among other things, that the qualitative effect of operator-like filtering is the same in both analysis scenarios, in the sense that the sparsity pattern of the innovation is essentially preserved.

A fundamental aspect of our formulation is that the whitening operator L is naturally tied to some underlying B-spline function, which will play a crucial role in what follows. The spline connection also provides a strong link with wavelets [UB03].

In this chapter, we review the foundations of spline theory and show how one can construct B-spline basis functions and wavelets that are tied to some specific operator L. The chapter starts with a gentle introduction to wavelets that exploits the analogy with Lego blocks. This naturally leads to the formulation of a multiresolution analysis of L2(ℝ) using piecewise-constant functions and a de visu identification of Haar wavelets. We then proceed in Section 6.2 with a formal definition of our generalized brand of splines – the cardinal L-splines – followed by a detailed discussion of the fundamental notion of the Riesz basis. In Section 6.3, we systematically cover the first-order operators with the construction of exponential B-splines and wavelets, which have the convenient property of being orthogonal. We then address the theory in its full generality and present two generic methods for constructing B-spline basis functions (Section 6.4) and semi-orthogonal wavelets (Section 6.5). The pleasing aspect is that these results apply to the whole class of shift-invariant differential operators L whose null space is finite-dimensional (possibly trivial), which are precisely those that can be safely inverted to specify sparse stochastic processes.

In this chapter, the probability density function of the hopcount and of the weight of the shortest path to the most nearby member of an anycast group consisting of m members (e.g. servers or peers) in a graph of N nodes is analyzed.

The results are applied to compute a performance measure η of the efficiency of anycast over unicast and to the server placement problem. The server placement problem asks for the number of (replicated) servers m needed such that any user in the network is not more than j hops away from a server of the anycast group with a certain prescribed probability. As in Chapter 18 on multicast, two types of shortest path trees are investigated: the regular k-ary tree and the irregular uniform recursive tree treated in Chapter 16. Since these two extreme cases of trees indicate that the performance measure η ≈ 1 — a log m, where the real number a depends on the details of the tree, it is believed that for trees in real networks (as the Internet) a same logarithmic law applies. An order calculus on exponentially growing trees further supplies evidence for the conjecture that η ≈ 1 — a log m for small m.

In peer-to-peer (P2P) networks (see e.g. Van Mieghem (2010a, Chapter 13)) such as Napster, Bit Torrent and Tribler, content is often either fully replicated at a peer or is split into chunks and stored over m peers. A major task of a member of the peer group lies in selecting the best peer among those m peers that possess the desired content.

The efficiency or gain of multicast in terms of network resources is compared to unicast. Specifically, we concentrate on a one-to-many communication, where a source sends a same message to m different, uniformly distributed destinations along the shortest path. In unicast, this message is sent m times from the source to each destination. Hence, unicast uses on average fN(m)= mE [HN] link-traversals or hops, where E [HN] is the mean number of hops to a uniform location in the graph with N nodes. One of the main properties of multicast is that it economizes on the number of link-traversals: the message is only copied at each branch point of the multicast tree to the m destinations. Let us denote by HN (m) the number of links in the shortest path tree (SPT) to m uniformly chosen nodes. If we define the multicast gain gN (m) = E [HN(m)] as the mean number of hops in the SPT rooted at a source to m randomly chosen distinct destinations, then gN(m) ≤ fN(m). The purpose here is to quantify the multicast gain gN(m). We present general results valid for all graphs and more explicit results valid for the random graph Gp(N) and for the k-ary tree. The analysis presented here may be valuable to derive a business model for multicast: “How many customers m are needed to make the use of multicast for a service provider profitable?”

This chapter presents some of the simplest and most basic queueing models. Un-fortunately, most queueing problems are not available in analytic form and many queueing problems require a specific and sometimes tailor-made solution.

Beside the simple and classical queueing models, we also present two other exact solvable models that have played a key role in the development of Asynchronous Transfer Mode (ATM). In these ATM queueing systems the service discipline is deterministic and only the arrival process is the distinguishing element. The first is the N*D/D/1 queue (Roberts, 1991, Section 6.2) whose solution relies on the Benes approach. The arrivals consist of N periodic sources each with period of D time slots, but randomly phased with respect to each other. The second model is the fluid flow model of Anick et al. (1982), known as the AMS-queue, which considers N on-off sources as input. The solution uses Markov theory. Since the Markov transition probability matrix has a special tri-band diagonal structure, the eigenvector and eigenvalue decomposition can be computed analytically.

We would like to refer to a few other models. Norros (1994) succeeded in deriving the asymptotic probability distribution of the unfinished work for a queue with self-similar input, modeled via a fractal Brownian motion. The resulting asymptotic probability distribution turns out to be a Weibull distribution (3.49).

Queueing theory describes basic phenomena such as the waiting time, the through-put, the losses, the number of queueing items, etc. in queueing systems. Following Kleinrock (1975), any system in which arrivals place demands upon a finite-capacity resource can be broadly termed a queueing system.

Queuing theory is a relatively new branch of applied mathematics that is generally considered to have been initiated by A. K. Erlang in 1918 with his paper on the design of automatic telephone exchanges, in which the famous Erlang blocking probability, the Erlang B-formula (14.18), was derived (Brockmeyer et al., 1948, p. 139). It was only after the Second World War, however, that queueing theory was boosted mainly by the introduction of computers and the digitalization of the telecommunications infrastructure. For engineers, the two volumes by Kleinrock (1975, 1976) are perhaps the most well-known, while in applied mathematics, apart from the penetrating influence of Feller (1970, 1971), the Single Server Queue of Cohen (1969) is regarded as a landmark. Since Cohen's book, which incorporates most of the important work before 1969, a wealth of books and excellent papers have appeared, an evolution that is still continuing today.

A queueing system

Examples of queueing abound in daily life: queueing situations at a ticket window in the railway station or post office, at the cash points in the supermarket, the waiting room at the airport, train or hospital, etc.