In this chapter we consider integral representations of functions using kernels that are ridge functions. The most commonly used integral representation with a ridge function kernel is that given by the Fourier transform. The function ψw := eix·w is, for each fixed w, a ridge function. Under suitable assumptions on an f defined on Rn, we have

For example, the above holds if f ∊ L1(Rn) n C(n) and,. While we have drifted into the complex plane, we can easily rewrite the above using only real-valued functions.

Another integral representation with a ridge function kernel may be found in John [1955], p. 11. We present it here without proof. Assume, i.e., f and its first partial derivatives are continuous functions, and f has compact support. Let Δx denote the Laplacian with respect to the variables x1, …, xn, i.e.,

Let Sn−1 denote the unit sphere in Rn, and let da be uniform measure on Sn−1 of total measure equal to the surface area of the unit sphere Sn−1, i.e., of total measure

For n and k odd positive integers, we have

For n an even positive integer and k any even non-negative integer, we have

The relationship between these integral formulæ and the inverse Radon transform can be found in John [1955], together with other similar integral representations.

In this chapter we review two additional integral representations. The first, using an orthogonal decomposition in terms of Gegenbauer polynomials is from Petrushev [1998]. The second is based upon ridgelets and was presented by Candès [1998] in his doctoral thesis, see also Candès [1999].

This monograph is an attempt to examine and study ridge functions as entities in and of themselves. As such we present what we consider to be various central properties of ridge functions. However, no encyclopedic claims are being made, and the topics chosen are those that we alone considered appropriate. In addition, most chapters contain, either explicitly or implicitly, unresolved questions. It is our hope that this monograph will prove useful and interesting to both researchers and the more casual reader. And, of course, all errors, omissions and other transgressions are totally our responsibility.

No monograph is written in a vacuum, and I would like to especially thank Carl de Boor, Vugar Ismailov and an anonymous referee for various comments and suggestions. Thanks also to Heinz Bauschke, Vitaly Maiorov, Simon Reich and Yuan Xu for their patience, and help with my various inquiries.

This monograph is about Ridge Functions. A ridge function is any multivariate real-valued function

F : Rn → R

of the form

F(x1, …, xn) = f(a1x1 + … + anxn) = f(a · x),

where x = (x1, …, xn) ∊ Rn are the variables, f is a univariate real-valued function, i.e., f : R → R, and a = (a1, …, an) ∊ Rn\{0} is a fixed vector. This vector a ∊ Rn\{0} is generally called the direction. In other words, a ridge function is a multivariate function constant on the parallel hyperplanes a · x = c, c ∊ R. It is one of the simpler multivariate functions. Namely, it is a superposition of a univariate function with one of the simplest multivariate functions, the inner product.

More generally, we can and will consider, for given d, 1 ≤ d ≤ n−1, functions F of the form

F(x) = f(Ax),

where A is a fixed d × n real matrix, and f : Rd → R. We call such functions Generalized Ridge Functions. For d = 1, this reduces to a ridge function.

Motivation

We see specific ridge functions in numerous multivariate settings without considering them of interest in and of themselves. We find them, for example, as kernels in integral formulæ. They appear in the Fourier transform

and its inverse. We see them in the n-dimensional Radon transform

and its inverse. Here the integral is taken with respect to the natural hypersurface measure dσ. It is possible to generalize the Radon transform still further by integrating over (n − d)-dimensional affine subspaces of Rn.

In this chapter we study one of the basic properties of ridge function decomposition, namely smoothness. In the first section we ask the following question. If

is smooth, does this imply that each of the fi is also smooth? In the second section we ask this same question with regard to generalized ridge functions, i.e., linear combinations of functions of the form f(Ax), where the A are d × n real matrices, and f : Rd → R.

Ridge Function Smoothness

Let Ck(Rn), k ∊ Z+, denote the usual class of real-valued functions with all derivatives of order up to and including k being continuous. Assume F ∊ Ck(Rn) is of the form

where r is finite, i.e., F ∊ M(a1, …, ar), and the ai are given pairwise linearly independent vectors in Rn. What can we say about the smoothness of the fi? Do the fi necessarily inherit all the smoothness properties of the F?

When r = 1 the answer is yes, and there is essentially nothing to prove. That is, if

F(x) = f1(a1 · x)

is in Ck(Rn) for some a1 ≠ 0, then for c ∊ Rn satisfying a1 · c = 1 and all t ∊ R we have that F(tc) = f1(t) is in Ck(R). This same result holds when r = 2. As the a1 and a2 are linearly independent, there exists a vector c ∊ Rn satisfying a1 · c = 0 and a2 · c = 1. Thus

F(tc) = f1(a1 · tc) + f2(a2 · tc) = f1(0) + f2(t).

Since F(tc) is in Ck(R), as a function of t, so is f2. The same result holds for f1.

In this final chapter we discuss the problem of the possibility of interpolation by functions from M(a1, …, ar) on straight lines. That is, assume we are given the straight lines {tbj + cj : t ∊ R}, bj ≠ 0, j = 1, …, m. The question we ask is when, for every (or most) choice of data gj(t), j = 1, …, m, do there exist functions G ∊ M(a1, …, ar) satisfying

G(tbj + cj) = gj(t), t ∊ R, j = 1, …, m?

Why interpolation on straight lines? Because that seems to be the most natural setting for interpolation from ridge functions.

In Section 12.1 we first show that interpolation by ridge functions on any set X in Rn is possible if and only if it is possible on every finite point set {x1, …, xk} ⊂ X. In Section 12.2 we show what happens when r = 1, i.e., when we have only one direction. We will show that we can interpolate from M(a) on the straight line {tb + c : t ∊ R} if and only if a · b ≠ 0, while we can never interpolate from M(a) to all given functions on the union of two straight lines. In Section 12.3 we consider the case of two directions, i.e., interpolation from M(a1, a2). We show exact conditions under which we can interpolate on two distinct straight lines. We also show how to reduce these conditions to more meaningful geometric conditions when we are in R2. In addition, by example, we show that while the data (the G) might be continuous on the union of two straight lines where interpolation fromM(a1, a2) is possible, this does not imply that the associated f1 and f2 in the representation

G(x) = f1(a1 · x) + f2(a2 · x)

can be taken to be continuous. In Section 12.4 we reprove the major result of Section 12.3 by a different method.

In this chapter we study the question of the existence and characterization of best approximations from the space of linear combinations of ridge functions with a finite number of directions. That is, from the space given by the restriction of

to a domain K ⊆ Rn, and to where all the fi(Aix) lie in an appropriate normed linear space. These normed linear spaces will be X = Lp(K), p ∊ (1, ∞), and X = C(K).

Section 8.1 contains some general results regarding existence and characterization of a best approximation from a linear subspace. In Section 8.2 we consider the space Lp(K) for p ∊ (1, ∞), and highlight the case p = 2, while Section 8.3 contains a few simple examples of that theory. In Section 8.4 we look at C(K) where, unfortunately, we only have results when approximating from linear combination of ridge functions with two directions. Very little seems to be known about these questions in other normed linear spaces.

General Results

In approximation theory, a set M in a normed linear space X is said to be an existence set (sometimes called a proximinal set) if to each G ∊ X there exists at least one best approximation to G from M. That is, to each G ∊ X there exists an F* ∊ M satisfying

A necessary condition for M to be an existence set is that it be closed. But closure, in general, is insufficient to guarantee existence.

Closed convex subsets of finite-dimensional subspaces are existence sets. But linear combinations of ridge functions of the form (8.1) are not finite-dimensional, unless K is very restricted. However, closed convex sets of a uniformly convex Banach space are existence sets.

In this chapter we consider ridge functions that are algebraic polynomials. While some of the material detailed here is a consequence of more general results, we also present results that are particular to polynomials. We start with a review of some of the basic notions. In Section 5.1 we consider homogeneous polynomials and present results on spanning, linear independence and interpolation by linear combinations of the (a · x)m for fixed m ∊ Z+, as we vary over a subset Ω of directions in Rn. In Section 5.2 we translate many of these results to the space of algebraic polynomials. Section 5.3 is concerned with what is called Waring's Problem for real polynomials (real linear forms). We consider the minimal number of linear combinations of ridge polynomials needed to represent any algebraic or homogeneous polynomial. Finally, in Section 5.4 we discuss generalized ridge functions that are polynomials. That is, we consider linear combinations of functions of the form p(Ax), where the A are fixed d × n matrices, and the p are d-variate polynomials.

Homogeneous Polynomials

As a matter of convenience we will, for a fixed direction a ∊ Rn\{0} and univariate polynomial p of degree m, term p(a · x) a ridge polynomial of degree m with direction a, and the polynomial (a · x)m will be called a ridge monomial of degree m with direction a.

For a given set Ω ⊆ Rn let

L(Ω) := {ƛa : a ∊ Ω, ƛ ∊ R}.

The set of ridge polynomials with direction a is the same as the set of ridge polynomials with direction ƛa for any ƛ ∊ R, ƛ ≠ 0. Thus L(Ω) is the set of directions we should or could be considering.

That is, P(Ω) is the set of all polynomials that vanish on L(Ω). Note that a homogeneous polynomial vanishes on L(Ω) if and only if it vanishes on Ω.

In this chapter we consider the problem of the uniqueness of the representation of a linear combination of a finite number of ridge functions. That is, assume we have two distinct representations for F of the form

where both k and l are finite. What can we say about these two representations? From linearity (3.1) is effectively equivalent to asking the following. Assume

for all x ∊ Rn, where r is finite, and the ai are pairwise linearly independent vectors in Rn. What does this imply regarding the fi?

The main result of the first section of this chapter is that, with minimal requirements, the fi satisfying (3.2) must be polynomials of degree ≤ r − 2. That is,we essentially have uniqueness of the representation of a finite linear combination of ridge functions up to polynomials of a certain degree. We extend this result, in the second section, to generalized ridge functions. Much of the material of this chapter is taken from Pinkus [2013], and generalizes a result of Buhmann and Pinkus [1999].

Ridge Function Uniqueness

We recall from Chapter 2 that B is any linear space, closed under translation, of real-valued functions f defined on R such that if there is a function g ∊ C(R) for which f − g satisfies the Cauchy Functional Equation (2.2), then f − g is necessarily a linear function.

As in Section 1.3, let denote the set of algebraic polynomials of total degree at most m in n variables. That is,

Theorem 3.1Assume (3.2) holds where r is finite, and theaiare pairwise linearly independent vectors in Rn. Assume, in addition, that fi ∊ B for i = 1, …, r. Then fi is a univariate polynomial of degree at most r − 2, i = 1, …, r.

Let K be a bounded set in Rn. For a given d × n matrix A, set

M(A;K) := {f(Ax) : x ∊ K, all possible f : Rd → R},

and for each p ∊ [1, ∞], let

Lp(A; K) := M(A; K) ∩ Lp(K).

That is, Lp(A; K) is the set of functions in Lp(K) of the form f(Ax). In this chapter we consider the question of the closure of

in Lp(K). This question has been studied with an eye to computerized tomography. It is relevant to the problem of the existence of best approximations, see Chapter 8, and also to the rate of convergence of some best approximation algorithms, see Chapter 9. It is also an interesting question in and of itself. In Section 7.1 we present various theorems providing conditions for when closure holds, and in Section 7.2 some examples of when closure is lacking. Some additional results are given in Section 7.3. In Section 7.4 we consider the case of C(K). For two directions closure in C(K) is equivalent to certain geometric properties of the set K and the directions. However, nothing seems to be known in the case of more than two directions. This chapter is more of a survey as the theorems are presented without proofs. The inclusion of their proofs would lead us too far afield.

The space Lp(K), p ∊ [1, ∞), is the standard linear space of functions G defined on K ⊆ Rn for which |G|p is Lebesgue integrable, and with norm given by

L∞(K) is the space of Lebesgue measurable, essentially bounded functions G, with norm

The first general results on closure seem to be due to Hamaker and Solmon [1978] who considered the case where p = 2, K is a disk with center at the origin in R2, and is a set of pairwise linearly independent vectors in R2.

In this chapter we consider the following questions. Can we characterize those pointsx1, …, xk ∊ Rn(any finite k) such that for every choice of data b1, …, bk (bj ∊ R, j = 1, …, k), there exists a function G ∊ M(A1, …, Ar) satisfying

G(xj) = bj, j = 1, …, k?

That is, for given fixed d × n matrices A1, …, Ar, do there exist functions f1, …, fr : Rd → R for which

For r = 1 this problem has a simple solution. Given a d × n matrix A, we want to know conditions on the points in Rn such that for every choice of b1, …, bk there exists a function f : Rd → R (depending on the xj and bj) such that

f(Axj) = bj, j = 1, …, k.

Obviously such a function exists if and only if

Axs ≠ Axt

for all s ≠ t, s, t ∊ {1, …, k}. And, in general, if for some i ∊ {1, …, r}, the values Aixj, j = 1, …, k, are all distinct, then it easily follows that we can interpolate as desired, independent of and without using the other Al, l ≠ i. The problem becomes more interesting and more difficult when, for each i, the k values Aixj, j = 1, …, k, are not all distinct.

In Section 11.1 we state some general, elementary results concerning interpolation at points. In Section 11.2 we detail necessary and sufficient conditions for when we can interpolate in the case of two directions, i.e., r = 2. In Section 11.3 we consider the case of r ≥ 3 directions, but only in R2, and present an exact geometric characterization for a large (but not the complete) set of points where interpolation is not always possible.

In this chapter we consider the following inverse problem. Assume that we are given a function F that we know is of the form

for some choice of positive integer r, unknown functions fi, and either known or unknown directions ai. The question we ask is how to determine these unknowns parameters based on our knowledge of F. In the first section we assume that we know the directions ai, while in the second section we assume they are unknown. In Section 4.3 we pose these same questions for generalized ridge functions. In the case of unknown Ai we are able only to analyze the case r = 1. Thematerial of Sections 4.1 and 4.2 may be found in Buhmann and Pinkus [1999].

Known Directions

Assume that we know an F of the form (4.1) with given directions ai. How can we theoretically identify the functions fi? (We will, of course, assume that the directions ai are pairwise linearly independent.) As we have seen, from the previous chapters, we have a degree of non-unicity. However, assuming that F is smooth and fi ∊ B for all i, then from Theorem 3.1 the fi are determined, at the very least, up to polynomials of degree at most r − 2.

Let us now detail how we might determine the fi. When r = 1 we need make no assumptions as

F(x) = f1(a1 · x).

Choosing c ∊ Rn such that a1 · c = 1, we have

F(tc) = f1(t),

which gives us f1. Similarly, for r = 2 we can find a c ∊ Rn satisfying a1 · c = 1 and a2 · c = 0, and thus

F(tc) = f1(t) + f2(0),

which determines f1 up to a constant. In this same manner we determine f2 up to a constant.

We are interested in algorithmic methods for finding best approximations from spaces of linear combinations of ridge functions. The main problem we will consider is that of approximating from the linear space

over some domain in Rn, where r is finite and each fi(Aix) is in an appropriate normed linear space X. Recall that the Ai are fixed d × n matrices and the dvariate functions Ai are the variables. That is, we are looking at the question of approximating by generalized ridge functions with fixed directions. We are also interested in the problem of approximating from the set of ridge functions with variable directions. This problem is significantly different.

We predicate these algorithmic approximation methods on the following basic assumption. For each i ∊ {1, …, r}, set

where Ai is a fixed d × n matrix and f(Aix) lies in the appropriate space. Let Pi be a best approximation operator to M(Ai), i.e., to each G the element PiG is a best approximation to G from M(Ai). The major assumption underlying the methods discussed in this chapter is that each Pi is computable (see Example 8.1). Based on this assumption we outline various approximation approaches.

In Section 9.1 we discuss approximation algorithms in a Hilbert space setting. The theory is the most detailed when M(A1, …, Ar) is closed. However, some convergence results are also known without the closure property. In Section 9.2 we generalize the above to consider a “greedy-type algorithm”. This permits us to deal with the possibility of an infinite number of directions. In Section 9.3 we consider the same problem as in Section 9.1, but in a uniformly convex and uniformly smooth Banach space.

In this chapter we consider the question of the density of linear combinations of ridge functions in the set of continuous functions on Rn, in the topology of uniform convergence on compact subsets of Rn. In Section 6.1 we consider a set of fixed directions and obtain necessary and sufficient conditions on the direction set for when we have density of linear combinations of all possible ridge functions with these directions. In Section 6.2 we discuss this same question with regard to generalized ridge functions. We discuss the question of density when permitting variable directions in Section 6.3. In Section 6.4 we ask for conditions on when a specific function in C(Rn) can be approximated by a linear combination of ridge function with given directions, without the density assumption. Much of the presentation of Sections 6.1, 6.2 and 6.4 is taken from Lin and Pinkus [1993]. In Section 6.5 we study a related but different question. Here we permit all possible directions, but restrict the set of permissible functions. We look at the class of ridge functions generated by shifts of a single function. This question is relevant in ridgelet analysis and in the MLP model in neural networks.

Density with Fixed Directions

In this section we consider the question of the density of linear combinations of ridge functions with a given set of directions. Let Ω denote any subset of vectors in Rn, and set

M(Ω) := span{f(a · x) : a ∊ Ω, f ∊ C(R)}.

That is, in M(Ω) we vary over all a ∊ Ω and all f ∊ C(R), and take arbitrary linear combinations of the elements of this set. This is our approximating set. The question we ask is one of density: when can elements of the above set approximate arbitrarily well any function in our class?