2.1 Background

The map $\operatorname {E}$ has a long history in the problem of finding the closest normal matrix to a given matrix, going back at least to Henrici [Reference Henrici30], who proved the following:

Proposition 2.1 (Henrici [Reference Henrici30]).

For any $A \in \mathbb {C}^{d \times d}$ ,

$$\begin{align*}\inf_{M \in \mathcal{N}_d} \|A - M\| \leq \left(\frac{d^3-d}{12}\operatorname{E}(A)\right)^{1/4}. \end{align*}$$

In other words, the distance from A to $\mathcal {N}_d$ is bounded above by a quantity proportional to $\operatorname {E}(A)^{1/4}$ . One virtue of this estimate is that $\operatorname {E}(A)$ is relatively easy to compute.

We now give an interpretation of $\operatorname {E}$ in terms of symplectic geometry, where we consider $\mathbb {C}^{d \times d} \approx \mathbb {C}^{d^2}$ as a symplectic manifold with its standard symplectic structure. This interpretation is not necessary for most of the paper, and is mainly included for context. As such, we give a somewhat informal treatment and avoid explicit definitions of any of the standard terminology from symplectic geometry. In our previous papers, we give short and elementary overviews of the necessary concepts from symplectic geometry, with a view toward understanding similar spaces of structured matrices (e.g., spaces consisting of unit norm tight frames); we refer the reader to [Reference Needham and Shonkwiler46, Section 2] and [Reference Needham and Shonkwiler47, Section 2.1] for more in-depth exposition.

Consider the action of the unitary group $\operatorname {SU}(d)$ on $\mathbb {C}^{d \times d}$ by conjugation. Let $\mathfrak {su}(d)$ denote the Lie algebra of $\operatorname {SU}(d)$ – that is, the traceless, skew-Hermitian $d\times d$ matrices – and let $\mathfrak {su}(d)^\ast $ denote its dual. It will be convenient to identify $\mathfrak {su}(d)^\ast $ with the space $\mathscr {H}_0(d)$ of $d \times d$ traceless Hermitian matrices via the isomorphism

$$ \begin{align*} \mathscr{H}_0(d) &\to \mathfrak{su}(d)^\ast \\ Y &\mapsto \big(X \mapsto \frac{i}{2}\mathrm{Tr}(XY)\big). \end{align*} $$

Then we have the following interpretation of $\operatorname {E}$ .

Proposition 2.2. The conjugation action of $\operatorname {SU}(d)$ on $\mathbb {C}^{d \times d}$ is Hamiltonian, with momentum map

(2.1) $$ \begin{align} \mu: \mathbb{C}^{d \times d} &\to \mathscr{H}_0(d) \approx \mathfrak{su}(d)^\ast \end{align} $$

(2.2) $$ \begin{align}\kern-5pt A &\mapsto [A,A^\ast]. \end{align} $$

The non-normal energy $\operatorname {E}$ is therefore the squared norm of a momentum map.

We omit the proof of Proposition 2.2, which is a straightforward calculation. In light of this result, one should expect the non-normal energy to have nice properties – see, for example, work of Kirwan [Reference Kirwan37] and Lerman [Reference Lerman40]. However, the specific properties of $\operatorname {E}$ (and related functions) that we derive below do not follow directly from the general theory.

2.2 Critical points of non-normal energy

Obviously, the global minima of the non-normal energy $\operatorname {E}$ are exactly the normal matrices. In fact, we now show that these are the only critical points. Throughout the paper, we use $\langle \cdot , \cdot \rangle $ to denote the real part of the Frobenius inner product on $\mathbb {C}^{d \times d}$ ,

and we use $D\mathrm {F}(A)$ to denote the derivative of a map $\mathrm {F}:\mathbb {C}^{d \times d} \to \mathbb {R}$ at $A \in \mathbb {C}^{d \times d}$ .

Proof. We claim that

(2.3) $$ \begin{align} \nabla \operatorname{E}(A) = -4[A,[A,A^\ast]]. \end{align} $$

Indeed, since $\operatorname {E}$ is the square of a momentum map (Proposition 2.2), this follows by general principles of symplectic geometry – see, for example, [Reference Kirwan37, Lemma 6.6] or [Reference Ness49, Lemma 6.1]. Let us additionally give an elementary derivation of this fact. Writing $\operatorname {E} = N \circ \mu $ , where $\mu $ is the momentum map (2.1) and $N:\mathbb {C}^{d \times d} \to \mathbb {R}$ is the norm-squared map $N(A) = \|A\|^2$ , we have, for any $A,B \in \mathbb {C}^{d \times d}$ ,

$$ \begin{align*} &\langle \nabla \operatorname{E}(A), B\rangle = D\operatorname{E}(A)(B) = DN(\mu(A)) \circ D\mu(A) (B) \\ &\quad= \langle \nabla N (\mu(A)), D\mu(A)(B)\rangle = \langle D\mu(A)^\vee \nabla N (\mu(A)), B\rangle, \end{align*} $$

where we use $D\mu (A)^\vee $ to denote the adjoint of $D\mu (A)$ with respect to the inner product $\langle \cdot , \cdot \rangle $ . It follows that

$$\begin{align*}\nabla \operatorname{E} (A) = D\mu(A)^\vee \nabla N (\mu(A)). \end{align*}$$

A straightforward calculation then shows that the adjoint is given by the formula

(2.4) $$ \begin{align} D\mu(A)^\vee(C) = [C + C^\ast, A]. \end{align} $$

It is also easy to show that $\nabla N(C) = 2C$ , so we conclude that

$$\begin{align*}\nabla \operatorname{E} (A) = [2\mu(A) + 2\mu(A)^\ast, A] = -4[A,[A,A^\ast]]. \end{align*}$$

Therefore, we have a critical point of $\operatorname {E}$ exactly when

$$\begin{align*}0 = [A,[A,A^\ast]]; \end{align*}$$

that is, when A and $[A,A^\ast ]$ commute. By Jacobson’s Lemma (stated below as Lemma 2.4), this implies that $[A,A^\ast ]$ is nilpotent. But $[A,A^\ast ]$ is Hermitian, so it is nilpotent if and only if it is the zero matrix, which happens precisely when A is normal.Footnote ¹

Remark 2.5. Theorem 2.3 might lead one to suspect that $\operatorname {E}$ is convex, but it is not. To see this, consider the normal matrices

$$\begin{align*}A_0 = \begin{bmatrix}0 & 1 \\ -1 & 0\end{bmatrix} \qquad \mbox{and} \qquad A_1 = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}. \end{align*}$$

Since they are normal, $\operatorname {E}(A_0) = 0 = \operatorname {E}(A_1)$ . However,

$$\begin{align*}\operatorname{E}((1-t)A_0 + tA_1) = 32t^2(1-t)^2> 0 \end{align*}$$

for all $0<t<1$ , so the interior of the line segment connecting $A_0$ and $A_1$ consists entirely of non-normal matrices, and hence $\operatorname {E}$ is not even quasiconvex, let alone convex. See Figure 2. Of course, we can pad $A_0$ and $A_1$ by zeros to get an analogous example for any $d> 2$ .

Figure 2

The graph of $\operatorname {E}$ restricted to the collection of real matrices of the form $\begin{bmatrix} 0 & x \\ y & 0 \end{bmatrix}$ .

On the other hand, Theorem 2.3 shows that $\operatorname {E}$ is an invex function. Recall that, as first defined by Hanson [Reference Hanson27] (later named by Craven [Reference Craven, Schaible and Ziemba13]), a function $f:\mathbb {R}^n \to \mathbb {R}$ is invex if there exists a function $\eta :\mathbb {R}^{n} \times \mathbb {R}^n \to \mathbb {R}^n$ such that

$$\begin{align*}f(x) - f(u) \geq \langle \eta(x,u), \nabla f(u) \rangle \qquad x,u \in \mathbb{R}^n. \end{align*}$$

A theorem of Craven and Glover [Reference Craven and Glover14] (see also [Reference Ben-Israel and Mond6]) says that a function is invex if and only if its critical points are all global minima; hence, $\operatorname {E}$ is invex.

2.3 Gradient flow of non-normal energy

Consider the negative gradient flow $\mathcal {F}: \mathbb {C}^{d \times d} \times [0,\infty ) \to \mathbb {C}^{d \times d}$ defined by

(2.5) $$ \begin{align} \mathcal{F}(A_0,0) = A_0, \qquad \frac{d}{dt}\mathcal{F}(A_0,t) = -\nabla\operatorname{E}(\mathcal{F}(A_0,t)). \end{align} $$

We pause here to note that there is a substantial history of applying flows like (2.5) to problems in numerical linear algebra, going back at least to Rutishauer’s work on the LU decomposition [Reference Rutishauser53, Section 11]. See Chu’s survey [Reference Chu12] for an introduction to this circle of ideas.

Since $\operatorname {E}$ is a real polynomial function on the real vector space $\mathbb {C}^{d \times d}$ , the gradient flow cannot have limit cycles or other bad behavior [Reference Łojasiewicz41], so Theorem 2.3 implies that, for any $A_0 \in \mathbb {C}^{d \times d}$ , the limit of the gradient flow is well-defined and normal.

From (2.3), we see that

(2.6) $$ \begin{align} \nabla \operatorname{E}(A) &= -4[A,[A,A^\ast]] = -4(A[A,A^\ast]-[A,A^\ast]A) \nonumber\\ &= 4 \left( \left. \frac{d}{d\epsilon}\right|{}_{\epsilon = 0} e^{\epsilon[A,A^\ast]}A e^{-\epsilon[A,A^\ast]}\right) = 4 \left( \left. \frac{d}{d\epsilon}\right|{}_{\epsilon = 0} e^{\epsilon[A,A^\ast]} \cdot A \right). \end{align} $$

Since $[A,A^\ast ]$ is traceless, $e^{\epsilon [A,A^\ast ]} \in \operatorname {SL}_d(\mathbb {C})$ for any $\epsilon $ , so the negative gradient flow lines $\mathcal {F}(A_0,t)$ produced by any $A_0$ stay within the conjugation orbit of $A_0$ . In particular, $A_\infty $ must have the same eigenvalues as $A_0$ . Since real matrices are invariant under gradient flow, we have thus proved:

If $A_0$ and $A_\infty $ are as in Theorem 2.6 and $\lambda _1, \dots , \lambda _d$ are their common eigenvalues, then the normality of $A_\infty $ implies that

$$\begin{align*}\|A_\infty\|^2 = \sum_{i=1}^d |\lambda_i|^2. \end{align*}$$

This immediately implies the following corollary.

A widely used statistic for describing the extent to which a matrix is non-normal is the Henrici departure from normality [Reference Henrici30]. For a matrix $A \in \mathbb {C}^{d \times d}$ with eigenvalues $\lambda _i$ , this is the quantityFootnote ²

$$\begin{align*}\mathrm{Hen}(A) = \|A\|^2 - \sum_{i=1}^d |\lambda_i|^2. \end{align*}$$

Corollary 2.9. Let $A_0 \in \mathbb {C}^{d \times d}$ and let $A_\infty $ be its limit under the gradient flow (2.5). The change in scale along gradient flow is equal to Henrici departure from normality,

$$\begin{align*}\|A_0\|^2 - \|A_\infty\|^2 = \mathrm{Hen}(A_0). \end{align*}$$

2.4 Bound on the distance to the limit of gradient flow

We now show that $A_\infty $ is not too much further from $A_0$ than the closest normal matrix, despite the fact that $A_\infty $ preserves features (spectrum, realness) that the closest normal matrix may not. We do so by a standard argument starting from a Łojasiewicz inequality.

Since $\operatorname {E}$ is the squared norm of a momentum map (Proposition 2.2), a result of Fisher [Reference Fisher20] gives us the desired inequalityFootnote ³:

Proposition 2.10 (Fisher [Reference Fisher20, Theorem 4.7]).

There exist constants $\epsilon , c> 0$ so that for all $A \in \mathbb {C}^{d \times d}$ with $\operatorname {E}(A) < \epsilon $ ,

$$\begin{align*}\|\nabla \operatorname{E}(A)\| \geq c \operatorname{E}(A)^{3/4}. \end{align*}$$

Now we follow a standard argument (see, e.g., Lerman [Reference Lerman40]) to get bounds on the distance from $A_0$ to $A_\infty $ . Certainly this distance is no larger than the length of the gradient flow path:

(2.7) $$ \begin{align} \|A_0 - A_\infty\| \leq \int_0^\infty \left\|\frac{d}{dt} \mathcal{F}(A_0,t)\right\|dt = \int_0^\infty \|\nabla \operatorname{E}(\mathcal{F}(A_0,t))\|dt. \end{align} $$

So long as $\operatorname {E}(\mathcal {F}(A_0,t)) < \epsilon $ ,

$$ \begin{align*} -\frac{d}{dt} (\operatorname{E}(\mathcal{F}(A_0,t)))^{1/4} &= -\frac{1}{4} \operatorname{E}(\mathcal{F}(A_0,t))^{-3/4} D\operatorname{E}(\mathcal{F}(A_0,t)))(-\nabla \operatorname{E}(\mathcal{F}(A_0,t))) \\ &= \frac{1}{4} \operatorname{E}(\mathcal{F}(A_0,t))^{-3/4}\|\nabla \operatorname{E}(\mathcal{F}(A_0,t))\|^2 \geq \frac{c}{4}\|\nabla\operatorname{E}(\mathcal{F}(A_0,t))\|, \end{align*} $$

where the last inequality follows since Proposition 2.10 implies $\operatorname {E}(\mathcal {F}(A_0,t))^{-3/4}\|\nabla \operatorname {E}(\mathcal {F}(A_0,t))\| \geq c$ .

Combining this with (2.7) yields:

$$\begin{align*}\|A_0 - A_\infty\| \leq \int_0^\infty \|\nabla \operatorname{E}(\mathcal{F}(A_0,t))\|dt \leq -\frac{4}{c} \int_0^\infty \frac{d}{dt}(\operatorname{E}(\mathcal{F}(A_0,t)))^{1/4} dt = \frac{4}{c} \operatorname{E}(A_0)^{1/4}. \end{align*}$$

Therefore, we have proved:

Proposition 2.11. There exist constants $\epsilon , c> 0$ so that, if $\operatorname {E}(A_0) < \epsilon $ , then

$$\begin{align*}\|A_0 - A_\infty\| \leq \frac{4}{c} \operatorname{E}(A_0)^{1/4}. \end{align*}$$

Comparing to the Henrici estimate (Proposition 2.1), we see that, at least when $\operatorname {E}(A_0)$ is small, the normal matrix $A_\infty $ we get by doing gradient descent is not much further from $A_0$ than the closest normal matrix is, even though $A_\infty $ has the same spectrum as $A_0$ and is real if $A_0$ is.

Remark 2.12. The closest normal matrix to a given $A_0 \in \mathbb {C}^{d \times d}$ can be computed explicitly by Ruhe’s algorithm [Reference Ruhe52],Footnote ⁴ but the actual closest normal matrix does not have the same spectrum as $A_0$ and may be complex even if $A_0$ is real (see discussion in Chu [Reference Chu11] and Guglielmi and Scalone [Reference Guglielmi and Scalone25]). This suggests that the gradient descent approach to finding a nearby normal matrix may be useful in situations where one is interested in preserving structural properties of the initialization. These observations are borne out by numerical experiments, and indeed $A_\infty $ gets relatively closer to the closest normal matrix when $A_0$ is almost normal to begin with: see Figure 3.

Figure 3

Left: We generated 10,000 initial matrices $A_0 \in \mathbb {C}^{20 \times 20}$ by letting the real and imaginary parts of each entry be drawn from a standard Gaussian and then normalizing so that $A_0$ has Frobenius norm 1. We computed the closest normal matrix $\widehat {A}$ using Ruhe’s algorithm [Reference Ruhe52] and $A_\infty = \displaystyle \lim _{t \to \infty } \mathcal {F}(A_0,t)$ using a very simple gradient descent with fixed step sizes, and then plotted the point $(\|\widehat {A}-A_0\|^2, \|A_\infty - A_0\|^2)$ . The ratios $\frac {\|A_\infty - A_0\|^2}{\|\widehat {A}-A_0\|^2}$ were all in the interval $[1.028,1.161]$ . Center: The same computations and visualization, except the initial matrices $A_0$ were all $20 \times 20$ real matrices. In this case the $\frac {\|A_\infty - A_0\|^2}{\|\widehat {A}-A_0\|^2}$ were all in the interval $[1.023,1.196]$ . Right: The same computations and visualization, but with nearly normal initial matrices $A_0 \in \mathbb {C}^{20 \times 20}$ . More precisely, we generated $B \in \mathbb {C}^{20 \times 20}$ by normalizing a matrix of standard complex Gaussians, found the closest normal matrix $\widehat {B}$ , then added an $\mathcal {N}(0,0.0075)$ random variate to the real and complex parts of each entry of $\widehat {B}$ , and let $A_0$ be the normalization of this matrix, so that $A_0$ has Frobenius norm 1 and is already close to being normal. In this case the $\frac {\|A_\infty - A_0\|^2}{\|\widehat {A}-A_0\|^2}$ were all in the interval $[1.009,1.036]$ . In all three plots, the solid line has slope 1 and the dashed line has slope $1.3$ . Code for these experiments is available on GitHub [Reference Shonkwiler54].

3.1 Gradient flow of restricted non-normal energy

The normal matrices in $\mathcal {U}_d$ are exactly the global minima of $\overline {\operatorname {E}}$ ; the goal is to show that almost every matrix in $\mathcal {U}_d$ flows to a normal matrix under the gradient flow:

Since $\overline {\operatorname {E}}$ is a polynomial function defined on a real-analytic submanifold of Euclidean space, it will have a Łojasiewicz exponent (cf. [Reference Bodmann and Haas7, Corollary 4.2]), and hence the gradient flow will have a single limit point [Reference Łojasiewicz41], proving the existence of $A_\infty $ .

Since the non-nilpotent matrices form an open, dense subset of $\mathcal {U}_d$ , Theorem 3.1 implies that almost every member of any neighborhood of a nonminimizing critical point will flow to a normal matrix; that is, a global minimum of $\overline {\operatorname {E}}$ . Hence, the nonminimizing critical points of $\overline {\operatorname {E}}$ cannot be basins of attraction. Since $\overline {\operatorname {E}}$ has a Łojasiewicz exponent, an argument analogous to [Reference Absil and Kurdyka1, Theorem 3] shows that all local minima must be basins of attraction. Hence we have the following corollary.

We have already shown that the gradient flow of $\overline {\operatorname {E}}$ converges to a single limit point $A_\infty $ . The remainder of this subsection will be devoted to proving the remaining statements of Theorem 3.1 through several supporting results. The strategy for proving the rest of the first sentence of Theorem 3.1 is to show that the gradient flow preserves non-nilpotency and that all nonminimizing critical points must be nilpotent. As with Theorem 2.6, the last sentence will follow because the real submanifold of $\mathcal {U}_d$ is invariant under the gradient flow.

Proposition 3.4. The intrinsic gradient of $\overline {\operatorname {E}}$ on $\mathcal {U}_d$ is

$$\begin{align*}\operatorname{grad} \overline{\operatorname{E}}(A) = -4([A,[A,A^\ast]] + \overline{\operatorname{E}}(A) A). \end{align*}$$

Proof. Geometrically, $\operatorname {grad} \overline {\operatorname {E}}(A)$ is the projection of the Euclidean gradient $\nabla \operatorname {E}(A)$ onto the tangent space $T_A \mathcal {U}_d = \mathrm {span}(\{A\})^\bot $ :

$$\begin{align*}\operatorname{grad} \overline{\operatorname{E}}(A) = \nabla \operatorname{E}(A) - \langle \nabla \operatorname{E}(A), A\rangle A. \end{align*}$$

We know from (2.3) that $\nabla \operatorname {E}(A) = -4[A,[A,A^\ast ]]$ , so the fact that $[A,A^\ast ]$ is Hermitian implies

(3.1) $$ \begin{align} \langle \nabla \operatorname{E}(A), A\rangle &= -4\mathrm{Re}\operatorname{tr}([A,[A,A^\ast]]^\ast A) = -4\mathrm{Re}\operatorname{tr}([A,A^\ast]A^\ast A - A^\ast [A,A^\ast]A) \nonumber\\ &= 4\mathrm{Re}\operatorname{tr}([A,A^\ast][A,A^\ast]) = 4\|[A,A^\ast]\|^2 = 4 \overline{\operatorname{E}}(A) \end{align} $$

by the linearity and cyclic invariance of trace, and the result follows.

Since $[A,A^\ast ]$ is traceless, notice that

$$\begin{align*}\operatorname{grad} \overline{\operatorname{E}}(A) = 4\left. \frac{d}{dt}\right|{}_{t=0} e^{-t \overline{\operatorname{E}}(A)} e^{t[A,A^\ast]} A e^{-t[A,A^\ast]} = 4\left. \frac{d}{dt}\right|{}_{t=0} (e^{t[A,A^\ast]},e^{-t \overline{\operatorname{E}}(A)}) \cdot A \end{align*}$$

is tangent to the $\operatorname {SL}_d(\mathbb {C}) \times \mathbb {C}^\times $ -orbit of A, where the action of $\operatorname {SL}_d(\mathbb {C}) \times \mathbb {C}^\times $ on $\mathbb {C}^{d \times d}$ is defined by .

We could use this to show that the negative gradient flow preserves non-nilpotency, but extending to the limit poses challenges, so we adopt a different approach. For $A \in \mathbb {C}^{d \times d}$ , define

where $\lambda _1, \dots , \lambda _d$ are the eigenvalues of A. The nilpotent matrices are precisely the vanishing locus of s.

Lemma 3.5. For any $A \in \mathcal {U}_d$ ,

$$\begin{align*}\langle -\operatorname{grad} \overline{\operatorname{E}}(A), \operatorname{grad} s(A) \rangle = 8 s(A) \overline{\operatorname{E}}(A), \end{align*}$$

where $\operatorname {grad} s(A)$ is the intrinsic gradient of s in $\mathcal {U}_d$ .

Proof. Note, first of all, that $\langle A, \operatorname {grad} s(A) \rangle = 0$ , since $\operatorname {grad} s(A) \in T_A \mathcal {U}_d = \mathrm {span}(\{A\})^\bot $ . Therefore,

$$ \begin{align*} \langle -\operatorname{grad} \overline{\operatorname{E}}(A), \operatorname{grad} s(A) \rangle & = \langle -\nabla \operatorname{E}(A) + 4\overline{\operatorname{E}}(A) A, \operatorname{grad} s(A) \rangle \\ & = \langle -\nabla \operatorname{E}(A), \operatorname{grad} s(A) \rangle \\ & = \langle -\nabla \operatorname{E}(A), \nabla s(A) - \langle \nabla s(A), A \rangle A\rangle \\ & = -\langle \nabla \operatorname{E}(A), \nabla s(A)\rangle + \langle \nabla s(A),A \rangle \langle \nabla \operatorname{E}(A), A \rangle \\ & = -\langle \nabla \operatorname{E}(A), \nabla s(A) \rangle + 4 \langle \nabla s(A), A \rangle \overline{\operatorname{E}}(A), \end{align*} $$

using (3.1) in the first and last equalities.

We know from (2.6) and the following sentence that $\nabla \operatorname {E}(A)$ lies in the conjugation orbit of A. But this means that $\nabla \operatorname {E}(A)$ must be tangent to the level set of s passing through A, since conjugation preserves eigenvalues, and hence fixes s. Therefore, $\langle \nabla \operatorname {E}(A), \nabla s(A) \rangle = 0$ and we have shown that

$$\begin{align*}\langle -\operatorname{grad} \overline{\operatorname{E}}(A), \operatorname{grad} s(A) \rangle = 4 \langle \nabla s(A), A \rangle \overline{\operatorname{E}}(A). \end{align*}$$

By definition of the gradient, the inner product is a directional derivative,

$$\begin{align*}\langle \nabla s(A), A \rangle = D s(A)(A) = \lim_{t \to 0} \frac{s(A + t A) - s(A)}{t} = \lim_{t \to 0} \frac{(1+t)^2 s(A) - s(A)}{t} = 2s(A), \end{align*}$$

completing the proof.

Proof. For any $A \in \mathcal {U}_d$ , Lemma 3.5 implies that

$$\begin{align*}\langle -\operatorname{grad} \overline{\operatorname{E}}(A), \operatorname{grad} s(A) \rangle = 8 s(A) \overline{\operatorname{E}}(A) \geq 0. \end{align*}$$

Therefore, $s(A)$ must be nondecreasing along the negative gradient flow lines of $\overline {\operatorname {E}}$ , so $s(A_t) \geq s(A_0)> 0$ for all $t \in [0, \infty )$ , and in the limit we also have $s(A_\infty ) \geq s(A_0)> 0$ . Hence, $A_t$ and $A_\infty $ must be non-nilpotent.

In other words, gradient flow preserves non-nilpotency, including in the limit, so we have completed the first step in our strategy for proving Theorem 3.1. We now proceed with the second step.

Proof. By Proposition 3.4, A is a critical point of $\overline {\operatorname {E}}$ if and only if

$$\begin{align*}0 = [A,[A,A^\ast]] + \overline{\operatorname{E}}(A)A. \end{align*}$$

If A is a nonminimizing critical point, then A is not normal, so $\overline {\operatorname {E}}(A) \neq 0$ and

$$\begin{align*}A = -\frac{1}{\overline{\operatorname{E}}(A)}[A,[A,A^\ast]]. \end{align*}$$

In other words, $A = [A,B]$ with $B = -\frac {1}{\overline {\operatorname {E}}(A)}[A,A^\ast ]$ . But then A certainly commutes with $[A,B]$ , so Jacobson’s Lemma (Lemma 2.4) implies that $[A,B]$ is nilpotent. Since $A=[A,B]$ , we conclude that A is nilpotent.

It is possible to prove an analogous statement to Proposition 2.11 in this setting as well, so gradient descent of $\overline {\operatorname {E}}$ , even though it preserves norms and (when applicable) realness, produces a limiting normal matrix $A_\infty $ which is not much further from $A_0$ than the closest normal matrix. Again, this conclusion is supported by numerical experiments: see Figure 4.

Figure 4

This is the same experimental setup as in Figure 3, except that now $A_\infty = \displaystyle \lim _{t \to \infty } \overline {\mathcal {F}}(A_0,t)$ . Left: $A_0 \in \mathbb {C}^{20 \times 20}$ ; all $\frac {\|A_\infty - A_0\|^2}{\|\widehat {A}-A_0\|^2} \in [1.060,1.198]$ . Center: $A_0 \in \mathbb {R}^{20 \times 20}$ ; all $\frac {\|A_\infty - A_0\|^2}{\|\widehat {A}-A_0\|^2} \in [1.046,1.253]$ . Right: $A_0 \in \mathbb {C}^{20 \times 20}$ is a small perturbation of a normal matrix; all $\frac {\|A_\infty - A_0\|^2}{\|\widehat {A}-A_0\|^2} \in [1.010,1.031]$ . In all three plots, the solid line has slope 1 and the dashed line has slope $1.3$ . Code for these experiments is available on GitHub [Reference Shonkwiler54].

3.2 Topology of unit norm normal matrices

The space of normal matrices $\mathcal {N}_d$ is a cone in $\mathbb {C}^{d \times d}$ and hence topologically trivial. However, the space $\mathcal {U}\mathcal {N}_d$ can potentially have interesting topology. Friedland [Reference Friedland21] argues that $\mathcal {U}\mathcal {N}_d$ is irreducible and the quasi-variety of its smooth points is connected. However, this is not quite enough to imply that $\mathcal {U}\mathcal {N}_d$ is connected, since irreducible real varieties can have connected components consisting entirely of nonsmooth points (see, e.g., [Reference Cahill, Mixon and Strawn9, Figure 2]). In this subsection, we show that $\mathcal {U}\mathcal {N}_d$ is connected and, in fact, that many of its low-dimensional homotopy groups vanish.

The key fact that we use when studying the topology of $\mathcal {U}\mathcal {N}_d$ is that it is closely related to the topology of the space of all non-nilpotent matrices in $\mathbb {C}^{d \times d}$ . For the rest of this subsection, we use $\mathcal {P}_d$ to denote the space of nilpotent matrices in $\mathbb {C}^{d \times d}$ and we let $\mathcal {M}_d = \mathbb {C}^{d \times d} \setminus \mathcal {P}_d$ . The relationship between the topologies of $\mathcal {U}\mathcal {N}_d$ and $\mathcal {M}_d$ is made precise by the following result.

In particular, $\mathcal {U}\mathcal {N}_d$ is homotopy equivalent to $\mathcal {M}_d$ , so our goal of characterizing the topology of the former space reduces to understanding that of the latter space. From such an understanding, we will deduce the main theorem of this subsection, stated below. In the following, we use $\pi _k(\mathcal {X},x_0)$ to denote the kth homotopy group of a space $\mathcal {X}$ with respect to a basepoint $x_0 \in \mathcal {X}$ , and write $\pi _k(\mathcal {X})$ in the case that $\mathcal {X}$ is path connected (in which case the result is independent of basepoint, up to isomorphism) – we refer the reader to [Reference Hatcher28, Chapter 4] for basic terminology and properties. We say that $\mathcal {X}$ is k-connected if $\pi _k(\mathcal {X},x_0)$ is the trivial group.

The proof will use two auxiliary topological results. The first follows from more general results on nilpotent cones, which are classical. We use [Reference Jantzen, Anker and Orsted35] as a general reference and explain how to deduce this particular result from the general results therein.

The following is a standard application of transversality (see [Reference Lee39, Chapter 6] and [Reference Hirsch32, Chapter 3]). Special cases of the result appear in, for example, [Reference Godbillon23, Theorem 2.3] and [Reference Ebert18, Theorem 1.1.4]. We give a proof sketch here for the sake of convenience.

Proof of Theorem 3.9.

By Corollary 3.8, it suffices to show that $\mathcal {M}_d$ is k-connected for all $2d-2$ . By a theorem of Whitney, the algebraic variety $\mathcal {P}_d$ can be expressed as a disjoint union of smooth manifolds [Reference Whitney59, Theorem 2], and, by Lemma 3.11, each of these has real codimension at least

$$\begin{align*}\mathrm{dim}(\mathbb{C}^{d \times d}) - \mathrm{dim}(\mathcal{P}_d) = 2d^2 - 2d(d-1) = 2d. \end{align*}$$

The theorem then follows from Lemma 3.12, since $\mathbb {C}^{d \times d}$ is k-connected for all k.

3.3 Topology of real unit norm normal matrices

Let $\mathcal {U}\mathcal {N}_d^{\mathbb {R}}$ denote the space of real, normal $d\times d$ matrices with Frobenius norm equal to one (so $\mathcal {UN}_{d}^{\mathbb {R}} \subset \mathcal {U}\mathcal {N}_d$ ). Adapting the arguments from the previous subsection, we will show the following.

The proof of the theorem follows the same general steps as that of Theorem 3.9. Let $\mathcal {P}_d^{\mathbb {R}}$ denote the $d \times d$ real nilpotent matrices, and let $\mathcal {M}_d^{\mathbb {R}} = \mathbb {R}^{d \times d} \setminus \mathcal {P}_d^{\mathbb {R}}$ denote the set of non-nilpotent matrices. By the same arguments used in the previous subsection, $\mathcal {M}_d^{\mathbb {R}}$ deformation retracts onto $\mathcal {UN}_d^{\mathbb {R}}$ , so it suffices to prove that $\mathcal {M}_d^{\mathbb {R}}$ is k-connected for all $k \leq d-2$ .

The main difference in the real case is that an analogue of Lemma 3.11 does not follow from general facts of nilpotent cones described in [Reference Jantzen, Anker and Orsted35], as the results therein are valid over algebraically closed fields. We obtain a decomposition of $\mathcal {P}_d^{\mathbb {R}}$ in analogy with the Whitney decomposition used in the proof of Theorem 3.9 from results of [Reference Heinzner, Schwarz and Stötzel29] and [Reference Böhm, Lafuente, Dearricott, Tuschmann, Nikolayevsky, Leistner and Crowley8].

Proof. It follows from a general theory of real reductive Lie group actions developed in [Reference Heinzner, Schwarz and Stötzel29] that $\mathbb {R}^{d \times d} \setminus \{0\}$ decomposes as a union of $\mathrm {GL}_d(\mathbb {R})$ -invariant (with respect to the conjugation action) smooth submanifolds $S_0 \cup S_1 \cup \cdots \cup S_k$ , where $S_0$ is exactly the open submanifold $\mathcal {M}_d^{\mathbb {R}}$ – see also [Reference Böhm, Lafuente, Dearricott, Tuschmann, Nikolayevsky, Leistner and Crowley8, Section 1] for a short exposition of these ideas. It is shown in [Reference Böhm, Lafuente, Dearricott, Tuschmann, Nikolayevsky, Leistner and Crowley8, Section 1.2] that (for the specific example of the conjugation action on $\mathbb {R}^{d \times d}$ ) the remaining submanifolds $S_i$ , $i> 0$ , are parameterized by Jordan canonical forms of nilpotent matrices. That is, fixing such a Jordan matrix J, we consider the corresponding set of nilpotent matrices as the homogeneous space $\mathrm {GL}_d(\mathbb {R})/\mathrm {stab}(J)$ , where $\mathrm {stab}(J)$ is the stabilizer of J under the conjugation action. To complete the proof, it suffices to show that the dimension of such a homogeneous space is at most $d^2-d$ , that is, to show that the stabilizer of any such Jordan matrix is at least of dimension d.

Let us now establish the claim made above. A nilpotent Jordan matrix J necessarily has all zeros on its diagonal, and is therefore characterized by the pattern of ones in the super diagonal (i.e., by the size of its Jordan blocks). An invertible real matrix $A = (a_{ij})_{i,j=1}^d$ lies in the stabilizer of J if and only if $AJ = JA$ . This matrix equation gives several constraints in the entries of A, and the number of independent constraints determines the dimension of the stabilizer.

In particular, since we aim to determine a lower bound on codimension, it suffices to consider the Jordan matrix which produces the largest number of constraints: that is, when J is the matrix whose superdiagonal consists of all ones (i.e., it has a single Jordan block). It is a standard fact (see, e.g., [Reference Gohberg, Lancaster and Rodman24, Theorem 9.1.1]) that, for this J, solutions of the equation $AJ = JA$ must be upper triangular Toeplitz matrices. In other words, elements of $\mathrm {stab}(J)$ are of the form

$$\begin{align*}\begin{bmatrix} a_1 & a_2 & a_3 & \cdots & a_{d-1} & a_d \\ 0 & a_1 & a_2 & \cdots & a_{d-2} & a_{d-1} \\ 0 & 0 & a_1 & \cdots & a_{d-3} & a_{d-2} \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & 0 & \cdots & a_1 & a_2 \\ 0 & 0 & 0 & \cdots & 0 & a_1 \end{bmatrix}. \end{align*}$$

Clearly, then, $\dim (\mathrm {stab}(J)) = d$ . This implies that the codimension of the associated submanifold is d. Since this is the submanifold of smallest codimension, this completes the proof.

4.1 Balancing matrices by gradient descent

As in the case of $\operatorname {E}$ , all critical points of $\operatorname {B}$ are global minima:

Proof. We first show that the gradient of the balanced energy is given by

(4.1) $$ \begin{align} \nabla \operatorname{B}(A) = -4[A,\operatorname{diag}([A,A^\ast])]. \end{align} $$

We write $\operatorname {B} = N \circ \mathrm {diag} \circ \mu $ , where $\mu $ is the momentum map (2.1), we consider the diagonalization operator as a linear map $\mathrm {diag}:\mathbb {C}^{d \times d} \to \mathbb {C}^{d \times d}$ , and N is the norm-squared map, as in the proof of Theorem 2.3. Following the logic of that proof, we then have that

$$\begin{align*}\nabla \operatorname{B}(A) = D\mu(A)^\vee \mathrm{diag}^\vee \nabla N (\mathrm{diag} \circ \mu(A)), \end{align*}$$

where the superscripts once again denote adjoints with respect to $\langle \cdot , \cdot \rangle $ . It is not hard to show that the map $\mathrm {diag}$ is self-adjoint and idempotent. Then

$$ \begin{align*} \nabla \operatorname{B}(A) &= D\mu(A)^\vee \mathrm{diag} (2 \cdot \mathrm{diag} \circ \mu(A)) = 2 D\mu(A)^\vee \mathrm{diag}(\mu(A)) \\ &= 2 [\mathrm{diag}(\mu(A)) + \mathrm{diag}(\mu(A))^\ast, A] = -4 [A, \mathrm{diag}([A,A^\ast])]. \end{align*} $$

The above shows that we have a critical point of $\operatorname {B}$ exactly when

$$\begin{align*}0 = [A,\operatorname{diag}([A,A^\ast])]. \end{align*}$$

Since the entries of $[A,\operatorname {diag}([A,A^\ast ])]$ are of the form

(4.2) $$ \begin{align} a_{ij}\left((\|A_i\|^2 - \|A^i\|^2) - (\|A_j\|^2 - \|A^j\|^2)\right), \end{align} $$

this means that $\|A_i\|^2 - \|A^i\|^2 = \|A_j\|^2 - \|A^j\|^2$ for all i and j such that $a_{ij} \neq 0$ .

In other words, A is a critical point of $\operatorname {B}$ if and only if $\|A_i\|^2 - \|A^i\|^2$ is independent of i. However, since

$$\begin{align*}\sum_{i=1}^d \left(\|A_i\|^2 - \|A^i\|^2\right) = \sum_{i=1}^d\|A_i\|^2 - \sum_{i=1}^d\|A^i\|^2 = \|A\|^2 - \|A\|^2 = 0, \end{align*}$$

this can only happen if all $\|A_i\|^2 - \|A^i\|^2 = 0$ ; that is, if A is balanced.

Remark 4.2. Theorem 4.1 shows that $\operatorname {B}$ is an invex function, but it is not quasiconvex. To see this, consider the matrices

$$\begin{align*}A_0 = \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{bmatrix} \qquad \text{and} \qquad A_1 = \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{bmatrix}, \end{align*}$$

which are the (entrywise square roots of the) adjacency matrices of the balanced graphs

respectively. Then $\operatorname {B}(A_0) = 0 = \operatorname {B}(A_1)$ , but $\operatorname {B}\left ((1-t)A_0 + t A_1\right ) = 8t^2(1-t)^2> 0$ for all $0 < t < 1$ .

As in the case of $\operatorname {E}$ , we can find global minima of $\operatorname {B}$ by gradient descent. Specifically, let $\mathscr {F}: \mathbb {C}^{d \times d} \times [0,\infty ) \to \mathbb {C}^{d \times d}$ be the negative gradient flow of $\operatorname {B}$ :

$$\begin{align*}\mathscr{F}(A_0,0) = A_0 \qquad \frac{d}{dt} \mathscr{F}(A_0, t) = -\nabla \operatorname{B}(\mathscr{F}(A_0,t)). \end{align*}$$

Since $\operatorname {B}$ is a real polynomial function on $\mathbb {C}^{d \times d}$ , Theorem 4.1 implies that $\lim _{t \to \infty } \mathscr {F}(A_0,t)$ is always well-defined and normal. Since the real matrices stay real under gradient flow, this limit will be real whenever $A_0$ is.

Moreover,

(4.3) $$ \begin{align} \nabla \operatorname{B}(A) = -4[A,\operatorname{diag}([A,A^\ast])] = 4\left(\left. \frac{d}{d\epsilon}\right|{}_{\epsilon = 0} e^{\epsilon \operatorname{diag}([A,A^\ast])} \cdot A\right) \end{align} $$

is tangent to the orbit of the diagonal subgroup $\operatorname {DSL}_d(\mathbb {C}) \leq \operatorname {SL}_d(\mathbb {C})$ acting by conjugation on $\mathbb {C}^{d \times d}$ . In particular, flowing $A_0$ by the gradient flow of $\operatorname {B}$ preserves not just the eigenvalues of $A_0$ , but also all principal minors of $A_0$ , including the diagonal entries of $A_0$ .

From the expression (4.2) for the entries of $-\frac {1}{4}\nabla \operatorname {B}(A)$ we see that, if there is $t_0 \geq 0$ so that the $(i,j)$ entry in $\mathscr {F}(A_0,t_0)$ vanishes, then the $(i,j)$ entry of $\mathscr {F}(A_0, t)$ will vanish for all $t \geq t_0$ . In graph terms, the gradient flow of $\operatorname {B}$ cannot sprout new edges in the graph. This also means that if $A_0$ is real, its entries cannot change sign under gradient descent of $\operatorname {B}$ . Thus, we have proved:

When we take $A_0$ to be the entrywise square root of the adjacency matrix of some weighted, directed graph $\mathcal {G}_0 = (\mathcal {V}_0, \mathcal {E}_0,w_0)$ , then we can sensibly interpret $A_\infty = \displaystyle \lim _{t \to \infty } \mathscr {F}(A_0,t)$ as the entrywise square root of the adjacency matrix of some balanced, weighted, directed graph $\mathcal {G}_\infty = (\mathcal {V}_\infty , \mathcal {E}_\infty ,w_\infty )$ with $\mathcal {V}_\infty = \mathcal {V}_0$ and $\mathcal {E}_\infty \subseteq \mathcal {E}_0$ . In other words, gradient descent of $\operatorname {B}$ balances $\mathcal {G}_0$ without introducing any new edges.

In the case of gradient descent of $\operatorname {E}$ , we saw that all nilpotent matrices flowed to the zero matrix. We see the same phenomenon here: if $\mathcal {G}_0$ is a weighted, directed, acyclic graph (DAG), then its adjacency matrix is nilpotent, as is the entrywise square root $A_0$ . The gradient flow $\mathscr {F}(A_0,t)$ will limit to the zero matrix, which makes sense: the only way to balance a weighted DAG is by driving all the weights to zero.

4.2 Preserving weights

Weighted DAGs provide an extreme example of the general phenomenon that gradient descent of $\operatorname {B}$ decreases the Frobenius norm. In graph terms, if $A_0$ is the entrywise square root of the adjacency matrix of a weighted, directed graph $\mathcal {G}_0$ , then the squared Frobenius norm

$$\begin{align*}\|A_0\|^2 = \sum_{i,j} |a_{ij}|^2 = \sum_{i,j} a_{ij}^2 \end{align*}$$

is precisely the sum of the weights in $\mathcal {G}_0$ . If the weights correspond to, for example, mass traversing between nodes in a network, then it may not make sense to balance the flows in the network by reducing the total mass in the system.

In order to preserve the sum of weights on $\mathcal {G}_0$ , we consider $\overline {\operatorname {B}}: \mathcal {U}_d \to \mathbb {R}$ , the restriction of $\operatorname {B}$ to $\mathcal {U}_d$ , and its gradient descent $\overline {\mathscr {F}}: \mathcal {U}_d \times [0, \infty ) \to \mathcal {U}_d$ given by

$$\begin{align*}\overline{\mathscr{F}}(A_0,0) = A_0 \qquad \frac{d}{dt} \overline{\mathscr{F}}(A_0,t) = -\operatorname{grad} \overline{\operatorname{B}}(\overline{\mathscr{F}}(A_0,t)). \end{align*}$$

In graph terms, if $A_0$ is the entrywise square root of an adjacency matrix for $\mathcal {G}_0$ with total weight 1, then $A_\infty $ is the entrywise square root of the adjacency matrix for a balanced graph $\mathcal {G}_\infty $ with total weight 1 whose vertices are the same as the vertices of $\mathcal {G}_0$ and whose edges are a subset of the edges of $\mathcal {G}_0$ . That is, gradient descent of $\overline {\operatorname {B}}$ balances $\mathcal {G}_0$ without introducing any new edges and without losing any overall weight.

The strategy for proving Theorem 4.5 is the same as for Theorem 3.1. The existence of a unique limit point $A_\infty $ follows from the fact that $\overline {\operatorname {B}}$ is a polynomial function on $\mathcal {U}_d$ , and hence has a Łojasiewicz exponent. The bulk of the argument is in showing that the gradient flow preserves non-nilpotency and that the nonminimizing critical points are nilpotent. The rest of the theorem will follow from the structure of $\operatorname {grad} \overline {\operatorname {B}}$ and the fact that the real submanifold of $\mathcal {U}_d$ is invariant under gradient flow.

First, we compute the intrinsic gradient of $\overline {\operatorname {B}}$ , which follows the same pattern as $\operatorname {grad} \overline {\operatorname {E}}$ :

Proposition 4.6. The intrinsic gradient of $\overline {\operatorname {B}}$ on $\mathcal {U}_d$ is

$$\begin{align*}\operatorname{grad} \overline{\operatorname{B}}(A) = -4([A,\operatorname{diag}([A,A^\ast])]+\overline{\operatorname{B}}(A)A). \end{align*}$$

Proof. We know that

$$\begin{align*}\operatorname{grad} \overline{\operatorname{B}}(A) = \nabla \operatorname{B}(A) - \langle \nabla \operatorname{B}(A),A\rangle A, \end{align*}$$

so the key is to use (4.1) and the fact that the diagonal of $[A,A^\ast ]$ is real to compute

$$ \begin{align*} \langle \nabla \operatorname{B}(A),A\rangle & = -4\mathrm{Re}\operatorname{tr}([A,\operatorname{diag}([A,A^\ast])]^\ast A) \\ & = -4\mathrm{Re}\operatorname{tr}(\operatorname{diag}([A,A^\ast])A^\ast A - A^\ast \operatorname{diag}([A,A^\ast])A) \\ & = 4\mathrm{Re}\operatorname{tr}(\operatorname{diag}([A,A^\ast])[A,A^\ast]) \\ & = 4\mathrm{Re}\operatorname{tr}(\operatorname{diag}([A,A^\ast])\operatorname{diag}([A,A^\ast])) \\ & = 4\|\operatorname{diag}([A,A^\ast])\|^2 \\ & = 4\overline{\operatorname{B}}(A) \end{align*} $$

using the linearity and cyclic invariance of trace.

Each entry of $\operatorname {grad} \overline {\operatorname {B}}(A)$ is a scalar multiple of the corresponding entry of A, so the fact that the negative gradient flow $\overline {\mathscr {F}}$ preserves zero entries and cannot change the sign of real entries follows immediately.

Next, we prove an analog of Lemma 3.5. Recall that $s(A) = \sum |\lambda _i|^2$ is the sum of the squares of the absolute values of the eigenvalues of A.

Lemma 4.7. For any $A \in \mathcal {U}_d$ ,

$$\begin{align*}\langle -\operatorname{grad} \overline{\operatorname{B}}(A),\operatorname{grad} s(A)\rangle = 8s(A)\overline{\operatorname{B}}(A). \end{align*}$$

Since $\langle -\operatorname {grad} \overline {\operatorname {B}}(A), \operatorname {grad} s(A) \rangle = 8s(A)\overline {\operatorname {B}}(A) \geq 0$ , $s(A)$ must be nondecreasing along the negative gradient flow lines of $\overline {\operatorname {B}}$ , so we have proved:

We know the balanced matrices are exactly the global minima of $\overline {\operatorname {B}}$ . Proposition 4.6 implies that A is a critical point of $\overline {\operatorname {B}}$ if and only if

$$\begin{align*}0 = [A,\operatorname{diag}([A,A^\ast])] + \overline{\operatorname{B}}(A)A. \end{align*}$$

When A is a non-minimizing critical point, $\overline {\operatorname {B}}(A) \neq 0$ and the same Jacobson’s Lemma argument as in Proposition 3.7 shows that A is nilpotent, proving:

This completes the proof of Theorem 4.5.

Figure 1 shows an application of this approach to balancing graphs, and Figure 6 shows a much larger example. In both cases, up to an overall normalization to ensure $\|A_0\| = 1$ , the non-zero entries in the starting matrix $A_0$ were populated by the absolute values of standard Gaussians.

Figure 6

Balancing a larger graph by the flow $\overline {\mathscr {F}}$ , with $A_0$ on the left and $A_\infty = \displaystyle \lim _{t \to \infty } \overline {\mathscr {F}}(A_0,t)$ on the right. The thickness of each edge is proportional to its weight. The underlying graph is a random planar graph with 100 vertices and 284 edges, constructed as the 1-skeleton of the Delaunay triangulation of 100 random points in the square; to make the visualization more comprehensible, the graph that is shown is a spring embedding, so the vertices are not at the locations of the original random points in the square.

4.3 Topology of unit norm balanced graphs

Let $\mathcal {UB}_d$ denote the space of balanced $d \times d$ matrices of unit Frobenius norm, and let $\mathcal {UB}_d^{\mathbb {R}}$ denote the subspace of balanced matrices with real entries. The topology of these spaces is tied to the topology of the relevant spaces of normal matrices, as we record in the following theorem.