Logarithmic norm

In mathematics, the logarithmic norm is a real-valued functional on operators, constructed from either a vector norm or an inner product, or directly from the induced operator norm. It quantifies key notions such as positive/negative definiteness in matrix theory, uniformly coercive or monotone vector fields in nonlinear analysis, and strong ellipticity in differential operators on function spaces, subject to specific boundary conditions.

The logarithmic norm has a wide range of applications in matrix theory, stability theory for initial and boundary value problems in differential equations, in nonlinear analysis, applied mathematics, and numerical analysis.

Comprehensive treatments of the theory and its applications can be found in recent monographs,¹² on which this article is based.

The logarithmic norm was introduced in 1958, independently by Germund Dahlquist³ and Sergei Lozinskiĭ⁴ for square matrices and bounded linear operators. The original purpose was to estimate solutions to linear differential equations ${\displaystyle {\dot {x}}=Ax+r}$, to construct sufficient conditions for stability, and to obtain norm bounds of perturbations due to the forcing function ${\displaystyle r}$.

Let ${\displaystyle A}$ be a square matrix and ${\displaystyle \|\cdot \|}$ be the operator norm induced by a given vector norm. The associated logarithmic norm ${\displaystyle \mu [A]}$ is defined by

${\displaystyle \mu [A]=\lim \limits _{h\rightarrow 0^{+}}{\frac {\|I+hA\|-1}{h}}\,,}$

where ${\displaystyle h}$ is real and ${\displaystyle I}$ is the identity matrix of the same dimension as ${\displaystyle A}$. The limit exists on account of the convexity of the matrix norm.

The term logarithmic norm is due to Lozinskiĭ, and refers to the fact that if ${\displaystyle {\dot {x}}=Ax}$ then

${\displaystyle {\mathrm {D} }_{t}^{+}\log \|x\|\,\leq \,\mu [A]\,,}$

where ${\displaystyle {\mathrm {D} }_{t}^{+}}$ denotes the upper right Dini derivative with respect to time. In other words, ${\displaystyle \mu [A]}$ is an upper bound for the (short-term) growth rate of the "logarithmic norm" of ${\displaystyle x(t)}$. Using logarithmic differentiation, this bound can also be written as the differential inequality

${\displaystyle {\mathrm {D} }_{t}^{+}\|x\|\,\leq \,\mu [A]\cdot \|x\|\,,}$

from which it follows that, for ${\displaystyle t\geq 0}$,

${\displaystyle \|{\mathrm {e} }^{tA}\|\,\leq \,{\mathrm {e} }^{t\mu [A]}\,.}$

This bound is always preferable to the Lipschitz bound ${\displaystyle {\mathrm {e} }^{t\|A\|}}$, since ${\displaystyle \mu [A]\leq \|A\|}$. Moreover, while ${\displaystyle \|A\|\geq 0}$, the logarithmic norm may take negative values, e.g. when ${\displaystyle A}$ is negative definite and the Euclidean norm is used. Thus ${\displaystyle \mu [A]\leq 0}$ implies that ${\displaystyle {\mathrm {e} }^{tA}}$ is a contraction, and hence the solutions ${\displaystyle x(t)}$ are stable in forward time.

The ability of the logarithmic norm to distinguish forward and reverse time evolution in dynamical systems is a key property. It may also establish whether a problem is well-posed. This asymmetry is also of importance in many applications. For example, in control theory, there is a fundamental need to discriminate between positive and negative feedback; this too can be characterized by the logarithmic norm.

Depending on the setting, the logarithmic norm is related to Lyapunov stability theory.⁵ Other bounds are similar to those obtained using Grönwall's lemma. Thus the state transition matrix ${\displaystyle \Phi (t,t_{0})}$ associated with a nonautonomous linear differential equation ${\displaystyle {\dot {x}}=A(t)x}$ satisfies the upper and lower bounds⁶⁷ for ${\displaystyle t\geq t_{0}}$,

${\displaystyle \exp \left(-\int _{t_{0}}^{t}\mu [-A(s)]\,{\mathrm {d} }s\right)\,\leq \,\|\Phi (t,t_{0})\|\,\leq \,\exp \left(\int _{t_{0}}^{t}\mu [A(s)]\,{\mathrm {d} }s\right).}$

More recently, the logarithmic norm has been extended and generalized to nonlinear maps⁸ as well as to unbounded operators,⁹ covering boundary value problems and elementary applications in elliptic and parabolic partial differential equations.¹

The logarithmic norm may in some literature appear under less informative names, such as the "matrix measure" (for matrices)¹⁰ and "one-sided Lipschitz constant" (for nonlinear maps),¹¹ although the concept neither relates to measure theory nor implies Lipschitz continuity.

For clarity and generality, the modern notation defines a greatest lower bound (or simply lower) logarithmic norm ${\displaystyle m[A]}$ as well as the classical least upper bound (upper) logarithmic norm by the two left and right limits

${\displaystyle m[A]\,=\,\lim _{h\rightarrow 0^{-}}{\frac {\|I+hA\|-1}{h}}\,;\qquad M[A]\,=\,\lim _{h\rightarrow 0^{+}}{\frac {\|I+hA\|-1}{h}}\,.}$

Here ${\displaystyle m[A]\leq M[A]\equiv \mu [A]}$ for all ${\displaystyle A}$, and the lower and upper logarithmic norms are related by ${\displaystyle m[A]\equiv -M[-A]}$.

Alternatively, if an inner product norm is used, the lower and upper logarithmic norms are defined as the best possible bounds such that for all ${\displaystyle x}$,

${\displaystyle m_{2}[A]\cdot \|x\|_{2}^{2}\,\leq \,{\mathrm {Re} }\,\langle x,Ax\rangle \,\leq \,M_{2}[A]\cdot \|x\|_{2}^{2}\,,}$

where ${\displaystyle \|x\|_{2}^{2}=\langle x,x\rangle }$ denotes the corresponding vector norm. In case of bounded linear operators in a Hilbert space setting, the inner product definition is equivalent to the original definition. But as the inner product definition also admits certain classes of unbounded operators it is more general. In the latter case one of the logarithmic norms may be infinite.

Further variants of use in semigroup theory are based on the limit

${\displaystyle M[A]\,=\,\lim _{h\rightarrow 0^{+}}{\frac {\|{\mathrm {e} }^{hA}\|-1}{h}}\,,}$

which is still equivalent to the classical definition if ${\displaystyle A}$ is a bounded linear operator. In case ${\displaystyle {\mathrm {e} }^{tA}}$ is a strongly continuous semigroup for ${\displaystyle t\geq 0}$, various modifications of this alternative are necessary, since the "vector field" ${\displaystyle A:x\mapsto y=Ax}$ is defined by the infinitesimal generator of the semigroup, i.e.,

${\displaystyle Ax\,=\,\lim _{h\rightarrow 0^{+}}{\frac {{\big (}{\mathrm {e} }^{hA}-I{\big )}x}{h}}\,.}$

The structure of the limit using the matrix exponential is shared by many other pairs of functionals, linking the vector field and the flow of a linear differential equation in a natural manner. For example, it holds that

${\displaystyle \alpha [A]=\lim _{h\rightarrow 0^{+}}{\frac {\rho [{\mathrm {e} }^{hA}]-1}{h}}\,;\qquad {\mathrm {trace} }[A]=\lim _{h\rightarrow 0}{\frac {\det[{\mathrm {e} }^{hA}]-1}{h}}\,,}$

where ${\displaystyle \rho [A]=\max _{k}|\lambda _{k}|}$ and ${\displaystyle \alpha [A]=\max _{k}{\mathrm {Re} }\,\lambda _{k}}$ are the (upper) spectral radius and (upper) spectral abscissa, respectively. Both limits exist, although neither the spectral radius nor the determinant is a convex functional. Moreover, for the trace, the left and right limits coincide. Moreover,

${\displaystyle {\mathrm {D} }_{t}\det \!{\big [}{\mathrm {e} }^{tA}{\big ]}\,=\,{\mathrm {trace} }[A]\cdot \det \!{\big [}{\mathrm {e} }^{tA}{\big ]}.}$

In spite of the similarities, there are major differences. Thus ${\displaystyle {\mathrm {D} }_{t}^{+}\|{\mathrm {e} }^{tA}\|\,\leq \,M[A]\cdot \|{\mathrm {e} }^{tA}\|\,,}$ where the important properties, enabling norm bounds, are that the operator norm is submultiplicative, and the logarithmic norm is subadditive. These properties do not hold for the spectral radius and abscissa, which are less powerful tools in the stability analysis of dynamical systems.

For every choice of norm, the basic properties (including some redundancy) of the logarithmic norm of a matrix are the following:¹

1. ${\displaystyle M[0]=0,\quad M[I]=1}$
2. ${\displaystyle M[-A]=-m[A]}$
3. Boundedness ${\displaystyle \,-\|A\|\,\leq \,m[A]\,\leq \,M[A]\,\leq \,\|A\|}$
4. Lower and upper bounds ${\displaystyle \,-{\mathrm {glb} }[A]\,\leq \,M[A]\,\leq \,\|A\|}$
5. Lower and upper bounds ${\displaystyle \,-\|A\|\,\leq \,m[A]\,\leq \,{\mathrm {glb} }[A]}$
6. Continuity ${\displaystyle \,{\big |}M[A]-M[B]{\big |}\,\leq \,\|A-B\|}$
7. Real translation ${\displaystyle \,M[A-\lambda I]=M[A]-{\mathrm {Re} }\,\lambda ,\qquad \lambda \in \mathbb {C} }$
8. Weak homogeneity ${\displaystyle \,M[\gamma \,A]=\gamma \cdot M[A],\qquad \gamma \in \mathbb {R} ^{+}}$
9. Sublinearity ${\displaystyle \,M[A+B]\,\leq \,M[A]+M[B]}$
10. Superlinearity ${\displaystyle \,m[A]+m[B]\,\leq \,m[A+B]}$
11. Convexity ${\displaystyle \,M[\theta A+(1-\theta )B]\,\leq \,\theta \,M[A]+(1-\theta )\,M[B],\qquad \theta \in [0,1]}$
12. Concavity ${\displaystyle \,\theta \,m[A]+(1-\theta )\,m[B]\,\leq \,m[\theta A+(1-\theta )B],\qquad \theta \in [0,1]}$
13. Lower and upper exponential bounds ${\displaystyle \,{\mathrm {e} }^{tm[A]}\,\leq \,\|{\mathrm {e} }^{tA}\|\,\leq \,{\mathrm {e} }^{tM[A]},\qquad t\geq 0}$
14. Positive definite inverse bound ${\displaystyle \,m[A]>0\quad \Rightarrow \quad \|A^{-1}\|\leq 1/m[A]}$
15. Negative definite inverse bound ${\displaystyle \,M[A]<0\quad \Rightarrow \quad \|A^{-1}\|\leq -1/M[A]}$
16. Spectral bound ${\displaystyle \,\alpha [A]\leq M[A].}$

As illustrated by properties 14 - 15, the condition ${\displaystyle m[A]>0}$ (${\displaystyle M[A]<0}$) generalizes positive (negative) definiteness to arbitrary norms and non-symmetric matrices, while also quantifying these notions. An indefinite matrix with respect to a given norm satisfies ${\displaystyle m[A]\leq 0\leq M[A]}$.

For the most common ${\displaystyle l^{p}}$ norms, the logarithmic norm of a (complex) matrix ${\displaystyle A}$ can be computed as follows, where ${\displaystyle a_{ij}}$ represents the matrix element in the ${\displaystyle i^{\mathrm {th} }}$ row and ${\displaystyle j^{\mathrm {th} }}$ column, and where ${\displaystyle A^{*}}$ denotes the complex conjugate of ${\displaystyle A}$ :

• ${\displaystyle M_{1}[A]\,=\,\max _{j}\left({\mathrm {Re} }\,a_{jj}+\sum _{i\neq j}|a_{ij}|\right)}$

• ${\displaystyle M_{2}[A]\,=\,\alpha \left[{\frac {A+A^{*}}{2}}\right]}$

• ${\displaystyle M_{\infty }[A]\,=\,\max _{i}\left({\mathrm {Re} }\,a_{ii}+\sum _{j\neq i}|a_{ij}|\right)}$

Let ${\displaystyle A}$ be a (possibly complex) ${\displaystyle n\times n}$ matrix. The symmetric positive definite square root of ${\displaystyle A^{*}A}$ and the Hermitian part of ${\displaystyle A}$ are denoted by

${\displaystyle |A|\,=\,{\sqrt {A^{*}A}}\qquad {\mbox{and}}\qquad {\mathrm {He} }(A)\,=\,{\frac {A+A^{*}}{2}}\,,}$

respectively. Their eigenvalue problems are ${\displaystyle |A|u=\sigma u}$ and ${\displaystyle {\mathrm {He} }(A)v=\mu v}$, where ${\displaystyle \sigma _{k}}$ are the ${\displaystyle n}$ singular values of ${\displaystyle A}$, and ${\displaystyle \mu _{k}}$ are the ${\displaystyle n}$ logarithmic values of ${\displaystyle A}$. The singular and logarithmic values are the ${\displaystyle n}$ stationary points of the two quadratic forms

${\displaystyle {\frac {u^{*}|A|u}{u^{*}u}}\qquad {\mbox{and}}\qquad {\frac {{\mathrm {Re} }\,v^{*}Av}{v^{*}v}}\,\equiv \,{\frac {v^{*}{\mathrm {He} }(A)v}{v^{*}v}}\,,}$

respectively. As for the relations to the ${\displaystyle n}$ eigenvalues ${\displaystyle \lambda _{k}}$ of ${\displaystyle A}$, it holds that

${\displaystyle {\mathrm {glb} }_{2}[A]\,=\,\min \sigma _{k}\,\leq \,\min |\lambda _{k}|\,\leq \,\max |\lambda _{k}|\,\leq \,\max \sigma _{k}\,=\,\|A\|_{2}}$

and

${\displaystyle m_{2}[A]\,=\,\min \mu _{k}\,\leq \,\min {\mathrm {Re} }\,\lambda _{k}\,\leq \,\max {\mathrm {Re} }\,\lambda _{k}\,\leq \,\max \mu _{k}\,=\,M_{2}[A]}$

together with

${\displaystyle |\det A|\,=\,\prod _{k}|\lambda _{k}|\,=\,\prod _{k}\sigma _{k}\,=\,\det |A|}$

and

${\displaystyle {\mathrm {Re\,trace} }[A]\,=\,\sum _{k}{\mathrm {Re} }\,\lambda _{k}\,=\,\sum _{k}\mu _{k}\,=\,{\mathrm {trace} }\left[{\mathrm {He} }(A)\right].}$

As the quadratic form ${\displaystyle x^{*}Ax}$ is generally complex valued, one can only find extrema of some related real function. For the logarithmic norms one takes the real part. Another option is to take the absolute value, to obtain the numerical radius of ${\displaystyle A}$, defined by

${\displaystyle \nu [A]\,=\,\sup _{x\neq 0}{\frac {|x^{*}Ax|}{x^{*}x}}\,.}$

Because ${\displaystyle {\mathrm {Re} }\,z\leq |z|}$, it follows that for all ${\displaystyle A}$,

${\displaystyle M_{2}[A]\leq \nu [A]\leq \|A\|_{2}.}$

The numerical radius is a matrix norm (but not an operator norm). However, as it is not defined by a quadratic form, the determination of ${\displaystyle \nu [A]}$ is more intricate. But to each matrix norm there is a logarithmic norm, and

${\displaystyle \lim _{h\rightarrow 0^{-}}{\frac {\nu [I+hA]-1}{h}}\,=\,m_{2}[A]\,;\qquad \lim _{h\rightarrow 0^{+}}{\frac {\nu [I+hA]-1}{h}}\,=\,M_{2}[A]\,,}$

i.e., the lower and upper logarithmic norms of the numerical radius coincide with the Euclidean logarithmic norms.¹ By the Hausdorff-Toeplitz theorem, the numerical range of a matrix ${\displaystyle A}$ is the set

${\displaystyle W(A)=\bigcap _{\varphi \in [0,2\pi ]}C(A;\varphi )}$

in the complex plane, given by the intersection of the half-planes

${\displaystyle C(A;\varphi )\,=\,{\big \lbrace }z\in {\mathbb {C} }\,:\,{\mathrm {Re} }\,{\mathrm {e} }^{-{\mathrm {i} }\varphi }z\leq M_{2}[{\mathrm {e} }^{-{\mathrm {i} }\varphi }A]{\big \rbrace }\,,}$

and the numerical radius is

${\displaystyle \nu [A]=\sup _{z\in W(A)}|z|=\max _{\varphi \in [0,2\pi ]}M_{2}[{\mathrm {e} }^{-{\mathrm {i} }\varphi }A]\,.}$

For matrix functions, the use of the von Neumann inequality and spectral sets provides the following classical result, using the Euclidean norm: let a polynomial ${\displaystyle P}$ be a such that ${\displaystyle |z|\leq 1\Rightarrow |P(z)|\leq 1}$. Then ${\displaystyle \|A\|_{2}\leq 1\Rightarrow \|P(A)\|_{2}\leq 1}$.

Using the logarithmic norm, this theorem can be generalized from unit circles to half-planes. Thus, if ${\displaystyle R}$ is a rational function such that ${\displaystyle {\mathrm {Re} }\,z\leq 0\Rightarrow |R(z)|\leq 1\,,}$ then ${\displaystyle M_{2}[A]\leq 0\Rightarrow \|R(A)\|_{2}\leq 1\,.}$

This property is of importance in numerical analysis, where implicit Runge-Kutta methods approximate a differential equation ${\displaystyle {\dot {x}}=Ax}$ by a recursion ${\displaystyle x_{k+1}=R(hA)x_{k}}$, where ${\displaystyle R}$ is a rational function, referred to as the method's stability function. The continuous system is stable with a contractive flow if ${\displaystyle M_{2}[A]\leq 0}$, while the discrete system is stable (a discrete contraction) if ${\displaystyle \|R(hA)\|_{2}\leq 1}$. Thus, if the stability function ${\displaystyle R}$ satisfies ${\displaystyle {\mathrm {Re} }\,z\leq 0\Rightarrow |R(z)|\leq 1}$, a condition known as A-stability, then the Runge-Kutta method produces stable solutions for all negative semidefinite vector fields ${\displaystyle A}$.

In a similar manner the theory of logarithmic norms can also be extended to unitarily invariant norms, e.g. the Hilbert-Schmidt norm for trace class operators, the Frobenius norm, and more generally the Schatten norms. As logarithmic values are only invariant under unitary similarity transformations, the logarithmic norms are defined subordinate to the matrix norms.

For real-valued nonlinear maps there are various possibilities to define operator norms. The least upper bound Lipschitz constant is an operator seminorm, which is all that is required. A corresponding lower Lipschitz constant can be included to define

${\displaystyle l[f]\cdot \|u-v\|\,\leq \,\|f(u)-f(v)\|\,\leq \,L[f]\cdot \|u-v\|}$

on a simply connected domain ${\displaystyle D}$. The corresponding lower and upper logarithmic Lipschitz constants are defined by

${\displaystyle m[f]=\lim _{h\rightarrow 0^{-}}{\frac {L[I+hf]-1}{h}}\,;\quad M[f]=\lim _{h\rightarrow 0^{+}}{\frac {L[I+hf]-1}{h}}\,,}$

where both limits exist if ${\displaystyle f}$ is Lipschitz continuous. For inner product norms, one can instead use the special definitions

${\displaystyle m_{2}[f]\cdot \|u-v\|_{2}^{2}\,\leq \,\langle u-v,f(u)-f(v)\rangle \,\leq \,M_{2}[f]\cdot \|u-v\|_{2}^{2}\,,}$

where one or both of the logarithmic Lipschitz constants may exist even when ${\displaystyle f}$ is not Lipschitz continuous. Although it is difficult to compute actual values of Lipschitz and logarithmic Lipschitz constants in practice, they retain their theoretical importance as useful tools in nonlinear analysis.¹²

All properties listed above for linear operators are retained, provided that operations like ${\displaystyle f+g}$ etc. are allowed on a joint domain of definition. In the special case when ${\displaystyle f}$ is Fréchet differentiable on a convex domain ${\displaystyle D}$, it further holds that

${\displaystyle L[f]=\sup _{x\in D}\|f'(x)\|\qquad {\mbox{and}}\qquad M[f]=\sup _{x\in D}M[f'(x)]\,,}$

where ${\displaystyle f'(x)}$ is the Jacobian matrix of ${\displaystyle f}$.

An operator having either ${\displaystyle m[f]>0}$ or ${\displaystyle M[f]<0}$ is called uniformly monotone, and the uniform monotonicity theorem states that

${\displaystyle m[f]>0\quad \Rightarrow \quad L[f^{-1}]\leq {\frac {1}{m[f]}}\,,}$

generalizing Properties 14 - 15 above, and quantifying the Uniform monotonicity theorem due to Browder and Minty (1963). Here, the Lipschitz constant of the inverse is defined on the image of ${\displaystyle f}$, while its lower logarithmic Lipschitz constant is defined on the domain of ${\displaystyle f}$.

Because composition is not distributive over addition for nonlinear maps, the von Neumann type bounds for 'rational functions of operators' have no direct counterparts in the nonlinear case. Nevertheless some results carry over. For example, in a fixed point iteration ${\displaystyle x_{k+1}=g(x_{k})}$ converging to a fixed point ${\displaystyle x_{\infty }}$, it can be shown that the error is bounded by

${\displaystyle \|x_{k+1}-x_{\infty }\|\,\leq \,L{\big [}g\circ (I-g)^{-1}{\big ]}\cdot \|x_{k+1}-x_{k}\|\,,}$

where the difference between the two most recent iterates is directly computable, and where any one of the three bounds

${\displaystyle L{\big [}g\circ (I-g)^{-1}{\big ]}\,\leq \,{\frac {L[g]}{1-M[g]}}\,\leq \,{\frac {L[g]}{1-L[g]}}\,,}$

can be used to obtain an error estimate. In order to guarantee the simple estimate ${\displaystyle \|x_{k+1}-x_{\infty }\|\,\leq \,\|x_{k+1}-x_{k}\|}$ using the third expression, one needs ${\displaystyle L[g]\leq 1/2}$ to ensure that the estimate is satisfied. If instead the central option is used, it is sufficient that ${\displaystyle L[g]<1}$ and ${\displaystyle M[g]\leq 0}$. Using the first (best) expression directly, a sharper result holds, but only for inner product norms, viz.,

${\displaystyle M_{2}[g]\leq {\frac {1}{2}}\quad \Rightarrow \quad L_{2}{\big [}g\circ (I-g)^{-1}{\big ]}\leq 1\,,}$

without further restrictions on ${\displaystyle L_{2}[g]}$. In other words, even slowly convergent iterations admit a simple error estimate, provided that the logarithmic Lipschitz constant of ${\displaystyle g}$ is less than ${\displaystyle 1/2}$. The bound relies on using the polarization identity, in the linear as well as the nonlinear case, and therefore only holds for the Euclidean norm.

For more complicated "rational functions" of ${\displaystyle f}$ there are only limited results. Consider a nonlinear system ${\displaystyle {\dot {x}}=f(x)}$ with ${\displaystyle M_{2}[f]\leq 0}$, discretized by the trapezoidal rule and the implicit midpoint method, using the same time step ${\displaystyle h}$. Both are A-stable Runge-Kutta methods, associated with the same rational stability function ${\displaystyle R(z)=(1+z/2)/(1-z/2)}$, so if ${\displaystyle f\equiv A}$ is a linear map, both methods are contractive (see above). In the nonlinear case, however, the trapezoidal rule yields the recursion

${\displaystyle u_{n+1}\,=\,\left(I-{\frac {hf}{2}}\right)^{-1}\circ \left(I+{\frac {hf}{2}}\right)(u_{n})}$

while the implicit midpoint method is

${\displaystyle v_{n+1}\,=\,\left(I+{\frac {hf}{2}}\right)\circ \left(I-{\frac {hf}{2}}\right)^{-1}(v_{n})\,.}$

Because the two nonlinear operators do not commute, only the second recursion is contractive under the assumption ${\displaystyle M_{2}[hf]\leq 0}$, i.e.,

${\displaystyle M_{2}[f]\,\leq \,0\quad \Rightarrow \quad L_{2}\left[\left(I+{\frac {hf}{2}}\right)\circ \left(I-{\frac {hf}{2}}\right)^{-1}\right]\,\leq \,1.}$

Thus A-stability is not sufficient. Instead a qualified condition, known as B-stability, is required for contractivity in the Euclidean norm¹³; the results again rely on the polarization identity. Of the two methods above, only the implicit midpoint method is B-stable, but there are other, advanced B-stable Runge--Kutta methods of high order of convergence.¹⁴¹⁵

For nonlinear maps boundedness and continuity are not equivalent. While the Lipschitzian approach to logarithmic norms above emphasizes continuity and stability, there is also an alternative, weaker approach, primarily linked to coercivity. Thus, for maps ${\displaystyle F}$ satisfying ${\displaystyle F(0)=0}$, defined on a star-shaped domain ${\displaystyle D}$ containing the origin, one can define an operator norm ${\displaystyle \|F\|=\sup _{x\in D}\|F(x)\|/\|x\|}$, together with an upper logarithmic norm

${\displaystyle {\hat {\mu }}[F]=\lim _{h\rightarrow 0^{+}}{\frac {\|I+hF\|-1}{h}}\,.}$

Likewise, in inner product spaces one may define lower and upper logarithmic norms by¹

${\displaystyle {\check {\mu }}_{2}[F]\cdot \|x\|_{2}^{2}\,\leq \,\langle x,F(x)\rangle \,\leq \,{\hat {\mu }}_{2}[F]\cdot \|x\|_{2}^{2}\,.}$

In dynamical systems it is of interest to estimate how far apart two solutions drift, given various types of perturbations. This concerns bounding the difference ${\displaystyle u-v}$, where

${\displaystyle {\dot {u}}=f(u)+r}$

${\displaystyle {\dot {v}}=f(v)\,.}$

From the differential inequality

${\displaystyle {\mathrm {D} }_{t}^{+}\|u-v\|\,\leq \,M[f]\cdot \|u-v\|+\|r\|\,,\qquad t\geq 0}$

the following general bound is obtained using logarithmic Lipschitz constants,

${\displaystyle \|u(t)-v(t)\|\,\leq \,{\mathrm {e} }^{tM[f]}\|u_{0}-v_{0}\|+\int _{0}^{t}{\mathrm {e} }^{(t-\tau )M[f]}\,\|r(\tau )\|\,{\mathrm {d} }\tau \,.}$

This bound applies to linear systems, non-autonomous systems, nonlinear systems as well as some problems with where ${\displaystyle f}$ represents an unbounded operator, and for any preferred choice of norm. Taking ${\displaystyle r\equiv 0}$ covers perturbed initial values, while taking ${\displaystyle u_{0}=v_{0}}$ covers the effect of forcing functions. In the latter case one can estimate

${\displaystyle \|u(t)-v(t)\|\,\leq \,{\frac {{\mathrm {e} }^{tM[f]}-1}{M[f]}}\cdot \|r\|_{\infty }\,,}$

where ${\displaystyle \|r\|_{\infty }=\max _{t\geq 0}\|r(t)\|}$. The bound generally holds on compact intervals, but if ${\displaystyle M[f]<0}$ it also holds on infinite intervals, showing that the problem is well posed, and that the error ${\displaystyle u-v}$ is uniformly bounded in terms of ${\displaystyle r}$ for all ${\displaystyle t\geq 0}$, i.e.,

${\displaystyle \|u(t)-v(t)\|\,\leq \,-\,{\frac {\|r\|_{\infty }}{M[f]}}\,.}$

In case of a stiff differential equation ${\displaystyle {\dot {u}}=f(u)}$ on ${\displaystyle [0,T]}$ it holds that

${\displaystyle {\mathrm {e} }^{-Tm[f]}\gg {\mathrm {e} }^{TM[f]}\,,}$

where the expression on the left represents the maximal growth in reverse time over a time interval of length ${\displaystyle T}$, while the right expression shows the maximal growth in forward time. Hence the condition

${\displaystyle s[f]:={\frac {m[f]+M[f]}{2}}\ll -\,{\frac {1}{T}}}$

is characteristic of stiffness: the average of the logarithmic Lipschitz constants is negative, and typically several orders of magnitude larger than ${\displaystyle 1/T}$. Here the functional ${\displaystyle s[f]}$ is referred to as the stiffness indicator.¹⁶

By instead using the logarithmic norms of the Jacobian matrix ${\displaystyle f'(u)}$, the criterion can be given a local interpretation along the solution trajectory. If an explicit method is used to integrate the equation, then the time step ${\displaystyle h}$ will be restricted by numerical stability requirements to be of order ${\displaystyle h\sim 1/\left|s[f'(u)]\right|}$.

Note that

${\displaystyle s[f'(u)]={\frac {\mu _{1}+\mu _{n}}{2}}\,,}$

where ${\displaystyle \mu _{1}}$ and ${\displaystyle \mu _{n}}$ are the largest and smallest logarithmic values (see above) of the Jacobian ${\displaystyle f'(u)}$. Because it is relatively expensive to compute the logarithmic norms, an alternative of similar utility is to compute the average of all logarithmic values. The scaled trace ${\displaystyle \tau [\cdot ]}$ is defined by

${\displaystyle \tau [f'(u)]\,:=\,{\frac {1}{n}}\sum _{k}\mu _{k}\,=\,{\frac {1}{n}}\,{\mathrm {trace} }[f'(u)]\,=\,{\frac {1}{n}}\left({\mathrm {div} }\,f\right)(u).}$

In practice, ${\displaystyle \tau [f'(u)]}$ serves well as a stiffness indicator, and is cheap to compute as it is simply the average of the diagonal elements of ${\displaystyle f'(u)}$. Moreover, it equals the scaled divergence of the vector field at ${\displaystyle u}$,

${\displaystyle {\frac {1}{n}}\left({\mathrm {div} }\,f\right)(u)\,.}$

A dynamical system with ${\displaystyle \,{\mathrm {div} }f\equiv 0\,}$ preserves phase volume (often referred to as area preserving). What is characteristic of stiff differential equations is that ${\displaystyle \,{\mathrm {div} }f\,}$ is large and negative, implying that phase volume is compressed along the solution. The phase volume compression rate ${\displaystyle \tau [f'(u)]}$ (per unit time) is indicative of stiffness at ${\displaystyle u}$, and quantifies step size restrictions along the trajectory for explicit methods.¹⁷

A differential operator ${\displaystyle {\mathcal {L}}}$ satisfying ${\displaystyle \langle u,{\mathcal {L}}u\rangle >0}$ is called elliptic. In particular, the logarithmic norm quantifies the concept of strong ellipticity. Applications are found in boundary value problems and partial differential equations. In the simplest case one considers piecewise continuously differentiable functions ${\displaystyle u,v}$ satisfying homogeneous Dirichlet conditions ${\displaystyle u(0)=u(1)=0}$, with the inner product

${\displaystyle \langle u,v\rangle \,=\,\int _{0}^{1}uv\,{\mathrm {d} }x\,}$

inducing the usual ${\displaystyle L^{2}}$ norm on the function space ${\displaystyle H_{0}^{1}(0,1)}$. Letting ${\displaystyle u'}$ denote ${\displaystyle {\mathrm {d} }u/{\mathrm {d} }x}$, integration by parts yields ${\displaystyle \langle u,v'\rangle =-\langle u',v\rangle }$. Hence, for the operator ${\displaystyle {\mathcal {L}}=-{\mathrm {d} }^{2}/{\mathrm {d} }x^{2}}$ we obtain

${\displaystyle \langle u,{\mathcal {L}}u\rangle \,=\,\langle u,-u''\rangle \,=\,\langle u',u'\rangle \,=\,\|u'\|_{2}^{2}\,\geq \,\pi ^{2}\,\|u\|_{2}^{2}\,,}$

where the last step is the Poincaré inequality. Equality is attained for the function ${\displaystyle u=\sin \pi x}$, implying that ${\displaystyle \pi ^{2}}$ is the best possible constant. Thus

${\displaystyle \langle u,{\mathcal {L}}u\rangle \,\geq \,m_{2}[{\mathcal {L}}]\cdot \|u\|_{2}^{2}\,\equiv \,\pi ^{2}\,\|u\|_{2}^{2}\,,}$

showing that ${\displaystyle {\mathcal {L}}}$ is strongly elliptic, with ${\displaystyle m_{2}[-{\mathrm {d} }^{2}/{\mathrm {d} }x^{2}]=\pi ^{2}>0}$ quantifying the ellipticity. (The upper logarithmic norm is ${\displaystyle +\infty }$, since the operator is unbounded.) An analogous derivation for Neumann conditions ${\displaystyle u(0)=u'(1)=0}$ results in ${\displaystyle m_{2}[{\mathcal {L}}]=\pi ^{2}/4}$, where the free boundary condition at ${\displaystyle x=1}$ reduces ellipticity compared with the case of Dirichlet conditions.

Considering the 1D Poisson equation ${\displaystyle -u''=f}$ with ${\displaystyle f\in L^{2}(0,1)}$ and homogeneous Dirichlet conditions, Property 14 above offers the solution bound

${\displaystyle \|u\|_{2}\,\leq \,{\frac {\|f\|_{2}}{m_{2}[{\mathcal {L}}]}}\,=\,{\frac {\|f\|_{2}}{\pi ^{2}}}\,.}$

Hence the solution ${\displaystyle u}$ is unique and depends continuously on the data, implying stability, where the inverse of the lower logarithmic norm is the stability constant. A smaller stability constant implies that the solution is more sensitive to perturbations in the data function ${\displaystyle f}$.

Similar results hold for finite element approximations, where the function space is replaced by (say) piecewise linear functions defined on a grid. This affects the logarithmic norm of ${\displaystyle {\mathcal {L}}}$, which becomes somewhat larger (since the minimizer in the Poincaré inequality cannot be represented). The theory can be extended to Sobolev norms and weak solutions, e.g. allowing "point loads" (Dirac functions in the right hand side), as is common in Galerkin methods, using different norms in the solution and data and spaces as needed.

If instead a finite difference method is used to solve ${\displaystyle -u''=f}$, the differential equation is approximated by an algebraic equation ${\displaystyle L{\tilde {u}}={\tilde {f}}}$. The matrix ${\displaystyle L}$ will typically inherit the ellipticity, i.e., ${\displaystyle m_{2}[L]>0}$, showing that ${\displaystyle L}$ is positive definite and therefore invertible.

The importance of strong ellipticity in selfadjoint differential operators can be illustrated by Euler buckling, where a slender column, fixed by articulated boundary conditions (homogeneous Dirichlet conditions as above), is subjected to an axial load ${\displaystyle F}$. The solution ${\displaystyle u}$ represents the deflection from the column's center line, and satisfies the eigenvalue problem

${\displaystyle -u''-{\frac {F}{EI}}\,u\,=\,0\,,}$

where ${\displaystyle E}$ is the elasticity module of the material and ${\displaystyle I}$ is the cross-section moment of inertia of the column. The logarithmic norm of the associated operator is

${\displaystyle m_{2}\left[-{\frac {{\mathrm {d} }^{2}}{{\mathrm {d} }x^{2}}}\,-\,{\frac {F}{EI}}\right]\,=\,\pi ^{2}-{\frac {F}{EI}}\,.}$

As long as ${\displaystyle F/(EI)<\pi ^{2}}$, the operator remains strongly elliptic, and ${\displaystyle u\equiv 0}$ is the unique solution. However, at the critical load ${\displaystyle F_{\mathrm {c} }/(EI)=\pi ^{2}}$ ellipticity is lost: the column fails (buckles) when ${\displaystyle F\geq F_{\mathrm {c} }}$. The zero solution is then no longer unique, in agreement with the Fredholm alternative, as the problem also admits nonzero (buckled) solutions satisfying the homogeneous boundary conditions.

The logarithmic norm can also be used in initial-boundary value partial differential equations ${\displaystyle u_{t}={\mathcal {A}}u+f(u)}$, where it quantifies the growth or decay rates of solutions. This is primarily of use in dissipative systems. The difference between two solutions decays if

${\displaystyle M_{2}[{\mathcal {A}}]\,+\,M_{2}[f]\,<\,0\,,}$

in which case the problem is well posed. This will typically require ${\displaystyle -{\mathcal {A}}}$ to be strongly elliptic and to dominate ${\displaystyle M_{2}[f]}$ for sufficient dissipation, such as in a reaction-diffusion equation.

For a more complicated differential operator, such as in convection-diffusion, ${\displaystyle M_{2}[{\mathcal {A}}]}$ may depend in a nontrivial way on the type of boundary conditions and the balance of convection and diffusion rates (the Péclet number), leading to questions of well posedness. For problems in several space dimensions, it is usually difficult to calculate the logarithmic norm, as it depends on the geometry of the computational domain where the boundary conditions are imposed.

References

References

1. Söderlind, G. (2024). "Logarithmic Norms." Springer Series in Computational Mathematics vol. 63. ISBN 978-3-031-74378-8 https://doi.org/10.1007/978-3-031-74379-5
2. Bullo, F. (2022). "Contraction Theory for Dynamical Systems". 1st ed., (19 Aug. 2022). Kindle Direct Publishing ISBN 979-8836646806
3. Dahlquist, G. (1958). "Stability and error bounds in the numerical integration of ordinary differential equations", Almqvist & Wiksell, Uppsala 1958
4. Lozinskiĭ, S. (1958). "Error estimates for the numerical integration of ordinary differential equations, Part I". (In Russian) Izv. Vyss. Uceb. Zaved. Matematika 6, pp 52-90 (1958)
5. Bullo, F. (2022). "Contraction Theory for Dynamical Systems". 1st ed., (19 Aug. 2022). Kindle Direct Publishing ISBN 979-8836646806
6. Desoer, C.; Haneda, H. (1972). "The measure of a matrix as a tool to analyze computer algorithms for circuit analysis". IEEE Transactions on Circuit Theory. 19 (5): 480–486. doi:10.1109/tct.1972.1083507.
7. Desoer, C. A.; Vidyasagar, M. (1975). Feedback Systems: Input-output Properties. New York: Elsevier. p. 34. ISBN 9780323157797.
8. Söderlind, G. (1986). "Bounds on nonlinear operators in finite-dimensional Banach spaces". Numerical Mathematics 50, pp 27-44.
9. Söderlind, G. (2006). "The logarithmic norm. History and modern theory", BIT Numerical Mathematics, 46(3):631-652.
10. Desoer, C.; Haneda, H. (1972). "The measure of a matrix as a tool to analyze computer algorithms for circuit analysis". IEEE Transactions on Circuit Theory. 19 (5): 480–486. doi:10.1109/tct.1972.1083507.
11. Dekker, K.; Verwer, J.G. (1984). "Stability of Runge-Kutta methods for stiff nonlinear differential equations." New York: North Holland.}
12. Söderlind, G. (1986). "Bounds on nonlinear operators in finite-dimensional Banach spaces". Numerical Mathematics 50, pp 27-44.
13. Butcher, J.C. (1975): "A stability property of implicit Runge--Kutta methods". BIT 15: 358-361.
14. Butcher, J.C. (2008) "Numerical methods for ordinary differential equations", 2nd ed. New York: Wiley
15. Hairer, E.; Wanner, G. (1991). "Solving orindary differential equations II. Stiff and differential-algebraic problems." Berlin-Heidelberg-New York: Springer.
16. Söderlind, G.; Jay, L.O.; Calvo, M. (2015) "Stiffness 1952--2012: Sixty years in search of a definition." BIT 55: 531-558.
17. Söderlind, G. (2024) "Logarithmic Norms." Springer Series in Computational Mathematics vol. 63. pp 281-286. ISBN 978-3-031-74378-8 https://doi.org/10.1007/978-3-031-74379-5