Derivative (multivariable calculus)

In mathematics, the derivative of a function at a point is the linear part of the best affine approximation to the function near the point. In one-variable calculus, this is the tangent line approximation. In multivariable calculus, the same property is generalized to define the derivative of a vector-valued function or function of a vector argument. Sometimes called the total derivative, in contrast with partial derivatives, the derivative approximates the function with respect to all of its arguments, not just a single one.¹ In many situations, this is the same as considering all partial derivatives simultaneously.

In functional analysis, particularly in infinite dimensions, the derivative in this sense is called the Fréchet derivative.²³

Let ${\displaystyle U\subseteq \mathbb {R} ^{n}}$ be an open subset. Then a function ${\displaystyle f\colon U\to \mathbb {R} ^{m}}$ is said to be differentiable at a point ${\displaystyle a\in U}$ if there exists a linear transformation ${\displaystyle Df_{a}\colon \mathbb {R} ^{n}\to \mathbb {R} ^{m}}$ such that⁴⁵

${\displaystyle \lim _{x\to a}{\frac {f(x)-f(a)-Df_{a}(x-a)}{\|x-a\|}}=0}$

where ${\displaystyle \|\ldots \|}$ denotes the norm of ${\displaystyle \ldots }$. The linear map ${\displaystyle Df_{a}}$ is called the derivative or differential of ${\displaystyle f}$ at ${\displaystyle a}$.⁴⁶ Here ${\displaystyle Df_{a}(x-a)}$ refers to applying the linear transformation ${\displaystyle Df_{a}}$ to the vector ${\displaystyle (x-a)}$; in coordinates, this is a matrix-vector product. Other notations for the derivative include ${\displaystyle D_{a}f}$ and ${\displaystyle Df(a)}$. A function is differentiable if its derivative exists at every point in its domain.

Conceptually, the definition of the derivative expresses the idea that ${\displaystyle Df_{a}}$ is the best linear approximation to ${\displaystyle f(a+h)-f(a)}$ for small ${\displaystyle h}$. This can be made precise by quantifying the error in the linear approximation ${\displaystyle Df_{a}}$. To do so, write

${\displaystyle f(a+h)=f(a)+Df_{a}(h)+\varepsilon (h),}$

where ${\displaystyle \varepsilon (h)}$ equals the error in the approximation. To say that the derivative of ${\displaystyle f}$ at ${\displaystyle a}$ is ${\displaystyle Df_{a}}$ is equivalent to the statement

${\displaystyle \varepsilon (h)=o(h),}$

where ${\displaystyle o}$ is little-o notation and means that ${\displaystyle \varepsilon (h)/\|h\|}$ tends to zero as ${\displaystyle h\to 0}$. The derivative ${\displaystyle Df_{a}}$ is the unique linear transformation for which the error term is this small, and this is the sense in which it is the best linear approximation to ${\displaystyle f(a+h)-f(a)}$.

The function ${\displaystyle f}$ is differentiable if and only if each of its components ${\displaystyle f_{i}\colon U\to \mathbb {R} }$ is differentiable, so when studying derivatives, it is often possible to work one coordinate at a time in the codomain. However, the same is not true of the coordinates in the domain. It is true that if ${\displaystyle f}$ is differentiable at ${\displaystyle a}$, then each partial derivative ${\displaystyle \partial f/\partial x_{i}}$ exists at ${\displaystyle a}$.⁷

The converse does not hold: it can happen that all of the partial derivatives of ${\displaystyle f}$ at ${\displaystyle a}$ exist, but ${\displaystyle f}$ is not differentiable at ${\displaystyle a}$.⁸ An example is the following function, which is continuous and has both partial derivatives zero at the origin, but is not differentiable there: $${\displaystyle f(x,y)={\begin{cases}{\frac {xy}{\sqrt {x^{2}+y^{2}}}}&(x,y)\neq (0,0)\\0&(x,y)=(0,0)\end{cases}}.}$$ (In polar coordinates, this function is ${\displaystyle f=r\cos \theta \sin \theta }$.)

Even the existence and linearity of all directional derivatives at a point is not sufficient for differentiability; the essential additional requirement is that the linear approximation hold uniformly as the increment tends to zero from all directions. An example is $${\displaystyle f(x,y)={\begin{cases}{\frac {x^{2}y}{x^{4}+y^{2}}}{\sqrt {x^{2}+y^{2}}}&(x,y)\neq (0,0)\\0&(x,y)=(0,0)\end{cases}}}$$ whose directional derivatives are all 0 at (0,0), but which fails to be differentiable there.

However, if all the partial derivatives of ${\displaystyle f}$ at ${\displaystyle a}$ exist in a neighborhood of ${\displaystyle a}$ and are continuous at ${\displaystyle a}$, then ${\displaystyle f}$ is differentiable at ${\displaystyle a}$.⁹ If ${\displaystyle f}$ is differentiable at a point, then the derivative of ${\displaystyle f}$ is the linear transformation corresponding to the Jacobian matrix of partial derivatives at the point.¹⁰

In some advanced calculus texts, the derivative is also called the differential.¹⁶ However, this term has several different, but closely connected meanings, in mathematics and the sciences.

When a differentiable function ${\displaystyle f\colon \mathbb {R} ^{n}\to \mathbb {R} }$ is scalar valued, the derivative of ${\displaystyle f}$ at ${\displaystyle a}$ may be written as the Jacobian matrix, which in this instance is a row matrix (a matrix consisting of elements in a single row, i.e., a row vector):

${\displaystyle Df_{a}={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}(a)&\cdots &{\frac {\partial f}{\partial x_{n}}}(a)\end{bmatrix}}.}$

The linear approximation property of the derivative implies that if

${\displaystyle \Delta x={\begin{bmatrix}\Delta x_{1}&\cdots &\Delta x_{n}\end{bmatrix}}^{\mathsf {T}}}$

is a small vector (where the ${\displaystyle {\mathsf {T}}}$ denotes transpose, so that this vector is a column vector), then

${\displaystyle f(a+\Delta x)-f(a)\approx Df_{a}\,\Delta x=\sum _{i=1}^{n}{\frac {\partial f}{\partial x_{i}}}(a)\,\Delta x_{i}.}$

Heuristically, this suggests that if ${\displaystyle dx_{1},\ldots ,dx_{n}}$ are infinitesimal increments in the coordinate directions, then

${\displaystyle df_{a}=\sum _{i=1}^{n}{\frac {\partial f}{\partial x_{i}}}(a)\,dx_{i}}$

and this is the differential of ${\textstyle f}$ at ${\textstyle a}$.¹¹¹² In fact, the notion of the infinitesimal, which is merely symbolic here, can be equipped with extensive mathematical structure. Techniques, such as the theory of differential forms, effectively give analytical and algebraic descriptions of objects like infinitesimal increments, ${\displaystyle dx_{i}}$. For instance, ${\displaystyle dx_{i}}$ may be inscribed as a linear functional on the vector space ${\displaystyle \mathbb {R} ^{n}}$. Evaluating ${\displaystyle dx_{i}}$ at a vector ${\displaystyle h}$ in ${\displaystyle \mathbb {R} ^{n}}$ measures how much ${\displaystyle h}$ points in the ${\displaystyle i}$-th coordinate direction. The differential ${\displaystyle df_{a}}$ is a linear combination of linear functionals and hence is itself a linear functional. The evaluation ${\displaystyle df_{a}(h)}$ is the directional derivative of ${\displaystyle f}$ along ${\displaystyle h}$. This point of view makes the derivative an instance of the exterior derivative.¹³

Being a linear form, the differential is naturally a covector. In a Euclidean space, a covector is naturally dual to a vector (its transpose). This vector is called the gradient of ${\displaystyle f}$, and points in the direction in which ${\displaystyle f}$ increases most rapidly.

Suppose now that ${\displaystyle f}$ is a vector-valued function, that is, ${\displaystyle f\colon \mathbb {R} ^{n}\to \mathbb {R} ^{m}}$. In this case, the components ${\displaystyle f_{i}}$ of ${\displaystyle f}$ are real-valued functions, so they have associated differential forms ${\displaystyle df_{i}}$. The differential ${\displaystyle df}$ amalgamates these forms into a single object and is therefore an instance of a vector-valued differential form.

If ${\displaystyle f\colon M\to N}$ is a mapping between differentiable manifolds, differentiability can be formulated by differentiability in any coordinate chart. Invariantly, the differential of ${\displaystyle f}$ at a point ${\displaystyle p\in M}$ is a linear map ${\displaystyle df_{p}\colon T_{p}M\to T_{f(p)}N}$ from the tangent space of ${\displaystyle M}$ at ${\displaystyle p}$ to that of ${\displaystyle N}$ at ${\displaystyle f(p)}$.¹⁴¹⁵ This is also known as the pushforward. This is closely related to the derivative as a linear approximation, since evaluating ${\displaystyle df_{p}}$ on a tangent vector gives the directional derivative of ${\displaystyle f}$ in that direction. Nevertheless, the differential is not quite the same thing as the derivative of a function between vector spaces: ${\displaystyle f}$ is a mapping of manifolds and not of the vector spaces ${\displaystyle T_{p}M}$ and ${\displaystyle T_{f(p)}N}$ where the linear approximation lives.

In applications such as thermodynamics, the language of differentials often has an additional conceptual role. Expressions such as ${\displaystyle dU=T\,dS-P\,dV}$ describe differentials of state functions and lead to questions about exactness, natural variables, and relations among partial derivatives. This use is related to the derivative of a multivariable function, but it is not just another name for the linear map ${\displaystyle Df_{a}}$ associated with an ordinary function.¹⁶¹⁷

The term total derivative is also used in more than one way. In some mathematical texts it denotes the full derivative ${\displaystyle Df_{a}}$, as opposed to any one partial derivative. In that sense, the total derivative is the linear map that accounts for variation in all coordinate directions simultaneously.

In many applied contexts, however, "total derivative" refers instead to the derivative of a composite dependence. For example, if

${\displaystyle z=f(x,y),\qquad x=x(t),\quad y=y(t),}$

then the total derivative of ${\displaystyle z}$ with respect to ${\displaystyle t}$ is

${\displaystyle {\frac {dz}{dt}}={\frac {\partial f}{\partial x}}{\frac {dx}{dt}}+{\frac {\partial f}{\partial y}}{\frac {dy}{dt}}.}$

This is the ordinary derivative of the composite function ${\displaystyle f(x(t),y(t))}$, computed by the chain rule. Equivalently, it is obtained by applying the derivative of ${\displaystyle f}$ to the velocity vector of the path ${\displaystyle t\mapsto (x(t),y(t))}$:

${\displaystyle {\frac {d}{dt}}f(x(t),y(t))=Df_{(x(t),y(t))}(x'(t),y'(t)).}$

This chain-rule sense of "total derivative" is common in physics, engineering, economics, and other applied fields. In mechanics, for instance, the total time derivative of a function ${\displaystyle F(q,p,t)}$ along a trajectory includes both explicit dependence on ${\displaystyle t}$ and implicit dependence through the time-dependent variables ${\displaystyle q(t)}$ and ${\displaystyle p(t)}$. In fluid mechanics, related terminology appears in the material derivative, which differentiates a quantity along the motion of a fluid parcel.¹⁸¹⁹

Another example is in classical mechanics, where the total derivative of a function that depends on phase space parameters and time is its partial derivative in time plus its Poisson bracket with the Hamiltonian ${\displaystyle H}$:²⁰ $${\displaystyle {\frac {df}{dt}}={\frac {\partial f}{\partial t}}+\{f,H\}}$$ Like the material derivative, total derivative in mechanics has the property that it is the derivative of the composite when pulled back to any Hamiltonian trajectory, but is still treated as a function of all phase space coordinates and time.

In comparative statics, total derivatives often describe how endogenous variables change with respect to exogenous variables in an implicitly defined system of equations.²¹ The endogeneous variables are generally not explicit functions of the exogeneous variables, other than through the implicit function theorem, and the total derivative is handled implicitly.

Thus, although "total derivative" can mean the derivative ${\displaystyle Df_{a}}$ in the sense above, the term also commonly refers to a derivative along a specified dependence or process. This ambiguity is one reason to distinguish between the derivative as a linear map and the various differential or total-derivative notations used in applications.

A form of the chain rule generalizes from one-variable calculus. It says that, for two functions ${\displaystyle f}$ and ${\displaystyle g}$, the derivative of the composite function ${\displaystyle f\circ g}$ at ${\displaystyle a}$ satisfies

${\displaystyle D(f\circ g)_{a}=Df_{g(a)}\circ Dg_{a}}$

where the composite on the right-hand side is the composition of linear maps. If the derivatives of ${\displaystyle f}$ and ${\displaystyle g}$ are identified with their Jacobian matrices, then the composite on the right-hand side is simply matrix multiplication.

Suppose that f is a function of two variables, x and y. If these two variables are independent, so that the domain of f is ${\displaystyle \mathbb {R} ^{2}}$, then the behavior of f may be understood in terms of its partial derivatives in the x and y directions. However, in some situations, x and y may be dependent. For example, it might happen that f is constrained to a curve ${\displaystyle y=y(x)}$. In this case, we are actually interested in the behavior of the composite function ${\displaystyle f(x,y(x))}$. The partial derivative of f with respect to x does not give the true rate of change of f with respect to changing x because changing x necessarily changes y, while the partial derivative assumes y is fixed. However, the chain rule takes such dependencies into account. Write ${\displaystyle \gamma (x)=(x,y(x))}$. Then, the chain rule says

${\displaystyle D(f\circ \gamma )_{x_{0}}=Df_{(x_{0},y(x_{0}))}\circ D\gamma _{x_{0}}.}$

By expressing the derivative using Jacobian matrices such as$${\displaystyle {\begin{array}{lcl}Df_{(x_{0},y(x_{0}))}&=&{\begin{bmatrix}{\frac {\partial f}{\partial x}}&{\frac {\partial f}{\partial y}}\end{bmatrix}}_{(x_{0},y(x_{0}))}\\D\gamma _{x_{0}}&=&{\begin{bmatrix}{\frac {\partial x}{\partial x}}\\{\frac {\partial y}{\partial x}}\end{bmatrix}}_{x_{0}}.\end{array}}}$$This becomes:

${\displaystyle {\frac {df(x,y(x))}{dx}}(x_{0})={\frac {\partial f}{\partial x}}(x_{0},y(x_{0}))\,{\frac {dx}{dx}}(x_{0})+{\frac {\partial f}{\partial y}}(x_{0},y(x_{0}))\,{\frac {dy}{dx}}(x_{0}).}$

Suppressing the evaluation at ${\displaystyle x_{0}}$ for legibility, we may also write this as

${\displaystyle {\frac {df(x,y(x))}{dx}}={\frac {\partial f}{\partial x}}{\frac {dx}{dx}}+{\frac {\partial f}{\partial y}}{\frac {dy}{dx}}.}$

This gives a straightforward formula for the derivative of ${\displaystyle f(x,y(x))}$ in terms of the partial derivatives of ${\displaystyle f}$ and the derivative of ${\displaystyle y(x)}$.

For example, suppose

${\displaystyle f(x,y)=xy.}$

The rate of change of f with respect to x is usually the partial derivative of f with respect to x; in this case,

${\displaystyle {\frac {\partial f}{\partial x}}=y.}$

However, if y depends on x, the partial derivative does not give the true rate of change of f as x changes because the partial derivative assumes that y is fixed. Suppose we are constrained to the line

${\displaystyle y=x.}$

Then

${\displaystyle f(x,y)=f(x,x)=x^{2},}$

and the total derivative of f with respect to x is

${\displaystyle {\frac {df}{dx}}=2x,}$

which we see is not equal to the partial derivative ${\displaystyle \partial f/\partial x}$. Instead of immediately substituting for y in terms of x, however, we can also use the chain rule as above:

${\displaystyle {\frac {df}{dx}}={\frac {\partial f}{\partial x}}+{\frac {\partial f}{\partial y}}{\frac {dy}{dx}}=y+x\cdot 1=x+y=2x.}$

While one can often perform substitutions to eliminate indirect dependencies, the chain rule provides for a more efficient and general technique. Suppose ${\displaystyle L(t,x_{1},\dots ,x_{n})}$ is a function of time ${\displaystyle t}$ and ${\displaystyle n}$ variables ${\displaystyle x_{i}}$ which themselves depend on time. Then, the time derivative of ${\displaystyle L}$ is

${\displaystyle {\frac {dL}{dt}}={\frac {d}{dt}}L{\bigl (}t,x_{1}(t),\ldots ,x_{n}(t){\bigr )}.}$

The chain rule expresses this derivative in terms of the partial derivatives of ${\displaystyle L}$ and the time derivatives of the functions ${\displaystyle x_{i}}$:

${\displaystyle {\frac {dL}{dt}}={\frac {\partial L}{\partial t}}+\sum _{i=1}^{n}{\frac {\partial L}{\partial x_{i}}}{\frac {dx_{i}}{dt}}={\biggl (}{\frac {\partial }{\partial t}}+\sum _{i=1}^{n}{\frac {dx_{i}}{dt}}{\frac {\partial }{\partial x_{i}}}{\biggr )}(L).}$

This expression is often used in physics for a gauge transformation of the Lagrangian, as two Lagrangians that differ only by the total time derivative of a function of time and the ${\displaystyle n}$ generalized coordinates lead to the same equations of motion. An interesting example concerns the resolution of causality concerning the Wheeler–Feynman time-symmetric theory. The operator in brackets (in the final expression above) is also called the total derivative operator (with respect to ${\displaystyle t}$).

For example, the total derivative of ${\displaystyle f(x(t),y(t))}$ is

${\displaystyle {\frac {df}{dt}}={\partial f \over \partial x}{dx \over dt}+{\partial f \over \partial y}{dy \over dt}.}$

Here there is no ${\displaystyle \partial f/\partial t}$ term since ${\displaystyle f}$ itself does not depend on the independent variable ${\displaystyle t}$ directly.

A total differential equation is a differential equation expressed in terms of total derivatives. Since the exterior derivative is coordinate-free, in a sense that can be given a technical meaning, such equations are intrinsic and geometric.

In economics, it is common for the total derivative to arise in the context of a system of equations.^{22: pp. 217–220} For example, a simple supply-demand system might specify the quantity q of a product demanded as a function D of its price p and consumers' income I, the latter being an exogenous variable, and might specify the quantity supplied by producers as a function S of its price and two exogenous resource cost variables r and w. The resulting system of equations

${\displaystyle q=D(p,I),}$

${\displaystyle q=S(p,r,w),}$

determines the market equilibrium values of the variables p and q. The total derivative ${\displaystyle dp/dr}$ of p with respect to r, for example, gives the sign and magnitude of the reaction of the market price to the exogenous variable r. In the indicated system, there are a total of six possible total derivatives, also known in this context as comparative static derivatives: dp/dr, dp/dw, dp/dI, dq/dr, dq/dw, and dq/dI. The total derivatives are found by totally differentiating the system of equations, dividing through by, say dr, treating dq/dr and dp/dr as the unknowns, setting dI = dw = 0, and solving the two totally differentiated equations simultaneously, typically by using Cramer's rule.