3
$\begingroup$

Consider, for example, the map $f: \mathbb{R}^{n \times n} \rightarrow \mathbb{R}^{n \times n}, f(A) = A^2.$ Then its differential is $df(A)(T) = AT+TA$. I would like a reference that states what this differential means and then how to obtain such results, but not necessarily in a completely rigorous way. I also understand that differentials can be defined and manipulated in the usual way for functionals (e.g. for the Lagrangian, leading to the Euler-Lagrange equations) and I'd like to see this done without developing the whole machinery of variational calculus.

In short, I'm looking for a clear treatment of differentials of operator-valued functions. I've tried looking up books on matrix calculus, calculus on normed vector spaces and variational calculus but haven't found anything suitable (the closest option was Cartan's Differential Calculus, but I'd like something more concrete). Where do people learn this sort of thing?

$\endgroup$
1
  • $\begingroup$ my go to reference is Cartan's book, and also Loomis and Sternberg's Advanced calculus (which is similar to Cartan's book, though there are some simpler exercises for differentiation). But since you don't want Cartan, you may not want Loomis either so I'd suggest you take a look at volume 1 of Bamberg and Sternberg's book. Volume 1 is essentially a slightly simplified version of Loomis and Sternberg's book in terms of rigour (though it has some extra topics) and generality. The nice part though, is they have many more worked out examples and they show you exactly how to do the computations. $\endgroup$ Commented Dec 31, 2020 at 7:52

4 Answers 4

Reset to default
5
$\begingroup$

Just compute the directional derivative, as you would in ordinary calculus. $df(A)(T) = \lim\limits_{h\to 0} \dfrac{f(A+hT)-f(A)}h$. Just do the matrix computation: \begin{align*} \frac{f(A+hT)-f(A)}h &= \frac{(A+hT)^2-A^2}h = \frac{h(AT+TA) + h^2T^2}h \\ &= (AT+TA) + hT^2 \to AT+TA \quad\text{as}\quad h\to 0. \end{align*} The point is that it's nothing different from calculus in Euclidean space, since the space of matrices is naturally a finite-dimensional Euclidean space.

Aside from other texts mentioned, Dieudonné's Treatise on Analysis is a standard reference. Differential Calculus in normed spaces appears in Volume 1.

$\endgroup$
3
$\begingroup$

The total derivative of a differential map $f\colon \Omega \subseteq \Bbb R^n \to \Bbb R^k$ at a point $x \in \Omega$, where $\Omega$ is open, is the unique linear map $Df(x)$ such that $$\lim_{h \to 0} \frac{f(x+h)-f(x)- Df(x)(h)}{\|h\|} = 0. $$Since matrix spaces are identified with Euclidean spaces themselves, it makes sense to compute derivatives of maps between matrix spaces. For instance, we have the chain rule $D(g\circ f)(x) = Dg(f(x))\circ Df(x)$, the total derivative of a linear map is itself, and if $B\colon \Bbb R^n \times \Bbb R^m \to \Bbb R^p$ is bilinear, its derivative is given by $$DB(x,y)(h,k) = B(x,k) + B(h,y).$$In your case, we can write $f(A) = A^2$ as $f(A) = g(\Delta(A))$, where $\Delta(A)= (A,A)$ is the (linear) diagonal map and $g(A,B) = AB$ is bilinear. So $$\begin{align} Df(A)(T) &= D(g\circ \Delta)(A)(T) = Dg(A,A) \circ D\Delta(A)(T) \\ &= Dg(A,A)(T,T) = g(A,T)+g(T,A) \\ &= AT+TA, \end{align}$$as wanted.

$\endgroup$
3
$\begingroup$

The right setting to talk about differentiability is the notion of a normed vector space. For example real $n\times n$ matrices are (obviously) a vector space, moreover you can introduce a norm on it. Also functionals in calculus of variations can often be written as a function between two normed vector spaces (the source being some vector space of functions, the target being the real numbers).

However, I'd recommend to start with something a bit simpler – learning how this formalism works in Euclidean spaces – and then learning the topic in more specialized contexts.

I'd recommend any of the following books:

  • W. Rudin's Principles of mathematical analysis,
  • T. Shifrin's Multivariable mathematics,
  • M. Spivak's Calculus on manifolds.

(Edit...) and these online materials:

$\endgroup$
6
  • $\begingroup$ Thank you for the references. I would still like to see calculus over normed vector spaces handled in a more intuitive way. I'm more interested in being able to compute these things. Is there no such reference? Physicists must deal with these computations all the time. Where do they learn this from? $\endgroup$ Commented Mar 1, 2020 at 18:54
  • $\begingroup$ LOL. Reverse alphabetical order, perhaps? Thanks for including my book. This appears as an exercise in my book. $\endgroup$ Commented Mar 1, 2020 at 19:43
  • $\begingroup$ KLC – These books have lots of exercises, so you can train the skill very quickly. @TedShifrin Your book is a piece of art – thanks for writing it! (PS. I sorted by the title). $\endgroup$ Commented Mar 1, 2020 at 23:54
  • $\begingroup$ I will add that I have 112 lectures on YouTube based on my text. Some of those might be helpful. The link is in my profile. $\endgroup$ Commented Mar 2, 2020 at 0:17
  • $\begingroup$ Great! I've made an edit and included them in the answer. $\endgroup$ Commented Mar 2, 2020 at 0:27
2
$\begingroup$

A book I've had for a long time (I think I purchased it from a university bookstore in 1981 or 1982) might be helpful. Although it's a bit weak on specific examples, the exposition is very straightforward and is accessible to someone with a fairly limited background (much less than for standard functional analysis texts, except maybe for Kreyszig's Introductory Functional Analysis with Applications, which might also be worth looking at). I'm including the contents because not much specific seems to be posted on the internet about it. Indeed, the only mention in Stack Exchange that I could find is this 4 November 2013 comment by me.

Leopoldo Nachbin, Introduction to Functional Analysis: Banach Spaces and Differential Calculus, translation of the 1976 Portuguese edition by Richard Martin Aron, Monographs and Textbooks in Pure and Applied Mathematics #60, Marcel Dekker, 1981, xii + 166 pages. Amer. Math. Monthly review

CONTENTS (pp. v-vi). PREFACE (pp. vii-ix).

PART I. BANACH SPACES (pp. 1-84).

1. Normed Spaces (pp. 3-9). 2. Banach Spaces (pp. 10-19). 3. Normed Subspaces (pp. 20-24). 4. Equivalent Norms (pp. 25-32). 5. Spaces of Continuous Linear Operators (pp. 33-42). 6. Continuous Linear Forms (pp. 43-49). 7. Isometries (pp. 50-51). 8. Cartesian Products and Direct Sums (pp. 52-56). 9. Cartesian Products of Normed Spaces (pp. 57-59). 10. Topological Direct Sums (pp. 60-62). 11. Finite Dimensional Normed Spaces (pp. 63-76). 12. Spaces of Continuous Multilinear Operators (pp. 77-84).

PART II. DIFFERENTIAL CALCULUS (pp. 85-160).

13. Differential Calculus in Normed Spaces (pp. 87-91). 14. The Differential in Normed Spaces (pp. 92-96). 15. Continuous Affine Tangent Mappings (pp. 97-98). 16. Some Rules of Differential Calculus (pp. 99-111). 17. The Scalar Variable Case (pp. 112-114). 18. The Lagrange Mean Value Theorem (pp. 115-123). 19. Mappings with Zero or Constant Derivatives (pp. 124-126). 20. Interchanging the Order of Differentiation and Limit (pp. 127-130). 21. Continuously Differentiable Mappings (pp. 131-132). 22. Partial Differentiation (pp. 133-142). 23. Natural Identifications for Multilinear Mappings (pp. 143-149). 24. Higher Order Differentiation (pp. 150-160).

NOTATION (pp. 161-162). BIBLIOGRAPHY (pp. 163-164). INDEX (pp. 165-166).

$\endgroup$

You must log in to answer this question.