<link href="../markdown.css" rel="stylesheet"></link>

<title>4.3. Orthogonal systems</title>

<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS_HTML">
    MathJax.Hub.Config({
        tex2jax: {
            inlineMath: [ ['$','$'], ["\\(","\\)"] ],
            displayMath: [ ['$$','$$'], ["\\[","\\]"] ],},
        TeX: {equationNumbers: {
            autoNumber: "AMS"
          },Augment: {  Definitions: {
           macros: {
             overbracket:  ['UnderOver','23B4',1],
             underbracket: ['UnderOver','23B5',1],
           }
         }}},
    });
</script>

$\renewcommand{\Re}{\operatorname{Re}}$
$\renewcommand{\Im}{\operatorname{Im}}$
$\newcommand{\erf}{\operatorname{erf}}$
$\newcommand{\dag}{\dagger}$
$\newcommand{\const}{\mathrm{const}}$
$\newcommand{\arcsinh}{\operatorname{arcsinh}}$


##[4.3. Orthogonal systems](id:sect-4.3)
------------------

> 1.  [Examples](#sect-4.3.1)
> 2.  [Abstract orthogonal systems: definition](#sect-4.3.2)
> 3.  [Orthogonal systems: approximation](#sect-4.3.3)
> 4.  [Orthogonal systems: approximation. II](#sect-4.3.4)
> 5.  [Orthogonal systems: completeness](#sect-4.3.5)

###[Examples](id:sect-4.3.1)

All systems we considered in the previous Section  were orthogonal i.e.
\begin{equation}
(X\_n, X\_m)=0\qquad \forall m\ne n \label{eq-4.3.1}
\end{equation}
with
\begin{equation}
(X,Y):=\int\_0^l X(x)\bar{Y} (x)\,dx,\qquad \\|X\\|^2:=(X,X).
\label{eq-4.3.2}
\end{equation}
where $\bar{Y}$ means complex-conjugate to $Y$.

**[Exercise 1.](id:exercise-4.3.1)** Prove it by direct calculation.
<!--\end{exercise}-->

Instead however we show that this nice property (and the fact that eigenvalues are real) is due to self-adjointness (the notion which we do not want to formulate at this time at least).

Consider $X,Y$ satisfying Robin boundary conditions
\begin{align}
&X'(0)-\alpha X(0)=0,\label{eq-4.3.3}\\\\
&X'(l)+\beta X(l)=0\label{eq-4.3.4}
\end{align} with
$\alpha,\beta\in \mathbb{R}$ (so $Y$ satisfies the same conditions). Note that \begin{multline}
(X'',Y)=\int X''(x)\bar{Y}(x)\,dx = \\\\
-\int X'(x)\bar{Y}'(x)\,dx + X' (l)\bar{Y}(l)- X' (0)\bar{Y}(0)= \\\\
-(X',Y') -\beta X (l)\bar{Y}(l)-\alpha X(0)\bar{Y}(0).\qquad
\label{eq-4.3.5}
\end{multline}
Therefore if we plug $Y=X\ne 0$ an eigenfunction, $X''+\lambda X=0$ we get $-\lambda \\|X\\|^2$ in the left-hand expression (with obviously real
$\\|X\\|^2\ne 0$) and also we get the real right expression (since $\alpha,\beta\in \mathbb{R}$); so $\lambda$ must be real: *all eigenvalues are real*.

Further, for $(X,Y'')$ we obtain the same equality albeit with $\alpha,\beta$ replaced by $\bar{\alpha},\bar{\beta}$ and therefore due to assumption $\alpha,\beta\in \mathbb{R}$
\begin{equation}
(X'',Y)= (X,Y'').
\label{eq-4.3.6}
\end{equation}
But then if $X,Y$ are eigenfunctions corresponding to *different* eigenvalues $\lambda$ and $\mu$ we get from (\ref{eq-4.3.6}) that $-\lambda(X,Y)=-\mu (X,Y)$ and $(X,Y)=0$ due to $\lambda\ne \mu$.

**[Remark 1.](id:rem-4.3.1)** For periodic boundary conditions we cannot apply these arghuments to prove that $\cos(2\pi nx/l)$ and $\cos(2\pi nx/l)$ are orthogonal since they correspond to the same eigenvalue; we need to prove it directly.
<!--\end{remark}-->

###[Abstract orthogonal systems: definition](id:sect-4.3.2)

Consider *linear space* $\mathsf{H}$, real or complex. From linear algebra course [standard
definition](http://en.wikipedia.org/wiki/Vector_space#Definition)

1.  $u+v=v+u\qquad \forall u,v\in \mathsf{H}$;
2.  $(u+v)+w=u+(v+w)\qquad \forall u,v,w\in \mathsf{H}$;
3.  $\exists 0\in \mathsf{H}: \ 0+u=u\qquad \forall u\in \mathsf{H}$;
4.  $\forall u\in \mathsf{H}\\ \exists (-u): u+(-u)=0 $;
5.  $\alpha(u+v)=\alpha u+ \alpha v \qquad \forall u,v\in \mathsf{H}\quad \forall\alpha \in \mathbb{R}$;
6.  $(\alpha+\beta)u=\alpha u+ \beta u \qquad \forall u\in
    \mathsf{H}\quad\forall\alpha,\beta \in \mathbb{R}$;
7.  $\alpha(\beta u)=(\alpha \beta)u \qquad \forall u\in
    \mathsf{H}\quad \forall\alpha,\beta \in \mathbb{R}$;
8.  $1u=u\qquad \forall u \in \mathsf{H}$.

For complex linear space replace $\mathbb{R}$ by $\mathbb{C}$.

Assume that on $\mathsf{H}$ *inner product* is defined:

1.  $(u+v,w)=(u,w)+(v,w)\qquad \forall u,v,w\in \mathsf{H}$;
2.  $(\alpha u,v)=\alpha (u,v) \qquad\forall u,v\in \mathsf{H} \quad \forall \alpha\in \mathbb{R}$;
3.  $(u,v)=\bar{(v,u)} \qquad \forall u,v\in \mathsf{H}$;
4.  $\\|u\\|^2:=(u,u)\ge 0 \qquad \forall u\in \mathsf{H}$ (it implies that it is real--if we consider complex spaces) and  $\\|u\\|=0 \iff u=0$.

**[Definition 1.](id:definition-4.3.1)**

a.  Finite dimensional real linear space with an inner product is called *Euclidean* space.
b.  Finite dimensional complex linear space with an inner product is called *Hermitian* space.
c.  Infinite dimensional linear space (real or complex) with an inner product is called *pre-Hilbert* space.

For Hilbert space we will need another property (completeness) which we
add later.
<!--\end{definition}-->

**[Definition 2.](id:definition-4.3.2)**

a.  System $\\{u\_n\\}$, $0\ne u\_n\in \mathsf{H}$ (finite or  infinite) is *orthogonal* if $(u\_m,u\_n)=0$ $\forall m\ne n$;
a.  Orthogonal system is *orthonormal* if $\\|u\_n\\|=1$ $\forall  n$, i.e. $(u\_m,u\_n)=\delta\_{mn}$ -- Kronecker symbol.
<!--\end{definition}-->

###[Orthogonal systems: approximation](id:sect-4.3.3)

Consider finite orthogonal system $\\{u\_n\\}$. Let $\mathsf{K}$ be its *linear hull*: the set of linear combinations $\sum\_n \alpha\_nu\_n$. Obviously $\mathsf{K}$ is a linear subspace of $\mathsf{H}$. Let $v\in \mathsf{H}$ and we try to find the best approximation of $v$ by elements of $\mathsf{K}$, i.e. we are looking for $w\in \mathsf{K}$ s.t. $\\|v-w\\|$ minimal.

**[Theorem 1.](id:thm-4.3.1)**

a.  There exists a unique minimizer;
b.  This minimizer is an orthogonal projection of $f$ to  $\mathsf{K}$, i.e. $w\in \mathsf{K}$ s.t. $(v-w)$ is orthogonal to all elements of $\mathsf{K}$;
c.  Such orthogonal projection is unique and $w=\sum\_n \alpha\_n  u\_n$ with \begin{equation}
\alpha\_n= \frac{(v,u\_n)}{\|u\_n\|^2}.
\label{eq-4.3.7}
\end{equation}
d.  $\\|v\\|^2=\\|w\\|^2+\\|v-w\\|^2$.
e.  $v=w \iff \\|v\\|^2=\\|w\\|^2$.
<!--\end{theorem}-->

*Proof.* (c) Obviously $(v-w)$ is orthogonal to $u\_n$ iff (\ref{eq-4.3.7}) holds. If (\ref{eq-4.3.7}) holds for all $n$ then $(v-w)$ is orthogonal to
all $u\_n$ and therefore to all their linear combinations.

(d)-(e) In particular $(v-w)$ is orthogonal to $w$ and then
\begin{equation\*}
\\|v\\|^2= \\|(v-w)+w\\|^2=\\|v-w\\|^2+
2\Re \underbracket{(v-w,w)}\_{=0}+\\|w\\|^2.
\end{equation\*}

(a)-(b) Consider $w'\in \mathsf{K}$. Then $\\|v-w'\\|^2=\\|v-w\\|^2+\\|w-w'\\|^2$ because $(w-w')\in \mathsf{K}$ and therefore it is orthogonal to $(v-w)$.
<!--\end{proof}-->

###[Orthogonal systems: approximation. II](id:"sect-4.3.4)

Now let $\\{u\_n\\}\_{n=1,2,\ldots,}$ be infinite orthogonal system. Consider its finite subsystem with $n=1,2,\ldots, N$, introduce $\mathsf{K}\_N$ for it and consider orthogonal projection $w\_N$ of $v$ on $\mathsf{K}\_N$. Then
\begin{equation\*}
w\_N= \sum\_{n=1}^N \alpha\_N u\_n
\end{equation\*}
where $\alpha\_n$ are defined by (\ref{eq-4.3.7}).
Then according to (d) of Theorem
\begin{equation\*}
\\|v\\|^2 =\\|v-w\_N\\|^2+\\|w\_N\\|^2\ge
\\|w\_N\\|^2=\sum\_{n=1}^N |\alpha\_n |^2\\|u\_n\\|^2.
\end{equation\*}
Therefore series in the right-hand expression below converges
\begin{equation}
\\|v\\|^2 \ge \sum\_{n=1}^\infty |\alpha\_n |^2\\|u\_n\\|^2
\label{eq-4.3.8}
\end{equation}
Really, recall that non-negative series can either converge or diverge to $\infty$.

Then $w\_N$ is a *Cauchy sequence*. Really, for $M\>N$
\begin{equation\*}
\\|w\_N-w\_M\\|^2= \sum\_{n=N+1}^M |\alpha\_n |^2\\|u\_n\\|^2\le \varepsilon\_N
\end{equation\*}
with $\varepsilon\_N\to 0$ as $N\to \infty$ because series in (\ref{eq-4.3.8}) converges.

Now we want to conclude that $w\_N$ converges and to do this we must
assume that every Cauchy sequence converges.

**[Definition 3.](id:definition-4.3.3)**

a.  $\mathsf{H}$ is *complete* if every Cauchy sequence converges in
    $\mathsf{H}$.
b.  Complete pre-Hilbert space is called *Hilbert space*.
<!--\end{definition}-->

**[Remark 2.](id:remark-4.3.2)** Every pre-Hilbert space could be completed i.e. extended to a complete space. From now on $\mathsf{H}$ is a Hilbert space.
<!--\end{remark}-->

Then we can introduce $\mathsf{K}$-- a closed linear hull of $\\{u\_n\\}\_{n=1,2,\ldots}$ i.e. the space of
\begin{equation}
\sum\_{n=1}^\infty \alpha\_n u\_n
\label{eq-4.3.9}
\end{equation}
with $\alpha\_n$ satisfying
\begin{equation}
\sum\_{n=1}^\infty |\alpha\_n |^2\\|u\_n\\|^2\<\infty.
\label{eq-4.3.10}
\end{equation}
(Linear hull would be a space of finite linear combinations).

Let $v\in \mathsf{H}$. We want to find the best approximation of $v$ by elements of $\mathsf{K}$. But then we get immediately

**[Theorem 2.](id:thm-4.3.2)** If $\mathsf{H}$ is a Hilbert space then [Theorem 1](#thm-4.3.1) holds for infinite systems as well.
<!--\end{theorem}-->

###[Orthogonal systems: completeness](id:sect-4.3.5)

**[Definition 4.](<id:definition-4.3.4)** Orthogonal system is *complete* if equivalent conditions below are satisfied:

a.  Its closed convex hull coincides with $\mathsf{H}$.
b.  If $v\in \mathsf{H}$ is orthogonal to all $u\_n$ then $v=0$.
<!--\end{definition}-->

**[Remark 3.](id:remark-4.3.3)**
Don't confuse completeness of spaces and completeness of orthogonal systems.

Our next goal is to establish completeness of some orthogonal systems and therefore to give a positive answer (in the corresponding frameworks) to the question in the end of the previous [Section 4.2](./S4.2.html): can we decompose any function into eigenfunctions? Alternatively: Is the general solution a combination of simple solutions?

_______

[$\Leftarrow$](./S4.2.html)&nbsp;&nbsp;[$\Uparrow$](../contents.html)&nbsp;&nbsp;[$\Rightarrow$](./S4.4.html)
