Fatou's Lemma, the Monotone Convergence Theorem and the Dominated Convergence Theorem

Consider integration on $\mathbb{R}^d$ as a mapping from the vector space of measurable functions on $\mathbb{R}^d$ to $[-\infty,\infty]$. If we restrict our attention to the collection of positive measurable functions, then a natural property to examine of this mapping is continuity.

If $ f_n \longrightarrow f$ a.e. does this imply that $\int f_n \longrightarrow \int f$ ?

e.g. $f_n(x) = n \cdot \chi_{[0,\frac{1}{n}]}(x)$

Then, $\{f_n\}$ is a sequence of measurable functions, $f_n \longrightarrow 0$ a.e. and so the zero function is the pointwise limit of $\{f_n\}$. So, $\int f = \int 0 = 0$. However, $\int f_n = 1 $ $\forall n\in{\mathbb{N}}$, so we get that $\int f < \lim_{n \rightarrow \infty} \int f_n$.

In general, the sequence of extended real numbers $\{f_n\}$ may not have a limit. Any sequence of extended real numbers does always have a $\limsup$ and a $\liminf$. So, in the above example, we have, in particular that $\int f < \liminf_{n \rightarrow \infty} \int f_n$ $\textit{theorem}$: Fatou's Lemma

Given $\{f_n\}$ a sequence of positive measurable functions and $f_n \longrightarrow f$ a.e. then $$\int f \leq \liminf_{n \rightarrow \infty} \int f_n$$ $\textit{proof}$:
The sequential properties of measurable functions imply that $f$ is a positive, measurable function. By definition, the Lebesgue integral of a positive measurable function is $$\int f \equiv \sup_g \left\lbrace\int g \right\rbrace$$ where the supremum is taken over all measurable $g$, $0 \leq g \leq f$, $g$ bounded and supported on a set of finite measure. Thus, to prove Fatou's Lemma, it suffices to show that for such $g$, $$\int g \leq \liminf_{n \rightarrow \infty} \int f_n$$ Let $g$ be a function satisfying the conditions of being bounded, measurable, supported on a set of finite measure and $0 \leq g \leq f$. Define $$g_n(x) = \min(g(x),f_n(x))$$ Then, $g_n$ are measurable, $g_n \longrightarrow g$ a.e. and the monotinicity of integration implies $\int g_n \leq \int f_n$. Applying the bounded convergence theorem gives $\int g_n \longrightarrow \int g$. In particular, $$\liminf_{n \rightarrow \infty} \int g_n = \int g \leq \liminf_{n \rightarrow \infty} \int f_n$$ Then, taking the sup over all $g$ that satisfy the above conditions, we get $$\sup_g \left\lbrace \int g \right\rbrace = \int f \leq \liminf_{n \rightarrow \infty} \int f_n$$ $\blacksquare$ Note that since the above integrals may not be finite, the above inequality holds in the extended sense. That is, we may get statements of the form $\int f \leq \infty$ or $\infty \leq \infty$ This proof of Fatou's lemma uses the bounded convergence theorem; which uses Egorov's theorem. With Fatou's Lemma, one can prove the monotone convergence theorem and the dominated convergence theorem which are both used in proving that $L^1(X,\mu)$ is complete in its metric (Riesz-Fischer). Alternatively, the monotone convergence theorem may be proven independently of the above results. If we assume that the monotone convergence theorem has been proven, we may obtain an alternate version of Fatou's lemma.

\textit{Alternate Fatou's Lemma}: Assume $\{f_n\}$ is a sequence of positive measurable functions. Then, $$\int \liminf_{n \rightarrow \infty} f_n \leq \liminf_{n \rightarrow \infty} \int f_n $$ Noe that $\{f_n\}$ need not necessarily have a pointwise limit, yet we are still able to obtain the above estimate since the limit inferior of a sequence of real numbers always exists. Of course, the above inequality also holds in the extended sense. $textit{proof}$ :
Define the following sequence of functions $$g_k(x) \equiv \inf_{i \geq k} f_i(x)$$ By this definition, $g_k$ are measurable, $g_k \leq f_k$ and $g_k \leq g_{k+1}$ since inf's increase as the index variable $i$ increases. $\{g_k\}$ actually has a pointwise limit since $$\lim_{k \rightarrow \infty} g_k(x) = \lim_{k \rightarrow \infty} \inf_{i \geq k} f_i(x)$$ which is exactly the definition of the limit inferior, $\liminf f_i$, which always exists and is measurable (Stein p. 29). So by construction, $0 \leq g_k \nearrow \liminf f_i$. Applying the monotone convergence theorem, we get $$\lim_{k \rightarrow \infty} \int g_k = \int \liminf f_i$$ In particular, $\lim_{k \rightarrow \infty} \int g_k = \liminf \int g_k \leq \liminf \int f_k$. Therefore, $$\int \liminf f_i \leq \liminf \int f_i$$ $\blacksquare$

\textit{Corollary 1}: Given $\{f_n\}$ a sequence of positive measurable functions with $0 \leq f_n \leq f$ and $f_n \longrightarrow f$ a.e. then $$\int f_n \longrightarrow \int f$$ This corollary establishes some conditions in which the operations of integration and (pointwise) limit may be swapped.
$\textit{proof}$ :
Note that the above conditions are slightly stronger than those of Fatou's lemma. So, applying Fatou's lemma gives us that $\int f \leq \liminf_{n \rightarrow \infty} \int f_n$. On the other hand, with $f_n \leq f$, the monotinicity of integration gives us that $\int f_n \leq \int f$. A priori, $\lim_{n \rightarrow \infty} \int f_n$ may not exist, but $\limsup_{n \rightarrow \infty} \int f_n$ does and $\limsup_{n \rightarrow \infty} \int f_n \leq \int f$. We now use the fact that for any sequence $\{a_n\}$ of real numbers, $\liminf_{n \rightarrow \infty} \{a_n\} \leq \limsup_{n \rightarrow \infty} \{a_n\}$. Since $\left\lbrace \int f_n \right\rbrace$ is really just a sequence of real numbers, we get $$\int f \leq \liminf_{n \rightarrow \infty} \int f_n \leq \limsup_{n \rightarrow \infty} \int f_n \leq \int f$$ So, $\liminf_{n \rightarrow \infty} \int f_n = \limsup_{n \rightarrow \infty} \int f_n \Longrightarrow \lim_{n \rightarrow \infty} \int f_n$ exists and equals $\int f$. $\blacksquare$ \textit{Corollary 2}: If $\{f_n\}$ is a sequence of measurable functions, $f_n \longrightarrow f$ a.e. and $\exists h \in{L^1}$ s.t. $-h \leq f_n$ $\forall n \in{\mathbb{N}}$ then $\int f \leq \liminf \int f_n$. If additionally, $f_n \leq f$ then again, $\int f_n \longrightarrow \int f$ $\textit{proof}$ :
Since $-h \leq f_n$, then adding $h$ to both sides of the inequality yields $0 \leq f_n+h$. Since $h\in{L^1}$, $h$ is, in particular, measurable. So, $\{f_n+h\}$ is a sequence of positive measurable functions converging a.e. to $f+h$. Applying Fatou's lemma gives $\int f+h \leq \liminf \int(f_n + h) = \liminf \int f_n + \int h$. Since $h\in{L^1}$, $\int h \leq \int |h| < \infty$, so subtracting $\int h$ from both sides gives $\int f \leq \liminf \int f_n$.
With the additional assumption that $f_n \leq f$, then we get that $-h \leq f_n \leq f$ so $0 \leq f_n + h \leq f+h$. Applying the previous corollary to the sequence $\{f_n+h\}$ gives us the interchange of limit and integration result. $\blacksquare$ $\textit{theorem}$ :
$\textit{Monotone Convergence Theorem}$
Given $\{f_n\}$ a sequence of positive measurable functions with $0 \leq f_n \nearrow f$ a.e. then $$\int f_n \longrightarrow \int f$$ The above conditions are stronger still than those of corollary 1, so there is nothing to prove.

The next corollary to the monotone convergence theorem is a result for series.

\textit{Corollary}: Assume that $\{a_j\}$ is a positive sequence of measurable functions. Consider the series $\sum_{j=1}^\infty a_j$. Then, $$\int \sum_{j=1}^\infty a_j = \sum_{j=1}^\infty \int a_j $$ In general, the above equality holds in the extended sense and the series may not converge. However, if the above equality holds in the finite (non-extended) sense, then the integrand function $\sum_{j=1}^\infty a_j$ must be finite a.e. That is, the series $\sum_{j=1}^\infty a_j$ converges a.e.
Another interpretation is that this corollary establishes conditions in which the linearity of the integral may be generalized to countable sums. $\textit{proof}$ :
We wish to construct a sequence of functions satisfying the conditions of the monotone convergence theorem. Define $$f_N = \sum_{j=1}^Na_j$$ Then, since $a_j \geq 0$, adding more terms to the above series only increases the value. So, $ 0 \leq f_1 \leq f_2 \leq \cdots \leq f_N \leq f_{N+1} \leq \cdots \leq f = \sum_{j=1}^\infty a_j$. The monotone convergence theorem implies that $$\lim_{N \rightarrow \infty} \int f_N = \int f$$ i.e. $$\lim_{N \rightarrow \infty} \int \sum_{j=1}^N a_j = \int \sum_{j=1}^\infty a_j$$ Since the Lebesgue integral is linear for a finite number of added terms, $$ \cdots = \lim_{N \rightarrow \infty} \sum_{j=1}^N a_j \int = \sum_{j=1}^\infty \int a_j$$ $\blacksquare$ This result will be a useful tool in the proof of Riesz-Fischer.

So, we have seen that pointwise convergence is not a sufficient condition to allow us to interchange the operations of limit and integration. What kinds of conditions do we need? Consider the following examples:
1. $f_n = \frac{1}{n}$ $\forall n \in{\mathbb{N}}$. Then, $f_n \longrightarrow 0$ (actually uniformly on $\mathbb{R}$) however, $\int_{\mathbb{R}} f_n = + \infty$ $\forall n \in{\mathbb{N}}$ whereas $\int_{\mathbb{R}} 0 = 0$.
2. $f_n = \frac{1}{n} \chi_{[\frac{-n}{2},\frac{n}{2}]}$ also converges uniformly to 0 on all of $\mathbb{R}$. $\int_{\mathbb{R}} f_n = 1$ $\forall n \in{\mathbb{N}}$
3. $f_n : [0,1] \longrightarrow \mathbb{R}$ converges uniformly to zero, is it true that $\lim_{n \rightarrow \infty} \int_{\mathbb{R}}f_n = 0?$ Yes, by the monotinicity of the integral, $\int_{[0,1]} f_n \leq \sup_{0 \leq x \leq 1}|f_n| \longrightarrow 0$ as $n \rightarrow \infty$.

The third example also illustrates an important fact: (Everywhere) uniform convergence forces convergence in the sup norm, which can in turn force bounds on integrals. The key difference in the third example is that each of the $f_n$ can be "dominated" by a single function which is $L^1$.
The next theorem is one of the most useful convergence results. $\textit{theorem}$ : Lebesgue Dominated Convergence Theorem

If $\{f_n\}$ is a sequence of measurable functions s.t. $|f_n| \leq g \in{(L^1,\mathbb{R}^d)}$ and $f_n \longrightarrow f$ a.e. then $$\int |f_n-f| \rightarrow 0$$ If we have the notion of the normed space $(L^1, ||\cdot||_1)$ then the conclusion of the DCT can be regarded as $||f_n-f||_1 \longrightarrow 0$. So in another sense, the DCT can be regarded as a theorem that establishes conditions in which pointwise convergence becomes convergence in the $L^1$ norm. For general measurable functions, neither this implication nor its converse are true. (Stein 2.8)

Example problem: Suppose $f_n: \mathbb{R}^d \longrightarrow [0,\infty]$ are measurable $\forall n\in{\mathbb{N}}$, $f_1 \geq f_2 \geq f_3 \geq \cdots \geq 0 $, $f_n \longrightarrow f$ a.e. and $f_1 \in{L^1}$. Show that $\lim_{n \rightarrow \infty} \int f_n = \int f$.
The dominated convergence theorem implies that $$\int |f_n-f| \longrightarrow 0$$ Using the triangle inequality and linearity of the integral, $$0 \leq \left| \int f_n - \int f \right|=\left| \int (f_n-f)\right| \leq \int |f_n-f| \longrightarrow 0$$ forces $$\int f_n \longrightarrow \int f$$ $\textit{proof}$ :
To show that $ \lim_{n \rightarrow \infty} \int |f_n-f|$ actually exists and equals zero, we will show that $\liminf \int |f_n-f| = \limsup \int |f_n-f| = 0$. This suggests that we should use Fatou's lemma in some way. Since $|f_n| \leq g$, then certainly $2g + |f_n-f| \geq 0$ and $2g-|f_n-f| \geq 0$ . Since $g\in{L^1}$, $g$ is, in particular, measurable. So, $2g \pm |f_n-f|$ being an algebraic combination of measurable functions is measurable. Also, since $f_n \longrightarrow f$ a.e. then $|f_n-f| \longrightarrow 0$ a.e. , so $2g \pm |f_n-f| \longrightarrow 2g$ a.e. Applying Fatou's lemma, $$0 \leq \int 2g \leq \liminf \int (2g + |f_n-f|) = \liminf \left( \int 2g + \int |f_n-f| \right) $$ where we are using the linearity of the integral. Recall some important properties of $\liminf $ and $\limsup$: If $\{a_n \}$ is a sequence of real numbers and $c$ is a constant, then $\liminf(c+a_n) = c+ \liminf a_n$. Since $\left\lbrace \int |f_n-f| \right\rbrace $ is really just a sequence of real numbers, then $$ \liminf \left( \int 2g + \int |f_n-f| \right) = \int 2g + \liminf \int |f_n-f|$$ Since $g\in{L^1}$, $\int 2g < \infty$ and so we are permitted to subtract $\int 2g$ from both sides of the above inequality to get $$0 \leq \liminf \int |f_n-f| $$ On the other hand, applying Fatou's lemma to the sequence $2g-|f_n-f|$, we get $$0 \leq \int 2g \leq \liminf \int (2g-|f_n-f|) = \int 2g + \liminf \left( -\int |f_n-f| \right) $$ $$0 \leq \liminf \left( -\int |f_n-f| \right)$$ We will now use two more properties of $\liminf$ and $\limsup$:
1. $\liminf a_n = - \limsup (-a_n)$
2. $\liminf a_n \leq \limsup a_n $
So, $$0 \leq \liminf \left( -\int |f_n-f| \right) = - \limsup \int |f_n-f|$$ $$0 \geq \limsup \int |f_n-f|$$ The final property of $\liminf$ and $\limsup$ we use is that $\liminf a_n \leq \limsup a_n$. Combining this property with (1) and (2), we obtain $$0 \leq \liminf \int |f_n-f| \leq \limsup \int |f_n-f| \leq 0$$ Which immediately implies that $\liminf \int |f_n-f| = \limsup \int |f_n-f|$ and so $\lim_{n \rightarrow \infty} \int |f_n-f|$ exists and equals 0.

Continuity of Measure (Stein Cor. 3.3)

Let $\{E_j\}$ be a sequence of Lebesgue measurable sets of $\mathbb{R}^d$. Then
(i) If $E_j \nearrow E = \cup_{J=1}^\infty $ then $m(E_j) \longrightarrow m(E)$
(ii) If $E_j \searrow E = \cup_{J=1}^\infty $ and $m(E_k) < \infty$ for some $k\in{\mathbb{N}}$ then $m(E_j) \longrightarrow m(E)$

Norms: a short overview

Recall: A $\textit{norm}$ (in the sense of analysis) is a map from a vector space to the non-negative real numbers
$|| \cdot ||: V \longrightarrow [0,\infty)$ with the following 3 properties :
(i) $||x|| \geq 0 \quad \forall{x \in{V}}$ and $||x||=0 \Longleftrightarrow x=0$ (positive-definite)
(ii) $||\alpha x||=|\alpha| \cdot ||x||$ $\forall \alpha\in{F}$, $x\in{V}$ (homogeneity)
(iii) $||x+y|| \leq ||x||+||y||$ (triangle inequality)

Any vector space equipped with a mapping satisfying (i)-(iii) above is called a $\textit{normed space}$

Examples:
1. The real numbers as a vector space together with the absolute value function $(\mathbb{R},|\cdot |)$ form a classic example of a normed space. Many other mappings that are defined by taking the absolute value of a real number inherit (i)-(iii) above and are thus a norm.
2. Consider the vector space of real-valued continuous functions defined on a closed interval $[a,b]$ (denoted $C[a,b]$). The norm $$||f||_p = \left(\int_a^b|f(x)|^p dx \right)^{\frac{1}{p}} \quad p\in{[1,\infty)}$$ When $p=1$, the above defines both a norm and a linear functional by the linearity and triangle inequality of the integral. The triangle inequality for the cases $ 1 < p < \infty $ is called Minkowski's inequality. For the $ p= \infty $ case, $$||f||_\infty \equiv \inf_{a \leq x \leq b}\{M \geq 0 : |f(x)| \leq M \quad a.e. \}$$ Since real-valued continuous functions on a closed bounded interval attain their sup and inf, the above norm is equivalent to $$||f||_{max} \equiv \max_{a \leq x \leq b} |f(x)| $$ When $C[a,b]$ is equipped with the norm $||f||_{max} \equiv \max_{a \leq x \leq b} |f(x)|$ this normed space is $\textit{complete}$: Sequences which are Cauchy with respect to the max norm also converge to an element in $C[a,b]$.

3. The mapping $||f||_1 \equiv \int_X|f(x)|d\mu(x)$ forms a $\textit{pseudonorm}$ on the vector space of integrable functions $L^1(X, \mu)$. Regarding (i), we may have $||f||=0$ for some $f$ that is not identically zero as long as $f = 0$ $\mu$-almost everywhere. e.g. The Dirichlet function $\chi_{\mathbb{Q}}=0$ $m-a.e.$ where $m$ is the usual Lebesgue measure on $\mathbb{R}$. To make a norm, we define the equivalence relation $f$~$g$ if $\mu(\{x|f(x) \neq g(x)\})=0$. Then, the set of equivalence classes of $L^1$ functions together with $|| \cdot ||$ forms a normed space.

Regarding (ii) above, the homogeneity requirement of norms has important implications:
$\textit{The powers of the norms must agree} $: Consider the following inequality
$||f||_X^2 \leq c||f||_Y$ If this inequality is true for all $f$, then in particular it is true if we replace $f$ with $\alpha f$.
$||\alpha f||_X^2 \leq c ||\alpha f||_Y$
$(|\alpha| ||f||_X)^2 \leq c |\alpha| ||f||_Y$
$|\alpha|^2 ||f||^2 \leq c |\alpha| ||f||_Y$
$|\alpha| ||f||_X^2 \leq c||f||_Y$
Letting $\alpha \longrightarrow 0$, we get $0 \leq c||f||_Y$...which is trivial. Also,
$||f||_X^2 \leq \frac{c}{|\alpha|} ||f||_Y$. Letting $\alpha \longrightarrow + \infty$, we get $||f||_X^2 = 0 \Longleftrightarrow f = 0$...nonsense!
Additionally, $\textit{the powers of the internal constants must agree}$. Consider the inequality
$||\alpha f|| \leq c||f||$. If true for all $f$, then again replace $f$ with $\alpha f$
$||\alpha \alpha f|| \leq c|| \alpha f|| $
$ | \alpha|^2 ||f|| \leq c |\alpha| ||f||$
$||f|| \leq \frac{c}{|\alpha |} ||f||$. Letting $| \alpha | \longrightarrow + \infty $ , we get
$||f|| = 0$...nonsense!

Regarding (iii) norms also satisfy the reverse triangle inequality: $$\left| ||x||-||y|| \right| \leq ||x-y|| \quad \forall x,y \in{(V,|| \cdot ||)} $$ Let $x,y$ be in a normed space and let $||x|| > ||y||$ for if they are equal, then the reverse triangle inequality is trivial.
$||x+y|| \leq ||x|| + ||y||$
$ ||x+y|| - ||y|| \leq ||x||$. $\leftarrow$ since this is true forall $x,y$, it is also true if we replace $x$ with $x-y$.
$||(x-y)+y|| - ||y|| \leq ||x-y||$
$||x||-||y|| \leq ||x-y||$. With both sides being non-negative, we may put absolute values on the left hand side
$\left| ||x||-||y|| \right| \leq ||x-y||$
Note that if we started with the above line, we could have reversed each of the above steps to obtain the triangle inequality. So, these two inequalities are equivalent in any normed space.

Application of the Arzela-Ascoli theorem

$\textbf{Application of the Arzel{\'a}-Ascoli Theorem:}$

Let $\{f_n\}_{n=1}^\infty$ be a sequence of measurable functions on [0,1] satisfying $\int_0^1 f_n^2(t)dt \leq M$ $\forall n\in{\mathbb{N}}$ and some $M \geq 0$. Define $g_n(x) = \int_0^xf_n(t)dt$. Show that $\exists \{g_{n_k}\} \subseteq \{g_n\}$ s.t. $g_{n_k} \longrightarrow g$ uniformly.

$\textit{Recall}$ : The Arzel{\'a}-Ascoli theorem states that if $\{g_n\}$ is an equicontinuous sequence of real-valued functions defined on a compact metric space that are uniformly bounded: $|g_n(x)| \leq M$ $\forall n$ and some $M \geq 0$, then, $\exists \{g_{n_k}\}$ s.t. $g_{n_k} \longrightarrow g\in{C(X)}$ uniformly.

$([0,1],|\cdot | )$ is a compact metric space...check. So, if we can show that $\{g_n\}$ is uniformly bounded and equicontinuous, then the Arzel{\'a}-Ascoli theorem will apply and give us the desired conclusion.
We now want to show that $\{g_n\}$ is uniformly bounded: By assumption, $f_n\in{L^2[0,1]}$. Let $h=1$ on $[0,1]$. Then H{\"o}lder's inequality with p=q=2 (Cauchy-Schwarz) gives us that $$||f_n \cdot 1||_1 \leq ||f_n||_2 \cdot ||1||_2$$ $$\int_0^1|f_n \cdot 1| \leq \left(\int_0^1|f_n|^2\right)^{\frac{1}{2}} \left(\int_0^1|1^2|\right)^{\frac{1}{2}} = \left(\int_0^1|f_n|^2\right)^{\frac{1}{2}}$$ Squaring both sides, $$\left(\int_0^1|f_n|\right)^2 \leq \int_0^1|f_n|^2 \leq M$$ Thus, $f\in{L^1[0,1]}$. Note that we actually showed that $L^p[a,b] \subseteq L^1[a,b] \quad \forall 1 \leq p < \infty$. To compare these inequalities with the $g_n$ terms, $$|g_n|^2=\left|\int_0^xf_n\right|^2 \leq \left(\int_0^x|f_n|\right)^2 \leq \left(\int_0^1|f_n|\right)^2 \leq M$$ $$|g_n|^2 \leq M \Longrightarrow |g_n| \leq \sqrt{M}$$ Thus, $\{g_n\}$ is a uniformly bounded sequence...check.
Now consider $0\leq x < z \leq 1$. $$|g_n(x)-g_n(z)|=\left|\int_0^xf_n - \int_0^z f_n \right|=\left|\int_x^zf_n\right|\leq \int_x^z|f_n \cdot 1|$$ Using H{\"o}lder's inequality again, $$\leq \left(\int_x^z|f_n|^2\right)^{\frac{1}{2}}\left(\int_x^z|1|^2\right)^{\frac{1}{2}} = \left(\int_x^z|f_n|^2\right)^{\frac{1}{2}}\sqrt{|x-z|}\leq \left(\int_0^1|f_n|^2\right)^{\frac{1}{2}}\sqrt{|x-z|} $$ $$ \leq \sqrt[4]{M}\sqrt{|x-z|}$$ So, $|g_n(x)-g_n(z)| \leq \sqrt[4]{M}\sqrt{|x-z|}$ which gives us a Lipschitz condition on $g_n$ $\forall n$. In the $\epsilon-\delta$ criterion for equicontinuity, if we choose $\delta \leq \frac{\epsilon^2}{\sqrt{M}}$ then $|g_n(x)-g_n(z)| \leq \epsilon$ making $\{g_n\}$ uniformly equicontinuous.