Sigma Algebra

Sigma Algebra

TLDR

We want to keep in mind the subset of $\mathbf{R}^n$ that obey the measure.
The most critical component that is the idea of countability of sets and its sum of measures. Thus we introduce the notion of $\sigma$-algebras and algebras.
$\sigma$-algebras provide a notion of “information”, and certain $\sigma$-algebras are smaller than others.
Borel $\sigma$-algebras are the smallest $\sigma$-algebras generated by open sets of $\mathbf{R}^n$. We can show that the same $\sigma$-algebras can be generated by a variety of types of sets.
There are several ways to construct $\sigma$-algebras from combining basic set algebras.

Introducing the Sigma Algebra

We should keep in mind the type of subsets of real numbers that we can measure. To do so, we define a logic on objects that can be measured, i.e. $\mathcal{L}^n$. Special types of sets impose a logic on the objects in the collection. As such, the first “logic” to be defined is the concept of an algebra.

Definition: Given an ambient space $\Omega$, an algebra $A$ endowed on the space satisfies the following constraints:

$\varnothing \in A$
If $X \in A$ then $X^C \in A$
If $X_1,…,X_n$ is a collection of finite countable sets in $A$, then its union $\cup_{i=1}^n X_i \in A$

If we change condition (3) to being just countable sets (finite or infinite), $A$ becomes a $\sigma$-algebra and is usually denoted with script $\mathcal{A}$. That is:

Definition: Given an ambient space $\Omega$, a $\sigma$-algebra $\mathcal{A}$ endowed on the space satisfies the following constraints:

$\varnothing \in \mathcal{A}$
If $X \in \mathcal{A}$ then $X^C \in \mathcal{A}$
If $X_1,…,X_n$ is a collection of finite countable sets in $\mathcal{A}$, then its union $\cup_{i=1}^n X_i \in \mathcal{A}$

Note that any algebra is a $\sigma$-algebra but not all $\sigma$-algebras as algebras.

Exercise: Prove that the intersection of two $\sigma$-algebras, $\mathcal{E}, \mathcal{F}$ both endowed on $\Omega$, forms a $\sigma$-algebra. Hint: show that property (1), (2), and (3) all hold in $\mathcal{E}\cap \mathcal{F}$.

Exercise: Prove that $\sigma$-algebras are closed under countble intersections. Hint: Define an arbitrary $\sigma$-algebra and use Demorgans law to show that if countable sets $X_1,…$ are in the $\sigma$-algebra, then its intersection must also be in the $\sigma$-algebra.

Special Sigma Algebras

The power set of any set is clearly a $\sigma$-algebra. The trivial $\sigma$-algebra on any set $\Omega$ is ${\varnothing, \Omega}$. Another trivial $\sigma$-algebra is ${\varnothing, X, X^C, \Omega}$ for any $X \in \Omega$.

We can even generate $\sigma$-algebras by taking intersections. As seen in the previous exercise, any two $\sigma$-algebras endowed on $\Omega$ is still a $\sigma$-algebra. Suppose $X$ is an element in $\Omega$. What is the smallest $\sigma$-algebra that still contains $X$?

This is a question of interest since we would ideally like to consider $\sigma$-algebras that contain $X$, which means it must contain $X^C$ and all countable unions that involve $X$ or $X^C$. We denote this as $\sigma(X)$.

Definition: Let $\mathcal{G}$ be the collection of all possible $\sigma$-algebras that can be made on $\Omega$ that contain $X$. The smallest $\sigma$-algebra endowed on $\Omega$ that contains a set $X\in\Omega$ is:

\[\sigma(X) = \bigcap_{\mathcal{F} : \mathcal{F} \in \mathcal{G}} \mathcal{F}\]

In otherwords, it is the intersection of all $\sigma$-algebras that contain $X$.

Why Sigma Algebras and Smallest Sigma Algebras

At this point, we should wonder why we care about $\sigma$-algebras. $\sigma$-algebras have the unique property in that they are closed under countable unions. Futhermore, we can think of this as related to measure theory in that:

The measure of nothing should be 0, and the measure of the entire set can be $\infty$ (or 1 in the case of probability). In fact, the measure of the entire set of interest can be any real number.
If we can measure one object, we should also be able to collectively measure everything but the object.
If we can measure each object in a potentially infinite and countable set, then we should be able to measure the combination of all of items in the set collectively.

Dealing with the smallest $\sigma$-algebra provides a notion of how much information is in a set. For example, suppose I draw random integers from ${1,2,3,4}$. I can generate the smallest $\sigma$-algebra from the set ${(1,2), (3,4)}$. This $\sigma$-algebra will be:

\[\mathcal{F} := \{\varnothing, \{1,2\}, \{3,4\}, \{1,2,3,4\} \}\]

Now suppose I generate the smallest $\sigma$-algebra from ${{1}, {2}, {3}, {4}}$. That is, I want the smallest $\sigma$-algebra that contains each possible outcome. Then the $\sigma$-algebra will be”

\[\mathcal{G} := \{\varnothing, \{1\}, \{2\}, \{3\}, \{4\}, \{1,2\}, \{2, 3\}, \{1,4\}, \{3,4\}, ..., \{1,2,3,4\} \}\]

It will have $2^4$ elements in it, and note that this $\sigma$-algebra is contains the previous one, i.e. $\mathcal{F} \subset \mathcal{G}$. We say that $\mathcal{G}$ is finer than $\mathcal{F}$. We also say that $\mathcal{G}$ provides more information that $\mathcal{F}$.

To see this, imagine someone stuck in a room and is drawing random numbers. You this person a set of index card, and each index card corresponds to an element of $\mathcal{F}$. They must return you the index card corresponding to the smallest element of $\mathcal{F}$ that contains the number from their draw. So if they draw a 2, they return the index card corresponding to ${1,2}$.

Now imagine there is another person stuck in another room, and they are playing the same random game with the same rules, except this time, they are given index card, where each card corresponds to an element of $\mathcal{G}$. They must return you the index card corresponding to the smallest element of $\mathcal{G}$ that contains the number of their draw.

In both rooms, the same game is being played, but you receive index cards with differing information. Clearly, the index cards from the $\mathcal{G}$ regime is more informative, as when a 2 is drawn, you will receive the card corresponding to ${2}$.

It is obvious that for any finite set, its power set is the finest $\sigma$-algebra. If all sets we want to learn about were finite, there would be no need to study measure theory. But obviously, real numbers are uncountably infinite, and rational numbers of countably infinite, so if we want to measure something as simple as an interval of these objects, we would need to study measure theory, and consider the smallest $\sigma$-algebras generated by sets of interest, i.e. $\sigma((0,1])$.

Exercise: Prove that $\mathcal{E} := {E \subseteq \mathbf{R}: E \text{ or }E^C \text{ is countable}}$ is a $\sigma$-algebra. Hint: We showed closure under countable intersections. We can use it to show $\mathcal{E}$ is also a $\sigma$-algebra.

Exercise: Prove that if $X \in \sigma(G)$ then $\sigma(X) \subseteq \sigma(G)$. Prove that is $X \subseteq G$, then $\sigma(X) \subseteq \sigma(G)$.

On Real Number Spaces

The Borel $\sigma$-algebras are the smallest $\sigma$-algebras that can be generated by open sets on $\mathbf{R}^n$. We denote this as $\mathcal{B}(\mathbf{R}^n)$. In general, the Borel $\sigma$-algebra can be defined on any set, and is defined as the smallest $\sigma$-algebra that is generated by all open sets of the given space. The open set in this case is defined with respect to any defined topology.

Thus the power set topology of ${1,2,3,4}$ is a $\sigma$-algebra (since power sets are $\sigma$-algebras), and it is also a Borel $\sigma$-algebra.

Now, we can generate Borel sets in various manners as well. For example, $\mathcal{B}(\mathbf{R})$ is the exact same set as:

$\sigma($ All closed intervals of $\mathbf{R})$
$\sigma($ All half closed intervals of $\mathbf{R})$
$\sigma($ All intervals of $\mathbf{R}$ with rational end points $)$

Exercise: Prove the 3 claims above. Hint: Use closure under countable unions and intersections. Use the result of the previous exercise to show $\subseteq \mathcal{B}(\mathbf{R})$ and $\supseteq \mathcal{B}(\mathbf{R})$.

Exercise: Provide a counterexample to show that a countable union of $\sigma$-algebras on subsets of $\Omega$ need not be a $\sigma$-algebra. Show it is an algebra. Show that a finite union of $\sigma$-algebras is a $\sigma$-algebra. Hint: The goal is to violate the rule that $\Omega \in \mathcal{A}$ if $\mathcal{A}$ is a $\sigma$-algebra.

Monotone Classes and Smallest Sigma Algebras

We introduce a new logical structure, the monotone class.

Definition: Let $\mathcal{M}$ be a collection such that:

If $A_1 \subseteq A_2 \subseteq …$, then $\cup A_i \in \mathcal{M}$
If $B_1 \supseteq B_2 \supseteq …$, then $\cap B_i \in \mathcal{M}$

Then $\mathcal{M}$ is a monotone class.

The smallest monotone class containing $X$, i.e. $\mathcal{M}(X)$ is defined the same way as we do for smallest $\sigma$-algebras.

Theorem: (Monotone Class Theorem) The smallest monotone class containing algebra $G$ is equivalent to the smallest $\sigma$-algebra containing $G$.

Proof: $\sigma(G)$ is clearly a monotone class due to closure under countable unions and intersections, which apply to monotone increasing or decreasing sets. But at this point we do not know if it is the smallest monotone set. So $\sigma(G) \supseteq \mathcal{M}$. Now we show the $\subseteq$ relation.

Let $\mathcal{D} := {E \in \mathcal{M} : E^C \in \mathcal{M}}$. We show that $\mathcal{D}$ is a monotone class.

$\mathcal{D} \subseteq \mathcal{M}$ is true by definition of $\mathcal{D}$. Now consider a sequence of increasing sets $A_1, A_2,…$ that exist in $\mathcal{M}$. $A_\infty$ must be in $\mathcal{M}$ by property of the monotone class. Now suppose in $\mathcal{M}$ we have $A_1^C, A_2^C,…$ are decreasing sets. $A_\infty^C$ is in $\mathcal{M}$ as well. Thereforem, $A_1, A_1^C,…$ and $A, A^C$ are in $\mathcal{D}$ since $\mathcal{D}$ collects all complementary sets. Thus $\mathcal{D}$ is a monotone class that contains the algebra $G$ (argument being that any finite union’s complement is also collected by $\mathcal{D}$). That is $\mathcal{D}$ is a monotone class that contains $G$, so $\mathcal{D} \supseteq \mathcal{M}$, so $\mathcal{D} = \mathcal{M}$. So $\mathcal{M}$ is closed under complementation.

Now, let $\mathcal{D} := {E \in \mathcal{M}: E \cap B \in \mathcal{M};\; \forall B \in G}$.

Again, $\mathcal{D} \subseteq \mathcal{M}$. Suppose we have $A_1 \subseteq A_2 \subseteq … $ in $\mathcal{M}$. Then $A_1 \cap B, A_2 \cap B,…$ are also increasing sets. Thus $A_\infty \cap B$ must be in $\mathcal{M}$ by property of monotone classes. Therefore, if $A_1,…$ are in $\mathcal{D}$, then $A_\infty$ must also be in $\mathcal{D}$. As $B\in G$, then $\mathcal{D}$ contains $G$ and is a monotone class once we argue for decreasing sets (via complementation of $A_1,…$). So $\mathcal{D} \supseteq \mathcal{M}$, therefore $\mathcal{D} = \mathcal{M}$. We can argue similarly for $\mathcal{D} := {E \in \mathcal{M} : E \cap B \in \mathcal{M}\; \forall B \in \mathcal{M}}$, so that $\mathcal{D} = \mathcal{M}$.

So $\mathcal{M}$ is identical to sets that are at least closed under complementation, and closed under finite intersections. Now consider any finite colection of sets $A_1,…,A_n \in \mathcal{M}$. Define $B_k = \cap_{i=1}^n A_i$. Thus $B_n \supseteq B_{n+1}$, and note that by closure under finite intersections, $B_n \in \mathcal{M}$ for any $n$. Since $\mathcal{M}$ is a monotone class, $B_\infty$ must also be in $\mathcal{M}$. This provides demonstrates closure under countable intersections.

So $\mathcal{M}$ is closed under complementation, and countable intersection, so it is a $\sigma$-algebra, that obviously contains $G$. Thus $\sigma(G) \subseteq \mathcal{M}$. By the first line, we have enough to say that $\sigma(G) = \mathcal{M}$. QED

Proof Technique Discussion

This proof contains several elements of typical proofs in measure theory.

We work with a simple set, i.e. the several definitions $\mathcal{D}$.
Each iteration of defining and redefining $\mathcal{D}$ is defined in a way to fulfill the requirements of an algebra.
- Fulfilling the requirements of an algebra allows us to say that $\mathcal{D}$ contains the algebra, and then we demonstrate $\mathcal{D}$ is a monotone class to equate $\mathcal{D} = \mathcal{M}$. This establishes algebra closure properties of $\mathcal{M}$ that ultimately derived from the contained algebra.
We expand the algebra properties by enhancing them with monotone class properties, to demonstrate the $\sigma$-algebra properties hold.

In other words, suppose we wish to show an “infinite” property holds for a set that has limited infinite properties (monotone class only requires closure of countable nested sequences whereas $\sigma$-algebras require closure of countable unions of any set). Then we do so by demonstrating:

Finite verions of the “infinite” properties hold.
Use the limited infinite properties to generalize to the desired inifinite property.

$\pi$ and $\lambda$ Systems

Sometimes, we do not wish to work with set logics that are as demanding as an algebra. We introduce two of such set logics:

Definition: $\mathcal{P}$ is a $\pi$-system if $A, B \in \mathcal{P}$ means that $A \cap B$ is in $\mathcal{P}$ too. That is, closure under finite unions.

Definition: $\mathcal{L}$ is a $\lambda$-system if:

$\varnothing \in \mathcal{L}$
If $A \in \mathcal{L}$ then $A^C \in \mathcal{L}$
If $A_1,A_2,…$ are all disjoint, then $\sqcup A_i \in \mathcal{L}$ too.

That is, every $\sigma$-algebra must be a $\lambda$-system, since $\sigma$-algebras require closure under disjoint unions as well. But the reverse need not be true. Counter examples are easy.

Exercise: Provide a counter example of a $\lambda$-system that is not a $\sigma$-algebra.

Lemma: Every $\pi$-system that is also a $\lambda$-system forms a $\sigma$-algebra.

Proof: Let $X$ be a $\pi$ and $\lambda$-system. $\varnothing$ and complementation is free since it is in the $\lambda$-system. Consider any set $A_1,…,A_n \in X$. Let $B_1 = A_1, B_2 = A_2 \setminus A_1$, and in general $B_k = A_k \setminus \cup_{j=1}^{k-1}A_{j}$.

$B_k$ is in $X$ since $A_k \in X$ and set differencing can be written as follows: $\setminus \cup_{i=1}^{k-1} A_i = \cap (\cup_{i=1}^{k-1} A_i)^C = \cap_{i=1}^{k-1} A_i^C$. As $A_1,…,A_n \in X$, the complements $A_i^C$ are in $X$ by the $\lambda$-system property. By the $\pi$-system property, finite intersections are closed. So for any finite $k$, $B_k \in X$. $B_\infty \in X$ by the $\lambda$-system closure of countable disjoint unions.

Now note that $B_\infty = A_\infty$, $A_\infty \in X$. Thus $X$ is closed under countable unions, and thus is a $\sigma$-algebra. QED

Lemma: A $\lambda$-system is closed under set differences i.e. if $A, B \in \mathcal{L}$ and $A \supset B$, then $A \setminus B \in \mathcal{L}$.

Proof: First, note that $A \setminus B = A \cap B^C$ and we can also use DeMorgans to write this as $(A^C \cup B)^C$. Now, $A^C \cup B \in \mathcal{L}$ since $A^C$ and $B$ are disjoint, indeed, because $A \supset B$. And since this union is in $\mathcal{L}$, its complement is in $\mathcal{L}$. So $A \setminus B \in \mathcal{L}$. QED

Exercise: An intersection of any $\lambda$-systems is still a $\lambda$-system. Hint: This proof follows closely with the proof on intersections of $\sigma$-algebras.

Exercise: Let $P$ be a $\pi$-system, and $l(P)$ be the smallest $\lambda$-system that contains $P$, both endowed on $X$. Show that $l(P)$ is a $\sigma$-algebra.

Hint: By the first lemma above, it is sufficient to show that $l(P)$ is a $\pi$-system. Do this by defining $\mathcal{D} :={E \subset X : A \cap B \in l(P);\; \forall A \in l(P)}$ and showing it is a $\lambda$-system that contains $P$. Then show that $\mathcal{D}$ is closed under finite intersections, and you are done.

Theorem: (Dynkin’s $\pi-\lambda$ Theorem): Let $P$ be a $\pi$-system on $\Omega$ and $L$ be a $\lambda$-system on $\Omega$. Also suppose $P \subset L$. Then $\sigma(P) \subseteq L$.

Proof: We start with $P \subseteq L$. As $l(P)$ is the smallest $\lambda$-system containing $P$, then $l(P) \subseteq L$. Now $l(P)$ is a $\sigma$-algebra containing $P$, so $l(P) \supseteq P$, and $\sigma(P) \subset l(P) \subset L$. Thus we have shown that $\sigma(P) \subseteq L$.

As every $\sigma$-algebra containing $P$ is also a $\lambda$-system containing $P$, we also have that $L(P) \subseteq \sigma(P)$, therefore, $L(P) = \sigma(P)$. QED

This theorem is important when we discuss measures agreeing on $\sigma$-algebras, and we will revisit this later on.

Navigation Structure

Table of contents