Sets and Probability Measures

Event as sets

Probability studies the repeatable (or ideal) experiments. The result of an experiment is called an outcome.

Definition 1. The set of all possible outcomes of an experiment is called the sample space and is denoted by $Ω$ .

Cardinality of sets

The cardinality of a set $Ω$ refers to the number of elements in this set, and is denoted by $card (Ω)$ or $| Ω |$ .

For finite sets, $card (Ω)$ is a natural number.
For the integer set, $card (Z) = ℵ_{0}$ , which is countable.
For the real number set, $card (R) = ℵ_{1} > ℵ_{0}$ .

The power set of a set $Ω$ is the set of all subsets, which is denoted by $2^{Ω}$ .

For $Z$ , $card (2^{Z}) = ℵ_{1} = card (R)$ .

In practical, sets in probability are: finite, countable, reals and their variants.

When we conduct an experiment, we want to know whether a subset occurs or not. For example, if we take a number from $R$ , the probability will be 0 because $card (R)$ is very large. Therefore, we are interested in $A \subset Ω$ in probability, where $A$ is a collection of subsets in probability.

Events and fields

Definition 2. The events are subsets of the sample space $Ω$ .

Remark. $\emptyset$ is called the impossible event. The set $Ω$ is called the certain event. Events $A$ and $B$ are called disjoint if their intersection is the empty set.

We do not need all the subsets of $Ω$ be events. It suffices for us to think of the collection of events as a subcollection $F$ of the set of all subsets of $Ω$ .

Definition 3. Any collection $F$ of subsets of $Ω$ which satisfies the following three conditions is called a field:

if $A, B \in F$ , then $A \cup B \in F$ and $A \cap B \in F$ (actually $A \cap B \in F$ is redundant);
if $A \in F$ , then $A^{c} \in F$ ;
the empty set $\emptyset$ belongs to $F$ .

Remark. Some implications from the properties of a filed $F$ :

if $A_{1}, \dots, A_{n} \in F$ , then $⋃_{i = 1}^{n} A_{i} \in F$ ;
$\emptyset^{C} = Ω \in F$ ;
$A \cap B = (A^{c} \cup B^{c})^{C} \in F$ .

what if we have an $Ω$ such that $card (Ω) = ℵ_{0}$ ? In this case, we may need $2^{Ω}$ to study the probability.

Definition 4. A collection $F$ of subsets of $Ω$ is called a $σ$ -field if it satisfies the following conditions:

$\emptyset \in F$ ;
if $A \in F$ , then $A^{c} \in F$ ;
if $A_{1}, A_{2}, \dots \in F$ , then $⋃_{i = 1}^{\infty} A_{i} \in F$ .

Remark. $σ$ -fields are closed under the operation of taking countable intersections.

Example. The smallest $σ$ -field associated with $Ω$ is the collection $F = {\emptyset, Ω}$ .

If $A \subset Ω$ , then $F = {\emptyset, A, A^{c}, Ω}$ is a $σ$ -field.

The power set of $Ω$ is a $σ$ -field.

With any experiment we may associate a pair $Ω, F$ , where $Ω$ is the set of all possible outcomes or elementary events and $F$ is a $σ$ -field of subsets of $Ω$ which contains \emph{all the events in whose occurrences we may be interested}; henceforth, to call a set $A$ an event is equivalent to asserting that $A$ belongs to the $σ$ -field in question.

Probability

Assume we have a “repeatable” experiment and we repeat the experiment a large number $N$ of times. Let $A \in Ω$ and $N (A)$ be the number of $A$ occurs in the $N$ trails. Intuitively, the ratio $N (A) / N$ appears to converge to a constant limit as $N$ increases. In practice, we have

$0 \leq N (A) / N \leq 1$ ;
if $A, B$ are disjoint, then $N (A \cup B) = N (A) + N (B)$ . (finite additive and countably additive)

Definition 5. A probability measure $P$ on ${Ω, F}$ is a function $P : F \to [0, 1]$ satisfying

$P (\emptyset) = 0$ ;
$P (Ω) = 1$ ;
if $A_{1}, A 2, \dots$ is a collection of disjoint members of $F$ , in that $A_{i} \cap A_{j} = \emptyset$ for all pairs $i, j$ satisfying $i \neq j$ , then $P (⋃_{i = 1}^{\infty} A_{i}) = \sum_{i = 1}^{\infty} P (A_{i}) .$

The triple $Ω, F, P$ , comprising a set $Ω$ , a $σ$ -field $F$ of subsets of $Ω$ , and a probability measure $P$ on $Ω, F$ , is called a \textbf{probability space}. We can associate a probability space $Ω, F, P$ with any experiment, and all questions associated with the experiment can be reformulated in terms of this space.

Remark. A probability measure is a special example of what is called a \emph{measure} on the pair ${Ω, F}$ . A measure is a function $μ : F \to [0, \infty)$ satisfying $μ (\emptyset) = 0$ together with (c) above. A measure $μ$ is a probability measure if $μ (Ω) = 1$ .

Lemma. We can deduce some lemmas from the definition.

$P (A^{c}) = 1 - P (A)$ ;
if $A \subseteq B$ , then $P (B) = P (A) + P (B ∖ A) \geq P (A)$ ;
$P (A \cup B) = P (A) + P (B) - P (A \cap B)$ ;
more generally, if $A_{1}, A_{2}, \dots, A_{n}$ are events, then $\begin{aligned} P (⋃_{i = 1}^{n} A_{i}) = & \sum_{i} P (A_{i}) - \sum_{i < j} P (A_{i} \cap A_{j}) + \sum_{i < j < k} P (A_{i} \cap A_{j} \cap A_{k}) - \dots \\ + (- 1)^{n + 1} P (A_{1} \cap A_{2} \cap \dots \cap A_{n}) \end{aligned}$

Lemma. Let $A_{1}, A_{2}, \dots$ be an increasing sequence of events, so that $A_{1} \subset A_{2} \subset A_{3} \subset \dots$ , and write $A$ for their limit: $A = ⋃_{i = 1}^{\infty} A_{i} = lim_{i \to \infty} A_{i} .$ Then $P (A) = lim_{i \to \infty} P (A_{i})$ . Similarly, if $B_{1}, B_{2}, \dots$ is a decreasing sequence of events, so that $B_{1} \supseteq B_{2} \supseteq B_{3} \supseteq \dots$ , then $B = ⋂_{i = 1}^{\infty} B_{i} = lim_{i \to \infty} B_{i}$ satisfies $P (B) = lim_{i \to \infty} P (B_{i})$

Some useful concepts

Conditional probability

What if we only care how many times does $A$ occur only when $B$ occurs? $N (A \cap B) / N (B)$ , the universe is changed.

Definition. If $P (B) > 0$ , then the \textbf{conditional probability} that $A$ occurs given that $B$ occurs is defined to be $P (A | B) = \frac{P (A \cap B)}{P (B)} .$

Definition 6. Suppose ${B_{i}}$ is a finite set. If $B_{i}$ are all disjoint and $⋃_{i = 1}^{n} B_{i} = Ω$ , then ${B_{i}}$ is called a \textbf{partition} of $Ω$ .

Lemma. For any events $A$ and $B$ such that $0 < P (B) < 1$ , $P (A) = P (A | B) P (B) + P (A | B^{c}) P (B^{c}) .$ More generally, let $B_{1}, B_{2}, \dots, B_{n}$ be a partition of $Ω$ such that $P (B_{i}) > O$ for all $i$ . Then $P (A) = \sum_{i} P (A | B_{i}) P (B_{i}) .$

Independence

Intuition: an event $A$ occurs doesn’t affect the probability of $A$ occurs when $B$ occurs, which means $P (A) = P (A | B)$ .

Definition 7. Events $A$ and $B$ are called independent if $P (A \cap B) = P (A) P (B) .$ More generally, a family ${A_{i} | i \in I}$ is called independent if $P (⋂_{i \in J} A_{i}) = \prod_{i \in J} P (A_{i})$ for all finite subsets $J$ of $I$ .

Remark. A common student error is to make the fallacious statement that $A$ and $B$ are independent if $A \cap B = \emptyset$ .

Remark. If the family ${A_{i} | i \in I}$ has the property that for all $P (A_{i} \cap A_{j}) = P (A_{i}) P (A_{j}) \forall i \neq j,$ then it is called \emph{pairwise independent}. Pairwise-independent families are not necessarily independent.

Completeness and product space

Lemma. If $F$ and $G$ are two $σ$ -fields of subsets of $Ω$ , then their intersection $F \cap G$ is a $σ$ -field also. More generally, if ${F_{i} | i \in I}$ is a family of $σ$ -fields of subsets of $Ω$ , then $G = ⋂_{i \in I} F_{i}$ is a $σ$ -field also.

Completeness: Let $Ω, F, P$ be a probability space. Any event $A$ which has zero probability, that is $P (A) = 0$ , is called \emph{null}. It may seem reasonable to suppose that any subset $B$ of a null set $A$ will itself be null, but this may be without meaning since $B$ may not be an event, and thus $P (B)$ may not be defined.

Definition 8. A probability space ${Ω, F, P}$ is called complete if all subsets of null sets are events.

Any incomplete space can be completed thus. Let $N$ be the collection of all subsets of null sets in $F$ and let $G = σ (F \cup N)$ be the smallest $σ$ -field which contains all sets in $F$ and $N$ . It can be shown that the domain of $P$ may be extended in an obvious way from $F$ to $G$ ; $Ω, G, P$ is called the completion of $Ω, F, P$ .

Product space: Suppose two experiments have associated probability spaces $Ω_{1}, F_{1}, P_{1}$ and $Ω_{2}, F_{2}, P_{2}$ respectively. The sample space of the pair of experiments, considered jointly, is the collection $Ω_{1} \times Ω_{2} = (ω_{1}, ω_{2}) | w_{1} \in Ω_{1}, ω_{2} \in Ω_{2}$ of ordered pairs. The appropriate $σ$ -field of events is \textbf{more complicated} to construct. \textbf{The family of all such sets, $F_{1} \times F_{2} = A_{1} \times A_{2} | A_{1} \in F_{1}, A_{1} \in F_{2}$ , is NOT in general a $σ$ -field.

Remark. There exists a unique smallest $σ$ -field $G = σ (F_{1} \times F_{2})$ of subsets of $Ω_{1} \times Ω_{2}$ which contains $F_{1} \times F_{2}$ . All we require now is a suitable probability function on $(Ω 1 \times Ω_{2}, G)$ . Let $P_{12} : F_{1} \times F_{2} \to [0, 1]$ be given by: $P (A_{1} \times A_{2}) = P_{1} (A_{1}) \times P_{2} (A_{2}) for A_{1} \in F_{1}, A_{2} \in F_{2} .$ It can be shown that the domain of $P_{12}$ can be extended from $F_{1} \times F_{2}$ to the whole of $G = σ (F_{1} \times F_{2})$ .

Definition 9. The probability space $(F_{1} \times F_{2}, G, P_{12})$ is called the product space of $(Ω_{1}, F_{1}, P_{1})$ and $(Ω_{2}, F_{2}, P_{2})$ . The measure $P_{12}$ is sometimes called the `product measure'.