Random Variables

Random variables

Definition 1. A random variable is a function $X : Ω \to R$ with the property that ${ω \in Ω | X (ω) \leq x} \in F$ for each $x \in R$ . Such a function is said to be $F$ -measurable.

Example 1. Tossing two dice. We define $X = {\begin{cases} 1 & get double (i = j) \\ 0 & not double (i \neq j) \end{cases} .$ Then $X = (i, j) \in R$ .

Remark. We are interested in $g (x) = P {ω | X (ω) = x}$ . But sometimes it doesn't work very well. Probability triplet is defined as ${Ω, F, P}$ . If $Ω$ is countable, then $g (x)$ is okay. But if $Ω$ is uncountable like intervals, then $P (X = x)$ doesn't make much sense because the cardinality is too large.

Definition 2. The distribution function of a random variable $X$ is the function $F : R \to [0, 1]$ given by $F (x) = P (X \leq x)$ .

Example 2. The distribution function of preceeding example is $F (x) = {\begin{cases} 0 & x \leq 0, \\ \frac{30}{36} & 0 < x \leq 1, \\ 1 & x > 1. \end{cases}$

Notice that $ω | X (ω) \leq x$ defines an event. It is an element in the corresponding $σ$ -field. Denote $A (x) = ω | X (ω) \leq x$ . Along with $A (x)$ , we can define $A^{c} (x) = {ω | X (ω) > x}, A (x, y) = A^{c} (x) \cap A (y) = {ω | x < X (ω) \leq y} .$ Tow points worth noting:

$F$ must be defined for all $x \in R$ .
$A (x)$ should belongs to $F$ . Otherwise, we cannot talk about the probability of $P (A (x))$ . Then the definition of distribution function is meaningless.

Lemma 1. A distribution function $F$ has the following properties:

$lim_{x \to - \infty} F (x) = 0$ , $lim_{x \to \infty} F (x) = 1$ ,
if $x < y$ , then $F (x) \leq F (y)$ ,
$F$ is right-continuous, that is $F (x + h) \to F (x)$ as $h ↓ 0$ . (left-continuous is not necessary)

Example 3. Indicator functions. A particular class of Bernoulli variables is very useful in probability theory. Let $A$ be an event and let $I_{A} : Ω \to R$ be the indicator function of $A$ ; that is, $I_{A} (ω) = {\begin{cases} 1 & if ω \in A, \\ 0 & if ω \in A^{c} . \end{cases}$ Then $I_{A}$ is a Bernoulli random variable taking the values 1 and 0 with probabilities $P (A)$ and $P (A^{c})$ respectively. Suppose ${B_{i} | i \in I}$ is a family of disjoint events with $A \subseteq ⋃_{i \in I} B_{i}$ . Then $I_{A} = \sum_{i} = I_{A \cap B_{i}},$ an identity which is often useful.

Lemma 2. Let $F$ be the distribution function of $X$ . Then

$P (X > x) = 1 - F (x)$ ,
$P (x < X \leq y) = F (y) - F (x)$ ,
$P (X = x) = F (x) - lim_{y ↑ x} F (y)$ .

A random variable $X$ with distribution function $F$ is said to have two “tails” given by $T_{1} (X) = P (X > x) = 1 - F (x), T_{2} (X) = P (X \leq - x) = F (- x),$ where $x$ is large and positive. The rates at which the $T_{i}$ decay to zero as $x \to \infty$ have a substantial effect on the existence or non-existence of certain associated quantities called the “moments” of the distribution.

Different random variables

Discrete random variables

Definition 3. The random variable $X$ is called discrete if it takes values in some countable subset ${x_{1}, x_{2}, \dots}$ , only, of $R$ . The discrete random variable $X$ has (probability) mass function $f : R \to [0, 1]$ given by $f (x) = P (X = x)$ .

We shall see that the distribution function of a discrete variable has jump discontinuities at the values $x_{1}, x_{2}, \dots$ and is constant in between; such a distribution is called atomic.

Continuous random variables

Definition 4. The random variable $X$ is called continuous if its distribution function can be expressed as $F (x) = \int_{- \infty}^{x} f (u) d u x \in R$ for some integrable function $f : R \to [0, \infty)$ called the **(probability) density function** of $X$ .

Some point worth noting:

The density function $f (x)$ is not unique. We can add some separate points to $f$ , and it doesn't affect the integration.
$F$ must be **absolutely continuous**. This implies $F$ is continuous. We can also deduce that the probability at certain point must be zero. \emph{i.e.}, $P (X = x) = 0$ .

There is another sort of random variable, called “singular”.

Random vectors

Definition

The random vector is a function $X : Ω \to R^{n}$ . For example, $X = (X, Y)$ for $n = 2$ . We can also define the distribution function for such $X$ . But we need to first introduce the ordering in $R^{n}$ .

Definition 5. By definition, $(x_{1}, y_{1}) < (x_{2}, y_{2})$ if and only if $x_{1} < x_{2}$ **AND** $y_{1} < y_{2}$ .

Definition 6. The joint distribution function of a random vector $X = (X_{1}, X_{2} \dots, X_{n})$ on the probability space ${Ω, F, P}$ is the function $F_{X} : R^{n} \to [0, 1]$ given by $F_{X} (x) = P (X \leq x)$ for $x \in R^{n}$ .

Remark . The joint probability $P (X \leq x) = P (X_{1} \leq x_{1}, \dots, X_{n} \leq x_{n})$ . ${X \leq x}$ is an abbreviation for the event ${ω \in Ω | X (w) \leq x}$ .

Lemma 3. The joint distribution function $F_{X, Y}$ of the random vector $(X, Y)$ has the following properties:

$lim_{x, y \to - \infty} F_{X, Y} (x, y) = 0, lim_{x, y \to \infty} F_{X, Y} (x, y) = 1$ ,
if $(x_{1}, y_{1}) \leq (x_{2}, y_{2})$ , then $F_{X, Y} (x_{1}, y_{1}) \leq F_{X, Y} (x_{2}, y_{2})$ ,
$F_{X, Y}$ is continuous from above, in that $F_{X, Y} (x + u, y + v) \to F_{X, Y} (x, y) a s u, v ↓ 0.$

Marginalization

$\begin{aligned} lim_{y \to \infty} F_{X, Y} & = F_{X} (x) = P (X \leq x), \\ lim_{x \to \infty} F_{X, Y} & = F_{Y} (y) = P (Y \leq y) . \end{aligned}$

The functions $F_{X}$ and $F_{Y}$ are called the “marginal” distribution functions of $F_{X, Y}$ . $F_{X, Y}$ can determine two marginals $F_{X}$ and $F_{Y}$ , but converse is NOT true.

Discrete and continuous distribution

Definition 7. The random variables $X$ and $Y$ on the probability space ${Ω, F, P}$ are called **(jointly) discrete** if the vector $(X, Y)$ takes values in some **countable** subset of $R^{2}$ only. The jointly discrete random variables $X, Y$ have **joint (probability) mass function** $f : R^{2} \to [0, 1]$ given by $f (x, y) = P (X = x, Y = y)$ .

Definition 8. The random variables $X$ and $Y$ on the probability space ${Ω, F, P}$ are called **(jointly) continuous** if their joint distribution function can be expressed as $F_{X, Y} (x, y) = \int_{u = - \infty}^{x} \int_{v = - \infty}^{y} f (u, v) d u d v x, y \in R,$ for some **integrable** function $f : R^{2} \to [0, \infty)$ called the **joint (probability) density function** of the pair $(X, Y)$ .