University is starting for some students who took A’levels in 2016. And, one of my ex-students told me to share/ summarise the things to know for probability at University level. Hopefully this helps. H2 Further Mathematics Students will find some of these helpful.

Random Variables

Suppose $X$ is a random variable which can takes values $x \in \chi$.

$X$ is a discrete r.v. is $\chi$ is countable.
$\Rightarrow p(x)$ is the probability of a value of $x$ and is called the probability mass function.

$X$ is a continuous r.v. is $\chi$ is uncountable.
$\Rightarrow f(x)$ is the probability density function and can be thought of as the probability of a value $x$.

Probability Mass Function

For a discrete r.v. the probability mass function (PMF) is

$p(a) = P(X=a)$, where $a \in \mathbb{R}$.

Probability Density Function

If $B = (a, b)$

$P(X \in B) = P(a \le X \le b) = \int_a^b f(x) ~dx$.

And strictly speaking,

$P(X = a) = \int_a^a f(x) ~dx = 0$.

Intuitively,

$f(a) = P(X = a)$.

Properties of Distributions

For discrete r.v.
$p(x) \ge 0 ~ \forall x \in \chi$.
$\sum_{x \in \chi} p(x) = 1$.

For continuous r.v.
$f(x) \ge 0 ~ \forall x \in \chi$.
$\int_{x \in \chi} f(x) ~dx = 1$.

Cumulative Distribution Function

For discrete r.v., the Cumulative Distribution Function (CDF) is
$F(a) = P(X \le a) = \sum_{x \le a} p(x)$.

For continuous r.v., the CDF is
$F(a) = P(X \le a ) = \int_{- \infty}^a f(x) ~dx$.

Expected Value

For a discrete r.v. X, the expected value is
$\mathbb{E} (X) = \sum_{x \in \chi} x p(x)$.

For a continuous r.v. X, the expected value is
$\mathbb{E} (X) = \int_{x \in \chi} x f(x) ~dx$.

If $Y = g(X)$, then

For a discrete r.v. X,
$\mathbb{E} (Y) = \mathbb{E} [g(X)] = \sum_{x \in \chi} g(x) p(x)$.

For a continuous r.v. X,
$\mathbb{E} (Y) = \mathbb{E} [g(X)] = \int_{x \in \chi} g(x) f(x) ~dx$.

Properties of Expectation

For random variables $X$ and $Y$ and constants $a, b, \in \mathbb{R}$, the expected value has the following properties (applicable to both discrete and continuous r.v.s)

$\mathbb{E}(aX + b) = a \mathbb{E}(X) + b$

$\mathbb{E}(X + Y) = \mathbb{E}(X) + \mathbb{E}(Y)$

Realisations of $X$, denoted by $x$, may be larger or smaller than $\mathbb{E}(X)$,

If you observed many realisations of $X$, $\mathbb{E}(X)$ is roughly an average of the values you would observe.

$\mathbb{E} (aX + b)$
$= \int_{- \infty}^{\infty} (ax+b)f(x) ~dx$
$= \int_{- \infty}^{\infty} axf(x) ~dx + \int_{- \infty}^{\infty} bf(x) ~dx$
$= a \int_{- \infty}^{\infty} xf(x) ~dx + b \int_{- \infty}^{\infty} f(x) ~dx$
$= a \mathbb{E} (X) + b$

Variance

Generally speaking, variance is defined as

$Var(X) = \mathbb{E}[(X- \mathbb{E}(X)^2] = \mathbb{E}[X^2] - \mathbb{E}[X]^2$

If $X$ is discrete:

$Var(X) = \sum_{x \in \chi} ( x - \mathbb{E}[X])^2 p(x)$

If $X$ is continuous:

$Var(X) = \int_{x \in \chi} ( x - \mathbb{E}[X])^2 f(x) ~dx$

Using the properties of expectations, we can show $Var(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$.

$Var(X)$
$= \mathbb{E} [(X - \mathbb{E}[X])^2]$
$= \mathbb{E} [(X^2 - 2X \mathbb{E}[X]) + \mathbb{E}[X]^2]$
$= \mathbb{E}[X^2] - 2\mathbb{E}[X]\mathbb{E}[X] + \mathbb{E}[X]^2$
$= \mathbb{E}[X^2] - \mathbb{E}[X]^2$

Standard Deviation

The standard deviation is defined as

$std(X) = \sqrt{Var(X)}$

Covariance

For two random variables $X$ and $Y$, the covariance is generally defined as

$Cov(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])]$

Note that $Cov(X, X) = Var(X)$

$Cov(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y]$

Properties of Variance

Given random variables $X$ and $Y$, and constants $a, b, c \in \mathbb{R}$,

$Var(aX \pm bY \pm b ) = a^2 Var(X) + b^2 Var(Y) + 2ab Cov(X, Y)$

This proof for the above can be done using definitions of expectations and variance.

Properties of Covariance

Given random variables $W, X, Y$ and $Z$ and constants $a, b, \in \mathbb{R}$

$Cov(X, a) = 0$

$Cov(aX, bY) = ab Cov(X, Y)$

$Cov(W+X, Y+Z) = Cov(W, Y) + Cov(W, Z) + Cov(X, Y) + Cov(X, Z)$

Correlation

Correlation is defined as

$Corr(X, Y) = \dfrac{Cov(X, Y)}{Std(X) Std(Y)}$

It is clear the $-1 \le Corr(X, Y) \le 1$.

The properties of correlations of sums of random variables follow from those of covariance and standard deviations above.