University is starting for some students who took A’levels in 2016. And, one of my ex-students told me to share/ summarise the things to know for probability at University level. Hopefully this helps. H2 Further Mathematics Students will find some of these helpful.

Random Variables

Suppose X is a random variable which can takes values x \in \chi.

X is a discrete r.v. is \chi is countable.
\Rightarrow p(x) is the probability of a value of x and is called the probability mass function.

X is a continuous r.v. is \chi is uncountable.
\Rightarrow f(x) is the probability density function and can be thought of as the probability of a value x.

Probability Mass Function

For a discrete r.v. the probability mass function (PMF) is

p(a) = P(X=a), where a \in \mathbb{R}.

Probability Density Function

If B = (a, b)

P(X \in B) = P(a \le X \le b) = \int_a^b f(x) ~dx.

And strictly speaking,

P(X = a) = \int_a^a f(x) ~dx = 0.


f(a) = P(X = a).

Properties of Distributions

For discrete r.v.
p(x) \ge 0 ~ \forall x \in \chi.
\sum_{x \in \chi} p(x) = 1.

For continuous r.v.
f(x) \ge 0 ~ \forall x \in \chi.
\int_{x \in \chi} f(x) ~dx = 1.

Cumulative Distribution Function

For discrete r.v., the Cumulative Distribution Function (CDF) is
F(a) = P(X \le a) = \sum_{x \le a} p(x).

For continuous r.v., the CDF is
F(a) = P(X \le a ) = \int_{- \infty}^a f(x) ~dx.

Expected Value

For a discrete r.v. X, the expected value is
\mathbb{E} (X) = \sum_{x \in \chi} x p(x).

For a continuous r.v. X, the expected value is
\mathbb{E} (X) = \int_{x \in \chi} x f(x) ~dx.

If Y = g(X), then

For a discrete r.v. X,
\mathbb{E} (Y) = \mathbb{E} [g(X)] = \sum_{x \in \chi} g(x) p(x).

For a continuous r.v. X,
\mathbb{E} (Y) = \mathbb{E} [g(X)] = \int_{x \in \chi} g(x) f(x) ~dx.

Properties of Expectation

For random variables X and Y and constants a, b, \in \mathbb{R}, the expected value has the following properties (applicable to both discrete and continuous r.v.s)

\mathbb{E}(aX + b) = a \mathbb{E}(X) + b

\mathbb{E}(X + Y) = \mathbb{E}(X) + \mathbb{E}(Y)

Realisations of X, denoted by x, may be larger or smaller than \mathbb{E}(X),

If you observed many realisations of X, \mathbb{E}(X) is roughly an average of the values you would observe.

\mathbb{E} (aX + b)
= \int_{- \infty}^{\infty} (ax+b)f(x) ~dx
= \int_{- \infty}^{\infty} axf(x) ~dx + \int_{- \infty}^{\infty} bf(x) ~dx
= a \int_{- \infty}^{\infty} xf(x) ~dx + b \int_{- \infty}^{\infty} f(x) ~dx
= a \mathbb{E} (X) + b


Generally speaking, variance is defined as

Var(X) = \mathbb{E}[(X- \mathbb{E}(X)^2] = \mathbb{E}[X^2] - \mathbb{E}[X]^2

If X is discrete:

Var(X) = \sum_{x \in \chi} ( x - \mathbb{E}[X])^2 p(x)

If X is continuous:

Var(X) = \int_{x \in \chi} ( x - \mathbb{E}[X])^2 f(x) ~dx

Using the properties of expectations, we can show Var(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2.

= \mathbb{E} [(X - \mathbb{E}[X])^2]
= \mathbb{E} [(X^2 - 2X \mathbb{E}[X]) + \mathbb{E}[X]^2]
= \mathbb{E}[X^2] - 2\mathbb{E}[X]\mathbb{E}[X] + \mathbb{E}[X]^2
= \mathbb{E}[X^2] - \mathbb{E}[X]^2

Standard Deviation

The standard deviation is defined as

std(X) = \sqrt{Var(X)}


For two random variables X and Y, the covariance is generally defined as

Cov(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])]

Note that Cov(X, X) = Var(X)

Cov(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y]

Properties of Variance

Given random variables X and Y, and constants a, b, c \in \mathbb{R},

Var(aX \pm bY \pm b ) = a^2 Var(X) + b^2 Var(Y) + 2ab Cov(X, Y)

This proof for the above can be done using definitions of expectations and variance.

Properties of Covariance

Given random variables W, X, Y and Z and constants a, b, \in \mathbb{R}

Cov(X, a) = 0

Cov(aX, bY) = ab Cov(X, Y)

Cov(W+X, Y+Z) = Cov(W, Y) + Cov(W, Z) + Cov(X, Y) + Cov(X, Z)


Correlation is defined as

Corr(X, Y) = \dfrac{Cov(X, Y)}{Std(X) Std(Y)}

It is clear the -1 \le Corr(X, Y) \le 1.

The properties of correlations of sums of random variables follow from those of covariance and standard deviations above.

Leave a Reply