Quick Summary (Probability)

University is starting for some students who took A’levels in 2016. And, one of my ex-students told me to share/ summarise the things to know for probability at University level. Hopefully this helps. H2 Further Mathematics Students will find some of these helpful.


Random Variables

Suppose X is a random variable which can takes values x \in \chi.

X is a discrete r.v. is \chi is countable.
\Rightarrow p(x) is the probability of a value of x and is called the probability mass function.

X is a continuous r.v. is \chi is uncountable.
\Rightarrow f(x) is the probability density function and can be thought of as the probability of a value x.

Probability Mass Function

For a discrete r.v. the probability mass function (PMF) is

p(a) = P(X=a), where a \in \mathbb{R}.

Probability Density Function

If B = (a, b)

P(X \in B) = P(a \le X \le b) = \int_a^b f(x) ~dx.

And strictly speaking,

P(X = a) = \int_a^a f(x) ~dx = 0.

Intuitively,

f(a) = P(X = a).

Properties of Distributions

For discrete r.v.
p(x) \ge 0 ~ \forall x \in \chi.
\sum_{x \in \chi} p(x) = 1.

For continuous r.v.
f(x) \ge 0 ~ \forall x \in \chi.
\int_{x \in \chi} f(x) ~dx = 1.

Cumulative Distribution Function

For discrete r.v., the Cumulative Distribution Function (CDF) is
F(a) = P(X \le a) = \sum_{x \le a} p(x).

For continuous r.v., the CDF is
F(a) = P(X \le a ) = \int_{- \infty}^a f(x) ~dx.

Expected Value

For a discrete r.v. X, the expected value is
\mathbb{E} (X) = \sum_{x \in \chi} x p(x).

For a continuous r.v. X, the expected value is
\mathbb{E} (X) = \int_{x \in \chi} x f(x) ~dx.

If Y = g(X), then

For a discrete r.v. X,
\mathbb{E} (Y) = \mathbb{E} [g(X)] = \sum_{x \in \chi} g(x) p(x).

For a continuous r.v. X,
\mathbb{E} (Y) = \mathbb{E} [g(X)] = \int_{x \in \chi} g(x) f(x) ~dx.

Properties of Expectation

For random variables X and Y and constants a, b, \in \mathbb{R}, the expected value has the following properties (applicable to both discrete and continuous r.v.s)

\mathbb{E}(aX + b) = a \mathbb{E}(X) + b

\mathbb{E}(X + Y) = \mathbb{E}(X) + \mathbb{E}(Y)

Realisations of X, denoted by x, may be larger or smaller than \mathbb{E}(X),

If you observed many realisations of X, \mathbb{E}(X) is roughly an average of the values you would observe.

\mathbb{E} (aX + b)
= \int_{- \infty}^{\infty} (ax+b)f(x) ~dx
= \int_{- \infty}^{\infty} axf(x) ~dx + \int_{- \infty}^{\infty} bf(x) ~dx
= a \int_{- \infty}^{\infty} xf(x) ~dx + b \int_{- \infty}^{\infty} f(x) ~dx
= a \mathbb{E} (X) + b

Variance

Generally speaking, variance is defined as

Var(X) = \mathbb{E}[(X- \mathbb{E}(X)^2] = \mathbb{E}[X^2] - \mathbb{E}[X]^2

If X is discrete:

Var(X) = \sum_{x \in \chi} ( x - \mathbb{E}[X])^2 p(x)

If X is continuous:

Var(X) = \int_{x \in \chi} ( x - \mathbb{E}[X])^2 f(x) ~dx

Using the properties of expectations, we can show Var(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2.

Var(X)
= \mathbb{E} [(X - \mathbb{E}[X])^2]
= \mathbb{E} [(X^2 - 2X \mathbb{E}[X]) + \mathbb{E}[X]^2]
= \mathbb{E}[X^2] - 2\mathbb{E}[X]\mathbb{E}[X] + \mathbb{E}[X]^2
= \mathbb{E}[X^2] - \mathbb{E}[X]^2

Standard Deviation

The standard deviation is defined as

std(X) = \sqrt{Var(X)}

Covariance

For two random variables X and Y, the covariance is generally defined as

Cov(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])]

Note that Cov(X, X) = Var(X)

Cov(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y]

Properties of Variance

Given random variables X and Y, and constants a, b, c \in \mathbb{R},

Var(aX \pm bY \pm b ) = a^2 Var(X) + b^2 Var(Y) + 2ab Cov(X, Y)

This proof for the above can be done using definitions of expectations and variance.

Properties of Covariance

Given random variables W, X, Y and Z and constants a, b, \in \mathbb{R}

Cov(X, a) = 0

Cov(aX, bY) = ab Cov(X, Y)

Latex Cov(W+X, Y+Z) = Cov(W, Y) + Cov(W, Z) + Cov(X, Y) + Cov(X, Z)

Correlation

Correlation is defined as

Corr(X, Y) = \dfrac{Cov(X, Y)}{Std(X) Std(Y)}

It is clear the -1 \le Corr(X, Y) \le 1.

The properties of correlations of sums of random variables follow from those of covariance and standard deviations above.

Leave a Comment

Contact Us

CONTACT US We would love to hear from you. Contact us, or simply hit our personal page for more contact information

Not readable? Change text. captcha txt

Start typing and press Enter to search