Quick Summary (Probability)

University is starting for some students who took A’levels in 2016. And, one of my ex-students told me to share/ summarise the things to know for probability at University level. Hopefully this helps. H2 Further Mathematics Students will find some of these helpful.

Random Variables

Suppose $X$ is a random variable which can takes values $x \in \chi$ .

$X$ is a discrete r.v. is $\chi$ is countable.
$\Rightarrow p(x)$ is the probability of a value of $x$ and is called the probability mass function.

$X$ is a continuous r.v. is $\chi$ is uncountable.
$\Rightarrow f(x)$ is the probability density function and can be thought of as the probability of a value $x$ .

Probability Mass Function

For a discrete r.v. the probability mass function (PMF) is

$p(a) = P(X=a)$ , where $a \in \mathbb{R}$ .

Probability Density Function

If $B = (a, b)$

$P(X \in B) = P(a \le X \le b) = \int_a^b f(x) ~dx$ .

And strictly speaking,

$P(X = a) = \int_a^a f(x) ~dx = 0$ .

Intuitively,

$f(a) = P(X = a)$ .

Properties of Distributions

For discrete r.v.
$p(x) \ge 0 ~ \forall x \in \chi$ .
$\sum_{x \in \chi} p(x) = 1$ .

For continuous r.v.
$f(x) \ge 0 ~ \forall x \in \chi$ .
$\int_{x \in \chi} f(x) ~dx = 1$ .

Cumulative Distribution Function

For discrete r.v., the Cumulative Distribution Function (CDF) is
$F(a) = P(X \le a) = \sum_{x \le a} p(x)$ .

For continuous r.v., the CDF is
$F(a) = P(X \le a ) = \int_{- \infty}^a f(x) ~dx$ .

Expected Value

For a discrete r.v. X, the expected value is
$\mathbb{E} (X) = \sum_{x \in \chi} x p(x)$ .

For a continuous r.v. X, the expected value is
$\mathbb{E} (X) = \int_{x \in \chi} x f(x) ~dx$ .

If $Y = g(X)$ , then

For a discrete r.v. X,
$\mathbb{E} (Y) = \mathbb{E} [g(X)] = \sum_{x \in \chi} g(x) p(x)$ .

For a continuous r.v. X,
$\mathbb{E} (Y) = \mathbb{E} [g(X)] = \int_{x \in \chi} g(x) f(x) ~dx$ .

Properties of Expectation

For random variables $X$ and $Y$ and constants $a, b, \in \mathbb{R}$ , the expected value has the following properties (applicable to both discrete and continuous r.v.s)

$\mathbb{E}(aX + b) = a \mathbb{E}(X) + b$

$\mathbb{E}(X + Y) = \mathbb{E}(X) + \mathbb{E}(Y)$

Realisations of $X$ , denoted by $x$ , may be larger or smaller than $\mathbb{E}(X)$ ,

If you observed many realisations of $X$ , $\mathbb{E}(X)$ is roughly an average of the values you would observe.

$\mathbb{E} (aX + b)$
$= \int_{- \infty}^{\infty} (ax+b)f(x) ~dx$
$= \int_{- \infty}^{\infty} axf(x) ~dx + \int_{- \infty}^{\infty} bf(x) ~dx$
$= a \int_{- \infty}^{\infty} xf(x) ~dx + b \int_{- \infty}^{\infty} f(x) ~dx$
$= a \mathbb{E} (X) + b$

Variance

Generally speaking, variance is defined as

$Var(X) = \mathbb{E}[(X- \mathbb{E}(X)^2] = \mathbb{E}[X^2] - \mathbb{E}[X]^2$

If $X$ is discrete:

$Var(X) = \sum_{x \in \chi} ( x - \mathbb{E}[X])^2 p(x)$

If $X$ is continuous:

$Var(X) = \int_{x \in \chi} ( x - \mathbb{E}[X])^2 f(x) ~dx$

Using the properties of expectations, we can show $Var(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$ .

$Var(X)$
$= \mathbb{E} [(X - \mathbb{E}[X])^2]$
$= \mathbb{E} [(X^2 - 2X \mathbb{E}[X]) + \mathbb{E}[X]^2]$
$= \mathbb{E}[X^2] - 2\mathbb{E}[X]\mathbb{E}[X] + \mathbb{E}[X]^2$
$= \mathbb{E}[X^2] - \mathbb{E}[X]^2$

Standard Deviation

The standard deviation is defined as

$std(X) = \sqrt{Var(X)}$

Covariance

For two random variables $X$ and $Y$ , the covariance is generally defined as

$Cov(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])]$

Note that $Cov(X, X) = Var(X)$

$Cov(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y]$

Properties of Variance

Given random variables $X$ and $Y$ , and constants $a, b, c \in \mathbb{R}$ ,

$Var(aX \pm bY \pm b ) = a^2 Var(X) + b^2 Var(Y) + 2ab Cov(X, Y)$

This proof for the above can be done using definitions of expectations and variance.

Properties of Covariance

Given random variables $W, X, Y$ and $Z$ and constants $a, b, \in \mathbb{R}$

$Cov(X, a) = 0$

$Cov(aX, bY) = ab Cov(X, Y)$

$Latex Cov(W+X, Y+Z) = Cov(W, Y) + Cov(W, Z) + Cov(X, Y) + Cov(X, Z)$

Correlation

Correlation is defined as

$Corr(X, Y) = \dfrac{Cov(X, Y)}{Std(X) Std(Y)}$

It is clear the $-1 \le Corr(X, Y) \le 1$ .

The properties of correlations of sums of random variables follow from those of covariance and standard deviations above.

About

KS Teng

KS has been teaching H2/H1 Mathematics and IB mathematics for the more than 10 years. Having taught students from all various Junior Colleges, KS adapts to students' abilities and help them better understand the topics. As someone who loves to teach mathematics and sees it as a truly useful tool in life, KS seeks to enable students to appreciate math. Therefore, his tuition mission is to motivate and cultivate students to be independent and confident thinkers.

Quick Summary (Probability)

Leave a Comment Cancel reply

Leave a Comment
Cancel reply