Multivariate Distributions

Let $X = (X_1 \ldots X_n)^T$ be an n-dimensional vector of random variables.
For all $x = (x_1, \ldots, x_n) \in \mathbb{R}^n$ , the joint cumulative distribution function of X satisfies
$F_{X_i}(x_i) = F_X (\infty, \ldots, \infty, x_i, \infty, \ldots, \infty)$

Clearly it is straightforward to generalise the previous definition to join marginal distributions. For example, the join marginal distribution of $X_i$ and $X_j$ satisfies
$F_X(x_1, \ldots, x_n) = \int_{-\infty}^{x_1} \ldots \int_{-\infty}^{x_n} f_x (u_1, \ldots, u_n) du_1 \ldots du_n$

If $X_1 = (X_1 \ldots X_k)^T$ and $X_2 = (X_{k+1} \ldots X_n)^T$ is a partition of X then the conditional CDF of X_2 given X_1 satisfies
$F_{X_2|X_1} (X_2|X_1) = P(X_2 \le x_2 | X_1 = x_1). If X has a PDF,$ latex f_X (\bullet) $, then the conditional PDF of$ latex X_2 $given$ latex X_1 $satisfies$ latex f_{X_2 | X_1} (X_2 | X_1) = \frac{f_X (X)}{f_{X_1}(X_1)} = \frac{f_{X_1 | x_2}(X_1 | X_2) f_{X_2}(X_2)}{f_{X_1}(X_1)} $and the conditional CDF is then given by$ latex F_{X_2 | X_1}(X_2 |X_1) = \int_{- \infty}^{x_{k+1}} \ldots \int_{- \infty}^{x_n} \frac{f_X (x_1, \ldots , x_k, u_{k+1}, \ldots , u_n)}{f_{X_1}(X_1)} du_{k+1} \dots du_n $where$ latex f_{X_1}(\bullet) $is the joint marginal PDF of$ latex X_1 $which is given by$ latex f_{X_1} (x_1, \ldots , x_k) = \int_{- \infty}^{\infty} \ldots \int_{- \infty}^{\infty} f_X (x_1, \ldots , x_K u_{k+1}, \ldots , u_n) du_{k+1} \ldots du_n $We next look at independence, which is something H2 Mathematics can easily relate to. Here, we say the collection X is independent if joint CDF can be factored into the product of the marginal CDFs so that$ latex F_X (x_1 \ldots , x_n) = F_{X_1} (x_1) \ldots F_{X_n}(x_n) $If X has a PDF,$ latex f_X(\bullet) $then independence implies that the PDF also factories into the product of marginal PDFs so that$ Latex f_X(x) = f_{X_1} (x_1) \ldots f_{X_n}(x_n) $. Using the above results, we have that if$ latex X_1 $and$ latex X_2 $are independent then$ latex f_{x_2|x_1}(x_2 | x_1) = \frac{f_X (X)}{f_{X_1}(X_1)} = \frac{f_{X_1}(X_1) f_{X_2}(X_2)}{f_{X_1}(X_1)} = f_{X_2}(X_2) $The above results tell us that having information about$ latex X_1 $tells us nothing about$ latex X_2 $. Let's continue to look further on the implications of independence. Let X and Y be independent random variables. Then for any events A and B,$ latex P(X \in A, Y \in B) = P(X \in A) P(Y \in B) $We can check this with,$ latex P(X \in A, Y \in B)latex = \mathbb{E}[1_{X \in A} 1_{Y \in B}]latex = P(X \in A) P(Y \in B) $In general, if$ latex X_1, \ldots, X_n $are independent random variables then$ latex \mathbb{E}[f_1 (X_1) f_2(X_2) \ldots f_n(X_n)] = \mathbb{E}[f_1(X_1)] \mathbb{E}[f_2(X_2)] \ldots \mathbb{E}[f_n(X_n)] $Moreover, random variables can also be conditionally independent. For example, we say that X and Y are conditionally independent given Z if$ latex \mathbb{E}[f(X)g(Y)|Z] = \mathbb{E}[f(X)|Z]\mathbb{E}[g(Y)|Z] $. The above will be used in the Gaussian copula model for pricing of collateralised debt obligation (CDO). [caption id="attachment_2829" align="alignnone" width="300"]<a href="http://theculture.sg/wp-content/uploads/2016/01/620x346xMultivariate_Gaussian_Fixed-1024x572.png.pagespeed.ic_.Tx7wEea-aO.png" rel="attachment wp-att-2829"><img src="http://theculture.sg/wp-content/uploads/2016/01/620x346xMultivariate_Gaussian_Fixed-1024x572.png.pagespeed.ic_.Tx7wEea-aO-300x168.png" alt="Source: www.turingfinance.com" width="300" height="168" class="size-medium wp-image-2829" /></a> Source: www.turingfinance.com[/caption] We let$ latex D_i $be the event that the$ latex i^{th} $bond in a portfolio defaults. Not reasonable to assume that the$ latex D_i $'s are independent though they are conditionally independent given Z so$ latex P(D_1, \ldots, D_n |Z ) = P(D_1|Z) \ldots P(D_n|Z) $is often easy to compute. Lastly, we consider the mean and covariance. The mean vector of X is given by$ Latex \mathbb{E}[X]:=(\mathbb{E}[X_1] \ldots \mathbb{E}[X_n])^T $and the covariance matrix of X satisfies$ latex \sum := \mathrm{Cov}(X) := \mathbb{E}[(X- \mathbb{E}[X])(X – \mathbb{E}[X])^T] $so the$ latex (i,j)^{th} $element of$ latex \sum $is simply the covariance of$ Latex X_i $and$ latex X_j $. The covariance matrix is symmetric and its diagonal elements satisfy$ latex \sum_{i, i} \ge 0 $and is also positive semi definite so that$ latex x^T \sum x \ge 0 $for all$ latex x \in \mathbb{R}^n $Then the correlation matrix$ latex \rho (X) $has$ latex (i, j)^{th} $element$ latex \rho_{ij} := \mathrm{Corr}(X_i, X_j) $. This is also symmetric, postiie semi-definite and has 1's along the diagonal. For any matrix$ latex A \in \mathbb{R}^{k \times n} $and vector$ latex a \in \mathbb{R}^k $,$ Latex \mathbb{E}[AX +a] = A \mathbb{E} [X] + a $(distributes linearly)$ latex \mathrm{Cov}(AX +a) = A \mathrm{Cov}(X) A^Tlatex \Rightarrow \mathrm{Var} (aX + bY) = a^2 \mathrm{Var}(X) + b^2 \mathrm{Var}(Y) + 2ab \mathrm{Cov}(X, Y) $Recall that if X and Y are independent, then$ latex \mathrm{Cov}(X, Y)=0$, and the converse is not true in general.

About

KS Teng

KS has been teaching H2/H1 Mathematics and IB mathematics for the more than 10 years. Having taught students from all various Junior Colleges, KS adapts to students' abilities and help them better understand the topics. As someone who loves to teach mathematics and sees it as a truly useful tool in life, KS seeks to enable students to appreciate math. Therefore, his tuition mission is to motivate and cultivate students to be independent and confident thinkers.

Comments

pingbacks / trackbacks

Review of Basic Probability – The Culture
[…] Multivariate Distributions […]
February 3rd, 2016 09:54 PM

Cart

Multivariate Distributions

Leave a Reply to Review of Basic Probability – The Culture Cancel reply