Introduction to Probability

Random Variables

A random variable denotes a quantity that is uncertain.

Exercise 2.1

Weather Prediction: let \(x\) denote the state (e.g. sunny, raining) and \(y\) denote some quantitative measure of interest (e.g temperature, wind, humidity).

Exercise 2.2

\[\int \int Pr(v, w, x, y, z) dw dy = \int Pr(v, x, y, z) dy = Pr(v, x, z)\]

\[\int Pr(v, x, z) dv = Pr(x, z)\]

Exercise 2.3

\[\begin{split}Pr(w, x, y, z) &= Pr(w, y, z \mid x) Pr(x)\\ &= Pr(w, z \mid x, y) Pr(y \mid x) Pr(x) & \quad & \text{(2.5)}\\ &= Pr(w, z \mid x, y) Pr(x, y) & \quad & \text{(2.6)}\\ &= Pr(z \mid w, x, y) Pr(w \mid x, y) Pr(x, y) & \quad & \text{(2.7)}\end{split}\]

Exercise 2.4

\[\begin{split}Pr(c = 2 \mid h = 1) &= \frac{Pr(h = 1 \mid c = 2) Pr(c = 2)}{Pr(h = 1)}\\ &= \frac{(0.8) (0.5)}{\sum_c Pr(h = 1, c)}\\ &= \frac{0.4}{\sum_c Pr(h = 1 \mid c) P(c)}\\ &= \frac{0.4}{ Pr(h = 1 \mid c = 1) P(c = 1) + Pr(h = 1 \mid c = 2) Pr(c = 2) }\\ &= \frac{0.4}{(0.5) (0.5) + (0.8) (0.5)}\\ &= \frac{8}{13}\end{split}\]

Exercise 2.5

No. Consider any general distribution \(Pr(y, z)\) where \(y\) and \(z\) are not independent. Any distribution \(Pr(x)\) that does not provide any information about \(y\) or \(z\) can be tacked on to the marginal distributions \(Pr(y)\) and \(Pr(z)\).

Exercise 2.6

\[\begin{split}\newcommand{\Perp}{\mathbin{\unicode{x2AEB}}} Pr(x \mid y = y^*) &= \frac{Pr(x, y = y^*)}{\int Pr(x, y = y^*) dx}\\ &= \frac{Pr(x) Pr(y = y^*)}{Pr(y = y^*) \int Pr(x) dx} & \quad & x \Perp y\\ &= Pr(x) & \quad & \int Pr(x) dx = 1\end{split}\]

Exercise 2.7

\[\begin{split}Pr(x, w) &= \int \int Pr(w, x, y, z) dz dy & \quad & \text{(2.2)}\\ &= \int \int Pr(w) Pr(z \mid y) Pr(y \mid x, w) Pr(x) dz dy & \quad & \text{initial assumption}\\ &= Pr(x) Pr(w) \int Pr(y \mid x, w) \int Pr(z \mid y) dz dy\\ &= Pr(x) Pr(w) & \quad & \text{(2.3)}\end{split}\]

Exercise 2.8

Let \(X\) denote the number of points on the face turned up with the corresponding probabilities.

\[\mathrm{E}[X] = \sum_x x \Pr(x) = (1) 12^{-1} + (2) 12^{-1} + (3) 12^{-1} + (4) 12^{-1} + (5) 6^{-1} + (6) 2^{-1} = 4.\bar{6}.\]

Since die rolls are independent, rolling the die twice is equivalent to

\[\mathrm{E}[X + X] = 2 \mathrm{E}[X] = 9.\bar{3}.\]

Exercise 2.9

\[\begin{split}\mathrm{E}[\kappa] &= \int \kappa Pr(x) dx\\ &= \kappa \int Pr(x) dx\\ &= \kappa\end{split}\]

\[\begin{split}\mathrm{E}[\kappa f[x]] &= \int \kappa f[x] Pr(x) dx\\ &= \kappa \int f[x] Pr(x) dx\\ &= \kappa \mathrm{E}[f[x]]\end{split}\]

\[\begin{split}\mathrm{E}[f[x] + g[x]] &= \int \left( f[x] + g[x] \right) Pr(x) dx\\ &= \int \left( f[x] Pr(x) + g[x] Pr(x) \right) dx\\ &= \int f[x] Pr(x) dx + \int g[x] Pr(x) dx\\ &= \mathrm{E}[f[x]] + \mathrm{E}[g[x]]\end{split}\]

\[\begin{split}\mathrm{E}[f[x] g[y]] &= \int \int f[x] g[y] Pr(x, y) dx dy & \quad & \text{(2.13)}\\ &= \int \int f[x] g[y] Pr(x) Pr(y) dx dy & \quad & \text{(2.11)}\\ &= \int f[x] Pr(x) dx \int g[y] Pr(y) dy\\ &= \mathrm{E}[f[x]] \mathrm{E}[g[y]]\end{split}\]

Exercise 2.10

\[\begin{split}\mathrm{E}\left[ (x - \mu)^2 \right] &= \mathrm{E}\left[ x^2 - 2 \mu x + \mu^2 \right]\\ &= \mathrm{E}[x^2] - 2 \mu \mathrm{E}[x] + \mu^2 & \quad & \text{(2.16), (2.15), (2.14)}\\ &= \mathrm{E}[x^2] - 2 \mathrm{E}[x]^2 + \mathrm{E}[x]^2 & \quad & \mu \triangleq \mathrm{E}[x]\\ &= \mathrm{E}[x^2] - \mathrm{E}[x]^2\end{split}\]