NotesAtHKU

Random variables

A numerical description of the outcome of a random experiment is called a random variable. It can be either discrete or continuous.

Notation

Random variables are denoted with capital letters $X$
The possible outcomes are denoted with regular letters $x$
Probability that the outcome of $X$ is $x$ is denoted by $P(X=x)$

Expected value and Variance

$E(X^n)=\sum(x^n\cdot P(X=x))$ gives the expected value, which represents the mean value (outcome) of the random variable.
$Var(X)=E(X^2)-E(X)^2$ gives the variance, which is a measure of the variability of the random variable's outcomes.

Operations of E(X) and Var(X)

$E(X+Y)=E(X)+E(Y)$ , which any addition/subtraction function within $E()$ can be expanded.

If $X$ and $Y$ are independent:

$Var(X+Y) = Var(X) + Var(Y),\quad\quad E(XY) = E(X)\cdot E(Y)$
For $Y=aX+b$ :
$E(Y)=aE(X)+b,\quad\quad Var(Y)=a^2Var(X)$

Probability distributions

A continuous random variable has probabilities as the area under its curve. Hence, $P(X=n)$ for any outcome $n$ is $0$ . A discrete random variable has specific probabilities assigned to an outcome.

Operation on continuous ranges

$P(X<x)=P(X\leq x)$
$P(X>x)=1 - P(X<x)$
$P(a\leq X\leq b)=P(X<b)-P(X<a)$

Operation on discrete ranges

$P(X<x)=P(X\leq x-1)$
$P(X>x)=1-P(X\leq x)$
$P(a\leq X\leq b)=P(X\leq b)-P(X\leq a-1)$

Probability distribution functions

A p.d.f is a continuous function that returns the probability of the given outcome. The following is an example of a p.d.f:

X\sim f(x) = \begin{cases} Kx^2 & 0 \leq x \leq 1 \\ 0 & \text{otherwise} \end{cases}

Using the function

$P(X=x)=0$
$P(a<X<b)=P(a\leq X\leq b)=\int_{a}^{b}f(x)dx$

Note that $\int_{\infty}^{-\infty}f(x)dx = 1$ , as the total probability of any event is 1. This condition must be true for $f(x)$ to be a valid p.d.f. Hence for the example $K=3$

Expected value

$E(X^n)=\int_{\infty}^{-\infty}x^nf(x)dx$

Cumulative distribution function

To obtain a c.d.f $X'\sim F(x)$ where $P(X'=x) = P(X < x)$ , all we have to do is integrate the p.d.f:

F(X)=\int f(x)dx

Using our example:

F(X)=\begin{cases} 1 & x > 1 \\ x^3 & 0 < x < 1 \\ 0 & x < 0 \end{cases}

And to convert a c.d.f back to its p.d.f, all we have to do is differentiate $F(x)$ :

f(x)=\frac{d}{dx}F(x)

Normal distribution

The Normal distribution is given by the following formula (which you don't have to memorize):

X\sim N(\mu, \sigma^2)=\frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}},\quad\quad\text{where }\underbrace{\mu}_{\text{mean}},\underbrace{\sigma^2}_{\text{variance}}

To calculate $P(X<x)$ , we have to standardize our $N$ by $Z\sim N(0, 1)$ , $P(X<x)=P(Z<\frac{x-\mu}{\sigma})$ . Here $Z$ is the standard normal variable.

Reading the Z-table

For $P(Z<z)=p$ , to find $p$ , locate the header and leftmost column in the z-table such that their sum is $z$ . The corresponding intersecting cell is $p$ .
$Z_p$ gives the value $z$ in $P(Z<z)=p$

Sampling

Sampling notation

For a given population of size $N$ , $\mu$ is the true mean and $\sigma^2$ is the true variance.

A sample is a subset of the population of size $n$ . The sample mean is $\bar{x}$ and the sample variance is $s^2$ . They can be calculated as:

\bar{x}=\frac{1}{n}\sum x_i,\quad\quad s^2=\frac{1}{n-1}\sum(x_i-\bar{x})^2

Definitions

The statistics for each random sample observation of the population are random.

The sampling distribution is the distribution of the statistics of the random samples.

Central Limit Theorem

The random variable $\bar{X}$ is the mean of a random sample of $n$ observations if:

The observations are independent.
The observations are identically distributed.
$n \geq 30$ .

\bar{X}=\frac{X_1+X_2+\dots+X_n}{n}

If $X$ is normally distributed:

\bar{X}\sim N(\mu, \frac{\sigma^2}{\sqrt{n}}),\quad \sqrt{n}\text{ is the standard error (deviation)}

The standard error captures how off the sample statistics are from the true population value. The value decreases as n increases.

The central limit theorem states that the sampling distribution of the sample mean approaches a normal distribution as the sample size increases, regardless of the shape of the population distribution.

Hypothesis testing

Test values

Test Statistic: Result that is calculated from the sample

Null Hypothesis $H_0$ : Hypothesis assumed correct

Alternative Hypothesis $H_1$ : Parameter if assumption shown wrong

Types of tests

$H_1$ in the form $H_1: p < \text{ or } p >$ → "one-tailed"
$H_1$ in the form $H_1: p \neq$ → "two-tailed"
Over / under / increase / decrease = one-tailed
Change / not = two-tailed

Carrying out tests

Identify the available data:
- Test statistic, $X$
- Significance level
- One tail or two tail
Execution:
- Define $X$ – the test statistic
- Define $H_0$ & $H_1$
- Assume $H_0$ true, substitute $p$ to distribution model → find $P(X \sim \text{t-value})$ to give the t-value.
- Compare: if $P(X \sim \text{t-value}) < \text{s.l.}$ → reject $H_0$
- Conclude in context of the question

Example of Hypothesis Testing

A seed producer claims that 96% of its beans turn golden. A random sample of 75 bean seeds was planted and 66 of these seeds turned golden. Test at 1% significance whether the producer is overstating the probability of the seeds turning golden.

Data identified:
- $n = 75, p = 0.96$
- Model hence is Binomial
- Test statistic value $x = 66$
- $s.l. = 0.01$
- One tail
Define $X$ – the test statistic:
- Let $X$ = the number of seeds that turn golden
Define $H_0$ & $H_1$ :
- $H_0: p = 0.96, H_1: p < 0.96$
Assume $H_0$ true, substitute $p$ to distribution model → find $P(X \sim \text{t-value})$ :
- $P(X \leq 66) = 0.00303 < 0.01$
Compare: if $P(X \sim \text{t-value}) < \text{s.l.}$ → reject $H_0$ :
- $H_0$ rejected
Conclude in context of the question:
- Producer is overstating …

Example if Two-Tailed

Test at 1% significance whether the producer is lying about the probability of the seeds turning golden.

Define $X$ – the test statistic:
- Let $X$ = the number of seeds that turn golden
Define $H_0$ & $H_1$ :
- $H_0: p = 0.96, H_1: p \neq 0.96$
Assume $H_0$ true, substitute $p$ to distribution model → find $P(X \geq \text{t-value})$ & $P(X \leq \text{t-value})$ if two-tailed:
- $P(X \leq 66) = 0.00303 < 0.005$
- $P(X \geq 66) = 1 - 0.00303 > 0.005$
Compare: if $P(X \sim \text{t-value}) < \text{s.l.} \div 2$ → reject $H_0$ :
- $H_0$ rejected (first tail)
Conclude in context of the question:
- Producer is lying and overstating…

Statistics