Statistics

$P(X=x)=\binom{n}{x} p^{x}(1-p)^{n-x},\quad x=0,\dots,n$

$\mathbb{E}[X]=np$

$\operatorname{Var}(X)=np(1-p)$

$P(X=k)=\frac{\lambda^{k} e^{-\lambda}}{k!},\quad k=0,1,2,\dots$

$\mathbb{E}[X]=\lambda$

$\operatorname{Var}(X)=\lambda$

$f(x)=\frac{1}{\sqrt{2\pi}\sigma} \exp\!\left[-\frac{(x-\mu)^2}{2\sigma^2}\right]$

$\mathbb{E}[X]=\mu$

$\operatorname{Var}(X)=\sigma^2$

$f(x)=\frac{1}{x\sigma\sqrt{2\pi}} \exp\!\left[-\frac{(\ln x - \mu)^{2}}{2\sigma^{2}}\right],\quad x>0$

$\mathbb{E}[X]=e^{\mu+\sigma^{2}/2}$

$\operatorname{Var}(X)= \left(e^{\sigma^{2}}-1\right)e^{2\mu+\sigma^{2}}$

$f(x;\nu)=\frac{x^{\nu/2-1} e^{-x/2}} {2^{\nu/2}\Gamma(\nu/2)},\quad x>0$

$\mathbb{E}[X]=\nu$

$\operatorname{Var}(X)=2\nu$

$f(x;\alpha,b)=\frac{\alpha b^{\alpha}}{x^{\alpha+1}},\quad x\ge b$

$\mathbb{E}[X]=\frac{\alpha b}{\alpha-1}\quad(\alpha>1)$

$\operatorname{Var}(X)= \frac{\alpha b^{2}}{(\alpha-1)^{2}(\alpha-2)}\quad(\alpha>2)$

$P(A\cup B)=P(A)+P(B)-P(A\cap B)$

$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$

$P(B_i \mid A) = \frac{P(A \mid B_i)\, P(B_i)}{P(A)}$

$\mu = E(X) = \sum_{i=1}^{n} x_i\, P(X = x_i)$

$\sigma^{2} = \operatorname{Var}(X) = E\!\left[(X - \mu)^{2}\right]$

$\mu = E(X) = \int_{-\infty}^{\infty} x\, f(x)\, dx$

$\sigma^{2} = \operatorname{Var}(X) = E[(X - \mu)^{2}] = \int_{-\infty}^{\infty} (x - \mu)^{2} f(x)\, dx = \int_{-\infty}^{\infty} x^{2} f(x)\, dx - \mu^{2}$

$Q(u) = \inf\{y : F(y) \ge u\},\quad 0<u<1$

Sample quantiles vs. theoretical quantiles.

The sample distribution matches the reference distribution.

$\text{A random vector is a vector of random variables } (X_1, \dots, X_k) \text{ considered jointly.}$

$F(x_1,\dots,x_k) = P(X_1\le x_1,\dots,X_k\le x_k)$

All observations have the same variance.

Observations have different variances, violating i.i.d. (Independent and identically distributed.) assumptions.

$\text{The sample mean } \bar{X}_n \text{ converges to the population mean } \mu \text{ as } n \to \infty.$

Variables must be i.i.d. (Independent and identically distributed.) with finite expectation.

$\text{The standardized sample mean approaches a normal distribution as } n \to \infty.$

Population: The entire group of objects about which inferences are made.
Sample: A subset of the population on which data are actually collected.
Statistic: A numerical summary of sample data (e.g., mean, variance, median).
(Page 6)

Frequentists treat θ as a fixed but unknown constant; randomness comes only from sampling.
Bayesians treat θ as a random variable with a prior distribution that is updated to a posterior using data.
(Page 8)

1. Method of moments
2. Least squares estimation (LSE)
3. Maximum likelihood estimation (MLE)

$B(θ̂) = E[θ̂] − θ $
$MSE(θ̂) = Var(θ̂) + (E[θ̂] − θ)² $
Biased: $B(θ̂)$ is not zero, but approaches zero as the dataset approaches
infinity.
Smaller MSE is preferred. (Page 13)

The method of moments matches theoretical moments of a distribution to sample moments, solving these equations for the unknown parameters.
(Page 15)
It basically looks at the mean and variance etc. of you data and tries to figure out the parameters of the distribution behind those numbers.

Least squares estimation minimizes the sum of squared errors (or weighted squared errors for heteroscedastic data).
(Page 17)
Basiacally if you have a lot of dots (data points) and you try to fit a line on them but they don't fit perfectly the least squares estimation chooses the line that makes all those errors as small as possible so the line is as accurate as possible.

$L(\theta) = \prod_{i=1}^{n} f(X_i \,;\, \theta)$
$\ell(\theta) = \sum_{i=1}^{n} \ln\big( f(X_i \,;\, \theta) \big)$
(Page 19)

MLE finds the parameter values that make the observed data most likely.
It does this by multiplying the probabilities of each data point (the likelihood)
and choosing the parameter values that maximize this product.
(Page 19)

The sample mean is defined as
$ \bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i, $
and it is a point estimator of the population mean μ.
(Page 26)

$\text{A }100(1-\alpha)\%\text{ confidence interval for }\theta\text{ is an interval }(l,u)\text{ such that }P(l<\theta<u)=1-\alpha.$
(Page 24)

$\alpha\text{ is a small value (e.g., 0.05 or 0.01) representing the probability that the }$
$\text{interval does not contain the true parameter }\theta.$
(Page 24)

$\text{For }\alpha=0.05,\text{ a 95\% confidence interval }(l,u)\text{ satisfies }P(l<\theta<u)=0.95.$

$\text{The sample mean is normally distributed: } \bar{X} \sim N\!\left(\mu,\; \frac{\sigma^{2}}{n}\right).$
(Page 26)

$\text{The standardized sample mean is } Z = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}}, \text{ and it follows } Z \sim N(0,1).$
=(Standard normal distr.)
(Page 26)

A z-score indicates how many standard deviations a value is above or below the mean.
(Page 28)

$\text{A 100(1-α)\% confidence interval for μ is } \bar{x} \pm z_{\alpha/2} \frac{\sigma}{\sqrt{n}}. $
(Page 29)

Because σ must be estimated from the data using the sample standard deviation S, which adds uncertainty. The resulting standardized statistic follows a t-distribution with n−1 degrees of freedom.
(Page 31)

$\text{A 100(1-α)\% confidence interval for μ is } \bar{x} \pm t_{\alpha/2,\,n-1} \frac{S}{\sqrt{n}}. $
(Page 32)

As n becomes large, the t-distribution approaches the standard normal distribution, so the CI becomes approximately:
$ \bar{x} \pm z_{\alpha/2} \frac{S}{\sqrt{n}}. $

(Page 33)

$ \bar{x} \pm z_{\alpha/2}\frac{\sigma}{\sqrt{n}}. $

(Page 34)

$ \bar{x} \pm t_{\alpha/2,\,n-1}\frac{S}{\sqrt{n}}. $

(Page 34)

The confidence interval forσ² is
$ \left(\frac{(n-1)S^{2}}{b},\; \frac{(n-1)S^{2}}{a}\right), $

where a and b satisfy
$ P(a < \chi^{2}_{n-1} < b) = 1-\alpha. $

(Page 35)

$\text{The T-statistic is defined as } T = \frac{\bar{X} - \mu}{S/\sqrt{n}} \text{ and follows a Student's } t\text{-distribution with } n-1 \text{ degrees of freedom.}$

(Page 31)

$\text{In a normal distribution, }1\sigma\text{ corresponds to }68.2\%,\;2\sigma\text{ to }95.4\%,\text{ and }3\sigma\text{ to }99.7\%\text{ confidence.}$
(Page 37)

$\text{In astronomy, the terms }1\sigma,\;2\sigma,\;3\sigma$
$\text{ are sometimes applied to non-normal distributions, leading to asymmetric or misleading uncertainty intervals.}$
(Page 37)

$\text{In frequentist statistics, }\theta\text{ is treated as a fixed constant, so only }P(X|\theta)$
$\text{ is a valid probability; }P(\theta|X)\text{ would require treating }\theta\text{ as random.}$
(Page 38)

$\text{MAP treats }\theta\text{ as a random variable with a prior, so the estimator maximizes the posterior }$
$P(\theta|X)\propto P(X|\theta)P(\theta),\text{ not just the likelihood.}$
(Page 38)

$\text{Marginalization removes a variable by integrating or summing}$$\text{ over it to obtain the PDF of a remaining variable, such as }$
$P(X)=\int P(X,Y)\,dY.$

(Page 39)

$\text{Conditioning expresses the probability of one variable given another, using }$
$P(X|Y)=\frac{P(X,Y)}{P(Y)}\text{ when }P(Y)>0.$

(Page 40)

$\text{Conditional probability represents updated knowledge of one variable after learning the value of another.}$
(Page 40)

$\text{The sum rule states that }P(A|B)+P(\bar{A}|B)=1\text{ for any statement A.}$
(Page 44)

$\text{The product rule states that }P(AB|C)=P(A|C)P(B|AC)=P(B|C)P(A|BC).$

(Page 44)

$P(B|AC)=\frac{P(A|BC)P(B|C)}{P(A|C)}.$

(Page 44)

$\text{The posterior }P(\theta|D,M)\text{ is the probability distribution for the parameters }$
$\theta\text{ after accounting for the data }D\text{ and model }M.$

ChatGPT:
Posterior: What we think the answer (θ) is after seeing the data.
Likelihood: How likely the data is if a certain answer (θ) were true.
(Page 45)

$\text{The likelihood }P(D|\theta,M)\text{ measures how well each parameter value explains the observed data.}$

ChatGPT:
Posterior: What we think the answer (θ) is after seeing the data.
Likelihood: How likely the data is if a certain answer (θ) were true.
(Page 45)

$\text{The prior }P(\theta|M)\text{ represents knowledge about the parameters before seeing the data.}$

ChatGPT:
Posterior: What we think the answer (θ) is after seeing the data.
Likelihood: How likely the data is if a certain answer (θ) were true.
(Page 45)

$\text{Because }P(X)\text{ is constant with respect to }\theta,\text{ the posterior satisfies } \log P(\theta|X)=\log P(X|\theta)+\log P(\theta).$
log(posterior)=log(likelihood)+log(prior)
(Page 46)

$\text{The MAP estimator is the value of }\theta\text{ that maximizes the posterior probability }P(\theta|X).$
(Page 46)

$\text{MAP includes the term }\log P(\theta),$
$\text{ so the estimator reflects both the data and prior beliefs about plausible parameter values.}$

MAP = likelihood + prior effects.
(Page 46)

$\text{The posterior describes the full range of parameter values supported by the data}, $
$\text{including uncertainty and multiple plausible solutions.}$

(Page 48)

$\text{A broad posterior indicates that the parameter is weakly constrained by the data.}$
(Page 48)

$\text{The posterior is }P(\theta|D,M)=\frac{P(D|\theta,M)P(\theta|M)}{P(D|M)}.$

(Page 48–49)

$\text{The likelihood is a Gaussian for the measured parallax }\varpi\text{ with mean }1/\theta\text{ and known variance }\sigma^{2}.$

(Page 50)

$\text{The posterior is }P(\theta|\varpi)\propto P(\varpi|\theta)P(\theta),\text{ where }P(\varpi|\theta)\text{ is the Gaussian likelihood and }P(\theta)\text{ is the chosen prior.}$

idk tbh
(Page 51)

$\text{An improper prior is a prior distribution that cannot be normalized because its total probability is infinite.}$

(Page 51)

$\text{An improper posterior has no defined mean, variance, or quantiles,}$
$ \text{so it cannot be used to summarize parameter uncertainty.}$

(Page 52)

$\text{A truncated prior limits }\theta\text{ to a finite range, ensuring the posterior is normalizable and yields meaningful summaries.}$

(Page 53)

$\text{The truncated prior sets }P(\theta)=\frac{1}{\theta_{\text{lim}}}\text{ for }0<\theta<\theta_{\text{lim}}\text{ and }0\text{ otherwise.}$

(Page 53)

$\text{Because the truncated prior restricts }\theta\text{ to a finite interval, the posterior integrates to a finite value.}$

(Page 54)

$\text{The prior dominates when the data are weak or uninformative,}$
$ \text{causing the likelihood to be broad relative to the prior.}$

(Page 55)

$\text{The prior influences the posterior by shaping how plausibility is distributed before considering the data,} $
$ \text{affecting the posterior especially when the data are weak.}$

(Page 56)

$\text{When the data are strong, the posterior distributions from different} $
$ \text{priors tend to converge because the likelihood dominates.}$

(Page 56)

$\text{MAP maximizes the posterior, while the posterior mean averages over the full posterior distribution,} $
$ \text{ so skewed or multimodal posteriors cause the values to differ.}$

(Page 57)

$\text{The posterior mean reflects the entire shape of the posterior and}$
$ \text{incorporates uncertainty more fully than a single maximizing value.}$

(Page 57)