statistics for atmospheric science

\(F_X(x)=P(X\leq x)\) continuous random variables

\(F_X(x)=\sum_{x_i< x}P(X=x_i)\)  discrete random variables

1. \(F_X\) monotonically increasing (\(0\leq F_X(x)\leq 1\))
2. \(lim_{x\rightarrow -\infty}F_X(x)=0,\;\;lim_{x\rightarrow \infty}F_X(x)=1\)
3. \(P(X \epsilon [a,b])=P(a\leq X\leq b)=F_X(b)-F_X(a)\)

Probability mass function (only for discrete variables!):

\(f_X(x)=P(X=x)\)

Probability density function (PDF, for continous random variables!):

\(f_X(x)=\frac{dF_X(x)}{dx}\)

proberties:

1. \(f_X(x)\geq 0\)
2. \(\int f_X(x)dx=1\;(cont.)\;\;\sum_{X\epsilon \Omega}f_X(x)=1\;(discrete)\)
3. \(P(X\epsilon [a,b])=P(a\leq X\leq b)=F_X(b)-F_X(a)\)

continuous random variables: 

Random variables X and Y are independent if for any x and y:

\(P(X\leq x, Y\leq y)=P(X\leq x)P(Y\leq y)=F(x)G(y)\)

where F(x) and G(x) are the corresponding CDFs.

discrete random variables:

Random variables X and Y are independent if for any \(x_i\)and \(y_i\):

\(P(X\leq x_i,Y\leq y_j)=P(X\leq x_i)P(Y\leq y_j)\)

Percentile: quantiles expressed in percentages: The 0.2 quantile is the 20th percentile

Quartiles: are 25th and 75th percentiles 

Median: is the 0.5-quantile

The nth moment \(\mu_n\) of a probability density \(f_X(x)\) is defined as:

• (cont.):  \(\mu_n=E(X^n)=\int x^n*f_X(x)dx\)
• (discr.):  \(\mu_n=E(X^n)=\sum x^n_k * f_X(x_k)\)

The n th central moment \(\mu'_n\) of a probability density \(f_X(x)\) is defined with respect to the first moment (\(\mu\)) as

 \(\mu_n'=E((X-\mu)^n)=\int (x-\mu)^n * f_X(x)dx \)

The expected value, also called the mean is defined as the first moment:

\(\mu=E(x)=\int x*f(x)dx \)

The expected value can be physically seen as the centroid of mass in physics.

The variance is defined as the second central moment:

\(\sigma^2=Var(x)=E((X-\mu)^2)=E(X^2)-\mu^2\)

The variance gives the spread around the expected value. 

The skewness is the third central moment divided by the standard diviation to the third power:

\(.\\\gamma_1=E[(\frac{X-\mu}{\sigma })^3 ]=\frac{E[(X-\mu )^3]}{(E[(X-\mu )^2])^{3/2}}\)

Kurtosis (measure of peakness)

The kurtosis of any univariate normal distribution is 3. It is common to compare the kurtosis of a distribution to this value. Distributions with kurtosis less than 3 are said to be platykurtic, although this does not imply the distribution is "flat-topped" as sometimes reported. Rather, it means the distribution produces fewer and less extreme outliers than does the normal distribution. An example of a platykurtic distribution is the uniform distribution, which does not produce outliers.

Der Exzess gibt die Differenz der Wölbung der betrachteten Funktion zur Wölbung der Dichtefunktion einer normalverteilten Zufallsgröße an.

The mode is the value that appears most often in a set of data. For a continuous probability distribution it is the peak. 

probability density function (PDF):

\(f(x)=\frac{1}{b-a}\;for\;x\;\epsilon\;[a,b]\)

cumulative distribution function (CDF):

\(F_X(x)=\left\{ \begin{array}{c} 0\;\;\;for\;x< a\\\frac{x-a}{b-a}\;\;\;for\;\;x\;\epsilon\;[a,b]\\1\;\;\;for\;x\geq b \end{array} \right.\)

\(f_X(x; \mu,\sigma)=\frac{1}{\sqrt{2\pi\sigma^2}}exp \left(-\frac{(x-\mu) ^2}{2\sigma^2}\right)=\mathcal{N}(\mu,\sigma)\)

A signal is said to be intermittent if rare events of large magnitude are separated by long periods with events of low magnitude. Spatial intermittency implies that the signal displays localized regions with events of large magnitude, and wide areas with events of low magnitude. 

• PDFs of intermittent flows are not Gaussian. 
• Kurtosis is often used as a measure of intermittency (high intermittency means high kurtosis) 

Yes, then:

• the distribution only consists of one constant 
• mean, median and mode are the same 

The Cauchy distribution 

• expected value, variance and standard deviation do not exist since it's integrals are infinite. 

Given a sequence of random variables \(X_1,X_2,...\) with mean \(\mu\) . Then it holds:

\(lim_{n\rightarrow \infty}\frac{1}{n}\sum^{n}_{i=1}X_i\rightarrow \mu\)

Given a sequence of independent and identical distributed random variables \(X_1,...,X_n\) with expected value \(\mu\) and variance \(\sigma^2\) , then the distribution of \(S_n=\frac{1}{n}(X_1+...+X_n)\)is approximately normal with mean \(\mu\) and variance \(\frac{1}{n}\sigma^2\) or, 

\(\sqrt{n}\left(\frac{1}{n}\sum\limits^n_{i=1}X_i-\mu \right)\xrightarrow d\mathcal{N}(0,\sigma)\)

the \(\xrightarrow d\) reads "converges in distribution to".

How large does n to be chosen? Depend on the underlying distributions of the sample sequences. 

For any random variable and c>0 there holds:

\(P(|X-E(X)|\geq c)\leq\frac{Var(X)}{c^2}\)

Quantities estimated from a given sample are often referred to as empirical or sample quantities. \(\hat{\;\mu} \)

The corresponding true or model quantities are often referred to as the theoretical. \(\mu\)

Given a sample \(x_1,...,x_N\)of a random variable X. Consider a parameter \(\Theta\) on X, e.g., the mean \(\mu\) .

Then the estimator \(\hat{\;\Theta}\) is a function of a sample (i.e., is a statistics) of the random variable X which assigns to the sample values which distribution depend on (and are close to) \(\Theta\) .

sample mean: \(\bar{x}=\widehat{\mu}=\frac{1}{N}\sum\limits^{N}_{i=1}x_i\)

sample variance: \(\widehat{Var}(x)=\frac{1}{N-1}\sum\limits^{N}_{i=1}(x_i-\bar{x})^2 \)

sample variance with known \(\mu\): \(\widehat{Var}(x)=\frac{1}{N}\sum\limits^{N}_{i=1}(x_i-\bar{x})^2 \)

standard deviation: \(\widehat{s}=\sqrt{\widehat{Var}(x)}\)

1. \(Z=\alpha+X\)
   • \(\mu_Z=\alpha+\mu_X,\;\;\;\sigma_Z^2=\sigma_X^2\)
2. \(Z=\alpha X\)

• \(\mu_Z=\alpha\mu_X,\;\;\;\sigma_Z^2=\alpha^2\sigma^2_X\)

1. \(Z=X+Y\)

• \(\mu_Z=\mu_X+\mu_Y,\;\;\;\sigma_Z^2=\sigma^2_X+\sigma^2_Y\)

1. \(Z=X*Y\)

   • \(\mu_Z=\mu_X*\mu_Y\)

Note the density function and the cumulative distribution of composed random variables (as X+Y or XY) is in general not easy to determine, although mean and variance can easily be determined. 

The histogram, which contains the relative occurrence divided by the bin width. 

non-trivial:

1. square-root choice: \(k=\sqrt{n}\)
2. Sturges' formula (assumes Gausssian): \(k=log_2n+1\)
3. Rice rule \(k=ceil(2n^{1/3})\)

The estimator \(\widehat{\Theta}\), as a function of the random variable X, is again a random variable. Therefore every estimator has an expected value and variance. 

An estimator is called consistent if:

\(P(|\widehat{\Theta}-\Theta|>\epsilon)\rightarrow0\;\;for\;\;N\rightarrow\infty\)

for all \(\epsilon >0\)

Example: The estimator for the expected value \(\widehat{\Theta}=\widehat{\mu}\) (the sample mean) is consistent (law of the large numbers). 

\(\widehat{\mu}=\frac{1}{N}\sum\limits^N_{i=1}x_i\)

\(MSE(\widehat{\Theta})=E[(\widehat{\Theta}-\Theta)^2]\)

The MSE is also called as risk 

\(Var(\widehat{\Theta})=E[(\widehat{\Theta}-E(\widehat{\Theta}))^2]\)

Bias: 

\(B(\widehat{\Theta})=E(\widehat{\Theta})-\Theta\)

An estimator is called unbiased if and only if 

\(B(\widehat{\Theta})=0\)

\(MSE(\widehat{\Theta})=Var(\widehat{\Theta})+(B(\widehat{\Theta}))^2\)

so for unbiased estimator it holds:

\(MSE(\widehat{\Theta})=Var(\widehat{\Theta})\)

1. The sample mean is an unbiased estimator of the expected value
2. The sample variance  \(\widehat{Var}(x)=\frac{1}{N}\sum\limits^N_{i=1}(x_i-\bar{x})^2\) is an not unbiased, but asymptotically unbiased estimator
3. However, the sample variance \(\widehat{Var}(x)=\frac{1}{N-1}\sum\limits^N_{i=1}(x_i-\bar{x})^2\) is an unbiased estimator. 

Given an estimate \(\widehat{\Theta}\) of \(\Theta\). An interval \((\widehat{\Theta}_L,\widehat{\Theta}_U)\) around \(\widehat{\Theta}\) is named a \((1-\alpha)\) confidence interval if 

\(P(\Theta\in (\widehat{\Theta}_L,\widehat{\Theta}_U))=1-\alpha\)

A 95% confidence interval covers the true value in 95% of the cases. 

t-Distribution: 

\(f_X(x;\nu)=c(\nu)\left(1+\frac{x^2}{\nu}\right)^{-\frac{\nu+1}{2}}\)

with a constant \(c(\nu)\). \(\nu\in\mathbb{N}\) is called the degree of freedom. For \(\nu\rightarrow \infty\) it converges to the \(\mathcal{N}(0,1)\) distribution. 

The t-Distribution is used to estimate the mean of a normally distributed population when the sample size is small and population standard deviation is unknown. 

The empirical mean \(\bar{x}\) has a distribution \(\mathcal{N}(\mu, \frac{\sigma}{\sqrt{n}})\) while

\(Z=\sqrt{n}(\bar{x}-\mu)/\sigma\)

has distribution \(\mathcal{N}(0,1)\) .

The value z is such that \(P(-z\leq Z\leq z)=1-\alpha\)

This yields: \(P(\bar{x}-z\frac{\sigma}{\sqrt{n}}\leq\mu\leq\bar{x}+z\frac{\sigma}{\sqrt{n}})=1-\alpha\)

The confidence interval can be easily found now. 

Significance tests aim at verification of a hypothesis based on statistical data:

1. Parametric tests consider hypothesis regarding parameters of the distribution 
2. Non-parametric tests consider hypotheses not involving paramters (e.g. distributaions are the same or different)

1. Formulate Null hypothesis and an alternative hypothesis 
2. Choose significance level \(\alpha\)
3. Choose significance test and test statistic; clarify assumptions to be made 
4. Calculate Null distribution and critical value 
5. Calculate test statistic and/or p-value 
6. Decide whether Null hypothesis is rejected or not 

Significance level and critical value are defined such that:

\(P(|T|\geq t_{crit})\equiv\alpha\)

i.e., the probability that T falls in the rejection region (Q) although the \(H_0\) (null hypothesis) is true (small probability).

Then the Null hypothesis is rejected in case \(p_{obs}\leq\alpha \) or \(|t_{obs}|\geq t_{crit}\)

error of the first kind or \(\alpha\)-error: Rection of the Null hypothesis \(H_0\) although it is true

Probability of this error is \(P(H_0 rejected\; | \;H_0 true)=\alpha\)

error of the second kind or \(\beta \)-error: No rejection of the Null hypothesis although it is wrong 

Probability of this error is \(P(H_0\;not\;rejected\;|\;H_0\;false)=\beta \)

The reduction ofthe one error leads to increase of the other, unless we can increase sample size!