QM

Parameter of the sample: b₀ and b₁ are parameter of the sample. In the regression model they are estimators which are based on the calculated straight line through our sample data. b0 and b1 are
unbiased estimators of ß₀ und ß₁

The coefficients of a linear regression model are b0, b1, … bn, whereby b0 is the y-intercept and b1, … bn are slope parameters. Every bi (for i ≥ 1) relates to a certain independent variable xi. The coefficients are determined by minimizing the sum of squared deviations between the actual data points and the regression line (yi – ŷi)2. The sum of the squared deviations represents the so called Sum of Squared Errors (SSE).

SSE is an important statistic because it is the basis for other statistics that assess how well the linear model fits the data. It is possible to compute the standard error of estimate (sε) from the SSE. sε has to be as little as possible and helps to evaluate different models. Nevertheless it cannot be used as an absolute measure since it has no upper limit. Furthermore it can be used to calculate the coefficient of determination (R2).

Short-Cut formulas make the calculation of those parameters a lot easier and they save computer power. Moreover, they can prevent computational errors due to minor inaccuracy by reducing the necessary steps to generate an answer.

In order to provide a good estimation for the regression model, the distribution of the error term has to follow the MARKOV-GAUSS CONDITIONS:

1. Expected Value is zero
2. The error terms are independent of the underlying dependent variable
3. Homoscedasticity: The variance remains constant for all error terms
4. There is no autocorrelation between the error terms

The best way to determine if those conditions are met is by plotting the residuals on the dependent variable. This way, any discrepancies can be spotted more easily. If one of those conditions is violated, you can fix it through a monotonous transformation: Log Transformation, quadratic transformation, radical transformation (root), reciprocal transformation.

The null hypothesis is always ��₀: ��_i = 0. This is because the rejection of a hypothesis is more powerful than the confirmation of the opposite hypothesis. Thereby, a coefficient is considered significant, if he remarkably differs from 0. The test itself consists of a simple t-test, with the test statistic:

The parameter in use is R-squared, which is referred to as the coefficient of determination. It is defined as follows (SST = Total Variation / SSR = Explained Variation / SSE = Unexplained Variation):

--> see picture

The R-squared should, however, not be taken too seriously in absolute terms, because it is often overrated. A value of 0.2 does not have to be useless, but also a value of 0.95 can be quite low in certain situations. Moreover, it is not possible for the R-squared to be lowered because of an additional variable. If this variable turns out to be absolutely useless, it will just not appear in the model, but the existing factors will not have less explanatory power because of that. A measure which would actually lower the value with additional independent variables is the ADJUSTED R2, which is therefore a more accurate measure.

The error terms are directly linked to the SSE, since the latter is the sum of all squared errors. The larger the error terms, the larger the SSE becomes and the smaller the R-squred. Error terms are basically variation that is not explained by the regression model.

Regression model of returns: μ = ß0 + ß1* μm + ε.

The market model implies that a stock return μ is linearly dependent on the equity market (represented by the return on market portfolios μm). Systematic risk is based on the market and measures the volatility of the asset price which is affected but also compensated by the market. Therefore the coefficient ß1 is called the stock’s beta coefficient, which measures how sensitive the stock’s rate of return is to changes in the level of the overall market.

Unsystematic risks are the so called firm-specific risks and are only a result of activities and events of one corporation and therefore they can be diversified. Thus, they can be regarded as non compensated risk. If an investor expects the markets to rise, it makes sense to hold a portfolio with a ß > 1. If he expects the markets to fall, it makes sense to hold a portfolio with a ß < 1. The systematic risk is measured by the coefficient of determination R2 and the unsystematic risk as a
consequence by 1-R2.

The regression table is usually divided into three parts.

The first delivers absolute values of the regression model, containing R, R-squared, adjusted R-squared, SEE (Standard Error of estimate, this is similar to the square root of the MSE) and number of observations.

The second part contains the ANOVA, which is an F-test considering the validity of the entire model, containing information about the SSE, SST and SSR.

The last part is considering single coefficients and their significance with a t-test.

The PREDICTION INTERVAL gives an interval of where the next observation will be located. This is usually a larger interval, because extreme events are likely to happen. For a 95% prediction interval, only one in 20 observations should be outside the borders.

The CONFIDENCE INTERVAL is usually smaller and it aims for the expected value. The CI gives the borders of where the actual optimal regression line of the population will probably be located. Due to aggregation (law of large numbers), better predictions can be made about the mean than of a single realization. Both measures are due to the uncertainty of the sample.

Both of those intervals experience an outward curvature. This is due to the fact that the predictions are the most precise around the mean of x. The farther out you go, the more unprecise the estimate becomes. Moreover, both intervals decrease with larger sample sizes n, because of the law of large numbers. The more observations have already been realized, the better future predictions become.

In the case of a single regression, the correlation between dependent and independent variable is equal to the square root of R-squares. However, the correlation does not actually make a valid point about the regression model. R-squared is a more important measure for this.

The correlation test determines a correlation between both variables. However, the test usually has very similar results to the t-test of a regular regression coefficient (beta), because it also measures the linear relationship between variables. The hypothesis is denoted as H₀: ρ = 0. If there is no linear relationship between variables, the (linear) correlation coefficient rho is also zero.

Homoscedasticity is one of the Gauss-Markov conditions for residuals, which has to be kept. It states that the variance of error terms is constant throughout all values of x. If the variance changes, the residuals are heteroscedastic. The easiest way to check for heteroscedasticity is looking at a plot of error terms.  A homoscedastic model will display a uniform cloud of dots, whereas heteroscedasticity will result in patterns such as a funnel shape, indicating greater error as the dependent variable increases.

SST, SSTR and SSE are Sum of Squares whereas MSTR and MSE are Mean Squares and hence the averages of the Sum of Squares. Basically the functions of calculating the F-value and therefore testing if the model has a good implication. In detail these are:

• SST: Sum of Squares Total, the total variation of y
• SSTR (= SSR): Sum of Squares for Regression, the variation which is explained by the regression mode.
• SSE: Sum of Squares for Error, the unexplained variation
• MSTR (= MSR): Mean Square for Regression, MTSR= SSR / k
• MSE: Mean Square for error, MSE = SSE / (n-k-1)

This implies: SST = SSTR + SSE. Basis for the mean of the Squares are the degrees of freedom. For the regression the degrees of freedom corresponds to k, the number of all estimated parameters. For the residual the degrees of freedom equate to (n-k-1). Finally the F value in ANOVA is calculated by dividing MSTR by MSE.

The F-test does not just look at a single coefficient, but at the explanatory power of the entire model. However, because there is no t-distribution, we assume the Fisher distribution.

The F-test is also more «robust» than a t-test. In contrast to the t-test, it has a lower alpha-error (stating an invalid model as valid is lower) and it is unaffected by multicollinearity, because it looks at the total SSE and not the contributions of each coefficient.

--> While the t-test only looks at one coefficient, the F-test looks at all of them simultaneously.

The Capital Asset Pricing Model states that the return on a stock is related to the return on the market through the following linear relationship: --> see picture

In here, et is the residual error. If the CAPM does explain the relation between the returns of the stock and that of the market, the intercept α should be zero. The coefficient β is a measure of the systematic or non-diversifiable risk. The higher it is, the more sensitive the stock is to the market. High beta stocks have more systematic risk than low beta stocks. The CAPM implies economically is that since high beta stocks have more risk, they should also earn higher returns.

The Sum of Squared Errors (SSE) corresponds directly to the standard deviation of the error term and affects the coefficient of determination R2 as well as the F-value. When the SSE is large, this implies also a large sε but therefore a small R2 and a small F value. In such instance the explanatory power of the regression model is poor. The smaller the SSE the better the explanatory power of the model. With a SSE and sε of 0 the regression model describes the relation of the dependent and the independent variables perfectly.

We test the regression coefficients to check whether the intercept coefficient is significantly different from zero and if the slope coefficients are significantly different from zero too. The regression coefficient is estimated and is therefore affected by some uncertainty. This uncertainty is measured by the standard error of each coefficient. With this information three different approaches exist to test whether the intercept is significantly different from zero:

• By dividing the estimated coefficient by its standard error we get the t-ratio aka the t-statistics. It tells us how many standard error units the coefficient is away from zero. As a rule of thumb, a tstatistic with an absolute value larger than 1.96 means that the corresponding coefficient is statistically different from zero. It is more than two standard deviations away from its mean.
• We can look at the p-value. If it is small enough, then we accept the idea that the corresponding coefficient is statistically different from zero
•  We can build a 95% confidence interval by adding plus/minus 1.96 times the standard error to the estimated coefficient. If zero is not included in this interval, this means that we can exclude the fact that the corresponding coefficient may be equal to zero

ALL these approaches are founded on the hypothesis of normally distributed errors with constant variance.

Multicollinearity occurs, if independent variables x are correlated among each other. Due to the ceteris paribus condition, the t-test only measures the impact if one variable changes. However, because this is unlikely due to high correlation, the t-test of correlated coefficients gives a very low result and might even reject all correlated factors as insignificant. In contrast to the t-test, the F-test cannot be tricked by multi-collinearity.

--> If the model has a very high F-value, but the factors are mostly insignificant, this is a good indicator of multicollinearity.

To compensate for multicollinearity, you have to try and neglect single independent variables or start with one and continue adding. It is an art form to find a model which has a large R-squared, but no autocorrelation.

The main objective of the Durbin-Watson test is to determine first-order autocorrelation among the residuals. This means it looks at the deviation from yesterday’s residual and looks for patterns. The resulting value d can take on values between 0 and 4.

A value of 2 implies no autocorrelation whatsoever. A significantly larger value indicates negative autocorrelation and a value close to zero implies strong positive autocorrelation.

There are some major differences between the Lagrange and the Kuhn-Tucker approach, even though both represent a saddle function. The largest of them being the inequalities in the Kuhn-Tucker restrictions. It is important that all the restrictions have to use a ≤ in order to define the set of feasible decisions.

Another difference is, that the multipliers are non-negative. In the world of Lagrange, they can only be negative, but for Kuhn-Tucker, it would not make sense.

This leads to the Complementary Condition: Either the multiplier or the restriction have to be equal to zero. If the multiplier is zero, the restriction is not binding and the optimal solution is not on its border. Therefore, the derivative on the border of the restriction is not zero. If the multiplier is positive, the restriction has actual value of the decision maker. Hence, the optimum is located on the border of said restriction.

--> Kuhn-Tucker only works for maximization problems. If there is a minimization problem, you need to switch signs and multiply by (-1).

The probability density function corresponds to the probability function, but in the continuous case. It is defined by the derivative of the Cumulative Distribution Function (CDF). In the continous case, the probability of a single realization is always zero, and we need to express the probability
by taking the integral in an interval under the curve

The cumulative distribution function (CDF) F(x) = P(X≤xi) indicates the probability that X takes maximally a value of x. It is called cumulative since it’s the accumulation of the probabilities. Thus,
the y-value for the highest x-value must always be 1.

A joint distribution function F(xi ,yk) = P(X≤xi,,Y≤yk) indicates the probability that X takes at most a value of x and Y at most a value of y. Unlike the previous functions that are based on one
random variable, this (and the following two) functions are used when two random variables exist

A conditional distribution function f(xi |Y=yk) describes the distribution of a variable X given the outcome of another variable Y. It is equal to the joint probability of the two variables divided
by the marginal probability of the given variable

The marginal distribution function fx(xi) = P(X=xi) indicates the probability of X = xi regardless of the value of Y

If we take n independent random variables with mean µ and variance σ2, then if n is large the sum of these random variables will be approximately normally distributed with mean nµ and variance nσ2. Thus, even though we might not know the shape of the distribution of the entire population, the central limit theorem says that we can treat the sampling distribution as if it were normal. Of course, in order for the conclusions of the theorem to hold, we do need a sample size that is large enough. The sampling distribution is asymptotically normal only if n ≥ 30. Many practices in statistics, such as those involving hypothesis testing or confidence intervals, make some assumptions concerning the population that the data was obtained from. One assumption that is initially
made in a statistics course is that the populations that we work with are normally distributed

The test statistic is used in hypothesis testing. In particular, it is calculated from the sample in order to decide whether the null hypothesis made on the population should be rejected.

Normal distribution: it plays a central role in econometrics and in statistics. The central limit theorem allows it to approximate several other distributions by using the normal distribution. Especially the standard normal distribution, as a special case of the normal distribution with mean zero and standard deviation one, is widely used in statistics for hypothesis testing etc. The normal
distribution is symmetric.

Student distribution: it is, like the normal distribution symmetric but it has fatter tails, which means that extreme outcomes are assigned with higher probability compared to the normal distribution. It is used in statistics when the variance of the population is unknown. For n≥30 it can be approximated by the normal distribution

Chi-Square distribution: it is the sum of squares of independently standard normally distributed random variables. The degrees of freedom the chi-square distribution corresponds to the number
of standard normally distributed variables that are summed up.

F distribution: it takes its name from Fischer. It is the ratio of two independent random variables which follow a chi-square distribution with v1 and v2 degrees of freedom, each divided by its degrees of freeom. The F-distribution is characterized by v1 and v2 degrees of freedom and is use in ANOVA (Analysis of Variance) to test if the populations of two samples have the same variance

A random variable is presumed to be lognormal distributed if the natural logarithm of the variable is normally distributed, that is if y = ln(x) is normally distributed (Y~N(µy, σy). The parameters for the lognormal-distribution are determined by the parameters of the normal distributed variables. In contrast to a (symmetric) normal distribution the graph is positively skewed (skewed to the right) and has only positive x-values. If ln(X)=1, then the lognormal-distribution results in the normal-distribution. An example is given by stock prices, which are usually lognormally distributed. When we calculate the
continuous returns by the taking the logarithm of the ratios, we end up with normally distributed returns.

The mean as a measure of central tendency is the expected value and thus the average value of all outcomes of the random variables. The expected value of a random variable is the value that one would expect to find if he could repeat the process infinite number of times and take the average of the values obtained. The expected value is a linear operator, and in the discrete case it is obtained by summing up the products of each probability with the corresponding value of the
random variable

The variance measures the dispersion of the distribution. It equals the expected quadratic
deviation of the random variables from the mean and is called the second central moment. The variance has no unit of measurement. It can be calculate as the expectation of the square minus
the square of the expectation. The variance is a nonlinear operator.

The standard deviation is the square root of the variance and therefore makes it easier to interpret the deviation of the random variables from the mean. It’s also a measure of dispersion
and it is commonly used in finance to express risk

Skewness is the third central moment and is a measure of the asymmetry of the distribution around its mean. It is defined as S = E((Y-μ)3)/σ3. In the case of:

• Positive skewness: the distribution has a longer tail to the right, the mean is greater than the median, which is turn greater than the mode. The mean is pulled up by the few very high observations
• Negative skewness: the distribution has a longer tail to the left, with mean being below the median and the mode. The mean is pulled down by the few very low observation

Kurtosis is the fourth central moment and measures the peakedness of the distribution with K = E((Y- μ)4)/σ4. Kurtosis of a normal distribution is 3. We distinguish between:

o Platykurtic: quite flat

o Leptokurtic: high and narrow

o Mesokurtic: like a triangle, resembles to a normal distribution

For Example: Leptokurtic distributions are found in asset returns, where there isn’t a continuous trading. Such security markets close over night or at weekends, so the information which has an influence on asset prices but is published when the market is closed will have an impact on prices when the market reopens. This causes a jump between the previous closing price and the opening price. As a result there will be higher frequencies of large negative or positive returns than would
be expected if the market were to trade continuously

The variance is the expectation of the square minus the square of the expectation. The covariance is the expectation of the product minus the product of the expectations of the two random
variables

In a variance-covariance matrix the covariances between several pairs of variables are displayed. It contains both the variances (as the covariances of a variable with itself) on the diagonal and the covariances of each variable with another variable everywhere else. Therefore the covariances occur twice in the matrix as cov (x,y) = cov (y,x), meaning the matrix is symmetric.

The correlation coefficient is a unit-free measure of the strength and the direction of a linear relationship between two variables. Contrary to the covariance the size of the correlation coefficient is not influenced by the values of the observations. For Example: a larger covariance could be more to do with having high values to the observations than to a closer association between two variables. It is calculated by dividing the covariance between X and Y by the product of the standard deviation of X and the standard deviation of Y. Correlation implies simply a measure of statistical association as there is no inference of causality in the statistic. It indicates how strong the linear association is between two variables, it cannot explain the changes in them. The larger the value is, the stronger the linear relationship. The coefficient is always between -1 and 1. The correlation is matrix is again a symmetric matrix which contains 1 in the diagonal (because the correlation of a variable with itself is always 1), while the correlations are located in the remaining of the
matrix, which is again symmetric