Empirical Methods in Management

• Extent to which the results of the experiment can be generalized – from sample to population
  => Degree to which findings are representative

• => no external validity without internal validity

• History (what happens during the experiment)

• Maturation (changes in the test units themselves)

• Mortality (survivorship bias)

• Instrumentation (changes to the measuring instrument)

• Experimenter effect (reaction to the experimenter due to age, sex, race, …)

• Socially desirable behavior and/or demand effects

• Selection bias (due to the improper selection of test units)

• Testing effects / reactivity (caused by the process of experimentation)

• Regression to the mean (test units move to the mean in the process of the experiment)

• True: randomization

• Quasi: no randomization

• Randomization

• Statistical control: measuring extraneous variables and adjusting for their effects through statistical methods (e.g. demographics).

• Matching: matching participants on a set of key variables before assigning them to conditions.

• Design control: using specific experimental designs to control for confounding effects (treating extraneous variables as additional IV’s)

• => between-group designs: one group gets only one treatment, study different groups

  • +: Simplicity, Lower fatigue and practice effects, Useful if it’s impossible to switch to other experimental conditions (e.g. male vs. female)

  • -: High number of participants necessary, particularly for complex designs (many IV’s with many levels), Weaker effects (manipulation needs to be strong)

• => within-subject designs: every participant gets several or all treatments, repeated testing

  • +: Economy (fewer participants necessary), Higher sensitivity (higher chance that manipulation has an effect)

  • -: Carry-over effects from one to condition to the next, Fatigue and practice effects, Remedy: Counterbalancing (random sequence)

1. Nominal: Numbers identify and classify objects (e.g., gender, social security number).

2. Ordinal: Numbers indicate relative positions of objects but not the magnitude of differences between them (e.g., preferences).
   
   Special case rating scale: actually an ordinal scale, but can in most cases be interpreted as interval scale
    

3. Interval: Intervals between data points on the scale are equal (e.g., temperature).

4. Ratio: All powers of the prior scales as well as a meaningful absolute zero (e.g., age, length).

Comparative (nonmetric)

• Paired comparison: What do you like more, Pepsi or Coke?

• Rank order: order your favourite softdrinks brands.

• Constant sum: distribute the 5 points to your favourite brands.
  

Noncomparative (metric)

• Continuous rating scale: place a mark on a continuous line

• Itemized rating scale

  • Likert: scale from 1 (strongly agree) to 5 (strongly disagree)

  • Semantic differential: Seven-point scale with bipolar labels (powerful - - - - - - - weak)

  • Stapel: Unipolar ten-point scale from -5 to +5 without 0.

Results are independent of examiner

- consistent results, if measurements are repeated

- free from random error (but not free from systematic error)

Whether what was tryed to measure was really measured.

Process of selecting test units

• Non Probability sampling

  • Convenience samples: samples drawn at the convenience of the interviewer

  • Judgmental samples: requires a judgment or an “educated guess” as to who should represent the population

  • Quota samples: specified percentages of the total sample for various types of individuals to be interviewed

  • Snowball samples: require respondents to provide the names of prospective respondents

• Probability Sampling (

  • Simple random sampling: the probability of being selected into the sample is “known” and equal for all members of the population.

  • Systematic sampling: way to select a random sample from a directory or list that is much more efficient than simple random sampling

  • Stratified sampling: separates the population into different subgroups and then samples all of these subgroups

  • Cluster sampling: method in which the population is divided into subgroups, called “clusters,” each of which could represent the entire population

68% within \(1\sigma\)

95% within \(2\sigma\)

error of falsely rejecting a null hypothesis

incorrectly retaining a false null hypothesis

Multiplied by 100 it represents the percentage of variation in the outcome that can be explained by the model

• No (perfect) multicollinearity: There should be no perfect linear relationship between two or more of the predictors

  • Variance inflation factor (VIF): can be used to assess and eliminate multicollinearity. VIF is a statistical value that identifies what independent variable(s) contribute to multicollinearity and should be removed. Any variable with VIF of greater than 10 should be removed.

• Normally distributed errors: it is assumed that the residuals in the model are normally distributed values with a mean of 0, i.e. they are most frequently zero, close to zero and rarely much greater than zero

• Homoscedasticity: at each level of the predictor variable(s), the variance of the residual terms should be constant

• Linearity: The inclusion of each independent variable preserves the straight-line assumptions of multiple regression analysis

Total SS = Explained SS + Residual (unexplained) Sum of Squares

1. Define the Problem
2. Determine Research Design
3. Design Data Collection Method and Form
4. Design Sample and Collect Data
5. Analyze and Interpret Data
6. Prepare the Research Report & Presentation

• Discovery
• Design
• Data
• Analysis
• Synthesis
• Report

• States a relationship between an independent and dependent variable.
• Based on literature review, exploratory research and/or theoretical reasoning
• Usually stated with a null and an alternative hypothesis.

• Rule 1. Avoid complexity. Use simple, audience-specific language if possible.
• Rule 2. Avoid leading and loaded questions. Use neutral questions.
• Rule 3. Avoid ambiguity. Be as specific and precise as possible.
• Rule 4. Avoid double-barreled questions. Ask about one topic at a time.
• Rule 5. Avoid making assumptions. Ask, don’t assume.
• Rule 6. Avoid burdensome questions. Use ‘top-of-mind questions’.

• Multiple contacts (mail & phone)
• Why important (strong appeals)
• Credible sponsor or affiliation
• Anonymity, confidentiality
• Personalization
• Incentives (monetary or nonmonetary)
• Survey length & design