Empirical Methods in Management

Process of selecting test units

• Non Probability sampling

  • Convenience samples: samples drawn at the convenience of the interviewer

  • Judgmental samples: requires a judgment or an “educated guess” as to who should represent the population

  • Quota samples: specified percentages of the total sample for various types of individuals to be interviewed

  • Snowball samples: require respondents to provide the names of prospective respondents

• Probability Sampling (

  • Simple random sampling: the probability of being selected into the sample is “known” and equal for all members of the population.

  • Systematic sampling: way to select a random sample from a directory or list that is much more efficient than simple random sampling

  • Stratified sampling: separates the population into different subgroups and then samples all of these subgroups

  • Cluster sampling: method in which the population is divided into subgroups, called “clusters,” each of which could represent the entire population

68% within \(1\sigma\)

95% within \(2\sigma\)

error of falsely rejecting a null hypothesis

incorrectly retaining a false null hypothesis

Multiplied by 100 it represents the percentage of variation in the outcome that can be explained by the model

• No (perfect) multicollinearity: There should be no perfect linear relationship between two or more of the predictors

  • Variance inflation factor (VIF): can be used to assess and eliminate multicollinearity. VIF is a statistical value that identifies what independent variable(s) contribute to multicollinearity and should be removed. Any variable with VIF of greater than 10 should be removed.

• Normally distributed errors: it is assumed that the residuals in the model are normally distributed values with a mean of 0, i.e. they are most frequently zero, close to zero and rarely much greater than zero

• Homoscedasticity: at each level of the predictor variable(s), the variance of the residual terms should be constant

• Linearity: The inclusion of each independent variable preserves the straight-line assumptions of multiple regression analysis

Total SS = Explained SS + Residual (unexplained) Sum of Squares

1. Define the Problem
2. Determine Research Design
3. Design Data Collection Method and Form
4. Design Sample and Collect Data
5. Analyze and Interpret Data
6. Prepare the Research Report & Presentation

• Discovery
• Design
• Data
• Analysis
• Synthesis
• Report

• States a relationship between an independent and dependent variable.
• Based on literature review, exploratory research and/or theoretical reasoning
• Usually stated with a null and an alternative hypothesis.

• Rule 1. Avoid complexity. Use simple, audience-specific language if possible.
• Rule 2. Avoid leading and loaded questions. Use neutral questions.
• Rule 3. Avoid ambiguity. Be as specific and precise as possible.
• Rule 4. Avoid double-barreled questions. Ask about one topic at a time.
• Rule 5. Avoid making assumptions. Ask, don’t assume.
• Rule 6. Avoid burdensome questions. Use ‘top-of-mind questions’.

• Multiple contacts (mail & phone)
• Why important (strong appeals)
• Credible sponsor or affiliation
• Anonymity, confidentiality
• Personalization
• Incentives (monetary or nonmonetary)
• Survey length & design

• Environment: Artificial
• Control: High
• Reactive Error: High
• Demand Artifacts: High
• Internal Validity: High
• External Validity: Low
• Time: Short
• Number of Units: Small
• Ease of implementation: High
• Cost: Low

• Environment: Realistic
• Control: Low
• Reactive Error: Low
• Demand Artifacts: Low
• Internal Validity: Low
• External Validity: High
• Time: Long
• Number of Units: Large
• Ease of implementation: Low
• Cost: High

• When you need to detect small differences between known stimulus objects (e.g., Pepsi and Coke).
• When you want a scale that is easily understood and applied.
• When you have fewer theoretical assumptions (e.g. about what all constitutes service quality, brand image, etc.).
• When you want to reduce halo or carryover effects from one judgment to another
• When respondent should make a trade-ˇoff (e.g. importance judgments)
• When you have no need to generalize beyond the stimulus objects scaled.
• When the ordinal nature of the data is sufficient for your planned data analysis.

1. Exploratory: helps learn more about the problem, terms and definitions, or identify research opportunities. Qualitative research

2. Descriptive: describes the phenomena of interest. Secondary data analysis, survey research

3. Experimental: uncovers underlying causes of a problem. Experiments

start with a broad literature review and theory, narrow it down to specific hypotheses and test these through the collection of data.

"Classical research approach"

move from very specific (“interesting but unexplained”) observations in our data to a detection of patterns, up to a formulation of tentative hypotheses and a theoretical framework.

"Big data approach" 

• Voluntary participation: Make sure that study participants are taking part in the study voluntarily and are not coerced.

• Informed consent: Inform participants about the procedures and risks involved in your research and get participants consent to participate.

• Anonymity: Assure participants that no one, including yourself, will be able to link the data to a specific individual (Not always possible. Then assure at least confidentiality)

• Confidentiality: Assure participants that identifying information about them acquired through your study will not be released to anyone outside the study.

• No data fabrication or manipulation

• Extent to which changes in the dependent variables(s) can be explained by the experimental manipulation and not by external factors
  => Degree to which a causal conclusion can be drawn

• Extent to which the results of the experiment can be generalized – from sample to population
  => Degree to which findings are representative

• => no external validity without internal validity

• History (what happens during the experiment)

• Maturation (changes in the test units themselves)

• Mortality (survivorship bias)

• Instrumentation (changes to the measuring instrument)

• Experimenter effect (reaction to the experimenter due to age, sex, race, …)

• Socially desirable behavior and/or demand effects

• Selection bias (due to the improper selection of test units)

• Testing effects / reactivity (caused by the process of experimentation)

• Regression to the mean (test units move to the mean in the process of the experiment)

• True: randomization

• Quasi: no randomization

• Randomization

• Statistical control: measuring extraneous variables and adjusting for their effects through statistical methods (e.g. demographics).

• Matching: matching participants on a set of key variables before assigning them to conditions.

• Design control: using specific experimental designs to control for confounding effects (treating extraneous variables as additional IV’s)

• => between-group designs: one group gets only one treatment, study different groups

  • +: Simplicity, Lower fatigue and practice effects, Useful if it’s impossible to switch to other experimental conditions (e.g. male vs. female)

  • -: High number of participants necessary, particularly for complex designs (many IV’s with many levels), Weaker effects (manipulation needs to be strong)

• => within-subject designs: every participant gets several or all treatments, repeated testing

  • +: Economy (fewer participants necessary), Higher sensitivity (higher chance that manipulation has an effect)

  • -: Carry-over effects from one to condition to the next, Fatigue and practice effects, Remedy: Counterbalancing (random sequence)

1. Nominal: Numbers identify and classify objects (e.g., gender, social security number).

2. Ordinal: Numbers indicate relative positions of objects but not the magnitude of differences between them (e.g., preferences).
   
   Special case rating scale: actually an ordinal scale, but can in most cases be interpreted as interval scale
    

3. Interval: Intervals between data points on the scale are equal (e.g., temperature).

4. Ratio: All powers of the prior scales as well as a meaningful absolute zero (e.g., age, length).

Comparative (nonmetric)

• Paired comparison: What do you like more, Pepsi or Coke?

• Rank order: order your favourite softdrinks brands.

• Constant sum: distribute the 5 points to your favourite brands.
  

Noncomparative (metric)

• Continuous rating scale: place a mark on a continuous line

• Itemized rating scale

  • Likert: scale from 1 (strongly agree) to 5 (strongly disagree)

  • Semantic differential: Seven-point scale with bipolar labels (powerful - - - - - - - weak)

  • Stapel: Unipolar ten-point scale from -5 to +5 without 0.

Results are independent of examiner

- consistent results, if measurements are repeated

- free from random error (but not free from systematic error)

Whether what was tryed to measure was really measured.