DOE_MSc

Descriptive study

Descriptive character, suitable for forming hypotheses but insufficient for testing them.

Cross sectional study

Sample with variables of interest are collected and described at a specific time “Quality of Life of master’s students.”

-> Survey is typical

Correlational study

Same as cross sectional, but variables are correlated:

“Relationship between the time that master’s students spend studying on their own and their general quality of life”

dentification and quantification of effects of relationship, not fully suitable for hypothesis testing.

Case control study                                                     

Examines how cases and controls differ from having been exposed previously Looking back at the exposition (retrospective)

Example with Pädos (Vergangenheit wird angeschaut, wieso wurden sie so?)

Cohort study

Cohort = group of people with comparable initial conditions

One group is exposed to an influencing factor afterwards the groups are compared

Randomized controlled (RCT)

Treatment and control are determined by a random process (randomization) before the intervention.

uitable for hypothesis testing.

Non-randomized controlled

Treatment and control are not determined by a random process (randomization) before the intervention. For example, control groups involve data obtained from previous studies («historical control») or from randomly allocated groups.

Input

Trial objects, tests objects, test persons, materials

Process

Process in which controllable and non-controllable factors influence the input

Controllable factors

Influencing factors whose strength can be adjusted within defined limits.

• Independent variable (IV)

Non-controllable factors

Influencing factors whose strength cannot be determined,

• but measured:
  • body weight, stress level, etc.
• or not measured:
  • random fluctuation in output, random fluctuation in measurement instruments

• Nuisance variables (= Störvariablen)

Output

Input changed by the process, result of the test/experiment.

• Dependent variable (DV)

Impact of factors on the variation of the output (focus of the study)

Ex. Body weight changes systematically when height changes:

Variation of the output, caused by nuisance factors (not in the focus of the study, for example gender)

Ex. Boys and girls differ in the level of body weight, that’s why we can divide the sample into two groups

Variation caused by measurement errors or random processes.

The Variance of the dependent variable (primary variance) should be attributed to the systematic variation of the independent variable (IV). The secondary variance should be controlled, and the error variance minimized.

If the relationship is linear, the primary variance can be maximized by the selection of extreme values in the IV

If the relationship is curvilinear, the primary variance can be maximized by Selection of optimal increments of IV

If the relationship is unknown, the primary variance could be maximized by selection of many levels of IV and the smallest possible gradations

• Keeping the experimental setups constant
• Repetition: Several measurements are repeated on the trial objects
• Randomization: the trial objects are allocated randomly to groups
• Blocking: Trial objects are assigned to blocks that can be considered homogenous in terms of one or more influencing variables. Example:

• Blinding: to reduce the placebo effect (“golden standard” for RCT)

Nuisance variables used as covariate to IV

• Subsequent statistical control: Analysis of covariance

Reliable measurement setup and instruments

- Standardization of the examination situation

- Standardized equipment and procedures

- Staff training- Selecting suitable measurement instruments

- Calibration and verification of measurement instruments

Measurements repetition / sample size

- Measurements are repeated several times → random errors are balanced out

- Large samples → random errors are balanced out on average

Suitable analytical methods

- Robust estimators in case of heterogenous error variance

Objectivity

Is given when the results are independent of personnel and calculation method.

Ex. Regardless of who reads a weighing scale, the result is the same.

Negative Ex. Reading error due to different viewing angles.

Reliability

Is the degree to which an instrument produces the same result each time under comparable conditions.

Ex. Weighting scale that always produces the same result at the same weight.

Negative Ex. Weighting scale changes the weighted amount over time

Validity

Is the extent to which an instrument measures what was intended.

Ex. Weighing scale that leads to the measurement result of 75 kg at 75 kg

Negative Ex. Use of weight scale with an insufficient accuracy class

Experimental design:

Trial and error

One-factor-at-a-time

Full factorial, two level design

Fractional factorial design

Trial and error

Characteristics: Research team tries different factors to answer a research question. They observe if the factor has an influence on the DV (trial), and if not (error), the team applies another one. This process is repeated until the research question can be answered (or the team gives up)

Advantages: none

Disadvantages: unsystematically, effortful, unable to study interactions, incomplete

Characteristics: Only one factor at a time is studied, instead of multiple factors at once.

Advantages: easy to implement, less effort, fast

Disadvantages: inefficient, large number of experimental runs, unable to study interactions

Ex. “Which factors influence the fuel consumption of a car?”

Characteristics: Several factors are of interest, and two levels of all factors are varied simultaneously.

As a result, all possible combinations of factors can be studied.

Advantages: systematically highest quality, able to study all main effects and interactions

Disadvantages: very effortful (exponential growth, 2^k), expensive

Ex. “Which factors influence the fuel consumption of a car?” 2³ = 8 experiment runs

Fractional factorial design

Characteristics: Same as “full factorial”, but only a selected “fraction” of the full factorial runs are tested.

Advantages: good choice if resources are limited

Disadvantages: unable to study all possible interactions

Population: Set of all (potentially explorable) elements that have a common characteristic or a common combination of characteristics.

Differentiation criteria: Each observation needs to meet criteria, to see whether it is part of the population or not:

• geographical aspects → Persons at Uni of Zurich
• temporal aspects (point in time or period) → in spring semester
• factual/content-related aspects → currently enrolled

The mean x is a point estimate of the true value u_o

Simple random sampling

With R

(Given is a .csv-file)

library(readr)

address <- read.csv("address.csv",head=TRUE, sep=",")

address_sample <- address[sample(699, 50, replace=FALSE),]

View(address_sample)

*Replace=False: Each element of the sample occurs only once!

Non-sampling error

Sampling error

Variability of sample means

Non-sampling error: A non-sampling error refers to an error that occurs during data collection, causing the data to differ from the true values.

• Coverage error: Part of the population cannot be identified. Example: The phone book for landlines serves as the basis for the sampling.

Defined Population: Inhabitants of a city

Ex. Under coverage: Entries in local phone book -> people without phone number are missing

Ex. Over coverage:§ Entries of doctors in the local phone book -> not necessarily inhabitants of the city

• Systematic nonresponse: Complete (unit-nonresponse) or partial (item-nonresponse). Absence of information on individual elements. Example: People generally refuse to answer one topic of a survey.

The difference between the mean of the population and the mean of the sample.

• Selection error: Not all the elements of the population have the same selection probability. Accordingly, the analysis does not reflect this. Example: Working people are less likely to answer the phone during the day. Use of an unsuitable estimator. Example: Empirical variance is calculated with 1/n instead of 1/(n-1).

The size of the standard error of the sample mean.

• The more heterogeneously the characteristic is distributed in the population, the greater the standard error of the sample mean.
• The smaller the sample, the larger the standard error

In statistics, an effect size is a number measuring the strength of the relationship between two variables in a statistical population, or a sample-based estimate of that quantity. It indicates the importance of the study results.

Statistical significance and importance of an effect:  the chance of having a significant result in hypothesis test is….

• Larger if sample size N increases
• Smaller if standard deviation S increases

If sample n is large … Effects can become significant that in fact have no importance/strength

If sample size n is small…Effects may not become significant that in fact have importance/strength

The differences of two measurement (new vs. current) can only be meaningfully assessed when standardized with a reference value.→ For differences in mean values, this is the pooled standard deviation (= Cohen’s d) Power analysis / determining the sample size.

Example: New medicine reduces blood pressure by 8.5 units: X_new – X_current = 8.5mmHg

Control group (nc = 64) and treatment group (nT = 64): Values of blood pressure

Mean xC = 146.4 mmHg, standard deviation sC = 15.6 mmHg

Mean xT = 137.9 mmHg, standard deviation sT = 17.7 mmHg

Estimation of effect size according to Cohen

How large must the total sample be for

• significance level a = 5
• power = 1 – b = 80%
• effect size medium to obtain a significant analysis of variance? (use cohens table!)

The Fourth Paradigm, introduced by Jim Gray in his lecture in 2007, marks an extension and renewal and not a replacement of current scientific paradigms. Jim Gray describes the Fourth Paradigm as a data-intensive science, which brings the fourth paradigm-shift to the nature of science.

First Experimental Science (Thousand years ago)

Description of natural phenomena

Second Theoretical Science (Last few hundred years)

Modelling and generalization: Newton’s Laws, Maxwell’s Equations …

Third Computational Science (Last few decades)

Modelling and generalization: Newton’s Laws, Maxwell’s Equations …Simulation of complex phenomena

Fourth Data-Intensive Science eScience

(Today)

Exploration of data: Analysis of database / files, using data management and statistics, unification of theory, experiment, and simulation synthesis of information technology and science Requires combination of statistics & computer science…Change of publishing data & literature (curation, access, preservation)

The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.

Arguments that speak in favor of it

• No need for a priori theories
• Large data quantities can cover an entire domain and provide a comprehensive solution.
• Using unbiased value-free data analysis, the data speaks for itself, free from human prejudgment and without scientific premises.
• Patterns and relations within large amounts of data are given «by nature».

Arguments that speak against it

• Large data quantities are created in an environment influenced by many factors:
  • Aim and purpose
  • Technology and platform
  • Regulatory environment
• Large data quantities cannot be interpreted outside of their context.
• There is no such thing as unbiased value-free data analysis …

Deductive approach -> Conclusion from the general to the specific

Drawing conclusions from theory and applying them to empirical data. The process starts with the theory from which empirically testable hypotheses are derived.

• Rejection based on data theory must be revised Falsification
• Non-rejection based on data theory is provisionally confirmed Verification

Example: p hacking

(Practical example: Deductive reasoning always follows necessarily from general or universal premises. If a sandwich is defined as "two or more slices of bread or a split roll having a filling in between," and a hot dog is defined as "a frankfurter; especially : a frankfurter heated and served in a long split roll" then one must deduce that any hot dog served in a split roll is a sandwich.

Inductive approach -> conclusion from the specific to the general

Drawing conclusions from empirical data on scientific theory to a higher level. The process starts with given data

• From the data, patterns are gradually worked
• By means of induction, hypotheses about theories are formed and theories are formulated

Example Google flu prediction

(Practical example: You observe 4 out of 6 co-workers ordering the same sandwich. From your observation, you then induce that the sandwich is probably good. Your reasoning is based on an observation of a small group as opposed to universals premises.)