DOE_MSc
DOE_MSc
DOE_MSc
Set of flashcards Details
| Flashcards | 60 |
|---|---|
| Language | English |
| Category | Computer Science |
| Level | University |
| Created / Updated | 28.10.2022 / 28.10.2022 |
| Weblink |
https://card2brain.ch/box/20221028_doemsc
|
| Embed |
<iframe src="https://card2brain.ch/box/20221028_doemsc/embed" width="780" height="150" scrolling="no" frameborder="0"></iframe>
|
7.1.3 Integration of abduction in the fourth paradigm
The theory is formed directly from the empirical data. Basically, it involves forming a conclusion from the information that is known.
(Practical Example: A half-eaten sandwich on the kitchen counter. Abduction will lead you to the best explanation. Your reasoning might be that your teenage son made the sandwich and then saw that he was late for school. In a rush, he put the sandwich on the counter and left.)
7.2 Quantitative empirical research
Quantitative empirical research
Research question:
How does the poliovirus vaccine work in children?
Operationalization
In a natural setting, children are vaccinated with different strains of the virus and analyzed. The children are allocated to the vaccination group (Treatment) and the comparison group (Control) by randomization. Ideally, each child is randomly assigned to either the vaccination group or the comparison group with a 50% probability.
Data ("small data")
- Population: approx. 1 million children from 6 to 9 years old
- Random sample: 750,000 children aged 6 to 9
Field phase
Some of the children were vaccinated (Treatment), some of the children were not (Control). The vaccination takes place under controlled standardized conditions.
Evaluation: Multilevel Analysis
Conclusion
Vaccinated children have a smaller rate of Polio cases (25) as unvaccinated children (71). The difference between Treatment- and control group is significant. The developed vaccine is successful.
7.2 Data-driven research
Data-driven research
Research question
Trend determination model
Setting
- Big data analysis
- Data-driven techniques
Operationalization
- Determination of an algorithm
- Data ("big data")
- Data basis: evaluation of Credit card details
- Random selection of credit card users: 10 million credit card transactions
Field phase
- Development of credit card purchases: analyzed over time and by region.
- Analyze shopping cart
Data Analytics
- Deep Learning / machine learning methods
Conclusion
The model shows how credit card purchases develop over time and that certain products and Services in certain regions have been bought excessively often. Certain trends can be derived.
8 A/B Testing
A/B testing (also bucket testing or split-run testing) is a method to compare different variants. The research questions are applied to two (A/B) or more randomized groups. The statistical analysis is done by t-test, ANOVA and more advanced methods.
8.1 Bandit algorithm
Besides answering the classic research question in A/B testing «Do variants A and B differ significantly? » a test can also be used to search dynamically for optimal solutions. Several variants are offered at the same time, the variant with the largest “success” gets the most traffic. The other variants are further developed with smaller data traffic. At one point, the most successful variant gets almost all traffic is thus the winner of the experiment.
<- In the beginning all variants (A/B/C) are starting with the same traffic.
8.1.1 Example classical A/B Testing
A/B Testing
Description
We want to check how different versions of our newsletter (which is send by e-mail) is changing the open rate. We have two versions of the newsletter.
We assign the newsletter recipients randomly into two groups. Each group receives a different version of the newsletter.
In the end we check which newsletter was opened more, thus this version wins and will be further used.
Advantages
- Planning and execution effort is low/simple.
- Easy to analyze the results of the test
Disadvantages
Depending on the prerequisites (number of variants, evaluation procedures, etc.) many observations are needed according to power analysis -> potential profit gets lost
8.1.1 A/B Testing with Bandit algorithm
Dynamic-algorithmic procedure
Description
We want to check how different versions of our newsletter (which is send by e-mail) is changing the open rate. We have two versions of the newsletter.
The experiment takes place over 4 days. We send out the newsletter every 6 hours. For the first batch, an equal count of recipients is assigned.
Depending on the success (the open rate), we change the allocation of recipients to the two different versions. The version which has the higher open rate receives more recipients. After 4 days we check which version has received more recipients. The version with the higher amount of recipients wins (= higher open rate)
Advantages
- Realtime data shows (potentially) quickly an outcome of the test
- The test can be adjusted
Disadvantages
The dynamic process produces ‘only’ one best result.
- No further models can be estimated
- No dependence of the conversion rate on other factors can be investigated
8.2 Error sources and pitfalls
Research design
A/B Testing cannot be used in all research questions, Ex. For a complete redesign of a website! (Quality aspects, survey, UX testing)
Sampling
Sampling bias / population is unknown, Example: Conducting an experiment on a webpage layout that includes only heavy users.
Sample size
Large samples Hypothesis test is significant, but information on effect sizes is missing.
Metrics
The measurement uses an unsuitable metric.
8.3 Manipulation / Dark patterns
Dark patterns are patterns of persuasion and influence, examples:
- Misdirection: The design purposefully focuses your attention on one thing to distract your attention from something else.
- Price Comparison Prevention: The retailer makes it hard for you to compare the price of an item with another item, so you cannot make an informed decision.
- Privacy Zuckering: You are tricked into publicly sharing more information about yourself than you really intended to. Named after Facebook CEO Mark Zuckerberg
9 Factorial Designs
How to arrange Factors and to get information out of them. One or more independent variables (= factors) with several levels are investigated simultaneously.
Factors = independent variables
The research team has several factors with several levels in which they have an interest, and levels of all factors are varied simultaneously. As a result, all possible combination of factor levels can be studied. The factor levels are determined before the experiment.
-> Interaction can be tested!
9.1 Full factorial designs - Single factorial
---
9.1 Full factorial designs - Multifactorial
---
9.2 Full factorial design
---
9.2 Fractional factorial design
---
9.3 Latin square
The more factors and the more characteristics, the larger the number of groups. Provided that there are NO interactions, the experiment can be conducted with a reduced number of groups By using Latin squares or related design, the number of groups required can be reduced compared to a full factorial design
= Relationship between a complete design and a Latin square (standard form)
Level ^ factors -1
3^4 = 84 -> 3^3 = 27
a. Each level of C is combined only once with each level of A and each level of B
-> Latin square is completely balanced in terms of the main effects
b. Each level of C occurs partially with other combinations of A and B
-> Latin square is only partially balance with respect to first-order interaction
10 Large data quantities
Volume (Datenmenge)
Describes the extreme amount of data. Increasing amounts of data must be stored and processed.
Important aspects for DoE
- Relation of the number of variables vs. sample size
- Relation of sample / administrative dataset vs. population
Velocity
(Geschwindigkeit)
Describes the increased speed with which the data must be produced, but also processed at the same time
Variety
(Vielfalt)
Describes the variety of different file structures: structured, semi-structured and unstructured.
Veracity
(Wahrhaftigkeit)
The uncertainty (veracity) of the data and data quality. In some cases, data from various sources does not arrive in the desired quality and can therefore not be used as intended or must be preprocessed at great expense
10.1 Made and found data
---
10.1 Made and found data (compared)
Made data
Are taken as part of a study, answers research question, data is generated by researcher
Found data
Collected for various reasons, serves documentary and administrative purposes, data is obtained administratively and technically
Grey area
Lies between: data from full surveys (=census), social media, representative surveys
10.2 Bias
Bias is the deviation (= Abweichung) between the population mean µ0 and the sample mean x. There are three elements determining the bias:
10.2.1Statistical paradise and paradox
Calculating the deviation between the sample mean and the mean in the population produces:
- statistical paradises: Have large data quantities → Big data (absolute)
- paradoxes in big data: It is not about the absolute size, but the relation!
- Often not the paradise we expected!
If the size n of the administrative dataset goes towards N, the bias becomes arbitrarily small. The absolute size n is meaningless without specifying N! Example: n = 2’000’000 “is big”, but the proportion is still small with f = n/N = 2’000’000 / 200’000’000 = 1%
