DM Exam
s
s
-
- 1 / 27
-
Lernkarten
Data-driven discovery (Data Mining Definition)
Models and patterns (defintion Data Mining)
Massive data sets (Data Mining Definition)
observation data sets (Data Mining Defition)
What's been done in data preprocessing?
Transform data into appropriate form
- Harmonize (data from multiple sources)
- Clean noise and duplicates
- Reduce records and features
What's been done in data postprocessing?
Drawing conclusions and utilizing the results.
Flow of Knowledge Discovery
Input Data -> Preprocessing -> Data Mining -> Postprocessing
Examples for simplicity (understandability)
The Problem of Interestingness
association rule length, decision tree size
Examples for validity (certainty)
The Problem of Interestingness
confidence, classification reliability, accuracy, certainty factor, rule strength, rule quality
Utility
The Problem of Interestingness
potential of usefulness, support association, noice threshold
Novelty
The Problem of Interestingness
not previously known knowledge, suprising knowdledge
Types of attributes
Categorical -> nominal, ordinal
Numeric -> interval, reatio
Data Quality:
Noise
Random error in data
Data Quality:
Artifacts
Error due to deterministic phenomenon or distortions
Data Quality:
Precision
Closeness of repeated measurements to one another
Data Quality:
Bias
A systematic variation of measurements from the quality being measured
Data Quality:
Accuracy
closeness of measurements to the true value of the quantity being measured
Data Quality:
Outliers
different characteristics from most other data objects or unusual with respect to typical values
Data Quality:
Missing Values
information that is not collected
Data Quality:
Inconsistent Values
closeness of repeated measurements to one another
Data Quality:
Duplicate Data
identical records that represent a single object.
Data Quality:
Timeliness
Do the data still represent the situation / behavior?
Data Quality:
Relevance
Do the data contain the relevant information ->
i.e. correct
sample, contains (all) possibly required information.
Characteristics of Data Sets
Dimensionality
number of attributes
curse of dimensionality
dimensionality reduction
Characteristics of Data Sets
Sparsity
amount of non-empty or non-zero values
Characteristics of Data Sets
Resolution
the smallest interval measurable
Boxplot Analysis
- box representation
- ends of the box are at the first and third quartiles, i.e., the height of the box is IRQ
- median is marked by a line within the box
- Whiskers: two lines outside the box extend to Minimum and Maximum
