IAI Chapter 8 Learning
Questions about the lecture 'Introduction to Artificial Intelligence' of the RWTH Aachen Chapter 8 Learning
Questions about the lecture 'Introduction to Artificial Intelligence' of the RWTH Aachen Chapter 8 Learning
-
- 1 / 58
-
Flashcards
What is the goal of learning?
Optimize behavior due to history of percepts, actions and world knowledge
What is the theoretical approach of learning?
Approximate ideal function f as good as possible with hypotheses h
What are the different kinds of feedback during learning? [3]
1. Supervised with correct output to compare with
2. Reinforced with rewards and punishment instead of correct answer
3. Unsupervised with no feedback // Predict next inputs
What are the characteristics of inductive learning? [3]
1. Input and correct output pair is given (x,f(x))
2. Find hypothesis function h which approximates f
3. Use additional bias which restrict the search space
What are the characteristics of decision tree (DT) for learning? [4]
1. Every PL formula (So-called boolean function) is representable by a DT
2. Trivial DT have one path per example // Memorizing but too big
3. Compact DT covering all examples // Idea by Occams Razor
4. For n attributes exist 2²^n possible functions
How is the decision tree (DT) algorithm for learning defined? [6]
Input are examples attributes and default value of goal predicates
Output is a DT
1. If examples are empty return default
2. Else if all examples have same classification return classification
3. Else if attributes are empty return MAJORITY-VALUE(examples)
4. Else choose best attribute and initialize DT with best as root
5. Forall value v_i of best find corresponding examples and build subtree
6. Return tree
What are the three cases which might appear in the decision tree (DT) algorithm for learning? [3]
1. If all examples are positive or negative then answer is Yes or No
2. If no more examples exist consider parent node examples for answer
3. If no more attributes exist some examples are identical
What is the explanation of case three of the decision tree (DT) algorithm for learning? [3]
1. An error occurred
2. Attributes are insufficient to distinguish
3. Consider other examples for answer
Name two applications of decision tree (DT) for learning? [2]
1. GASOIL expert system divides crude oil from gas with 2500 rules
2. Flight simulator of a Cessna
What is the definition of evaluating learning? [4]
1. Collect large set of examples E
2. Separate E into disjoint training Tr and test set Te
3. Use Tr to generate a hypothesis H
4. Test H with Te
What is the problem of eliminating from hypotheses space?
Usually too big
What is the definition of considering the best hypothesis in learning? [3]
1. If inconsistent due to new example modify the extension
2. After modification test all old examples
3. Dead end possible → uncontrolled backtracking
How do we modify the extension in the considering the best hypothesis algorithm in learning? [2]
1. Enlarge for FN (Generalization)
2. Shrink for FP (Specialization)
What are the characteristics of probably approximately correct (PAC) learning? [2]
1. Criteria for approximation of hypothesis h
2. How many examples are necessary for h is within eps of f with probability 1-delta for small delta and eps!=0
What are the characteristics of decision list (DL) for learning? [3]
1. Consists of several tests for a conjunction of bounded many literals
2. If test is successful return value
3. Else try next test in list
What is the problem of decision list (DL) for learning?
DL represents only restricted class of logical formulas
How is the decision list (DL) algorithm for learning defined? [5]
Input are examples
Output is a DL, No or failure
1. If examples are empty return No
2. Test nonempty subset examples_i thus only positive or negative with t
3. If no such test t exist return failure
4. If examples_i are assigned positive set o to Yes else to No
5. Return DL with t, outcome o and DL’ of examples-examples_i
What are the characteristics of information theory by Shannon and Weaver (1949)? [2]
1. Information content of an answer to a Yes/No query
2. I(P(v_1),…,P(v_n)) = Sum_i -(P(v_i)*log2P(v_i)) in bits
What is the equivalent of processors in neural nets of the human brain?
Neurons
What is the equivalent of connections in neural nets of the human brain?
Synapses
How is information processed both for neural nets and in the human brain?
Information is processed locally and in parallel
In which categories is the human brain better than a computer concerning neural nets? [4]
1. Computational units // 106 < 1011
2. Storage units // DISK:1013 < 1014
3. Bandwidth // 1010 bits/sec < 1014 bits/sec
4. Neuron updates/sec // 106 (10³ gates per update) < 1014
In which category is the computer ‘better’ than a human brain concerning neural nets? [2]
1. Storage units // RAM:1014 > 1011
2. Cycle time // 10⁻9 [serial] < 10⁻³ [parallel]
What are the facts of neural nets? [6]
1. Speed through parallelism
2. Robust if parts of network are damaged // Graceful degradation
3. Designed for inductive learning
4. Noise does not affect network
5. Bad transparency
6. Additional knowledge has no positive effect
What are units of neural nets?
Nodes (neurons) of the network
What are input/output units of neural nets?
Nodes connected to the environment
What are hidden units of neural nets?
Nodes which are neither in- nor output
What are links of neural nets?
Connections of nodes with weight (usually a real number)
What are activation levels of neural nets?
Value computed by units with weighted inputs
What are the components of the unit function of neural nets? [2]
1. Input and 2. activation function
What is the definition of the input function for neural nets?
Input function organizes the input links // Sum of weight and active units
What is the definition of the activation function g for neural nets?
Activation function g usually nonlinear returns value
What are possible forms of the activation function g for neural nets? [3]
1. Step 2. sign and 3. sigmoid
What holds for the step and sign function for neural nets?
Possible to simulate step by sign with additional input
How can we use step function for boolean function in neural nets? [3]
1. AND with t=1.5 and W=1
2. OR with t=0.5 and W=1
3. NOT with t=-0.5 and W=-1
What are the two main topologies of neural nets? [2]
1. Feed forward (FF) and 2. recurrent
What is the definition of layers in neural nets? [3]
1. No links between nodes of the same layer and to the 'next-next' layer
2. Only links to the next layer
3. For counting number of layers input layer is ignored
What are the effects of the neural nets size? [3]
1. Too small → desired function might not be possible
2. Too big → memorizing without generalization // Overfitting
3. Usually much less than 2^n / n for any boolean function
Name two heuristics for topology choice for neural nets? [2]
1. Max reduce and 2. min increase
What is the definition of max reduce in neural nets? [2]
1. Start with maximal number of connections
2. Reduce iteratively with information theory
