IAI Chapter 8 Learning

Questions about the lecture 'Introduction to Artificial Intelligence' of the RWTH Aachen Chapter 8 Learning

0.0 (0)

R. B.

Étudier

R. B.

À propos de ce fichier

This flashcard set delves into advanced neural network concepts, suitable for university-level computer science students. It covers key topics like feed-forward and recurrent neural nets, their topologies, activation functions, and learning algorithms such as back-propagation. The flashcards explore practical applications like NETtalk and zip code recognition, along with theoretical aspects like the Hopefield theorem and Boltzmann machines. This set is ideal for those studying artificial intelligence, offering insights into neural net functions, characteristics, and their comparison with human brain processes.

Cartes-fiches

Utilisateurs

Langue

Anglais

Catégorie

Informatique

Niveau

Université

Créé / Mis à jour

04.02.2017 / 13.02.2017

Fichiers de cartes similaires

Name two heuristics for topology choice for neural nets? [2]

1. Max reduce and 2. min increase

What is the definition of max reduce in neural nets? [2]

1. Start with maximal number of connections

2. Reduce iteratively with information theory

What is the definition of min increase in neural nets? [2]

1. Start with minimum number of connections

2. Increase iteratively with information theory

What is the definition of a feed-forward (FF) neural net?

Directed acyclic graph

What are perceptrons for FF neural nets?

A special case of FF without hidden units

Which functions can be represented with FF neural nets? [3]

1. With zero hidden layers every linearly separable function is possible

2. With one hidden layer every continuous function is possible

3. With two hidden layers every function is possible

What is the form of g for FF neural nets?

For fixed topology and g functions have parametrized forms

What is learning for FF neural nets?

Learning becomes search for correct parameters // Nonlinear regression

Name two recurrent neural nets models? [2]

1. Hopefield nets and 2. Boltzmann machines

What is the definition of the Hopefield nets in neural nets? [3]

1. Symmetric bi-directional links Wij=Wji

2. All units are both in- and output

3. Associative memory with matching input

What is the theorem regarding Hopefield nets in neural nets?

A Hopefield net reliably stores 0.138*N examples

What is the definition of the Boltzmann machine in neural nets? [4]

1. Symmetric weights

2. Units which are neither in- nor output exist

3. Stochastic activation functions

4. State transition are like simulated annealing search

What is the difference between FF and recurrent neural nets? [3]

1. FF are best understood

2. Recurrent nets are unstable, oscillate or chaotic

3. Brain is massively recurrent

What are the basic learning steps for neural nets? [4]

1. Initially randomly assign weights to links

2. For each example compare NN with correct output and update weights // Epoch

3. Error = O_NN – O_C

4. Update rule is W = W + a*I*Error

What is the theorem by Rosenblatt (1960) for learning of neural nets?

For possible functions update rule converges to correct output

What are the learning steps for back-propagation for multi-layer FF NN?

First update weights at output layer then subsequent ones

Name two applications solved with neural nets? [2]

1. NETtalk (1987) and 2. recognizing Zip Codes (1989)

What is the application of NETtalk?

Written English text into speech

What was the effect of NETtalk back in time?

Paved the way for commercial success of neural nets

What is the application of Recognizing Zip Codes?

Recognize handwritten zip codes

What is the goal of learning?

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

Optimize behavior due to history of percepts, actions and world knowledge

What is the theoretical approach of learning?

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

Approximate ideal function f as good as possible with hypotheses h

What are the different kinds of feedback during learning? [3]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

1. Supervised with correct output to compare with

2. Reinforced with rewards and punishment instead of correct answer

3. Unsupervised with no feedback // Predict next inputs

What are the characteristics of inductive learning? [3]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

1. Input and correct output pair is given (x,f(x))

2. Find hypothesis function h which approximates f

3. Use additional bias which restrict the search space

What are the characteristics of decision tree (DT) for learning? [4]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

1. Every PL formula (So-called boolean function) is representable by a DT

2. Trivial DT have one path per example // Memorizing but too big

3. Compact DT covering all examples // Idea by Occams Razor

4. For n attributes exist 2²^n possible functions

How is the decision tree (DT) algorithm for learning defined? [6]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

Input are examples attributes and default value of goal predicates

Output is a DT

1. If examples are empty return default

2. Else if all examples have same classification return classification

3. Else if attributes are empty return MAJORITY-VALUE(examples)

4. Else choose best attribute and initialize DT with best as root

5. Forall value v_i of best find corresponding examples and build subtree

6. Return tree

What are the three cases which might appear in the decision tree (DT) algorithm for learning? [3]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

1. If all examples are positive or negative then answer is Yes or No

2. If no more examples exist consider parent node examples for answer

3. If no more attributes exist some examples are identical

What is the explanation of case three of the decision tree (DT) algorithm for learning? [3]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

1. An error occurred

2. Attributes are insufficient to distinguish

3. Consider other examples for answer

Name two applications of decision tree (DT) for learning? [2]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

1. GASOIL expert system divides crude oil from gas with 2500 rules

2. Flight simulator of a Cessna

What is the definition of evaluating learning? [4]

Commandes clavier:

= tourner,

= avant/arrière,

= faire défiler

1. Collect large set of examples E

2. Separate E into disjoint training Tr and test set Te

3. Use Tr to generate a hypothesis H

4. Test H with Te

What is the problem of eliminating from hypotheses space?

Usually too big

What is the definition of considering the best hypothesis in learning? [3]

1. If inconsistent due to new example modify the extension

2. After modification test all old examples

3. Dead end possible → uncontrolled backtracking

How do we modify the extension in the considering the best hypothesis algorithm in learning? [2]

1. Enlarge for FN (Generalization)

2. Shrink for FP (Specialization)

What are the characteristics of probably approximately correct (PAC) learning? [2]

1. Criteria for approximation of hypothesis h

2. How many examples are necessary for h is within eps of f with probability 1-delta for small delta and eps!=0

What are the characteristics of decision list (DL) for learning? [3]

1. Consists of several tests for a conjunction of bounded many literals

2. If test is successful return value

3. Else try next test in list

What is the problem of decision list (DL) for learning?

DL represents only restricted class of logical formulas

How is the decision list (DL) algorithm for learning defined? [5]

Input are examples

Output is a DL, No or failure

1. If examples are empty return No

2. Test nonempty subset examples_i thus only positive or negative with t

3. If no such test t exist return failure

4. If examples_i are assigned positive set o to Yes else to No

5. Return DL with t, outcome o and DL’ of examples-examples_i

What are the characteristics of information theory by Shannon and Weaver (1949)? [2]

1. Information content of an answer to a Yes/No query

2. I(P(v_1),…,P(v_n)) = Sum_i -(P(v_i)*log2P(v_i)) in bits

What is the equivalent of processors in neural nets of the human brain?

Neurons

What is the equivalent of connections in neural nets of the human brain?

Synapses

1 / 58

Cartes-fiches

What is the definition of min increase in neural nets? [2]

1. Start with minimum number of connections

2. Increase iteratively with information theory

What is the definition of a feed-forward (FF) neural net?

Directed acyclic graph

What are perceptrons for FF neural nets?

A special case of FF without hidden units

Which functions can be represented with FF neural nets? [3]

1. With zero hidden layers every linearly separable function is possible

2. With one hidden layer every continuous function is possible

3. With two hidden layers every function is possible

What is the form of g for FF neural nets?

For fixed topology and g functions have parametrized forms

What is learning for FF neural nets?

Learning becomes search for correct parameters // Nonlinear regression

Name two recurrent neural nets models? [2]

1. Hopefield nets and 2. Boltzmann machines

What is the definition of the Hopefield nets in neural nets? [3]

1. Symmetric bi-directional links Wij=Wji

2. All units are both in- and output

3. Associative memory with matching input

What is the theorem regarding Hopefield nets in neural nets?

A Hopefield net reliably stores 0.138*N examples

What is the definition of the Boltzmann machine in neural nets? [4]

1. Symmetric weights

2. Units which are neither in- nor output exist

3. Stochastic activation functions

4. State transition are like simulated annealing search

What is the difference between FF and recurrent neural nets? [3]

1. FF are best understood

2. Recurrent nets are unstable, oscillate or chaotic

3. Brain is massively recurrent

What are the basic learning steps for neural nets? [4]

1. Initially randomly assign weights to links

2. For each example compare NN with correct output and update weights // Epoch

3. Error = O_NN – O_C

4. Update rule is W = W + a*I*Error

What is the theorem by Rosenblatt (1960) for learning of neural nets?

For possible functions update rule converges to correct output

What are the learning steps for back-propagation for multi-layer FF NN?

First update weights at output layer then subsequent ones

Name two applications solved with neural nets? [2]

1. NETtalk (1987) and 2. recognizing Zip Codes (1989)

What is the application of NETtalk?

Written English text into speech

What was the effect of NETtalk back in time?

Paved the way for commercial success of neural nets

What is the application of Recognizing Zip Codes?

Recognize handwritten zip codes

Étudier

IAI Chapter 8 Learning

Créer ou copier des fichiers d'apprentissage

Cartes-fiches

Veuillez vous connecter

Google

Apple ID

Microsoft

Switch edu-ID

Edulog

Google

Apple ID

Microsoft

Switch edu-ID

Edulog