Download Machine Learning

Meta-Learning
General Concepts
Current Status of Machine Learning
An abundance of learning algorithms
But no guidelines to select a learning
algorithm according to the characteristics
of the data
Decision trees
Algorithm
Neural Networks
Support Vector Machines
Data
Unsolved problem: What Works Well Where?
Decision trees
Support Vector Machines
Dataset
Neural Networks
An Example
• Figure shows class-conditional
probability distribution for both
classes over the plane.
• Classes are distributed in an
XOR manner.
• Training set of size two thousand.
• Testing set size of five thousand.
Misclassification Patterns
Decision Tree
Naïve Bayes
Misclassification Patterns
Perceptron
Neural Network
Misclassification Patterns
Nearest Neighbor
Another Example from Research
Try new sigmoid
function σ
Neural Network B
Neural Network A
Accuracy :
83% (2.3%)
Datasets
Accuracy :
90% (1.8%)
Have we proved anything at all?
Datasets
Fundamental Questions
Typical experiments fail to provide answer to two questions:
Algorithm Design
Why does the proposed
change in the algorithm
design produced better
performance on that set of tasks?
Data Characterization
What characteristics of a sample drawn
from a fixed but unknown
distribution are more decisive in
affecting learning behavior?
An Example of How to Understand Learning
Performance: The Case of Naïve Bayes
P(y|x)
Misclassified
Projection
X1
Class A
Class B
X2
Class Decomposition
A
A
A A
A
A
A
A
A A
A
A
{ (x,y’) | y’ = (A,1) }
B
B
B B
B
B
{ (x,y’) | y’ = (A,2) }
{ (x,y’) | y’ = (B,1) }
Results
Results
Current Forms of Data Characterization
P(y|x)
Class A
Class B
X1
• No. of classes
• No. of features
• Class entropy
X2
Easy Problem
Difficult Problem
New Forms of Data Characterization
Levels of Learning
Base Level:
We want to produce a model from one single task.
Experience does not accumulate across tasks
Hypothesis improves with number of examples
Hypothesis does not improve across tasks
Meta Level:
Systems are more efficient through experience
Experience or meta-knowledge accumulates across tasks
Learning takes place across tasks
Concepts learned: what learning mechanism is best for
this task?
Structured Tasks
Definition of Meta-learning
Meta-learning is the study of principled methods that
exploit meta-knowledge to obtain different models
and solutions by adapting machine learning and data
mining processes.
General View of AI:
The Need for Meta-Intelligence
Meta-Learning (Learning-to-Learn)
Model
Family of Models
Meta-Planning Meta-Reasoning Meta-Representation