Lecture 2 + Lecture 2 (IDK)

Spam classification, intent classification, machine translation, image classification

A program learns from experience E with respect to tasks T and performance measure P, if performance at T (measured by P) improves with E

T = classify the dialog act of an utterance, P = fraction of utterances correctly classified, E = a set of utterances labeled with dialog acts

Supervised (labels), Unsupervised (no labels), Reinforcement (reward system)

The expected number of bits needed to encode the class of a randomly drawn instance under the optimal code

When all instances belong to one class

For [+9, -5]: –(9/14)log(9/14) – (5/14)log(5/14) = 0.940

The ability to perform well on test examples beyond the training data; memorization alone is not enough

The preference of a learning algorithm for one solution/interpretation over another, independent of the observed data

Expected number of bits to encode the class (⊕ or ⊖) of a randomly drawn member of S under optimal code

Entropy before split – entropy after split (weighted by probability of each branch)

When a model is too simple to capture patterns (e.g., decision tree always predicting majority class)

When a model memorizes training data idiosyncrasies (e.g., one leaf per training instance, zero training error but poor generalization)

Weights or structure selected by the learning algorithm

Parameters that control other parameters; tunable but not selected by the learning algorithm

TP / (TP + FP) → Of all messages labeled spam, fraction actually spam

TP / (TP + FN) → Of all actual spam messages, fraction correctly labeled as spam

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

T: Classify the dialog act of an utterance, P: The fraction of utterances correctly classified, E: A set of utterances labeled with their dialog acts.

T: Drive on public highways using vision sensors, P: Average distance traveled before an error, E: Sequence of images and steering commands from a human driver.

They take a long time to create, it's easy to make mistakes, and they are difficult to maintain.

It allows a learning algorithm to prioritize one solution or interpretation over another, independent of the observed data.

If all instances have the same label, or if there are no features left to split on.

Select the best test to split the data on, split the data according to the test, and recurse on each subset of the data.

The expected number of bits needed to encode the class of a randomly drawn member of set S.

A hill-climbing search.

Information gain.

When instances are described by attribute-value pairs, the target function is discrete-valued, and the data might be noisy.

The fraction of items identified as belonging to a class that actually do belong to that class.

The fraction of items that actually belong to a class that you have successfully identified.

Low precision is typically worse, as it means accidentally labeling an important message as spam.

The harmonic mean of precision and recall.

Micro F1 calculates metrics globally across all classes, while Macro F1 calculates metrics for each class individually and then averages them.

1) Eliminate entries/attributes with missing values. 2) Estimate missing values (e.g., fill with the mean or median).

Having a negative value for an attribute like 'Age'.

Unexpected values that can statistically distort a dataset and negatively impact the performance of a machine learning model.

Measurement noise, faulty input, or rare events.