Supervised, Unsupervised, and Beyond

Where the feedback comes from

The classical split is supervised vs unsupervised. Supervised learning consumes pairs of features and labels; the loss measures label fit. Unsupervised learning has no labels and looks for structure (clusters, components, neighborhoods). The newer flavors change where the feedback comes from but the engineering question stays the same: what signal tells the system it improved?

The newer cousins

Semi-supervised mixes a small labeled set with a large unlabeled set.
Self-supervised generates its own labels from parts of the input (next-token prediction, masked patches).
Reinforcement learning takes feedback from environment reward, often delayed.
Active learning picks which examples to label next to maximize information gain.

The honest unsupervised question

If you say unsupervised but a human reviews every cluster after every run, that human is part of the system. Cluster usefulness is judged by whether the labels they produce help downstream tasks, not by silhouette score alone.

Code

Audit where the feedback signal lives·python

feedback_source = {
    "supervised": "explicit human labels in the dataset",
    "semi_supervised": "small labeled set + structure of unlabeled set",
    "self_supervised": "held-out part of the input itself",
    "unsupervised": "distance / density / structure assumptions",
    "reinforcement": "reward signal from environment",
    "active": "queries to a human oracle, ranked by uncertainty",
}

A simple semi-supervised setup with sklearn·python

from sklearn.semi_supervised import SelfTrainingClassifier
from sklearn.linear_model import LogisticRegression

base = LogisticRegression(max_iter=500)
ssl = SelfTrainingClassifier(base, threshold=0.85)
ssl.fit(X_partial, y_partial)  # y has -1 for unlabeled rows

Where the feedback comes from

The newer cousins

The honest unsupervised question

Code

External links

Exercise

Progress

Comments 0