Feature Scaling and Encoding

When scaling matters

Linear regularization, k-nearest neighbors, k-means, and neural-network optimization react to numeric magnitude. A salary measured in dollars can dominate a visit count simply because its numbers are larger. Tree-based models usually do not need scaling because their splits depend on ordering rather than distance.

Pick a scaler from the distribution

Use StandardScaler for reasonably symmetric values, RobustScaler when outliers would distort the mean and standard deviation, and MinMaxScaler when bounded inputs are required. Long-tailed amounts often benefit from a log transform before scaling. Remember that future values can exceed the training min and max.

Encoding categoricals

One-hot for low-cardinality unordered categories, compatible with most models.
Ordinal only when order is real; arbitrary integer codes invent a false distance.
Target encoding for high cardinality, always estimated inside training folds to prevent leakage.
Hashing when a stored vocabulary is impractical and collisions are acceptable.

Plan for unseen categories

Production will contain a city, product, or code absent from training. Decide whether the encoder errors, maps it to an explicit unknown bucket, or hashes it. Monitor the unknown rate because silently collapsing every new category can hide meaningful drift.

Pipeline placement

Scalers and encoders belong inside a ColumnTransformer and model pipeline. Fit them on training only and apply the same fitted objects to validation and production. Select columns by stable names rather than position, then inspect the transformed names and output shape.

Test the serialized path

Load the saved pipeline in a fresh process and score raw rows with reordered columns, missing optional values, and unseen categories. If a notebook cell or hand-maintained feature order is still required, training and serving do not yet share one contract.

Code

Mixed numeric + categorical preprocessing·python

from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline

numeric_pipe = Pipeline([
    ("impute", SimpleImputer(strategy="median")),
    ("scale", StandardScaler()),
])
categorical_pipe = Pipeline([
    ("impute", SimpleImputer(strategy="most_frequent")),
    ("onehot", OneHotEncoder(handle_unknown="ignore")),
])
preprocess = ColumnTransformer([
    ("num", numeric_pipe, numeric_cols),
    ("cat", categorical_pipe, categorical_cols),
])

Target encoding done safely with category_encoders·python

from category_encoders import TargetEncoder

te = TargetEncoder(cols=["merchant_id", "city"])  # fit per CV fold
X_tr_enc = te.fit_transform(X_tr, y_tr)
X_val_enc = te.transform(X_val)

When scaling matters

Pick a scaler from the distribution

Encoding categoricals

Plan for unseen categories

Pipeline placement

Test the serialized path

Code

External links

Exercise

Progress

Comments 0