iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

Scaling & Encoding

By · Reviewed & last updated · Part of Machine Learning Tutorial

Many models (linear regression, SVM, KNN, neural nets) are sensitive to feature scale. Distance- and gradient-based methods assume features are roughly the same magnitude.

Three scalers + when to use which

EXAMPLE
from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
from sklearn.pipeline       import Pipeline
from sklearn.linear_model   import LogisticRegression
from sklearn.model_selection import train_test_split

# 1) StandardScaler — subtract mean, divide by std. Default pick.
#    Good for normally-distributed features, gradient-based models.
#    Sensitive to outliers (they distort mean + std).
scaler = StandardScaler()
Xtr_s = scaler.fit_transform(Xtr)
Xte_s = scaler.transform(Xte)            # use train stats on test

# 2) MinMaxScaler — squash to [0, 1].
#    Good for bounded features, image pixels, anything that must stay positive.
#    Also sensitive to outliers — one extreme value compresses everything else.

# 3) RobustScaler — uses median + IQR. Resistant to outliers.
#    Good when your data has heavy tails.

# ALWAYS fit on the training set, then transform train + test with the same fit.
# Wrap in a Pipeline so cross-validation does it correctly
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf',    LogisticRegression(max_iter=500)),
])
pipe.fit(Xtr, ytr)
pipe.score(Xte, yte)

# What DOESN'T need scaling:
#   - Tree-based models (Random Forest, GBDT, XGBoost) — splits don't care about magnitude
#   - Naive Bayes — probabilities don't care
#   - Anything with one feature

Why it matters

Fitting the scaler on the full dataset (then splitting) is data leakage. Always: split → fit on train → transform both. Pipelines + cross-validation enforce this for you.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
pre = ColumnTransformer([
    ('num', StandardScaler(), ['age', 'income']),
    ('cat', OneHotEncoder(handle_unknown='ignore'), ['country']),
])
Xtr_t = pre.fit_transform(Xtr)
Try it Yourself »

Exercise

Standardise numeric features.

from sklearn.preprocessing import

Discussion

Loading…