Machine learning interviews test whether you truly understand the fundamentals, not just whether you can call a library function. Interviewers want to hear you reason about trade-offs: why a model overfits, when to trust an accuracy number, and how to diagnose a training curve. This guide walks through the machine learning interview questions that come up again and again in 2026, with concise, technically accurate answers and Python examples you can practice explaining out loud.

Core Concepts: Bias, Variance, and Overfitting

Bias is error from overly simplistic assumptions that cause a model to miss relevant relationships (underfitting). Variance is error from excessive sensitivity to small fluctuations in the training data (overfitting). A high-bias model performs poorly on both training and test data; a high-variance model fits training data well but generalizes poorly. The trade-off exists because reducing one typically increases the other. The goal is to minimize total expected error, which decomposes into bias squared, variance, and irreducible noise. Techniques like regularization increase bias slightly to reduce variance substantially, often improving generalization.

You detect overfitting when training performance is high but validation performance is significantly worse, or when the validation loss starts rising while training loss keeps falling. To prevent it, use regularization (L1/L2), gather more data, reduce model complexity, apply dropout in neural networks, use early stopping, and validate with cross-validation. L2 regularization (ridge) shrinks weights toward zero to reduce variance, while L1 (lasso) drives some weights exactly to zero, performing implicit feature selection.

from sklearn.linear_model import Ridge, Lasso
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split

X, y = make_regression(n_samples=500, n_features=20, noise=15.0, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# alpha controls regularization strength: higher alpha = more bias, less variance
ridge = Ridge(alpha=1.0).fit(X_train, y_train)
lasso = Lasso(alpha=0.1).fit(X_train, y_train)

print("Ridge test R^2:", ridge.score(X_test, y_test))
print("Lasso test R^2:", lasso.score(X_test, y_test))
print("Lasso zeroed features:", (lasso.coef_ == 0).sum())

Evaluation Metrics: Precision, Recall, and F1

Precision is the fraction of predicted positives that are actually positive: TP / (TP + FP). Recall (sensitivity) is the fraction of actual positives that were correctly identified: TP / (TP + FN). The F1 score is the harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall). Prioritize precision when false positives are costly, such as flagging legitimate transactions as fraud. Prioritize recall when false negatives are costly, such as missing a disease diagnosis. F1 balances both and is useful on imbalanced datasets where accuracy is misleading.

With imbalanced classes, a model can achieve high accuracy by simply predicting the majority class. If 99 percent of samples are negative, a model that always predicts negative is 99 percent accurate but useless for finding positives. In these cases, use precision, recall, F1, the precision-recall curve, or ROC-AUC, and consider techniques like class weighting, resampling, or threshold tuning to address the imbalance.

from sklearn.metrics import precision_score, recall_score, f1_score, classification_report

y_true = [0, 1, 1, 0, 1, 1, 0, 1]
y_pred = [0, 1, 0, 0, 1, 1, 1, 1]

print("Precision:", precision_score(y_true, y_pred))
print("Recall:   ", recall_score(y_true, y_pred))
print("F1:       ", f1_score(y_true, y_pred))
print(classification_report(y_true, y_pred, digits=3))

Validation and Optimization

K-fold cross-validation splits the data into k equal folds, trains on k-1 folds, and validates on the remaining fold, repeating this k times so every sample is used for validation exactly once. The performance estimate is the average across folds. It gives a more reliable estimate of generalization than a single train/test split because it reduces the variance of the estimate and uses all data for both training and validation. For classification with imbalanced classes, use stratified k-fold to preserve class proportions in each fold.

Gradient descent minimizes a loss function by iteratively moving parameters in the direction of the negative gradient, scaled by a learning rate. Batch gradient descent computes the gradient over the entire dataset per step, which is stable but slow. Stochastic gradient descent (SGD) updates on one sample at a time, which is noisy but fast and can escape shallow local minima. Mini-batch gradient descent uses small batches, balancing stability and speed, and is the standard in deep learning. Adaptive optimizers like Adam adjust the learning rate per parameter using running estimates of gradient moments. A learning rate that is too high causes divergence; too low causes slow convergence.

import numpy as np

# Minimal batch gradient descent for linear regression
X = np.c_[np.ones(100), np.random.randn(100, 1)]
true_w = np.array([2.0, -3.0])
y = X @ true_w + 0.1 * np.random.randn(100)

w = np.zeros(2)
lr = 0.1
for _ in range(1000):
    grad = (2 / len(X)) * X.T @ (X @ w - y)  # gradient of MSE
    w -= lr * grad

print("Learned weights:", np.round(w, 3))

Feature Engineering and Algorithms

Feature engineering is the process of transforming raw data into features that better represent the underlying problem to the model. It includes scaling and normalization, encoding categorical variables (one-hot, target, or ordinal encoding), creating interaction terms, extracting date components, binning continuous variables, and handling missing values. Good features often improve performance more than swapping algorithms, because they encode domain knowledge the model cannot infer on its own. Always fit transformations like scalers on the training set only and apply them to the test set to avoid data leakage.

Choose logistic regression when you need an interpretable, fast, linear baseline and the decision boundary is roughly linear. Choose a random forest when relationships are nonlinear, features interact in complex ways, and you can trade some interpretability for accuracy. Random forests handle mixed feature types, are robust to outliers, and require less feature scaling, but they are larger, slower to predict, and harder to explain. Gradient boosting methods like XGBoost or LightGBM often outperform random forests on tabular data at the cost of more careful tuning.

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
clf = RandomForestClassifier(n_estimators=200, max_depth=6, random_state=42)

# stratified 5-fold cross-validation scored on F1
scores = cross_val_score(clf, X, y, cv=5, scoring="f1")
print("Mean F1:", round(scores.mean(), 4), "+/-", round(scores.std(), 4))

clf.fit(X, y)
top = sorted(zip(clf.feature_importances_, range(X.shape[1])), reverse=True)[:5]
print("Top feature indices:", [i for _, i in top])

Frequently Asked Questions

You need a working understanding of linear algebra (vectors, matrices, dot products), calculus (gradients and the chain rule), and probability and statistics (distributions, expectation, Bayes rule). You rarely need to derive proofs from scratch, but you should be able to explain why gradient descent moves in the negative gradient direction or how regularization affects the loss function. Focus on intuition backed by enough formalism to answer follow-up questions.

Memorizing full implementations is less valuable than understanding how each algorithm works, its assumptions, complexity, and trade-offs. For a coding round you may implement something simple like k-nearest neighbors or gradient descent, so practice those. For most conceptual questions, being able to explain when and why you would use an algorithm matters far more than reciting code.

Structure your answers: define the concept, explain the trade-offs, and connect it to a concrete scenario or project. Talk through your reasoning rather than jumping to an answer, and be honest about uncertainty. Demonstrating that you can diagnose problems, such as reading a learning curve or choosing the right metric for an imbalanced dataset, signals real experience. Practicing out loud with a mock interview tool helps you deliver these answers clearly and confidently.