Time-Series Forecasting & Uncertainty — Interview Questions

A question bank for time-series forecasting and uncertainty-quantification interviews — the kind grid operators and utilities, and the forecasting teams at Amazon (retail demand & energy), Google, and Meta actually ask. Each question has a thorough, say-it-out-loud answer. Deep dives: Probabilistic Forecasting & Prediction Intervals · Conformal Prediction: Coverage-Guaranteed Intervals · Forecast Evaluation & Backtesting.

---

Q1. Why can't you use plain k-fold cross-validation on a time series?

Answer. Standard k-fold shuffles rows and lets any fold serve as validation. On a time series that leaks the future into the past: a model validated on Monday can have trained on Tuesday, which is impossible in production and inflates your score. Time series have autocorrelation (today depends on yesterday) and often trend/seasonality drift, so order carries information that shuffling destroys.

Use walk-forward (a.k.a. rolling / expanding-window) validation instead: always train on the past, test on the future, then roll forward.

   EXPANDING WINDOW                       SLIDING WINDOW (fixed train length)
   [==train==]|test|                      [==train==]|test|
   [====train====]|test|                     [==train==]|test|
   [======train======]|test|                    [==train==]|test|
   time ->                                 time ->

This mirrors deployment (you only ever know the past), respects autocorrelation, and gives an honest estimate of forward error. Amazon and grid-forecasting teams expect you to volunteer "walk-forward, never shuffled" the instant time series come up.

---

Q2. Recursive vs direct multi-step forecasting — what's the tradeoff?

Answer. To forecast h steps ahead you can:

• Recursive (iterated): train one one-step model, feed its prediction back in as input, and roll forward h times. Simple and data-efficient, but errors compound — step-2 uses step-1's error as if it were truth, so uncertainty snowballs with horizon.
• Direct: train a separate model per horizon (ŷ_{t+1}, ŷ_{t+2}, … each with its own model). No error feedback, often better at long horizons, but you train h models and lose the coupling between horizons (forecasts can be jagged).

A common middle ground is multi-output / seq2seq models (one model predicts the whole horizon at once), and DirRec hybrids. Say when you'd pick each: recursive for short horizons or scarce data; direct/multi-output for long horizons where compounding hurts.

---

Q3. Why is MAPE a dangerous accuracy metric, and what do you use instead?

Answer. MAPE = mean of |y − ŷ| / |y|. Its traps:

• Blows up near zero: if actual load (or demand) approaches 0, the percentage error explodes or divides by zero.
• Asymmetric: it penalizes over-forecasts more than under-forecasts, biasing models to predict low.
• Undefined/meaningless for series that hit zero or go negative.

Alternatives, matched to the goal:

For load forecasting, MASE (or plain MAE plus a seasonal-naïve baseline) is the safe default; reserve MAPE for strictly-positive, away-from-zero series where stakeholders insist on a percentage.

---

Q4. What do PICP and MPIW mean, and why report both?

Answer. They score a prediction interval, not a point.

• PICP (Prediction Interval Coverage Probability) = the fraction of actuals that land inside the interval. It should be ≈ the nominal level 1 − α (e.g. 0.90 for a 90% band). PICP below target = overconfident (too narrow); above = wastefully wide.
• MPIW (Mean Prediction Interval Width) = the average width q_hi − q_lo. Smaller is better at the target coverage.

You must report both because either alone is gameable: you get PICP = 1.0 by making the band infinitely wide, and MPIW ≈ 0 by making the band useless. The real objective is the narrowest band that still hits the coverage target — high PICP and low MPIW. When comparing two interval models, fix PICP at the target and prefer the smaller MPIW.

---

Q5. Point forecast vs probabilistic forecast — when does the difference matter?

Answer. A point forecast is a single number; a probabilistic forecast is a distribution / quantiles / an interval. The difference matters whenever the decision is asymmetric or tail-driven:

• Reserve procurement: you buy generation to cover the 95th percentile of load, not the mean — a shortfall (blackout) costs far more than a surplus.
• Staffing / inventory / safety stock: you size to a service level (a quantile), not the average.
• Energy trading / bidding: you price the risk, which lives in the spread.

The point forecast literally cannot express "cover to the 95th percentile." Whenever the cost of being wrong is lopsided, you need the interval. If the loss were perfectly symmetric and you only cared about the average, a point forecast (the mean) would suffice — but that is rare in operations. See Probabilistic Forecasting & Prediction Intervals.

---

Q6. Explain quantile regression and pinball loss.

Answer. To predict the τ-th quantile you minimize pinball (quantile) loss, an asymmetric absolute error:

   L_tau(y, yhat) = max( tau*(y - yhat), (tau - 1)*(y - yhat) )

     under-predict (y > yhat): weight tau
     over-predict  (y < yhat): weight (1 - tau)

For τ = 0.9, under-prediction is weighted 0.9 and over-prediction 0.1 — a 9:1 penalty. The minimizer therefore keeps raising ŷ until only 10% of actuals exceed it: that point is the 90th percentile. MSE, being symmetric, targets the mean; pinball's asymmetry is exactly what targets a chosen quantile. Fit τ = 0.05, 0.50, 0.95 and you have a median plus a 90% interval, with no distributional assumption — ideal for skewed load. Watch for quantile crossing (q05 > q95 on some rows); sort/clip or use monotone models.

---

Q7. How does split conformal prediction guarantee coverage?

Answer. Split conformal converts a claimed interval into a guaranteed one:

1. Split data into train and calibration (disjoint).
2. Fit base model f on train.
3. On each calibration point compute a nonconformity score sᵢ = |yᵢ − f(xᵢ)|.
4. Set q = the ⌈(n+1)(1−α)⌉-th smallest score.
5. For any new x, output f(x) ± q.

The guarantee: under exchangeability, P(y_new ∈ interval) ≥ 1 − α. The intuition is pure ranking — the new test score is one more exchangeable draw among n+1, so choosing q at rank ⌈(n+1)(1−α)⌉ caps the miss probability at α. No distribution is ever assumed, which is why it wraps any model. The (n+1) and ceiling are what make it exact in finite samples (the plain 90th percentile of n points slightly under-covers). More calibration data → tighter certified q.

---

Q8. Why does exchangeability break in time series, and how do you fix conformal for it?

Answer. Conformal's guarantee needs exchangeability — the joint distribution unchanged by reordering. Time series violate it two ways:

• Autocorrelation: consecutive points are dependent, so calibration and test scores aren't interchangeable draws.
• Distribution drift: seasons, growth, weather, new load (EVs, data centers) shift the target over time.

Naive random-split conformal can then silently lose coverage. Fixes:

ACI trades the exact finite-sample guarantee for a long-run coverage guarantee that survives drift — the right choice for a live forecasting service. See Conformal Prediction: Coverage-Guaranteed Intervals.

---

Q9. How do ensembles produce an uncertainty estimate, and why must it be calibrated?

Answer. Train several models — different seeds, features, or algorithms (this family includes deep ensembles, MC dropout, and bootstrap resampling). For each input, look at the spread (standard deviation or min–max range) of their forecasts:

• where the members agree → familiar input → low uncertainty;
• where they disagree → unusual input → high uncertainty.

That spread is a cheap, general uncertainty signal — one standard source among several, alongside parametric and quantile approaches. But it's a heuristic: the std of a handful of models has no reason to equal the true standard deviation, and in practice it's usually too narrow (overconfident). So you must calibrate it before promising coverage — for example, feed a nonconformity score like |y − mean| / std through conformal calibration, which rescales the raw spread so the band actually hits 1 − α. The general fact: conformal calibration can be applied on top of any raw uncertainty score, including an ensemble's spread — it's just an ordinary choice of score, nothing special.

---

Q10. Foundation time-series models (Chronos, TimesFM) — when does zero-shot help?

Answer. These are large models pretrained on huge, diverse collections of time series, meant to forecast a new series zero-shot (no per-series training), much like an LLM answering a fresh prompt. When they help:

• Cold start / little history: a new store, meter, or region with few observations, where a bespoke model would overfit.
• Many series, low effort: thousands of series where training one model each is impractical — a strong zero-shot baseline in one call.
• Rapid prototyping: get a credible forecast in minutes before investing in a tailored model.

When they don't clearly win:

• You have abundant history + strong known structure (clear daily/weekly seasonality, weather drivers): a well-tuned classical or gradient-boosted model with good features often matches or beats them and is cheaper to run.
• Hard operational constraints (latency, on-prem, interpretability, guaranteed intervals): a big foundation model can be overkill, and you still need conformal on top for real coverage — these models give point/limited probabilistic output, not a coverage guarantee.

The honest framing: treat them as a strong zero-shot baseline to beat, not an automatic winner. Always benchmark against a seasonal-naïve baseline (Q11) and a tuned local model.

---

Q11. How do you pick a baseline, and why is seasonal-naïve the right one here?

Answer. A baseline is the cheap model your fancy model must beat — without it you have no idea whether your effort added anything. The right baseline exploits the series' dominant structure:

• Naïve: ŷ_t = y_{t−1} (last value). Good for random-walk-like series.
• Seasonal-naïve: ŷ_t = y_{t−s} where s is the season length. For hourly load, s = 24 (same hour yesterday) or s = 168 (same hour last week). Load is strongly daily/weekly periodic, so "same hour last week" is a shockingly hard baseline to beat.
• Drift / seasonal-naïve + trend: adds a slope for growing series.

Seasonal-naïve wins as the load baseline because it captures the biggest signal (daily/weekly cycle) for free, sets the reference for MASE (error scaled by the seasonal-naïve one-step error), and instantly exposes models that add cost but no accuracy. Interview move: state the baseline first, report your model's skill relative to it, and be suspicious of any model that can't clearly beat seasonal-naïve.

---

Q12. Coding — walk-forward split, pinball loss, and the split-conformal q.

Answer. Three short, interview-favorite building blocks. Keep them simple and correct.

(a) Walk-forward (expanding-window) split — yields (train_idx, test_idx) pairs that never look into the future:

def walk_forward(n, initial, horizon):
    """Expanding-window splits over n points.
    initial = first train size; horizon = test block length. Train is always the PAST."""
    start = initial
    while start + horizon <= n:
        train_idx = range(0, start)              # everything before the cut
        test_idx = range(start, start + horizon) # the immediate future block
        yield list(train_idx), list(test_idx)
        start += horizon                          # roll forward

# for n=10, initial=4, horizon=2 -> train[0:4]test[4:6], train[0:6]test[6:8], train[0:8]test[8:10]

(b) Pinball (quantile) loss — the asymmetric error that targets quantile tau:

def pinball(y, yhat, tau):
    """Average pinball loss; minimizing it fits the tau-th quantile."""
    total = 0.0
    for a, p in zip(y, yhat):
        d = a - p
        total += max(tau * d, (tau - 1) * d)     # under-pred weight tau, over-pred weight 1-tau
    return total / len(y)

# pinball([100], [90], 0.9) -> max(0.9*10, -0.1*10) = 9.0   (under-shooting q0.9 hurts 9x)

(c) Split-conformal quantile q — the padding for 1 − alpha coverage:

import math
def conformal_q(scores, alpha):
    """q = the k-th smallest nonconformity score, k = ceil((n+1)(1-alpha))."""
    n = len(scores)
    k = min(math.ceil((n + 1) * (1 - alpha)), n)  # clip: if k>n, sample certifies only the max
    return sorted(scores)[k - 1]

# scores=[42,55,63,71,88,96,110,125,140,210], alpha=0.1 -> k=ceil(11*0.9)=10 -> q=210
# a new point yhat=8400 -> interval [8400-210, 8400+210] = [8190, 8610], coverage >= 90%

Follow-ups interviewers push on: walk-forward must never shuffle; pinball's asymmetry is what pins the quantile (symmetric loss → mean); and the (n+1)/ceiling in conformal_q is what makes coverage exact, not the plain percentile. Have the k = ceil((n+1)(1-alpha)) formula memorized — it's the single most-asked conformal detail.

---

Resources

• Hyndman & Athanasopoulos — Forecasting: Principles and Practice (FPP3), incl. cross-validation, baselines, MASE — https://otexts.com/fpp3/
• Koenker & Bassett — Quantile Regression — https://www.jstor.org/stable/1913643
• Angelopoulos & Bates — A Gentle Introduction to Conformal Prediction — https://arxiv.org/abs/2107.07511
• Romano, Patterson & Candès — Conformalized Quantile Regression — https://arxiv.org/abs/1905.03222
• Gibbs & Candès — Adaptive Conformal Inference — https://arxiv.org/abs/2106.00170
• Ansari et al. — Chronos: Learning the Language of Time Series — https://arxiv.org/abs/2403.07815
• Das et al. — TimesFM: A decoder-only foundation model for time-series forecasting — https://arxiv.org/abs/2310.10688
• GEFCom2014 — probabilistic energy forecasting competition — https://doi.org/10.1016/j.ijforecast.2016.02.001
• Deep dives: Probabilistic Forecasting & Prediction Intervals · Conformal Prediction: Coverage-Guaranteed Intervals · Forecast Evaluation & Backtesting