Time-Series Forecasting Fundamentals

TL;DR — A time series is data with a clock: the order of the rows carries information. That one fact changes everything. You must never shuffle it, never let the future leak into the past, and never split it randomly. The core vocabulary is trend, seasonality, autocorrelation, stationarity; the core geometry is a forecast origin (the "now") and a horizon (how far ahead). To predict many steps you go recursive (feed predictions back) or direct (one model per step). The whole job is a staged pipeline: ingest → features → model → forecast, with a hard rule at every stage — respect time. Break that rule and you get a model that looks brilliant offline and fails the moment it meets tomorrow.

---

1. Simple explanation

Most machine learning treats each row as independent. Shuffle the rows, split them randomly, and nothing breaks. A time series is different. Each row has a timestamp, and the rows are in order. Yesterday's electricity demand tells you a lot about today's. The order is the signal.

Our running example is short-term load forecasting (STLF): predict the hourly electricity demand on a power grid — measured in megawatts (MW) — for the next 24 hours. Grid operators need this to decide how many power plants to switch on. Too little and the lights go out; too much and money burns.

Analogy — reading a person's daily routine. Imagine you watch one person for weeks. They wake at 7am, coffee at 7:30, gym on Mondays, big dinner on Sundays. After a while you can predict their next hour from the clock and their recent actions. You do not shuffle the days — that would destroy the routine. You do not peek at what they do at 8am to predict 7am — that is cheating. And if you want to guess the whole afternoon, you either predict 2pm, then use that guess to help predict 3pm (recursive), or you train a separate "what-do-they-do-at-3pm" expert (direct).

Electricity load is exactly this, at grid scale:

• A daily rhythm (low at 4am, peaks in the evening) — this is seasonality.
• A weekly rhythm (weekends differ from weekdays) — seasonality again, longer period.
• A slow drift upward over years as population grows — this is trend.
• "This hour looks like the last hour" — this is autocorrelation.

Learn to name these four things and half the job is done.

---

2. Diagram

                         THE FORECASTING GEOMETRY
   past (known)                    │  future (unknown, to predict)
   ─────────────────────────────── │ ──────────────────────────────────►  time
   load: ... 3.1 3.4 3.9 4.2 4.6   │   ?    ?    ?   ...   ?
                                   ▲
                            FORECAST ORIGIN ("now", t0)
                                   │◄──────── HORIZON H = 24h ────────►│
                                   t0+1  t0+2  t0+3   ...            t0+24

   SEASONALITY (daily)      TREND (slow drift)     AUTOCORRELATION
   MW ▲   ╱╲    ╱╲    ╱╲     MW ▲        ______      corr(load_t, load_{t-k})
      │  ╱  ╲  ╱  ╲  ╱  ╲       │   _____╱             1 ┤█
      │ ╱    ╲╱    ╲╱    ╲      │__╱                     ┤█ █          █(lag 24)
      └───────────────────►     └──────────────►        ┤█ █ ▁ ▁ ▁ ▁ █
        24h  48h  72h             years                 └┴─┴─┴─┴─┴─┴─┴──► lag k
                                                          1 2 3 ...  24

   THE PIPELINE (each stage must RESPECT TIME)
   ┌─────────┐   ┌──────────────┐   ┌────────┐   ┌───────────┐
   │ INGEST  │──►│  FEATURES    │──►│ MODEL  │──►│ FORECAST  │
   │ raw MW, │   │ lags, rolling│   │ fit on │   │ H steps   │
   │ weather │   │ calendar     │   │ PAST   │   │ ahead     │
   └─────────┘   └──────────────┘   └────────┘   └───────────┘
        └── no shuffle · no future leak · fit transforms on train only ──┘

---

3. How it works

3.1 What makes a time series different

Autocorrelation is the mathematical heart. It is the correlation of the series with a lagged copy of itself. For load, autocorrelation is high at lag 1 (last hour), spikes at lag 24 (same hour yesterday), and spikes again at lag 168 (same hour last week). Those spikes tell you which lag features will be useful.

Stationarity matters because many classical models assume it. A series is (weakly) stationary if its mean and variance do not change over time and its autocorrelation depends only on the gap between points, not on where you are. Raw load is not stationary — it trends and it cycles. You handle that by differencing (subtract the value 24 hours ago) or by adding trend/seasonal features so the model can absorb it.

3.2 Forecast origin and horizon

Two words you will use in every design review:

• Forecast origin (t0): the "now". The last timestamp you actually have data for. Everything at or before t0 is known; everything after is unknown.
• Horizon (H): how many steps ahead you predict. Day-ahead STLF has H = 24 hourly steps.

A forecast is a function from known history up to t0 to values at t0+1 … t0+H. The origin moves forward in production: every hour a new reading arrives, the origin advances, and you re-forecast. This moving origin is exactly what backtesting replays (see Forecast Evaluation & Backtesting).

3.3 Multi-step forecasting: recursive vs direct vs multi-output

You rarely want just one step. Three standard strategies:

Rule of thumb: recursive for short horizons or classical models; direct when a few specific horizons matter and error compounding hurts; multi-output when you use a deep model that naturally emits a sequence.

3.4 The golden rule and time-series leakage

The golden rule: never shuffle, never use random k-fold, never let information from the future touch the past. Time only flows one way — your validation must too.

Leakage in time series means a feature or a preprocessing step secretly used future information that would not exist at prediction time. It makes offline scores look amazing and production scores collapse. The four classic leaks:

Leakage audit — the lag-1 check. A load_lag_1 feature must exactly equal the target shifted forward by one step. If df["load_lag_1"].iloc[t] does not equal df["load"].iloc[t-1], your lag is misaligned and may be pulling from the future. A one-line assertion (see §5) catches it. Do this for every lag and rolling feature before you trust a single metric.

3.5 Walk-forward validation (respecting time)

Instead of random folds you slide a window forward through time. Two shapes:

EXPANDING window                     ROLLING (sliding) window
train ██████──test──                 train  ████──test──
train ████████──test──               train    ████──test──
train ██████████──test──             train      ████──test──
(train grows; keeps all history)     (train fixed length; forgets old data)

Train is always before test in time. Optionally leave a gap/embargo between them so a lag feature cannot straddle the boundary. This is covered in depth in Forecast Evaluation & Backtesting.

3.6 The staged pipeline (and how it maps to workflow stages)

A forecasting system is a pipeline with four stages, each a checkpoint where the time rule is enforced:

Each stage is an independent, testable unit with a clear contract. That means a forecasting pipeline can be organized as workflow stages and run by an orchestrator — one stage's output is the next stage's input, with retries and validation between them. If you are wiring this into a larger system, the staging patterns in Agent Orchestration apply directly: sequential stages, a shared data contract, and a gate between steps.

---

4. The math

Notation. Let y_t be the load (MW) at hour t. History up to the origin is y_1, …, y_{t0}. We want ŷ_{t0+h} for h = 1 … H.

Autocorrelation at lag k (how much a value resembles its own past):

r_k = Σ_{t=k+1}^{n} (y_t − ȳ)(y_{t−k} − ȳ)  /  Σ_{t=1}^{n} (y_t − ȳ)²

r_k near 1 means strong positive autocorrelation at gap k. For load, r_1, r_24, r_168 are all high.

Differencing to reduce non-stationarity. First difference: Δy_t = y_t − y_{t−1}. Seasonal (daily) difference: Δ_24 y_t = y_t − y_{t−24}.

Recursive multi-step. With a 1-step model f:

ŷ_{t0+1} = f(y_{t0}, y_{t0−1}, …)
ŷ_{t0+2} = f(ŷ_{t0+1}, y_{t0}, …)     ← the prediction feeds back in
…
ŷ_{t0+h} = f(ŷ_{t0+h−1}, …)

Direct multi-step. Train H models f_1 … f_H, each mapping the same known history to a specific horizon:

ŷ_{t0+h} = f_h(y_{t0}, y_{t0−1}, …)     for h = 1 … H, no feedback

Seasonal-naïve forecast (the baseline you must beat). Copy the value one full season back — for hourly load with daily season s = 24:

ŷ_{t0+h} = y_{t0+h−24}

Worked numeric example — recursive vs seasonal-naïve on hourly load.

Say the last few known loads (GW) ending at origin t0 (which is 23:00) are:

hour   ...  20:00  21:00  22:00  23:00 (= t0)
load        4.6    4.4    4.1    3.8

And exactly 24 hours earlier the loads for the next three hours (00:00, 01:00, 02:00 of the previous day) were 3.5, 3.2, 3.0.

Seasonal-naïve simply copies yesterday:

ŷ(00:00) = 3.5   ŷ(01:00) = 3.2   ŷ(02:00) = 3.0

Recursive with a toy persistence-with-decay model f(y) = 0.95 · y_last (each hour keeps 95% of the previous load — a stand-in for the overnight decline):

ŷ(00:00) = 0.95 · 3.8      = 3.61
ŷ(01:00) = 0.95 · 3.61     = 3.43   ← uses the PREVIOUS prediction
ŷ(02:00) = 0.95 · 3.43     = 3.26

Now suppose the actual loads turn out to be 3.5, 3.2, 3.0. Absolute errors:

              00:00        01:00        02:00      MAE
seasonal-naïve |3.5−3.5|=0  |3.2−3.2|=0  |3.0−3.0|=0   0.00
recursive toy  |3.61−3.5|=.11 |3.43−3.2|=.23 |3.26−3.0|=.26  0.20

The seasonal-naïve baseline nails it here because load repeats daily — which is exactly why you must beat seasonal-naïve before claiming a model is good. Notice also the recursive error grows each step (0.11 → 0.23 → 0.26): that is error compounding, the core weakness of recursion.

---

5. Real code

Runnable with pandas/numpy. It builds a load-like series, engineers time-safe features, runs the lag-1 leakage audit, splits by time (no shuffle), and produces both a recursive and a seasonal-naïve forecast.

"""Time-series fundamentals on hourly electricity load:
build features safely, audit for leakage, split by TIME, forecast H steps ahead."""
import numpy as np
import pandas as pd

# ---------- 1. INGEST: a synthetic but realistic hourly load series ----------
rng = np.random.default_rng(0)
n = 24 * 60                                   # 60 days of hourly data
idx = pd.date_range("2024-01-01", periods=n, freq="h")
hour = idx.hour.values
dow  = idx.dayofweek.values                   # 0=Mon … 6=Sun

daily   = 2.0 * np.sin((hour - 6) / 24 * 2 * np.pi)   # low at night, peak evening
weekly  = np.where(dow >= 5, -0.8, 0.0)               # weekends lower
trend   = np.linspace(0, 0.5, n)                      # slow upward drift
noise   = rng.normal(0, 0.15, n)
load    = 5.0 + daily + weekly + trend + noise        # GW
df = pd.DataFrame({"load": load}, index=idx)

# ---------- 2. FEATURES: lags, rolling stats, calendar (all knowable at t0) ----------
df["load_lag_1"]   = df["load"].shift(1)              # last hour
df["load_lag_24"]  = df["load"].shift(24)             # same hour yesterday
df["load_lag_168"] = df["load"].shift(168)            # same hour last week
df["roll_mean_24"] = df["load"].shift(1).rolling(24).mean()   # shift(1) FIRST -> no peeking at t
df["hour"] = df.index.hour
df["dow"]  = df.index.dayofweek
df = df.dropna()                                      # drop rows without full history

# ---------- 3. LEAKAGE AUDIT: lag-1 must equal target shifted by one ----------
# For every row, load_lag_1 should equal the previous row's load. If not, the lag leaks.
audit = np.allclose(df["load_lag_1"].values[1:], df["load"].values[:-1])
assert audit, "LEAK: load_lag_1 does not equal load.shift(1) — feature is misaligned!"
print("leakage audit passed: load_lag_1 == load.shift(1)")

# ---------- 4. SPLIT BY TIME (never shuffle) ----------
H = 24                                                # 24-hour horizon
train, test = df.iloc[:-H], df.iloc[-H:]             # test is the LAST day, strictly after train
print(f"train ends {train.index[-1]}, forecast origin = t0, horizon = {H}h")

# ---------- 5a. SEASONAL-NAIVE forecast: copy the value 24h earlier ----------
seasonal_naive = df["load"].shift(24).iloc[-H:]      # y_{t0+h-24}

# ---------- 5b. RECURSIVE forecast with a tiny fitted model ----------
# Fit load_t ~ a * load_{t-1} + b on TRAIN ONLY, then roll it forward H steps.
x = train["load_lag_1"].values
y = train["load"].values
a, b = np.polyfit(x, y, 1)                            # 1-step linear model
print(f"fitted 1-step model: load_t = {a:.3f}*load_lag1 + {b:.3f}")

last = train["load"].iloc[-1]                         # value at origin t0
recursive = []
for _ in range(H):
    nxt = a * last + b                                # predict one step
    recursive.append(nxt)
    last = nxt                                        # FEED PREDICTION BACK (recursive)
recursive = pd.Series(recursive, index=test.index)

# ---------- 6. Compare with MAE ----------
mae = lambda p: float(np.mean(np.abs(p.values - test["load"].values)))
print(f"MAE seasonal-naive : {mae(seasonal_naive):.3f} GW")
print(f"MAE recursive toy  : {mae(recursive):.3f} GW")

Typical output (numbers vary slightly with the random seed):

leakage audit passed: load_lag_1 == load.shift(1)
train ends 2024-02-28 23:00:00, forecast origin = t0, horizon = 24h
fitted 1-step model: load_t = 0.9xx*load_lag1 + 0.xx
MAE seasonal-naive : 0.2xx GW
MAE recursive toy  : 0.5xx GW

The seasonal-naïve baseline is usually hard to beat on load, which is the whole point of §6.

---

6. Real-world example

Day-ahead grid load forecast for a regional operator.

• Setting. A balancing authority must submit, by 10:00 today, an hourly load forecast for all 24 hours of tomorrow (H = 24). This feeds the day-ahead energy market: how much generation to schedule.
• Data. Five years of hourly load (MW), plus a temperature forecast from the weather service. Peak summer load ≈ 25,000 MW; overnight winter trough ≈ 12,000 MW.
• Signal. Load has a strong daily cycle (evening peak), a weekly cycle (weekday > weekend), a yearly cycle (summer AC + winter heating), and it is highly temperature-driven (a 1°C rise on a hot day can add hundreds of MW of air-conditioning).
• Origin & horizon. Origin is 10:00 today; the horizon covers hours 24–47 ahead (all of tomorrow). Because a specific horizon matters and error compounding is costly at the evening peak, operators often use direct models per hour-of-day, or a multi-output deep model.
• Features. load_lag_24, load_lag_168, rolling 24h mean, hour, dow, holiday flag, and forecast temperature. The holiday flag matters — a public holiday looks like a Sunday even on a Tuesday.
• Why the baseline is sacred. Seasonal-naïve ("tomorrow = last week, same hour") already gets within a few percent MAPE because load is so repetitive. A model earns its keep only by beating that — usually on hot/cold snaps and holidays, where naïve fails.
• Consequence of leakage. If an engineer fits the temperature scaler on all five years (including the future), the offline MAPE looks ~10% better than reality. In production the model under-forecasts the summer peak, the operator schedules too little generation, and they buy expensive emergency power. The bug is a scaler fit across the train/test boundary — pure leakage.

The pipeline (ingest weather + load → build lag/calendar/weather features → fit direct hourly models → emit 24 values with intervals) is exactly the four-stage design in §3.6, run fresh every morning as the origin advances.

---

7. Interview questions companies actually ask

Q [Amazon Forecasting / AWS] "Why can't you just use random k-fold cross-validation on a
   time series?"
  A Because k-fold puts future rows into the training set and past rows into the test set.
    The model then 'learns' from the future and its offline score is fantasy. Time only flows
    one way, so validation must too: use walk-forward (expanding or rolling) splits where the
    train slice is always strictly before the test slice, ideally with a gap/embargo so lag
    features can't straddle the boundary.

Q [Google / DeepMind] "Explain recursive vs direct multi-step forecasting and their trade-off."
  A Recursive trains one 1-step model and feeds each prediction back as the next input — simple
    and uses the series' own dynamics, but errors compound over the horizon. Direct trains H
    separate models, one per horizon, so there's no error feedback and each horizon is tuned
    independently — but it's H models to maintain and it ignores cross-step structure.
    Multi-output emits the whole horizon vector at once (LSTM/N-BEATS). Pick recursive for short
    horizons, direct when specific far horizons matter, multi-output for deep sequence models.

Q [Meta / infra] "What is data leakage in a time series and give a concrete example?"
  A Using information at training time that wouldn't exist at prediction time. Classic examples:
    fitting a StandardScaler on the whole series (its mean includes future values), a rolling
    mean computed across the train/test boundary, or a feature derived from the target. Fix by
    fitting all transforms on the train slice only and computing lag/rolling features causally
    (shift before rolling).

Q [Uber / Michelangelo] "How would you audit a lag feature for leakage?"
  A Assert the identity that defines it: load_lag_1 must exactly equal load.shift(1). If
    df['load_lag_1'][t] != df['load'][t-1] for any row, the lag is misaligned and may pull from
    the future. Run the same check for every lag and rolling feature before trusting any metric.

Q [Netflix] "Your model beats the mean baseline. Is it good?"
  A Not necessarily. For a strongly seasonal series like load, the right baseline is
    seasonal-naïve (copy the value one season back), not the mean. Seasonal-naïve is very strong
    on load, so beating the mean proves nothing. A model is 'good' only when it beats
    seasonal-naïve, especially on the hard cases: holidays and temperature snaps.

Q [Two Sigma / quant] "What does stationarity mean and why do you care?"
  A A series is (weakly) stationary if its mean, variance, and autocorrelation structure don't
    change over time. Many classical models (ARIMA) assume it. Raw load isn't stationary — it
    trends and cycles — so you difference it (first and seasonal differences) or add
    trend/seasonal features so the model can absorb the non-stationarity.

Q [Microsoft / energy] "What features would you engineer for hourly load forecasting?"
  A Calendar: hour-of-day, day-of-week, holiday flag. Lags: lag_1, lag_24 (yesterday same hour),
    lag_168 (last week same hour). Rolling: 24h mean/std computed causally. Weather: forecast
    temperature (and its non-linear effect — heating and cooling both raise load). The lag_24
    and lag_168 features capture the daily and weekly seasonality directly.

Q [Robinhood / any DS] "What are forecast origin and horizon, and why do they move?"
  A The origin is 'now' — the last timestamp with data. The horizon is how many steps ahead you
    predict (24 for day-ahead hourly). In production the origin advances every time new data
    arrives, so you re-forecast on a rolling basis. Backtesting replays this moving origin over
    history to estimate real performance.

Q [Stripe] "How do you handle missing readings and timezone issues at ingest?"
  A Reindex to a complete hourly grid, fill short gaps causally (forward-fill or interpolate
    using only past values), flag long gaps, and normalize everything to a single timezone —
    watching daylight-saving transitions, which create a duplicated or missing hour that will
    silently break lag_24 alignment if ignored.

---

8. When to use / tradeoffs

   USE a time-series-aware approach when:
     ✓ rows are timestamped and their ORDER carries information
     ✓ there is autocorrelation, seasonality, or trend to exploit
     ✓ you predict FORWARD in time (a horizon), not a static label
   DON'T over-engineer when:
     ✗ the target is truly independent of time — then it's ordinary regression/classification
     ✗ you have almost no history (cold start) — a seasonal-naïve baseline or a foundation model
       may beat a custom model you can't train
   HONEST LIMITS:
     ✗ leakage is easy to introduce and hard to see — always run the audit
     ✗ recursive forecasts compound error; long horizons get unreliable fast
     ✗ regime changes (a new large factory, an EV boom) break patterns the model learned
     ✗ a single point forecast hides risk — you often need INTERVALS (see 14-2)
   THE CORE REMINDER:
     the value of a time series lives in its ORDER. Protect that order at every pipeline stage
     and most disasters never happen.

---

9. Summary + related articles

• A time series carries information in its order — never shuffle, never random-split.
• The four names to master: trend, seasonality, autocorrelation, stationarity.
• Forecast origin = "now"; horizon = how far ahead. The origin moves in production.
• Multi-step: recursive (feedback, compounds error), direct (one model per step), multi-output (whole vector at once).
• Leakage = future info touching the past: shuffled folds, scalers fit on the whole series, boundary-crossing rolling features, target-derived features. Audit every lag with lag_1 == shift(1).
• Validate with walk-forward (expanding/rolling) splits, and beat seasonal-naïve before celebrating.
• Build it as a staged pipeline (ingest → features → model → forecast), each stage time-safe — and organizable as orchestrated workflow stages.

Related: Forecasting Models: From SARIMA to Foundation Models · Forecast Evaluation & Backtesting · Agent Orchestration

Resources

• Hyndman & Athanasopoulos, Forecasting: Principles and Practice (free online, the standard text) — https://otexts.com/fpp3/
• Rob Hyndman on cross-validation for time series (walk-forward) — https://robjhyndman.com/hyndsight/tscv/
• Bontempi et al., "Machine Learning Strategies for Time Series Forecasting" (recursive vs direct) — https://link.springer.com/chapter/10.1007/978-3-642-36318-4_3
• scikit-learn TimeSeriesSplit (walk-forward CV) — https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html
• EIA hourly electricity demand by balancing authority — https://www.eia.gov/opendata/
• Kaggle: hourly energy consumption datasets — https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption