ML Foundations
Check forecast target availability before a temporal split
Chronological feature rows can still carry future training labels. Audit target times against the model fit boundary, including the forecast horizon.
Evidence and testing
Review record: Tensor Almanac automated evidence gate (no independent human review). Example versions, recorded outputs and limitations appear below.
A past feature row can have a future label
Consider a model fitted once immediately before time 8, using only observations with timestamps strictly below 8. It then stays fixed while forecasts are issued after observing times 8 and 9. Each sample uses three observations ending at forecast origin t and predicts the observation at t+2. Sorting samples by origin preserves their chronology, but does not establish when their labels became available.
The executed fixture has observations at integer times zero through eleven. Selecting training origins below 8 admits origins 2 through 7. The last two of those rows require labels at times 8 and 9, which were unavailable under the stated fit contract. The repair keeps only rows whose target timestamp is also strictly below 8. Training origins then end at 5, with their latest label at 7.
References: [2]
Carry origin and target times beside the arrays
The samples helper constructs three-element feature windows with NumPy sliding_window_view, which returns moving windows with a window dimension appended to the trimmed input shape. It keeps only windows with an in-range future target and returns origin and target-time arrays alongside the feature and label arrays. [1]
Run python examples/forecast_target_boundary.py. The training mask checks both origin and target availability. Assertions verify the two excluded rows, the retained target boundary, disjoint training and evaluation rows, feature-window alignment and the single-sample boundary case. Three invalid length settings are rejected. Values deliberately equal timestamps so an indexing error is visible; this is not a forecasting dataset or an accuracy experiment.
The first evaluation feature window contains observations at times 6, 7 and 8. Earlier observations used during training can legitimately provide historical context for a forecast made after observing time 8. Overlap in past inputs is therefore a different question from admitting unavailable future labels during the earlier fit.
Executed with Python 3.12.14 and NumPy 2.3.5 on CPU. Exact stdout and source-byte SHA-256 are preserved in private evidence.
"""Audit target availability for a fixed-fit temporal holdout."""
import numpy as np
def samples(values, history, horizon):
if values.ndim != 1 or history < 1 or horizon < 1:
raise ValueError("one-dimensional data and positive lengths required")
if len(values) < history + horizon:
raise ValueError("not enough observations for one labeled sample")
origins = np.arange(history - 1, len(values) - horizon)
inputs = np.lib.stride_tricks.sliding_window_view(values, history)[:len(origins)]
target_times = origins + horizon
return inputs, values[target_times], origins, target_times
values = np.arange(12, dtype=np.float64)
inputs, targets, origins, target_times = samples(values, history=3, horizon=2)
fit_before = 8
naive_train = origins < fit_before
train = naive_train & (target_times < fit_before)
test = origins >= fit_before
crossing = naive_train & (target_times >= fit_before)
assert origins[naive_train].tolist() == [2, 3, 4, 5, 6, 7]
assert origins[crossing].tolist() == [6, 7]
assert target_times[crossing].tolist() == [8, 9]
assert origins[train].tolist() == [2, 3, 4, 5]
assert target_times[train].max() == 7
assert origins[test].tolist() == [8, 9]
assert np.array_equal(inputs[test][0], [6, 7, 8])
assert targets[test].tolist() == [10.0, 11.0]
assert not np.any(train & test)
# Each feature window ends at its forecast origin; the target is two steps later.
assert np.array_equal(inputs[:, -1], values[origins])
assert np.array_equal(targets, values[origins + 2])
one = samples(np.arange(5), history=3, horizon=2)
assert one[0].shape == (1, 3) and one[3].tolist() == [4]
rejected = 0
for data, history, horizon in ((values, 0, 2), (values, 3, 0), (values[:4], 3, 2)):
try:
samples(data, history, horizon)
except ValueError:
rejected += 1
assert rejected == 3
print("origin-only training times:", origins[naive_train].tolist())
print("unavailable training target times:", target_times[crossing].tolist())
print("availability-checked training times:", origins[train].tolist())
print("evaluation origin/target times:", origins[test].tolist(), target_times[test].tolist())
print("first evaluation feature window:", inputs[test][0].tolist())
print("invalid length fixtures rejected:", rejected)
Recorded output
origin-only training times: [2, 3, 4, 5, 6, 7] unavailable training target times: [8, 9] availability-checked training times: [2, 3, 4, 5] evaluation origin/target times: [8, 9] [10, 11] first evaluation feature window: [6.0, 7.0, 8.0] invalid length fixtures rejected: 3
References: [1]
Choose gaps from availability rather than window width
TimeSeriesSplit documents gap as the number of samples removed from the end of each training set before its test set. In this regular, one-origin-per-step fixture, removing two trailing training samples implements the chosen two-step target boundary. The example directly checks timestamps; it does not execute TimeSeriesSplit. [2]
A row-count gap is not a universal elapsed-time rule. Irregular observations, delayed label arrival, aggregated targets and changing forecast horizons need their actual availability times. For this fixed-fit experiment the comparison is strictly before 8; a system that refits after observing time 8 has a different contract. The helper assumes integer positive history and horizon lengths and immediate observation availability. No fitted preprocessing, estimator, rolling refit or uncertainty evaluation is included.
References: [2]