walk_forward_mda

Defined in fynance.research

walk_forward_mda(model_factory, X, y, *, train=252, test=63, step=None, purge=0, metric=None, n_repeats=5, seed=0, feature_names=None)

Mean-decrease-accuracy feature importance, evaluated out-of-fold.

Splits X/y with the purged walk-forward generator fynance.data.split.walk_forward. Per fold: fit model_factory() once on the train window, score it on the (unpermuted) test window, then – without refitting – for each repeat and each feature, permute that feature’s column within the test window only and re-score. The score drop (baseline - permuted) is the importance signal; it is averaged across folds x repeats (mean -> importances, std -> stds).

Parameters:

model_factorycallable

Zero-argument callable returning a fresh model exposing fit(X, y) -> self / predict(X) -> ndarray (fynance.core.SignalModel). Called once per fold, so every fold starts from an untrained model.

Xnumpy.ndarray, shape (n_samples, n_features)

Feature matrix, strictly time-ordered (row t must not depend on rows after t).

ynumpy.ndarray, shape (n_samples,)

Target aligned with X.

train, testint

Train and test window lengths, forwarded to walk_forward.

stepint, optional

Roll step (defaults to test; see walk_forward).

purgeint

Embargo between train and test windows, forwarded as-is.

metriccallable, optional

(y_true, y_pred) -> float, higher is better. Defaults to the negative mean squared error (-mean((y_true - y_pred) ** 2)).

n_repeatsint

Number of independent permutation draws per feature per fold.

seedint

Master seed. Fold i / repeat r draws its permutations from numpy.random.default_rng(seed + i * 1000 + r), so the result is fully reproducible for a fixed seed and independent across folds and repeats.

feature_nameslist of str, optional

Labels carried through to ImportanceResult.feature_names; must match X.shape[1] if given.

Returns:

ImportanceResult

See ImportanceResult.

Raises:

ValueError

If X is not 2-D, y is not 1-D, their lengths mismatch, train + test exceeds the number of samples (no fold would fit), or feature_names has the wrong length.

Examples

A closed-form linear model recovers the planted feature (column 0) as the most important, well above the pure-noise columns:

>>> import numpy as np
>>> from fynance.research.importance import walk_forward_mda
>>> class LinearModel:
...     def fit(self, X, y):
...         design = np.column_stack([X, np.ones(len(X))])
...         self.coef_, *_ = np.linalg.lstsq(design, y, rcond=None)
...         return self
...     def predict(self, X):
...         design = np.column_stack([X, np.ones(len(X))])
...         return design @ self.coef_
>>> rng = np.random.default_rng(0)
>>> n, k = 300, 3
>>> X = rng.standard_normal((n, k))
>>> y = 2.0 * X[:, 0] + 0.1 * rng.standard_normal(n)
>>> result = walk_forward_mda(LinearModel, X, y, train=150, test=50,
...                            n_repeats=3, seed=0)
>>> result.n_folds
3
>>> bool(result.importances[0] > result.importances[1])
True
>>> bool(result.importances[0] > result.importances[2])
True