pbo¶

Defined in fynance.research

pbo(returns_panel, n_blocks=16, metric=None)[source]

Probability of backtest overfitting via combinatorially symmetric CV.

Implements the CSCV procedure of Bailey, Borwein, Lopez de Prado & Zhu (2015): split the T periods into n_blocks contiguous blocks; for every way to assign half the blocks to an in-sample (IS) half and the other half to the complementary out-of-sample (OOS) half (\(\binom{n\_blocks}{n\_blocks / 2}\) splits total), pick the config that scores best on the IS half by metric, then locate that same config’s OOS-half performance among all configs’ OOS performances as a relative rank \(r \in (0, 1)\) (average-tie rank divided by n_configs + 1, so the best OOS config sits close to 1 and the worst close to 0). The rank logit is \(\lambda = \ln(r / (1 - r))\); pbo is the share of splits with \(\lambda \le 0\), i.e. where the in-sample winner performs at or below the OOS median — a config search that overfits systematically sends the IS winner to the bottom half OOS.

Parameters:

returns_panelnumpy.ndarray, shape (T, n_configs): Per-period returns, one column per configuration tried (see returns_panel to build this from Experiment runs).
n_blocksint: Number of contiguous blocks to split T into. Must be even (a balanced IS/OOS split) and no greater than 16 — beyond that \(\binom{n\_blocks}{n\_blocks/2}\) exceeds 12,870 splits.
metriccallable, optional: metric(returns) -> float scoring a 1-D per-period return stretch, higher is better. Defaults to an annualization-free Sharpe (mean over std of the returns) — deliberately not fynance.metrics.sharpe, which assumes a price/equity curve rather than a returns series.

Returns:

PBOResult: The overfitting diagnostics (see PBOResult).

Raises:

ValueError: If returns_panel is not 2-D, n_blocks is odd, n_blocks implies more than 12,870 IS/OOS splits, or T < n_blocks.

Examples

>>> import numpy as np
>>> from fynance.research import pbo
>>> rng = np.random.default_rng(0)
>>> panel = rng.normal(0.0, 0.01, size=(400, 8))  # pure noise, no edge
>>> result = pbo(panel, n_blocks=8)
>>> 0.0 <= result.pbo <= 1.0
True
>>> result.logits.shape
(70,)