Crypto backtest parameter sensitivity: inspect the neighborhood

Crypto backtest parameter sensitivity asks how a strategy's reported results change when nearby settings are tested while other assumptions stay fixed. Inspect a neighborhood around the chosen configuration, including combinations of changes, rather than only its best cell. A broad stable region can reduce concern about one precise setting, but it cannot establish future profitability or repair biased data and unrealistic fills.

A lookback of 20 candles should prompt a practical question: what happens at 18 and 22? If a tiny adjustment destroys the result, investigate which trades changed before treating 20 as a meaningful discovery.

Define the neighborhood before viewing its results

Write down the chosen configuration, why each parameter exists and what a nearby value means in that parameter's units. A two-candle adjustment to a 20-candle lookback is a different intervention from changing a volatility stop multiplier from 2.0 to 2.2. A categorical choice such as market versus limit entry needs a separate comparison; it has no natural numeric neighbor.

For an initial diagnostic, choose a small declared grid around an existing candidate. Keep the venue, instrument, data version, evaluation dates, capital, sizing rule, execution convention and cost model fixed. Save the full configuration for each cell.

Use plausible increments derived from the rule's meaning. An integer lookback cannot move by 0.1 candle. A threshold rounded internally to two decimals may make several apparent settings identical. Record the effective values after rounding and reject impossible combinations visibly.

This is distinct from asking which parameter set maximizes a score. QuantConnect describes optimization as selecting parameters against an objective and warns that close fitting to historical data can weaken out-of-sample performance. Its documentation supports that distinction; it does not supply a universal safe neighborhood size. QuantConnect optimization introduction

Two identical winners, two different neighborhoods

The following two tables are invented educational result sets, not outputs from a market replay. Imagine two candidate crypto rules whose settings include a lookback and a stop-distance multiplier. Every cell represents a hypothetical net portfolio return over the same evaluation interval, with the same declared modeled costs. The examples illustrate how to read a surface; they do not describe KLINEO results.

Candidate A has a broad favorable neighborhood:

Candidate B has an isolated high cell:

Both chosen center cells report 8.0%. Candidate A's nine-cell median is 7.5%, and its range is 8.0 − 7.1 = 0.9 percentage points. Candidate B's median is 0.1%, and its range is 8.0 − (−1.2) = 9.2 percentage points. Four of B's eight neighbors lose money.

These summaries describe the chosen grid, not a probability distribution of future returns. Nearby runs reuse much of the same history and trades. Nine cells are not nine independent confirmations. A's smaller range supports further investigation of local stability; B asks for a trade-level explanation of its central spike.

Do not turn these numbers into universal pass thresholds. A flat surface can also be uniformly unprofitable. A favorable return surface may hide unacceptable drawdown, near-total exposure or very few trades.

Inspect combinations and preserve hidden dimensions

Changing one setting at a time is useful for finding obvious cliffs. It can miss interactions. A shorter lookback and a tighter stop may each look harmless alone while their combination changes both entry timing and exit frequency. Include the diagonal cells when two settings can interact.

For more than two parameters, say what every displayed cell represents. Is a third setting held fixed, averaged across its tested values or selected separately to maximize each cell? These operations answer different questions. Taking the best third-setting result for every pair creates an envelope of winners, not one deployable configuration.

The backtesting.py tutorial explicitly projects higher-dimensional results by averaging over other settings. Its heatmap makes that aggregation visible in code. Retain the underlying combinations so a smooth projection cannot conceal unfavorable individual runs. backtesting.py parameter heatmap tutorial

Mark failed, inadmissible and untested cells separately. A blank is not a zero return. If randomized scenarios affect fills or sampling, use paired scenarios where appropriate and preserve the configuration manifest described in random-seed reproducibility. Changing the random scenario at every cell can mix parameter effects with simulation noise.

Check whether the runs really share an evaluation window

A longer lookback can change when indicators become available. That can remove early trades and make the apparent sensitivity partly a change in evaluated history. The backtesting.py API warns that rolling indicators with missing warm-up values delay the backtest's start until all declared indicators are available. This is documented engine behavior, not a claim about every simulator. backtesting.py indicator warm-up warning

Declare a common tradable start after sufficient prior warm-up data for every tested setting. Record initialization and open-position handling. If identical start conditions are impossible, disclose the difference rather than silently comparing unlike paths.

For each cell, retain net return, marked-equity drawdown, trade count, exposure time, turnover and costs. Inspect the changed trades. One parameter value might skip a single large loss; another might simply produce no entries because a rounded threshold never activates. Neither behavior is explained by a heatmap color.

A stable surface can inherit an unstable assumption

Suppose every setting uses target-first fills when both stop and target lie inside the same candle. All nine cells can look stable because they share the same favorable convention. Varying lookback and stop distance does not test that convention's accuracy.

Keep that question separate: the intrabar-data guide explains which sequence questions finer candles can resolve and which remain unknown. The backtest evidence worksheet covers costs and execution assumptions. Fixing a data defect may require regenerating the entire surface.

Local stability also says little about unobserved market conditions. Compare the frozen diagnostic across declared chronological windows without changing its grid to rescue weak results. The walk-forward testing guide addresses how to keep later evaluation separate from earlier selection.

Write a finding that someone can act on

Use this worksheet before deciding what to investigate next:

A suitable finding might read: “The declared nine-cell neighborhood remains favorable, but all cells depend on unresolved same-candle exits.” It names the evidence and its limit.

If inspecting the neighborhood leads to choosing a different configuration, record that selection decision. Repeatedly expanding the grid until it looks reassuring creates additional research history. The overfitting trial ledger covers that broader selection problem; a sensitivity table cannot reconstruct discarded attempts.

Method and limits

This article is AI-assisted synthesis of primary software documentation with original hypothetical surfaces and a proposed review worksheet. The medians, ranges and neighbor count were recomputed. No crypto dataset, parameter sweep or live strategy was run. The procedure diagnoses dependence within a declared model and history; it does not certify a strategy or remove the losses described in the risk disclosure.