Crypto backtest data quality: audit six candle slots

Crypto backtest data quality begins with a reproducible input contract: venue, instrument, market type, interval boundaries, timestamp units and a policy for missing, duplicate or unfinished records. Validate those inputs before interpreting strategy performance. A file checksum identifies the bytes used; it does not establish that the candles are complete, causal or executable, and a high row count can conceal missing intervals.

A useful first audit is small enough to inspect by hand. Six expected one-minute candles can expose several mistakes that disappear inside a multi-year return chart.

Declare the interval before counting the rows

Define the instrument identity and the time range explicitly. Include the exchange, base and quote assets, spot or derivative market, interval, timezone and timestamp convention. Use a half-open range when appropriate: include the start and exclude the end. Then derive expected intervals only for periods when the instrument was within the declared trading scope.

Binance identifies spot klines by their opening time. Its documentation also distinguishes the timezone used to interpret candle intervals from the UTC interpretation of start and end request timestamps. These details matter when joining archives or reconstructing a requested range. Binance spot candlestick documentation

A daily candle at UTC midnight and a daily candle aligned to another timezone can aggregate different trades. Matching the displayed date does not make their open, high, low and close comparable. Save the interval definition alongside the source file rather than relying on a column name such as “date.”

Historical eligibility is another boundary. Do not expect pre-listing candles or remove delisted instruments merely because the modern downloader cannot enumerate them. The historical-universe guide separates missing price coverage from retrospective asset selection.

An eight-row file that supplies only two accepted candles

Here is an original hypothetical raw fixture. The intended range is 10:00 UTC inclusive through 10:06 exclusive, with six finalized one-minute candles expected for one invented instrument. Assume all six intervals are eligible under the exercise's stated universe. Prices and volumes are invented and carry no market evidence.

Eight raw rows exceed the expected count of six, but the file is incomplete. Row 8 lies outside the requested range. Rows 2 and 3 are identical duplicates. Rows 4 and 5 conflict at one timestamp. No row covers 10:03. Row 6 has a high below its open and close. Row 7 has negative volume.

Under this exercise's explicit policy, remove the out-of-range row, collapse the identical duplicate, quarantine both conflicting rows and reject the two invalid rows pending investigation. Only 10:00 and 10:01 remain accepted. Accepted unique coverage is 2 / 6 = 33.33%, rounded.

Within the seven in-range raw rows, two are accepted, one is an identical duplicate removed and four are quarantined. That accounting reconciles the raw rows without claiming that quarantine repaired the missing data. Timestamp presence is five of six expected slots; validated coverage is two of six. Those are different measurements.

Validate keys and values separately

Use a complete key containing venue, instrument, market, interval and normalized opening time. A timestamp alone is not unique across several coins or venues. Identify identical duplicates separately from conflicting duplicates, and preserve the original source provenance for both.

A deterministic “last row wins” rule can make a pipeline repeatable while silently choosing between conflicting values. If the later row is an explicit provider correction, retain that fact and its retrieval time. If the conflict has no explanation, quarantine or report it instead of treating file order as evidence of correctness.

For a conventional positive-price OHLCV series, check finite numeric values, a high at least as large as open and close, a low no larger than open and close, high no lower than low, and nonnegative volume. Apply the schema appropriate to the product; a generic price rule should not be copied unexamined to every derivative or transformed series.

These invariants catch malformed records. They do not prove the recorded prices were accurate. A candle with an incorrect but internally consistent high can pass every simple range check. Compare suspicious observations with authoritative source records where available and record the resolution.

Normalize timestamp units from the source contract

A timestamp parsed in the wrong unit can move observations by orders of magnitude. Avoid relying solely on a guessed digit count. Use the documented schema, retain the original value and verify the normalized result against a known interval boundary.

Binance's public-data repository states that spot archive timestamps from January 1, 2025 onward use microseconds. Its archive documentation also supplies checksums and explains that archived files can later be replaced to correct issues. A downloader spanning schema periods must preserve those distinctions. Binance public-data format and integrity notes

For a simple verification, the chosen values 1,735,689,600,000 milliseconds and 1,735,689,600,000,000 microseconds both represent January 1, 2025 at 00:00:00 UTC. Dividing by 1,000 or 1,000,000 respectively gives the same Unix-seconds value, 1,735,689,600. This arithmetic check does not replace checking every input file's actual declared unit.

After normalization, sort by the complete key and check alignment to the specified interval. Inspect daylight-saving conversions if local display times enter the pipeline. A local display label should not silently alter the underlying UTC sequence used by the strategy.

Investigate gaps before manufacturing flat candles

A missing interval can reflect no trading, a collection failure, a source limitation or instrument availability. The interpretation depends on the provider. Coinbase's candle endpoint explicitly warns that historical data may be incomplete and that intervals without ticks do not receive published data. An absent candle from that endpoint is therefore not automatically proof of a downloader outage. Coinbase Exchange product candles

Forward-filling the previous close with zero volume creates a synthetic record. That may be an explicit representation for a documented no-trade interval, but it cannot recover an unknown intrabar range or prove that an order would fill. If an outage hid a large movement, a flat synthetic candle can change triggers, volatility estimates and modeled exits.

Keep a flag distinguishing source candles from constructed ones. Declare how each strategy component handles a flagged record. One component may skip evaluation, another may retain a stale indicator state, and another may halt the run. The chosen behavior changes the tested strategy and belongs in its configuration.

For unresolved gaps, run declared sensitivity cases only when they answer a useful question. Do not invent high and low values and label the resulting return a historical reconstruction. If the missing interval can determine a trade's outcome, retain that outcome as unresolved.

Separate finalized archives from received data

The latest REST candle may still be developing. A complete archive downloaded later can contain a final close that was unavailable when the live decision occurred. Validate finalization and availability separately from structural data quality.

The look-ahead audit explains how an accurate finalized value can still leak into an earlier decision. For a walk-forward evaluation, also check whether forward labels cross model-freeze boundaries. Passing an OHLC range check answers neither timing question.

Store the raw response, retrieval time, requested parameters, schema version, normalization code version and checksums. Preserve corrected downloads alongside earlier versions when practical. A later rerun should identify whether changed results came from revised data or revised strategy logic.

For a forward comparison, keep the observations actually received at the time. Replacing them silently with a later archive prevents meaningful diagnosis of paper-trading signal mismatches.

Close the audit with a bounded claim

Publish expected intervals, unique observed intervals, accepted intervals, identical duplicates, conflicts, invalid rows, missing slots and provisional or constructed records. State how those counts overlap. For example, one missing slot has no raw row to include in the rejected-row total.

Then inspect whether any rejected or missing interval affects entries, exits, warmup or valuation. A strategy that requires 30 accepted consecutive candles may remain unready even after one missing candle has passed. Preserve state and warmup effects instead of deleting the gap and treating nonadjacent records as consecutive minutes.

KLINEO's backtest methodology describes the scope of its published source and candle-quality checks. Apply the same evidence discipline to the exact input used by a particular run. A clean fixture validates a checking rule; it does not certify all market history or the resulting strategy.

Method and limits

This article was prepared with AI assistance using linked primary exchange documentation. The eight-row fixture, reconciliation policy and coverage calculation are original educational constructions. Counts, OHLC invariants and timestamp conversions were independently checked. No exchange archive was downloaded or KLINEO market dataset certified for this article. Complete, well-formed data still cannot remove modeling, execution or the broader risks of trading.