Crypto trading signal precision measures how many emitted positive signals match a predefined positive outcome. Recall measures how many of all labeled positive opportunities the signal captures, including opportunities it misses. Both require a complete evaluation universe and a fixed label rule. They describe classification on the evaluated sample, not executed-trade win rate, profitability or the accuracy of future signals.
A signal log can show which alerts were right. It cannot establish recall if it contains only alerts: the missed opportunities are absent. Start with the schedule on which the rule could have made a decision, then attach both its decisions and the later labels.
Define an opportunity before finding the winners
An opportunity here is a scheduled evaluation row, whether or not a favorable move follows. It does not mean a trade that was necessarily available at an executable price.
For an original hypothetical experiment, define 100 consecutive hourly decision windows for one spot pair on one named venue. Every window has complete reference prices and a recorded signal decision. At each hourly close, freeze the signal's inputs; later, label the window positive if the next hourly close exceeds the starting close by strictly more than 0.5%. All other complete outcomes, including exactly 0.5%, are negative.
This is a chosen directional price label. It excludes fees, funding, slippage, path-dependent exits and fill availability. A positive label means that the reference-price test passed; it does not mean an actual position earned money. The starting close is a research reference, not a promised entry fill. Another experiment might label returns after a stated cost model, but that would be a different target.
Fix the venue, instrument, clock, horizon, return convention and missing-data policy before reviewing results. Selecting only hours that later moved sharply changes the universe using the answer. The look-ahead bias guide explains the separate requirement that signal inputs were available when the decision was made.
Count the four outcomes
In the invented 100-window sample, 20 windows eventually receive positive labels and 80 receive negative labels. The signal flags 20 windows. The matrix uses decisions as rows and realized labels as columns:
| Frozen signal decision | Later positive label | Later negative label | Row total |
|---|---|---|---|
| Emit positive signal | 12 true positives (TP) | 8 false positives (FP) | 20 |
| Emit no positive signal | 8 false negatives (FN) | 72 true negatives (TN) | 80 |
| Column total | 20 | 80 | 100 |
Precision asks about the first row. Recall asks about the first column. Google's confusion-matrix explanation distinguishes these four cells and their row and column totals.
Precision = TP/(TP+FP)
Recall = TP/(TP+FN)
The sample has precision 12 / 20 = 60% and recall 12 / 20 = 60%. The percentages happen to match because emitted positives and actual positives both total 20. They are different conditional fractions even when their displayed values coincide. Scikit-learn's precision and recall metric definitions specify these denominators.
Do not call FP / (TP + FP) the false-positive rate. Here that fraction is 40%: the share of emitted signals that were false. The conventional false-positive rate uses actual negatives, FP / (FP + TN), giving 8 / 80 = 10%.
Move the threshold without changing the sample
Suppose the same frozen rule assigns scores to all 100 windows. Scores are ranking values here, not claimed probabilities. Use the convention “emit when score is greater than or equal to the threshold.” A tie at the threshold is included; this must match the implementation.
The following three thresholds use the same labels and nested selections. All figures are hypothetical, constructed to expose the tradeoff.
| Score threshold | Signals | TP | FP | FN | TN | Precision | Recall |
|---|---|---|---|---|---|---|---|
| At least 0.50 | 20 | 12 | 8 | 8 | 72 | 60% | 60% |
| At least 0.70 | 10 | 8 | 2 | 12 | 78 | 80% | 40% |
| At least 0.90 | 5 | 3 | 2 | 17 | 78 | 60% | 15% |
The move from 0.50 to 0.70 removes six false signals and four true ones. Precision rises, but recall falls because twelve positive windows are now missed. Raising the threshold again removes five true positives and no false positives. Precision falls to 60% despite the stricter score.
For a fixed score vector, raising the threshold cannot add positive predictions, so recall cannot increase when actual positives exist. Precision need not rise monotonically in a finite sample: it depends on which labels disappear. Scikit-learn's precision-recall curve interface computes the pairs across thresholds and specifies its inclusive comparison.
Freeze threshold selection before the final evaluation period. Scikit-learn's threshold-tuning guidance warns against fitting a classifier and tuning its threshold on the same data. For time-dependent labels, preserve chronological validation and inspect purged label intervals; an ordinary shuffled split can share outcome information across partitions.
Keep the base rate and empty cases visible
The positive-label fraction in this sample is 20%. Emitting a signal on every window would therefore give 20% precision and 100% recall. That is an exact all-positive comparison on this dataset, not a target return or a universal benchmark. A different instrument or label threshold can change the positive fraction substantially.
Predicting no positive windows would correctly classify the 80 negative ones, yielding 80% overall accuracy while missing every positive. Google's classification metric guide explains why accuracy can conceal failures in an imbalanced dataset.
Handle empty denominators explicitly:
- No positive predictions means
TP + FP = 0: precision is undefined. - No actual positive labels means
TP + FN = 0: recall is undefined. - No eligible labeled observations means neither metric is estimable.
Report these states with their counts. A software library may replace undefined values with zero, one or a missing value under a configured convention; that replacement is not an observed success rate. Scikit-learn documents the behavior through its zero_division option in the metric reference linked above.
Separate signal quality from execution outcomes
A classifier's output can pass into risk gates, order submission and fills. Preserve those stages. An emitted signal rejected for insufficient available capital remains an emitted signal in this classifier-level matrix. If evaluating the combined decision system instead, define its final positive action before counting and retain the rejection reasons.
For example, suppose only 12 of the 20 emitted signals become completed positions, and nine finish with positive net P&L. Their executed-trade win rate is 9 / 12 = 75%. That does not alter the signal's 60% precision: the population and outcome definition differ. The trading expectancy and win-rate guide then asks how large the net wins and losses were.
Repeated alerts for the same future move can also inflate apparent evidence. Choose one row per scheduled decision or a declared event-deduplication rule, then apply it to signals and non-signals alike. Overlapping windows and simultaneous coins can share market shocks; 100 rows need not supply 100 independent observations.
Even improved precision does not establish economic value. Extra signal activity may increase portfolio turnover, while missed positives may coincide with capital already committed elsewhere. Evaluate those financial constraints separately. If the score is presented as an event probability, inspect probability calibration and Brier score; threshold counts alone cannot show that a score of 0.70 means a 70% event frequency.
Save a reproducible signal receipt
| Field | What the worked experiment records |
|---|---|
| Universe | 100 scheduled hourly windows; one venue and pair |
| Label | Next hourly reference close rises strictly more than 0.5% |
| Decisions | Frozen score and threshold; equality included |
| Counts | TP, FP, FN, TN; their sum reconciles to 100 |
| Denominators | Emitted positives, actual positives and positive fraction |
| Exceptions | Missing prices, missing scores and unresolved horizons counted separately |
| Selection | Threshold chosen before the final evaluation period |
| Execution | Risk rejection, orders, fills and completed net outcomes kept separately |
A window whose future horizon has not finished is unresolved, not negative. A missing score is not automatically an intentional no-signal decision. Declare whether such failures are outside a classifier-only analysis or count as no action in an end-to-end system assessment, and retain their totals so exclusions cannot silently improve the report.
Method and limits
This AI-assisted educational article uses primary classification documentation and an original hypothetical opportunity matrix. The threshold counts, percentages and subset relationships were checked directly. No historical crypto dataset, trained model, KLINEO evaluation or future accuracy forecast was produced. The worksheet makes a sample's classification evidence inspectable; it does not establish profitability or a stable prediction rate across market regimes.