Check AI trading analysis for hallucinations by splitting it into individual claims, matching each factual claim to a source with the same market and time, and independently recalculating derived numbers. Mark missing evidence as unverified and forecasts as inference. A real citation, repeated answer or confident tone does not establish that a claim is correct.
The useful question is not whether an entire paragraph sounds trustworthy. It is whether each premise that could change a decision survives a specific check. One invented indicator value can hide inside an otherwise accurate explanation.
This guide provides a reusable claim worksheet and a fictional example. It checks evidence quality; it does not identify a profitable strategy or recommend a trade.
First separate errors from predictions
NIST describes confabulation as confidently presented erroneous or false content, including outputs that contradict their input. Its Generative AI Profile also recommends reviewing and verifying sources and citations in generated outputs. That gives a practical starting point: inspect the support for a statement instead of judging its writing style. NIST Generative AI Profile, section 2.2 and action MS-2.5-003.
Different statements require different checks:
| Claim type | Example | Appropriate check |
|---|---|---|
| Observed fact | “The completed hourly close was 100 USDT.” | Retrieve the matching venue, symbol, interval and candle. |
| Derived quantity | “The close increased 5%.” | Recalculate from the stated numerator and denominator. |
| Attributed statement | “The exchange announced maintenance.” | Open the original announcement and check its scope and date. |
| Interpretation or forecast | “Buyers are in control, so price will rise.” | Identify assumptions and a falsifiable condition; do not treat it as an observed fact. |
A forecast can fail without having been fabricated. Conversely, a forecast that happens to succeed does not make an invented input true. Review the evidence available at the decision time, separately from what the market did later.
The joint SEC, NASAA and FINRA investor alert warns that AI-generated investment information can be inaccurate or incomplete, and can produce faulty output even with accurate input. The alert addresses investment information broadly; it is not an assessment of a particular crypto tool. Investor.gov alert on AI and investment fraud.
Give every material claim a row
Copy the exact claim before rewriting it. A later correction should not erase the original error. Then record the following fields in a note or spreadsheet:
| Worksheet field | What to save |
|---|---|
| Claim and type | One checkable assertion, classified using the table above. |
| Scope | Venue, exact pair or contract, spot or derivatives, interval, timezone and units. |
| Evidence locator | Original URL or saved response, plus the exact passage or field that supports the claim. |
| Clock | Observation time, retrieval time and analysis time. |
| Check method | Direct comparison, independent calculation, or an explicit statement that evidence is unavailable. |
| Verdict and consequence | Supported for this scope, contradicted, unverified, or inference; then retain, correct, remove or hold. |
Prioritize claims that affect whether a condition is satisfied, position size, order permissions or account state. A misspelled asset name may be editorial; confusing two assets with similar names can change the instrument being traded.
“Supported” is deliberately narrow. A source can support a historical observation without supporting a prediction built on it. The market-data freshness guide explains the clock checks in more detail. For announcement authenticity and repeated reporting, use the crypto news verification guide.
Work through a flawed AI paragraph
All figures and the proposed order below are hypothetical. They are a teaching exercise, not a market capture. Assume a fictional spot pair quoted in USDT, with a completed hourly close of 100, preceding hourly close of 96, and completed-hour volume of 150 base units. The preceding 20 completed hours have mean volume of 100 base units. No RSI history is provided.
An AI draft says:
“The hourly close is 100, up 5%. Volume is three times its recent average. RSI is 72, confirming another rise. Buying 40 units requires only 400 USDT.”
Audit each assertion separately:
| Assertion | Independent check | Result |
|---|---|---|
| Hourly close is 100 | Compare with the supplied completed candle in the stated scope. | Supported within the fictional snapshot. |
| Price rose 5% | (100 − 96) ÷ 96 × 100 = approximately 4.17%. | Contradicted; correct the number. |
| Volume is three times average | 150 ÷ 100 = 1.5 times average. | Contradicted; it is 50% above average. |
| RSI is 72 | Required price history and calculation settings are absent. | Unverified; do not invent a replacement value. |
| Another rise is confirmed | No observation can establish the next price move as certain. | Forecast presented too strongly; identify it as inference. |
| Forty units require 400 USDT | 40 × 100 = 4,000 USDT before fees. | Contradicted; the draft understates notional tenfold. |
The corrected factual note contains the supplied close, approximately 4.17% change, volume of 1.5 times the stated average, and 4,000 USDT notional before fees. It omits RSI until the needed evidence exists. This correction still provides no reason to approve the proposed order.
If the analysis produces a machine-readable proposal, structured-output validation separately checks its contract, units, instrument rules and current account constraints. Correct prose does not settle those checks.
Use a calculator or deterministic calculation for arithmetic. Asking the same model to “double-check” can help find candidates for review, but its second explanation is not the independent computation. Whether an error is labeled hallucination, unit confusion or arithmetic failure, a material mismatch needs resolution before the result can support a decision.
A working link is only the first citation check
Open the source and locate the actual support. Check that the publisher is relevant, the cited page says what the AI claims, and the statement applies to the same instrument and period. A documentation page about an endpoint does not prove a current market value returned by that endpoint.
For example, Binance's Spot REST documentation describes its 24-hour ticker as rolling-window statistics. It separately documents candlestick requests by symbol and interval, with open and close times in the response. A genuine 24-hour ticker response does not establish the close of a particular hourly candle. These are Binance data definitions, not a statement about KLINEO's available integrations. Binance Spot market-data documentation.
If a cited page disappears, preserve the claim as unverified while seeking an appropriate replacement. Absence of accessible evidence does not prove the statement false. If sources disagree, record the disagreement and inspect their definitions, timestamps and origin rather than counting links.
Check what happens when evidence is missing
A realistic failure case is a partial data response: the close arrives, but the indicator's required history does not. The assistant may retain an RSI value from an earlier analysis or fill the gap with a plausible number. A polished paragraph can hide that the data conditions changed.
In a controlled review or paper exercise, remove a required input and inspect whether the output identifies the missing field. Also try a mismatched symbol or an outdated observation. Expected behavior should be written beforehand: flag the mismatch and withhold the affected conclusion. Passing a few examples is evidence about those examples, not a measured hallucination rate for future use.
A confidence score does not resolve this gap. The AI confidence-score guide explains why a score needs a defined target and calibration evidence. Two assistants agreeing on the same unsupported premise also leave that premise unsupported.
Close the review with a usable decision record
Retain the original output, evidence, calculations, unresolved claims and corrected version together. Record which material claims remain unverified and what evidence would resolve them. A useful outcome can be “insufficient evidence.”
Repeated checks also create a history for model-drift monitoring. A change in outputs or missing-input frequency needs comparison with a declared baseline; it cannot be diagnosed from one flawed paragraph alone.
Only after factual checks should a separate review consider whether an idea fits the account, costs and execution constraints. The AI trade-analysis review guide covers that broader decision. Correct facts are a necessary input to that review, not proof that a trade should be made.
Editorial method: prepared with AI assistance using the primary sources cited above. The worksheet and numerical case are original educational constructions, not a live trading record, measured model evaluation or personalized advice. Report a correction.