AI trading explanations: how to check faithfulness

A faithful AI trading explanation describes factors that actually influenced the recorded decision. Check it by linking each stated reason to saved inputs and rule results, then replay controlled changes to those reasons and unrelated details. Fluent prose, correct arithmetic and a citation can coexist with an explanation that misstates why a proposal was produced.

“The system proposed this trade because momentum and liquidity improved” sounds inspectable. It becomes useful only when the reviewer can identify the measurements, recover the proposal and test whether those factors mattered. An explanation written after a decision can be a convincing reconstruction rather than a reliable account of what produced it.

The practical aim is to find specific contradictions and missing dependencies. No short checklist can certify a model's complete internal reasoning or establish that a trading idea will make money.

Separate accuracy, faithfulness and permission

Three questions often get compressed into “does the explanation look right?” Keep them separate:

A correct price can appear in an explanation even if the decision never used it. A perfectly faithful explanation can describe a bad rule. Passing an eligibility check does not mean an order was approved, submitted or filled.

Jacovi and Goldberg distinguish faithfulness to a model's decision process from how plausible its explanation appears to people. Their methodological argument is particularly useful here: agreement from a reader is not a substitute for evaluating the relationship between explanation and model behavior. Faithfulness evaluation framework.

This concern has experimental support. Turpin and colleagues changed biasing features in prompts for GPT-3.5 and Claude 1.0, including answer ordering and suggested answers. Outputs could follow those influences while generated explanations omitted them. These were particular language-model tasks and historical models; their results do not supply a failure rate for a current trading assistant. They do show why a persuasive account needs a separate check. Research on unfaithful explanations.

Freeze the decision before asking why

Save the exact evidence bundle, model and prompt version, relevant tool responses, proposed action and explanation under one run identifier. If a deterministic rule evaluates the proposal, save its version, inputs and individual gate results too. Preserve the order: an AI proposal, a rule's rejection and an order response are different events.

Ask for a concise account of observable evidence and recorded decisions. A request for more elaborate prose or private internal reasoning does not create an audit trail. The useful objects are recoverable inputs, explicit computations, tool calls and outputs.

Link each material sentence to an object. “Volume was elevated” needs a volume measurement and baseline. “The spread check passed” needs the actual spread calculation and rule result. “Risk rejected the proposal” needs that rejection event, rather than the AI's guess about it. The data provenance guide covers preserving the source and transformation chain behind those objects.

Record which layer the explanation claims to describe. A post-processing rule may reject an AI-generated candidate for a spread violation. That trace explains the rejection. It does not, by itself, explain why the AI originally selected the candidate.

A hypothetical checker with testable reasons

Consider an invented spot-market eligibility checker. This is a teaching fixture, not captured market data, a model evaluation or a proposed profitable strategy. Its three mandatory conditions are an hourly close-to-close increase of at least 0.8%, hourly volume at least 1.5 times the preceding twenty completed hours' mean, and a quoted spread no wider than 12 basis points.

All three must pass. A failed condition makes the candidate ineligible. If none fails but required evidence is missing, the result is incomplete. Eligible means only that this checker passes; sizing, risk and execution decisions remain separate.

The fictional snapshot contains:

The hourly return is (50,500 / 50,000 − 1) × 100 = 1%. The volume ratio is 180 / 100 = 1.8. The quote midpoint is (50,490 + 50,510) / 2 = 50,500. Spread in basis points is (50,510 − 50,490) / 50,500 × 10,000, approximately 3.96. These numbers pass all three conditions.

A matching explanation would say: “The candidate passed this checker because the completed hourly return was 1%, the volume ratio was 1.8 and the quoted spread was 3.96 basis points, satisfying its three limits.” That sentence can be traced to computations and gate results.

“A bullish headline persuaded the system to enter” would misdescribe this checker. It has no headline input and no authority to enter a position. Even if a bullish headline existed, its truth would not make it a cause of the eligibility result.

Change one dependency and check the consequence

Run each test below independently from the original snapshot in a sandbox with execution disabled. Predict the result before seeing the replay. Keep the policy version fixed, and record the exact changed fields and resulting gate states.

The first intervention changes the quote midpoint consistently with the altered close while preserving its 20 USDT width. It is a controlled fixture, not a claim that real quotes would follow that path.

These results follow directly from the specified checker. If a replay keeps the candidate eligible after volume falls to 1.3 times its baseline, investigate an implementation, input-routing or version mismatch. Changing the explanation alone would hide the failure.

If the gate correctly fails but the explanation still says “all conditions passed,” the reporting layer failed. Keep both records and correct that layer. A different headline changing this checker's result would expose an undeclared dependency; it could be legitimate in some other policy, but would contradict this one.

Evidence removal asks a narrower question

The ERASER benchmark evaluates extracted textual rationales using, among other measures, comprehensiveness and sufficiency. Its classification-based tests compare output probabilities after removing the rationale or retaining only it. Those are distinct questions about whether the selected evidence influenced, or could support, a prediction. ERASER methodology, section 4.2.

For a trading review, adapt the questions carefully. Remove a claimed supporting observation and ask what changed. Then retain the claimed evidence, together with the policy and necessary context, and ask whether the stated decision remains reproducible. The checker's removal case above should produce an incomplete record because the policy explicitly requires volume. Replacing missing volume with zero would test a different input.

For a black-box model, unchanged output after removal is a warning to investigate, not conclusive proof that the explanation lied. Equivalent evidence may remain elsewhere, several factors may substitute for one another, or the changed input may be unlike anything the model normally sees. Search for duplicate summaries and cached tool results before drawing a conclusion.

Likewise, a new answer after deletion might reflect confusion from a broken input format. Preserve a valid schema and distinguish “unknown” from an invented neutral value. Do not call these adapted checks ERASER scores unless the actual required model probabilities and evaluation definitions are available. An assistant's written “80% confidence” is not automatically such a probability.

Make the audit repeatable

Use one worksheet row per claim: run identifier; exact explanation sentence; linked evidence; claimed decision layer; planned intervention; predicted effect; observed effect; and disposition. Mark each claim supported, contradicted or unresolved, with a concrete reason. A missing trace belongs in unresolved, not supported.

Replay the component being audited. To test an AI proposal, recompute it from the changed evidence before generating a fresh explanation; supplying the old proposal and asking the model to defend it tests a different task. To audit a summary of rule results, provide the authoritative trace and check whether the summary follows it. Agreement at that reporting layer cannot establish the original proposal's causes.

For a generative model, preserve available sampling settings and repeat paired baseline and intervention runs when outputs vary. Keep every result and report counts rather than choosing the most persuasive pair. A deterministic rule can be checked exactly; a small sample of model responses supports a narrower behavioral observation. Neither procedure reveals every internal computation. Turpin and colleagues also explicitly describe their counterfactual consistency tests as capable of finding failures without proving full faithfulness. Study limitations.

Approval should use this evidence before the reviewer reads a polished conclusion. The automation bias guide develops that review order. After a loss, keep the same audit standard: precommitment rules for revenge trading help prevent changing evidence requirements to justify a desired response.

Finally, return to the actual proposal. The AI trade analysis review covers instrument identity, sizing, costs and the separate execution decision. A faithful account of an eligibility result settles none of those automatically.

For the next explanation you review, choose its strongest causal sentence, identify the saved evidence it names and write down one change that should alter the recorded decision. If the trace cannot support that test, record exactly what remains unknown before letting the explanation influence an action.