AI trading context window evidence loss occurs when information needed for a decision is omitted from the model's input, removed during conversation management or lost in a summary. Prevent it by defining required evidence, inspecting the final submitted input and checking that material facts survive compression. A large context window or a confident answer does not establish that every required source was considered correctly.
A workflow can retrieve an exchange notice successfully and still produce an answer that lacks its exception. The notice may never reach the final request, or its summary may preserve “order management available” while dropping “new orders paused.” Review the evidence that reached this particular analysis, rather than the documents available somewhere in the system.
Separate three places where evidence can disappear
Before submission: a retrieval step returns a document, but a size cap, relevance filter or history policy excludes it. The retrieval log proves acquisition; it does not prove inclusion in the model request. Over-limit behavior depends on the API and application: a request may fail, or the application may shorten it. Do not assume every service silently drops the oldest text.
During compression: a retained summary replaces longer evidence. Anthropic's compaction documentation describes replacing older conversation turns with a server-written summary. That documents a context-management mechanism, not a guarantee that a trading exception will survive. Check the replacement against facts the decision requires.
During interpretation: the necessary passage remains in the input, but the answer overlooks or misuses it. This is not literal truncation. The Lost in the Middle research paper found position-dependent performance on multi-document question answering and key-value retrieval, including weaker use of information in the middle. Its tested models and tasks do not establish a failure rate for today's model or your trading workflow. They justify testing placement instead of equating presence with correct use.
Google's Gemini long-context guide also distinguishes single-item retrieval from retrieving multiple specific pieces of information, whose accuracy can vary with context. Neither source establishes trading performance.
Budget the final request, not the source folder
Start with the actual endpoint's accounting rules. Instructions, conversation history, tool definitions, tool results, attachments and generated output can consume different allowances. For Claude, its context-window documentation explicitly includes system prompts, messages, tool definitions and output. Other model families can account for input, output and thinking differently.
The following is a hypothetical shared 16,000-token request ceiling, chosen for arithmetic rather than taken from a model specification:
| Allocation | Illustrative tokens |
|---|---|
| Instructions and tool definitions | 2,000 |
| Retained conversation | 1,500 |
| Reserved additional tool results | 1,000 |
| Reserved output | 2,000 |
| Safety margin | 500 |
| Evidence available to pack | 9,000 |
| Total | 16,000 |
Count the serialized request with the provider's supported method, including wrappers around evidence. Repeat after new tool results or compression. A word count is not a token count, and a reserve cannot prevent an unexpected result from exceeding its allocation. Define what happens when it does: shorten optional material, retrieve a bounded excerpt or stop the affected analysis.
Work through a pack that fits but misses a condition
Suppose a fictional perpetual-market review requires five evidence objects, E1–E5. Two other objects are optional background. Every token count and policy in this exercise is invented.
A naive packing rule visits the rows in order, retains a whole object if it fits and skips it otherwise. With 9,000 tokens available, E1–E4 use 6,000. E5 does not fit in the remaining 3,000, so it is skipped. E6 fits; E7 does not. The final evidence uses 8,000 tokens, but includes only four of five required objects.
| ID and role | Raw tokens | Naive pack | Deliberate compact pack |
|---|---|---|---|
| E1 completed-candle snapshot, required | 1,500 | Included | 700 |
| E2 account exposure and reservations, required | 1,000 | Included | 500 |
| E3 instrument restrictions, required | 2,500 | Included | 1,800 |
| E4 booked funding ledger, required | 1,000 | Included | 600 |
| E5 venue incident notice, required | 3,500 | Omitted | 2,400 |
| E6 recent news background, optional | 2,000 | Included | 1,500 |
| E7 older research notes, optional | 2,500 | Omitted | 500 |
| Total | 14,000 | 8,000 tokens | 8,000 tokens |
Both packs use the same space. Required-object coverage changes from 4/5, or 80%, to 5/5, or 100%. Overall object coverage in the naive pack is 5/7, about 71.4%. Neither percentage is a model-accuracy score. Required coverage is a completeness check under the review's declared scope; optional documents must not compensate for a missing required one. If there are no required objects, report the metric as not applicable and question whether the scope was defined, rather than dividing by zero.
Now inspect what E5 retained. The fictional notice says cancellations remain available while new orders are paused for a named market and interval. A summary that retains its source ID but says only “order management available” passes an ID-presence check and loses the decisive condition. The compact pack is acceptable only after verifying that the exception, affected market, effective interval and source locator all survive.
This example does not prescribe compression ratios. Some passages cannot be reduced safely enough to fit. Remove optional material first; if required evidence still cannot fit, divide the research into bounded questions or return an incomplete assessment. Any combined answer still needs its own coverage check.
Keep a fact checklist beside the evidence manifest
The manifest should identify each evidence object's source, captured version, observation or effective time, instrument scope, required status and submitted location. Retain the original object separately. The data-provenance guide covers that source lineage; this worksheet asks whether its relevant content survived into the request.
For each material conclusion, record the supporting passage and any exception that can reverse it. A reusable review card asks:
- Which required objects were retrieved, submitted, summarized or omitted, and why?
- Which exact fields or clauses must survive for this conclusion?
- Does each cited locator resolve to the captured version actually submitted?
- Did the answer retain qualifications, units, scope and effective times?
- Which claim must be withheld if one of those checks fails?
A citation is useful evidence of a reference, but its presence does not establish correct interpretation. Compare the answer with the captured passage using claim-by-claim hallucination checks. Do not let the model's own statement that it “reviewed all sources” replace this check.
If two surviving passages disagree, preserve both and apply the separate contradictory-source review. Shortening away one side creates apparent agreement without resolving the conflict.
Test loss deliberately and preserve the result
Create a fixed fixture containing an exception such as the fictional pause above. Move it between the beginning, middle and end of the evidence; add irrelevant history; then trigger the application's actual truncation or summarization path. Check whether the final request retains the clause and whether the answer correctly limits its conclusion. Record multiple runs when generation varies; one successful answer does not establish reliable coverage.
Also remove E5 deliberately. The expected result is “incident status unverified,” with dependent claims withheld. The missing-data policy guide explains why absence cannot become permission. Keep current execution eligibility in independent checks; a well-packed research answer cannot establish that conditions remain unchanged later.
Save the submitted request, evidence manifest, summaries, omissions and output. Use deterministic replay to inspect which inputs a historical run actually consumed. A later run with restored evidence is a different record, even if its prose looks similar.
Prepared with AI assistance under the editorial policy, this article proposes an original token-budget and coverage worksheet. It does not describe a verified KLINEO packing feature, test a live model or establish a safe trading outcome.