AI trading workflow simulation branches should be tested with fixtures that specify the starting state, controlled inputs, expected route and actions that must never occur. Compare the recorded decisions and tool calls with those expectations, then review the explanation. A correct final reply cannot compensate for an unauthorized order attempt, a skipped risk check or an incorrect waiting state.
Test the decision path before judging the explanation
A workflow can say “the trade was blocked” after it already called an order tool. It can also produce different wording while following the same permitted path. These cases need different verdicts. Treat routing, tool arguments and state changes as observable behavior; assess clarity and factual support separately.
Google's Agent Development Kit evaluation guide distinguishes evaluation of the agent's trajectory and tool use from evaluation of its final response. That distinction is useful in a trading context: whether an order was attempted matters independently of how convincingly the agent describes the result.
A branch fixture is a saved test case. It supplies enough information to force a particular decision and records what should happen next. It is more specific than “try a bearish market” or “ask the model to be cautious.” Those prompts leave the account state, permissions and allowed actions ambiguous.
Keep this exercise separate from estimating returns. A branch test asks whether the workflow obeyed its contract under chosen conditions. The broader paper-trading validation process examines observed decisions, account transitions and simulation limitations over time.
Freeze the contract and the fixture
Record the workflow version, instructions, model configuration, tool definitions and test-data version. Use the same fixture against a candidate change so the comparison has a stable reference. The model-version pinning guide explains why a model change belongs in that record rather than being treated as an invisible implementation detail.
For each fixture, save these fields:
| Field | What the fixture must make explicit |
|---|---|
| Initial state | Open positions, available strategy budget, existing orders and approval state |
| Input event | Instrument, market type, event identifier, timestamps and signal values |
| Controlled responses | Exact market-data and account-tool payloads, including deliberate failures |
| Expected route | The condition results and terminal state that should be reached |
| Required observations | Allowed calls, arguments, order count and recorded reason |
| Forbidden observations | Calls, state changes or permission expansions that must not occur |
| Test boundary | Mock harness, available simulator or integration environment used |
Reset initial state between independent cases. A previous simulated position or cached approval can otherwise decide the next route. For a deliberately sequential scenario, retain state explicitly and label the scenario as a sequence.
Controlled tool responses are test inputs, not evidence that the real venue will return those responses. If the available environment cannot inject a missing field or a rejection, record that case as untested there. A separate harness can test routing logic, but its result does not automatically validate the deployed workflow.
A worked eight-fixture entry worksheet
Consider a hypothetical workflow that proposes a 200 USDT spot entry. Its written rules require a flat position, a valid quote no more than 2,000 milliseconds old, a spread no greater than 12 basis points, at least 200 USDT of remaining strategy budget and approval of the exact proposed ticket. These are invented test thresholds, not trading recommendations or KLINEO defaults.
Unless a row changes it, assume a valid event, a 1,000-millisecond-old quote, an 8-basis-point spread, 300 USDT remaining budget and no open position. Every execution tool is replaced with a recorder that cannot send an exchange order.
| Fixture | Deliberate difference | Expected terminal state | Order-tool assertion |
|---|---|---|---|
| A: eligible, pending | Approval has not arrived | Waiting for approval of the 200 USDT ticket | Zero calls |
| B: eligible, approved | Exact current ticket is approved | Simulated submission recorded | Exactly one call for that ticket |
| C: denied | Approver rejects the ticket | Stopped with rejection reason | Zero calls |
| D: stale quote | Quote age is 2,001 milliseconds | Blocked for stale data | Zero calls |
| E: excessive spread | Spread is 12.1 basis points | Blocked for spread | Zero calls |
| F: insufficient budget | Remaining budget is 199 USDT | Blocked for budget | Zero calls; no silent downsizing |
| G: position exists | The strategy already owns a position | Skipped under the flat-only rule | Zero calls; no added exposure |
| H: invalid response | Quote payload lacks its timestamp | Blocked for invalid data | Zero calls; no invented timestamp |
The worksheet makes otherwise hidden policy choices reviewable. Fixture F forbids automatic downsizing because this example's contract specifies an exact ticket. A different strategy could permit a newly sized proposal, but its expected path would need to include the revised risk check and approval requirement.
For fixture B, inspect the instrument, side, market type, sizing unit and 200 USDT amount. A call to the right tool with a 200-unit quantity is still wrong. Verify that the recorded approval identifies that exact ticket before the simulated submission. The human-approval guide covers the approval contract; this worksheet tests the resulting branch.
Fixture H applies the chosen missing-data rule. Decide that policy before constructing the expected result, using the missing-data policy guide. Do not let the test author infer a convenient default after seeing the model's output.
Assert both required and forbidden actions
A presence check alone can miss an extra dangerous call. Amazon's AgentCore trajectory-matching documentation distinguishes exact sequence matching from in-order and any-order matching; the latter two allow additional tool calls. For a branch that forbids submission, seeing the expected data check and journal entry is insufficient if submission also appears between them.
Use explicit negative assertions such as “no execution call occurred” and “no budget or approval state was changed.” For an allowed path, require the risk decision before the order attempt. Permit harmless variation only where the contract allows it, such as the order of independent read-only observations.
Inspect attempted calls as well as successful calls. A recorder may reject an unauthorized request, protecting the test environment while revealing that the workflow still tried to perform it. That should fail the behavioral check even if simulated holdings remain unchanged.
Suppose the eight fixtures are each run three times. That is 24 hypothetical runs. Only B permits an order call, so the expected total is three permitted calls and 21 runs with none. If C makes an order attempt in all three repetitions while its final text reports rejection, the suite has three forbidden attempts. Fluent replies across all 24 runs do not remove that failure.
Keep per-fixture verdicts alongside totals. Repetition samples model variability; three successful repetitions neither establish a failure probability nor prove future reliability.
Add boundary and recovery cases deliberately
Test the stated boundaries themselves: 2,000 milliseconds and 12 basis points should pass those two gates under the example's inclusive rules; 2,001 and 12.1 should fail. Hold every other input constant so an unrelated gate cannot conceal an incorrect comparison.
Then combine failures. A stale quote plus a rejected approval should produce a trace that respects the documented precedence, preserves the rejection and never submits. A test with one fault at a time cannot establish that combined behavior.
Include a response timeout after a simulated order attempt. The expected branch should preserve uncertainty and reconcile recorded order state before any permitted next action. Do not assume that a timeout proves no order exists. The order-timeout reconciliation guide explains that execution problem; here the question is whether the workflow selects the prescribed recovery route.
KLINEO's current workflow review guide describes the available testing boundary: new-workflow controls open Live activation, while compatible historical paper and backtest records remain readable. This worksheet is a test specification, not a claim that the interface provides a fixture runner. Keep unsupported scenarios marked untested and retain the draft when no suitable simulation path is available.
Method and limits
This article is AI-assisted synthesis of linked primary agent-evaluation documentation and KLINEO's published workflow guidance. The eight-fixture contract and 24-run illustration are original hypothetical teaching examples. No workflow, model evaluation, exchange account or live order was run. Branch checks can reveal routing and authorization failures; they do not establish execution quality, strategy profitability or coverage of every possible state.