A crypto workflow research tool circuit breaker temporarily blocks calls to a failing dependency, then admits limited recovery probes. Closed allows normal calls; open rejects new calls; half-open tests whether service has recovered. Define the failure rule, dependency scope and evidence fallback before using it. A breaker protects research capacity across requests; it does not establish that missing evidence is negative, cancel existing work or close trading positions.
Suppose a news lookup keeps timing out while candle retrieval works. Retrying the news tool on every workflow run spends capacity on the same outage. Blocking every tool removes useful candle evidence too. The design question is which calls to suppress, how to test recovery and what the remaining evidence permits.
The policy below is an original educational design. Its thresholds and timings are hypothetical, and it does not describe configurable KLINEO controls.
Give each breaker a specific job
In Martin Fowler's circuit-breaker explanation, a wrapper records failures and stops invoking the protected function after a threshold is reached. His examples also distinguish a consecutive-failure counter from a failure-frequency approach. Choose and name the rule; “three failures” alone does not identify either one.
A research breaker might protect a provider's news-search operation under one credential scope. A separate candle endpoint can remain usable. Choose the key from the actual failure boundary: provider, operation, environment and relevant quota or credential scope. Avoid recording the credential secret itself. Splitting every symbol into a separate breaker can hide an outage shared by the whole endpoint; combining unrelated providers can unnecessarily suppress healthy evidence.
This control has a narrower purpose than an account or workflow kill switch. It governs dependency admission. It grants no trading permission, cancels no orders and proves nothing about account exposure.
Write the three-state contract
Microsoft's Circuit Breaker pattern describes normal admission, immediate rejection and limited recovery traffic. It explains why restricted half-open traffic matters: flooding a recovering service can make it fail again.
| State | Research-call admission | Evidence result | Transition in this article's policy |
|---|---|---|---|
| Closed | Admit within other limits | Validate each returned response | Open when the declared window meets the failure threshold |
| Open | Reject before contacting the provider | Explicitly unavailable; no synthetic empty search | After the wait, allow a controlled probe |
| Half-open | One probe at a time; reject ordinary calls | A probe must pass the same research contract | Two consecutive valid probes close it; any counted failure reopens it |
The wait expires into permission to test, not proof of recovery. A green status page or successful static health check may not exercise the authenticated search operation that failed. For this policy, probes use a small read-only request representative of that operation and must return valid, sufficiently fresh evidence. They receive deadlines and consume request allowance.
The research timeout guide covers how long one attempt and its answer may wait. The breaker adds memory across calls. Both controls are necessary for this design: without an attempt deadline, a half-open probe could occupy its slot indefinitely.
Decide what counts before counting it
Resilience4j's official documentation distinguishes count-based and time-based windows, a minimum number of calls, and configurable recorded or ignored exceptions. Those are implementation choices, not interchangeable meanings of “failure rate.”
For the following worksheet, count completed provider attempts after response validation. Include timeouts, connection failures and provider-service errors. Count an invalid payload only when the wrapper has established that the request was valid and the response violated the expected provider contract. A local parser bug needs investigation; do not automatically diagnose a remote outage from it.
Exclude a locally rejected invalid query, caller cancellation and a breaker rejection from both the numerator and denominator. A rejected call never tested the dependency. Treat an authentication error as a separate credential/configuration stop requiring correction, rather than expecting an automatic cooldown to repair it. Handle throttling according to the provider's documented quota scope and retry instructions; do not send a recovery probe before its permitted time.
The missing-data policy remains binding. An unavailable news tool cannot support “no relevant news.” Whether the note can continue depends on whether news verification was required for the question.
Check a complete opening and recovery timeline
Assume a fictional news endpoint, serialized calls, no retries and an initially empty counter. Use the last six eligible completed attempts, require six before evaluating, and open at a failure rate of at least 50%. Wait 20 seconds after opening. In half-open, admit one probe at a time and require two consecutive successes. Reset the closed-state window after successful recovery. These are teaching choices, not recommended defaults.
Times below are seconds relative to the first completion. Ordinary-attempt rows name completion times; probe rows show admission through completion. Each listed rejection is one attempted admission.
| Time | Observation | Arithmetic or admission check | Resulting state |
|---|---|---|---|
| 0, 2, 4, 6, 8 | Success, failure, success, failure, success | Two failures in five; minimum six not reached | Closed |
| 10 | Sixth attempt fails | Three of six = 50%; threshold met | Open until 30 |
| 11, 15, 19 | Three callers request news | All rejected locally; window stays three of six | Open |
| 30–31 | First recovery probe succeeds | One consecutive valid probe | Half-open |
| 34–35 | Second recovery probe fails | Reset probe success count; wait 20 seconds from 35 | Open until 55 |
| 55–56 | New recovery probe succeeds | One consecutive valid probe | Half-open |
| 57–58 | Next recovery probe succeeds | Two consecutive valid probes; clear old window | Closed |
The two waits are 10 + 20 = 30 and 35 + 20 = 55. The three local rejections do not turn the original failure fraction into six of nine. Nor does the success at 31 combine with the success at 56: the intervening failure ended that recovery trial.
Six ordinary calls and four probes reached the provider, for ten dependency attempts. Thirteen admission requests appear in the table when the three rejected callers are included. Keep those totals separate when reporting attempted demand versus provider traffic.
At low traffic, a minimum-call rule can leave a breaker closed after several failures. That is a limitation of this fictional six-call policy, not evidence that the service is healthy. Select and test the rule against the actual request pattern rather than importing these numbers.
Protect the probe from a recovery stampede
A “one probe” rule must have an explicit ownership scope. If four workers each maintain an independent breaker, four simultaneous probes can pass. Use coordinated admission when a shared limit is required, or document the aggregate bound of independent instances. Make the state change and probe-slot claim atomic within that ownership model.
A breaker also does not recall calls admitted before it opened. Tag observations with the state generation and define how late completions are recorded. A success from an older in-flight call should not accidentally release the current half-open gate. Cancellation propagation addresses the separate work of stopping downstream scheduling and accounting for activity already started.
Amazon's guidance on retries and jitter explains how retries add load to a struggling service and how synchronized clients create bursts. For this design, every retry passes admission again; an open rejection never becomes a tight retry loop. Probe calls also remain subject to request cost budgets. A healthy dependency can still exceed the run's spending allowance.
Keep a recovery record readers can inspect
Log the breaker key, policy version, eligible-call count, counted failures, state transitions, open-until time, probe identity and fallback chosen. Record provider attempts separately from local rejections. Do not interpret suppression as improved provider reliability by silently removing failed calls from the record.
Before relying on the design, rehearse these cases:
| Fixture | Required observation |
|---|---|
| Threshold reached exactly | Open at the stated inclusive boundary |
| Many workers arrive after the wait | Only the permitted aggregate probes reach the provider |
| Probe returns quickly with malformed or stale evidence | Apply the declared validation result; do not close on transport success alone |
| Required news lookup is suppressed | Withhold the news-dependent conclusion |
| Old call succeeds after reopening | Preserve its history without overriding the current recovery trial |
Method and limits
This AI-assisted article uses the linked primary software and architecture documentation and an original state-transition worksheet. The arithmetic and call totals were checked. No provider outage, exchange account, KLINEO breaker implementation or live recovery behavior was tested. A successful recovery probe supports a limited admission decision; it cannot establish that the next answer will be accurate. Report a correction.