Building a Python Circuit Breaker for Async Workloads with Thresholds and Common Mistakes to Avoid

A circuit breaker decides when not to call

A Python circuit breaker temporarily stops calls to a dependency after enough relevant failures. It is not a retry counter.

Retries ask whether another attempt may recover one operation. A circuit breaker asks whether new operations should reach a dependency that already appears unhealthy.

That distinction matters during cascading failures. When a slow provider consumes every connection, worker, and retry budget, continuing to call it can spread the outage into otherwise healthy parts of the system.

A useful breaker has three states:

closed -- failures exceed threshold --> open
open -- recovery timeout expires ----> half-open
half-open -- probe succeeds ----------> closed
half-open -- probe fails -------------> open

closed allows calls and records selected failures. open rejects immediately. half-open allows a controlled probe to test whether recovery is real.

The difficult parts are failure classification, threshold design, half-open concurrency, state scope, and interaction with retries and fallbacks.

The manual example was executed with Python 3.13.5 on August 17, 2026. The RelPrim example was verified against RelPrim 0.10.0 source and documentation on the same date.

Count failures that describe dependency health

A circuit breaker should not count every exception.

A payment provider returning 400 for an invalid currency should not move the breaker toward open. Neither should an asyncio.CancelledError caused by a user disconnect.

The breaker should represent one health question, such as:

Can the primary catalog endpoint serve reads right now?

That question determines both the failure classifier and the state scope.

Thresholds depend on traffic shape

A consecutive-failure threshold is simple and works for low-volume dependencies.

open after 5 matching failures in a row

It reacts quickly, but one brief burst can open the circuit even when the dependency succeeds most of the time.

Higher-volume systems often use a rolling failure rate:

open when failure rate >= 50%
and at least 20 calls were observed
within the last 30 seconds

The minimum-throughput condition matters. One failed request should not produce a 100 percent failure rate and open a global breaker.

Open duration should be long enough to reduce pressure, but not so long that the application ignores a recovered provider. The correct value follows recovery characteristics and fallback cost, not a universal default.

Half-open concurrency is the hidden race

When the open timeout expires, many waiting callers can arrive at once. Allowing all of them through creates a recovery stampede.

The simplest strategy permits one probe. Other callers continue receiving an open-circuit response until that probe completes.

A small number of parallel probes can give stronger evidence for high-volume systems, but it requires an explicit policy:

• how many probes may run;
• how many must succeed;
• whether one failure reopens immediately;
• what happens when probes disagree.

One probe is conservative. Several probes reduce the risk that one lucky success closes the breaker too early, but add race conditions.

A small async implementation

This implementation uses an asyncio.Lock to serialize transitions and allow only one half-open probe. It injects a monotonic clock so state changes can be tested without real waiting.

from __future__ import annotations

import asyncio
import time
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from enum import StrEnum
from typing import TypeVar


T = TypeVar("T")
Clock = Callable[[], float]


class BreakerState(StrEnum):
    CLOSED = "closed"
    OPEN = "open"
    HALF_OPEN = "half_open"


class CircuitOpenError(RuntimeError):
    pass


@dataclass(slots=True)
class AsyncCircuitBreaker:
    failure_threshold: int
    open_seconds: float
    record_failure_on: tuple[type[BaseException], ...]
    clock: Clock = time.monotonic

    _state: BreakerState = field(
        default=BreakerState.CLOSED,
        init=False,
    )
    _failures: int = field(default=0, init=False)
    _opened_at: float | None = field(default=None, init=False)
    _probe_in_flight: bool = field(default=False, init=False)
    _lock: asyncio.Lock = field(
        default_factory=asyncio.Lock,
        init=False,
    )

    async def run(
        self,
        operation: Callable[[], Awaitable[T]],
    ) -> T:
        probe = await self._before_call()

        try:
            result = await operation()
        except asyncio.CancelledError:
            await self._release_cancelled_probe(probe)
            raise
        except Exception as exc:
            await self._record_failure(exc)
            raise
        else:
            await self._record_success()
            return result

    async def _before_call(self) -> bool:
        async with self._lock:
            if self._state is BreakerState.OPEN:
                opened_at = (
                    self._opened_at
                    if self._opened_at is not None
                    else self.clock()
                )

                if self.clock() - opened_at < self.open_seconds:
                    raise CircuitOpenError("Circuit breaker is open")

                self._state = BreakerState.HALF_OPEN
                self._probe_in_flight = True
                return True

            if self._state is BreakerState.HALF_OPEN:
                if self._probe_in_flight:
                    raise CircuitOpenError(
                        "Recovery probe already running"
                    )

                self._probe_in_flight = True
                return True

            return False

    async def _record_success(self) -> None:
        async with self._lock:
            self._state = BreakerState.CLOSED
            self._failures = 0
            self._opened_at = None
            self._probe_in_flight = False

    async def _record_failure(self, exc: Exception) -> None:
        async with self._lock:
            if not isinstance(exc, self.record_failure_on):
                self._probe_in_flight = False
                return

            self._failures += 1

            if (
                self._state is BreakerState.HALF_OPEN
                or self._failures >= self.failure_threshold
            ):
                self._state = BreakerState.OPEN
                self._opened_at = self.clock()
                self._probe_in_flight = False

    async def _release_cancelled_probe(self, probe: bool) -> None:
        if not probe:
            return

        async with self._lock:
            self._probe_in_flight = False

The implementation is intentionally process-local. A single breaker instance must be reused by all calls that should share health state.

Production libraries may add rolling windows, slow-call thresholds, snapshots, events, and multiple probes. State transitions must remain synchronized.

Scope is an architectural decision

A breaker that is too broad lets one failing path block healthy traffic. A breaker that is too narrow never accumulates enough evidence to protect the system.

Useful scopes include:

• per provider when one provider has one shared health profile;
• per endpoint when capabilities fail independently;
• per region when outages are geographically isolated;
• per tenant only when quotas or routing are tenant-specific.

A global breaker across unrelated providers is usually wrong. So is creating a new breaker for every request, because its state disappears before it can protect anything.

Process-local state reduces pressure from each application instance but does not create one global health view. Distributed state adds latency, coordination failure modes, and another dependency.

Local breakers are often preferable unless the system truly needs coordinated global shedding.

Retries belong inside operations, breakers between them

A useful mental model is:

logical operation
  -> breaker admission
      -> bounded retries with backoff
          -> one downstream attempt

The breaker decides whether an operation may begin. Retry handles transient failures inside that operation.

If the breaker counts every retry attempt, one user request can consume the entire failure threshold. That may be intentional, but the threshold must account for it.

Making Python Exponential Backoff Safe with Jitter, Retry Budgets and Production Trade-offs covers retry budgets and load amplification in more detail.

Fallback usually runs after the primary breaker rejects or the primary operation exhausts its policy. Give each fallback dependency its own breaker. Otherwise a failure in the primary can incorrectly poison the backup’s health state.

Using the circuit breaker in RelPrim

RelPrim exposes an async circuit breaker with closed, open, and half-open states, an async lock, one concurrent recovery probe, configurable failure types, and a state snapshot.

import httpx
from relprim import (
    CircuitBreaker,
    RetryPolicy,
    async_operation,
    fallback_chain,
)


class TemporaryCatalogError(RuntimeError):
    pass


primary_breaker = CircuitBreaker(
    name="primary_catalog",
    failure_threshold=5,
    recovery_timeout_seconds=30,
    record_failure_on=(
        httpx.ConnectError,
        httpx.ConnectTimeout,
        httpx.ReadTimeout,
        TemporaryCatalogError,
    ),
)


result = await (
    async_operation("load_product_catalog", load_primary_catalog)
    .with_circuit_breaker(primary_breaker)
    .with_retry(
        RetryPolicy(
            max_attempts=3,
            retry_on=(
                httpx.ConnectError,
                httpx.ConnectTimeout,
                httpx.ReadTimeout,
                TemporaryCatalogError,
            ),
        )
    )
    .with_fallbacks(
        fallback_chain(
            ("cached_catalog", load_cached_catalog),
        )
    )
    .run(product_id)
)

snapshot = await primary_breaker.snapshot()

print(result.value)
print(snapshot.state)

RelPrim 0.10.0 checks the breaker for each retry attempt, so matching attempt failures contribute to the threshold. Configure the threshold with that composition in mind.

An open breaker raises CircuitBreakerOpenError. Keep that error out of the retryable exception set when an open circuit should move directly to fallback instead of repeatedly checking the same breaker.

The current breaker is process-local. Share the same instance within the process for operations that need one health state.

See the RelPrim advanced usage guide for the current builder API.

Test transitions with a fake clock

A fake clock makes the open period deterministic.

import pytest


class FakeClock:
    def __init__(self) -> None:
        self.now = 0.0

    def __call__(self) -> float:
        return self.now

    def advance(self, seconds: float) -> None:
        self.now += seconds


@pytest.mark.asyncio
async def test_half_open_allows_one_recovery_probe() -> None:
    clock = FakeClock()
    breaker = AsyncCircuitBreaker(
        failure_threshold=2,
        open_seconds=10,
        record_failure_on=(ConnectionError,),
        clock=clock,
    )

    async def fail() -> str:
        raise ConnectionError("provider unavailable")

    for _ in range(2):
        with pytest.raises(ConnectionError):
            await breaker.run(fail)

    with pytest.raises(CircuitOpenError):
        await breaker.run(fail)

    clock.advance(10)

    async def recover() -> str:
        return "ok"

    assert await breaker.run(recover) == "ok"
    assert breaker._state is BreakerState.CLOSED

Also test concurrent half-open callers, ignored validation errors, cancellation during a probe, and a failed probe reopening the breaker.

Common mistakes that weaken the breaker

The most damaging mistakes are usually structural:

• counting every exception as provider failure;
• using one global breaker for unrelated dependencies;
• creating a new breaker per call;
• allowing every half-open caller through;
• combining retry layers without understanding what the threshold counts;
• closing after one weak probe in a high-volume system;
• treating open state as proof that recovery will happen;
• hiding fallback use from metrics and users.

A circuit breaker reduces exposure to a failing dependency. It does not repair that dependency or guarantee that the next probe will succeed.

Before you ship

Define the health question represented by the breaker before choosing a threshold. Scope its state to the provider, endpoint, region, or tenant that actually shares failure behavior.

Count only failures that describe dependency health. Preserve cancellation, synchronize state transitions, and limit half-open concurrency. Decide whether the threshold counts transport attempts or completed logical operations, then align retry configuration with that choice.

Record state transitions, rejected calls, probe outcomes, time spent open, failure classification, and fallback activation. A breaker that opens frequently may be protecting the system correctly, or it may be masking an unstable dependency or a threshold that is too sensitive.

A circuit breaker is a decision to stop sending work, not a promise that the dependency has recovered.

How do you scope circuit breaker state in your system, and does one failed operation count once or once per retry attempt?