Skip to content
Bart Rozycki
RelPrim docsCircuit breaker

Examples

Example: circuit breaker

A complete, runnable example of a circuit breaker with a fallback. A failing dependency opens the breaker, later calls skip it and use a cached value, and a probe closes it again after recovery.

When a dependency is down, calling it on every request wastes time and adds load to a system that is trying to recover. This example uses a circuit breaker to stop calling a failing service, serves a cached value meanwhile, and lets one probe call check for recovery.

The problem

A product page shows the stock level from an inventory service. During an outage every request waits for the inventory call to fail before showing anything, and the service receives a steady stream of calls it cannot answer. A slightly stale cached value would be acceptable for a while.

Implementation

The demo inventory service fails until it is marked healthy. A FakeClock replaces the breaker’s monotonic clock so the example can move time forward instead of waiting 30 seconds.

import asyncio

from relprim import CircuitBreaker, async_operation, fallback_chain


class InventoryUnavailableError(Exception):
    pass


class DemoInventoryService:
    def __init__(self) -> None:
        self.calls = 0
        self.healthy = False

    async def stock_level(self, sku: str) -> int:
        self.calls += 1
        if not self.healthy:
            raise InventoryUnavailableError("inventory service is down")
        return 7


async def cached_stock_level(sku: str) -> int:
    return 5  # last known value, possibly stale


class FakeClock:
    """Moves time forward on demand instead of waiting 30 seconds."""

    def __init__(self) -> None:
        self.now = 0.0

    def __call__(self) -> float:
        return self.now


async def main() -> None:
    inventory = DemoInventoryService()
    clock = FakeClock()
    breaker = CircuitBreaker(
        name="inventory",
        failure_threshold=3,
        recovery_timeout_seconds=30,
        record_failure_on=(InventoryUnavailableError,),
        clock=clock,
    )
    stock_level = (
        async_operation("stock_level", inventory.stock_level)
        .with_circuit_breaker(breaker)
        .with_fallbacks(fallback_chain(("cached_stock", cached_stock_level)))
    )

    async def request(label: str) -> None:
        result = await stock_level.run("SKU-42")
        primary = result.report.attempts[0]
        outcome = primary.error.type if primary.error else "succeeded"
        state = (await breaker.snapshot()).state
        print(
            f"{label}: value={result.value} primary={outcome} "
            f"breaker={state} inventory calls={inventory.calls}"
        )

    for number in range(1, 6):
        await request(f"request {number}")

    clock.now += 30  # the recovery timeout has passed
    inventory.healthy = True
    await request("request 6")


asyncio.run(main())

It prints:

request 1: value=5 primary=InventoryUnavailableError breaker=closed inventory calls=1
request 2: value=5 primary=InventoryUnavailableError breaker=closed inventory calls=2
request 3: value=5 primary=InventoryUnavailableError breaker=open inventory calls=3
request 4: value=5 primary=CircuitBreakerOpenError breaker=open inventory calls=3
request 5: value=5 primary=CircuitBreakerOpenError breaker=open inventory calls=3
request 6: value=7 primary=succeeded breaker=closed inventory calls=4

What happens

  1. Requests 1–3 call the inventory service, which fails. Each failure matches record_failure_on and is counted. The cached value is served by the fallback. The third failure reaches failure_threshold=3, and the breaker opens.
  2. Requests 4–5 do not call the inventory service at all: the call count stays at 3. The open breaker rejects the attempt immediately with CircuitBreakerOpenError, and the fallback serves the cached value.
  3. After recovery_timeout_seconds=30, the breaker lets one call through as a probe (the half-open state). Request 6 is that probe. The service has recovered, so the probe succeeds and the breaker closes. If it had failed, the breaker would open again for another recovery timeout.

The breaker state is shared: reuse one CircuitBreaker for every call to the same dependency. breaker.snapshot() returns its state, failure count and open time. On a rejected attempt the attempt metadata has circuit_breaker_open=True; the report-level key of the same name describes only the final attempt, which here is the fallback.

Why this matters

A retry asks “should this call try again?”. A circuit breaker asks “should anyone call this dependency right now?”. Retries help with brief, independent failures; a breaker turns a sustained outage into fast, cheap rejections across all callers, and gives the dependency room to recover. Combined with a fallback, users see a degraded answer instead of an error or a long wait.

Production note

Choose record_failure_on carefully: only failures that indicate an unhealthy dependency should open the breaker. A 404 for a missing product is not an outage. Breaker state lives in the process, so each instance of a service keeps its own breaker. Leave CircuitBreakerOpenError out of retry_on: retrying a rejected call immediately is rejected again.