Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion packages/cache/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,21 @@ echo $data;
| `Hazelcast` | Hazelcast over its Memcached protocol. |
| `Sharding` | Spreads keys across several adapters. |
| `Pool` | Checks an adapter out of a `utopia-php/pools` pool per call. |
| `CircuitBreaker` | Wraps an adapter so a failing cache stops being called. |
| `CircuitBreaker` | Wraps an adapter so a failing cache stops being read — see [what an open circuit sheds](#what-an-open-circuit-sheds). |

## What an open circuit sheds

`CircuitBreaker` sheds reads while its circuit is open, and still lets writes through.

Shedding reads is the point: the dependency is sick, the read would fail anyway, and refusing it early costs nothing. A write is not the same thing, because a cache write is the repair. Refusing it holds the miss rate at 100% for as long as the circuit stays open, so the traffic the breaker diverted keeps arriving at whatever it was diverted to well after the cache itself is healthy. A cache cannot warm up while it is forbidden to remember anything.

So `save()`, `saveWithLease()` and `touch()` go to the adapter even while the circuit is open, and return their usual fallback if they fail. `load()`, `list()`, `getSize()` and `ping()` are shed as before.

Two details keep that from costing anything:

**The write bypasses the breaker while open, rather than reporting to it.** The verdict is already made, so one more data point cannot change it, and what decides when the circuit closes should be the probes half-open schedules rather than repair traffic arriving at whatever rate the fallback path happens to generate. While the circuit is *not* open, a write reports normally, so a failing cache can still open the circuit.

**One failed repair ends the attempts for that open episode.** A repair is worth one timeout to learn whether the cache accepts writes and worth nothing after that: against an adapter that is not answering, retrying on every request would add its timeout to every request, which is the cost an open circuit exists to avoid. The first failure suppresses the rest of the episode, and the guard clears as soon as the circuit is no longer open — so a cache that refused writes is tried again next time, not written off for the life of the process.

## System requirements

Expand Down
61 changes: 58 additions & 3 deletions packages/cache/src/Cache/Adapter/CircuitBreaker.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,18 @@

class CircuitBreaker implements Adapter, Feature\Leasable, Feature\Telemetry
{
/**
* Whether this open episode has already shown the cache cannot be written.
*
* A repair attempt is worth one timeout to find out, and worth nothing after
* that: if the adapter is not answering at all, retrying the write on every
* request would add its timeout to every request, which is the cost an open
* circuit exists to avoid. One attempt per episode keeps the repair where it
* pays — a cache that is already well again, behind a circuit that has not
* closed yet — and drops it where it does not.
*/
private bool $repairUnavailable = false;

public function __construct(
private readonly Adapter $adapter,
private readonly UtopiaCircuitBreaker $breaker,
Expand All @@ -34,10 +46,53 @@ public function load(string $key, int $ttl, string $hash = ''): mixed
return $this->delegate(__FUNCTION__, \func_get_args(), false);
}

/**
* Route a write, which an open circuit must not refuse outright.
*
* Shedding reads while the dependency is sick is what a breaker is for: the
* read would fail anyway, so refusing it early costs nothing. A write is not
* the same thing — it is the repair. Refusing it holds the miss rate at 100%
* for as long as the circuit stays open, so the traffic the breaker diverted
* keeps arriving at whatever it was diverted to well after the cache itself is
* healthy. The cache cannot warm up while it is forbidden to remember
* anything.
*
* While open the write bypasses the breaker rather than reporting to it. The
* verdict is already made, so one more data point cannot change it, and what
* decides when the circuit closes should be the probes half-open schedules,
* not repair traffic arriving at whatever rate the fallback path happens to
* generate.
*
* @param array<mixed> $args
*/
private function repair(string $method, array $args, mixed $fallback): mixed
{
if (! $this->breaker->isOpen()) {
// Not open: the write reports to the breaker like any other call, so a
// failing cache can still open the circuit. This also clears the guard
// below, scoping it to a single open episode.
$this->repairUnavailable = false;

return $this->delegate($method, $args, $fallback);
}

if ($this->repairUnavailable) {
return $fallback;
}

try {
return $this->adapter->{$method}(...$args);
} catch (\Throwable) {
$this->repairUnavailable = true;

return $fallback;
}
}

public function save(string $key, array|string $data, string $hash = ''): bool|string|array
{
/** @var bool|string|array<int|string, mixed> $result */
$result = $this->delegate(__FUNCTION__, \func_get_args(), false);
$result = $this->repair(__FUNCTION__, \func_get_args(), false);

return $result;
}
Expand All @@ -61,15 +116,15 @@ public function saveWithLease(string $key, array|string $data, string $hash, str
}

/** @var bool|string|array<int|string, mixed> $result */
$result = $this->delegate(__FUNCTION__, \func_get_args(), false);
$result = $this->repair(__FUNCTION__, \func_get_args(), false);

return $result;
}

public function touch(string $key, string $hash = ''): bool
{
/** @var bool $result */
$result = $this->delegate(__FUNCTION__, \func_get_args(), false);
$result = $this->repair(__FUNCTION__, \func_get_args(), false);

return $result;
}
Expand Down
158 changes: 158 additions & 0 deletions packages/cache/tests/Cache/Unit/RepairWhileOpenTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
<?php

declare(strict_types=1);

namespace Utopia\Tests\Unit;

use PHPUnit\Framework\TestCase;
use Utopia\Cache\Adapter\CircuitBreaker;
use Utopia\Cache\Adapter\Memory;
use Utopia\CircuitBreaker\CircuitBreaker as UtopiaCircuitBreaker;

/**
* An open circuit sheds reads and still lets the cache repair itself.
*
* Shedding reads while the dependency is sick is what a breaker is for — the read
* would fail anyway, so refusing it early costs nothing. A write is not the same
* thing, because a cache write is the repair. Refusing it holds the miss rate at
* 100% for as long as the circuit stays open, so the traffic the breaker diverted
* keeps arriving at whatever it was diverted to well after the cache is healthy.
*/
final class RepairWhileOpenTest extends TestCase
{
private function openBreaker(): UtopiaCircuitBreaker
{
$breaker = new UtopiaCircuitBreaker(timeout: 30);
$breaker->trip();

return $breaker;
}

public function testAnOpenCircuitStillWritesThrough(): void
{
$adapter = new Memory();
$cache = new CircuitBreaker($adapter, $this->openBreaker());

$this->assertSame('value', $cache->save('key', 'value'));
$this->assertSame('value', $adapter->load('key', 60), 'The write reached the adapter.');
}

public function testReadsAreStillShedWhileOpen(): void
{
$adapter = new Memory();
$adapter->save('key', 'value');

$cache = new CircuitBreaker($adapter, $this->openBreaker());

// Repairing is not reopening. The point of the breaker is to stop asking a
// sick dependency questions it cannot answer.
$this->assertFalse($cache->load('key', 60));
}

public function testTouchIsTreatedAsAWrite(): void
{
$adapter = new Memory();
$adapter->save('key', 'value');

$cache = new CircuitBreaker($adapter, $this->openBreaker());

$this->assertTrue($cache->touch('key'));
}

public function testAFailingWriteStillReturnsTheFallback(): void
{
$cache = new CircuitBreaker(new FailingAdapter(), $this->openBreaker());

// Attempting the repair must not turn a dead dependency into an exception
// the caller did not have to handle before.
$this->assertFalse($cache->save('key', 'value'));
}

/**
* The bound that makes this safe as the only behaviour.
*
* A repair is worth one timeout to find out whether the cache is writable, and
* worth nothing after that. If every request retried the write against a cache
* that is not answering, each would pay that timeout on top of the fallback it
* was already paying — which is the cost an open circuit exists to avoid.
*/
public function testOneFailedRepairStopsTheRestOfTheEpisodeTrying(): void
{
$adapter = new CountingFailingWriteAdapter();
$cache = new CircuitBreaker($adapter, $this->openBreaker());

for ($i = 0; $i < 5; $i++) {
$this->assertFalse($cache->save('key', 'value'));
}

$this->assertSame(1, $adapter->saves, 'Only the first repair attempt reaches a cache that refused one.');
}

public function testAHealthyCacheKeepsBeingRepaired(): void
{
$adapter = new Memory();
$cache = new CircuitBreaker($adapter, $this->openBreaker());

// Nothing has failed, so nothing is suppressed.
$this->assertSame('one', $cache->save('a', 'one'));
$this->assertSame('two', $cache->save('b', 'two'));

$this->assertSame('one', $adapter->load('a', 60));
$this->assertSame('two', $adapter->load('b', 60));
}

/**
* The guard is scoped to one open episode, so a cache that was unwritable
* while the circuit was open is tried again next time rather than being
* written off for the life of the process.
*/
public function testTheGuardClearsWhenTheCircuitIsNotOpen(): void
{
$adapter = new CountingFailingWriteAdapter();
$breaker = new UtopiaCircuitBreaker(threshold: 1, timeout: 30);
$cache = new CircuitBreaker($adapter, $breaker);

$breaker->trip();
$this->assertFalse($cache->save('key', 'value'));
$this->assertFalse($cache->save('key', 'value'));
$this->assertSame(1, $adapter->saves);

// A fresh breaker stands in for the circuit having closed again.
$closed = new CircuitBreaker($adapter, new UtopiaCircuitBreaker(threshold: 99));
$this->assertFalse($closed->save('key', 'value'));
$this->assertSame(2, $adapter->saves, 'A closed circuit attempts the write again.');
}

public function testWritesStillReportToAClosedCircuit(): void
{
$adapter = new FailingAdapter();
$breaker = new UtopiaCircuitBreaker(threshold: 1);
$cache = new CircuitBreaker($adapter, $breaker);

// A failing write is evidence of a sick cache, so it must still be able to
// open the circuit rather than being invisible to it.
$this->assertFalse($cache->save('key', 'value'));
$this->assertTrue($breaker->isOpen());
}

public function testAClosedCircuitIsUnaffected(): void
{
$adapter = new Memory();
$cache = new CircuitBreaker($adapter, new UtopiaCircuitBreaker());

$this->assertSame('value', $cache->save('key', 'value'));
$this->assertSame('value', $cache->load('key', 60));
}
}

final class CountingFailingWriteAdapter extends FailingAdapter
{
public int $saves = 0;

public function save(string $key, array|string $data, string $hash = ''): bool|string|array
{
$this->saves++;

return parent::save($key, $data, $hash);
}
}
Loading