feat: add lada-cache:calibrate command for auto TTL calibration

README.md

@@ -167,7 +167,105 @@ php artisan lada-cache:disable
 
 # Re-enable cache
 php artisan lada-cache:enable
+
+# Auto-calibrate per-model TTLs from Redis OBJECT IDLETIME (see Auto-calibration below)
+php artisan lada-cache:calibrate              # dry-run
+php artisan lada-cache:calibrate --apply      # persist results
+```
+
+## Auto-calibration
+
+Picking the right TTL per model is a guessing game without production data.
+`lada-cache:calibrate` removes the guesswork: it samples Redis `OBJECT IDLETIME`
+for cached keys belonging to each Lada-cached model, combines that with recent
+cache activity, and derives a per-model TTL via:
+
 ```
+calibrated_ttl = max(ceil(P95 * safety_factor), floor(previous_ttl / 2))
+```
+
+The floor term protects against survivor bias — `OBJECT IDLETIME` can only
+sample keys that haven't yet been evicted, so successive runs would otherwise
+shrink TTLs monotonically toward zero.
+
+Results are stored in the `lada_cache_calibrations` table (shipped via package
+migration) and consumed by `TtlResolver` between the `HasLadaTtl` interface
+and the static `model_ttls` config map.
+
+### Enable
+
+```env
+LADA_CACHE_CALIBRATION_ENABLED=true        # default: false
+LADA_CACHE_CALIBRATION_SAFETY_FACTOR=2.0   # P95 multiplier
+LADA_CACHE_CALIBRATION_MIN_SAMPLES=50      # skip models with fewer samples
+LADA_CACHE_CALIBRATION_CACHE_TTL=300       # in-memory map cache, seconds
+LADA_CACHE_CALIBRATION_SCHEDULE_INTERVAL=7 # run once every 7 days; 0 = no auto-schedule
+```
+
+Then publish & run the package migration:
+
+```bash
+php artisan vendor:publish --tag=migrations
+php artisan migrate
+```
+
+### Auto-scheduled run
+
+The package auto-registers the calibration cron via
+`callAfterResolving(Schedule::class)` when calibration is enabled and
+`schedule_interval` is greater than zero. The default is **once every 7 days**
+at 03:00 — long enough for TTLs to converge on real access patterns without
+bombing Redis with daily scans.
+
+To customise, set `LADA_CACHE_CALIBRATION_SCHEDULE_INTERVAL` to the number of
+days between runs, or set it to `0` and register the command yourself:
+
+```php
+// routes/console.php (Laravel 11+) or app/Console/Kernel.php
+Schedule::command('lada-cache:calibrate --apply')->weekly();
+```
+
+The auto-registered job runs with `->onOneServer()`, `->withoutOverlapping()`,
+and `->runInBackground()` — safe under multi-server Horizon deployments.
+
+### Safety
+
+- Refuses to run when Redis `maxmemory-policy` is `*-lfu` (IDLETIME is
+  unsupported under LFU eviction — using it would calibrate every TTL
+  toward zero).
+- Dry-run by default; `--apply` is required to mutate `lada_cache_calibrations`.
+- Skips models with fewer than `min_samples` data points (including 0).
+- Pipelined `OBJECT IDLETIME` calls and cursor-driven `SSCAN`/`SCAN` keep
+  the command non-blocking even against millions of cached keys.
+
+### Activity-aware calibration
+
+While calibration is enabled, Lada records lightweight per-table activity and
+uses it on the next `lada-cache:calibrate` run. Each model is labeled with a
+signal source:
+
+| Signal | When | Effect |
+|--------|------|--------|
+| `idletime_only` | Activity read unavailable or Redis lookup failed | Original IDLETIME-only TTL |
+| `no_activity` | reads + writes below `min_reads_for_signal` | Original IDLETIME-only TTL |
+| `write_heavy` | invalidates / (hits+misses) ≥ `write_heavy_ratio` | Skip survivor-bias floor (writes were going to invalidate anyway) |
+| `read_heavy` | otherwise | Hit-ratio proportional control toward `target_hit_ratio` |
+
+Tuning knobs (all `LADA_CACHE_CALIBRATION_*` env vars):
+
+```env
+LADA_CACHE_CALIBRATION_ACTIVITY_LOOKBACK_HOURS=168      # 7 days
+LADA_CACHE_CALIBRATION_MIN_READS_FOR_SIGNAL=10
+LADA_CACHE_CALIBRATION_WRITE_HEAVY_RATIO=0.5
+LADA_CACHE_CALIBRATION_TARGET_HIT_RATIO=0.80
+LADA_CACHE_CALIBRATION_HIT_RATIO_DEADBAND=0.05
+LADA_CACHE_CALIBRATION_HIT_RATIO_LEARNING_RATE=0.30
+LADA_CACHE_CALIBRATION_HIT_RATIO_MAX_STEP=0.20
+LADA_CACHE_CALIBRATION_ACTIVITY_BUCKET_TTL_SECONDS=604800
+```
+
+The calibration log line includes per-signal counters so you can graph the
+distribution of read-heavy vs write-heavy models across runs.
 
 ## Known Issues and Limitations
 

config/lada-cache.php

@@ -65,6 +65,115 @@
     */
     'expiration_time' => env('LADA_CACHE_EXPIRATION', 0),
 
+    /*
+    |--------------------------------------------------------------------------
+    | Auto-calibration
+    |--------------------------------------------------------------------------
+    |
+    | The `lada-cache:calibrate` Artisan command samples Redis OBJECT IDLETIME
+    | for cached keys belonging to each Lada-cached model, combines that signal
+    | with recent cache activity, and derives a per-model TTL via
+    |
+    |     calibrated_ttl = max(ceil(P95 * safety_factor), floor(previous_ttl / 2))
+    |
+    | The floor term guards against survivor bias — OBJECT IDLETIME can only
+    | sample keys that haven't yet been evicted, so successive runs would
+    | otherwise shrink TTLs monotonically toward zero.
+    |
+    | Results are stored in the `lada_cache_calibrations` table and consumed
+    | by TtlResolver between the HasLadaTtl interface and the static
+    | `model_ttls` map (see "Per-model TTL overrides" above).
+    |
+    | Safety:
+    |   - The command refuses to run when Redis maxmemory-policy is *-lfu
+    |     (IDLETIME is unsupported under LFU eviction).
+    |   - `--apply` is required to persist; the default is a dry-run table.
+    |   - Models with fewer than `min_samples` data points are skipped.
+    |
+    | Designed for periodic use, e.g. weekly: `lada-cache:calibrate --apply`.
+    |
+    */
+    'calibration' => [
+        'enabled' => (bool) env('LADA_CACHE_CALIBRATION_ENABLED', false),
+        'safety_factor' => (float) env('LADA_CACHE_CALIBRATION_SAFETY_FACTOR', 2.0),
+        'min_samples' => (int) env('LADA_CACHE_CALIBRATION_MIN_SAMPLES', 50),
+        'cache_ttl' => (int) env('LADA_CACHE_CALIBRATION_CACHE_TTL', 300),
+
+        // How many --apply rows to buffer before issuing a single bulk UPSERT.
+        // With ~500 cached models a batch of 100 reduces DB round-trips ~5x.
+        'batch_size' => (int) env('LADA_CACHE_CALIBRATION_BATCH_SIZE', 100),
+
+        // Number of days between auto-scheduled calibration runs.
+        // Default 7 = once a week. Set 0 to disable the auto-schedule (you can
+        // still invoke `php artisan lada-cache:calibrate --apply` manually or
+        // register it yourself in `routes/console.php`).
+        'schedule_interval' => (int) env('LADA_CACHE_CALIBRATION_SCHEDULE_INTERVAL', 7),
+
+        // Recent activity is recorded automatically while calibration is enabled.
+        // The calibrate command can enrich the IDLETIME signal with per-table
+        // read / write counts. With activity data:
+        //   - `write_heavy` tables skip the survivor-bias floor so an invalidation
+        //     -dominated workload doesn't inflate TTL into wasted memory.
+        //   - `read_heavy` tables run through a convergent hit_ratio controller
+        //     that pulls TTL toward `target_hit_ratio` by bounded steps.
+        // Without activity (Redis down or cold window) the command
+        // falls back to the original IDLETIME-only behavior.
+
+        // How far back to aggregate activity buckets, in hours.
+        // Default 168 = 7 days, matching the retention default below.
+        'activity_lookback_hours' => (int) env('LADA_CACHE_CALIBRATION_ACTIVITY_LOOKBACK_HOURS', 168),
+
+        // Minimum reads+writes in the lookback window before the signal is
+        // trusted. Below this, the activity adjustment is skipped and the run
+        // is labeled 'no_activity'. Guards against fitting TTL to thin data.
+        'min_reads_for_signal' => (int) env('LADA_CACHE_CALIBRATION_MIN_READS_FOR_SIGNAL', 10),
+
+        // invalidates / (hits+misses) ≥ this ratio classifies the table as
+        // write_heavy. Default 0.5 = invalidations equal reads.
+        'write_heavy_ratio' => (float) env('LADA_CACHE_CALIBRATION_WRITE_HEAVY_RATIO', 0.5),
+
+        // Read-heavy proportional control parameters (HitRatioAdjustment).
+        // Defaults: pull toward 80% hit ratio, ±20% per-run step,
+        // 30% learning rate, 5% deadband for hysteresis.
+        'target_hit_ratio' => (float) env('LADA_CACHE_CALIBRATION_TARGET_HIT_RATIO', 0.80),
+        'hit_ratio_deadband' => (float) env('LADA_CACHE_CALIBRATION_HIT_RATIO_DEADBAND', 0.05),
+        'hit_ratio_learning_rate' => (float) env('LADA_CACHE_CALIBRATION_HIT_RATIO_LEARNING_RATE', 0.30),
+        'hit_ratio_max_step' => (float) env('LADA_CACHE_CALIBRATION_HIT_RATIO_MAX_STEP', 0.20),
+
+        // In-memory aggregation limits before forcing a flush.
+        'activity_flush_max_batch' => (int) env('LADA_CACHE_CALIBRATION_ACTIVITY_FLUSH_MAX_BATCH', 100),
+        'activity_flush_max_seconds' => (float) env('LADA_CACHE_CALIBRATION_ACTIVITY_FLUSH_MAX_SECONDS', 5.0),
+
+        // Per-bucket retention in seconds. Default = 7 days.
+        'activity_bucket_ttl_seconds' => (int) env('LADA_CACHE_CALIBRATION_ACTIVITY_BUCKET_TTL_SECONDS', 86400 * 7),
+    ],
+
+    /*
+    |--------------------------------------------------------------------------
+    | Per-model TTL overrides
+    |--------------------------------------------------------------------------
+    |
+    | Map of model FQCN to TTL in seconds. Models listed here override the
+    | global expiration_time. Resolution order (first non-null wins):
+    |   1. $model->getLadaTtl() if model implements
+    |      Spiritix\LadaCache\Contracts\HasLadaTtl
+    |   2. lada_cache_calibrations.calibrated_ttl
+    |      (auto-calibration via lada-cache:calibrate, see "Auto-calibration" above)
+    |   3. config('lada-cache.model_ttls.<FQCN>')
+    |   4. config('lada-cache.expiration_time') (global)
+    |
+    | Examples:
+    |   App\Models\City::class       => 86400 * 30, // 30 days for rarely-changing data
+    |   App\Models\Tournament::class => 300,        // 5 minutes for hot state
+    |   App\Models\Order::class      => null,       // fall through to global
+    |
+    | 0 = persist forever (cache until tag-based invalidation).
+    |
+    */
+    'model_ttls' => [
+        // App\Models\City::class => 86400 * 30,
+    ],
+
     /*
     |--------------------------------------------------------------------------
     | Cache Granularity

database/migrations/2024_01_01_000001_create_lada_cache_calibrations_table.php

@@ -0,0 +1,28 @@
+<?php
+
+declare(strict_types=1);
+
+use Illuminate\Database\Migrations\Migration;
+use Illuminate\Database\Schema\Blueprint;
+use Illuminate\Support\Facades\Schema;
+
+return new class extends Migration
+{
+    public function up(): void
+    {
+        Schema::create('lada_cache_calibrations', function (Blueprint $table): void {
+            $table->id();
+            $table->string('model_class')->unique();
+            $table->string('table_name')->index();
+            $table->unsignedInteger('calibrated_ttl');
+            $table->json('metrics');
+            $table->timestamp('calibrated_at');
+            $table->timestamps();
+        });
+    }
+
+    public function down(): void
+    {
+        Schema::dropIfExists('lada_cache_calibrations');
+    }
+};

src/Cache.php

@@ -4,6 +4,7 @@
 
 namespace Spiritix\LadaCache;
 
+use Illuminate\Support\Facades\Config;
 use Illuminate\Support\Facades\Log;
 use Throwable;
 
@@ -28,21 +29,32 @@ public function __construct(
         private readonly Encoder $encoder,
         ?int $expirationTime = null,
     ) {
-        $this->expirationTime = $expirationTime ?? (int) config('lada-cache.expiration_time', 0);
+        $this->expirationTime = $expirationTime ?? Config::integer('lada-cache.expiration_time', 0);
     }
 
     public function has(string $key): bool
     {
         return (bool) $this->redis->exists($this->redis->prefix($key));
     }
 
-    public function set(string $key, array $tags, mixed $data): void
+    /**
+     * Persist a cached query result under the given key with optional per-call TTL.
+     *
+     * TTL resolution:
+     *   - $ttl > 0  : SET key value EX $ttl (expires after $ttl seconds)
+     *   - $ttl = 0  : SET key value (no expiration — persist forever, rely on tag invalidation)
+     *   - $ttl null : fall back to the global expirationTime (which itself follows the same > 0 / = 0 rule)
+     *
+     * @param  array<string>  $tags
+     */
+    public function set(string $key, array $tags, mixed $data, ?int $ttl = null): void
     {
         $key = $this->redis->prefix($key);
         $value = $this->encoder->encode($data);
+        $effectiveTtl = $ttl ?? $this->expirationTime;
 
-        if ($this->expirationTime > 0) {
-            $this->redis->set($key, $value, 'EX', $this->expirationTime);
+        if ($effectiveTtl > 0) {
+            $this->redis->set($key, $value, 'EX', $effectiveTtl);
         } else {
             $this->redis->set($key, $value);
         }
@@ -59,6 +71,9 @@ public function get(string $key): mixed
         return $encoded === null ? null : $this->encoder->decode($encoded);
     }
 
+    /**
+     * @param  array<string>  $tags
+     */
     public function repairTagMembership(string $key, array $tags): void
     {
         $prefixedKey = $this->redis->prefix($key);
@@ -75,7 +90,10 @@ public function repairTagMembership(string $key, array $tags): void
     public function flush(): void
     {
         try {
-            $connectionPrefix = (string) (config('database.redis.options.prefix') ?? '');
+            // `database.redis.options.prefix` may be `false` (legacy "no prefix") in addition to
+            // null / empty string / string — `Config::string()` is too strict here.
+            $rawPrefix = config('database.redis.options.prefix');
+            $connectionPrefix = is_string($rawPrefix) ? $rawPrefix : '';
 
             // Fetch all Lada keys as returned by the connection (includes connection prefix if set)
             $keys = $this->redis->keys($this->redis->prefix('*'));
@@ -84,7 +102,7 @@ public function flush(): void
                 // Strip the connection-level prefix so the driver applies it exactly once when deleting
                 $toDelete = $connectionPrefix !== ''
                     ? array_map(
-                        static fn(string $k): string => str_starts_with($k, $connectionPrefix) ? substr($k, strlen($connectionPrefix)) : $k,
+                        static fn (string $k): string => str_starts_with($k, $connectionPrefix) ? substr($k, strlen($connectionPrefix)) : $k,
                         $keys
                     )
                     : $keys;