Skip to content

fix(server): evict model-suffixed cache keys on auto-reload - #2297

Open
alireza-aminzadeh wants to merge 1 commit into
NVIDIA-NeMo:developfrom
alireza-aminzadeh:fix/issue-2263-auto-reload-cache
Open

fix(server): evict model-suffixed cache keys on auto-reload#2297
alireza-aminzadeh wants to merge 1 commit into
NVIDIA-NeMo:developfrom
alireza-aminzadeh:fix/issue-2263-auto-reload-cache

Conversation

@alireza-aminzadeh

@alireza-aminzadeh alireza-aminzadeh commented Aug 17, 2026

Copy link
Copy Markdown

Description

nemoguardrails server --auto-reload never applied config edits on a running server. Two bugs:

  1. Cache-key mismatch: _get_rails() stores instances as config_id:model_name (because GuardrailCheckRequest.model is required on /v1/checks), but the watchdog only deleted the bare config_id. Eviction was a silent no-op for real traffic. The events-history restore path had the same mismatch.
  2. Docker bind-mount inotify can stop delivering events with no error. The existing 5s monitor loop now polls file mtimes and evicts stale configs, and it restarts the observer if that thread has died.

Eviction now matches every cache key that includes the changed config id (bare, :model suffix, and hyphen-joined multi-config keys) and stores events history under those same keys.

Related Issue(s)

Verification

  • uv run --locked pytest tests/server/test_auto_reload.py -q — 16 passed
  • Focused unit tests cover model-suffixed eviction, events-history restore keys, single-config path mapping, mtime polling (edit and delete), and the watchdog handler

AI Assistance

  • No AI tools were used.
  • AI tools were used; a human reviewed and can explain every change (tool: ___).

Checklist

  • I've read the CONTRIBUTING guidelines.
  • This PR links to a triaged issue assigned to me.
  • My PR title follows the project commit convention.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • I've noted any verification beyond CI and any checks I couldn't run.
  • I did not update generated changelog files manually.
  • I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
  • @mentions of the person or team responsible for reviewing proposed changes.

--auto-reload never dropped cached LLMRails instances because the watchdog looked up the bare config id while /v1/checks stores config_id:model. Evict every matching key and poll mtimes so Docker bind-mount inotify stalls still reload.

Signed-off-by: Alireza Aminzadeh <syeedalireza@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: L status: needs triage New issues that have not yet been reviewed or categorized.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: server --auto-reload never picks up config changes on a running server

1 participant