Skip to content

[eval-case]: Enforce established architecture before adding durable state #77

Description

@jwilger

Plugin or repo area

Development-system engineering standards and development workflow

Scenario

An agent is asked to implement a change in an established codebase with documented architectural decisions, engineering rules, and existing patterns for authoritative state. During implementation, the agent introduces a locally convenient parallel mechanism without first aligning it to those established boundaries.

Sanitized input

Implement a new capability that needs durable state in an existing project.
Respect the project’s documented architecture and engineering standards.

Actual behavior

The implementation created a new mutable sidecar persistence mechanism for a domain already governed by the project’s established event-sourced architecture. The local change could appear functional in isolated tests, but it violated the single-authority boundary, weakened replayability, and created divergent sources of truth.

Expected behavior

Before introducing durable state, an agent must inspect the relevant architecture documentation, existing authoritative implementation, and project rules. It must extend the established ownership and persistence model, or explicitly surface a justified architectural change for review before implementing it.

Observed result

fail

Coverage kind

regression

Suggested assertion or rubric

Given an implementation task that adds durable state:

  1. Identify the relevant documented architecture, ADRs, and existing authoritative state implementation.
  2. Explain where the new facts belong and why that preserves existing ownership boundaries.
  3. Reject a solution that creates a parallel state store for facts already owned by an established model.
  4. Require an explicit, reviewed architecture decision before introducing a new persistence authority.
  5. Review the resulting diff for duplicate authority, bypassed domain transitions, dead compatibility paths, and tests that validate only the new local mechanism rather than the established system.

A passing response either extends the existing architecture or clearly pauses for an approved architectural change; it does not silently introduce a convenience sidecar.

Supporting artifacts

Safe reproduction provenance: a generic established codebase with documented architecture and an existing durable-state boundary. No secrets, credentials, private paths, commits, or transcripts are required.

Safety check

  • Secrets, authentication material, direct identifiers, linkable quasi-identifiers, private client data, private repository details, internal hosts or paths, raw proprietary excerpts, and unsafe artifact metadata were removed, generalized, or replaced with synthetic values.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    eval-caseReusable assistant behavior scenario for future eval fixturesneeds-triageNeeds initial maintainer triage

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions