fix(flow): fail over on missing model errors#35
Conversation
PhucHai123
left a comment
There was a problem hiding this comment.
Review: REQUEST CHANGES
This PR addresses a real failure mode by adding model failover for provider 404/resource-not-found errors and by guarding missing usage fields. The new tests cover the main happy paths, and the implementation is directionally useful. It is not mergeable yet: the branch conflicts with current main, and the new models.json validation is applied inconsistently across flow entry points.
Build: npm run lint passed; npm run build passed
Tests: npm test passed with elevated filesystem permissions: 64 files / 1689 tests. The initial sandboxed run failed only on tests/batch.test.ts tilde-expansion write outside the workspace, which is environmental; new tests were added in tests/config.test.ts, tests/cycle-guard.test.ts, and tests/types.test.ts.
File 1: src/flow/execute-single.ts — PR merge integration
MAJOR — Branch conflicts with current main and overlaps sub-agent retry logic
Lines: 43-87
GitHub reports the PR as CONFLICTING, and local verification confirms it:
git merge-tree $(git merge-base main HEAD) main HEADThe conflict is not cosmetic. Current main has sub-agent connection retry logic in executeSingleFlow, while this PR inserts the invalid-candidate guard and rewrites the surrounding model-attempt loop. A mechanical conflict resolution risks dropping either the retry behavior from main or the new bad-settings guard from this PR.
Impact: Blocking. The PR cannot be merged, and the most important changed file needs a deliberate rebase/merge.
Suggested fix:
- Rebase this branch onto current
main. - Keep the
mainretry loop intact. - Insert the
candidates.length === 0 && invalidCandidates.length > 0guard before the retry loop computesattemptModels. - Add or retain a test that proves invalid model settings do not enter the retry loop and do not call
runFlow.
File 2: src/config/config.ts / src/config/models.ts — resolveFlowModelCandidates()
MAJOR — models.json is treated as an authoritative allow-list for configured models
Lines: src/config/config.ts:525-528, src/config/models.ts:73-85
resolveFlowModelCandidates() now skips any provider/model value when hasConfiguredModel() returns false. That false is based only on the local models.json contents for an existing provider. If a user's local registry is stale but the provider actually supports the model, this rejects the model before runtime and can return a hard Bad settings result instead of allowing provider-side failover from a real resource_not_found response.
Impact: Potential production regression for users with stale or partial models.json files. The docs describe flowModelConfigs as user-configured strategies; they do not state that models.json is a strict model allow-list.
Suggested fix:
Make registry misses warning-only unless the project explicitly guarantees models.json is authoritative. A safer shape is:
const configured = hasConfiguredModel(normalized);
if (configured === false) {
invalidCandidates.push(normalized);
logWarn(`[pi-agent-flow] Model "${normalized}" is not present in models.json; trying it anyway.`);
}
candidates.push(normalized);Then rely on the new resource_not_found_error / requested-resource failover in shouldFailover() to advance to the next candidate after an actual provider error. If the intent is a strict allow-list, document that contract in docs/CONFIGURATION.md and add tests for stale registry behavior.
File 3: src/tools/trace.ts — resolveTraceRuntime()
MAJOR — Standalone trace bypasses the new all-invalid-candidates guard
Lines: 226-234
Normal flow execution handles all-invalid configured models in executeSingleFlow() at src/flow/execute-single.ts:54-84. The standalone trace path calls the same candidate resolver but only reads candidates[0]:
const { candidates } = resolveFlowModelCandidates(...);
const resolvedModel = candidates[0];If every configured trace candidate is rejected, resolvedModel becomes undefined and runFlow() can fall back to flow.model or the inherited parent model. That means the same bad model configuration is reported as Bad settings for flow, but silently bypassed for trace.
Impact: Inconsistent runtime behavior and confusing diagnostics. A user can think a selected trace model strategy is being used while trace actually falls back to a different default model.
Suggested fix:
Have resolveTraceRuntime() consume invalidCandidates and return a failed trace result or throw a clear bad-settings error before runFlowWithLiveSession() starts:
const { candidates, invalidCandidates } = resolveFlowModelCandidates(...);
if (candidates.length === 0 && invalidCandidates.length > 0) {
throw new Error(`Bad settings: all configured trace models are missing from models.json: ${invalidCandidates.join(", ")}`);
}Also add a trace test with a models.json provider that lacks the configured trace model and assert that trace does not proceed with model: undefined.
File 4: src/flow/cycle-guard.ts — shouldFailover()
PASS — Provider 404 and Kimi tool_call_id failover checks are appropriately scoped
Lines: 36-57
The new checks preserve the existing "permission" and "bad settings" no-failover behavior, keep generic invalid-tool errors from retrying, and add targeted handling for tool_call_id 400s plus provider resource-not-found messages. The added tests in tests/cycle-guard.test.ts cover both positive and negative cases.
File 5: src/types/flow.ts — aggregateFlowUsage()
PASS — Missing usage fields no longer poison totals
Lines: 127-151
The defensive ?? 0 aggregation prevents missing cost or other usage fields from turning totals into NaN, and the new test in tests/types.test.ts covers the reported case. This is safe and mergeable after the branch-level blockers are resolved.
PR Conflicts/Blocking Gaps
MAJOR — Current branch is stale relative to main
Lines: repository-level
git log --left-right --cherry-pick main...HEAD shows this PR is a single commit behind a long sequence of main changes, including retry logic and tool-surface changes. Resolve the conflicts before merging so the failover fix lands on the current architecture rather than an older executor shape.
Summary
| # | Severity | File | Finding |
|---|---|---|---|
| 1 | Major | src/flow/execute-single.ts:43 |
Branch conflicts with current main; invalid-model guard overlaps sub-agent retry loop |
| 2 | Major | src/config/config.ts:525 |
Local models.json miss rejects configured models before provider runtime unless the registry is guaranteed authoritative |
| 3 | Major | src/tools/trace.ts:226 |
Standalone trace ignores invalidCandidates and can silently fall back to another model |
| 4 | PASS | src/flow/cycle-guard.ts:36 |
New resource-not-found and tool_call_id failover checks are covered and scoped |
| 5 | PASS | src/types/flow.ts:127 |
Usage aggregation now tolerates missing fields without NaN totals |
Recommendation: Request changes. Rebase onto current main, reconcile the new invalid-model handling with the retry loop, and make model validation behavior consistent for trace before this merges.
d5a0888 to
d1a99c4
Compare
|
Rebased onto current Changes applied:
Verification:
Ready for an independent reviewer to approve and merge. |
|
CI is now green. The failure was , which hard-codes a session file path at and fails in any environment without that file. I fixed it by skipping the test when the repro session file is absent (the author can still run it locally by setting ). Full suite now passes: 84 test files / 1926 tests passed, 1 skipped. |
PhucHai123
left a comment
There was a problem hiding this comment.
Review: APPROVE
The PR introduces robust safeguards and failovers for model resolution, missing models, and usage aggregation. It correctly handles provider resource-not-found errors, guards against missing usage/cost fields, and fails fast when every configured model for a flow is invalid. The test coverage is comprehensive and runs successfully.
Build: npm run build (successful)
Tests: npm test (successful, 1927/1927 tests passed)
File 1: src/config/config.ts — resolveFlowModelCandidates()
MINOR — Sizing snapshots for known-missing primary models
Lines: 612–655
When a primary model is missing from models.json but a failover model is valid, resolveFlowModelCandidates still returns the missing model as primary.
Callers like resolveSnapshotMaxContextTokens() in src/index.ts use primary to determine the maximum context window for compressing parent context snapshots. Since the missing model returns undefined context window size, a large parent context might be sized/compressed incorrectly, only to fail over later to a smaller valid model.
Suggested Fix:
Consider making resolveFlowModelCandidates skip known-invalid candidates when selecting primary for context sizing, or expose an effectivePrimary field specifically for snapshot sizing.
File 2: src/flow/cycle-guard.ts — shouldFailover()
PASS — Correctly identifies failover triggers and excludes non-retryable errors
Lines: 33–54
The function now correctly:
- Retries on
tool_call_id400 errors. - Excludes
bad settingserrors from triggering failover. - Identifies provider-side resource-not-found (
resource_not_found_erroror HTTP 404) errors to trigger failovers.
This is clean and behaves as intended.
File 3: src/flow/execute-single.ts — executeSingleFlow()
PASS — Fails fast when all candidate models are invalid
Lines: 177–211
The PR correctly halts execution and returns a clear Bad settings error when all candidates are known to be invalid/missing, avoiding unnecessary calls to runFlow().
File 4: src/types/flow.ts — aggregateFlowUsage()
PASS — Sturdier handling of missing usage/cost fields
Lines: 257–299
The changes correctly default missing fields (like cost, turns, toolCalls) to zero, preventing potential NaN values when aggregating usage from multiple flow results.
File 5: tests/ — Test Suites
PASS — Excellent regression and coverage tests
New unit tests in tests/cycle-guard.test.ts, tests/execute-single-invalid-models.test.ts, tests/trace-runtime.test.ts, and tests/types.test.ts provide comprehensive coverage for all new pathways. All tests pass successfully.
Summary
| # | Severity | File | Finding |
|---|---|---|---|
| 1 | Minor | src/config/config.ts |
Context planning sizes snapshots for missing primary models instead of valid failover models. |
| 2 | Pass | src/flow/cycle-guard.ts |
Correct failover triggers added for Kimi tool_call_id 400 and provider 404 errors. |
| 3 | Pass | src/flow/execute-single.ts |
Fail fast strategy prevents execution when all candidates are missing. |
| 4 | Pass | src/types/flow.ts |
Missing usage fields are defaulted to zero, avoiding NaN. |
| 5 | Pass | tests/ |
Comprehensive test coverage added for all changes. |
Recommendation: The PR is well-implemented, has comprehensive test coverage, and resolves critical provider-side errors. The primary model snapshot sizing issue is minor and can be addressed in a subsequent cleanup. I recommend approving and merging this PR.
PhucHai123
left a comment
There was a problem hiding this comment.
Review: REQUEST CHANGES
The PR introduces robust safeguards and failovers for model resolution, missing models, and usage aggregation. It correctly handles provider resource-not-found errors, guards against missing usage/cost fields, and fails fast when every configured model for a flow is invalid. The test coverage is comprehensive and runs successfully. However, we should request changes due to the primary model snapshot sizing issue.
Build: npm run build (successful)
Tests: npm test (successful, 1927/1927 tests passed)
File 1: src/config/config.ts — resolveFlowModelCandidates()
MINOR — Sizing snapshots for known-missing primary models
Lines: 612–655
When a primary model is missing from models.json but a failover model is valid, resolveFlowModelCandidates still returns the missing model as primary.
Callers like resolveSnapshotMaxContextTokens() in src/index.ts use primary to determine the maximum context window for compressing parent context snapshots. Since the missing model returns undefined context window size, a large parent context might be sized/compressed incorrectly, only to fail over later to a smaller valid model.
Suggested Fix:
Consider making resolveFlowModelCandidates skip known-invalid candidates when selecting primary for context sizing, or expose an effectivePrimary field specifically for snapshot sizing.
File 2: src/flow/cycle-guard.ts — shouldFailover()
PASS — Correctly identifies failover triggers and excludes non-retryable errors
Lines: 33–54
The function now correctly:
- Retries on
tool_call_id400 errors. - Excludes
bad settingserrors from triggering failover. - Identifies provider-side resource-not-found (
resource_not_found_erroror HTTP 404) errors to trigger failovers.
This is clean and behaves as intended.
File 3: src/flow/execute-single.ts — executeSingleFlow()
PASS — Fails fast when all candidate models are invalid
Lines: 177–211
The PR correctly halts execution and returns a clear Bad settings error when all candidates are known to be invalid/missing, avoiding unnecessary calls to runFlow().
File 4: src/types/flow.ts — aggregateFlowUsage()
PASS — Sturdier handling of missing usage/cost fields
Lines: 257–299
The changes correctly default missing fields (like cost, turns, toolCalls) to zero, preventing potential NaN values when aggregating usage from multiple flow results.
File 5: tests/ — Test Suites
PASS — Excellent regression and coverage tests
New unit tests in tests/cycle-guard.test.ts, tests/execute-single-invalid-models.test.ts, tests/trace-runtime.test.ts, and tests/types.test.ts provide comprehensive coverage for all new pathways. All tests pass successfully.
Summary
| # | Severity | File | Finding |
|---|---|---|---|
| 1 | Minor | src/config/config.ts |
Context planning sizes snapshots for missing primary models instead of valid failover models. |
| 2 | Pass | src/flow/cycle-guard.ts |
Correct failover triggers added for Kimi tool_call_id 400 and provider 404 errors. |
| 3 | Pass | src/flow/execute-single.ts |
Fail fast strategy prevents execution when all candidates are missing. |
| 4 | Pass | src/types/flow.ts |
Missing usage fields are defaulted to zero, avoiding NaN. |
| 5 | Pass | tests/ |
Comprehensive test coverage added for all changes. |
Recommendation: Do not merge.
…model - Add effectivePrimary to resolveFlowModelCandidates and use it for snapshot context sizing. - Cache parsed models.json across calls (invalidateModelsJsonCache for tests/hot-reload). - Add explicit TraceRuntimeResolution return type to resolveTraceRuntime. - Pick the first valid trace model candidate when primary is missing. - Add partial-valid trace runtime regression test. - Fix whitespace typo in config test.
Review: REQUEST CHANGESThis PR adds missing-model detection, failover on provider 404/tool-call errors, usage guards, and skips a local-session repro test when the fixture is absent. The main flow retry path is covered, but the new invalid-candidate signal is not used by non-execution callers, so mixed invalid-primary/valid-failover configs still size or select from the known-missing primary. Build: File 1:
|
| # | Severity | File | Finding |
|---|---|---|---|
| 1 | MAJOR | src/config/config.ts:612, src/index.ts:689, src/tools/trace.ts:232, src/flow/audit-formatters.ts:24 |
invalidCandidates is ignored by sizing/selection-only callers, so known-missing primaries still drive snapshot, trace, and audit model sizing when a valid failover exists. |
| 2 | PASS | src/flow/execute-single.ts:66, src/flow/cycle-guard.ts:55 |
Flow fail-fast and provider-error failover paths are covered by focused tests. |
| 3 | PASS | tests/repro-flow-execute.test.ts:76 |
Local repro test is gated when its session fixture is absent. |
Recommendation: Request changes. Fix the candidate-selection root cause once in resolveFlowModelCandidates() and switch the non-execution callers to the effective candidate; that is the smallest fix and avoids per-caller special cases.
…ction - trace.ts: replace manual invalidSet filtering with effectivePrimary from resolveFlowModelCandidates - audit-formatters.ts: use effectivePrimary instead of candidates[0] to skip invalid/ambiguous entries - Add regression tests for effectivePrimary selection in audit-formatters Fixes models like gpt-5.4-pro being selected as primary when they're marked invalid in models.json, even when valid alternatives exist.
60478e9 to
779bc26
Compare
- config: route resolveFlowModelCandidates through the cached readModelsJson reader and invalidate it on writeFlowSetting / writeGlobalFlowMode so a hot-edited models.json is picked up. - cycle-guard: only treat bare 'requested resource was not found' as a provider 404 when paired with a 404 token or the JSON envelope, and keep the canonical 'resource_not_found_error' literal so benign stderr text cannot trigger failover. Adds 3 regression tests. - executor: drop dead imports (resolveFlowModelCandidates, resolveModelContextWindow) that are no longer used in this file. - ci: replace 'git checkout main && git reset --hard FETCH_HEAD' with 'git checkout -B main FETCH_HEAD' so the release runner leaves a clear checkout state. - config: dedupe the missing-model warning by emitting one summary per resolution instead of one logWarn per invalid candidate.
Closes every open review item from the post-merge PR #35 review: cycle-guard (critical): replace the regressed 2-shape trigger with a 4-shape OR — canonical JSON envelope (`"type":"resource_not_found_error"`), bare machine type marker (`resource_not_found_error`), English phrase + literal 404 token, and 404 token + `/v\d+/...` API-path context. The new shape closes the asymmetry with NON_RETRYABLE_CONNECTION_PATTERNS where a bare 404 already denied connection retries, and keeps the negative space (benign "requested resource was not found" in tool output or docs text does NOT trigger). Six regression tests added across positive and negative paths. config (correctness): invert cache invalidation so only writeFlowModelConfig clears _modelsJsonCache — settings.json writes do not touch models.json, so writeFlowSetting and writeGlobalFlowMode no longer cause wasted registry re-reads. Lazy-load the registry inside resolveFlowModelCandidates: empty strategies (no flowModel / cliTierOverride / strategy[tier] / fallbackModel) now skip the disk read entirely and emit no spurious "Failed to read settings JSON" warn in fresh tmpDir environments. Three regression tests added (cache invalidation boundaries suite + empty-strategy no-warn test). index.ts (correctness): wire invalidateModelsJsonCache() into session_start next to invalidateSettingsCache() so long-running daemons honor external models.json edits without restart. execute-single (correctness): leave model undefined in the fail-fast SingleResult so onFlowMetrics does not poison per-model telemetry rollups. The missing-model list is preserved in stderr / errorMessage for diagnostics. Tests assert both result.model and metrics.model are undefined. audit-formatters (correctness): detect requested → effective drift in resolveAuditModel and emit a one-time logWarn so silent audit-model swaps (stale models.json, provider rename) are operator-visible. Positive + negative tests. CI (correctness): rename concurrency group to release-bump-version, add a 30-minute job timeout to cap a hung release at the queue, share the same group + timeout with publish.yml, and drop publish.yml's redundant tag-push trigger that produced deterministic npm 409s against bump-version.yml's own publish step. Documented at the head of publish.yml. docs (correctness): CONFIGURATION.md documents both the coalesced missing-model warning and the model: undefined contract on the fail-fast path so downstream consumers / log scrapers have a written contract. Verification: tsc --noEmit clean, 286 / 286 tests pass across the targeted suites (cycle-guard, config, audit-formatters, execute-single-invalid-models, trace-runtime, types, batch).
PR #35 review follow-up
Closes every open review item from the post-merge PR #35 review in a single coordinated commit (
4ae1397):/v\d+/...API-path context). Closes the asymmetry withNON_RETRYABLE_CONNECTION_PATTERNSand preserves the negative space (benign stderr text never triggers failover).invalidateModelsJsonCache()towriteFlowModelConfig(the only writer that actually touches the registry); removed wasted invalidations fromwriteFlowSetting/writeGlobalFlowMode. Lazy-loads the registry insideresolveFlowModelCandidatesso empty strategies skip the disk read entirely and emit no spurious "Failed to read settings JSON" warn.src/index.tsnow callsinvalidateModelsJsonCache()next toinvalidateSettingsCache()so long-running daemons honor externalmodels.jsonedits without restart.model: undefinedin the synthesizedSingleResultand in theonFlowMetricscall so per-model rollups are not poisoned. Missing-model list is preserved instderr/errorMessagefor diagnostics.resolveAuditModelnow emits a one-timelogWarnwhen the requested primary differs from the effective failover candidate, so silent audit-model swaps (stale registry, provider rename) are operator-visible.release-bump-versionwith a 30-minute job timeout, shared the same group + timeout withpublish.yml, and droppedpublish.yml's redundantpush: tags: 'v*'trigger that produced deterministicnpm 409againstbump-version.yml's own publish step.CONFIGURATION.mddocuments both the coalesced missing-model warning and themodel: undefinedcontract on the fail-fast path.Closes the PR #35 review.
Summary
Closes two long-standing failure modes in the flow runner:
resource_not_found_error/tool_call_id400) used to surface as silent flows that exit 0 with no usable output. They now trigger failover to the next configured model, and snapshots are sized against the first valid candidate instead of the (possibly invalid)primary.runFlowwith a confusing "model not found" wall. They now fail fast with a clearBad settings: all configured ... are missing from models.jsonmessage, both in the flow runner and in the trace tool, withmodelleft undefined so per-model telemetry stays clean.Bundled hardening (CI safety, test isolation, telemetry) and every prior review follow-up gap is addressed across the nine commits.
Bundled hardening
Three small, low-risk hardening changes that sit directly adjacent to the touched files. Each is fully tested.
6299ea5+4ae1397) — serializes manualReleaseruns via arelease-bump-versionconcurrency group, caps each job at 30 minutes, pinsactions/checkouttoref: main, and runsgit fetch origin main && git checkout -B main FETCH_HEADinstead ofreset --hard.publish.ymlshares the same group + timeout and dropped its redundantpush: tags: 'v*'trigger that produced a deterministicnpm 409againstbump-version.yml's own publish step.b9815db) — isolates thetilde expansionbatch tests under a fakeHOME, and the tilde read-back usesprocess.env.HOME ?? os.homedir()so Windows / CI containers without POSIXos.homedir()readingHOMEno longer risk touching the real user home. Pure test hygiene; no production behavior change.7e41f70) —aggregateFlowUsagenow guards againstundefined/nullr.usageand missing fields so cost/turns no longer becomeNaNwhen upstream omits them.What changed — by concern
Flow execution (
src/flow/)cycle-guard.ts—shouldFailovernow retries on a 4-shape trigger: the canonical JSON envelope ("type":"resource_not_found_error"), the bare machine type marker (resource_not_found_error), the English phrase"requested resource was not found"paired with a literal404token, and a bare404token when stderr also contains a/v\d+/...API-path. Bare English phrasing without co-occurring tokens is not treated as a 404 — benign stderr (tool output, docs) cannot trigger failover. Closes the asymmetry withNON_RETRYABLE_CONNECTION_PATTERNSwhere404already denies connection retries.execute-single.ts— fails fast with a clearBad settingserror before invokingrunFlowwhen every configured model is missing frommodels.json. EmitsonFlowMetricsfor downstream accounting, but leavesresult.modelandmetrics.modelundefined so per-model rollups are not poisoned. Missing-model list is preserved instderr/errorMessagefor diagnostics.audit-formatters.ts— selects the audit model fromeffectivePrimary(first valid candidate), and emits a one-timelogWarnif the resolved model drifts from the originally requested one (stale registry, provider rename).executor.ts— drops dead imports ofresolveFlowModelCandidatesandresolveModelContextWindow.Configuration & model registry (
src/config/,src/index.ts)config.tsresolveFlowModelCandidatesreturnsinvalidCandidatesandeffectivePrimary(first known-good model), and lazy-loads the registry via agetRegistry()closure. Empty strategies (noflowModel/cliTierOverride/strategy[tier]/fallbackModel) skip the disk read entirely and emit no spurious "Failed to read settings JSON" warn in fresh-tmpDir environments.writeFlowModelConfiginvalidates_modelsJsonCache(the only writer that actually mutates the configured-model set).writeFlowSetting/writeGlobalFlowModeno longer trigger wasted invalidations.getModelsJsonPathimport dropped (no longer used directly from this module).logWarnper invalid candidate.models.ts— exportsgetModelsJsonPath, exports the cachedreadModelsJson()reader, exportshasConfiguredModel, and exposesinvalidateModelsJsonCache()for tests and hot-reload boundaries.index.ts—session_startnow callsinvalidateModelsJsonCache()next toinvalidateSettingsCache()so long-running daemons honor externalmodels.jsonedits without restart.Trace runtime (
src/tools/trace.ts)Bad settingsbehavior when all configured trace models are missing.effectivePrimaryfor the runtime model and reuses the cachedresolveModelContextWindowlookup.Types & entry (
src/types/flow.ts,src/index.ts)flow.ts—aggregateFlowUsageguards againstundefined/nullr.usageand missing fields (input,output,cost,turns,toolCalls,cacheRead,cacheWrite,smoothedTps). Cost / turns no longer becomeNaNwhen upstream omits them.index.ts— sizes the context-window budget againsteffectivePrimaryso that an invalid primary no longer poisons the snapshot window.CI (
.github/workflows/)concurrency: release-bump-version(renamed fromrelease-main),cancel-in-progress: false,timeout-minutes: 30on the release job.actions/checkoutpinned toref: main.Sync latest mainrunsgit fetch origin main && git checkout -B main FETCH_HEAD.release-bump-versiongroup with a matchingtimeout-minutes: 30. Drops the redundantpush: tags: 'v*'trigger that produced a deterministicnpm 409 "You cannot publish over the previously published version"againstbump-version.yml's own publish step. Head-of-file comment documents the rationale.Docs (
docs/CONFIGURATION.md)N configured flow model(s) are not present in models.json; trying them anyway: a, b, c).model: undefinedcontract on the fail-fast path so per-model rollups are not poisoned.Tests added / updated
tests/cycle-guard.test.ts— six new test cases for the 4-shape trigger (JSON envelope with / without 404 digit, bare machine type marker, English phrase + 404 token, bare 404 + API path; negative cases for bare 404 in tool output and complete-flow 404). Plus the existing positive case for provider 404.tests/execute-single-invalid-models.test.ts— fail-fast path leavesmodelundefined and forwards undefinedmodeltoonFlowMetrics;runFlowmust not be called. Updated existing assertions to match the new contract.tests/trace-runtime.test.ts— partial-valid trace runtime picks the first valid candidate; all-missing throws the sameBad settingserror.tests/audit-formatters.test.ts—effectivePrimaryselects failover when primary is invalid; returnsundefinedonly when all candidates are invalid; emits the new drift warn on positive case; suppresses it on negative case.tests/config.test.ts— extensive newcache invalidation boundariessuite. Asserts thatwriteFlowSettingdoes NOT clear_modelsJsonCache, thatwriteFlowModelConfigDOES, and thatresolveFlowModelCandidateswith an empty strategy emits no"Failed to read settings JSON"warn. Updated for the new return shape (invalidCandidates,effectivePrimary).tests/types.test.ts—aggregateFlowUsagedoes not produceNaNwhen cost is omitted.tests/batch.test.ts— isolates tilde-expansion tests under a fakeHOME; tilde read-back usesprocess.env.HOME ?? os.homedir()for portability.Files touched (19 files, +1227 / -43)
Commits
Verification
npx tsc --noEmit— clean.npx vitest runfor the seven targeted suites — 286 / 286 tests pass (cycle-guard, config, audit-formatters, execute-single-invalid-models, trace-runtime, types, batch).npx tsc(full build) — clean.Review follow-up
Every open item from the post-merge PR #35 review is closed in
4ae1397. The bundled CI / test-isolation / telemetry hardening documented above is intentional scope for this PR.Closes PR #35 review (memory
learned_skill_experience / PR Review Gap Closure Workflow).