Skip to content

Commit 8df7f3e

Browse files
V48 Gate 5 (spec-impl): Co-locate tests with owners
Move parse/PCC/measure suites to owning packages; core vs edges layout; agent-generics keeps composition tests only.
1 parent 13f8558 commit 8df7f3e

54 files changed

Lines changed: 1252 additions & 590 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.docs/AGENTS.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,25 @@
5353
- Any inline code comment that cites an accepted QA finding shorthand must always carry the fully-qualified tag `[VERSION]-Gate[N]-F[ID]` (e.g. `V48-Gate3-F26-B`), never a bare `F26-B`-style tag. The same fully-qualified tag must always be discoverable in the specification/QA ledger file(s) (e.g. the finding's own `### V48-Gate3-F26` heading in `.qa/BITCODE_V48_QA.md`), so a reader can go from either direction — code comment to spec entry, or spec entry to every citing code comment — with a single grep. This keeps finding-to-fix traceability intact without requiring the surrounding file or commit to already establish which version/gate is active.
5454
- Once implementation starts on a gate branch, do not stop at partial progress unless blocked by missing external input or explicit user pause. A gate branch is ready to stop only when the gate's acceptance criteria are implemented, specified, tested, documented, committed, pushed, and pull-requested for closure into the version branch.
5555
- Treat gate and promotion workflow health as part of gate closure. Gate pull requests into version branches must be green through the **active + draft** gate-quality / canon-quality surface (not prior-era `check-vN-*` suites). Repository-wide living product CI (uapi lint/typecheck/build/Jest) must remain greenable during draft work. Version pull requests into `main` must pass the version promotion workflow, which performs promotion-grade validations and commits the standalone `BITCODE_SPEC.txt` pointer change only after those validations pass.
56+
- **Test co-location with package ownership (required):** unit tests live in the package that **owns** the unit under test — never under a consumer because it imports the unit. Examples: `@bitcode/parsing` tests under `packages/parsing/`; PrepareConciseContext (PCC) under `packages/generic-generations/failsafes/` even while an LLM-bound factory is still transitional in `@bitcode/agent-generics`. **Implementer exception:** when package B *implements for itself* a base/specific class from a primitive/base in package A, B's specialization tests co-locate in B; A's base tests stay in A. Do not re-host A's base suite under B.
57+
- **Tests co-locate with the owning package (required):** unit tests live in the package that **owns** the unit under test — not in a consumer, re-export host, or monorepo root. Law:
58+
- **Primitive package** → primitive contracts
59+
- **Generic base package** → base contracts (even if an LLM factory is *temporarily hosted* in another package for execution coupling)
60+
- **Implementing package** → only tests that *compose/specialize* the base for that package
61+
- **Product package** → product-only behavior
62+
Anti-pattern: parking `@bitcode/parsing` or MeasureAgent base tests under `@bitcode/agent-generics` because factories re-export or host them. Pilot co-location: `packages/parsing`, `packages/generic-generations/failsafes` (PCC), `packages/generic-measurements/measure-agent`.
63+
- **Test organization: core vs edges (required for backend packages as they migrate):** package unit tests live under two categories so suites stay readable and useful:
64+
| Category | Purpose | Growth |
65+
| --- | --- | --- |
66+
| **Core** | Extremely clear default / happy-path behavior of the package or subsystem | Stable — grows only when core API/behavior changes |
67+
| **Edges** | Exhaustive edge cases, debug flags, failures, bounds, regression pins | Grows as edges are discovered |
68+
**Layout (both folder and filename required):**
69+
```text
70+
src/__tests__/core/<topic>.core.test.ts
71+
src/__tests__/edges/<topic>.edges.test.ts
72+
src/__tests__/support/ # shared fixtures only — not a third test class
73+
```
74+
Default package `test` runs **both**; `test:core` / `test:edges` are convenience only — **edges are never optional** for commit/CI green. New tests go into the correct folder immediately; do not add flat `__tests__/*.test.ts` on opted-in packages. Pilot exemplars: `@bitcode/agent-generics` (core/edges), `@bitcode/generic-generations-failsafes` (PCC + prepared-context), `@bitcode/parsing`. When reorganizing, also **elevate**: clear descriptors, drop stale/clutter tests, make core files teach the product by reading alone. Human guide: `CONTRIBUTING.md` §8.0.
5675
- **REQUIRED — never commit until all living CI checks are run locally and completely green.** This is absolute for every commit that may land on a shared branch, gate PR, or production path. **All commits must be green for production deployment** — a red commit is undeployable product debt, not “CI will catch it later.”
5776
- **Hard ban:** do **not** `git commit` (and do **not** push) while lint, typecheck, build, or required package/Jest suites are red, skipped, or only partially run for the change set.
5877
- **Before every commit**, the living local CI mirror must be green. It is **enforced by `.githooks/pre-commit`** (`pnpm run hooks:install` once per clone):

.docs/FAMILIARIZATION.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -514,7 +514,7 @@ Grouped by role. Names are `@bitcode/<name>` unless noted.
514514
| Package | Responsibility |
515515
| --- | --- |
516516
| `execution-generics` | `Execution` state tree, `Executor`, sequential/parallel/pipe |
517-
| `agent-generics` | Agent = Executor + PTRR composition over generations |
517+
| `agent-generics` | Agent = Executor + PTRR composition over generations; tests under `__tests__/core|edges` (see CONTRIBUTING §8.0) |
518518
| `tools-generics` | `Tool` class, factories, MCP bridges |
519519
| `pipelines-generics` | ExecutionPipeline primitives / stream hooks (no phases; phases are SDIVF-only) |
520520
| `generic-pipelines-execution-pipeline-sdivf` | SDIVF base loop (`packages/generic-pipelines/execution-pipeline-sdivf`) |

CONTRIBUTING.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -405,6 +405,52 @@ Connected-services orientation: [`.docs/BITCODE_CONNECTED_SERVICES.md`](.docs/BI
405405

406406
## 8. Testing, documenting, and proving
407407

408+
### 8.0 Package co-location and core vs edges
409+
410+
**Package co-location (required):** unit tests live in the package that **owns**
411+
the unit under test. Do not put package A’s suite under package B because B is a
412+
consumer (e.g. parsing tests belong in `packages/parsing/`, not
413+
`agent-generics`; PCC belongs in `packages/generic-generations/failsafes/`).
414+
415+
**Implementer exception:** when package B *implements for itself* a base or
416+
specific class from a primitive/base in A, B’s specialization tests co-locate in
417+
B. A’s base tests stay in A. Do not re-host A’s base suite under B.
418+
419+
Package tests (backend first; other packages migrate progressively) are also
420+
split so **core behavior stays obvious** while edge coverage can grow without
421+
burying it.
422+
423+
| Category | Role | Growth |
424+
| --- | --- | --- |
425+
| **Core** | Default / happy-path behavior — a new reader should learn the package from these files alone | Stable unless core API/behavior changes |
426+
| **Edges** | Exhaustive edges: failures, bounds, debug env, odd inputs, regression pins | Grows continuously |
427+
428+
**Physical law (folder + filename both required):**
429+
430+
```text
431+
packages/<pkg>/src/__tests__/
432+
core/<topic>.core.test.ts
433+
edges/<topic>.edges.test.ts
434+
support/ # fixtures only
435+
```
436+
437+
| Script | Meaning |
438+
| --- | --- |
439+
| `pnpm -C packages/<pkg> test` | **Both** core and edges (commit / CI bar) |
440+
| `test:core` / `test:edges` | Convenience filters — **not** a way to skip edges before commit |
441+
442+
**Heuristic:** if it teaches “how this package works by default,” it is **core**.
443+
If it pins a corner, flag, failure, or bug, it is **edges**.
444+
445+
When reorganizing a package, also **elevate** tests: clearer `describe`/`it`
446+
names, remove stale or redundant cases, keep core files short and didactic.
447+
448+
**Pilots:** `@bitcode/agent-generics` (composition only); `@bitcode/parsing`;
449+
`@bitcode/generic-generations-failsafes` (PCC + prepared-context);
450+
`@bitcode/generic-measurements-measure-agent` / `absolutes`;
451+
`@bitcode/parsing`. Other packages adopt the same layout when next touched.
452+
Full agent law: `.docs/AGENTS.md`.
453+
408454
### 8.1 REQUIRED: never commit until full local CI is completely green
409455

410456
**Never commit** (and **never push**) until the living required CI surface has

packages/agent-generics/README.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,27 @@
33
Agent **primitives**: execution, registries, generation-layer factories,
44
`factoryAgent` / `factoryQuickAgent`.
55

6+
## Tests (core vs edges)
7+
8+
| Category | Path | Role |
9+
| --- | --- | --- |
10+
| **Core** | `src/__tests__/core/*.core.test.ts` | Default / happy-path package contracts |
11+
| **Edges** | `src/__tests__/edges/*.edges.test.ts` | Flags, failures, bounds, telemetry corners |
12+
13+
```bash
14+
pnpm test # both (required green)
15+
pnpm test:core # convenience
16+
pnpm test:edges # convenience — not a commit skip
17+
```
18+
19+
Law: `.docs/AGENTS.md`, `CONTRIBUTING.md` §8.0. Core should teach Thinkings +
20+
failsafe *composition* + PTRR step tools without reading edges.
21+
22+
**Not here:** PrepareConciseContext (PCC) base pins live in
23+
`@bitcode/generic-generations-failsafes` (`packages/generic-generations/failsafes/`).
24+
This package may still *host* the transitional LLM-bound factory and may test how
25+
agents *use* PCC; it must not re-host the PCC base suite.
26+
627
## Hierarchy
728

829
```

packages/agent-generics/jest.config.cjs

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,13 @@
11
const path = require('path');
22
const { createJestConfig } = require('../../tests/jest.base.cjs');
33

4+
// Core vs edges: see .docs/AGENTS.md and CONTRIBUTING §8.0.
5+
// testMatch: core and edges globs only; both always required green.
46
module.exports = createJestConfig(__dirname, {
5-
testMatch: ['**/__tests__/**/*.test.(ts|tsx)'],
7+
testMatch: [
8+
'**/__tests__/core/**/*.core.test.(ts|tsx)',
9+
'**/__tests__/edges/**/*.edges.test.(ts|tsx)',
10+
],
611
moduleNameMapper: {
712
'^@bitcode/logger$': path.join(__dirname, '../logger/src/logger.ts'),
813
// Deep subpaths not covered by package root map
@@ -14,5 +19,5 @@ module.exports = createJestConfig(__dirname, {
1419
'^@bitcode/generic-artifacts-compose$': path.join(__dirname, '../generic-artifacts/compose/src/index.ts'),
1520
'^@bitcode/parsing$': path.join(__dirname, '../parsing/src/parsing.ts'),
1621
},
17-
collectCoverageFrom: ['src/**/*.{ts,tsx}', '!src/**/*.d.ts'],
22+
collectCoverageFrom: ['src/**/*.{ts,tsx}', '!src/**/*.d.ts', '!src/**/__tests__/**'],
1823
});

packages/agent-generics/package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@
88
"build": "tsc --noEmit",
99
"typecheck": "node ../../node_modules/typescript/bin/tsc -p tsconfig.typecheck.json --noEmit",
1010
"test": "jest",
11+
"test:core": "jest --testPathPattern=__tests__/core/",
12+
"test:edges": "jest --testPathPattern=__tests__/edges/",
1113
"test:watch": "jest --watch",
1214
"test:coverage": "jest --coverage",
1315
"type-check": "pnpm run typecheck"

packages/agent-generics/src/__tests__/agent-llms-registry.test.ts renamed to packages/agent-generics/src/__tests__/core/agent-llms-registry.core.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,8 @@ jest.mock('@bitcode/generic-llms', () => {
4747
});
4848

4949
import { Execution } from '@bitcode/execution-generics';
50-
import { AgentExecution } from '../execution/AgentExecution';
51-
import { AgentLLMsRegistry } from '../execution/AgentLLMsRegistry';
50+
import { AgentExecution } from '../../execution/AgentExecution';
51+
import { AgentLLMsRegistry } from '../../execution/AgentLLMsRegistry';
5252

5353
const ENV_KEYS = [
5454
'BITCODE_LLM_PROVIDER',

packages/agent-generics/src/__tests__/failsafe-generation-sequence.test.ts renamed to packages/agent-generics/src/__tests__/core/failsafe-generation-sequence.core.test.ts

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
// @ts-nocheck
22
/**
3-
* Pins for createFailsafeGenerationSequence — the Failsafes sequence is
4-
* selection (PCC) -> task (CS, xchunks when triggered) -> repair-only (SC):
5-
* - PCC always runs ONE selection Thinkings (key-selection schema, keys only)
6-
* - CS runs ONE task Thinkings when the composed request fits
7-
* - SC adds ZERO generations when the task output is schema-complete
3+
* CORE — FailsafeGenerationSequence default success path.
4+
*
5+
* Teaches: selection (PCC) → task (ChunkThenSum) → repair-only (Stitch);
6+
* PCC selection Thinkings; one task Thinkings when request fits; Stitch adds
7+
* zero generations when task output is schema-complete.
88
*/
99
import { z } from 'zod';
1010
import { Execution } from '@bitcode/execution-generics';
11-
import { StepExecution } from '../execution';
12-
import { createFailsafeGenerationSequence } from '../steps/failsafe-sequence';
13-
import { factoryChunkThenSum, factoryStitchUntilComplete } from '../generations/llm-bound-factories';
11+
import { StepExecution } from '../../execution';
12+
import { createFailsafeGenerationSequence } from '../../steps/failsafe-sequence';
13+
import { factoryChunkThenSum, factoryStitchUntilComplete } from '../../generations/llm-bound-factories';
1414

1515
const outputSchema = z.object({ title: z.string(), score: z.number() });
1616

@@ -73,8 +73,8 @@ afterEach(() => {
7373
delete process.env.BITCODE_DEBUG_SKIP_THINKINGS_JUDGE_AND_STRUCTURED_OUTPUT;
7474
});
7575

76-
describe('createFailsafeGenerationSequence composition', () => {
77-
it('runs selection -> task -> repair-only and returns the {context, output, finalOutput} envelope', async () => {
76+
describe('CORE: FailsafeGenerationSequence default composition', () => {
77+
it('runs selection task repair-only and returns {context, output, finalOutput}', async () => {
7878
const counter = { calls: 0 };
7979
const { root, step } = makeRootAndStep(makeScriptedLLM(counter));
8080
const sequence = createFailsafeGenerationSequence({ outputSchema });

packages/agent-generics/src/__tests__/failsafe-sequence-order-and-failure.test.ts renamed to packages/agent-generics/src/__tests__/core/failsafe-sequence.core.test.ts

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,17 @@
11
// @ts-nocheck
22
/**
3-
* Pins for createFailsafeGenerationSequence:
4-
* - composition ORDER: prepare (seq-0) -> chunk (seq-1) -> stitch (seq-2)
5-
* - a rejecting LLM (e.g. the callLlmWithTimeout 'timed out after Nms'
6-
* rejection) propagates through the whole sequence as a clean failure —
7-
* no retry, no hang, no partial envelope.
3+
* CORE — FailsafeGenerationSequence structure and hard failure propagation.
4+
*
5+
* Teaches: prepare (seq-0) → chunk (seq-1) → stitch (seq-2); a rejecting LLM
6+
* fails the whole sequence cleanly (no hang / silent retry).
7+
*
8+
* Detailed failsafe call counts and debug filters: failsafe-generation-sequence
9+
* and edges as applicable.
810
*/
911
import { z } from 'zod';
1012
import { Execution } from '@bitcode/execution-generics';
11-
import { StepExecution } from '../execution';
12-
import { createFailsafeGenerationSequence } from '../steps/failsafe-sequence';
13+
import { StepExecution } from '../../execution';
14+
import { createFailsafeGenerationSequence } from '../../steps/failsafe-sequence';
1315

1416
const outputSchema = z.object({ title: z.string(), score: z.number() });
1517

@@ -43,8 +45,8 @@ function collectNodes(node: any, out: any[] = []): any[] {
4345
return out;
4446
}
4547

46-
describe('createFailsafeGenerationSequence composition order', () => {
47-
it('hosts prepare under seq-0, chunk under seq-1 and stitch under seq-2', async () => {
48+
describe('CORE: FailsafeGenerationSequence composition order', () => {
49+
it('hosts prepare under seq-0, chunk under seq-1, and stitch under seq-2', async () => {
4850
const llm = async (input: any) => ({
4951
content: scriptedContent(input),
5052
usage: { totalTokens: 5 },
@@ -67,8 +69,8 @@ describe('createFailsafeGenerationSequence composition order', () => {
6769
});
6870
});
6971

70-
describe('createFailsafeGenerationSequence failure propagation', () => {
71-
it('surfaces an LLM timeout rejection from the very first call — exactly one invocation, no retry', async () => {
72+
describe('CORE: FailsafeGenerationSequence failure propagation', () => {
73+
it('surfaces an LLM timeout from the first call — one invocation, no silent retry', async () => {
7274
const counter = { calls: 0 };
7375
const llm = async () => {
7476
counter.calls++;

packages/agent-generics/src/__tests__/prepared-task-user-payload.test.ts renamed to packages/agent-generics/src/__tests__/core/prepared-task-user-payload.core.test.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,15 @@
66
*/
77
import { z } from 'zod';
88
import { Execution } from '@bitcode/execution-generics';
9-
import { StepExecution } from '../execution';
10-
import { createFailsafeGenerationSequence } from '../steps/failsafe-sequence';
9+
import { StepExecution } from '../../execution';
10+
import { createFailsafeGenerationSequence } from '../../steps/failsafe-sequence';
1111
import {
1212
factoryChunkThenSum,
1313
factoryPrepareConciseContext,
1414
factoryReason,
1515
isPreparedTaskInput,
1616
buildPreparedTaskLlmPayload,
17-
} from '../generations/llm-bound-factories';
17+
} from '../../generations/llm-bound-factories';
1818

1919
function makeRootAndStep(llm?: (input: any) => Promise<any>) {
2020
const root = new Execution('agent-root') as any;

0 commit comments

Comments
 (0)