Skip to content

Commit 0ead44b

Browse files
aneiderhiseraneiderhiser
authored
Consolidate Tuva agent workflow context (#1327)
1 parent da0b493 commit 0ead44b

4 files changed

Lines changed: 393 additions & 113 deletions

File tree

‎.github/release.yml‎

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,24 @@ changelog:
33
labels:
44
- ignore-for-release
55
categories:
6-
- title: 🚨 Breaking Changes 🚨
6+
- title: Breaking Changes
77
labels:
8-
- Semver-Major
98
- breaking-change
10-
- title: 🎉 Exciting New Features 🎉
9+
- title: New Features
1110
labels:
12-
- Semver-Minor
1311
- enhancement
14-
- title: 🪲 Bug Fixes 🪲
12+
- title: Bug Fixes
1513
labels:
16-
- Semver-Minor
1714
- bug
15+
- title: Terminology and Value Sets
16+
labels:
17+
- terminology
18+
- title: Connectors
19+
labels:
20+
- connector
21+
- title: Documentation
22+
labels:
23+
- docs
1824
- title: Other Changes
1925
labels:
2026
- "*"

‎AGENTS.md‎

Lines changed: 190 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -1,99 +1,191 @@
1-
# Tuva Local Development Workflow
1+
# Tuva Agent Operating Manual
22

3-
Follow this workflow for all work in this repository unless the user explicitly overrides it.
3+
This file is the canonical agent context for the Tuva repository. Agents must read
4+
and follow it for all Tuva work, including work performed from separate git
5+
worktrees. Keep workflow details here rather than duplicating them in local Codex
6+
skills or nested agent files.
47

58
## Project Context
69

7-
- The Tuva Project is an open-source healthcare data model and analytics framework built with dbt and SQL.
8-
- It transforms raw healthcare data (claims, clinical, eligibility, pharmacy) into a standardized model used for:
9-
- Risk adjustment (CMS-HCC)
10-
- Quality measures (HEDIS)
11-
- Cost and utilization analytics
12-
- Clinical and population health analytics
10+
- The Tuva Project is an open-source healthcare data model and analytics
11+
framework built with dbt and SQL.
12+
- It transforms raw healthcare data, including claims, clinical, eligibility, and
13+
pharmacy data, into a standardized model used for:
14+
- Risk adjustment, including CMS-HCC.
15+
- Quality measures, including HEDIS-oriented marts.
16+
- Cost and utilization analytics.
17+
- Clinical and population health analytics.
18+
- The docs site under `docs/` is a Docusaurus project hosted by Netlify at
19+
`www.thetuvaproject.com`. It documents the package and includes data
20+
dictionaries generated from model YAML metadata.
1321

1422
## Architecture Context
1523

1624
- Input Layer (`models/input_layer`):
1725
- Defines the contract user data must conform to before package processing.
18-
- Users map their raw data to this contract in their own dbt project, and Tuva refs those models.
26+
- Users map raw data to this contract in their own dbt project; Tuva refs those
27+
input models.
1928
- Claims Preprocessing (`models/claims_preprocessing`):
2029
- Normalizes claims and runs service category and encounter grouping logic.
2130
- Core Data Model (`models/core`):
2231
- Cleaned claims and clinical outputs expected to remain relatively stable.
23-
- Treat breaking changes carefully and communicate them clearly.
24-
- Data Marts (`models/*` outside core/input/preprocessing):
25-
- Advanced marts for measures, groupers, and risk models.
26-
- Terminology and Value Sets (`seeds/` definitions):
27-
- Seed definitions in repo are primarily schema/header definitions.
28-
- Underlying terminology/value set content is loaded from S3 via hooks.
32+
- Treat breaking changes carefully and make them explicit in issues, PRs, and
33+
release labels.
34+
- Data Marts (`models/data_marts` and other marts outside input/core/preprocessing):
35+
- Advanced marts for measures, groupers, risk models, analytics, and derived
36+
outputs.
37+
- When changing mart behavior or schema, update SQL, YAML descriptions/tests,
38+
and any docs/data dictionary source affected by the change.
39+
- Terminology, value sets, and data assets (`seeds/`):
40+
- Top-level package seed CSVs are primarily definitions or headers.
41+
- Larger underlying terminology/value-set content is loaded from public object
42+
storage through seed hooks and version vars.
43+
- Data asset changes must preserve cross-warehouse loading behavior.
44+
45+
## Mandatory Local Rules
46+
47+
- Use local DuckDB for development unless the user explicitly asks for another
48+
warehouse.
49+
- Run dbt from `integration_tests` using local profiles from `~/.dbt` or
50+
`DBT_PROFILES_DIR`.
51+
- Prefer `scripts/dbt-local` for local dbt commands.
52+
- Set `TUVA_DBT_PROFILE` or `DBT_PROFILE` only when local profile selection is
53+
needed.
54+
- Keep `integration_tests/dbt_project.yml` on `profile: default` before pushing.
55+
- Treat `integration_tests/profiles/*` as CI-only configuration, not local
56+
runbooks.
57+
- Treat `.github/workflows/*` as CI pipeline definitions, not local runbooks.
58+
- Keep `.github/workflows/create-release.yml` intact unless the task explicitly
59+
targets release automation.
60+
- Never merge to `main`; the user reviews and merges PRs.
61+
62+
## Seed And Data Safety
2963

30-
## Local Environment Rules (Mandatory)
31-
32-
- Treat `integration_tests/profiles/*` as CI-only configuration. Do not use these files for local development runs.
33-
- Treat `.github/workflows/*` as CI-only pipeline definitions, not local runbooks.
34-
- Changes to `.github/workflows/*` are valid when the task is a CI workflow feature/fix.
35-
- Keep `.github/workflows/create-release.yml` intact unless release workflow changes are explicitly requested.
36-
- For docs changes under `docs/*`, run local docs build validation before push/PR.
37-
- Use local DuckDB for development unless the user explicitly asks for MotherDuck or another warehouse.
38-
- Run dbt from `integration_tests` using local profiles from `~/.dbt` (or `DBT_PROFILES_DIR` if set).
39-
- Set `TUVA_DBT_PROFILE` (or `DBT_PROFILE`) for local profile selection when needed.
40-
- Keep `integration_tests/dbt_project.yml` on `profile: default` before push.
41-
- For data model and seed feature/bug work, validate using the shipped synthetic seed definitions in `integration_tests/seeds/*` on local DuckDB.
42-
43-
## Standard Local Docs Commands
44-
45-
For docs changes (`docs/**`), run:
46-
47-
- `cd docs && npm ci`
48-
- `cd docs && npm run build`
49-
50-
## Standard Local dbt Commands
51-
52-
Use the repo wrapper:
64+
- Do not modify `integration_tests/seeds/*` without explicit user approval.
65+
- For local validation, use the integration test defaults:
66+
- `use_synthetic_data: true`
67+
- `synthetic_data_size: small`
68+
- For data model, data mart, seed, terminology, or value-set work, validate
69+
against the shipped synthetic data on local DuckDB.
70+
- If a change appears to require new synthetic columns or synthetic data rows,
71+
stop and ask for explicit generation requirements before editing
72+
`integration_tests/seeds/*`.
73+
- Seed feature or bug work in top-level `seeds/*` is allowed when requested, but
74+
must be validated through `integration_tests` and local DuckDB before PR.
75+
76+
## Issue-First Worktree Workflow
77+
78+
Use this workflow for normal Tuva issue work, especially when the user points to a
79+
GitHub issue or asks the agent to create one.
80+
81+
1. Issue creation:
82+
- If the user asks the agent to create an issue, create it in
83+
`tuva-health/tuva`.
84+
- Include title, problem statement, acceptance criteria, constraints,
85+
validation expectations, and one release-note label.
86+
- If acceptance criteria or the release-note label are ambiguous, ask the
87+
user. If a conservative default is clear, choose it and record the
88+
assumption in the issue.
89+
2. Planning:
90+
- Read the issue body and comments.
91+
- Inspect the relevant repo context.
92+
- Produce a concise implementation plan before creating a worktree or making
93+
code changes.
94+
3. Branch and worktree:
95+
- After the plan is accepted, run `scripts/start-issue-worktree <issue-number>
96+
[slug]`.
97+
- The helper creates or reuses a GitHub-linked branch named
98+
`codex/issue-<n>-<slug>` from `origin/main`.
99+
- The helper creates or reuses a local worktree under
100+
`/Users/aaronneiderhiser/code/tuva-worktrees/issue-<n>-<slug>`.
101+
- All implementation happens in that worktree. Do not reuse the main checkout
102+
for issue implementation.
103+
4. Implementation:
104+
- Keep changes focused on the issue acceptance criteria.
105+
- Preserve cross-warehouse SQL portability.
106+
- Update docs and YAML metadata when model behavior, schema, or user-facing
107+
fields change.
108+
5. PR:
109+
- Rebase or update from `origin/main` before pushing when needed.
110+
- Push the linked branch and open a PR to `main`.
111+
- The PR body must include summary, validation, release-note label, and
112+
`Closes #<issue-number>`.
113+
- Mirror the issue release-note label onto the PR.
114+
6. CI:
115+
- PR open still triggers the default Snowflake `dbt run`.
116+
- After PR creation, post `/ci snowflake dbt seed dbt run`.
117+
- Monitor checks, fix failures in the same worktree, push, and continue until
118+
the relevant checks pass.
119+
- Leave merge to the user.
120+
121+
`scripts/start-dev-branch <topic>` remains available for older non-issue flows,
122+
but issue-linked worktrees are preferred.
123+
124+
## Release Labels And Notes
125+
126+
Use exactly one release-note disposition on each issue and PR:
127+
128+
- `breaking-change`
129+
- `enhancement`
130+
- `bug`
131+
- `docs`
132+
- `terminology`
133+
- `connector`
134+
- `ignore-for-release`
135+
136+
Optional version labels such as `v0.18.0` may be used for triage, but they do not
137+
drive release note categories.
138+
139+
Release behavior:
140+
141+
- `.github/workflows/create-release.yml` creates a draft release when the package
142+
version changes in `dbt_project.yml` on `main`.
143+
- GitHub generated release notes are built from merged PRs between tags.
144+
- `.github/release.yml` groups those notes by PR labels.
145+
- Because release notes are PR-label based, mirror the issue release-note label
146+
onto the PR.
147+
- Use `Closes #<issue-number>` in PR bodies so merge closes the issue and keeps
148+
issue-to-release traceability.
149+
150+
## Local Validation
151+
152+
Standard dbt commands:
53153

54154
- `scripts/dbt-local deps`
55155
- `scripts/dbt-local debug`
56156
- `scripts/dbt-local build --select <selector>`
57-
- `scripts/dbt-local build --full-refresh` before push for model/seed feature or bug work
58-
59-
Equivalent explicit form:
157+
- `scripts/dbt-local build --full-refresh`
60158

61-
- `dbt --project-dir integration_tests --profiles-dir ~/.dbt <command>`
159+
Validation expectations by change type:
62160

63-
Outside contributor bridge helper:
161+
- Docs changes under `docs/**`:
162+
- `cd docs && npm ci`
163+
- `cd docs && npm run build`
164+
- Models, marts, macros, seeds, terminology, or value sets:
165+
- `scripts/dbt-local deps`
166+
- Run the narrowest useful `scripts/dbt-local build --select <selector>`.
167+
- Run `scripts/dbt-local build --full-refresh` before PR.
168+
- CI workflow changes:
169+
- Validate workflow YAML and available local workflow linting tools.
170+
- Confirm PR-open default remains `run-snowflake`.
171+
- Confirm `/ci run|build[-warehouse]` mappings still behave as documented.
64172

65-
- `scripts/outside-pr-bridge <pr-number>`
173+
`dbt seed`, `dbt run`, and `dbt build` may require internet access because some
174+
seed and value-set content is loaded from public object storage.
66175

67176
## CI Command Workflows
68177

69-
- Default on PR open: run `run-snowflake` (Snowflake-only `dbt run`).
178+
- Default on PR open: Snowflake-only `dbt run`.
70179
- Comment commands on PRs:
71180
- `/ci run` runs `dbt run` across all supported warehouses.
72181
- `/ci run-<warehouse>` runs `dbt run` on one warehouse.
73182
- `/ci build` runs `dbt build --full-refresh` across all supported warehouses.
74183
- `/ci build-<warehouse>` runs `dbt build --full-refresh` on one warehouse.
75-
- Supported warehouses: `snowflake`, `bigquery`, `databricks`, `fabric`, `redshift`, `duckdb`.
76-
- If a PR changes seed/config files that require a full refresh, use a build command (`/ci build*`) instead of a run command (`/ci run*`).
77-
78-
## Git and PR Workflow
79-
80-
- Never commit directly to `main`.
81-
- Start every new feature/bug task from a clean, up-to-date `main`:
82-
- Preferred: `scripts/start-dev-branch <topic>`
83-
- Equivalent manual flow: clean working tree -> `git fetch origin main` -> `git checkout main` -> `git pull --ff-only origin main` -> create a concise, descriptive feature branch
84-
- Use succinct, straightforward branch names and PR titles that clearly describe the change.
85-
- Make focused commits with clear messages.
86-
- Push branch and open PR to `main`.
87-
- Monitor CI checks by default after opening a PR until all required checks pass.
88-
- If any check fails, troubleshoot, fix, push, and continue monitoring until green.
89-
- Never merge to `main`; the user merges.
90-
91-
## Seed/Data Safety
92-
93-
- Do not modify `integration_tests/seeds/*` without explicit user approval.
94-
- If a change appears to require seed changes, stop and ask first.
95-
- If new synthetic columns/data are required, ask for explicit generation requirements first (for example expected date ranges).
96-
- Seed feature/bug work in top-level `seeds/*` is allowed, but must be validated via `integration_tests` + local DuckDB before push.
184+
- `/ci snowflake dbt seed dbt run` runs Snowflake seed followed by Snowflake run.
185+
- Supported warehouses: `snowflake`, `bigquery`, `databricks`, `fabric`,
186+
`redshift`, `duckdb`.
187+
- If a PR changes seed/config files that require a full refresh, use a
188+
seed-refreshing CI command before run/test-only commands.
97189

98190
## SQL Portability Rules
99191

@@ -105,26 +197,12 @@ Outside contributor bridge helper:
105197
- Microsoft Fabric
106198
- Redshift
107199
- DuckDB
108-
109-
## Build Requirements
110-
111-
- `dbt seed` and `dbt build` may require internet access because some seeds/value sets are loaded from public S3 resources.
112-
113-
## Execution Contract Per Task
114-
115-
For each user task:
116-
117-
1. Restate goal briefly and start execution.
118-
2. Implement code changes directly.
119-
3. Validate locally by change type and report exact commands/results:
120-
- docs changes: `cd docs && npm ci && npm run build`
121-
- data model/seed changes: run `scripts/dbt-local` commands from `integration_tests` with local DuckDB
122-
- CI workflow changes: run available local workflow lint/syntax checks and confirm `/ci run|build[-warehouse]` behavior plus PR-open `run-snowflake` default
123-
4. Push/open PR and monitor CI until required checks are green by default, then report final status.
200+
- Prefer existing Tuva macros and package patterns over warehouse-specific SQL.
124201

125202
## Outside Contributor PR Workflow
126203

127-
Use this for PRs that do not trigger CI because they originate outside the Tuva GitHub org.
204+
Use this for PRs that do not trigger CI because they originate outside the Tuva
205+
GitHub org.
128206

129207
Invocation:
130208

@@ -136,18 +214,24 @@ task: outside-pr <pr-number>
136214
Deterministic flow:
137215

138216
1. Sync local `main` to clean `origin/main`.
139-
2. Fetch source PR refs and prefer `pull/<pr-number>/merge` so validation runs in clean-main merge context.
140-
3. Run local validation in DuckDB (`scripts/dbt-local deps` and `scripts/dbt-local build --full-refresh`).
141-
4. If local validation fails, stop and report failures. Do not create bridge PR.
142-
5. If local validation passes, push a bridge branch named `codex/outside-pr-<pr-number>-ci-<timestamp>`.
143-
6. Open a new PR to `main` with title prefix `[outside-pr <pr-number>]` and body linking the source PR.
144-
7. Monitor CI checks until all required checks are green; if any fail, troubleshoot/fix/push and continue until green.
217+
2. Fetch source PR refs and prefer `pull/<pr-number>/merge` so validation runs
218+
in clean-main merge context.
219+
3. Run local validation in DuckDB with `scripts/dbt-local deps` and
220+
`scripts/dbt-local build --full-refresh`.
221+
4. If local validation fails, stop and report failures. Do not create a bridge PR.
222+
5. If local validation passes, push a bridge branch named
223+
`codex/outside-pr-<pr-number>-ci-<timestamp>`.
224+
6. Open a new PR to `main` with title prefix `[outside-pr <pr-number>]` and body
225+
linking the source PR.
226+
7. Monitor CI checks until all required checks are green; if any fail,
227+
troubleshoot, fix, push, and continue until green.
145228

146229
Use `scripts/outside-pr-bridge <pr-number>` to execute this flow.
147230

148231
## Invocation Shortcut: `tuva-dev/`
149232

150-
If a user message starts with `tuva-dev/`, treat it as a request to run this exact local Tuva workflow end-to-end for a feature or bug task.
233+
If a user message starts with `tuva-dev/`, treat it as a request to run this Tuva
234+
workflow for a feature, bug, issue, or outside PR task.
151235

152236
Preferred user format:
153237

@@ -161,15 +245,21 @@ constraints:
161245
- <optional guardrails>
162246
```
163247

164-
Outside contributor format:
248+
Fallback parsing rules:
165249

166-
```text
167-
tuva-dev/
168-
task: outside-pr 1219
169-
```
250+
- If only one sentence is provided after `tuva-dev/`, treat it as the task.
251+
- If the task references an existing issue, follow the issue-first worktree
252+
workflow.
253+
- If acceptance is missing and no issue exists, infer concise acceptance criteria
254+
and record them in the issue or PR.
170255

171-
Fallback parsing rules:
256+
## Output Contract
257+
258+
For each task, report:
172259

173-
- If only one sentence is provided after `tuva-dev/`, treat it as `task`.
174-
- If `acceptance` is missing, propose concise acceptance criteria and proceed.
175-
- Assume local DuckDB + `scripts/dbt-local` validation unless user says otherwise.
260+
- Issue and PR URLs when created.
261+
- Branch and worktree path.
262+
- What changed and why.
263+
- Local validation commands run and key outcomes.
264+
- CI commands/check status when run.
265+
- Any blockers, assumptions, or required user decisions.

‎docs/agent.md‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,8 @@
1-
# The Tuva Project Docs -- AI Agent Guide
1+
# Tuva Docs Agent Guide
22

3-
The Tuva Project Docs is a docusaurus project contained in the root docs folder which this file is inside of. The purpose of the docs site is to document the Tuva Project, especially the main Tuva repo. The docs site is hosted via Netlify and is available on the web at www.thetuvaproject.com.
3+
The canonical Tuva agent instructions live in the repository root:
44

5-
The site contains data dictionaries for many data tables in the Tuva Project, including:
6-
- Input Layer
7-
- Core Data Model
8-
- Data Marts
5+
- `../AGENTS.md`
96

10-
The data marts use code that extracts table names, column names, descriptions, and data types from the YML files in the Tuva package.
7+
Follow the root guide for docs work, including Docusaurus validation, data
8+
dictionary updates, release labels, and PR workflow.

0 commit comments

Comments
 (0)