Issue
The agent enrichment pipeline currently relies on prompt instructions to produce JSON, then
parses and validates the raw text response afterward. This is fragile and makes structured
extraction dependent on prompt compliance rather than model-level enforcement.
Current behavior:
- BaseAgent calls
llm.invoke(prompt) directly
- raw text is parsed with _parse_json_value / _parse_json_object
- parsed data is validated afterward with TypeAdapter
This means:
- malformed JSON can still be returned by the model
- prompt wording is carrying too much responsibility
- provider behavior is inconsistent across the fallback chain
- agent logic contains repeated parse/recovery code that should not be in the happy path
Goal
Move structured extraction to model-level enforcement using .with_structured_output(...)
where supported, so agent tasks receive validated Python objects instead of raw JSON text.
Scope
- audit provider support across the current FreeTierLLM fallback chain
- add a structured-output wrapper at the LLM factory/base-agent layer
- map each enrichment task to an explicit Pydantic schema
- update agent flows to use structured invocation instead of manual JSON parsing
- define fallback behavior for providers that do not support structured output
- add tests for supported, unsupported, and failover scenarios
Suggested approach
- Add a schema-aware LLM accessor in the config/base-agent layer.
- Implement a _run_structured(prompt, schema) helper in BaseAgent.
- Start with one low-risk enrichment task such as SocialMedia or Contact.
- Expand to Education, PoliticalBackground, FamilyMember, and CrimeRecord.
- Retain prompt-based JSON parsing only as a controlled fallback if required.
Issue
The agent enrichment pipeline currently relies on prompt instructions to produce JSON, then
parses and validates the raw text response afterward. This is fragile and makes structured
extraction dependent on prompt compliance rather than model-level enforcement.
Current behavior:
llm.invoke(prompt)directlyThis means:
Goal
Move structured extraction to model-level enforcement using
.with_structured_output(...)where supported, so agent tasks receive validated Python objects instead of raw JSON text.
Scope
Suggested approach