Bug
The trace summarization pipeline (posthog/temporal/llm_analytics/trace_summarization/) is missing $ai_evaluation from its AI event type lists:
constants.py — AI_EVENT_TYPES tuple is missing $ai_evaluation
sampling.py — Two hardcoded WHERE event IN (...) SQL clauses are missing $ai_evaluation
This means any $ai_evaluation events within a trace are silently excluded from:
- Trace fetching (via
queries.py which uses the AI_EVENT_TYPES constant)
- Trace sampling (via the hardcoded SQL in
sampling.py)
The canonical Node.js list in nodejs/src/ingestion/ai/process-ai-event.ts includes $ai_evaluation, and the event type is used across the codebase (schema, taxonomy, usage reports, etc.).
Impact
Traces that contain evaluation events will have incomplete summaries — the evaluation results are dropped before the LLM summarizer sees them.
Expected behavior
$ai_evaluation should be included alongside the other AI event types in both the constants tuple and the sampling SQL queries.
Bug
The trace summarization pipeline (
posthog/temporal/llm_analytics/trace_summarization/) is missing$ai_evaluationfrom its AI event type lists:constants.py—AI_EVENT_TYPEStuple is missing$ai_evaluationsampling.py— Two hardcodedWHERE event IN (...)SQL clauses are missing$ai_evaluationThis means any
$ai_evaluationevents within a trace are silently excluded from:queries.pywhich uses theAI_EVENT_TYPESconstant)sampling.py)The canonical Node.js list in
nodejs/src/ingestion/ai/process-ai-event.tsincludes$ai_evaluation, and the event type is used across the codebase (schema, taxonomy, usage reports, etc.).Impact
Traces that contain evaluation events will have incomplete summaries — the evaluation results are dropped before the LLM summarizer sees them.
Expected behavior
$ai_evaluationshould be included alongside the other AI event types in both the constants tuple and the sampling SQL queries.