TECH_DEBT: Structured CQL debug output for measureeval $evaluate (#1668)

* Add DebugSections enum and MeasureEvaluationResult DTO

Adds two model classes that will support structured debug output from measure evaluation.

* Plumb DebugSections through evaluator and controller

Wires the new DebugSections type through the measure-evaluation call path and switches the $evaluate endpoint to return MeasureEvaluationResult.

* Capture and emit structured CQL debug info on $evaluate

Wires the real debug data capture into the MeasureEvaluator.

* Drop javassist exception dependency and add DebugSections tests

Polish to finish the debug-logging series.

* Add integration tests for the $evaluate debug capture path

* Cap trace tree size and log debug activations

* Add Spring converter, dev docs, and clarify cache bypass

Three small follow-up improvements bundled together since each is independently small and they share theme (developer ergonomics around the new debug parameter).

* Address review feedback: sanitize logs, omit null debugInfo, guard null subjects

* Preserve bare MeasureReport response shape when debug is empty

Author: c-schuler

Java/measureeval/DEBUG-LOGGING.md

@@ -0,0 +1,154 @@
+# Measure Evaluation Debug Output
+
+The `$evaluate` endpoint on the Measure Definition API can return structured debug data
+alongside the standard FHIR `MeasureReport`. This is intended for measure authoring and
+troubleshooting — not for production monitoring or analytics.
+
+## Endpoint
+
+```
+POST /api/measureeval/measure-definition/{id}/$evaluate?debug={sections}
+```
+
+Request body: the same FHIR `Parameters` resource as a non-debug evaluation
+(`periodStart`, `periodEnd`, `subject`, `additionalData`).
+
+## The `debug` query parameter
+
+`debug` is a comma-separated list of sections to populate. Spring binds the value through
+a custom converter so the controller receives a `Set<DebugSections>` directly.
+
+| Value                                 | Meaning                          |
+|---------------------------------------|----------------------------------|
+| _(parameter omitted)_                 | No debug data (fast path)        |
+| `false` or empty                      | No debug data (fast path)        |
+| `true` or `all`                       | Every section                    |
+| `groups`                              | Population counts + errors       |
+| `expressions`                         | CQL expression → result map      |
+| `librarydebug`                        | Per-library node-level results   |
+| `messages`                            | CQL engine messages              |
+| `traces`                              | Hierarchical execution trace     |
+| `debuglog`                            | Human-readable rendered log      |
+| `expressions,traces` (any combination)| Just those sections              |
+
+Token matching is case-insensitive and tolerates surrounding whitespace.
+Unknown tokens are silently ignored; the recognised ones around them still parse.
+
+## Response shape
+
+Without debug — bare FHIR `MeasureReport`, unchanged from the pre-debug behaviour:
+
+```json
+{ "resourceType": "MeasureReport", ... }
+```
+
+With debug — wrapper object carrying the report alongside the requested debug sections:
+
+```json
+{
+  "measureReport": { "resourceType": "MeasureReport", ... },
+  "debugInfo": {
+    "groups": [
+      {
+        "id": "group-1",
+        "populations": [
+          { "type": "initial-population", "count": 1, "subjects": ["Patient/p1"] }
+        ]
+      }
+    ],
+    "expressionResults": { "Initial Population": "[Encounter/enc-1]" },
+    "traces": [ ... ],
+    "truncated": true
+  }
+}
+```
+
+Sections that weren't requested (or that the engine didn't produce data for) are omitted
+from `debugInfo` rather than emitted as `null`.
+
+The `truncated` flag is present and set to `true` only if a section had to be capped to
+protect the response from runaway memory use. Currently this only happens when the trace
+tree exceeds `MeasureEvaluator.MAX_TRACE_FRAMES` (10,000 frames).
+
+## Wire-format compatibility
+
+The default response (no `debug` query parameter) is unchanged: clients continue to receive
+a bare FHIR `MeasureReport`. The wrapper envelope is only emitted when callers opt in by
+passing `debug=...`, so existing consumers do not need to migrate.
+
+## Performance considerations
+
+- **Without debug**: identical to the pre-change behaviour. Uses
+  `R4MultiMeasureService.evaluate(...)` and returns immediately.
+- **With debug**: routes through `R4MultiMeasureService.evaluateSingleMeasureCaptureDef(...)`
+  to capture the underlying `EvaluationResult` map. There is some overhead in this path even
+  when only the `groups` section is requested.
+- **With `traces` or `all`**: the CQL engine's tracing flags are turned on, which adds
+  per-expression instrumentation cost. On a realistic measure, this can multiply evaluation
+  time several-fold and produce response payloads in the MB range. Use only against
+  representative test inputs, not full production patient bundles.
+- The 10,000-frame trace cap (`MeasureEvaluator.MAX_TRACE_FRAMES`) is a soft floor against
+  pathological cases. If you hit it routinely on legitimate measures, raise the constant or
+  request only a subset of debug sections.
+
+## Examples
+
+```bash
+# Fast path (production behaviour)
+curl -X POST -H "Authorization: Bearer ..." \
+  -H "Content-Type: application/json" \
+  -d @parameters.json \
+  https://.../api/measureeval/measure-definition/my-measure/\$evaluate
+
+# Just see population counts plus the report
+curl -X POST -H "Authorization: Bearer ..." \
+  -H "Content-Type: application/json" \
+  -d @parameters.json \
+  "https://.../api/measureeval/measure-definition/my-measure/\$evaluate?debug=groups"
+
+# Everything (use sparingly)
+curl -X POST -H "Authorization: Bearer ..." \
+  -H "Content-Type: application/json" \
+  -d @parameters.json \
+  "https://.../api/measureeval/measure-definition/my-measure/\$evaluate?debug=all"
+
+# Targeted: expression results plus a human-readable log
+curl -X POST -H "Authorization: Bearer ..." \
+  -H "Content-Type: application/json" \
+  -d @parameters.json \
+  "https://.../api/measureeval/measure-definition/my-measure/\$evaluate?debug=expressions,debuglog"
+```
+
+## Observability
+
+When `debug` is non-empty, the controller emits an INFO log line at the start of the
+request identifying the measure id and the requested sections:
+
+```
+Measure evaluation requested with debug sections [EXPRESSIONS, TRACES] for measure my-measure
+```
+
+This is silent on the fast path so production logs aren't polluted.
+
+## Caching note
+
+The `MeasureEvaluatorCache` keeps compiled evaluators keyed by measure id, but the
+controller intentionally bypasses the cached evaluator's `evaluate(...)` method on the
+debug path. The cached evaluator was compiled with the service-wide `link.cql-debug` flag
+baked in; the request-specific `debug` parameter may differ. So the controller looks the
+measure up via the cache (for the side effect of registering the libraries with the
+`LibraryResolver`, used by `CqlLogAppender`), then calls
+`MeasureEvaluator.compileAndEvaluate(...)` which produces a per-request evaluator with the
+correct engine flags. This costs an extra compile per request — acceptable because debug
+is opt-in and rare.
+
+## Related code
+
+| File                                              | Role                                                       |
+|---------------------------------------------------|------------------------------------------------------------|
+| `models/DebugSections.java`                       | Enum of available sections + `parse(String)`               |
+| `models/MeasureEvaluationResult.java`             | Response DTO (`measureReport` + optional `debugInfo`)      |
+| `converters/DebugSectionsConverter.java`          | Spring MVC `String` → `Set<DebugSections>` converter       |
+| `configs/WebMvcConfig.java`                       | Registers the converter with the formatter registry        |
+| `controllers/MeasureDefinitionController.java`    | `$evaluate` endpoint                                       |
+| `services/MeasureEvaluator.java`                  | Compile + evaluate + `buildDebugInfo` + `buildTraceTree`   |

Java/measureeval/src/main/java/com/lantanagroup/link/measureeval/configs/WebMvcConfig.java

@@ -0,0 +1,25 @@
+package com.lantanagroup.link.measureeval.configs;
+
+import com.lantanagroup.link.measureeval.converters.DebugSectionsConverter;
+import org.springframework.context.annotation.Configuration;
+import org.springframework.format.FormatterRegistry;
+import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
+
+/**
+ * Registers project-specific Spring MVC converters/formatters so controllers can declare
+ * strongly-typed request parameters instead of taking raw Strings and parsing inline.
+ */
+@Configuration
+public class WebMvcConfig implements WebMvcConfigurer {
+
+    private final DebugSectionsConverter debugSectionsConverter;
+
+    public WebMvcConfig(DebugSectionsConverter debugSectionsConverter) {
+        this.debugSectionsConverter = debugSectionsConverter;
+    }
+
+    @Override
+    public void addFormatters(FormatterRegistry registry) {
+        registry.addConverter(debugSectionsConverter);
+    }
+}

Java/measureeval/src/main/java/com/lantanagroup/link/measureeval/controllers/MeasureDefinitionController.java

@@ -3,6 +3,8 @@
 import ca.uhn.fhir.context.FhirContext;
 import com.fasterxml.jackson.annotation.JsonView;
 import com.lantanagroup.link.measureeval.entities.MeasureDefinition;
+import com.lantanagroup.link.measureeval.models.DebugSections;
+import com.lantanagroup.link.measureeval.models.MeasureEvaluationResult;
 import com.lantanagroup.link.measureeval.models.RelatedArtifactInfo;
 import com.lantanagroup.link.measureeval.repositories.MeasureDefinitionRepository;
 import com.lantanagroup.link.measureeval.services.MeasureDefinitionBundleValidator;
@@ -15,10 +17,8 @@
 import io.opentelemetry.api.trace.Span;
 import io.swagger.v3.oas.annotations.Operation;
 import io.swagger.v3.oas.annotations.Parameter;
-import javassist.NotFoundException;
 import org.apache.commons.text.StringEscapeUtils;
 import org.hl7.fhir.r4.model.Bundle;
-import org.hl7.fhir.r4.model.MeasureReport;
 import org.hl7.fhir.r4.model.Parameters;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -30,6 +30,7 @@
 import org.springframework.web.server.ResponseStatusException;
 
 import java.util.List;
+import java.util.Set;
 
 @RestController
 @RequestMapping("/api/measureeval/measure-definition")
@@ -136,7 +137,7 @@ public String getMeasureLibraryCQL(
 
         try {
             return CqlUtils.getCql(measureDefinition.getBundle(), libraryId, range);
-        } catch (NotFoundException e) {
+        } catch (CqlUtils.ResourceNotFoundException e) {
             throw new ResponseStatusException(HttpStatus.NOT_FOUND, e.getMessage(), e);
         }
     }
@@ -146,24 +147,53 @@ public String getMeasureLibraryCQL(
     @Operation(summary = "Evaluate a measure against data in request body", tags = {"Measure Definitions"})
     @Parameter(name = "id", description = "The ID of the measure definition", required = true)
     @Parameter(name = "parameters", description = "The parameters to use in the evaluation", required = true)
-    @Parameter(name = "debug", description = "Whether to log CQL debugging information during evaluation", required = false)
-    public MeasureReport evaluate(@AuthenticationPrincipal PrincipalUser user, @PathVariable String id, @RequestBody Parameters parameters, @RequestParam(required = false, defaultValue = "false") boolean debug) {
+    @Parameter(name = "debug",
+            description = "Which debug sections to populate on the response. " +
+                    "Accepts `true`/`all` (every section), `false`/empty (no sections), " +
+                    "or a comma-separated list of: groups, expressions, librarydebug, messages, traces, debuglog.",
+            required = false)
+    public Object evaluate(
+            @AuthenticationPrincipal PrincipalUser user,
+            @PathVariable String id,
+            @RequestBody Parameters parameters,
+            @RequestParam(required = false, defaultValue = "false") Set<DebugSections> debug) {
 
         if (user != null){
             Span currentSpan = Span.current();
             currentSpan.setAttribute("user", user.getEmailAddress());
         }
 
-        // Ensure that a measure evaluator is cached (so that CQL logging can use it)
+        // Resolve the cached evaluator. Two things this gives us:
+        //   1. A cheap existence check for the measure (returns null -> 404).
+        //   2. Side effect: ensures the measure's libraries are registered with
+        //      MeasureEvaluatorCache's LibraryResolver, which CqlLogAppender uses
+        //      to translate CQL log locators back to source ranges.
+        // We intentionally do NOT call evaluator.evaluate(...) here. The cached
+        // evaluator was compiled once at first-use time with `linkConfig.cqlDebug`
+        // baked in as its only debug flag; reusing it would either log too much or
+        // too little relative to the per-request `debug` parameter. So we always
+        // recompile via compileAndEvaluate(...) with the explicit per-request
+        // section set, which sets the engine's debug/tracing flags correctly for
+        // this specific call.
         MeasureEvaluator evaluator = evaluatorCache.get(id);
 
         if (evaluator == null) {
             throw new ResponseStatusException(HttpStatus.NOT_FOUND, "Measure definition not found");
         }
 
+        if (!debug.isEmpty()) {
+            _logger.info("Measure evaluation requested with debug sections {} for measure {}",
+                    StringEscapeUtils.escapeJava(String.valueOf(debug)),
+                    StringEscapeUtils.escapeJava(id));
+        }
+
         try {
-            // But recompile the bundle every time because the debug flag may not match what's in the cache
-            return MeasureEvaluator.compileAndEvaluate(FhirContext.forR4(), evaluator.getBundle(), parameters, debug);
+            MeasureEvaluationResult result = MeasureEvaluator.compileAndEvaluate(
+                    FhirContext.forR4(), evaluator.getBundle(), parameters, debug);
+            // Preserve the original wire contract: when no debug sections are
+            // requested, return a bare MeasureReport. Only emit the wrapper for
+            // callers who opted in via the `debug` query parameter.
+            return debug.isEmpty() ? result.getMeasureReport() : result;
         } catch (Exception e) {
             throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage(), e);
         }

Java/measureeval/src/main/java/com/lantanagroup/link/measureeval/converters/DebugSectionsConverter.java

@@ -0,0 +1,24 @@
+package com.lantanagroup.link.measureeval.converters;
+
+import com.lantanagroup.link.measureeval.models.DebugSections;
+import org.springframework.core.convert.converter.Converter;
+import org.springframework.stereotype.Component;
+
+import java.util.Set;
+
+/**
+ * Adapts the {@code ?debug=...} query parameter from raw String to {@code Set<DebugSections>}
+ * at the Spring MVC request-binding edge. Lets controllers declare the strongly-typed parameter
+ * directly instead of parsing the value inline.
+ *
+ * <p>Registered via {@link com.lantanagroup.link.measureeval.configs.WebMvcConfig}.
+ * Delegates parsing to {@link DebugSections#parse(String)} so the wire contract stays in one place.
+ */
+@Component
+public class DebugSectionsConverter implements Converter<String, Set<DebugSections>> {
+
+    @Override
+    public Set<DebugSections> convert(String source) {
+        return DebugSections.parse(source);
+    }
+}

Java/measureeval/src/main/java/com/lantanagroup/link/measureeval/models/DebugSections.java

@@ -0,0 +1,52 @@
+package com.lantanagroup.link.measureeval.models;
+
+import java.util.*;
+
+public enum DebugSections {
+    GROUPS,
+    EXPRESSIONS,
+    LIBRARY_DEBUG,
+    MESSAGES,
+    TRACES,
+    DEBUG_LOG;
+
+    private static final Map<String, DebugSections> ALIASES = Map.of(
+            "groups", GROUPS,
+            "expressions", EXPRESSIONS,
+            "librarydebug", LIBRARY_DEBUG,
+            "messages", MESSAGES,
+            "traces", TRACES,
+            "debuglog", DEBUG_LOG
+    );
+
+    public static Set<DebugSections> parse(String input) {
+        if (input == null || input.isBlank() || "false".equalsIgnoreCase(input.trim())) {
+            return EnumSet.noneOf(DebugSections.class);
+        }
+        String trimmed = input.trim();
+        if ("true".equalsIgnoreCase(trimmed) || "all".equalsIgnoreCase(trimmed)) {
+            return EnumSet.allOf(DebugSections.class);
+        }
+        Set<DebugSections> result = EnumSet.noneOf(DebugSections.class);
+        for (String token : trimmed.split(",")) {
+            String key = token.trim().toLowerCase(Locale.ROOT);
+            DebugSections section = ALIASES.get(key);
+            if (section != null) {
+                result.add(section);
+            }
+        }
+        return result;
+    }
+
+    public static boolean needsDebugLogging(Set<DebugSections> sections) {
+        return sections.contains(EXPRESSIONS)
+                || sections.contains(LIBRARY_DEBUG)
+                || sections.contains(MESSAGES)
+                || sections.contains(DEBUG_LOG);
+    }
+
+    public static boolean needsTracing(Set<DebugSections> sections) {
+        return sections.contains(TRACES)
+                || sections.contains(DEBUG_LOG);
+    }
+}