Cache parsed FHIRPath expressions - #117
Conversation
evaluateExpression re-ran the ANTLR lexer and parser on every call. For the search-parameter indexing path, where a small fixed set of expressions is evaluated against every resource, that re-parse is the dominant cost (measured ~6-10x faster eval once cached on a representative Patient/Observation search-param set). Cache the parse tree by expression string. The tree is read-only during evaluation and initialize() already resets all per-eval state, so reuse is semantics-preserving; the HL7 spec suite passes unchanged. Invalid expressions still throw before the cache stores, so error behaviour is unchanged. The cache is unbounded, sized by the caller's distinct expressions (a fixed search-parameter set in the indexing use case).
|
I did some bechmark with claude see the source for the benchmark and the report. FHIRPath parse cache: evaluation throughputFor fifteen distinct expressions each parsed 20,000 times resulting to 300k evaluation.
Parsing accounted for approximately 89% of per-evaluation cost. It is now incurred once per distinct ApplicabilityThe measured workload is the search-parameter indexing pattern: a fixed expression set evaluated The improvement is proportional to the expression reuse ratio. A caller evaluating each expression MeasurementsFive runs per variant, each in a fresh JVM.
The distributions do not overlap: the slowest cached run (486 ms) exceeds the fastest uncached run Procedure
Environment: JDK 21.0.8 (Temurin), Linux 6.8, 16 cores, 31 GB RAM. Limitations: measurement covers the JVM target only; the JS, Wasm and Native targets were not Benchmark source
package dev.ohs.fhir.fhirpath
import dev.ohs.fhir.model.r4.Resource
import kotlin.test.Test
import kotlin.time.measureTime
import kotlinx.serialization.json.Json
/**
* Ad-hoc throughput benchmark for repeated evaluation of a fixed expression set (the engine's
* search-parameter indexing pattern). Prints a `FHIRPATHBENCH` line; not an assertion test.
*/
class ParseCacheBenchmark {
private val json = Json { ignoreUnknownKeys = true }
private val patient: Resource =
json.decodeFromString(
"""{"resourceType":"Patient","id":"p1","active":true,"gender":"female","birthDate":"1990-01-01",
"name":[{"family":"Otieno","given":["Amina"]}],"address":[{"city":"Kibera"}],
"identifier":[{"system":"urn:id","value":"X1"}],
"managingOrganization":{"reference":"Organization/o1"}}"""
)
private val observation: Resource =
json.decodeFromString(
"""{"resourceType":"Observation","id":"o1","status":"final",
"code":{"coding":[{"system":"http://loinc.org","code":"29463-7"}]},
"subject":{"reference":"Patient/p1"},"encounter":{"reference":"Encounter/e1"},
"effectiveDateTime":"2026-01-01","valueQuantity":{"value":50,"unit":"kg"}}"""
)
private val patientExprs =
listOf(
"Patient.name",
"Patient.name.given",
"Patient.name.family",
"Patient.birthDate",
"Patient.gender",
"Patient.active",
"Patient.address.city",
"Patient.identifier.value",
"Patient.managingOrganization.reference",
)
private val obsExprs =
listOf(
"Observation.code.coding.code",
"Observation.subject.reference",
"Observation.encounter.reference",
"Observation.status",
"Observation.effectiveDateTime",
"Observation.valueQuantity.value",
)
@Test
fun benchmark() {
val engine = FhirPathEngine.forR4()
val work = listOf(patient to patientExprs, observation to obsExprs)
val exprsPerRound = work.sumOf { it.second.size }
val iterations = 20_000
repeat(2_000) {
work.forEach { (res, exprs) -> exprs.forEach { engine.evaluateExpression(it, res) } }
}
val evals = iterations.toLong() * exprsPerRound
val elapsed = measureTime {
repeat(iterations) {
work.forEach { (res, exprs) -> exprs.forEach { engine.evaluateExpression(it, res) } }
}
}
val ns = elapsed.inWholeNanoseconds
val perEvalNs = ns / evals
val evalsPerSec = evals * 1_000_000_000L / ns
println(
"FHIRPATHBENCH evals=$evals elapsed_ms=${elapsed.inWholeMilliseconds} " +
"per_eval_ns=$perEvalNs evals_per_sec=$evalsPerSec"
)
}
} |
@jingtang10 Concerning your question about where this improvement is applicable and when it is not. The 9x is not really a constant and would be different, depending on the expression and number of evaluations being run. |
1 participant
evaluateExpression re-ran the ANTLR lexer and parser on every call. For the search-parameter indexing path, where a small fixed set of expressions is evaluated against every resource, that re-parse is the dominant cost (measured ~6-10x faster eval once cached on a representative Patient/Observation search-param set).
Cache the parse tree by expression string. The tree is read-only during evaluation and initialize() already resets all per-eval state, so reuse is semantics-preserving; the HL7 spec suite passes unchanged. Invalid expressions still throw before the cache stores, so error behaviour is unchanged. The cache is unbounded, sized by the caller's distinct expressions (a fixed search-parameter set in the indexing use case).