perf: Analyze rules before running to limit context [DHIS2-21245]

[enricocolasante]

DHIS2-21245 — Use rule engine analyzer to build SupplementaryData and RuleEngineContext once per program

Summary

This PR optimises the tracker import rule engine by building the RuleEngineContext once per
program per request (instead of once per enrollment/event), gating expensive DB fetches behind
flags from the rule-engine-jvm analyzer, and replacing the org unit group lookup strategy with
one driven by the org units in the import payload.

---

Changes

Rule engine context built once per program

DefaultProgramRuleService now calls programRuleEngine.analyzeContextRequirements(rules, variables)
once per program to obtain a RuleContextRequirements object. The RuleEngineContext (rules,
variables, supplementary data, constants) is constructed once and reused for every enrollment and
event belonging to that program.

needsAllEvents gate

RuleContextRequirements.getNeedsAllEvents() is true only when rules reference
newest-event / previous-event variables or V{event_count}. Saved events are fetched from the
database only for programs where this flag is set, eliminating the DB call entirely for programs
that don't need it.

Supplementary data driven by payload org units

RuleContextRequirements.getNeedsOrgUnitGroups() is a boolean. When true,
SupplementaryDataProvider now:

1. Collects all org unit UIDs from payload enrollments, tracker events, single events, and
   preheat-resolved saved enrollments.
2. Runs a single JDBC query — WHERE ou.uid = ANY(:orgUnitUids) — returning, for each org unit,
   the groups it belongs to (keyed by both group UID and group code).

The old approach extracted group identifiers from rule condition strings via regex and loaded each
group through OrganisationUnitGroupService.getOrganisationUnitGroup(String), which delegates to
getByUid and pulls full Hibernate entity graphs.

---

Bug fix: d2:inOrgUnitGroup now resolves by UID or code

Previously, d2:inOrgUnitGroup('identifier') silently returned false for any non-UID
identifier. The old implementation called getOrganisationUnitGroup(String) which delegates to
getByUid, so only exact UID matches worked.

The new JDBC query matches on both oug.uid and oug.code, so rule authors can now write
d2:inOrgUnitGroup('MY_GROUP_CODE') and it will work as expected.

---

Performance

The load profile runs 4 concurrent import users, 15 s ramp + 180 s sustained, across three
import scenarios — 60 requests each, using custom TrackerTest version with different metadata:

Scenario	Program	UID
ANC import	Antenatal care visit (event program)	lxAQ7Zs9VYR
Child Programme import	Child Programme (tracker program)	IpHINAT79UW
MNCH import	MNCH / PNC (tracker program)	uy2gU8kT1jF

Before the test runs, all program rule variables for ANC (lxAQ7Zs9VYR) and
Child Programme (IpHINAT79UW) are patched to source type DATAELEMENT_CURRENT_EVENT.
This ensures needsAllEvents = false for those programs, so the test exercises the fast
path introduced by this PR.

Three program rules using d2:inOrgUnitGroup were added to the SL demo DB to exercise the new
supplementary data path:

Rule	Program	Condition	Action
Perf test - in Public facilities	ANC (lxAQ7Zs9VYR)	d2:inOrgUnitGroup('oRVt7g429ZO')	SHOWWARNING
Perf test - in Rural	ANC (lxAQ7Zs9VYR)	d2:inOrgUnitGroup('GGghZsfu7qV')	SHOWWARNING
Perf test - in Urban	Child Programme (IpHINAT79UW)	d2:inOrgUnitGroup('f25dqv3Y7Z0')	SHOWWARNING

MNCH has no program rules, so the rule engine is skipped for it entirely. It serves as a control:
any latency change there reflects general import overhead, not rule engine behaviour.

Allocation (run 24345151024)

	Baseline	Candidate	Delta	Delta %
Total	19.83 GB	13.22 GB	−6.62 GB	−33.4%
DefaultProgramRuleService	7.44 GB	0.66 GB	−6.77 GB	−91.1%
% of total	37.5%	5.0%

The 91% reduction in DefaultProgramRuleService allocations comes from two sources:

• The SupplementaryDataProvider no longer loads Hibernate entity graphs for org unit groups on
  every call — a single JDBC query replaces repeated getByUid calls that each inflated an
  OrganisationUnitGroup with all its associations.
• The needsAllEvents gate eliminates the fetchSavedRuleEventsByEnrollment DB fetch for
  programs (ANC, Child Programme) whose rules don't reference event-history variables.

CPU (run 24348612905)

	Baseline	Candidate	Delta	Delta %
Total samples	7849	6446	−1403	−17.9%

Notable decreases by DHIS2 leaf frame:

Frame	Baseline	Candidate	Delta
Calculator.evalFunction	25	0	−25
TrackerBundle$$Lambda…test	15	0	−15

Calculator.evalFunction disappearing entirely from the candidate confirms the needsAllEvents
gate is working — the rule expression evaluator is no longer called for programs that don't need
saved events.

Latency (run 24345151024, run 24348612905)

Program	Metric	Baseline	Candidate	Delta	Delta %
ANC	p95	892ms / 971ms	665ms / 517ms	−227ms / −454ms	−25% / −47%
Child Programme	p95	932ms / 907ms	550ms / 548ms	−382ms / −359ms	−41% / −40%
MNCH	p95	1195ms / 1136ms	1231ms / 1140ms	+36ms / +4ms	+3% / +0.4%

ANC and Child Programme see consistent 25–47% p95 improvements. MNCH is essentially flat — it
has more complex rules and its saved-event fetch path is still exercised.

---

Open question: SL demo DB program rule variable type

The performance test patches program rule variables for ANC and Child Programme to
DATAELEMENT_CURRENT_EVENT at runtime to exercise the needsAllEvents = false path. Should we
update the SL demo DB to permanently set the existing program rule variables for those programs to
DATAELEMENT_CURRENT_EVENT, so that the test no longer needs to patch them and the demo data
reflects a more typical real-world configuration?

[teleivo]

maybe unlikely but what if a groups code matches anothers uid?

[enricocolasante]

Bad things will happen 😨
We need to work with the assumption that it will not happen because the grammar itself has no way to differentiate between a UID and a CODE.
If we have an orgUnitGroupA with UID h4w96yEMlzO and orgUnitGroupB with CODE h4w96yEMlzO, then a rule using a condition like d2:inOrgUnitGroup('h4w96yEMlzO') has no way to know what the user meant.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud