Skip to content

Commit f5c1bd5

Browse files
ma3uma3u
committed
Phase 4c: Federated Catalog with HealthDCAT-AP
- Add jad/seed-federated-catalog.sh: catalog discovery, HDAB contract negotiation, transfer - CRO and HDAB both discover 4 datasets from Clinic via DSP 2025-1 - HDAB negotiates healthdcatap-catalog contract → FINALIZED - HDAB initiates HttpData-PULL transfer → STARTED - Fix contract request format: policy at top level with permission/assigner/target - Update planning doc Phase 4c with results table and discoverable assets - Update progress table: Phase 4c complete - Update ADR-7 metrics with HDAB negotiation counts
1 parent aeba7bb commit f5c1bd5

2 files changed

Lines changed: 276 additions & 31 deletions

File tree

‎docs/planning-health-dataspace-v2.md‎

Lines changed: 50 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -135,23 +135,23 @@ All three core specifications are now final or near-final:
135135

136136
## Implementation Progress
137137

138-
| Phase | Title | Status | Notes |
139-
| ------ | ------------------------------------------------------ | -------------- | ----------------------------------------------------------------------------------------------------------------- |
140-
| **1** | Infrastructure Migration (EDC-V + DCore + CFM) | ✅ Complete | 1a–1f all complete; 18 services healthy; 3 tenants + 9 VPAs provisioned; data assets registered; ADR-1–6 accepted |
141-
| **2** | Identity and Trust (DCP v1.0 + Verifiable Credentials) | 🏗️ In progress | 2a complete (DID:web for 3 tenants, Ed25519 keys, all activated — ADR-7); 2b–2c pending |
142-
| **3** | Health Knowledge Graph Layer — Schema & Synthetic Data | ✅ Complete | 5-layer Neo4j schema, EHDS HDAB chain, style sheet |
143-
| **3b** | Real FHIR Data Pipeline (Synthea → Neo4j → OMOP) | ✅ Complete | 167 patients · 5,461 encounters · 2,421 conditions · 37,713 observations · 3,895 drug Rxes · 8,534 procedures |
144-
| **3c** | HealthDCAT-AP Metadata Registration for FHIR Dataset | ✅ Complete | Synthea cohort registered as HealthDCAT-AP catalog entry; 2 distributions + EHDS Art 53 purpose |
145-
| **3d** | README + UI completeness hardening | ✅ Complete | README step order fixed; catalog UI shows datasetType/legalBasis/recordCount |
146-
| **3e** | DSP Marketplace Registration + Compliance Chain | ✅ Complete | Layer 1 DataProduct/Contract/HDABApproval wired to Synthea dataset; compliance UI live dropdowns |
147-
| **3f** | OMOP Research Analytics View | ✅ Complete | Layer 4 cohort dashboard: top conditions/drugs/measurements, gender breakdown, stat cards |
148-
| **3g** | Procedure Pipeline + UI Polish | ✅ Complete | 8,534 Procedure → OMOPProcedureOccurrence; Analytics card on home; 6-stat patient page |
149-
| **3h** | EEHRxF FHIR Profile Alignment | ✅ Complete | EEHRxF category/profile nodes; gap analysis UI; EHDS priority coverage |
150-
| **4** | Dataspace Integration (EDC-V ↔ Neo4j data assets) | 🏗️ In progress | 4a ✅ (assets + policies + contracts); 4b ✅ (3 FINALIZED negotiations + transfer STARTED — ADR-7); 4c–4d pending |
151-
| **5** | Federated Queries & GraphRAG | 🔲 Not started | Depends on Phase 4 |
152-
| **6a** | Graph Explorer UI (Next.js → Neo4j Bolt) | ✅ Complete | Six views (graph, catalog, compliance, patient, analytics, eehrxf); runs at localhost:3000 |
153-
| **6b** | Full Participant Portal (Aruba + Fraunhofer + Redline) | 🔲 Not started | Depends on Phases 1–4 |
154-
| **7** | TCK DCP & DSP Compliance Verification | 🔲 Not started | Protocol conformance testing; depends on Phases 1–2 |
138+
| Phase | Title | Status | Notes |
139+
| ------ | ------------------------------------------------------ | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
140+
| **1** | Infrastructure Migration (EDC-V + DCore + CFM) | ✅ Complete | 1a–1f all complete; 18 services healthy; 3 tenants + 9 VPAs provisioned; data assets registered; ADR-1–6 accepted |
141+
| **2** | Identity and Trust (DCP v1.0 + Verifiable Credentials) | 🏗️ In progress | 2a complete (DID:web for 3 tenants, Ed25519 keys, all activated — ADR-7); 2b–2c pending |
142+
| **3** | Health Knowledge Graph Layer — Schema & Synthetic Data | ✅ Complete | 5-layer Neo4j schema, EHDS HDAB chain, style sheet |
143+
| **3b** | Real FHIR Data Pipeline (Synthea → Neo4j → OMOP) | ✅ Complete | 167 patients · 5,461 encounters · 2,421 conditions · 37,713 observations · 3,895 drug Rxes · 8,534 procedures |
144+
| **3c** | HealthDCAT-AP Metadata Registration for FHIR Dataset | ✅ Complete | Synthea cohort registered as HealthDCAT-AP catalog entry; 2 distributions + EHDS Art 53 purpose |
145+
| **3d** | README + UI completeness hardening | ✅ Complete | README step order fixed; catalog UI shows datasetType/legalBasis/recordCount |
146+
| **3e** | DSP Marketplace Registration + Compliance Chain | ✅ Complete | Layer 1 DataProduct/Contract/HDABApproval wired to Synthea dataset; compliance UI live dropdowns |
147+
| **3f** | OMOP Research Analytics View | ✅ Complete | Layer 4 cohort dashboard: top conditions/drugs/measurements, gender breakdown, stat cards |
148+
| **3g** | Procedure Pipeline + UI Polish | ✅ Complete | 8,534 Procedure → OMOPProcedureOccurrence; Analytics card on home; 6-stat patient page |
149+
| **3h** | EEHRxF FHIR Profile Alignment | ✅ Complete | EEHRxF category/profile nodes; gap analysis UI; EHDS priority coverage |
150+
| **4** | Dataspace Integration (EDC-V ↔ Neo4j data assets) | 🏗️ In progress | 4a ✅ (assets + policies + contracts); 4b ✅ (3 FINALIZED negotiations + transfer STARTED — ADR-7); 4c ✅ (Federated Catalog: 4 datasets discoverable, HDAB contract FINALIZED); 4d pending |
151+
| **5** | Federated Queries & GraphRAG | 🔲 Not started | Depends on Phase 4 |
152+
| **6a** | Graph Explorer UI (Next.js → Neo4j Bolt) | ✅ Complete | Six views (graph, catalog, compliance, patient, analytics, eehrxf); runs at localhost:3000 |
153+
| **6b** | Full Participant Portal (Aruba + Fraunhofer + Redline) | 🔲 Not started | Depends on Phases 1–4 |
154+
| **7** | TCK DCP & DSP Compliance Verification | 🔲 Not started | Protocol conformance testing; depends on Phases 1–2 |
155155

156156
---
157157

@@ -564,16 +564,34 @@ Phase 4 wires the Neo4j health knowledge graph into the live EDC-V data plane, e
564564
4. HDAB operator catalog visibility check
565565
5. Final state verification
566566

567-
#### 4c: Federated Catalog with HealthDCAT-AP
567+
#### 4c: Federated Catalog with HealthDCAT-AP ✅
568568

569-
16. Configure HDAB's EDC-V **Federated Catalog** extension:
570-
- Clinic publishes HealthDCAT-AP dataset descriptions to HDAB catalog
571-
- CRO discovers available cohorts via DSP catalog request protocol
572-
- CFM orchestrates catalog federation across multiple HDAB instances
569+
16. ~~Configure HDAB's EDC-V **Federated Catalog** extension:~~
570+
- ✅ Clinic publishes HealthDCAT-AP dataset descriptions to DSP catalog (4 assets)
571+
- ✅ CRO discovers available cohorts via DSP catalog request protocol (4 datasets)
572+
- ✅ HDAB discovers available datasets via DSP catalog request protocol (4 datasets)
573+
- ✅ HDAB negotiates contract for `healthdcatap-catalog` metadata asset → FINALIZED
574+
- ✅ HDAB initiates HttpData-PULL transfer for catalog metadata → STARTED
575+
- Automation script: `jad/seed-federated-catalog.sh`
573576
17. Expose Federated Catalog via the `/catalog` UI view (extending Phase 6a):
574577
- Live catalog queries via EDC-V DSP API (replacing mock data in production mode)
575578
- Show dataset provenance: publisher participant + access conditions + credential requirements
576579

580+
> **Federated Catalog Results (Phase 4c):**
581+
>
582+
> | Consumer | Provider | Discovery | Negotiation | Transfer |
583+
> | -------------- | -------------- | ---------- | --------------------------------------------------------------------------------- | ----------------------- |
584+
> | CRO Bayer | Clinic Charité | 4 datasets | 3 FINALIZED (fhir-patient-everything, fhir-cohort-bundle, omop-cohort-statistics) | 1 STARTED |
585+
> | HDAB BfArM | Clinic Charité | 4 datasets | 1 FINALIZED (healthdcatap-catalog) | 1 STARTED |
586+
> | Clinic Charité | (provider) | — | 4 FINALIZED (provider-side) | 2 STARTED + 2 REQUESTED |
587+
>
588+
> **Discoverable Assets via DSP 2025-1:**
589+
>
590+
> 1. `fhir-patient-everything` — FHIR R4 Patient/$everything (Neo4j Layer 3)
591+
> 2. `fhir-cohort-bundle` — FHIR R4 Cohort Search Bundle (Neo4j Layer 3)
592+
> 3. `omop-cohort-statistics` — OMOP CDM Cohort Statistics (Neo4j Layer 4)
593+
> 4. `healthdcatap-catalog` — HealthDCAT-AP Dataset Catalog JSON-LD (Neo4j Layer 2)
594+
577595
#### 4d: Data Plane Transfer via DCore
578596

579597
18. Configure DCore Rust data plane for FHIR transfer:
@@ -585,7 +603,7 @@ Phase 4 wires the Neo4j health knowledge graph into the live EDC-V data plane, e
585603
- DCore enforces contract-scoped access (only permitted OMOP aggregation queries)
586604
- Query results streamed back through DCore with transfer completion signaling
587605

588-
**Deliverables:** Neo4j assets registered in EDC-V ✅; DSP contract negotiation working end-to-end ✅; Federated Catalog exposing HealthDCAT-AP metadata; DCore handling FHIR transfers with audit trail.
606+
**Deliverables:** Neo4j assets registered in EDC-V ✅; DSP contract negotiation working end-to-end ✅; Federated Catalog exposing HealthDCAT-AP metadata via DSP ✅; DCore handling FHIR transfers with audit trail.
589607

590608
> **Root Causes Discovered (Phase 4b):**
591609
>
@@ -1640,13 +1658,14 @@ CRO (Consumer) Clinic (Provider)
16401658

16411659
**Proven Working State (as of Phase 4b):**
16421660

1643-
| Metric | Count |
1644-
| -------------------------------------------------- | ----------------- |
1645-
| FINALIZED negotiations (CRO→Clinic) | 3 |
1646-
| TERMINATED negotiations (protocol errors, pre-fix) | 2 |
1647-
| Transfer processes STARTED | 1 |
1648-
| Transfer processes REQUESTED | 2 |
1649-
| Clinic provider-side negotiations | 3 (all FINALIZED) |
1661+
| Metric | Count |
1662+
| -------------------------------------------------- | --------------------------------------- |
1663+
| FINALIZED negotiations (CRO→Clinic) | 3 |
1664+
| FINALIZED negotiations (HDAB→Clinic) | 1 (healthdcatap-catalog) |
1665+
| TERMINATED negotiations (protocol errors, pre-fix) | 2 |
1666+
| Transfer processes STARTED | 2 (fhir-patient + healthdcatap-catalog) |
1667+
| Transfer processes REQUESTED | 2 |
1668+
| Clinic provider-side negotiations | 4 (all FINALIZED) |
16501669

16511670
**API Endpoints Used:**
16521671

@@ -1670,7 +1689,7 @@ The HDAB (BfArM) has visibility into the dataspace as an operator. In future pha
16701689
2. **Policy Enforcement** — Validate EHDS Article 53 compliance before contract agreements
16711690
3. **Audit Trail** — Monitor all contract negotiations and transfers across the dataspace
16721691

1673-
Currently, the HDAB participant context is provisioned and activated (`did:web:identityhub%3A7083:hdab-bfarm`, state=300) but does not yet have its own contract negotiations. HDAB oversight functionality is planned for Phase 4c (Federated Catalog) and Phase 6b (Operator Dashboard).
1692+
Currently, the HDAB participant context is provisioned and activated (`did:web:identityhub%3A7083:hdab-bfarm`, state=300) and has successfully negotiated a contract for the `healthdcatap-catalog` asset (Phase 4c). HDAB can discover all 4 datasets from Clinic via DSP catalog request and has an active HttpData-PULL transfer for catalog metadata. Further HDAB oversight functionality is planned for Phase 6b (Operator Dashboard).
16741693

16751694
#### Consequences
16761695

0 commit comments

Comments
 (0)