Conversation
- Records the proposed (not accepted) design for asset-level, read-only SPARQL access requested in OpenMetadata #33224, distinguishing it from the existing admin-only and glossary-scoped query paths. - Captures source-derived feasibility findings, the A/B/C/D visibility example, alternative architectures, and open decisions requiring owner sign-off before implementation. - Lays out a small-commit implementation roadmap and cross-repo contract checklist with ai-platform #1299 to guide future work.
- Captures a bounded web/source research pass on Jena/Fuseki access control facilities (jena-permissions, Fuseki graph ACL, TDB tuple filters, DatasetGraphFilteredView) evaluated against the repo's pinned Jena 6.2.0 and current RDF projection architecture. - Compares candidates and recommends a request-local sanitized-model approach over adapter-based filtering, since only the former confines reads by construction without unverified engine coverage. - Proposes a minimal, unapproved experiment with concrete acceptance tests to validate the recommended approach before it becomes an ADR. - Feeds the pending ADR at docs/adr/2026-09-14-authorized-sparql.md and flags open engineering unknowns (predicate-to-field mapping, rebuild consistency, cross-pod cache invalidation) that block it.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
🔴 Playwright Results — workflow failedValidated commit ✅ 0 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky ⚙️ No test failures. This run failed in CI infrastructure or reporting (see Pipeline and setup failures), not because of your changes — no test action needed from you. Pipeline and setup failures (6)
Performance⚪ Performance metrics unavailable; see the CI and reporting failures above.
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
Adds an opt-in Testcontainers-based subclass that reruns the sanitized sanitized-model experiment suite with retrieval against an isolated, memory-capped Fuseki 6.2.0 container instead of in-process Jena, closing the previously open question of whether the approach holds against a real triple store. Updates the ADR and experiment writeup with the run results, commands, and a new finding that TDB2 canonicalizes literal terms, which required reading the unrestricted reference through the same store to keep comparisons valid.
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
| @@ -0,0 +1,445 @@ | |||
| package org.openmetadata.service.rdf; | |||
There was a problem hiding this comment.
💡 Quality: New Java files missing Apache-2.0 license header
The six new .java files start directly with the package statement, while existing test files in this tree (e.g. AsyncOperationsConfigurationTest.java, BindJsonTest.java) begin with the standard Apache-2.0 / Collate license header block, which the repo's Java conventions require. Add the standard header to each new file so spotless/checkstyle and the license convention pass.
Was this helpful? React with 👍 / 👎
fmcardoso
changed the title
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
| @@ -0,0 +1,614 @@ | |||
| package org.openmetadata.service.rdf; | |||
There was a problem hiding this comment.
💡 Quality: New IT file missing Apache-2.0 license header
RdfAuthorizationAlignmentIT.java starts directly with the package declaration and has no Apache-2.0 license header, which java.md requires for all Java files (and which most IT files in this tree carry). Add the standard Collate/Apache-2.0 header block above the package statement.
Was this helpful? React with 👍 / 👎
Code Review 👍 Approved with suggestions 0 resolved / 2 findingsResearch and prototype for asset-level SPARQL RBAC with comprehensive documentation, ADR, design alternatives and 112 passing tests. Add Apache-2.0 license headers to the six new Java test files ( 💡 Quality: New Java files missing Apache-2.0 license header📄 openmetadata-service/src/test/java/org/openmetadata/service/rdf/SanitizedModelBuilder.java:1 📄 openmetadata-service/src/test/java/org/openmetadata/service/rdf/SanitizedModelExperimentTest.java:1 📄 openmetadata-service/src/test/java/org/openmetadata/service/rdf/SanitizedModelFixture.java:1 📄 openmetadata-service/src/test/java/org/openmetadata/service/rdf/SanitizedModelFusekiTest.java:1 📄 openmetadata-service/src/test/java/org/openmetadata/service/rdf/SanitizedQueryProfile.java:1 📄 openmetadata-service/src/test/java/org/openmetadata/service/security/policyevaluator/PolicyContextFixture.java:1 The six new 💡 Quality: New IT file missing Apache-2.0 license headerRdfAuthorizationAlignmentIT.java starts directly with the package declaration and has no Apache-2.0 license header, which java.md requires for all Java files (and which most IT files in this tree carry). Add the standard Collate/Apache-2.0 header block above the package statement. 🤖 Prompt for agentsOptionsDisplay: compact → Counting what did not apply, without listing it. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
1 participant
Summary
Related to #33224 · Companion chat integration: https://github.com/open-metadata/ai-platform/issues/1299
Draft for design/security/performance feedback—not production asset RBAC. This PR adds research, a proposed ADR, and a test-only experiment for read-only SPARQL over caller-authorized RDF facts. Existing admin-only and glossary-scoped boundaries are unchanged. No production endpoint, schema, POM, configuration or UI changes are introduced. This PR should not close #33224 yet.
Reading guide
Required semantics
Use the effective authenticated caller and ordinary OpenMetadata discovery, field and detail restrictions. There is no least-restrictive override for RDF.
Authorization must apply before evaluation, not by filtering result rows: a hidden asset can otherwise change COUNT/ASK, joins, EXISTS, or connect two visible assets through a path. For example, if A→B→C and A→D, and B is hidden, A has one visible direct upstream and no visible path to C through B.
Candidate design and why we selected it for the experiment
Why this baseline: once admission and query confinement are correct, hidden facts are physically absent from the model the query reads. It reuses installed Jena ARQ and the existing local-query pattern, without initially repartitioning the projection or shipping policy decisions across a new Fuseki trust boundary. Unknown mappings, ownership conflicts and retrieval-budget overflow fail closed.
This is a simplicity/auditability choice, not a demonstrated performance win or approved production architecture. The complete field mapping and the query profile still need security validation.
Alternatives and the additional primary-source web check
DatasetGraphFilteredView: predicate-filtered dataset access.SecurityContextView: query-context filtering exists.Important correction from the second pass: Fuseki is not inherently limited to a static registry. Its 6.2.0
AuthorizationServiceis pluggable andDataAccessCtl.controlledDatasetaccepts it. Dynamic integration need not mirror policies into static ACLs. The research addendum, pushed in4e409314cd7, corrects the earlier overbroad static-registry characterization and records the additional sources and performance decision gate.Research conclusion: this bounded second pass found no clearly better drop-in solution. It does keep custom server-side filtering and a narrower query contract on the table if measurements reject local materialization. It is not an exhaustive survey or a proof of safety for any alternative.
Evidence now included in the PR
112 tests passed locally: 56 in-process + the same 56 with Fuseki-backed retrieval, zero failures/errors/skips. This is a recorded worktree run, not a claim about PR CI.
PolicyEvaluator, but test-supplied catalog/policy context—not full caller/role resolution.graph/knowledge; overflow and unknown fact mappings fail closed.The test report includes commands, image provenance and formatting verification. TDB2 literal canonicalization required reading the unrestricted reference through the same store. One arm64 run on four tables and coarse container-memory samples are not scale or per-request latency evidence. No production API, ingestion or UI tests apply to this test-only scope.
Concrete gaps found
om:labelType/om:tagStateare projected onto shared tag nodes, losing the owning asset context needed for authorization. Normal populated attributes currently fail closed.VIEW_BASIC. Finer-grained usage/sample-data/tests/queries/custom-property restrictions are not demonstrated.Latency and scalability: decision gate, not a solved problem
The initial target is roughly 2 seconds for the complete server-side graph request, including candidate discovery, authorization, RDF retrieval, model construction, evaluation and serialization. LLM SPARQL generation and answer composition are excluded. Scope, concurrency and the eventual SLO percentile still need agreement.
We do not yet know whether this approach meets that target. Per-asset policy/DB work and copying an authorized graph can both dominate; existing caches do not prove otherwise.
Next, combine real authorization integration with a bounded performance experiment:
Do not meet the target with silent candidate truncation, partial aggregates, weaker permissions, or an unreviewed cross-request model cache. Earlier experiment-document wording that postpones all scale measurements until mapping is complete is superseded by this combined integration/performance next step.
Open contract decision: discovery versus direct-read permissions
Unresolved: should SPARQL include assets the caller can discover through search/browse, assets they can directly read, or an explicitly defined combination?
The code investigation found different authorization paths: direct reads evaluate policies against a specific resource, while discovery uses type-level checks and filtering. Search-policy compilation may skip unsupported conditions. Consequently, discovery results must not be assumed equivalent to direct-read permissions: omitted deny conditions can admit extra candidates, while omitted allow conditions can exclude readable assets.
Rechecking every candidate can remove unauthorized assets, but cannot recover assets missing from candidate discovery. This matters for complete COUNT/ASK answers. Full policy compilation alone also does not establish completeness when the search index may lag.
This is a clarification required to implement “ordinary OpenMetadata permissions,” not approval to choose the more permissive path or silently redefine discovery. The first direct-read alignment test will provide evidence, not settle the product contract.
Questions for peers
Checklist
UI screenshots: not applicable.
Summary by Gitar
This will update automatically on new commits.