Enforce association lifecycle invariants - #4460
Conversation
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
There was a problem hiding this comment.
Pull request overview
This PR enforces association lifecycle invariants in Schema Registry so that associations cannot end up in invalid STRONG/WEAK combinations, and so IMPORT-mode subjects cannot receive schemas via association requests (preventing “successful” no-op schema drops).
Changes:
- Prevent upsert from creating topic-owned (STRONG) associations (with an IMPORT-mode exemption for replication use-cases).
- Reject schemas provided alongside associations when the subject is in IMPORT mode (instead of silently dropping them).
- Enforce uniform lifecycle per resource across association types (reject mixed STRONG/WEAK, while allowing “convert all types together” repairs).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| core/src/main/java/io/confluent/kafka/schemaregistry/storage/AbstractSchemaRegistry.java | Adds server-side enforcement for (1) “upsert can’t create STRONG”, (2) “no schema in IMPORT”, and (3) uniform per-resource lifecycle across association types. |
| core/src/test/java/io/confluent/kafka/schemaregistry/rest/RestApiAssociationTest.java | Adds/updates REST-level integration tests covering mixed-lifecycle rejection, upsert-creation restrictions, and IMPORT-mode schema rejection. |
| client/src/main/java/io/confluent/kafka/schemaregistry/client/rest/entities/requests/AssociationCreateOrUpdateInfo.java | Updates defaulting rules to reject schema on upsert-creation paths (schema implies STRONG/topic-owned). |
| client/src/main/java/io/confluent/kafka/schemaregistry/client/rest/entities/requests/AssociationCreateOrUpdateOp.java | Mirrors the upsert-creation schema rejection/defaulting behavior for batch operation entities. |
| client/src/test/java/io/confluent/kafka/schemaregistry/client/rest/entities/requests/AssociationsRequestTest.java | Adds unit tests validating the new defaulting/validation behavior for create vs upsert-creation scenarios. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 8 changed files in this pull request and generated 1 comment.
Files not reviewed (2)
- core/src/test/java/io/confluent/kafka/serializers/protobuf/test/Ref.java: Generated file
- core/src/test/java/io/confluent/kafka/serializers/protobuf/test/Root.java: Generated file
fdc19ea to
6891131
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
core/src/main/java/io/confluent/kafka/schemaregistry/storage/AbstractSchemaRegistry.java:2057
- In the dry-run create-without-resourceId fallback, the resource selection is based only on association types present in the request. If the most-recent resource sharing (name, namespace, type) does not yet have those types, it will be invisible to the query and an older (e.g., orphaned) resource can be selected instead, causing equivalence / frozen / subject-change checks to compare against the wrong resource. Consider selecting the most-recent resource using all association types first, then filtering to the requested types for the per-type map.
List<Association> fallback = getAssociationsByResourceName(
request.getResourceName(), request.getResourceNamespace(),
request.getResourceType(), new ArrayList<>(infosByType.keySet()), null);
// The fallback can return entries from multiple resourceIds sharing (name, namespace,
// type) — e.g. an orphan from a deleted topic alongside a live one. Narrow to a single
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
core/src/main/java/io/confluent/kafka/schemaregistry/storage/AbstractSchemaRegistry.java:2555
- The comment says a run ends at a repeat of an association type, but the loop only breaks on op-type changes. Since
AssociationOpRequest#validatenow rejects repeated association types within a request, this comment is misleading and should be updated to match the actual behavior.
// A run ends at a change of op type, and also at a repeat of an association type:
// a request may only carry one entry per type, and successive ops on the same type
// are meant to apply one after another rather than collapse into a single request.
f849605 to
a53decf
Compare
|
What
Fixes DGS-24501, DGS-24537, DGS-24538, DGS-24539, DGS-24582, DGS-24631, DGS-24646
Seven association bugs reduce to three invariants that weren't being enforced. Each is
fixed at the point the invariant belongs rather than at the symptom.
Invariant 1 — an upsert may not create a topic-owned (STRONG) association
A STRONG association is owned by its topic and can only be created together with the
topic. Previously an upsert could create one, and would promote to STRONG on its own
whenever a schema was supplied.
IncrementalAlterConfigscreated a topic-owned association on avanilla topic. Now rejected.
SET key={subject, schema}silently created a STRONG association thecaller never asked for. Now rejected.
associations and surfaced as
An association already exists for subject '...', whichis not the real reason. The request now fails on its actual cause.
Enforced in two places: the schema half in
applyDefaultson the request entities, thelifecycle half in
AbstractSchemaRegistrywhere the subject's mode is known.Subjects in IMPORT mode are exempt, so cluster linking can still replicate an
established frozen STRONG association to a destination.
Evolving the schema of an association that already exists — the common alter-configs
flow — is unaffected, since these defaults only apply while creating.
Invariant 2 — a subject in IMPORT mode receives its versions by replication
call reported success having registered nothing. Now rejected.
The existing "subject must have an active version" guard was correct but was skipped
whenever a schema was supplied; with a schema no longer accepted during IMPORT, that
guard applies and the association is refused until the subject's versions land.
Invariant 3 — a resource has one lifecycle across all association types
A topic is either topic-owned (STRONG) or shared (WEAK), never a mix.
CreateAssociationaccepted a shared key plus a topic-owned value inone request.
IncrementalAlterConfigsadded a shared association to a topic thatalready had a topic-owned one.
checkUniformLifecyclecomputes the lifecycle the resource would end up with per type— from the request where it supplies one, from the existing association where it does
not, and from storage for types the request leaves alone — and rejects anything that
leaves the resource mixed. Converting every type in one request is therefore allowed.
Two details that make this work:
a sibling type is invisible to it. The check re-queries all types, which is what makes
DGS-24539 detectable.
matched by (name, namespace). Several resources can share those — an orphan from a
deleted topic beside a live one — so
mostRecentlyUpdatedResourcenarrows to oneresource before evaluating. Merging per type instead would stitch types across
resources and report a mix that never existed. The pre-existing by-name fallback now
goes through the same helper, so the equivalence, subject-change and frozen checks all
judge the same resource.
Behavior changes
IncrementalAlterConfigswhile passing a schema nowfails; topic-owned associations must be created with the topic.
IMPORT mode.
ignored.
Checklist
Please answer the questions with Y, N or N/A if not applicable.
References
JIRA:
Test & Review
Open questions / Follow-ups