Merge branch 'master' into json-schema-ssrf-guard

Author: uwezkhan

.formatignore

@@ -12,3 +12,7 @@ tests/integration/schema_registry/data/proto/TestProto_pb2.py
 tests/integration/schema_registry/data/proto/common_proto_pb2.py
 tests/integration/schema_registry/data/proto/exampleProtoCriteo_pb2.py
 tests/integration/schema_registry/data/proto/metadata_proto_pb2.py
+
+# Hand-maintained type stub. tools/style-format.sh runs clang-format on any
+# file that isn't .py, and that corrupts the stub. Keep it out.
+src/confluent_kafka/cimpl.pyi

.semaphore/semaphore.yml

@@ -8,7 +8,7 @@ execution_time_limit:
 global_job_config:
   env_vars:
     - name: LIBRDKAFKA_VERSION
-      value: v2.14.2
+      value: v2.15.0
   prologue:
     commands:
       - checkout
@@ -203,6 +203,7 @@ blocks:
         - name: Build and Tests with 'classic' group protocol
           commands:
             - sem-version python 3.11
+            - sem-version java 17
             - pip install uv
             # use a virtualenv
             - uv venv _venv --python "$(command -v python)" && source _venv/bin/activate
@@ -221,6 +222,7 @@ blocks:
         - name: Build, Test, and Report coverage
           commands:
             - sem-version python 3.11
+            - sem-version java 17
             - pip install uv
             # use a virtualenv
             - uv venv _venv --python "$(command -v python)" && source _venv/bin/activate
@@ -245,6 +247,7 @@ blocks:
           commands:
             - export ARCH=arm64
             - sem-version python 3.11
+            - sem-version java 17
             - pip install uv
             # use a virtualenv
             - uv venv _venv --python "$(command -v python)" && source _venv/bin/activate

CHANGELOG.md

@@ -1,10 +1,48 @@
 # Confluent Python Client for Apache Kafka - CHANGELOG
 
-## v2.xx.x
+## v2.15.0
+
+### [KIP-932](https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka) Queues for Kafka – Now in **Preview**
+
+confluent-kafka-python 2.15.0 adds a **Preview** implementation of the KIP-932 share consumer (Queues for Kafka). Members of a share group cooperatively consume from the same partitions with per-record acquire/acknowledge semantics and redelivery, providing queue-like consumption on top of Kafka.
+
+- New `ShareConsumer` and `DeserializingShareConsumer` clients: subscribe, batch poll (`poll()` returns a `Messages` batch), and close (with context-manager support).
+- Implicit (default) and explicit acknowledgement modes (`share.acknowledgement.mode`), with `AcknowledgeType` ACCEPT / RELEASE / REJECT and `Message.delivery_count()`.
+- Synchronous and asynchronous commit (`commit_sync` / `commit_async`) and an acknowledgement-commit callback.
+- Runtime SASL credential updates (`set_sasl_credentials`) and new `IllegalStateException` / `ConcurrentModificationException` exceptions.
+- (#2217, #2239, #2241, #2252, #2253, #2265, #2273, #2277, #2279)
+
+See the [Share Consumer guide](docs/kip-932-share-consumer.md) and the [Share consumers section of librdkafka's INTRODUCTION.md](https://github.com/confluentinc/librdkafka/blob/master/INTRODUCTION.md#share-consumers-queues-for-kafka).
+
+**Note:** The share consumer is currently in **Preview** and should not be used in production environments. Its public interfaces may change before General Availability, and known limitations apply (see the guide). The share consumer is single-threaded and not thread-safe. It requires a broker with share groups enabled (generally available in Apache Kafka 4.2.0).
+
+### Features
+
+- New optional install `confluent-kafka[oauthbearer-aws]` provides AWS IAM-based
+  OAUTHBEARER authentication via AWS STS `GetWebIdentityToken`. Activate by setting `sasl.oauthbearer.method=oidc`,
+`sasl.oauthbearer.metadata.authentication.type=aws_iam`, and
+`sasl.oauthbearer.config="region=...,audience=..."`. See the
+[AWS IAM OAUTHBEARER guide](docs/oauthbearer-aws.md) for full configuration, and
+[`examples/oauth_oidc_ccloud_aws_iam.py`](examples/oauth_oidc_ccloud_aws_iam.py)
+for a worked example.
+
+### Enhancements
+
+- Add support for union-of-pools and auto pool mapping for Schema Registry (#2182)
 
 ### Fixes
 
-- Fix Encryption fails when expanded union types have two references to the same record (#2262)
+- Fix Encryption fails when expanded union types have two references to the same record (@ChristophMajcenAtXxxlutz, #2262)
+- Make orjson optional in JSON serdes with stdlib json fallback (#2281)
+- Handle non-http errors during retries (#2292)
+- Use TLS certification verification with Hashicorp Vault (#2294)
+
+
+confluent-kafka-python v2.15.0 is based on librdkafka v2.15.0, see the
+[librdkafka release notes](https://github.com/confluentinc/librdkafka/releases/tag/v2.15.0)
+for a complete list of changes, enhancements, fixes and upgrade considerations.
+
+
 
 ## v2.14.2 - 2026-06-03
 

README.md

@@ -24,6 +24,7 @@ Unlike the basic Apache Kafka Python client, `confluent-kafka-python` provides:
 
 - **High Performance & Reliability**: Built on [`librdkafka`](https://github.com/confluentinc/librdkafka), the battle-tested C client for Apache Kafka, ensuring maximum throughput, low latency, and stability. The client is supported by Confluent and is trusted in mission-critical production environments.
 - **Comprehensive Kafka Support**: Full support for the Kafka protocol, transactions, and administration APIs.
+- **Queues for Kafka (Preview)**: A `ShareConsumer` ([KIP-932](https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka)) for queue-like, cooperative consumption with per-record acknowledgement. See the [Share Consumer guide](docs/kip-932-share-consumer.md). **Preview** — not recommended for production.
 - **AsyncIO Producer**: A fully asynchronous producer (`AIOProducer`) for seamless integration with modern Python applications using `asyncio`.
 - **Seamless Schema Registry Integration**: Synchronous and asynchronous clients for Confluent Schema Registry to handle schema management and serialization (Avro, Protobuf, JSON Schema).
 - **Improved Error Handling**: Detailed, context-aware error messages and exceptions to speed up debugging and troubleshooting.
@@ -263,7 +264,10 @@ for topic, f in fs.items():
 ```
 ## Thread safety
 
-The `Producer`, `Consumer`, and `AdminClient` are all thread safe.
+The `Producer`, `Consumer`, and `AdminClient` are all thread safe. The
+`ShareConsumer` (Preview) is **not** thread safe — a single instance must not
+be used concurrently from multiple threads (see the
+[Share Consumer guide](docs/kip-932-share-consumer.md)).
 ## Install
 
 ```bash
@@ -277,6 +281,9 @@ pip install "confluent-kafka[protobuf,schemaregistry]" # Protobuf
 
 # With Data Contract rules (includes CSFLE support)
 pip install "confluent-kafka[avro,schemaregistry,rules]"
+
+# With AWS IAM OAUTHBEARER authentication (mints JWTs via AWS STS GetWebIdentityToken)
+pip install "confluent-kafka[oauthbearer-aws]"
 ```
 
 **Note:** Pre-built Linux wheels do not include SASL Kerberos/GSSAPI support. For Kerberos, see the source installation instructions in [INSTALL.md](INSTALL.md).
@@ -304,6 +311,20 @@ When using Data Contract rules (including CSFLE) add the `rules`extra, e.g.:
 pip install "confluent-kafka[avro,schemaregistry,rules]"
 ```
 
+To authenticate to a Kafka cluster using AWS IAM (when running on EC2, EKS, ECS,
+Fargate, or Lambda with an IAM role attached), add the `oauthbearer-aws` extra:
+
+```bash
+pip install "confluent-kafka[oauthbearer-aws]"
+```
+
+Activation is config-only — set `sasl.oauthbearer.method=oidc`,
+`sasl.oauthbearer.metadata.authentication.type=aws_iam`, and
+`sasl.oauthbearer.config="region=...,audience=..."`. The client mints fresh
+JWTs via AWS STS on every token refresh — no static credentials, no Python-side
+imports. See [`examples/oauth_oidc_ccloud_aws_iam.py`](examples/oauth_oidc_ccloud_aws_iam.py)
+for a worked example.
+
 **Install from source**
 
 For source install, see the *Install from source* section in [INSTALL.md](INSTALL.md).

docs/Makefile

@@ -55,6 +55,12 @@ rst2md:
 	$(PANDOC) --wrap=none --from=rst --to=gfm --output=kip-848-migration-guide.md kip-848-migration-guide.rst
 	@echo
 	@echo "KIP-848 Migration Guide Markdown file generated: kip-848-migration-guide.md"
+	$(PANDOC) --wrap=none --from=rst --to=gfm --output=kip-932-share-consumer.md kip-932-share-consumer.rst
+	@echo
+	@echo "KIP-932 Share Consumer guide Markdown file generated: kip-932-share-consumer.md"
+	$(PANDOC) --wrap=none --from=rst --to=gfm --output=oauthbearer-aws.md oauthbearer-aws.rst
+	@echo
+	@echo "AWS IAM OAUTHBEARER guide Markdown file generated: oauthbearer-aws.md"
 
 html:
 	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html

docs/index.rst

@@ -7,10 +7,13 @@ Guides
    - :ref:`Configuration Guide <pythonclient_configuration>`
    - :ref:`Transactional API <pythonclient_transactional>`
    - :ref:`KIP-848 Migration Guide <pythonclient_migration_kip848>`
+   - :ref:`KIP-932 Share Consumer (Queues for Kafka) <pythonclient_kip932_share_consumer>`
+   - :ref:`AWS IAM OAUTHBEARER <pythonclient_oauthbearer_aws>`
 
 Client API
    - :ref:`Producer <pythonclient_producer>`
    - :ref:`Consumer <pythonclient_consumer>`
+   - :ref:`ShareConsumer <pythonclient_shareconsumer>`
    - :ref:`AdminClient <pythonclient_adminclient>`
    - :ref:`SchemaRegistryClient <schemaregistry_client>`
 
@@ -34,6 +37,8 @@ Supporting classes
     - :ref:`ConsumerGroupState <pythonclient_consumer_group_state>`
     - :ref:`Uuid <pythonclient_uuid>`
     - :ref:`ElectionType <pythonclient_election_type>`
+    - :ref:`Messages <pythonclient_messages>`
+    - :ref:`AcknowledgeType <pythonclient_acknowledge_type>`
 
     - Errors:
        - :ref:`KafkaError <pythonclient_kafkaerror>`
@@ -45,6 +50,8 @@ Supporting classes
        - :ref:`ValueSerializationError <serde_error_serializer_value>`
        - :ref:`KeyDeserializationError <serde_error_deserializer_key>`
        - :ref:`ValueDeserializationError <serde_error_deserializer_value>`
+       - :ref:`IllegalStateException <pythonclient_illegalstateexception>`
+       - :ref:`ConcurrentModificationException <pythonclient_concurrentmodificationexception>`
 
     - Admin API
        - :ref:`NewTopic <pyclient_admin_newtopic>`
@@ -89,6 +96,7 @@ Experimental
 
    - :ref:`SerializingProducer <serde_producer>`
    - :ref:`DeserializingConsumer <serde_consumer>`
+   - :ref:`DeserializingShareConsumer <serde_share_consumer>`
 
 Legacy
    These classes are deprecated and will be removed in a future version of the library.
@@ -427,6 +435,16 @@ Consumer
    :members:
    :noindex:
 
+.. _pythonclient_shareconsumer:
+
+*************
+ShareConsumer
+*************
+
+.. autoclass:: confluent_kafka.ShareConsumer
+   :members:
+   :noindex:
+
 .. _serde_consumer:
 
 ************************************
@@ -438,6 +456,17 @@ DeserializingConsumer (experimental)
 
    :inherited-members:
 
+.. _serde_share_consumer:
+
+*****************************************
+DeserializingShareConsumer (experimental)
+*****************************************
+
+.. autoclass:: confluent_kafka.DeserializingShareConsumer
+   :members:
+
+   :inherited-members:
+
 .. _pythonclient_producer:
 
 ********
@@ -640,6 +669,24 @@ Message
 .. autoclass:: confluent_kafka.Message
    :members:
 
+.. _pythonclient_messages:
+
+********
+Messages
+********
+
+.. autoclass:: confluent_kafka.Messages
+   :members:
+
+.. _pythonclient_acknowledge_type:
+
+***************
+AcknowledgeType
+***************
+
+.. autoclass:: confluent_kafka.AcknowledgeType
+   :members:
+
 .. _pythonclient_topicpartition:
 
 **************
@@ -775,6 +822,24 @@ KafkaException
 .. autoclass:: confluent_kafka.KafkaException
    :members:
 
+.. _pythonclient_illegalstateexception:
+
+**********************
+IllegalStateException
+**********************
+
+.. autoclass:: confluent_kafka.IllegalStateException
+   :members:
+
+.. _pythonclient_concurrentmodificationexception:
+
+********************************
+ConcurrentModificationException
+********************************
+
+.. autoclass:: confluent_kafka.ConcurrentModificationException
+   :members:
+
 .. _pyclient_error_consumer:
 
 ************
@@ -1098,4 +1163,20 @@ https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
 `KIP-848 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol>`_ - Migration Guide
 ==================================================================================================================================================
 
-.. include:: kip-848-migration-guide.rst
+.. include:: kip-848-migration-guide.rst
+
+
+.. _pythonclient_kip932_share_consumer:
+
+`KIP-932 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka>`_ - Share Consumer (Queues for Kafka)
+===============================================================================================================================
+
+.. include:: kip-932-share-consumer.rst
+
+
+.. _pythonclient_oauthbearer_aws:
+
+AWS IAM OAUTHBEARER Authentication
+==================================
+
+.. include:: oauthbearer-aws.rst