Consumers with different groups create separate MQClientInstance per binding, causing thread explosion

[wxpptang]

When using spring-cloud-starter-stream-rocketmq with many consumer bindings (e.g. 70+), each consumer creates its own MQClientInstance, resulting in massive thread overhead. In our production service with 71 consumer bindings, the JVM spawned ~1,700 threads solely from RocketMQ (total 2,200+), most of which
are redundant Netty I/O, Rebalance, PullMessage, and Heartbeat threads.

Root Cause

RocketMQConsumerFactory.initPushConsumer() calls RocketMQUtils.getInstanceName() to set each consumer's instanceName:

// RocketMQConsumerFactory.java line 77-78
consumer.setInstanceName(
RocketMQUtils.getInstanceName(rpcHook, consumerProperties.getGroup()));

RocketMQUtils.getInstanceName() appends System.nanoTime() to ensure uniqueness:

// RocketMQUtils.java line 85-86
instanceName.append(identify).append(separator).append(UtilAll.getPid())
.append(separator).append(Long.toString(System.nanoTime(), 36));

Since each consumer gets a unique instanceName, they each generate a unique clientId (IP@instanceName), and MQClientManager.getOrCreateMQClientInstance() creates a separate MQClientInstance for each one. Every MQClientInstance spawns ~20-24 threads (Netty EventLoop, PullMessageService, RebalanceService,
ScheduledExecutor, etc.).

Why this is unnecessary

RocketMQ's MQClientInstance.consumerTable uses consumer group name as the key. Consumers with different groups connecting to the same NameServer can safely share a single MQClientInstance — this is RocketMQ's native supported behavior. The unique instanceName per consumer is unnecessary when groups are already
distinct.

Impact

┌────────────────────────┬────────────────────────┐
│ Metric │ Actual (71 bindings) │
├────────────────────────┼────────────────────────┤
│ MQClientInstance count │ 72 (should be 1) │
├────────────────────────┼────────────────────────┤
│ RocketMQ threads │ ~1,700 (should be ~24) │
├────────────────────────┼────────────────────────┤
│ Total JVM threads │ 2,200+ │
└────────────────────────┴────────────────────────┘

Expected Behavior

Consumers connecting to the same NameServer should share a single MQClientInstance by default, unless explicitly configured otherwise.

Suggested Fix

Provide a configurable instanceName strategy. For example:

Option A — Use a fixed instanceName by default (group name uniqueness already prevents conflicts):

// RocketMQConsumerFactory.java
consumer.setInstanceName(
RocketMQUtils.getInstanceName(rpcHook, consumerProperties.getGroup()));
// change to:
consumer.setInstanceName(
RocketMQUtils.getSharedInstanceName(rpcHook));

Option B — Expose a configuration property:

spring:
cloud:
stream:
rocketmq:
binder:
share-client-instance: true # default true

Option C — At minimum, provide a hook (e.g. ConsumerEndpointCustomizer timing fix or a ClientInstanceCustomizer callback) so users can override the instanceName after pushConsumer is created but before start() is called. Currently there is no such hook — ConsumerEndpointCustomizer.configure() is called before
afterPropertiesSet(), at which point pushConsumer is still null.

Why users cannot work around this

The framework's lifecycle leaves no clean extension point:

createConsumerEndpoint() → new Adapter() [pushConsumer = null]
ConsumerEndpointCustomizer → configure() [pushConsumer = null, can't modify]
afterPropertiesSet() → onInit() → Factory [pushConsumer created with unique instanceName]
start() → pushConsumer.start() [MQClientInstance created, too late]

• BeanPostProcessor — doesn't work because RocketMQInboundChannelAdapter is never registered as a Spring bean (created via new in doBindConsumer)
• ConsumerEndpointCustomizer — called before pushConsumer exists
• BinderCustomizer — can access the binder but not the individual consumers

The only workaround is classpath shadowing of RocketMQUtils, which is fragile and not maintainable.

Environment

• spring-cloud-starter-stream-rocketmq: 2023.0.3.2
• rocketmq-client: 5.3.1
• Spring Boot: 3.x
• Java: 17

[uuuyuqi]

Thanks for the detailed report. I checked the current 2025.1.x branch and can confirm the core issue described here is real.

In the current binder implementation, both push and pull consumers set instanceName via RocketMQUtils.getInstanceName(...), and that method appends System.nanoTime(). In RocketMQ 5.3.1, ClientConfig.buildMQClientId() includes instanceName, and MQClientManager.getOrCreateMQClientInstance() reuses instances by clientId. As a result, multiple consumer bindings in the same application end up with different clientIds, so they do not reuse the same MQClientInstance. This makes the resource and thread amplification described in this issue possible.

I also checked whether users can work around this through the current customization hooks. At the moment, that is difficult: ConsumerEndpointCustomizer.configure(...) runs before the underlying DefaultMQPushConsumer is created, while the consumer itself is created later in RocketMQInboundChannelAdapter.onInit(). In other words, when the customization hook is called, there is no DefaultMQPushConsumer instance available yet to override instanceName; and by the time the consumer exists, that customization phase is already over. So today there is indeed no clean extension point for users to change instanceName before startup.

So from my side, the issue itself is valid.

That said, the exact fix may need some design discussion in the community. Sharing one fixed instanceName for all bindings may help in many cases, especially across different consumer groups, but it may not be sufficient for every case, especially when multiple bindings use the same consumer group. Since RocketMQ manages consumers by group inside one MQClientInstance, the final solution likely needs a safe sharing strategy or an explicit/configurable instanceName policy rather than a blanket change.