Is your feature request related to a problem? Please describe.
The Kafka source does not allow configuring connections.max.idle.ms. The consumer always uses the client default of 540000ms (9 minutes), so there is no way to have the client recycle idle connections sooner. This is a problem when connecting to brokers that close idle connections earlier than the client default, which leads to avoidable reconnection churn.
Describe the solution you'd like
Add a per-topic connections_max_idle option to the Kafka source that controls connections.max.idle.ms for the consumer. It should default to the current client default (540000ms) so existing pipelines are unchanged. This brings the source in line with the Kafka sink, which already exposes this option.
Describe alternatives you've considered (Optional)
Lowering max_poll_interval was considered but only affects the poll loop and does not control how long idle connections are kept open.
Additional context
N/A
Is your feature request related to a problem? Please describe.
The Kafka source does not allow configuring
connections.max.idle.ms. The consumer always uses the client default of 540000ms (9 minutes), so there is no way to have the client recycle idle connections sooner. This is a problem when connecting to brokers that close idle connections earlier than the client default, which leads to avoidable reconnection churn.Describe the solution you'd like
Add a per-topic
connections_max_idleoption to the Kafka source that controlsconnections.max.idle.msfor the consumer. It should default to the current client default (540000ms) so existing pipelines are unchanged. This brings the source in line with the Kafka sink, which already exposes this option.Describe alternatives you've considered (Optional)
Lowering
max_poll_intervalwas considered but only affects the poll loop and does not control how long idle connections are kept open.Additional context
N/A