fix: collect KafkaTopicSample on Kafka 4.x (DescribeConfigs version)

[shashank-reddy-nr]

Summary

KafkaTopicSample is missing on Kafka 4.x clusters while KafkaBrokerSample, KafkaProducerSample, KafkaConsumerSample, and KafkaOffsetSample continue to report.

Root Cause: The integration issues DescribeConfigs requests at API version 0, which Kafka 4.0 removed (the broker's minimum supported version is now v1, per KIP-896). The v0 request is rejected by the broker, setTopicInfo() returns an error, and topicWorker() continues before the topic metric set is created — so the topic sample is dropped entirely. Broker/producer/consumer/offset samples are JMX- or higher-level-API-based, so they remain unaffected.

This was reproduced and verified on a real Kafka 3.8.1 (works) vs 4.0.0 (topic samples missing) cluster, confirmed both via the integration's -pretty output and in NRDB.

---

Changes

• src/topic/topic_collection.go, src/broker/broker_collection.go: Derive the DescribeConfigs request version from the negotiated KAFKA_VERSION instead of hardcoding 0.
  • Logic: >= 2.0.0 → v2, >= 1.1.0 → v1, else v0. (Sarama v1.43.x supports DescribeConfigs up to v2; Kafka 4.0 supports 1–4). IncludeSynonyms is now set only when the version supports it (>= 1).
• src/args/args.go: Raise the default KAFKA_VERSION from 1.0.0 to 2.1.0. This ensures topic/broker config collection works against Kafka 4.x out of the box without requiring manual user configuration.
• src/args/args_test.go: Updated TestDefaultArgs expectation to match the new default.
• CHANGELOG.md: Added unreleased bug-fix entry.

---

Why both changes are needed

Sarama refuses to send a request whose requiredVersion() exceeds config.Version (within broker.go). Simply fixing the hardcoded version is insufficient; with the old 1.0.0 default, a v2 (or even v1) request is rejected client-side. Bumping the default ensures functionality for Kafka 4.x users without forcing manual configuration changes. Deriving the version (rather than hardcoding v2) ensures we do not regress support for genuinely older brokers.

---

Testing

• Unit Tests: go test ./src/... passes.
• Manual/Agent Testing: Ran against a live single-node KRaft cluster:
  • Kafka 3.8.1: All sample types present (baseline, unchanged).
  • Kafka 4.0.0 (Before): KafkaTopicSample = 0, DescribeConfigs errors.
  • Kafka 4.0.0 (After, no KAFKA_VERSION set): KafkaTopicSample present for all topics, 0 DescribeConfigs errors; confirmed in NRDB.

---

Compatibility / Notes

• No behavior change for Kafka < 4.0 except the higher default protocol version (2.1.0), which negotiates down per-API via ApiVersions.
• Future Considerations: Upgrade Sarama to a release that natively supports Kafka 4.x (DescribeConfigs v4 / flexible versions), and improve error handling to make a malformed KAFKA_VERSION non-fatal (falling back to default instead of aborting the whole run).

[abhishuraina]

LGTM, make sure you've tested it on kafka V4