You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Visualise Kafka Streams topologies in the Kpow Workflows UI.
14
+
- Aggregate and Expose Kafka Streams metrics via Kpow [Prometheus Endpoints](https://docs.factorhouse.io/kpow-ee/features/prometheus/) (for alerting, etc).
16
15
17
16
See the [Kpow Kafka Streams Feature Guide](https://docs.factorhouse.io/kpow-ee/features/kafka-streams/) for full documentation.
18
17
@@ -52,10 +51,10 @@ Include the agent as a dependency in your Kafka Streams application.
52
51
53
52
In your application, just before you start your KafkaStreams instance:
54
53
55
-
* Create a new [io.factorhouse.kpow.StreamsRegistry](https://github.com/factorhouse/kpow-streams-agent/blob/main/src/java/io/factorhouse/kpow/StreamsRegistry.java) instance.
56
-
* Register your KafkaStreams and Topology instances with the StreamsRegistry.
54
+
- Create a new [io.factorhouse.kpow.StreamsRegistry](https://github.com/factorhouse/kpow-streams-agent/blob/main/src/java/io/factorhouse/kpow/StreamsRegistry.java) instance.
55
+
- Register your KafkaStreams and Topology instances with the StreamsRegistry.
The StreamsRegistry is a *single-threaded process* that performs these actions **once every minute**:
83
+
The StreamsRegistry is a _single-threaded process_ that performs these actions **once every minute**:
85
84
86
-
* Capture metadata about each registered Kafka Streams application.
87
-
* Produce snapshots to the Kpow internal `__oprtr_snapshot_state` topic.
85
+
- Capture metadata about each registered Kafka Streams application.
86
+
- Produce snapshots to the Kpow internal `__oprtr_snapshot_state` topic.
88
87
89
88
The StreamsRegistry **does not talk directly to Kpow**. Kpow reads streams data from the snapshot topic.
90
89
@@ -111,13 +110,14 @@ If you pass no metric filters to the `StreamsRegistry` constructor then the defa
111
110
112
111
Kpow's streams agent metric filters work very similar to Micrometer's [meter filters](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/meter-filters.adoc).
113
112
114
-
Metric filters can either `ACCEPT` or `DENY` a metric. The filter itself is a Java predicate which takes in the [org.apache.common.MetricName](https://kafka.apache.org/0110/javadoc/org/apache/kafka/common/MetricName.html#group()) class. This allows you to filter metrics by name, tags or group.
113
+
Metric filters can either `ACCEPT` or `DENY` a metric. The filter itself is a Java predicate which takes in the [org.apache.common.MetricName](<https://kafka.apache.org/0110/javadoc/org/apache/kafka/common/MetricName.html#group()>) class. This allows you to filter metrics by name, tags or group.
115
114
116
115
Metric filters are applied sequentially in the order they are configured in the registry. This allows for stacking of deny and accept filters to create more complex rules:
### The Workflows UI is showing "Configure Streams Topology"
268
268
269
269
This could happen for a few reasons:
270
270
271
271
1.`register` method has not been called.
272
272
2. Invalid connection details passed to `StreamsRegistry` constructor. If this is the case you will see Kafka producer exceptions in the logs of your streams application.
273
-
3. Telemetry is still being calculated. After a fresh deployment, it might take up to 2 minutes for initial streams telemetry to be calculated.
273
+
3. Telemetry is still being calculated. After a fresh deployment, it might take up to 2 minutes for initial streams telemetry to be calculated.
274
274
275
275
You can verify `StreamsRegistry` is sending telemetry to your Kafka Cluster by using Data Inspect in Kpow:
276
276
277
-
* Select topic `__oprtr_snapshot_state`
278
-
* Choose `Transit / JSON` as the key deserializer
279
-
* Choose `Last 15 minutes` as the window
280
-
* Enter the following kJQ filter: `.key[0] == :streams`
277
+
- Select topic `__oprtr_snapshot_state`
278
+
- Choose `Transit / JSON` as the key deserializer
279
+
- Choose `Last 15 minutes` as the window
280
+
- Enter the following kJQ filter: `.key[0] == :streams`
281
281
282
282

283
283
@@ -287,6 +287,6 @@ If you have any issues contact [support@factorhouse.io](mailto:support@factorhou
0 commit comments