journald source reports component_events_received_bytes even when the downstream components are blocked.

[jcantrill]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Issue

Given vector is configured to read from a jouranald source, through several transforms, and then to an http based output, it continues to report 'component_received_bytes_total' when the output is down and imposing back-pressure.

Analysis of the code suggests this is related to the code logic and its interaction with the sink's retry logic:

1. Journald reads batch from journalctl (lines 498-522) - INSTANT
2. Metrics emitted (lines 617-624) - INSTANT - component_received_bytes_total increments
3. send_batch().await BLOCKS (line 626) - BLOCKS FOR ~5 MINUTES
4. After 5 minutes, send completes
5. Loop back to step 1 - instantly reads next batch and emits metrics
6. Blocks again for 5 minutes

Proposed solution:

Modify the log to emit metrics when self.source.out.send_batch(self.events).await is 'Ok'

Question

1. While this looks to resolve the issue I'm wondering if it matches the intention of the metric:

• Current issue is not isolated to this source as many of the others also emit before sending downstream
• Is it reasonable to report "received" events contingent upon the deliver to the next component? Sent events are reported upon successful enqueuing to the downstream channel so maybe?

2. What might an alternate to resolve the issue

Configuration

Version

v0.54.0

Debug Output

Example Data

No response

Additional Context

No response

References

• Ref: https://redhat.atlassian.net/browse/LOG-7538

[jcantrill]

cc @vparfonov