Open
Conversation
Add wire-format protos for the linux.journald data source, modeled on the Android logcat data source. New files: - protos/perfetto/config/linux/journald_config.proto: Config message with min_prio, filter_identifiers, and filter_units fields. - protos/perfetto/config/linux/BUILD.gn: GN build target. - protos/perfetto/trace/linux/journald_event.proto: JournaldEventPacket message with per-entry fields (timestamp, pid, tid, uid, gid, prio, tag, message, comm, exe, systemd_unit, hostname, transport, monotonic_ts) and a Stats submessage. - protos/perfetto/trace/linux/BUILD.gn: GN build target. Modified files: - protos/perfetto/config/data_source_config.proto: Import journald config and add journald_config field (id 140). - protos/perfetto/trace/trace_packet.proto: Import journald event and add journald_event field (id 126) to the oneof data block. - protos/perfetto/config/BUILD.gn: Add linux:@type@ to deps. - protos/perfetto/trace/BUILD.gn: Add linux:@type@ to non_minimal deps.
Add enable_perfetto_journald GN flag that gates the journald data source
in traced_probes. The flag is enabled on Linux non-Android standalone
builds that have enable_perfetto_traced_probes set.
Also add a pkg_config("pkgconfig_libsystemd") target to gn/BUILD.gn so
that targets building the journald data source can pull in the libsystemd
compile/link flags via pkg-config.
The flag is placed in a separate declare_args() block (after the block
containing enable_perfetto_traced_probes) so it can reference that
variable without triggering the GN "variable defined in same
declare_args() call" restriction.
Implement a new data source that reads log entries from the systemd journal via libsystemd's sd-journal API. The data source: - Opens both system and user journals (SD_JOURNAL_LOCAL_ONLY). - Supports filtering by syslog priority, SYSLOG_IDENTIFIER, and _SYSTEMD_UNIT via sd_journal match/conjunction/disjunction filters. - Seeks to the journal tail on start so only new entries are captured. - Watches the journal file descriptor for new-entry notifications and drains entries in batches of up to 500. - Extracts 14 fields per entry (timestamp, pid, tid, uid, gid, prio, tag, message, comm, exe, systemd_unit, hostname, transport, monotonic_ts) and writes them as JournaldEventPacket protos. - Emits a Stats sub-message on Flush with counters for total, skipped, and failed entries. - Gated behind the enable_perfetto_journald GN flag (Linux-only).
Adds full parsing and storage of JournaldEventPacket from the
linux.journald data source into Trace Processor.
Changes:
- src/trace_processor/tables/linux_tables.py: new JournaldLogTable
with columns ts, utid, prio, tag, msg, uid, comm, systemd_unit,
hostname, transport.
- src/trace_processor/tables/BUILD.gn: add linux_tables.py.
- src/trace_processor/importers/proto/linux_probes_parser.{h,cc}:
parses JournaldEventPacket, resolves PIDs to utids, interns strings
and inserts rows into JournaldLogTable.
- src/trace_processor/importers/proto/linux_probes_module.{h,cc}:
registers for TracePacket field 126, tokenises each journald event
into its own sorted packet (REALTIME → trace time conversion), then
dispatches to the parser.
- src/trace_processor/importers/proto/BUILD.gn: adds the new sources
and the linux:zero proto dep to the 'full' source_set.
- src/trace_processor/importers/proto/additional_modules.cc: registers
LinuxProbesModule.
- src/trace_processor/storage/trace_storage.{h,cc}: adds
journald_log_table() / mutable_journald_log_table() accessors and
the linux_tables_py.h include.
- src/trace_processor/trace_processor_impl.cc: calls AddStaticTable
for journald_log_table and adds linux_tables_py.h include.
- src/trace_processor/perfetto_sql/stdlib/prelude/after_eof/views.sql:
adds the public journald_logs SQL view.
- test/trace_processor/diff_tests/parser/linux/tests.py: new diff
test Linux.journald_basic verifying end-to-end parsing.
- test/trace_processor/diff_tests/include_index.py: registers Linux
test suite.
Add a new Perfetto UI plugin for displaying journald logs from the journald_logs SQL table exposed by trace processor. The plugin mirrors the Android log plugin (com.android.AndroidLog) in structure, with the following key differences: - Table: journald_logs (vs android_logs) - Priority scale: 0=EMERG...7=DEBUG (syslog convention, lower=worse) vs Android's 2=Verbose...7=Fatal (higher=worse) - Priority filter SQL: prio <= minimumLevel (inverted vs Android) - Default minimumLevel: 7 (show everything, DEBUG and above) - Depth/color mapping inverted: 0→purple(EMERG/ALERT/CRIT), 1→red(ERR), 2→orange(WARNING), 3→grey(NOTICE/INFO), 4→green(DEBUG) - Extra columns: systemd_unit instead of machine_id - No multi-machine filter Files created/modified: - ui/src/public/track_kinds.ts: Add JOURNALD_LOGS_TRACK_KIND - ui/src/core/default_plugins.ts: Register dev.perfetto.JournaldLog - ui/src/plugins/dev.perfetto.JournaldLog/logs_track.ts: Track renderer - ui/src/plugins/dev.perfetto.JournaldLog/logs_panel.scss: Styles - ui/src/plugins/dev.perfetto.JournaldLog/logs_panel.ts: Panel + filtering - ui/src/plugins/dev.perfetto.JournaldLog/index.ts: Plugin entry point
staticfloat
force-pushed
the
sf/journald
branch
from
45ae425 to
8ecbd7d
Compare
Member
|
Hi. Thanks for the contribution! Really great to see this happening. However there are several concerns I have with this patch as it stands:
This is my high level comments: I'm really excited to see this happen but I want to make sure this goes in while fitting in with the overall direction of the project! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a
linux.journalddata source to capture journal messages on systemd systems, inspired by the Android logcat viewer. This PR satisfies #3288.The design is fairly straightforward;
libsystemdis added as an optional dependency,traced_probesis taught how to open a connection to the system journal, and messages are serialized in a new protobuf message and displayed in a new UI plugin that mimics that of the logcat messages.This PR was co-written by AI; while I iterated on it until the look and feel felt correct, there may be context or use cases that I am unaware of. I am open to all review comments, and will do my best to address them.