Fixing awful.spawn read_lines closure leak#4105
Open
soderstrom-rikard wants to merge 1 commit into
Open
Conversation
awful.spawn.read_lines() has a memory leak caused by a mutual closure reference cycle between start_read and finish_read that survives multiple Lua GC cycles. This happens because GI_SCOPE_TYPE_ASYNC callbacks always use LGI's autodestroy guard mechanism (see lgi/marshal.c:marshal_2c_callable), there is an unavoidable one-cycle delay between the callback firing and the Lua reference being released. After that, the mutual cycle between start_read and finish_read requires ANOTHER cycle to detect as unreachable. In practice this means the closures, their upvalues (stream, line_callback, done_callback, accumulated stdout strings etc.) survive 3+ GC cycles per easy_async call. With awful.widget.watch running at 1 Hz (volume widget default), this accumulates ~50-200 KB of Lua objects per second faster than Lua's incremental GC (with default gcpause=200) reclaims them, resulting in ~180 MB per hour or 4.32 GB per day of leaked memory. The ffi closures themselves (via ffi_closure_alloc) are NOT tracked by Lua's allocator, so growing ffi memory does not increase GC pressure, making the problem worse. This patch fixes this by breaking the start_read <-> finish_read mutual cycle. This is achieved by clearing all upvalues and making local copy of the done_callback (for use in the protected_call).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Affects: awesome v4.3 and likely all prior versions
Tested on: awesome 4.3-6, lua-lgi 0.9.2-14, Lua 5.3.6
PROBLEM
awful.spawn.read_lines() has a memory leak caused by a mutual closure reference
cycle between start_read and finish_read that survives multiple Lua GC cycles.
OBSERVED SYMPTOM
awesome memory grows from ~400 MB at startup to 4+ GB over 1-2 days when any
widget using awful.widget.watch (or anything that calls easy_async on a timer)
is in use.
Forcing two GC cycles from awesome-client confirms the leak is entirely in
Lua-tracked objects, not native memory:
ROOT CAUSE
In spawn.read_lines(), start_read and finish_read are declared as upvalues and
then close over each other:
This creates a mutual reference cycle:
finish_read <--> start_read (each is an upvalue of the other)
Additionally, LGI's ffi closure for finish_read (created by read_line_async)
holds a Lua registry reference (target_ref) to finish_read. This reference is
released only after:
Step 1: LGI's autodestroy guard is finalized by Lua GC
(guard was created in closure_callback when the callback fired)
Step 2: Lua GC detects finish_read is now unreachable
Because GI_SCOPE_TYPE_ASYNC callbacks always use LGI's autodestroy guard
mechanism (see lgi/marshal.c:marshal_2c_callable), there is an unavoidable
one-cycle delay between the callback firing and the Lua reference being
released. After that, the mutual cycle between start_read and finish_read
requires ANOTHER cycle to detect as unreachable.
In practice this means the closures, their upvalues (stream, line_callback,
done_callback, accumulated stdout strings etc.) survive 3+ GC cycles per
easy_async call. With awful.widget.watch running at 1 Hz (volume widget
default), this accumulates ~50-200 KB of Lua objects per second faster than
Lua's incremental GC (with default gcpause=200) reclaims them, resulting in
~180 MB per hour or 4.32 GB per day of retained memory.
The ffi closures themselves (via ffi_closure_alloc) are NOT tracked by Lua's
allocator, so growing ffi memory does not increase GC pressure, making the
problem worse.
FIX
Break all upvalue references inside done() before calling done_callback. This
eliminates both the start_read/finish_read mutual cycle and all other upvalue
chains (stream, line_callback, done_callback) in a single step. After this:
needed since the mutual reference was already broken)
This reduces from 3+ GC cycles to 2, and eliminates the mutual-cycle detection
overhead entirely.
SUGGESTED GC TUNING (companion change, rc.lua or early in awesome config)
Even with this patch, Lua's default GC settings (gcpause=200, meaning GC starts
a new cycle when memory reaches 200% of the post-collection size) allow large
amounts of short-lived objects to accumulate before collection. For a desktop
WM with frequent async spawns, more aggressive settings are advisable:
This is independent of the spawn.lua fix and provides a safety margin for any
other sources of short-lived object accumulation.