Skip to content

feat: add should_summarize_callback param to LLMContextSummarizer #4795

Description

@Jap1959

Problem Statement

Currently, LLMContextSummarizer._should_summarize() runs on every
LLMFullResponseStartFrame and always performs token estimation
(LLMContextSummarizationUtil.estimate_context_tokens()), even when
thresholds are far from being reached. There is no way to override
or customize this trigger logic without subclassing.

Proposed Solution

Add a should_summarize_callback parameter that, when
provided, replaces the built-in threshold checks entirely — giving
users full control over when summarization triggers and avoiding
unnecessary token estimation overhead.

Alternative Solutions

Current Workaround

Subclassing LLMContextSummarizer and overriding _should_summarize()
is technically possible since it uses a single underscore:

class CustomContextSummarizer(LLMContextSummarizer):
def init(self, *, context, should_summarize_fn=None, **kwargs):
super().init(context=context, **kwargs)
self._should_summarize_fn = should_summarize_fn

def _should_summarize(self) -> bool:
    if self._should_summarize_fn:
        return self._should_summarize_fn(self._context)
    return super()._should_summarize()

Why This Is Not Ideal

  • _should_summarize is a private method (single underscore),
    not a documented extension point
  • Subclassing is fragile — internal changes to the method
    signature or behavior in future versions would silently break it
  • Users shouldn't need to subclass just to customize trigger logic
  • No way to inject CustomContextSummarizer into
    LLMAssistantAggregator without patching internals

Additional Context

No response

Would you be willing to help implement this feature?

  • Yes, I'd like to contribute
  • No, I'm just suggesting

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions