Clamping to context window prevents artificial context limits, distinct from maxTokens

[DanielThomas]

What happened?

The fix (
09f1059#diff-1a93fd67549d15259de0044dba9e2650cf5e6c41a51de3c4994992c9867ca80b) for #5595 clamps max_tokens to the reported context window.

This prevents plugins from intentionally setting a smaller context window for UI feedback/automatic compact, because it's no longer tracking the maximums supported by the API due to the clamp:

Error: Model stopped because it reached the maximum output token limit. The response may be incomplete.

Steps to reproduce

• Report a smaller context size than supported by the API in a registered model
• Note that it clamps to that limit, instead of the maximum limit of the model

Expected behavior

Plugins should be able to set a context window size smaller than is supported by the model to cause UI feedback and compaction at lower thresholds (for instance, I find even 400K tokens into a 1M Anthropic model is pushing it)

Version

No response

[legacy7838-create]

Interested in working on this. @legacy7838-create

The root cause is that clampMaxTokensToContext() in simple-options.ts uses model.contextWindow for both compaction thresholds and API output limit clamping. Plugins that set a smaller contextWindow for early compaction get their maxTokens squeezed below the model's actual output capacity.

Proposed fix: add a maxContextWindow?: number field to Model for the hard API limit, falling back to contextWindow when unset. Clamp uses maxContextWindow ?? contextWindow instead of contextWindow directly.

[Claes1981]

I think this bug hit me too after latest Pi update, when using the @agnishc/edb-herald extension. This is the output: (error after a single "The")

The

Error: Model stopped because it reached the maximum output token limit. The response may be incomplete.

Update: Get the same response also by just sending a simple chat message in Pi.

Downgrading Pi to version 0.80.2 removes the issue.

[xezpeleta]

I've also been hit by this issue. Currently, the only workaround to continue the session is to switch to another model for a turn, compact and then switch back to the original model.

[highlyunavailable]

This is a really nasty one, it does mean I have to swap models unless I manually compact before hitting max context locally. This is made worse by the fact that auto-compact doesn't happen at all unless it was on the user turn - auto compact won't fire mid agent-turn which usually guarantees a max context crash-compaction.

[vitor-alves]

I am also being affected by this bug.
pi version 0.80.3

[alexian79]

God bless you guys. I was haunted by this for a while, stopping at 16386 tokens of thinking generation. Local llama.cpp running GLM 5.2 at only 5t/s each stop was painful. Downgraded to 0.80.2 and it is gone.

[highlyunavailable]

This may be fixed in latest, no longer reproing the "context full" problem in 0.80.6

[Claes1981]

This may be fixed in latest, no longer reproing the "context full" problem in 0.80.6

Tested version 0.80.6, but for me same error message as I got with version 0.80.3.
Version 0.80.2 works fine.

[badlogic]

i'm not sure how we cpuld possibly "fix" this on our end. a model has a context size. all calculations are based on that. if you set a lower size then the calculations will use that. expecting the actual size to be used is not something that can be supported with a single context window size.

i'd propose this:

• change the compation thresholds to align with your wishes
• create an extension that overrides the footer to display the "dumbzone".

[Claes1981]

Hmm... It seems I had a too low value of contextWindow in my ~/.pi/agent/models.json file. It was much lower than the specified context size of the model in my Llama.cpp llama-server presets file.

After increasing contextWindow in ~/.pi/agent/models.json , also Pi version 0.80.6 seems to work fine for me.