fix(core): Merge system messages for strict providers

[kayareyouokay]

Summary

Merge volatile instructions and observation memory into one system message for providers that do not support multiple system messages.

Keep split system messages for Anthropic, Vertex Anthropic, OpenAI, and OpenRouter. This keeps the stable instruction prefix available for prompt caching.

How to test

1. Run pushd packages/@n8n/agents && pnpm test src/runtime/__tests__/message-list.test.ts src/runtime/__tests__/agent-runtime.test.ts && popd.
2. Run pushd packages/@n8n/agents && pnpm format:check && pnpm typecheck && pnpm lint && popd.
3. Configure an AI Assistant custom OpenAI-compatible endpoint that accepts one leading system message only.
4. Run a task that triggers mid-run observation compaction.
5. Confirm that the next model request contains one system message and the task continues.

Related Linear tickets, Github issues, and Community forum posts

Fixes #37589

Review / Merge checklist

• I have seen this code, I have run this code, and I take responsibility for this code.
• PR title and summary are descriptive.
• Docs updated or follow-up ticket created. Documentation is not required for this internal behavior change.
• Tests included.
• PR Labeled with Backport to Beta, Backport to Stable, or Backport to v1 (if the PR is an urgent fix that needs to be backported)

[cubic-dev-ai]

No issues found across 4 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Shadow auto-approve: would not auto-approve. Auto-approval skipped because this PR is from an external contributor.

Re-trigger cubic</sub>

[n8n-assistant]

Hey @kayareyouokay,

Thank you for your contribution. We appreciate the time and effort you’ve taken to submit this pull request.

Before we can proceed, please ensure the following: • Your PR references the GitHub issue it fixes (or, for feature requests, a link to the corresponding community forum post). • Tests are included for any new functionality, logic changes or bug fixes. • The PR aligns with our contribution guidelines.

Why the linked issue matters: Our teams pick up work from the issue, not from individual pull requests — the issue is what reaches them, with your PR linked to it. So please make sure the issue contains everything needed to judge the change: a clear problem description, reproduction steps, and the expected behaviour. If the issue is thin, add the missing context there rather than only in the PR description.

Regarding new nodes: We no longer accept new nodes directly into the core codebase. Instead, we encourage contributors to follow our Community Node Submission Guide to publish nodes independently.

If your node integrates with an AI service that you own or represent, please email nodes@n8n.io and we will be happy to discuss the best approach.

About review timelines: While we plan to review it as soon as possible, we are currently unable to provide an exact timeframe. Our goal is to begin reviews within a month, but this may change depending on team priorities. We will reach out when the review begins.

Please also note that other contributors may have opened pull requests for the same issue. We keep them all open so the reviewing team can choose the approach that fits best. Once the issue is resolved, the remaining pull requests are closed — this is not a judgement on the quality of your work, and we're grateful for it either way.

Thank you again for contributing to n8n.

[kayareyouokay]

Hey @mutdmour please have a look on this whenever you've got the time.