Skip to content

test(core-components): cover invalid variable patterns in prompt template #214

Description

@Victor-w-Madeira

Context

The Prompt Template variable extraction goes through two different parsers (Python's string.Formatter in f-string mode and mustache_template_vars + validate_mustache_template in mustache mode). Both reject a documented set of invalid patterns by raising ValueError, but the current spec core-components/prompt-template-component-regression.spec.ts only covers valid, well-formed variables.

A regression where the parser silently accepts a previously invalid pattern (or stops rejecting an invalid one) would slip past the suite today. Negative-path coverage is also useful as a snapshot of the contract — if Langflow ever loosens the validation, the diff in this spec makes the change visible.

Source references (Langflow upstream):

  • src/lfx/src/lfx/interface/utils.py:62-83extract_input_variables_from_prompt() uses Formatter().parse() to walk the template and deduplicates by seen set
  • src/lfx/src/lfx/base/prompts/api_utils.py:13-27_INVALID_CHARACTERS list (space, comma, dot, colon, etc.) used by _check_input_variables()
  • src/lfx/src/lfx/base/prompts/api_utils.py:85-101_check_input_variables() and _check_for_errors() raise ValueError("Input variables contain invalid characters or formats") when an invalid char is found
  • src/lfx/src/lfx/base/prompts/api_utils.py:127-177validate_prompt() branches on is_mustache; mustache path calls validate_mustache_template() which rejects section/inverted/triple-brace syntax
  • src/lfx/src/lfx/utils/mustache_security.pyvalidate_mustache_template() with the per-pattern rejection logic

Scope (proposed)

f-string mode (toggle off, default)

# Input Expected behavior
1 {} (empty braces) Error toast/notification surfaces "Input variables contain invalid characters or formats"; no handle created
2 {var-name} (hyphen in identifier) Error surfaces; no handle created
3 {var.attr} (dot notation) Error surfaces; no handle created
4 {{var}} (double braces in f-string mode) Treated as literal escape — no handle created (this is the documented Python behavior, not an error)
5 {name} and {name} (duplicate) Exactly one name handle is rendered, not two

mustache mode (toggle on — depends on companion issue)

# Input Expected behavior
6 {{ var }} (spaces inside) Error surfaces; no handle created
7 {{var.attr}} (dot notation) Error surfaces; no handle created
8 {{#section}}{{/section}} (mustache section) Error "Complex mustache syntax is not allowed" surfaces; no handle created
9 {{{var}}} (triple braces / unescaped HTML) Error surfaces; no handle created

Tests 6–9 can be deferred to a second milestone if the use_double_brackets companion spec (#213) is not yet merged — they require the toggle helper to flip mustache mode on.

Tags

All tests: @stable @regression @components. None should carry @release — these are defensive contract tests, not happy-path flows.

Deliverables

  • tests/tests-automations/regression/core-components/prompt-template-invalid-patterns-regression.spec.ts
  • docs/core-components/prompt-template-invalid-patterns-regression.md
  • QA-CHECKLIST.md — add new bullets under §3.2 covering invalid-pattern rejection (one per parser mode, or one bullet listing the cases)
  • docs/core-components/prompt-template-component-regression.md — extend the "What this test does not cover" section with a bullet pointing at the new prompt-template-invalid-patterns-regression.spec.ts (and replace the existing generic "Variable name validation" line if you do). Reference the spec file directly — do not cite this issue, which will close once delivered.

Out of scope

  • The toggle behavior itself — covered by the companion issue
  • LLM/runtime errors when running a flow with an invalid template (this spec validates the input-time parser, not the build-time flow execution)
  • Tool Mode interaction — covered by tool-mode.spec.ts

Notes

  • The error surface in the UI is the validation toast/notification raised when update_build_config re-processes the template after genericModalBtnSave. The implementer should verify whether the error is shown as a toast, an inline message under the field, or a tooltip — this affects the selector strategy.
  • Test 5 (duplicate variable) is a positive-path assertion mixed in with the rejection cases on purpose: it documents deduplication as part of the parser contract.
  • Test 4 ({{var}} in f-string mode rendering as literal) is positive-path-as-negative-coverage: confirms the parser does not create a handle even though the pattern looks plausible. This is the documented Python .format() escape behavior.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions