Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion QA-CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -441,7 +441,7 @@
- [x] Output formatting (JSON via output_schema, Markdown, plain text) → `agent-structured-output.spec.ts`

#### 7.8 Unified Provider Setup — 1.11.0 additions
- [~] OpenAI Compatible as a first-class unified model provider: setup with base URL + key, models discovered, usable by a flow → `core-functionality/model-provider/openai-compatible-provider-setup.spec.ts` (6 tests, validated on 1.12.0.dev15 against `https://api.openai.com/v1` — the endpoint the issue itself suggests, so **no new CI secret**: the base URL defaults there and the bearer is the existing `OPENAI_API_KEY`, overridable via `OPENAI_COMPATIBLE_TEST_BASE_URL` / `_API_KEY` / `_MODEL`. **5 are `@stable`**: the two-variable form with its asymmetric required/optional Save gate; the live-only catalog — this is the first provider with **no static rows at all**, so unconfigured it contributes 0 models, asserted differentially against Azure AI Foundry's seed catalog in the same run; an unresolvable `.invalid` base URL and a real endpoint with a bogus key both rejected on the `validate-provider` **body** (HTTP 200 + `valid:false`, DNS vs `Authentication failed for the OpenAI-compatible endpoint`) with nothing persisted; live discovery returning **exactly** the endpoint's own `/v1/models` id set, twice over (once per model type); and one discovered id running a Basic Prompting flow to a sentinel reply through the provider-qualified `OpenAI Compatible-<id>-option`. **Partial because the 6th — configuring the provider through Settings — is quarantined (`test.fixme`, no `@stable`) against a confirmed product defect:** Save persists **only the base URL**, because the frontend fires the two `POST /api/v1/variables/` writes concurrently and the primary (secret) key write is rejected `400 Invalid OpenAI-compatible base URL` — validated against provider variables that do not yet include the base URL its sibling request is creating — with **nothing surfaced in the UI**. Measured 3/3 through the UI and isolated against the API on the same instance: sequential writes 201/201, concurrent 201/400. So an authenticated OpenAI-compatible endpoint cannot be configured from Settings at all; the assertions are unchanged and lifting the quarantine is a deliverable of [LE-2124](https://datastax.jira.com/browse/LE-2124))
- [x] OpenAI Compatible as a first-class unified model provider: setup with base URL + key, models discovered, usable by a flow → `core-functionality/model-provider/openai-compatible-provider-setup.spec.ts` (6 tests, **all 6 `@stable`**, validated on 1.12.0.dev19 against `https://api.openai.com/v1` — the endpoint the issue itself suggests, so **no new CI secret**: the base URL defaults there and the bearer is the existing `OPENAI_API_KEY`, overridable via `OPENAI_COMPATIBLE_TEST_BASE_URL` / `_API_KEY` / `_MODEL`. Covered: the two-variable form with its asymmetric required/optional Save gate; the live-only catalog — this is the first provider with **no static rows at all**, so unconfigured it contributes 0 models, asserted differentially against Azure AI Foundry's seed catalog in the same run; an unresolvable `.invalid` base URL and a real endpoint with a bogus key both rejected on the `validate-provider` **body** (HTTP 200 + `valid:false`, DNS vs `Authentication failed for the OpenAI-compatible endpoint`) with nothing persisted; live discovery returning **exactly** the endpoint's own `/v1/models` id set, twice over (once per model type), polled on that terminal shape because the catalog is observable mid-registration; one discovered id — enabled for the provider by the test itself, since a `default:false` id never reaches the node's dropdown — running a Basic Prompting flow to a sentinel reply, gated on the **persisted** node carrying `{name, provider: "OpenAI Compatible"}` rather than a stored credential name, which upstream [#14311](https://github.com/langflow-ai/langflow/pull/14311) stopped writing (#1334); and configuring the provider through Settings persisting **both** variables — quarantined against [LE-2124](https://datastax.jira.com/browse/LE-2124) (the concurrent `POST /api/v1/variables/` writes dropped the key, silently) and **un-quarantined on 1.12.0.dev19**, where concurrent API writes answer 201/201 3/3 and the test passes 3/3 with its assertions unchanged)
- [x] Azure AI Foundry in the unified provider setup: configuration accepts deployment names, provider appears configured → `core-functionality/model-provider/azure-ai-foundry-provider-setup.spec.ts` (6 tests, all validated against a live Azure resource on 1.12.0.dev15 — 3/3 clean `--retries=0 --workers=1` runs, every test force-failed. 4 of them need **no** Azure account and carry the surface on every lane: the two-variable form, the Foundry-only deployment hint asserted differentially against OpenRouter, the read-only unconfigured panel, an unresolvable endpoint rejected with `valid:false` and nothing persisted, and a deployment name absent from every catalog accepted and stored as `Azure AI Foundry::llm::<name>`. The other two — real credentials configuring the provider through Settings, and a real inference addressed by the deployment name — now run there too: the three `AZURE_AI_FOUNDRY_*` secrets are wired into the daily shard step and `manual.yml`'s Docker job by #1270, proven by a dispatch that reported `6 passed` where the lane had said `4 passed, 2 skipped`. They still skip with the concrete reason wherever the credentials are absent — a local run without the `.env` block, `pr-validation.yml` and the external-URL job, both unwired on purpose: #1216 keeps an unrelated PR off that account's health, and #1055 keeps us from storing our credentials in an instance we do not own)

---
Expand Down
Loading
Loading