Tool readiness is invisible, and an unusable oas_spec is stored without complaint

[lonelycode]

A Tool that exists is not a Tool that works, and nothing on the Tool says which it is.

D3 — A Tool needs four separate things to be true, and none is visible on the Tool

1. operations enabled
2. privacy score ≤ the LLM's
3. attached to an App
4. listed in the Chat's Default Tools

(plus a config push on the edge — see the config-push issue)

The Tool page has a usage chart, a spec confirmation and an operations list, and says nothing about any of it.

Fix: a readiness panel on the Tool page — four checks, each linking to the screen that fixes it. This one change would remove most of an entire demo's second act.

A3 — oas_spec is typed string but must contain base64

POST /tools with a raw JSON spec returns 201 and the Tool looks fine in the list. GET /tools/{id}/spec-operations then returns 500 — illegal base64 data, forever. The Tool can never have operations enabled, so it is permanently dead, with nothing on it saying why.

Fix: validate and reject at create time, or accept both and normalise. Storing an unusable value is the worst of the three options. Typing it []byte (as Filter.Script is) would make the encoding self-documenting.

H3 — Server-side spec constraints are strict and unstated up front

• OAS 3.x only
• a non-empty servers whose first entry is used verbatim as the base URL — so a relative /api/v3, as the published Swagger Petstore uses, breaks
• an operationId on every operation in the document, not just the ones you enable
• no external or remote $ref

We hand-wrote two fixture specs because no published spec we tried satisfied the validator.

Fix: validate in the wizard, before create, with the four rules listed and the failing operation named. That page is the natural home for this checklist.

H2 — "Import from URL" needs a CORS-open host

It is a browser fetch() from the admin SPA, so a normal spec URL fails with "Failed to load specification", which reads as a bad URL.

Fix: proxy the fetch server-side, or say what actually happened.

---

Acceptance

• Tool page carries a readiness panel with the four checks, each linking to its fix
• oas_spec cannot be stored in a form that makes the Tool permanently unusable
• The import wizard states the four spec constraints and names the failing operation
• Import-from-URL either works for non-CORS hosts or explains itself

Related and already fixed: #481 (btoa/UTF-8 encoding). From the UX & UI review of the demo set (D3, A3, H2, H3).

[probelabs]

Hello @lonelycode, thank you for this detailed and well-structured report. Your breakdown of the issues surrounding tool readiness and OpenAPI specification handling is incredibly helpful.

Based on your description, I understand there are four key areas of concern:

1. Invisible Readiness: The UI for a Tool doesn't show whether it's actually usable, which depends on four distinct conditions (operations enabled, privacy score, app attachment, and inclusion in a chat's default tools).
2. Invalid Spec Storage: The /tools endpoint accepts a raw JSON oas_spec, even though it expects base64-encoded data. This creates a permanently broken tool that consistently fails with a 500 error when its operations are accessed.
3. Unstated Spec Constraints: The server-side validation for specs is strict (requires OAS 3.x, servers field, universal operationId, no remote $ref) but doesn't provide this feedback to the user during the import process.
4. CORS Issues with URL Import: The "Import from URL" feature fails for most standard URLs due to being a client-side fetch() without a server-side proxy, leading to a generic and misleading error message.

You've correctly identified that these issues combined create a very challenging user experience for what is a core feature of the product.

To help the team investigate further, could you confirm if you have observed any specific error messages in the browser's developer console or in the AI Studio's server logs when the GET /tools/{id}/spec-operations call fails? Any additional log data would be valuable.

The team will review this feedback for consideration in future development. Thank you again for your thorough contribution.

References

• api/catalogue_handlers.go:121-146 – The CreateTool handler, which processes the oas_spec on tool creation.
• api/catalogue_handlers.go:217-238 – The GetToolSpecOperations handler, where the base64 decoding error likely originates.
• catalogue/oas_parser.go:102-120 – The Parse function responsible for validating the OpenAPI specification.

---

Powered by Visor from Probelabs

💡 TIP: You can chat with Visor using /visor ask <your question>