Add intelligent CKAN instance detection to ckan_smart_search orchestrator

[KlementMultiverse]

Context

Issue #17 proposes a ckan_smart_search orchestrator tool that accepts natural language queries like "find climate datasets in Italy." This would dramatically improve accessibility for non-technical users who don't know which CKAN portal to query or its native search syntax.

However, the current implementation approach assumes users either:

1. Already know which CKAN instance they want to query (and pass a URL)
2. Manually specify the country/region before searching

This creates friction for the primary use case: "I'm looking for data about X, in country Y" — without requiring users to know that Italy's data portal is dati.gov.it, Canada's is open.canada.ca, etc.

Problem

Today, a user asking "find renewable energy statistics for Denmark" would need to:

• Know that Denmark uses an open data portal
• Know its URL is the correct CKAN instance
• Or provide it manually before the search runs

This defeats the purpose of a natural language interface. The orchestrator should be smart enough to route queries to the right CKAN instance based on geographic and contextual clues.

Proposed Solution

Add an optional CKAN instance detector to ckan_smart_search that:

1. Parses geographic references from the query ("Denmark", "Italy", "Canada") using existing tools or a lightweight regex/NLP approach
2. Maps countries to known CKAN instances using a configuration file (e.g., src/config/ckan-instances.json) with entries like:

   {
     "IT": {"url": "https://dati.gov.it", "names": ["italy", "italia"]},
     "CA": {"url": "https://open.canada.ca", "names": ["canada"]},
     "US": {"url": "https://catalog.data.gov", "names": ["usa", "united states"]}
   }

3. Defaults to user-provided instance if geography is ambiguous or not detected
4. Logs the detected instance in responses so users understand where the search ran

Why This Matters

Without detection, ckan_smart_search is still more convenient than raw API calls, but it doesn't feel "smart" — it still requires domain knowledge (knowing which portal to use). With detection, it becomes a true natural language interface that non-technical users can trust.

Good Example

User: "Where can I find water quality data in France?"

Server (internal):

• Detects "France" → maps to https://www.data.gouv.fr
• Runs search: package_search(q='water quality')
• Response: "Found 47 datasets on data.gouv.fr about water quality..."

User doesn't need to know about data.gouv.fr; the orchestrator figured it out.

Success Criteria

• ckan-instances.json added with 10+ major CKAN portals (data.gov, dati.gov.it, open.canada.ca, etc.)
• Detector integrated into ckan_smart_search workflow
• Graceful fallback if geography can't be parsed
• Updated docs/guide/ with examples showing geographic queries

---

Contributed by Klement Gunndu

[aborruso]

Thanks for the detailed write-up. After thinking it through, I don't think this belongs in the core MCP server — and here's why.

MCP is a connector, not an orchestrator

An MCP server is fundamentally a connector: it exposes atomic, composable tools that an AI agent can call. The "smartness" lives in the agent layer, not in the server itself. Baking geographic routing into this server would break this principle: we'd be hardcoding domain knowledge (country → portal mappings) into a layer that's supposed to stay generic.

The static ckan-instances.json approach is a step backward

The server already has ckan_find_portals, which queries a live registry of ~950 CKAN portals worldwide. Replacing that with a static JSON file covering "10+ portals" would reduce coverage by two orders of magnitude and introduce a maintenance burden that belongs nowhere near a transport-layer connector.

It would introduce hardcoded errors

The issue suggests mapping France → https://www.data.gouv.fr. That portal is not CKAN — querying it with CKAN tools will fail. An LLM agent can reason about this at runtime and redirect to data.europa.eu instead. A static config file would crystallize this kind of error permanently.

The orchestration pattern is already solved — at the right layer

The exact flow described in this issue (parse geography → find portal → verify → search with bilingual query → handle fallback if portal is down) is already implemented as a skill (ckan-mcp) that runs at the agent layer. That's where this logic belongs: the LLM reasons, the MCP server executes primitives.

We tested the two example queries from this issue today against the skill:

"Find violence data in Argentina"
The skill called ckan_find_portals(country="Argentina"), identified datos.gob.ar as the national portal, verified it was reachable, detected the OR-query parser quirk and switched to query_parser=text at runtime, and returned 46 relevant datasets — all without any hardcoded mapping in the server.

"Where can I find water quality data in France?"
The skill correctly recognized that data.gouv.fr is not CKAN and redirected to data.europa.eu with country=FR and a French-language query (qualité eau), returning datasets like "Qualité de l'eau de la rade de Brest et de ses bassins" (Brest métropole) and "PLN QUALITE EAU Hors baignade 2022" (Région Occitanie). A static config mapping France → data.gouv.fr would have failed at the first API call.

Summary

	Issue #18 proposal	Current approach (skill)
Country → portal mapping	Static JSON (~10 entries)	ckan_find_portals — live registry (~950 portals)
Error handling	Generic fallback	Contextual LLM reasoning
Maintenance	Manual, baked into the repo	Centralized in the registry
Architecture	Orchestration logic in the server	Orchestration logic in the agent ✓
France edge case	Maps to non-CKAN data.gouv.fr ❌	Redirects to data.europa.eu ✓

The MCP server should stay lean and generic. Closing this in favour of the skill-based approach.

---

The skill lives in this repo at skills/ckan-mcp/ and can be installed in one command:

npx skills install github:ondata/ckan-mcp-server/skills/ckan-mcp

Thanks again for taking the time to open this — the use case you described is exactly the right problem to solve, just at the right layer.