Testing v0.4.122 through an MCP client, not curl, showed ckan_find_relevant_datasets
answering its own documented example badly. defibrillatori Comune di Lecce on
dati.gov.it used to return the catalog's only Lecce dataset — because the search
returned exactly one result. With recall restored it returns 679, and the top three
became Trento, Martina Franca and Desio while the Lecce dataset fell out of the window.
The wrapping fix did not cause this; it removed the cover. Three defects, all older:
scoreTextFieldawarded the whole field weight when any query term matched. "Comune di Martina Franca" and "Comune di Lecce" both scored a full holder match, so the right dataset could not outrank the wrong ones. It now scores the share of terms the field carries.- the stopword list was English-only, so
dicounted as a term: "Provincia Autonoma di Trento" earned a full holder match on a query asking for Lecce. Italian stopwords added. \bis ASCII-only in JavaScript, so a term ending in an accented letter never found its word boundary:mobilitàscored 0 against "mobilità urbana",qualitàagainst "qualità dell'aria". On a catalog that is mostly not in English this silently sank every accented query. Replaced with Unicode lookarounds.
Looking for more of the same family turned up three more, all cases of a local filter running over a truncated or wrongly-normalised set:
ckan_find_relevant_datasetsnever called the parser probe.portals.jsonused to cover it; removingforce_text_fieldleft it sending boolean queries to the parser that ignores them. On dati.comune.milano.itaria OR acquareturned 0 there against 87 fromckan_package_search. Same probe now applies to both.ckan_organization_searchbuilds a Solr wildcard, which bypasses the analysis chain, so the pattern must be pre-normalised the way CKAN builds a name slug. It lowercased but did not fold accents:cittàreturned 0 whilecittamatched 135 datasets.ckan_tag_listappliedtag_queryafter faceting, withfacet.limitset to the caller'slimit. On dati.gov.it 53 tags contain "citta" and none is in the top 100, so the filter answered "no tags" while they existed. The facet is now widened when a filter is given.
openspec/specs/ckan-search/spec.md described the parser as a property of
ckan_package_search alone, which is what let the second caller go unnoticed — and after
yesterday it was also wrong, still describing the per-portal default that was removed.
Rewritten as a property of the query-building path, naming every tool that shares it.
Also raised the candidate window to at least 50: the local ranking only sees what Solr
returns first, and limit: 3 shrank it to 15 — enough when a search returned a handful
of results, not enough now.
defibrillatori Comune di Lecce puts the Lecce dataset first again. 532 tests, 5 added.
How it was missed: yesterday's verification checked result counts through
ckan_package_search, never the ranked output of ckan_find_relevant_datasets — the
third most used tool in the telemetry, and the second caller of resolveSearchQuery.
Counting results proves recall, not usefulness.
Ships #534: the text:(...) wrapper is reserved for the queries dismax cannot serve, the escaping preserves unary operators and balanced grouping, and the parser probe measures two terms taken from the catalog on every portal. force_text_field is gone from portals.json.
The usage review surfaced an LLM client reformulating the same request against
www.dati.gov.it/opendata for three hours on 29 July. The data was there from the first
attempt: bonifica siti contaminati Piemonte returns 22 datasets on that portal, the
Piedmont contaminated-sites registry first. Our text:(...) rewrite was returning 5.
package_search hands a colon-free query to Solr's dismax parser with q.op=AND,
mm='2<-1 5<80%' and qf='name^4 title^4 tags^2 groups^2 text' (ckan/lib/search/query.py).
dismax has no boolean syntax, so A OR B collapses into A AND B: on dati.gov.it
aria OR Milano returns 59, exactly what aria AND Milano returns. A colon takes the query
off dismax, which is what the wrapper exploits — the wrapped form returns 3421. The same
switch is why it hurts everything else: it drops the qf boosts that rank titles and tags
first, searches the catch-all text field alone, and ANDs every term instead of applying
mm. This is CKAN's own default, not a per-portal defect, which is why the same pattern
showed up on Milano, Toscana, Sicilia, Ucraina and open.canada.ca.
Measured on dati.gov.it: defibrillatori Comune di Lecce 678 -> 1, qualità aria Milano
650 -> 51, musei roma arte opere catalogo 9 -> 0 — the second most repeated query of the
semester, answered with an empty page while nine datasets existed. On 40 real queries from
telemetry the separation is clean: with a boolean operator the wrapper helped 8 times and
hurt none; without one it helped none and hurt 10, six of them down to zero.
So the wrapper now applies only to queries carrying a boolean operator. The shape that an
LLM client generates from a user's request — a run of keywords, no operators — is left to
the portal's own parser, boosts and mm included.
probePortalParser was rewritten and now runs for every portal, configured ones included;
force_text_field is gone from portals.json (8 entries) so there is one source of truth.
The old probe asked data OR dati, two words common enough to saturate: on Milano data
alone and data OR dati both return 2564, so it read the portal as healthy while
aria OR acqua returned 0 against 54 and 33 for the single terms. It now picks two terms
from the catalog itself — single-word tag facets between 0.5% and 30%, falling back to
frequent title words — and compares A, B, A OR B, text:(A OR B). An OR returning
fewer hits than either operand is not being honoured, and the wrapper is the answer only if
the wrapped form returns more. data.stadt-zuerich.ch, where the text field returns 0 for
every query, is correctly left alone.
Cost: a plain query pays nothing, since nothing but a boolean query can be wrapped. The
probe costs 5 extra rows=0 calls the first time a boolean query reaches a portal in a
session, then nothing.
Verified e2e against live portals: dati.gov.it 5 -> 22 and 0 -> 9 on the two queries from
the telemetry, Milano aria OR acqua 0 -> 87, Zurich unchanged at 10 and 172. 516 tests
pass, 18 added.
Patch release for the fix in #532: a 404 on datastore_search_sql now says the portal does not expose the SQL endpoint, instead of sending the caller to ckan_package_show for a resource_id that was already valid. Also ships the telemetry pipeline fixes and the DEPLOYMENT.md/CLAUDE.md realignment from earlier today.
Six things the deployment guide got wrong, all verified against the live system:
- tool count stuck at 7;
/healthreports 20 tools, 7 resources, 6 prompts on v0.4.120 - bundle "~400KB" in two places, while the doc's own sample output said 541 KiB; rebuilt: 533 KB
- the free tier presented as the configuration in use. Workers Observability keeps 7 days of telemetry on this account (measured: events at 7 days back, none at 8), which is the paid retention — the free-tier request and CPU limits are not the ones this deployment operates under
demo.ckan.orgin three curl examples, which CLAUDE.md forbids for tests- the release workflow told you to commit and push straight to
main, in both files. Now branch, PR, squash-merge, then tag on the merged commit - a stale
Co-Authored-Byline in the sample commit
Also corrected the troubleshooting entry calling CPU-limit errors "rare": 22 in the first four days of September against 11 in all of August. Left flagged as unexplained.
Left alone on purpose: archived OpenSpec proposals, historical LOG entries, and demo.ckan.org where it appears as an example for end users (README, EXAMPLES, SKILL) rather than as a test target.
A 404 on datastore_search_sql fell into the generic datastore_search branch of
formatCkanError, which answers "get a valid resource_id first: call ckan_package_show".
On a portal that does not expose SQL the resource_id is fine and ckan_package_show
hands back the same one, so a client following the hint retries into the same 404.
SQL access is optional in CKAN and widely disabled: datastore_search_sql?sql=SELECT 1
returns 404 on Toronto's portal and 200 on dati.comune.messina.it, while
datastore_search answers 200 on both. A bad table name comes back as a 400, not a 404,
so a 404 on that action means the endpoint itself is missing. The hint now says so and
points at ckan_datastore_search.
Verified e2e: Toronto returns the new hint; Messina counts 202.906 rows in a single SQL call. 498 tests pass, one added.
A usage review of worker_events_flat.jsonl surfaced three defects in the measurement itself, all confirmed against the live Observability API:
- The query filtered only on
$metadata.type = cf-worker, with noservicefilter, so events fromopensdmx-mcp(another Worker on the same account) were archived here: 7 out of 31 events in a 48h probe, 589sdmx_*records in the May-July history. Added$metadata.service = ckan-mcp-server(env overrideCF_SCRIPT_NAME); the probe then returned 24/24 ckan events. The existing records cannot be cleaned: retention is 3 days. $workers.outcomedisappeared from the API on 2026-07-12 (the object now carriesspanId/traceIdinstead). Not a regression of ours: the archiver's only refactor is 7e78b32, March. The replacement was already there and unused:src/worker.tslogs its ownstatusper call, pluscache_hit,duration_msandlimit, and the archiver was throwing all four away.resolve_outcome()now reads$workers.outcome, thensource.status, then$metadata.level, and writesunknownrather than assumingok. On a 47-event live sample it resolved every record (40 ok, 7 error, no unknown), CPU-limit crashes included. July-September stays null and cannot be backfilled: retention is 3 days.worker_daily_stats.shhad been writingok:0, errors:0on every row since that date. It now derives the error flag fromerror IS NOT NULL OR outcome IN (...), which stays valid across the schema change. March figures are unchanged; August and September are populated again (2026-09-03: 240 calls, 220 ok, 20 errors).
Flat schema gains script_version, request_id, trigger, cache_hit, duration_ms, limit. Without the first, a CPU-limit event cannot be tied to a release, and those events carry no tool or server either; they now do carry the version, so September's 22 CPU errors in 4 days (against 11 in the whole of August) become attributable from here on, not retroactively. cache_hit and limit answer the two questions this review could only guess at: cache fragmentation from URL spelling, and page size on the long datastore loops. Older records are padded to the same field set on rewrite.
Usage, excluding March (evaluation traffic: 2.953 calls, 242 portals) and the sdmx events: ~470-1.450 calls a month, median 13,5 a day, very spiky. ckan_datastore_search leads (1.142 since June), and on 2026-08-24 a single client generated 392 calls with 6 distinct queries paging Toronto's datastore. Callers spell the same portal in three or four ways (dati.gov.it/opendata, www. variant, trailing slash; open.canada.ca in three forms), which fragments the cache key.
Releases PR #531 (env-proxy refusal, multicast/reserved/site-local ranges, see 2026-09-04 entry). Also:
npm audit fix: axios 1.13.2 -> 1.20.0 (29 open advisories, including NO_PROXY/SSRF bypasses), follow-redirects -> 1.16.0, wrangler -> 4.129.0, vitest -> 4.1.11. Remaining findings (body-parser, express, qs) need express 5, a major: left for a dedicated change.src/server.tsandsrc/worker.tswere still at 0.4.118: the 0.4.119 bump skipped them because CLAUDE.md's release step lists only the three JSON files. Fixed here.- Verified: build, worker build, 497 tests, real HTTP e2e (search on dati.gov.it, datastore on Messina,
169.254.169.254still refused).
Compared our isBlockedIp and request chain against the SSRF hardening shipped in datagouv-mcp v1.0.0 (PR #126, external report). Connect-time pinning, redirect re-checks and IPv4-mapped/6to4/NAT64 unwrapping were already in place; their specific vector (fetching producer-supplied URLs from catalog metadata) does not apply here. Two gaps closed:
proxy: falseon the axios path: withHTTP_PROXY/HTTPS_PROXYset, axios connected to the proxy and the SSRF-safe lookup validated only the proxy's IP, not the target's.isBlockedIpnow also rejects 224.0.0.0/4 (multicast), 240.0.0.0/4 (reserved), ff00::/8 (IPv6 multicast) and fec0::/10 (site-local). Running their 37 test cases against our function surfaced exactly these.
Second reply from Tempy (25 August, seen on the portal - the notification was in the inbox but the question was already answered before I looked): the advisory curation team confirmed that publication of repository advisories to the Global Advisory Database is queued, reviewed manually, and that "the advisory publication queue stretches back to June 2026", with work under way to reduce it.
So nothing is wrong with the records. The pattern we measured fits the explanation exactly: the only two advisories in the global database are the oldest ones (March and May), and CVE-2026-61612 (June) sits right at the edge of the backlog. Nothing on our side accelerates this, and there is no action left - the ticket auto-closed after seven days and is best left closed.
Worth keeping: Dependabot alerts are driven from the advisory in the database, not from the CVE assignment (Support, 24 August). The thirteen CVEs this repository now holds are a public reference, not downstream protection - that arrives only when the advisory clears the queue.
First reply from GitHub Support (Tempy). It attributes the delay to an industry-wide surge in CVE publication and records being processed in order of arrival - which answers a question the ticket did not ask. The useful part is a statement that undercuts that very explanation:
The associated advisory in the GitHub Advisory Database is published independently of the CVE record, and Dependabot alerts are driven from the advisory - not the CVE assignment.
If alerts are driven from the advisory, the CVE backlog cannot explain twelve published advisories being absent from the database. Replied restating the scope, with a new data point: GHSA-x32r, published 2026-08-20, is at its third working day and still not in the database either - so this is not confined to the older records.
State unchanged: 14 published advisories, 2 in the global database, 13 carrying a CVE. Linked from the reply: "Inside the Advisory Database and what happens when vulnerability volume breaks records", which suggests the bottleneck is database curation rather than CVE issuance. If that is the answer, "queued, and the queue is long" would be enough - and is worth asking for explicitly.
The 2026-08-15 narrowing round came back green: GHSA-vqff -> CVE-2026-76894, GHSA-c499 -> CVE-2026-76895, GHSA-3369 -> CVE-2026-76896, GHSA-v3j5 -> CVE-2026-76897. Arguing dependency for GHSA-c499 (one missing neutralization pass produces both the injection and the spoofing) worked as well as splitting did for the others - no advisory had to be re-published as a new one. Every published advisory now carries a CVE except GHSA-x32r, requested today.
SECURITY.md updated. The propagation gap is unchanged and remains the subject of ticket #4682889: a CVE is necessary but not sufficient, and the global database still lists two of fourteen.
0xRomSec reported that isBlockedIp misses the IPv6 ranges that embed an IPv4 address: NAT64 (64:ff9b::/96, 64:ff9b:1::/48), 6to4 (2002::/16) and IPv4-compatible (::/96). Verified against the real function - all eight cases in the report return false. 64:ff9b::a9fe:a9fe is 169.254.169.254 written in IPv6, and the function is the single decision point for both the literal guard and the DNS-resolution guard, so the gap applied to both halves.
Fixed structurally rather than by appending a fifth prefix: parseIpv6 expands any spelling (compressed, expanded, upper-case, leading zeros, dotted-quad tail, zone id) into eight 16-bit words, and the classifier decides on values. Unparseable input fails closed. Added 2001::/32 (Teredo) and IPv4 198.18.0.0/15 as hardening - both embed public addresses and open no path to RFC1918, so they sit outside the vector.
The DNS half needed no change: createSsrfSafeLookup already passes all: true with family: family || 0 and iterates every result, so an AAAA record was always inspected - it was the classifier that waved it through. Checked before writing the fix, since a family: 4 there would have made the patch half a fix.
Tests: the eight advisory cases, four non-canonical spellings of the same address, four unparseable literals, 198.18.0.0/15 boundaries, plus three guard-level regressions that exercise the actual bypass chain (literal guard, assertHostnameResolvesSafe, connection-time lookup) rather than the classifier alone. 495 passing. Both build targets checked - http.ts is shared with the Workers bundle.
Released the full round: npm 0.4.119 (CI, provenance), GitHub release with dxt/skill assets, Cloudflare deployed and verified in prod (http://[64:ff9b::a9fe:a9fe] rejected, normal search intact), MCP Registry 0.4.119 isLatest=true.
API note, same class as the missing comments endpoint: there is no POST .../security-advisories/{ghsa}/publish - 404, and the OpenAPI description lists only get, patch, cve and forks for that path. Publishing an advisory is UI-only. PATCH (description, patched_versions) and POST .../cve do work from the API.
GitHub issued CVEs for the three advisories still pending from the July batch: GHSA-38f8 (CVE-2026-76811, High), GHSA-vmrr (CVE-2026-76812, Critical) and GHSA-q5gv (CVE-2026-76813, High). All three were already published and patched (0.4.110 / 0.4.111), so no action was required on the advisories themselves. The 2026-08-15 narrowing round (GHSA-3369, GHSA-v3j5, GHSA-c499, GHSA-vqff) is still without an outcome.
SECURITY.md now carries a GHSA → CVE → fixed-in table for all thirteen published advisories, plus a supported-versions note. Until now the CVE mapping existed only scattered across LOG entries.
Propagation remains the real gap. GET /advisories?affects=@aborruso/ckan-mcp-server still returns only two records (CVE-2026-33060, CVE-2026-53509) out of thirteen published advisories, so Dependabot has never alerted downstream users about the other eleven — including the Critical SSRF-to-cloud-metadata one. CVE-2026-61612 has had its CVE since 2026-06-22 without propagating, which argues against a plain review backlog. Escalated to GitHub Support as ticket #4682889 (org ondata, category Repositories / repository features), 2026-08-20. The portal's AI triage confirmed the reading before letting the ticket through: the documented review window after publication is 72 hours, nothing in the docs describes a further maintainer action to force propagation, and inspecting why some advisories were promoted and others not requires Support. Draft kept in tmp/github-support-advisory-propagation.md.
GitHub issued CVE-2026-73844 (GHSA-6f9w), CVE-2026-73845 (GHSA-83x6) and CVE-2026-73846 (GHSA-78x9), and rejected three others under CNA rule 4.2.11 — each advisory described more than one independently fixable vulnerability. The requests originate from GHSA-p5c9, where the reporters (Gal3m, mrostamipoor) asked for one CVE per finding.
Rather than split published advisories into new public ones, each was narrowed to a single defect, with the rest demoted to impact or deployment context:
GHSA-3369— kept the second-order SSRF (destination derived from resourceurlmetadata); dropped thePromise.allamplification claim, which a concurrency cap fixes independently. CWE-400 removed; vectorS:U/C:L/A:L(4.7) →S:C/C:L/A:N(4.0, still Medium).GHSA-v3j5— kept the missingOrigin/Hostvalidation. The MCP spec (Basic/Transports) draws the line for us: origin validation is MUST, loopback binding and authentication are SHOULD. The latter two stay as deployment context. CWE-306 and CWE-1327 removed.GHSA-c499— argued dependency rather than narrowing: one missing neutralization pass at the render boundary produces both the prompt injection and the content spoofing, and no patch fixes one without the other. The remediation section was rewritten as a single control (it previously listed two, which read as two defects). CWE-79 removed.
Also requested the CVE for GHSA-vqff, which was in the reporters' list but had neither a CVE nor a rejection — the request had apparently never been submitted.
API notes: PATCH /repos/{o}/{r}/security-advisories/{ghsa} accepts summary, description, cwe_ids, cvss_vector_string (mutually exclusive with severity); POST .../{ghsa}/cve re-requests the CVE. There is no comments endpoint (404, absent from the OpenAPI description) — advisory comments were posted by replying to the GitHub notification email.
Worth noting for expectations: a CVE is necessary but not sufficient for Dependabot alerts. Only 2 of the repo's advisories are in the global GitHub Advisory Database; CVE-2026-61612 has had a CVE since 2026-06-22 and is still not propagated.
GHSA-r8hw-3fch-r42w reported the v0.4.108 SSRF fix as bypassable by DNS rebinding, with a PoC pointing ckan_package_search at 7f000001.7f000001.rbndr.us. Reproduced: the PoC is blocked. Both halves of that label decode to 127.0.0.1, so the name never rebinds — it is a plain DNS-name-to-loopback, the case v0.4.108 already closed. Replaying it against localtest.me with a listener on 127.0.0.1:8054 is rejected by createSsrfSafeLookup, and a genuine rebinding fake-DNS (public IP first, loopback second) through the axios agent produces exactly one lookup with the connection pinned to the validated address.
The class was right on the wrong path, though. safeFetch() — used by sparql_query and the MQA quality tools — validated the hostname with assertHostnameResolvesSafe() and then let undici resolve it a second time, which is a real TOCTOU window. getSafeDispatcher() now builds an undici.Agent({ connect: { lookup: createSsrfSafeLookup(dns) } }) and safeFetch passes it on every hop, so the socket connects to the address that was just checked. assertHostnameResolvesSafe() stays as defence in depth for runtimes without a dispatcher (Workers, where the CF sandbox blocks internal egress anyway).
undici becomes an explicit dependency: it was only present transitively through wrangler, i.e. absent in production installs. The import specifier is assembled at runtime (["und","ici"].join("")) because esbuild constant-folds "undici" + "" and would otherwise bundle all of undici into dist/index.js (207 KB → 969 KB) and break the browser-platform Workers build.
Same trap on the other side: because the specifier is dynamic, the DXT bundle cannot contain undici either, and a .dxt unpacked by Claude Desktop has no node_modules to resolve it from — the dispatcher would have been null and the pin silently inert. pack:dxt now copies node_modules/undici into dxt-staging/server/node_modules/ (+1.7 MB uncompressed), verified by resolving it from a copy of the staging dir outside the repo.
engines.node moves to >=18.17.0, undici 6's own floor — the old >=18.0.0 would have promised installs that cannot work.
Also fixed the serverInfo version, hardcoded at 0.4.108 in src/server.ts and src/worker.ts while the package was at 0.4.117.
Field names, titles, cell values and every other short string coming from a third-party portal were interpolated into markdown structure unescaped. Since every response here is read by a model, that has two effects: a newline ends the construct and opens a line that reads as server-authored — indistinguishable from the > **Note**: lines this server writes to instruct the model — and an unescaped | adds table cells, shifting later values under the wrong header.
Worst case was ckan_analyze_datasets, which used none of the existing defenses and rendered the DataStore Data Dictionary (info.notes, publisher free text) straight into a bullet list. A newline let a portal fabricate a field entry, in the very tool an agent calls to learn which fields exist.
sanitizeInline now sits in utils/formatting.ts beside wrapUntrusted and safeUrlText: collapses newlines, escapes pipes, and replaces backticks with U+02BC so a value cannot close the inline code span it is rendered in. It promotes the private copy that lived in datastore.ts and retires the three ad-hoc .replace(/[\r\n]+/g, ' ') copies, which stripped newlines but escaped neither pipes nor backticks.
Long free text keeps wrapUntrusted — a fence states "this is data" better than escaping — and was already safe. The JSON path was never affected. URL query parameters use encodeURIComponent, not the markdown escaper: & and # would otherwise pass through.
Three review rounds on #45, each finding real gaps. Two are worth recording. The first automated sweep matched markdown +=, so every renderer's opening let markdown = \# ...`` escaped it — all four top-level headings. And the sweep wrapped two scoring weights that are our own numbers, not portal strings; both reverted. A grep-driven audit produced a false sense of completeness twice; if anything else surfaces, the answer is a generative test that enumerates portal fields, not a fourth grep.
485 tests, 15 of them new.
Ships the two DataStore output fixes below, plus .greptile/rules.md: the automated review on #43 produced one confident false positive, so the invariants a generic reviewer cannot know are now stated in the repo.
Both markdown renderers in src/tools/datastore.ts cut the record table at 8 columns with no notice. On the Messina electoral-lists resource (14 columns) the table stopped right before cognome, nome, sesso and voti: a model reading it saw an election dataset with no votes and no candidate names, and nothing told it anything was missing. Row truncation was already handled properly (... and N more records, Total Records); columns were not.
Both renderers now append a note naming the omitted columns and pointing at the way to retrieve them (fields parameter for search, an explicit SELECT list for SQL, or response_format: "json"). The JSON path never had the bug — compactDatastoreResult passes every column through.
Found while checking what the GovInsider piece on the OKFN Brazil/Uruguay pilot added to docs/future-ideas.md (nothing new — it covers the same pilot already recorded on 2026-06-11), but its failure mode is exactly this: the model fills a gap it cannot see.
Surfaced by the end-to-end check above: datastore_search_sql on SELECT * returns CKAN's internal _full_text column, which repeats the whole row as one concatenated string. It was taking the first table slot and pushing out a real column, and the same query reported 15 columns via SQL against 14 via datastore_search. _id was already filtered; _full_text now is too, in both renderers and in the JSON output (where it was pure token waste). The two tools now agree on the column count.
465 tests pass; verified end to end against dati.comune.messina.it.
First release published from CI. No functional change: this exists to exercise the release workflow added earlier today end to end — tag guard, OIDC authentication, provenance attestation — rather than discovering whether it works during a release that actually matters.
Worked on the first try: the job went green in 34s and the attestation binds the tarball to ondata/ckan-mcp-server, workflow release.yml, ref refs/tags/v0.4.115, on a GitHub-hosted runner. npm view @aborruso/ckan-mcp-server@0.4.115 dist.attestations returns an SLSA v1 provenance; the same query on the hand-published 0.4.114 returns nothing.
One snag worth remembering: mcp-publisher failed with an expired JWT, and re-running login github then died with incorrect_device_code before the browser step. Neither error named the real cause — the local binary was 1.5.0 from 6 March, and the device-auth flow changed by 1.8.0. Updating the binary fixed it. When the registry token expires, check the publisher version too: both were installed the same day and go stale together.
npm view @aborruso/ckan-mcp-server@0.4.114 dist.attestations came back empty — packages were published by hand from a local machine, with nothing binding a tarball to the commit that produced it. Adopters could check which version was current (see below) but not where it came from.
- New
.github/workflows/release.yml: triggers onv*tags, verifies the tag matchespackage.jsonbefore anything else (npm publishes are irreversible after 72h), thennpm ci→ build → tests →npm publish --provenance. Guard tested both ways;npm test -- --runconfirmed against the current 461-test suite. package.jsonhad norepositoryfield — a hard prerequisite for provenance, which would have failed the publish. Set toondata/ckan-mcp-server.- Release workflow in
CLAUDE.mdrewritten rather than extended: step 5 now warns that pushing the tag is the publish, step 9 says explicitly not to runnpm publishby hand (two paths would collide onEPUBLISHCONFLICT), and step 10 must wait for the CI run to go green because the MCP Registry validates that the npm version exists. .readme-full.mdadded to.gitignore: theprepack/postpackpair swaps in the short npm README, so a local publish failing between the two hooks leaves the wrongREADME.mdin the tree, onegit add .away from being committed.- Authentication is trusted publishing (OIDC), not a secret. The first draft used an
NPM_TOKEN;npm profile getthen surfaced npm's own warning that "tokens that bypass 2FA are being restricted for direct publishing", which pointed at the mechanism npm now recommends instead. Trusted publishing needs no stored credential, emits provenance by default, and — because the trusted publisher names the GitHub repo explicitly — also settles the open question about the npm scope (@aborruso) differing from the GitHub org (ondata). The workflow upgrades npm to ≥ 11.5.1 explicitly: Node 22 ships npm 10.x, which falls back to token auth silently and would publish unattested.--provenanceis kept although implied, so that degradation fails the job instead of passing quietly. - Not yet active: the trusted publisher has to be registered by hand on npmjs.com (package → Settings → Trusted Publisher → GitHub Actions →
ondata/ckan-mcp-server/release.yml). Until then the workflow runs and fails at the publish step. Note that the workflow filename is part of that identity.
Published and verified against the public endpoint: the registry now serves 0.4.114 with isLatest: true (updated 2026-08-03T06:01:07Z), and the old 0.4.83 record dropped to isLatest: false.
The official MCP Registry entry for io.github.aborruso/ckan-mcp-server had been stuck at 0.4.83 since 2026-03-12 — 31 patch releases and almost five months behind npm, while still flagged isLatest: true. Clients resolving the server through the registry were pointed at a build predating the v0.4.108 SSRF remediation, and 0.4.83 is still installable from npm.
- Root cause, not a one-off slip: the Release Workflow in
CLAUDE.mdlisted the version bump forpackage.jsonandmanifest.jsonbut neverserver.json, andnpm publishdoes not touch the registry. The drift was structural and would have kept growing. - Fixed
server.json(bothversionandpackages[0].version— two fields, easy to half-update) and rewrote the release workflow:server.jsonadded to step 1, a new step 10 formcp-publisher publishplaced afternpm publishsince the registry validates that the npm version exists, plus acurlone-liner to verify the published entry. - Surfaced by an unsolicited vendor email selling a £395 "MCP Readiness Audit". The sales pitch was worthless — the remedy it offered has nothing to do with the defect — but the three technical claims all checked out under verification. Worth recording: the finding was real and cost the sender two
curlcalls, which is exactly how long it would have taken us to catch it ourselves with a check in the release procedure. - Registry tokens (
.mcpregistry_*) verified: gitignored, never committed.
structuredContent capped like the text (closes #39).
The v0.4.113 cap applied to content[].text only, so any client reading the structured channel got the full payload — ckan_tag_list limit=1000 on dati.gov.it: ~50K of text against 65,382 uncapped characters. The limit was a fiction for those clients.
- New
jsonToolResult(): truncates once viatruncateJsonand parses the result back intostructuredContent, so the two channels cannot disagree and_truncated/_original_countreach structured readers too. Applied to 13 call sites. ckan_status_showandckan_find_portalspair structured output with Markdown text, so there is no truncated JSON to derive it from: they cap it on its own via the newcappedStructured(). My first pass left them uncapped calling them "bounded by shape" — wrong, as review pointed out:status_showis echoed straight from the portal, and the 50-result limit onfind_portalsbounds the number of entries, not the length of their titles and URLs. Only theall_fields=falsecount branches ofckan_organization_list/ckan_group_liststay outside the cap: a single integer.- Correction: the deferral in v0.4.113 claimed capping would drop rows from
datastore-table-ui. That was wrong and never verified — the UI resource is commented out insrc/resources/index.ts:18and never registered, andckan_datastore_searchreturns nostructuredContentat all. No exception was needed. Same failure mode as the bug being fixed: a plausible claim about the code that nobody checked. - 7 new tests (461 total), covering a payload long in a single field rather than in many, and a bounded-count list whose entries are individually oversized. Docs realigned:
README.md,docs/JSON-OUTPUT.md,docs/DECISIONS.md,CLAUDE.mdall asserted the uncapped behaviour. - Released end to end: tag, GitHub release with DXT and skill, npm
0.4.114, Cloudflare version36b611fd. Verified on the public endpoint after waiting for edge propagation —tag_list limit=1000: 49,938 characters on both channels, byte-identical,_truncatedset.status_showandfind_portalslegitimately differ across channels: Markdown text against JSON structured output, two renderings of the same data. - Process note: check edge propagation with an active probe, not a fixed wait. The v0.4.113 verification measured the old code because it ran seconds after deploy, and briefly looked like a broken release.
Released the same day: tag, GitHub release with DXT and skill, npm 0.4.113, Cloudflare version 1cac40ea. Also merged #38 (docs: document user-facing limits, contributed by averyquinnhq), which closed #37, and disabled the claude-code-review workflow: it cannot pass on fork PRs, since GitHub withholds secrets and the OIDC token for pull_request events from forks, so the job dies on token setup regardless of the diff. The file is kept in the repo. CONTRIBUTING.md now also states that build:tsc is not a gate — a contributor exhausted 8 GB of heap on it before reporting they could not pass it, because the warning lived only in CLAUDE.md.
JSON output always parseable (issue #39).
The docs promised "always valid JSON"; the code did not deliver it. Surfaced while reviewing PR #38 (an AI contribution by averyquinnhq), which documented the real behaviour and contradicted issue #37 — the PR was right.
- Live repro:
ckan_tag_list limit=1000on dati.gov.it → 50,046 characters,JSON.parsefails. Not a corner case: an ordinary call. truncateJson: the last-resort branch cut the serialized string (truncateText(JSON.stringify(...)), with an in-code comment admitting "may produce invalid JSON"). It now empties arrays progressively and, when that is not enough, degrades to a valid{_truncated, _error}.SHRINKABLE_KEYSextended (+rows,datasets,portals,facets,fields) with a sacrifice order: bulk rows first,fieldslast.- 8 tools + 4 Resources used
truncateText(JSON.stringify(...))directly: routed throughtruncateJson.sparql_queryappended/* output truncated */to cut JSON (JSON has no comments): rewritten. - 4 tools with no cap at all:
ckan_get_mqa_quality,ckan_get_mqa_quality_details(bare JSON.stringify),ckan_find_portals,ckan_status_show(markdown). - A third unparseable path: no
catchhonouredresponse_format— every error came back as prose even withjson. NewformatError(); errors now return{error, _error: true}plusisError: true. Zod validation errors stay textual (emitted by the SDK before the handler runs). structuredContentstays uncapped (65,382 characters against 50,000 of text on tag_list): capping it would drop rows fromdatastore-table-ui, which consumes it. Decision deferred, documented indocs/DECISIONS.md.- 10 new tests (
truncateJsonhad none), 453 passing. E2e:tag_listfrom PARSE-FAIL to parse-ok; real errors parseable in json mode. CONTRIBUTING.md: section on AI-assisted contributions (disclosure, small diffs, verifiable claims).- Review follow-up:
addDemoFooter()was appended to JSON output inquality.ts, breaking parsing on Workers — now wraps the Markdown branch only, insidetruncateText. AddedisError: trueto the two non-dati.gov.it guards. ThetruncateJsonfallback now degrades further so it always respects small limits. 454 passing.
Security — Round 3: hardening (closes the last group of advisories in triage).
- Error reflection:
makeCkanRequestno longer embeds the upstream body (JSON.stringify(decodedData)) in the error returned to the caller — now a generic action-scoped message, with detail on stderr only (truncated).worker.tscatch-all: removederror.messagefrom the JSON-RPCdatafield (server-side logging only). Closes the semi-blind read channel of the SSRF. - postMessage UI (
resources/datastore-table-ui.ts): the host origin is pinned from the reply to theui/initializehandshake; messages carrying data are accepted only from that origin, and outbound messages use an explicit target origin (never'*'). - Prompt injection on org/group: extended the c499 containment to the
organization.tsandgroup.tsrenderers —descriptionin an untrusted block (wrapUntrusted), newlines collapsed in lists. - 3 new tests (error no-leak, org/group description fencing); 443 passing. E2e: generic error on 404 (no internal body), normal requests fine. Worker build fine.
Security — Round 2: three distinct low-cost bugs.
- MQA allowlist bypass (
isValidMqaServer,tools/quality.ts): unanchored regex replaced by URL parsing plus exact host comparison (dati.gov.it/www.dati.gov.it). Suffix (dati.gov.it.attacker.com) and userinfo (dati.gov.it@attacker.com) tricks are now rejected. - Decompression bomb / unbounded buffering (
utils/http.ts): response size cap (maxContentLength/maxBodyLengthon axios plus a byte check onarrayBufferin the fetch branch, default 32MB) and decompression output cap (maxOutputLengthon gunzip/brotli/inflateSync plus a check on DecompressionStream, default 64MB). Override viaCKAN_MAX_RESPONSE_BYTES/CKAN_MAX_DECOMPRESSED_BYTES. Stops OOM/stalls from hyper-compressed payloads. - Cache-key collision (
utils/cache.ts):canonicalizeParamsnow produces typed canonical JSON (recursive sort) instead of ak=vjoin with unescaped&;buildCacheKeyframes it inJSON.stringify([url,action,canon]).{q:"budget",rows:10}and{q:"budget&rows=10"}no longer collide. - 2 new tests (plus updated regressions); 440 passing. E2e: valid MQA host reaches data.europa.eu, bypass rejected, normal requests fine. Worker build fine.
- Further hardening in progress for upcoming releases.
Security — Round 1: SSRF cluster on the fetch path (GHSA-vmrr, GHSA-38f8; GHSA-8hxx clarified):
- Centralized
safeFetch()inutils/http.ts:redirect:"manual"plus re-validation of every hop (validateServerUrl+assertHostnameResolvesSafe), bounded hops,httpsOnlyoption. Closes redirect-SSRF (e.g. a public endpoint 302-ing to169.254.169.254) without breaking legitimate canonical redirects. Used bysparql_query(3 fetches) and by the MQA metrics fetch inquality.ts. assertHostnameResolvesSafeis now fail-closed: it distinguishes "DNS module absent (Workers) → no-op" from "resolution failed → throw". Previously a DNS error let the request proceed (fail-open / TOCTOU).- GHSA-8hxx: verified that WHATWG
URLalready normalizes IPv4 encodings (int/hex/octal/short) to dotted-decimal beforevalidateServerUrl→ the existing check already covers them. No code added (the advisory PoC tested the regex against raw strings, not the parsed hostname). Only a comment and regression tests. The remaining::7f00:1(IPv4-compatible IPv6) is not routable, as the advisory itself concedes. - MQA fetch: constant host (
data.europa.eu), so hardening for consistency rather than a real vector. - 6 new tests (redirect→internal blocked, redirect→non-HTTPS rejected, fail-closed on DNS error, IPv4 encodings blocked). 438 passing. E2e: Wikidata via
safeFetchfine, internal IP blocked. Worker build fine. - Further security hardening in progress for upcoming releases.
Security hardening — 3 advisories, "poison and door" (environmental risk before the knives):
- GHSA-3369 (second-order SSRF,
ckan_list_resources): source-portal probing is now opt-in (check_source_portaldefaults tofalse). It used to be ON: listing a dataset's resources contacted hosts and ports taken from the dataset's own data (confused deputy + port-scan oracle + amplification viaPromise.all). Added: ports other than 80/443 dropped inextractSourcePortal(useshostname, nothost), fan-out capped at 10 probes. - GHSA-c499 (indirect prompt injection): free-text portal fields (
notes, resourcedescription) were rendered verbatim in the output. They are now wrapped in a delimiteduntrustedblock with a warning (wrapUntrusted), with inner fences neutralized; portal URLs are scheme-validated (http/https only) and rendered as inline code (safeUrlText);ckan_list_resourcestable cells are neutralized (|, newlines). Containment, not a complete fix — documented for integrators. - GHSA-v3j5 (exposed HTTP transport): binds to
127.0.0.1by default (was0.0.0.0),enableDnsRebindingProtectionplusallowedHosts/allowedOrigins.docker-compose.ymlpublishes on127.0.0.1:3000:3000(withCKAN_HTTP_HOST=0.0.0.0inside the container). New env vars:CKAN_HTTP_HOST,CKAN_HTTP_ALLOWED_HOSTS,CKAN_HTTP_ALLOWED_ORIGINS. Closing this door downgrades the whole SSRF cluster from "remote" to "local". - 4 new tests; 432 passing. Verified e2e against a real HTTP deployment (loopback bind, 403 on disallowed Host, no probing by default, notes fenced). Worker build fine.
- Docs: README (HTTP env var table), SKILL (source-portal now opt-in), docker/README, docker-compose.
- Not done yet (fetch SSRF cluster, MQA regex, cache, decompression bomb, postMessage UI): knives and hardening, in later rounds.
- Security fix (GHSA-798p-78g2-v556): close DNS-name SSRF bypass —
validateServerUrlonly checked the hostname string, so a name resolving to an internal IP (e.g.lvh.me→127.0.0.1,*.nip.io→ cloud IMDS) bypassed the guard. Added DNS resolution + validation of every resolved IP, with connection pinning via a customlookupagent (closes DNS-rebinding and redirect-to-internal) on the Node/axios path; pre-resolution check on the fetch-basedsparql_query(HTTPS-only). ExtractedisBlockedIpshared by literal and resolved-IP guards.maxRedirects: 5on CKAN requests. - Hardening: the network-exposed HTTP transport now refuses to start without
CKAN_ALLOWED_DOMAINS(default-deny), unless explicitly opted out withCKAN_HTTP_ALLOW_ALL=true(logs a warning). stdio stays open. Cloudflare Worker unaffected (CF sandbox already blocks internal addresses). - 11 new tests (isBlockedIp, SSRF-safe lookup, allowlist gate, DNS-bypass on sparql). Verified end-to-end against a real HTTP deployment.
- Reported by: EchoSkorJjj
ckan_package_show: surface DCAT-AP fields already returned by package_show but previously hidden in markdown output — Rights Holder (holder_namevia readDcatExtra), Publisher (publisher_name), Update Frequency (frequency), Language (language), Access Rights (access_rights). Each printed only when present; no new tools or API calls.conforms_todeliberately excluded (renders as raw JSON). Verified on dati.gov.it (DCAT-AP-IT) and open.canada.ca (no regression).
- Security fix (GHSA-g84h-j7jj-x32p): block
ip6-localhostandip6-loopbackSSRF bypass — hostname aliases present in/etc/hostson Linux that resolve to::1but bypassed the existing SSRF filter (GHSA-3xm7-qw7j-qc8v); replaced singlelocalhostcheck with a blocked-hostnameSet; 2 new unit tests added - Reported by: hibrian827
- Fix (scoring):
ckan_find_relevant_datasetsnow scoresholder_name(DCAT-AP_ITdct:rightsHolder) andpublisher_name(dct:publisher) as distinct weighted fields, separate fromorganization - Rationale: on federated catalogs (e.g.
dati.gov.it, but the pattern applies to any portal harvesting from sub-publishers),organizationis the harvesting catalog (e.g.regione-puglia), NOT the data owner. Queries like "datasets from Comune di Lecce" previously scored 0 on the owner field when the dataset was harvested via Regione Puglia or a local action group, missing the actualrightsHolder - The fields are read from
extras[](the authoritative DCAT-AP_IT location on Italian portals) with fallback to root-level. On dati.gov.it,package_searchexposesholder_nameandpublisher_nameboth inextras[](correct DCAT values) and at root (often overwritten by the harvester with the organization name); reading only the root would be wrong. The root-level fallback preserves correct behavior on non-DCAT-AP_IT portals (data.gov, open.canada.ca) - Bug surfaced on real-world Puglia datasets:
defibrillatori-esterni(extras.holder=Comune di Mesagne, root.holder=GAL Terra dei Messapi, organization=GAL Terra dei Messapi) anddefibrillatori-dae-progetto-comune-cardioprotetto(extras.holder=Comune di Lecce, organization=Regione Puglia) - Defaults:
holder=4(peer withtitle— actual institutional owner per DCAT-AP_IT),publisher=2(lower because sometimes a technical role like "Redazione OD" rather than the institution) - API:
weightsobject accepts two new optional fields (holder,publisher); backward-compatible — clients not setting them get the improved scoring by default - Types: added
holder_name?: stringandpublisher_name?: stringtoCkanPackageinterface (previously accessed via index signature) - Added internal helper
readDcatExtra(dataset, key)that encapsulates the extras-first, root-fallback lookup - Score breakdown markdown and JSON outputs include
holderandpublisherper dataset - Validated against live
package_searchresponses on dati.gov.it: defibrillatori-esterni (Mesagne) 6 → 12, comune-cardioprotetto (Lecce) 10 → 13
- Source portal DataStore fallback + LLM error hints (see 2026-05-18 entries below)
- Source portal DataStore fallback:
ckan_list_resourcesnow probes the source portal when a resource hasdatastore_active=falseand its download URL belongs to a different CKAN instance (harvested dataset pattern). Addssource_datastore_activeandsource_portal_urlfields to output. Newcheck_source_portalparameter (defaulttrue) to skip extra HTTP calls. NewextractSourcePortal()utility inurl-generator.ts. Scoped to CKAN-to-CKAN harvesting (detects/resource/{uuid}/URL pattern). 12 new tests → 399 total. - LLM error hints: add
CkanApiErrorclass tomakeCkanRequest(carriesstatus+action); addformatCkanError()with hint table mapping HTTP status/action → actionable suggestion for the LLM (404 datastore →ckan_package_show, 404 package →ckan_package_search, 400 SQL → check columns, 503 → retry, etc.) - Replace raw
error.messageinterpolation in all tool catch blocks (datastore, package, organization, group, analyze, portal-discovery, quality) withformatCkanError() - Replace fragile string-match in
organization.ts(includes('CKAN API error (500)')) witherror instanceof CkanApiError && error.status === 500 - Tests: 9 new unit tests for
CkanApiErrorandformatCkanError→ 387 total (381 pass, 6 skipped)
- Security: add optional domain allowlist (
CKAN_ALLOWED_DOMAINSenv var) invalidateServerUrl(); blocks requests to unlisted public domains — CERT-AgID MCP recommendation - Security: add structured audit logging to stderr for all
makeCkanRequestcalls in Node modes (stdio + http); fields:ts,server,action,cache_hit, query params — CERT-AgID MCP recommendation - Tests: 8 new tests (4 allowlist, 3 audit log) → 383 total (377 pass, 6 skipped)
- Add
cache_hitfield to worker telemetry log entries getLastCacheHit()exported fromsrc/utils/http.ts;worker.tslogs after response withcache_hit: true/false- Enables cache hit rate analysis in
worker_events_flat.jsonl
- Add per-portal upstream rate limiter (
src/utils/rate-limiter.ts) - Token bucket algorithm, one independent bucket per hostname
- Integration in
makeCkanRequest: cache hit bypasses limiter; only upstream fetches consume tokens - Config:
CKAN_RATE_LIMIT_ENABLED,CKAN_RATE_LIMIT_RPS(default 5),CKAN_RATE_LIMIT_BURST(default 10),CKAN_RATE_LIMIT_MAX_WAIT_MS(default 5000) - Per-call bypass via
opts.rateLimit: false; disabled by default in Vitest runs RateLimitErrorthrown when wait exceedsmaxWaitMs(includes hostname + wait ms)- 13 new unit tests; suite now 370 tests, all green
- OpenSpec:
add-upstream-rate-limiter
- Add read-through HTTP cache layer in
makeCkanRequest(src/utils/cache.ts) - Backends: Cloudflare Cache API on Workers, bounded in-memory LRU on Node
- Action-based TTL: metadata 300s, datastore 60s, status 3600s; fallback configurable
- Env vars:
CKAN_CACHE_ENABLED,CKAN_CACHE_TTL_DEFAULT,CKAN_CACHE_MAX_ENTRIES,CKAN_CACHE_MAX_ENTRY_BYTES - Per-call bypass via
makeCkanRequest(..., { cache: false }); errors and oversize payloads never cached - 29 new unit tests; suite now 357 tests, all green
- Measured locally: 348 ms cache miss → 13 ms cache hit on identical
package_searchcalls (~27×) - OpenSpec:
add-http-cache-layer(proposal + design + spec)
- Fix
resolveSearchQuery:*:*and fielded queries (e.g.title:X) are no longer wrapped intext:(...)when auto-detected parser override is "text" —escapeSolrQuerywas turning*:*into\*\:\*, returning 0 results on portals like dati.comune.messina.it
- Fix
ckan_package_search: quoted phrases (e.g."aree protette") now work insidetext:(...)wrapping — removed"fromescapeSolrQueryspecial chars so phrase queries are preserved
- Improve tool parameter descriptions for Code Mode compatibility (group, tag, package tools)
- Add
.describe()to all parameters inckan_group_list,ckan_group_show,ckan_group_search - Add
.describe()to all parameters inckan_tag_list - Improve
ckan_package_showparameter descriptions (server_url example, id UUID/slug hint)
- Add
dati.regione.sicilia.it(Sicily open data portal) toportals.jsonwithforce_text_field: false— CKAN 2.6.8 does not support thetext:(...)query wrapper
- Show
Portal Locale(locale_default) inckan_status_showmarkdown output - Add query language hint to
ckan_package_searchdescription: check locale before searching - Update skill query construction rule: dynamic locale check via
ckan_status_showinstead of hardcoded portal table
- Add
data.gov.ua(Ukraine open data portal) toportals.jsonwith explicitforce_text_field: falseto prevent auto-detection from incorrectly wrapping Solr queries intext:(...), which caused 0 results
- fix(worker): return 405 immediately for GET /mcp — prevents bot scanners from hanging the Worker and consuming CPU quota
- fix(http): create server + transport per request — SDK 1.26.0 security fix enforces stateless transport cannot be reused across requests
- fix(worker): create server + transport per request — SDK 1.27.1 enforces stateless transport cannot be reused across requests
- chore(deps): update
@modelcontextprotocol/sdk1.25.2 → 1.27.1 — fixes Claude Desktop crash with MCP protocol2025-11-25
- fix(
tools/organization.ts):ckan_organization_searchlowercases pattern before Solr query — fixes case-sensitive search (e.g. "Roma" → no results)
-
fix(
tools/organization.ts):ckan_organization_searchnow showsview_urlin markdown table and JSON output;ckan_organization_showJSON includesview_url— all usingportals.jsoncustom patterns -
evals(
evals/tool-selection): tool selection eval pipeline — synthetic NL query generation (Gemini, 1583 records), train/eval split, fine-tuning Qwen2.5-0.5B with Unsloth on Colab T4; 86.3% accuracy on 8 tools; model published at huggingface.co/aborruso/ckan-tool-selector
- fix(
tools/sparql.ts): add; charset=utf-8to POST Content-Type — fixes accented chars corruption in SPARQL queries (issue #22)
- docs(
tools/datastore.ts): add security note tockan_datastore_search_sql— clarifies SQL forwarding boundary; bump v0.4.86 - security(
tools/sparql.ts): applyvalidateServerUrl()tosparql_query— blocks SSRF via private IPs (gap from GHSA-3xm7-qw7j-qc8v); 1 new test
- security(
utils/http.ts): addvalidateServerUrl()— blocks SSRF via private/loopback IPs, link-local, and non-HTTP/S protocols; 15 new tests
- fix(
worker.ts): add OPTIONS preflight handler at/mcp— was returning 405, blocking CORS-based connectors (e.g. Dify)
- chore(
package.json,server.json): fix mcpName toio.github.aborruso/ckan-mcp-server(ondata org not authorized); bump to v0.4.83 - chore(
package.json,server.json): addmcpNamefor MCP Registry publication (io.github.ondata/ckan-mcp-server); bump to v0.4.82
- fix(
worker.ts+worker_telemetry_flatten.py):ckan_find_portalsnow logscountry,language,has_datastore,min_datasets— previously alwaysnullinworker_events_flat.jsonl - fix(
worker.ts):sparql_querynow logs correctserverfield — was always""because logging code readserver_urlbut SPARQL tool usesendpoint_url; fix:a['server_url'] ?? a['endpoint_url'] ?? ''
- fix(skill): bilingual query rule made conditional — use native language only on monolingual portals (dati.gov.it → IT, data.gov → EN); bilingual only on multilingual portals (data.europa.eu, open.canada.ca)
- fix(skill): geographic qualifiers (city/region/country) must never be OR-joined with topic terms — always use
fqor AND; OR-joining a place name inflates results with off-topic datasets
- feat(
package.ts): auto-detect Solr parser mode for unknown portals via probe querydata OR dati(2 parallel rows=0 calls); if text count > default count × 2 → use text:(...) wrapping; result cached per portal URL for session lifetime - feat(
portal-config.ts): addisPortalSearchExplicitlyConfigured()to distinguish known vs unknown portals - fix(
package.ts): auto-convertNOWdate math to ISO dates forissuedandmodifiedfields inqandfq— these are CKAN extra fields not native Solr fields, soNOWsyntax returned 0 results on all portals tested (dati.gov.it, catalog.data.gov); ISO dates work universally - fix(
package.ts): fixcontent_recentclause to use ISO date forissuedfield (same root cause) - docs(
tools.md): clarify date math limitation —NOWonly works onmetadata_modified/metadata_created;issued/modifiedrequire explicit ISO dates (now auto-converted by server)
- fix(
http.ts): removeReferer,Sec-Fetch-*,Upgrade-Insecure-Requestsfrom axios headers — these triggered WAF block on BA Data (data.buenosaires.gob.ar) and other portals with strict WAF rules; dati.gov.it unaffected
- fix(
ckan_find_portals): deduplicate portals by hostname, preferring https over http - feat: new tool
ckan_find_portals— discovers CKAN portals from datashades.info registry (~950 portals); filters by country, keyword, min_datasets, language, has_datastore; LLM translates country to English
- fix: use
z.coerce.number()for all numeric tool parameters — fixes validation errors when MCP clients pass numbers as strings (closes #16)
- feat:
package_shownow includesapi_json_urlfor dataset and each resource (direct CKAN API JSON link) - fix:
api_json_urluses portal-specific API path viagetPortalApiPathinstead of hardcoded/api/3/action
- fix:
package_showJSON now includeshvd_category,applicable_legislation,frequency,language,publisher_name,holder_name - fix:
fqparameter docs warn about correct Solr OR syntax andextras_prefix for CKAN extras fields
- feat: new MCP prompt
ckan-search-hvd— guided HVD search usinghvd.category_fieldfromportals.json; fallback to keyword search for unconfigured portals
- feat:
ckan_status_shownow shows HVD dataset count when portal hashvd.category_fieldconfigured inportals.json
- feat:
ckan_package_searchandckan_package_showJSON output now includeview_urlfield pointing to the dataset page on the source portal
- feat: SPARQL endpoint config in
portals.json—sparql.endpoint_url+sparql.methodper portal (Italy:lod.dati.gov.it/sparql, GET-only) - feat:
ckan_status_shownow shows SPARQL endpoint when configured for the portal - fix:
sparql_query— GET fallback on 403/405 for endpoints that reject POST; User-Agent set toMozilla/5.0 (compatible; CKAN-MCP-Server/1.0)(required by AWS WAF on lod.dati.gov.it) - refactor:
portal-config.ts— addedSparqlConfigtype,getSparqlConfig(endpointUrl),getPortalSparqlConfig(serverUrl); tool count 18→19
- improve:
sparql_query— validate SELECT-only, auto-inject LIMIT (default 25, max 1000), truncate output at CHARACTER_LIMIT; +11 tests (310 total) - remove:
europa_dataset_searchtool and related files (src/tools/europa.ts,src/utils/europa-http.ts, Europa types insrc/types.ts, docs, tests); tool count 19→18
- feat: add
sparql_querytool — execute SPARQL SELECT against any public HTTPS endpoint (e.g. data.europa.eu/sparql, DBpedia, Wikidata) - feat(europa): update
europa_dataset_searchdescription to suggestsparql_queryfor publisher aggregations and queries not exposed as facets - test: add 12 tests for
sparql_query(querySparqlEndpoint, formatSparqlMarkdown, formatSparqlJson)
- feat(europa): add
publishertoALLOWED_FACETSineuropa_dataset_search— now shows top publishers per query
- fix(europa):
q=*now correctly returns all 1.7M+ datasets — Europa API ignoresqwhen omitted; sendingq=*was causing Elasticsearch to return only ~6k results; match-all queries (*,*:*) now omit theqparameter
- feat: expose Europa API facets in
europa_dataset_search— country, format, categories, and 15 more facet types now rendered as tables (markdown) and compact objects (JSON, top 15 items per facet) - feat: add
is_hvdboolean filter toeuropa_dataset_search— search only among High Value Datasets - refactor: filter Europa facets to 8 useful ones (country, categories, format, is_hvd, scoring, language, subject, hvdCategory), resolve multilingual titles to requested lang, JSON output 52KB→12KB
- feat: compact JSON output for heavy tools —
package_search,package_show,organization_list/show,group_list/show,datastore_search/search_sqlnow return only essential fields in JSON mode (~70% token reduction) - feat:
truncateJson()— JSON-safe truncation that shrinks arrays instead of cutting mid-string, always produces valid JSON - fix: filter
_idfield from datastore JSON output (already done in markdown) - docs: add
docs/JSON-OUTPUT.md— complete field schema for all tools in JSON mode - feat: add
europa_dataset_searchtool for European Data Portal (data.europa.eu) — searches 1.7M+ datasets across all EU countries with country filter, multilingual support, and HVD badge - new files:
src/utils/europa-http.ts,src/tools/europa.ts,tests/integration/europa.test.ts - types: add
EuropaDataset,EuropaDistribution,EuropaMultilingualField,EuropaLabelledValueinterfaces - JSON output filters multilingual fields to requested language only (compact response)
- fix(http): add AbortController 30s timeout to Workers fetch — prevents hang when CKAN server is slow (root cause of Worker timeout errors)
- packaging: add DXT one-click install support —
manifest.json+npm run pack:dxtscript producesckan-mcp-server.dxtfor Claude Desktop - docs: add "One-click install" section in README Claude Desktop section
- release workflow: DXT artifact now uploaded to GitHub releases
- tools: add HVD note on synthesis queries — when
q=*:*+ org/tag facets (orrows=0) on dati.gov.it, auto-fetch real-time HVD count and append EU Reg. 2023/138 note to markdown output - portals: add
hvd.category_fieldconfig to dati.gov.it inportals.json - utils: export
getPortalHvdConfig()fromportal-config.ts
- worker: enable Cloudflare Workers Logs (
[observability]inwrangler.toml) - worker: log structured JSON for every tool call (tool, server, q, fq, id, sql, etc.)
- portals: add
data.stadt-zuerich.ch(City of Zurich) with customorganization_view_urlpointing to CKAN backend - docs: add
data.stadt-zuerich.chto verified portals table insrc/README.md - docs: add
data.opentransportdata.swissto known issues (API on separate domain, requires API key) - tools: add "no data found" note to all no-results responses to discourage LLM hallucination
- docs: add LLM hallucination note to README Troubleshooting section
- docs: rewrite README intro — plain-language description, two-path table (local vs hosted), audience statement
- docs: add MIT license badge to README badge row
- docs: add "Use it in your favorite tool" section (ChatGPT, Claude Code, Claude Desktop, Gemini CLI, VS Code)
- docs: backup of previous README saved as README.bak.md
- feat: new tool
ckan_analyze_datasets— search + DataStore schema introspection in one call; includesinfo.label/info.notesfrom DataStore Dictionary when available - feat: new tool
ckan_catalog_stats— portal overview (total datasets, categories, formats, organizations) via single faceted query - refactor:
quality.tstools migrated from deprecatedserver.tool()toregisterTool()with full annotations - tests: 287 passing (+15 new tests)
- fix: HTTP transport —
/.well-known/oauth-authorization-servernow returns JSON 404 instead of HTML; fixes Claude Code HTTP transport connection failure - fix:
ckan_datastore_search—limitmin changed 1→0; allows column discovery without fetching data - docs:
ckan_datastore_searchdescription updated — fields always returned,limit=0pattern documented
- refactor: domain types for all tool files —
CkanTag,CkanResource,CkanPackage,CkanOrganization,CkanField,CkanDatastoreResultinsrc/types.ts;anyreduced 32 → 1 - refactor: extract rendering functions from handler closures → named exports in
datastore.ts,organization.ts,group.ts,status.ts; +26 unit tests - fix: datastore table — skip
_idcolumn, increase cell truncation 50 → 80 chars - fix: org/group show — dataset heading now shows
showing M of N returned — T total - tests: 191 → 272; all passing
- fix: datastore table — skip
_idcolumn, increase cell truncation 50→80 chars - fix: org/group show —
## Datasets (N)→## Datasets (showing M of N returned — T total) - tests: 270 → 272; all passing
- refactor: extract markdown rendering from handler closures into exported functions in
datastore.ts,organization.ts,group.ts,status.ts - add 26 unit tests across 4 new test files (
datastore-formatting,organization-formatting,group-formatting,status-formatting) - test count: 244 → 270; all passing
- refactor: add CKAN domain types (
CkanTag,CkanResource,CkanPackage,CkanOrganization,CkanField,CkanDatastoreResult) tosrc/types.ts - replace
anyin exported tool functions acrosspackage.ts,datastore.ts,organization.ts,group.ts,quality.ts,tag.ts— 32 → 1 remaining (internal handler variable) - no behavioral change; 244 tests passing
ckan_list_resources: addformat_filterparam (case-insensitive, client-side) — e.g. 72 resources → 8 CSV; header shows "Total: 72 (showing 8 CSV)"ckan_package_search: OR tip on zero results — when a plain multi-term query returns 0, suggest the OR version (e.g."a b c"→"a OR b OR c")ckan_package_search: accent fallback — if query returns 0 results and contains accented chars, retry with accent-stripped query; note shown in outputckan_package_show: always show DataStore status per resource✅ Available/❌ Not available/❓ Not reported by portal- Previously silent when field absent (e.g. dati.gov.it); now explicit
- Disable DataStore Table UI component (MCP Apps) pending use-case design
- Reverted
registerAppTool→server.registerToolin datastore.ts and package.ts - Removed
_meta.ui,structuredContentfrom tool responses - Commented out UI resource registration in resources/index.ts
- Skipped UI test suite (6 tests)
- All source files preserved (
src/ui/,src/resources/datastore-table-ui.ts) - Tests: 227 passed, 6 skipped
- Reverted
- Add workflow guidance ("Typical workflow: ...") to all 15 tool docstrings
- Inspired by datagouv-mcp pattern; steers LLMs toward correct multi-step usage
- Add
ckan_list_resourcestool (16th tool)- Compact table of resources: name, format, size, DataStore flag, resource ID
- Helps LLMs assess available files before deciding how to access data
- Tested against dati.gov.it and dati.comune.messina.it (DataStore active)
- Update
docs/future-ideas.mdwith datagouv-mcp analysis - Tests: 233 (was 228)
- DataStore Table UI: add hyperlinks on dataset titles, open in new tab
- DataStore Table UI: increase spacing (body padding, header/controls gap, th/td padding)
- Fix Workers: use
Accept-Encoding: identityin fetch branch to avoid gzip decompression failures- Cloudflare Workers
DecompressionStreamwas silently failing on gzip responses Accept-Encoding: identityprevents server from sending compressed responses- Eliminates the binary garbage output on all CKAN API calls from Workers
- Cloudflare Workers
- Fix MCP Apps: add timeout fallback to SEP-1865 handshake
- If host doesn't respond to
ui/initializewithin 1.5s, sendinitializedanyway - Handles hosts with partial spec support without blocking the widget forever
- Fixed in both
src/ui/datastore-table.htmland inlined HTML insrc/resources/datastore-table-ui.ts
- If host doesn't respond to
- Fix MCP Apps: implement mandatory SEP-1865 handshake in DataStore Table Viewer HTML
- Widget now sends
ui/initializerequest to host on load - After host response, sends
ui/notifications/initialized - Without this handshake, compliant hosts (MCPJam, Goose, etc.) never send
tool-result - Fixed in both
src/ui/datastore-table.htmland inlined HTML insrc/resources/datastore-table-ui.ts - Confirmed working in MCPJam (interactive table renders correctly)
- Widget now sends
-
Fix MCP Apps message listener to use correct JSON-RPC method per ext-apps spec:
ui/notifications/tool-result(wasui/toolResult) with data inmsg.params.structuredContent- Fixed in both
src/ui/datastore-table.htmland inlined HTML insrc/resources/datastore-table-ui.ts - Kept fallbacks for older/alternative message shapes
-
Audit and improve Zod parameter descriptions across all tools for "code mode" SDK compatibility:
ckan_organization_list/show/search: added.describe()on all parameters (previously none)ckan_datastore_search: added.describe()on all parameters (previously none)ckan_datastore_search_sql: added.describe()onserver_urlandsql(with double-quote hint)ckan_package_search: improvedfqdescription (was missing)ckan_find_relevant_datasets: improved tool description with "when to use vs ckan_package_search", improvedqueryandweightsdescriptions
- Fix MCP Apps implementation (was broken in v0.4.40):
_meta.ui.resourceUrimoved to tool DEFINITION (not result) usingregisterAppToolfrom@modelcontextprotocol/ext-apps/server- URI scheme changed from
ckan-ui://toui://ckan/(required by spec) - Resource now uses
RESOURCE_MIME_TYPEfrom ext-apps (required by clients) - Resource registered via
registerAppResourceinstead ofserver.registerResource - Data passed to UI via
structuredContentin tool result; HTML listener updated to handleui/toolResultJSON-RPC format - Table viewer extended to
ckan_package_search: datasets flattened to title/organization/formats/num_resources/modified/license columns - Added
@modelcontextprotocol/ext-appsas runtime dependency
⚠️ MCP Apps not yet supported by Claude.ai/Claude Desktop clients:_meta.ui.resourceUriis silently ignored — the interactive table viewer never appears. The server-side implementation is correct and ready; waiting for client support. Feature is effectively dormant until Anthropic ships MCP Apps in public clients.- Add DataStore Table Viewer (MCP Apps interactive UI)
- New MCP resource
ckan-ui://datastore-tableserves self-contained HTML table viewer ckan_datastore_searchnow returns_meta.ui.resourceUri+_meta.ui.datafor MCP Apps clients- Interactive features: sortable columns (numeric/date/string type-aware), text filter, pagination (10/25/50/100 rows/page)
- Works in Node.js and Cloudflare Workers (HTML inlined in TypeScript module, no fs dependency)
- Non-breaking: text/markdown output unchanged; non-MCP-Apps clients ignore
_meta - Files:
src/ui/datastore-table.html,src/resources/datastore-table-ui.ts, updatedsrc/tools/datastore.tsandsrc/resources/index.ts - Tests: 228 passing (7 new)
- Ideas:
docs/future-ideas.mdupdated with MCP Apps section - OpenSpec:
add-datastore-table-viewercreated and implemented
- New MCP resource
- Created issue #11 for one-click MCP server installation support
- Proposes
claude://install-mcp-serverprotocol integration - Based on Anthropic Desktop Extensions announcement
- Updated proposal: Two one-click installers to serve different use cases
- 🚀 "Try it now" → HTTP Worker (instant, zero install, shared quota)
- 💪 "Install locally" → npx (unlimited, requires Node.js)
- User journey: try demo first, install locally when ready for production
- Files: GitHub issue #11 + comment
Objective: Strongly encourage users to install locally instead of using shared Cloudflare Workers demo.
Changes:
- Added prominent banner in README recommending local npm installation
- Simplified Installation section - npm as primary method
- Repositioned Workers endpoint as "testing only" option
- Added visible footer to all tool responses when running on Workers
- Added HTTP headers to Workers responses (X-Service-Notice, X-Recommendation)
- Updated all MCP client configuration examples to prioritize local install
- Added note in DEPLOYMENT.md clarifying it's internal team documentation
Footer shown to Workers users:
ℹ️ Demo instance (100k requests/day shared quota). For unlimited access: https://github.com/ondata/ckan-mcp-server#installation
Files modified:
- README.md (banner, installation section, client config examples)
- src/utils/formatting.ts (isWorkers(), addDemoFooter() functions)
- src/tools/*.ts (7 tool files - applied footer to markdown responses)
- src/worker.ts (HTTP headers for debugging)
- docs/DEPLOYMENT.md (internal documentation note)
No breaking changes - All existing functionality preserved
- Fix: Replace emoji flags with SVG images from Twemoji CDN
- Problem: Country flag emojis (🇮🇹, 🇺🇸, 🇨🇦, 🇦🇺, 🇬🇧, 🇨🇭) not visible on Linux/desktop browsers lacking color emoji fonts
- Solution: Use
<img>tags pointing tohttps://cdn.jsdelivr.net/gh/twitter/twemoji@latest/assets/svg/ - Benefits: Works on all platforms, consistent appearance, better accessibility
- Files:
website/src/pages/index.astro(6 flag replacements),website/src/styles/global.css(removed.emoji-flagclass) - Impact: Flags now visible on all devices (Linux, Windows, macOS, mobile)
- Enhancement: add portal entries and API base URLs for catalog.data.gov, open.canada.ca, data.gov.au, and opendata.swiss
- Fix: align server and worker reported version with package version
- Files:
src/portals.json,package.json,package-lock.json,src/server.ts,src/worker.ts - No breaking changes
- Feature: add support for custom API paths in portal configuration
- Feature: add data.gov.uk portal support (uses
/api/action/instead of/api/3/action/) - Enhancement: extend portal configuration with
api_pathfield - Enhancement: dynamic API path construction based on portal config
- Files:
src/utils/portal-config.ts,src/utils/http.ts,src/portals.json - No breaking changes
- Fix: align server and worker reported version with package version
- Fix: update worker health tool count
- Files:
src/server.ts,src/worker.ts
- Docs: clarify GitHub release notes formatting (use here-doc + --notes-file)
- Files:
AGENTS.md,CLAUDE.md
- Tests: adjust MQA metrics details fixture to include scoring entries
- Files:
tests/integration/quality.test.ts
- MQA: add detailed quality reasons tool with metrics flag parsing
- MQA: add guidance note to use metrics endpoint for score deductions
- Tests: cover detailed MQA reasons output and guidance note
- Docs: list
ckan_get_mqa_quality_detailstool - Files:
src/tools/quality.ts,tests/integration/quality.test.ts,README.md,docs/architecture-flow.md
- None
- Docs: clarify natural language date-field mapping for package search and document
content_recentusage with example - Files:
src/tools/package.ts,src/server.ts,src/worker.ts,package.json,package-lock.json
- Workers: align browser-like headers for fetch path to avoid 403 on dati.gov.it
- Files:
src/utils/http.ts,package.json,package-lock.json
- Workers: decode compressed responses via DecompressionStream when available
- Docs: avoid demo.ckan.org in tests (use https://www.dati.gov.it/opendata)
- Files:
src/utils/http.ts,AGENTS.md,CLAUDE.md,package.json,package-lock.json
- Fix Workers build by avoiding static node:zlib import while keeping decompression in Node
- Files:
src/utils/http.ts,package.json,package-lock.json
- Decode compressed/binary CKAN responses (gzip/br/deflate) to fix DataStore calls on Messina portal
- Tests: cover gzip, brotli, deflate payloads for HTTP client
- Files:
src/utils/http.ts,tests/unit/http.test.ts,package.json,package-lock.json
- ckan_package_show: clarify dates (Issued/Modified vs harvest) and add metadata_harvested_at
- Resources: surface Access Service endpoints and effective download URL fallback
- Docs: add SPARQL examples + CKAN vs SPARQL comparison
- Tests: add package_show formatting/unit coverage
- Files:
src/tools/package.ts,tests/fixtures/responses/package-show-success.json,tests/unit/package-show-formatting.test.ts,docs/sparql-examples.md,src/server.ts,src/worker.ts,package.json,package-lock.json
- Resolve portal hostname to API URL in CKAN requests
- Tests: add unit coverage for URL resolution
- Files:
src/utils/http.ts,tests/unit/http.test.ts,src/server.ts,src/worker.ts,package.json,package-lock.json
- Add ANAC open data portal entry and aliases
- Files:
src/portals.json,src/server.ts,src/worker.ts,package.json,package-lock.json
- Use browser-like headers (UA + Sec-* + Referer) to avoid WAF blocks on some portals
- Deps: npm audit fix (hono transitive)
- Files:
src/utils/http.ts,tests/unit/http.test.ts,src/server.ts,src/worker.ts,package.json,package-lock.json,README.md
- Source: https://www.philschmid.de/mcp-best-practices
- Score: 4/6 (B+ grade)
- Doc:
docs/mcp-best-practices-evaluation.md - Key findings:
- ✅ Excellent: Tool naming, flat arguments
- ✅ Good: Instructions as context
⚠️ Partial: Outcomes focus, curation, pagination metadata
- Top recommendations:
- Add structured pagination metadata to all responses
- Create outcome-focused tools (discover_datasets, find_organization)
- Improve tool categorization (beginner/advanced/expert)
- Extracted with:
agent-browser get text 'article'(readability-like)
- Fix MQA quality score maximum from 450 to 405 (correct max: 100+100+110+75+20)
- Files modified:
src/tools/quality.ts:442,tests/integration/quality.test.ts:279,302
- Fix MQA identifier normalization to handle dot separators (e.g.
c_g273:D.1727->c_g273-d-1727) - Workers deploy: https://ckan-mcp-server.andy-pr.workers.dev (2026-01-26)
- Fix metrics parsing in Workers by switching to fetch and mocking fetch in tests
- Fix Worker metrics parsing fallback to ensure dimension scores are populated
- Fix metrics JSON-LD parsing in Workers to restore dimension score breakdown
- Ensure ld+json responses are parsed even when returned as strings
- MQA quality output now includes dimension score breakdown with ✅/
⚠️ indicators - Metrics endpoint link added for direct score inspection
- Non-max dimension(s) highlighted for quick diagnosis
- Files:
src/tools/quality.ts,tests/integration/quality.test.ts,tests/fixtures/responses/mqa-metrics-success.json
- Feature: Add dimension score breakdown with ✅/
⚠️ indicators and non-max dimensions - Source: Fetch metrics JSON-LD from data.europa.eu for scoring details
- Output: Markdown and JSON include metrics endpoint link and derived scores
- Files:
src/tools/quality.ts,tests/integration/quality.test.ts,tests/fixtures/responses/mqa-metrics-success.json
- Fix: Added custom color definitions to
tailwind.config.mjs - Colors: navy (#0A1628), data-blue (#0066CC), teal (#0D9488), coral (#F97316), amber (#F59E0B), cream (#FFFEF9)
- Impact: CTA section and all custom-colored elements now render correctly with proper contrast
- Before: Custom Tailwind classes (bg-navy, text-data-blue, etc.) were ignored, causing transparent backgrounds and unreadable text
- After: All colors apply correctly, navy CTA section has proper dark background with white/gray text
- Website: Created production-ready landing page in
website/directory - Stack: Astro v5 + React + Tailwind CSS + TypeScript strict mode
- Deployment: GitHub Actions workflow for automatic GitHub Pages deployment
- URL: https://ondata.github.io/ckan-mcp-server/
- Content:
- Hero section with value proposition for open data researchers
- Features section (6 key capabilities)
- Quick start with copy-paste configs (Claude Desktop, VS Code, Cloudflare Workers, global npm)
- Use cases (researchers, data scientists, students, journalists, etc.)
- Supported portals showcase (dati.gov.it, data.gov, data.europa.eu, etc.)
- SEO optimized (meta tags, Open Graph, sitemap)
- Responsive design (mobile-first, accessible WCAG AA)
- Assets:
- SVG favicon with network graph icon
- manifest.json for PWA support
- robots.txt and sitemap
- Script for PNG favicon generation (
generate-favicons.sh)
- Files:
website/src/pages/index.astro(main landing page)website/src/layouts/Layout.astro(SEO layout)website/src/components/Footer.astrowebsite/public/favicon.svg,manifest.json,robots.txt.github/workflows/deploy-website.yml(deployment automation)website/README.md(documentation)
- Build: 396 packages, builds successfully in ~1s
- Deployment trigger: Push to main branch with changes in
website/directory
- README: Added "Exploring the Server" section before "Manual Testing"
- Tool: MCP Inspector for interactive server exploration
- Usage:
npx @modelcontextprotocol/inspector node dist/index.js - Features: Browse tools/resources, test calls with auto-complete, real-time responses, debug errors
- Impact: Developers can quickly explore and test server without manual JSON-RPC
- Published to npm:
@aborruso/ckan-mcp-server@0.4.17 - Aligned with GitHub tag
v0.4.17
- Fix: Normalize identifiers for data.europa.eu lookups (lowercase, collapse hyphens)
- Fix: Retry with disambiguation suffixes (
~~1,~~2) when base identifier 404s - Result: MQA quality now matches portal IDs for datasets like Beinasco (with
~~1) - Improved errors: clearer message when identifier is not aligned
- Files modified:
src/tools/quality.ts,tests/integration/quality.test.ts - Deployed: Cloudflare Workers v0.4.17
- Published to npm:
@aborruso/ckan-mcp-server@0.4.16 - Aligned with GitHub tag
v0.4.16
- Bug fix: Identifier transformation for data.europa.eu API compatibility
- Issue: CKAN identifiers with colon separator (e.g.,
c_f158:224c373e...) were not recognized by MQA API - Root cause: data.europa.eu uses hyphen-separated identifiers (
c_f158-224c373e...) - Solution: Replace colons with hyphens before API call:
.replace(/:/g, '-') - Impact: MQA quality metrics now work for all dati.gov.it datasets, including municipal portals
- Example: Messina air quality dataset now returns score 405/560 (Eccellente)
- File modified:
src/tools/quality.ts(line 41) - Deployed: Cloudflare Workers v0.4.16
- Feature: Added
ckan_get_mqa_qualitytool for retrieving quality metrics from data.europa.eu MQA API - Scope: Only works with dati.gov.it datasets (server validation enforced)
- Data source: Queries https://data.europa.eu/api/mqa/cache/datasets/{identifier}
- Identifier logic: Uses
identifierfield from CKAN metadata, falls back tonameif identifier is empty - Metrics returned:
- Overall score (max 405 points)
- Accessibility (URL status, download availability)
- Reusability (license, contact point, publisher)
- Interoperability (format, media type)
- Findability (keywords, category, spatial/temporal coverage)
- Output formats: Markdown (default, human-readable) or JSON (structured data)
- Error handling: Dataset not found, MQA API unavailable, invalid server URL
- Tests: +11 integration tests (212 total, all passing)
- Server validation (www/non-www dati.gov.it URLs)
- Quality retrieval with identifier
- Fallback to name field
- Error scenarios (404, network errors)
- Markdown formatting (complete/partial data, availability indicators)
- Documentation: README.md (new Quality Metrics section), EXAMPLES.md (usage example with expected metrics)
- Files:
src/tools/quality.ts(new, 194 lines)src/server.ts(register quality tools)tests/integration/quality.test.ts(new, 11 tests)tests/fixtures/responses/mqa-quality-success.json(new)tests/fixtures/responses/package-show-{with,without}-identifier.json(new)
- OpenSpec: Proposal in
openspec/changes/add-mqa-quality-tool/(4 requirements, 11 scenarios)
- Feature: Auto-convert NOW-based date math for
modifiedandissuedfields - Problem: CKAN Solr supports
NOW-XDAYSsyntax only onmetadata_modifiedandmetadata_createdfields - Solution: New
convertDateMathForUnsupportedFields()automatically converts queries likemodified:[NOW-30DAYS TO NOW]to ISO datesmodified:[2025-12-23T... TO 2026-01-22T...] - Supported fields:
modified,issued(auto-converted) |metadata_modified,metadata_created(native NOW support) - Supported units: DAYS, MONTHS, YEARS (singular and plural forms)
- Tests: +10 unit tests (201 total, all passing)
- Documentation: Updated tool description with NOW syntax limitations and examples
- Files:
src/utils/search.ts,src/tools/package.ts,tests/unit/search.test.ts - No breaking changes: Backward compatible - existing queries work unchanged
- Fix: Escape Solr special characters when forcing
text:(...)queries - Tests: Added unit coverage for escaping and forced parser output
- Files:
src/utils/search.ts,tests/unit/search.test.ts,README.md
- Test counts updated: README.md (184→190), PRD.md (130→190)
- Version updated: PRD.md version 0.4.7→0.4.12
- Date updated: PRD.md last updated 2026-01-10→2026-01-17
- Verification: All 13 tools, 7 resources, 5 prompts implemented and documented
- Files:
README.md,PRD.md
- Issue templates: Added bug report and feature request YAML forms
- Bug report: CKAN portal URL, steps, expected/actual, error, Node version
- Feature request: problem/use case, proposed solution, alternatives
- Auto-labels:
bugandenhancement
- Issue chooser: Routes questions to Discussions Q&A
- PR template: Description, related issue, test/docs checklist
- Files:
.github/ISSUE_TEMPLATE/{bug_report,feature_request,config}.yml,.github/PULL_REQUEST_TEMPLATE.md
- Documentation: Created complete demo video documentation suite in
docs/video/demo-script.md- Commands and technical notes for 4 use casesdemo-expected-results.md- Actual test results with data samplesdemo-recording-guide.md- Step-by-step recording guide with voiceover scriptspre-recording-checklist.md- Practical day-of checklistdemo-fallback-options.md- Comprehensive fallback strategies for 8 scenariosdemo-timing-report.md- Performance analysis and timing verification
- Testing: Verified all demo commands working with dati.gov.it
- Portal overview: 67,614 datasets, top 10 organizations (~5s)
- Targeted search: 263 Milano transport datasets (~10s)
- Dataset details: Complete metadata with CSV/JSON resources (~10s)
- DuckDB analysis: DESCRIBE, SUMMARIZE, SAMPLE all working (~8s total)
- Target: 5-7 minute video demonstrating MCP convenience for Italian open data
- Status: Ready for recording with high confidence level
- Feature: Added
ckan://{server}/.../datasetsresource templates for group, organization, tag, and format filters - Fix: Map
ckan://hostnames to portal API base URLs (e.g., dati.gov.it → /opendata) - Fix: Format filtering now matches
res_formatanddistribution_format(with case variants) - Docs: Updated README and future ideas with new URI templates
- Docs: Updated
docs/proposta-spunti-datos-gob-es-mcp.mdmarking resource templates as completed - Tests: Added unit tests for dataset filter resource templates
- Files: New
src/resources/dataset-filters.ts, updates insrc/resources/index.ts,src/worker.ts
- Fix: Prompt arguments now coerce numeric strings (e.g., rows) for MCP prompt requests
- Docs: Updated evaluation notes for 0.4.11
- No breaking changes: Prompt names and outputs unchanged
- Feature: Added 5 guided MCP prompts (theme, organization, format, recent datasets, dataset analysis)
- Docs: README and new
docs/prompts.mdupdated with usage examples - Tests: Added prompt unit tests; total now 184 tests (all passing)
- Files: New
src/prompts/*, updates insrc/server.ts,src/worker.ts, README.md
- Security: Updated @modelcontextprotocol/sdk from 1.25.1 to 1.25.2 (fixes HIGH severity ReDoS vulnerability)
- Testing: Added 49 new unit tests for package.ts scoring functions
- Coverage: Improved from 37.33% to 38.63% (package.ts: 12.5% to 15%)
- Total tests: 179 tests (all passing, +49 from 130)
- Documentation: Corrected test coverage claims (was "113 tests, 97%+" now accurate "179 tests, ~39%")
- Deployment: Added npm audit check to DEPLOYMENT.md
- Files modified: package.json, src/server.ts, src/worker.ts, README.md, CLAUDE.md, docs/DEPLOYMENT.md
- New file: tests/unit/package-scoring.test.ts
- No breaking changes: All existing functionality preserved
- Added: 49 new unit tests for package.ts scoring functions
- Coverage improvement: package.ts from 12.5% to 15%
- Overall coverage: 37.33% to 38.63%
- Total tests: 130 to 179 tests (all passing)
- New test file: tests/unit/package-scoring.test.ts
- Functions tested:
- extractQueryTerms (10 tests)
- escapeRegExp (6 tests)
- textMatchesTerms (10 tests)
- scoreTextField (6 tests)
- scoreDatasetRelevance (17 tests with edge cases)
- Exports: Made internal functions testable (extractQueryTerms, escapeRegExp, textMatchesTerms, scoreTextField)
- Impact: Better coverage of dataset relevance scoring logic
- Fix: Corrected test coverage claims in README.md and CLAUDE.md
- Previous claim: "113 tests, 97%+ coverage"
- Actual values: 130 tests passing, ~37% overall coverage
- Utility modules: 98% coverage (excellent)
- Tool handlers: 12-20% coverage (needs improvement)
- Impact: Documentation now accurately reflects project state
- Files modified: README.md, CLAUDE.md
- Added: npm audit check to DEPLOYMENT.md (Step 4.5)
- Added: Security audit to pre-release checklist
- Recommendation: Always run
npm auditbefore production deployment
- Fix: Update @modelcontextprotocol/sdk from 1.25.1 to 1.25.2
- Reason: Resolves HIGH severity ReDoS vulnerability (GHSA-8r9q-7v3j-jr4g)
- Tests: All 130 tests passing
- Audit: 0 vulnerabilities
- Fix: On CKAN 500, fall back to
package_searchfacets for org counts - Output: Facet lists show top 10; suggest
response_format: jsonandfacet_limit
- MCP tool awareness: Gemini now selects appropriate tool from 15 available
- Loads tool list on startup via
tools/list - Passes available tools to Gemini with descriptions
- Gemini chooses tool and generates arguments based on query type
- Examples:
ckan_organization_listfor "organizations with most datasets" ckan_find_relevant_datasetsfor smart searchesckan_tag_listfor tag statistics
- Loads tool list on startup via
- Multi-type results: UI handles datasets, organizations, tags
- Organization cards show package count
- Dataset cards show resources and org name
- Status shows tool being used ("Using ckan_organization_list...")
- Fallback: Defaults to
ckan_package_searchif Gemini fails - Fix: Query "quali organizzazioni con il maggior numero di dataset" now works correctly
- UI redesign: Dark theme with data editorial aesthetic
- Typography: DM Serif Display + IBM Plex Sans
- Color scheme: Deep charcoal (#0f1419) with cyan accent (#06b6d4)
- Glass morphism effects, gradient text, subtle grid background
- Smooth animations: slide-in, hover transitions, status pulse
- Collapsible settings panel with icon-based controls
- Enhanced dataset cards with hover lift and glow
- Custom scrollbar, loading shimmer, SVG icons throughout
- Conversation context: Added history management
- Gemini receives conversation history for contextual refinement
- Users can ask follow-up queries ("only from Tuscany", "last 5 years")
- History limited to 10 messages (5 exchanges) to avoid token overflow
- Reset button to clear conversation and start fresh
- UX improvements: Better visual hierarchy, spacing, interaction patterns
- Responsive: Mobile-friendly layout maintained
- Web GUI: Replaced landing with MCP-backed chat UI (vanilla + Tailwind)
- MCP: Added JSON-RPC search flow with dataset cards
- Fix: Added
Acceptheader for MCP 406 requirement - Fix: Normalize natural-language queries before search
- Gemini: Added API key input and NL→Solr query call
- Config: Added per-portal search parser config
- Tool: Added query parser override for package search and relevance
- Tool: Added
ckan_find_relevant_datasets - Docs: Updated README/EXAMPLES
- Tests: Added relevance scoring checks
- Workers: /health version/tools updated
- Tool: Added
ckan_datastore_search_sql - Docs: Updated README/EXAMPLES/PRD for SQL support
- Tests: Added SQL fixture and checks
- Tags: Added
ckan_tag_listwith faceting and filtering - Groups: Added
ckan_group_list,ckan_group_show,ckan_group_search - Docs: Updated README with examples and tool list
- Tests: Added tag/group fixtures and tests
- npm package: Added
.npmignoreto exclude dev artifacts
- Date formatting: ISO
YYYY-MM-DDoutput, tests aligned - HTTP transport: Single shared transport per process
- Registration: Centralized tool/resource setup via
registerAll() - Docs: Updated CLAUDE/PRD/REFACTORING notes
- New section in EXAMPLES.md: "Understanding Solr Field Types: Exact vs Fuzzy Search"
- Documents difference between
type=string(exact match) andtype=text(fuzzy) - String fields: res_format, tags, organization, license, state, name (case-sensitive)
- Text fields: title, notes, author, maintainer (normalized, fuzzy enabled)
- Practical example:
res_format:CSV(43,836 results) vsres_format:csv(0 results) - Links to CKAN Solr schema on GitHub
- Explains why some searches are exact and others are fuzzy
- Documents difference between
- Impact: Users understand when exact matching is required vs when fuzzy search works
-
Production deployment: Server now live on Cloudflare Workers
- Public endpoint:
https://ckan-mcp-server.andy-pr.workers.dev - Global edge deployment (low latency worldwide)
- Free tier: 100,000 requests/day
- Bundle size: 398KB (minified: 130KB gzipped)
- Cold start time: 58ms
- Public endpoint:
-
New files:
src/worker.ts(95 lines): Workers entry point using Web Standards transportwrangler.toml(5 lines): Cloudflare Workers configuration
-
New npm scripts:
build:worker: Compile for Workers (browser platform, ESM format)dev:worker: Local testing with wrangler devdeploy: Build and deploy to Cloudflare
-
Architecture:
- Uses
WebStandardStreamableHTTPServerTransportfrom MCP SDK - Compatible with Workers runtime (no Node.js APIs)
- Stateless mode (no session management)
- JSON responses enabled for simplicity
- CORS enabled for browser access
- Uses
-
Testing: All 7 MCP tools verified in Workers environment
- Health check: ✅ Working
- tools/list: ✅ Returns all 7 tools
- ckan_status_show: ✅ External CKAN API calls working
- Response times: < 2s for typical queries
-
Documentation:
- Updated README.md with "Deployment Options" section
- Added Option 4 to Claude Desktop config (Workers HTTP transport)
- Created OpenSpec proposal in
openspec/changes/add-cloudflare-workers/
-
No breaking changes: stdio and self-hosted HTTP modes still fully supported
- Dual build system: Node.js bundle unchanged
- Existing tests (113) all passing
- Version bumped to 0.4.0
- npm Publication: Published to npm registry as
@aborruso/ckan-mcp-server- Package size: 68.6 KB (236 KB unpacked)
- Public access configured
- Installation time: 5 min → 30 sec (90% faster)
- User actions: 6 steps → 2 steps (67% reduction)
- Global Command Support: Added
binfield to package.json- Direct command:
ckan-mcp-server(no node path required) - Works system-wide after global install
- Direct command:
- Documentation Enhancement: Three installation options in README
- Option 1: Global installation (recommended)
- Option 2: Local project installation
- Option 3: From source (development)
- Platform-specific paths (macOS, Windows, Linux)
- GitHub Release: Tagged v0.3.2 with release notes
- Impact: Low barrier to entry, standard npm workflow, better discoverability
- Overall Rating: 9.0/10 (local evidence; external distribution not verified)
- Distribution readiness: 9/10 (metadata and CLI entry point verified)
- Testing: 113 tests passing; coverage 97%+ (2026-01-09)
- Status: Packaging and docs production-ready; npm/GitHub release require external verification
- See
docs/evaluation-v0.3.2.mdfor full assessment
- Added unit tests for HTTP error branches and URL generator org paths
- Tests:
tests/unit/http.test.ts,tests/unit/url-generator.test.ts
- Tests:
npm test: 113 tests passingnpm run test:coverage: 97.01% statements, 89.36% branches, 100% functions, 96.87% lines
- New Section: "Advanced Query Examples" in README.md
- 4 real-world examples tested on dati.gov.it portal
- English explanations with Italian query terms maintained
- Each example includes: use case, query, techniques, results
- Example 1: Fuzzy search + date math + boosting (871 healthcare datasets)
- Example 2: Proximity search + complex boolean (306 air quality datasets)
- Example 3: Wildcard + range + field existence (5,318 regional datasets)
- Example 4: Date ranges + exclusive bounds (demonstrates precise constraints)
- Solr Syntax Reference: Quick reference table for all query operators
- Impact: Users have practical, tested examples for advanced searches
- Tool Description: Enhanced
ckan_package_searchtool description with comprehensive Solr query syntax- Added boolean operators (AND, OR, NOT, +, -, grouping)
- Added wildcards, fuzzy search, proximity search
- Added range queries (inclusive/exclusive bounds)
- Added date math (NOW-1YEAR, NOW/DAY, etc.)
- Added field existence checks
- Added boosting/relevance scoring (^, ^=)
- 15+ inline examples in tool description
- EXAMPLES.md: New "Advanced Solr Query Features" section (~280 lines)
- Fuzzy search examples (edit distance matching)
- Proximity search (words within N positions)
- Boosting examples (relevance scoring)
- Field existence checks
- Date math with relative dates
- Complex nested queries
- Range queries with different bounds
- Wildcard patterns
- Practical advanced examples
- Impact: LLMs calling MCP server now have comprehensive query syntax reference
- Portal-Specific URLs: Introduced configuration system for non-standard CKAN portals
- New
src/portals.json: Configurable mapping for portals likedati.gov.it - New
src/utils/url-generator.ts: Utility for generating context-aware view URLs - Fixed issue where
dati.gov.itlinks pointed to standard CKAN paths instead of custom/view-dataset/paths - Automated replacement of
{id},{name}and{server_url}placeholders in URL templates - Updated
ckan_package_search,ckan_package_show,ckan_organization_listandckan_organization_showtools to use the new system
- New
- MCP Resource Templates: Direct data access via
ckan://URI schemeckan://{server}/dataset/{id}- Dataset metadatackan://{server}/resource/{id}- Resource metadatackan://{server}/organization/{name}- Organization metadata- New
src/resources/module (5 files, ~240 lines)
- Tests: 101 tests total (was 79), 100% passing
- Cleanup: Removed
src/index-old.ts, standardized to English
- Overall Rating: 9/10 - Production-ready with excellent architecture
- Improvements from v0.2.0:
- Removed legacy code (index-old.ts)
- Standardized documentation to English
- Added MCP Resource Templates
- Expanded test suite (79 → 101 tests)
- Remaining enhancements: Caching layer, configurable limits, authentication
- See
docs/evaluation-v0.3.0.mdfor full assessment
- Test Suite: Added comprehensive automated testing infrastructure
- 79 tests total: 100% passing
- Unit tests (25): formatting utilities, HTTP client
- Integration tests (54): all 7 CKAN API tools
- Coverage: vitest with v8 coverage support
- Test fixtures for all CKAN endpoints + error scenarios
- Scripts:
npm test,npm run test:watch,npm run test:coverage
- Documentation: Translated to English
- README.md: comprehensive project overview
- EXAMPLES.md: detailed usage patterns
- CLAUDE.md: AI assistant instructions
- OpenSpec: Added change proposals
- Test suite implementation proposal
- Documentation translation spec
- Major refactoring: Restructured codebase from monolithic file to modular structure
- Before: 1 file (
src/index.ts) - 1021 lines - After: 11 organized modules - 1097 total lines
- Structure:
src/ ├── index.ts (39) # Entry point ├── server.ts (12) # MCP server config ├── types.ts (16) # Types & schemas ├── utils/ # Utilities (88 lines) │ ├── http.ts # CKAN API client │ └── formatting.ts # Output formatting ├── tools/ # Tool handlers (903 lines) │ ├── package.ts # 2 tools │ ├── organization.ts # 3 tools │ ├── datastore.ts # 1 tool │ └── status.ts # 1 tool └── transport/ # Transports (39 lines) ├── stdio.ts └── http.ts - Benefits:
- Smaller files (max 350 lines vs 1021)
- Localized and safe changes
- Isolated testing possible
- Simplified maintenance
- Zero breaking changes
- Performance: Build time 16ms, bundle 33KB (unchanged)
- Testing: All 7 tools working
- Before: 1 file (
- Created
REFACTORING.md- Complete refactoring documentation - Updated
CLAUDE.md- Updated with new modular structure - Updated
PRD.md- Added npm publication requirement- Goal: Simple installation like PyPI in Python
npm install -g ckan-mcp-servernpx ckan-mcp-server
- Comprehensive testing on https://www.dati.gov.it/opendata
- Server status: CKAN 2.10.3, 66,937 datasets
- COVID search: 90 datasets found
- Organization search: Regione Toscana (10,988 datasets)
- Faceting statistics: Top orgs, formats, tags
- Dataset details: Vaccini COVID-19 2024 (Puglia)
- Response times: 3-5 seconds (network + CKAN API)
- All 7 tools working perfectly
- Code refactored and modular
- Fully tested and functional
- Documentation complete
- Ready for npm publication
- New tool:
ckan_organization_search- search organizations by name pattern- Simple input: only
pattern(automatic wildcards) - Output: only matching organizations (zero datasets downloaded)
- Efficient: server-side filtering, token savings
- Example: pattern "toscana" -> 2 orgs, 11K total datasets
- Simple input: only
- Initial release
- MCP server for CKAN open data portals
- 7 tools: package_search, package_show, organization_list, organization_show, organization_search, datastore_search, status_show
- Build system: esbuild (ultra-fast, 47ms build)
- Fixed TypeScript memory issues by switching from tsc to esbuild
- Corrected dati.gov.it URL to https://www.dati.gov.it/opendata
- Created CLAUDE.md for repository guidance
- Tested successfully with dati.gov.it (4178 datasets on "popolazione" query)