Skip to content

DOCS-6643 Derive help-table URLs from the nav, not the file path - #1069

Open
mariadb-stefan-hinz wants to merge 1 commit into
mainfrom
DOCS-6643-help-url-from-nav
Open

mariadb-stefan-hinz wants to merge 1 commit into
mainfrom
DOCS-6643-help-url-from-nav

Conversation

@mariadb-stefan-hinz

Copy link
Copy Markdown
Contributor

Closes DOCS-6643.

The defect

markdown_extractor.py built each help topic's URL from the page's path on disk:

url_path = str(Path(path).resolve().relative_to(REPO_ROOT)).removesuffix(".md")

GitBook publishes a page at its position in SUMMARY.md, and lowercases the slug. Where the two disagree the generator emits a URL that only resolves through a 307 redirect33 of 1,095 topics, split into two causes:

  • 12 by case — the generated Plugin API pages under server/reference/plugins/api-plugin/, the only mixed-case content files in server/reference/ (MDBF-1241).
  • 21 by nav positionCREATE TABLE, DROP TABLE, SET NAMES, the whole subqueries/ cluster, and others.

These URLs are written into mysql.help_topic and ship with the server. Nothing watches them, and a redirect that is later retired becomes a dead HELP pointer in a released branch.

The fix

build_nav_url_map() walks server/SUMMARY.md and gives each entry its parent's URL plus its own slug: the file's own path component, lowercased, using the directory name for a README.md.

A file listed more than once is published at the occurrence that has children, failing that the first. That rule is what separates two entries sitting side by side in the nav:

first listing published at
ALTER TABLE (alter-table/README.md) Server Usage → Tables, no children …/reference/sql-statements/data-definition/alter/alter-table (its 2nd listing, which has a child)
CREATE TABLE (create-table.md) Server Usage → Tables, no children server-usage/tables/create-table

A page listed in no SUMMARY.md is not published anywhere, so no URL is correct for it. Those keep the path-derived fallback and are now reported to help-tables/unlisted_pages.txt151 today, which is the known orphan backlog, not a regression.

Verification

Measured against the live site, not against the model.

  • The nav map reproduces the published URL for all 1,095 topics — I diffed it against a ground-truth set built by curling every path-derived URL without -L and taking the redirect target where there was one. 1,095 match, 0 mismatch, 0 unresolved.
  • After the change, check_help_urls.py reports 1,095 direct 200, 0 not (exit 0).
  • Proven red by mutation: disabling the nav lookup restores exactly the 33 redirects, and the check exits 1 naming each one with its target.
  • codespell (CI-exact invocation) and doc-lint.sh clean on the changed files.

Drive-by fixes

  • help-tables/check_help_urls.py — the standing check the ticket asks for. HEADs every docs URL in the generated SQL, 16 at a time, fails on anything that is not a direct 200, retries once on a network-level error so a DNS blip in a run of a thousand is not a finding. It reads both the URL: line and the url column: a description long enough to be truncated loses its URL: line, so matching only those would skip exactly the topics that are already damaged.
  • graphify-out/ added to EXCLUDED_DIRS — the artifacts are git-ignored so CI never sees them, but a local run turned GRAPH_REPORT.md into a help topic.
  • The generator's three output files are git-ignored — they were dirtying the working tree on every run.
  • HELP_TABLES_PIPELINE.md updated: the URL Format section now states the nav rule, and check_help_urls.py has a Components entry.

Note for the reviewer

This supersedes the /README hunk of MDEV-40377-help-generator.patch, attached to MDEV-40377 — nav_slug() already maps a README.md to its directory. That patch's other two fixes are untouched and still needed:

  • 272 topic names keep their markdown backslash escapes (AES\_ENCRYPT), so HELP AES_ENCRYPT cannot match them;
  • 29 topics lose their URL: line to truncation, GRANT, ALTER TABLE, CREATE TABLE, EXPLAIN and FLUSH among them.

Both are already shipped in 10.11's fill_help_tables.sql.

🤖 Generated with Claude Code

markdown_extractor.py built each help topic's URL from the page's path on
disk. GitBook publishes a page at its position in SUMMARY.md instead, and
lowercases the slug, so where the two disagreed the generator emitted a URL
that only resolved through a 307 redirect. 33 of 1,095 topics were affected:
12 by case (the generated Plugin API pages, the only mixed-case content under
server/reference/) and 21 by nav position, CREATE TABLE and DROP TABLE among
them. These URLs ship inside mysql.help_topic, nothing watches them, and a
redirect that is later retired turns into a dead HELP pointer in a released
branch.

build_nav_url_map() walks server/SUMMARY.md and gives each entry its parent's
URL plus its own slug: the file's own path component, lowercased, using the
directory name for a README.md. A file listed more than once is published at
the occurrence that has children, failing that the first — which is why
ALTER TABLE keeps its reference/ URL while CREATE TABLE, listed beside it
under Server Usage > Tables with no children of its own, does not.

Verified against the live site rather than against the model: the map
reproduces the published URL for all 1,095 topics, and check_help_urls.py
now reports 1,095 direct 200s and no redirects. Reverting the lookup
reproduces exactly the 33 redirects.

A page listed in no SUMMARY.md is not published anywhere, so no URL is
correct for it; those keep the path-derived fallback and are now reported to
help-tables/unlisted_pages.txt (151 today, the known orphan backlog).

Drive-by fixes:
- check_help_urls.py, the standing check: HEADs every docs URL in the
  generated SQL and fails on anything that is not a direct 200. It reads both
  the URL: line and the url column, because a description long enough to be
  truncated loses its URL: line and would otherwise go unchecked.
- graphify-out/ added to EXCLUDED_DIRS. The artifacts are git-ignored so CI
  never sees them, but a local run turned GRAPH_REPORT.md into a help topic.
- The generator's three output files are git-ignored; they were dirtying the
  working tree on every run.

Note for review: this supersedes the /README hunk of the
MDEV-40377-help-generator.patch attached to MDEV-40377 (nav_slug() already
maps a README.md to its directory). That patch's other two fixes — topic
names keeping their markdown backslash escapes, and the URL: line being cut
off by truncation — are untouched and still needed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant