DX-2361: Add GitHub Actions to validate external URLs and internal anchor fragments (#1865)

* DX-2361: Add GitHub Actions to validate external URLs and internal anchors

- Extend validate_mintlify_docs.py with --check-anchors flag that extracts
  GFM-slugified heading anchors, {#custom-id} syntax, and <a id/name> elements
  from target MDX files and verifies every internal #fragment resolves
- Update validate-docs.yml to run anchor check on every PR
- Add check-external-links.yml workflow: weekly scheduled (Mon 07:00 UTC) and
  manual-trigger run that HEAD-checks all external HTTP/HTTPS URLs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* DX-2361: Run external link check on every pull request

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* DX-2361: Fix exit code not failing on broken external links

External link failures were reported but never factored into the
exit code, so CI was passing despite 404s. Added has_broken_external
check and a summary line for external link results.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* DX-2361: Switch external link checker to lychee

Replace sequential Python/requests approach with lycheeverse/lychee-action.
Lychee runs checks in parallel, has built-in retry logic, and is purpose-built
for link checking — much faster on a repo with 1500+ external URLs.

Add lychee.toml to exclude tyk-owned domains, localhost, placeholder URLs,
and sites known to block bots (LinkedIn, Facebook). Accept 429 as non-broken
to handle rate-limited responses gracefully.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix mirror-pr workflow failing on PR bodies with special characters

GitHub Actions expands ${{ }} expressions before the shell runs, so
backticks and $ signs in PR bodies were injected raw into the script
and interpreted as shell command substitution.

Fix: write static content via a single-quoted heredoc (no shell
expansion), then append the PR body via printf with an env var
($PR_BODY). Shell variables accessed as "$VAR" are never
re-interpreted, making any PR body content safe.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix lychee checking internal/relative links

Add scheme = ["https", "http"] to lychee.toml so lychee only attempts
to check http/https URLs. Without this, lychee tries to resolve
root-relative paths (/img/..., /page/...) as file URIs and fails.
Internal links are already validated by validate-docs.yml.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix lychee erroring on root-relative paths

scheme = ["https", "http"] didn't help because lychee fails to build
the URL before the scheme filter runs. Set base = "https://tyk.io" so
root-relative paths (/img/..., /page/...) are resolved to
https://tyk.io/... and then silently skipped by the existing tyk.io
exclude rule, rather than producing "cannot convert path to URI" errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix lychee config: base -> base_url

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: sharadregoti

.github/workflows/check-external-links.yml

@@ -0,0 +1,23 @@
+name: Check External Links
+
+on:
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  external-links:
+    name: Check External URLs
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5  # v4
+
+    - name: Check external links
+      uses: lycheeverse/lychee-action@v2
+      with:
+        args: --config lychee.toml '**/*.md' '**/*.mdx'
+        fail: true

.github/workflows/mirror-pr-to-build-deploy.yml

@@ -32,26 +32,28 @@ jobs:
             # Create new mirror PR
             echo "Creating new mirror PR..."
           
-            # Create PR body content using printf to avoid shell parsing issues
-            printf '%s\n' \
-              "**🔗 Auto-generated mirror PR for Mintlify preview**" \
-              "" \
-              "**Original PR:** #${{ github.event.number }}" \
-              "**Author:** @${{ github.event.pull_request.user.login }}" \
-              "" \
-              "## Purpose" \
-              "This PR provides a Mintlify preview link for reviewing documentation changes." \
-              "" \
-              "## Preview Link" \
-              "The Mintlify preview will be available once this PR is processed." \
-              "" \
-              "## ⚠️ Important Notes" \
-              "- **Do not merge this PR directly**" \
-              "- This PR will be auto-merged when the original PR #${{ github.event.number }} is merged" \
-              "- Make all comments and reviews on the original PR #${{ github.event.number }}" \
-              "" \
-              "## Changes" \
-              "${{ github.event.pull_request.body }}" > pr_body.txt
+            # Write PR body to file via env var to safely handle special characters
+            # (backticks, $, quotes) without shell re-interpretation
+            cat > pr_body.txt << 'ENDBODY'
+**🔗 Auto-generated mirror PR for Mintlify preview**
+
+**Original PR:** #${{ github.event.number }}
+**Author:** @${{ github.event.pull_request.user.login }}
+
+## Purpose
+This PR provides a Mintlify preview link for reviewing documentation changes.
+
+## Preview Link
+The Mintlify preview will be available once this PR is processed.
+
+## ⚠️ Important Notes
+- **Do not merge this PR directly**
+- This PR will be auto-merged when the original PR #${{ github.event.number }} is merged
+- Make all comments and reviews on the original PR #${{ github.event.number }}
+
+## Changes
+ENDBODY
+            printf '%s\n' "$PR_BODY" >> pr_body.txt
 
             # Escape the title properly to handle special characters and spaces
             ESCAPED_TITLE=$(printf '%s' "🔄 Preview: ${{ github.event.pull_request.title }}" | sed 's/"/\\"/g')
@@ -69,6 +71,7 @@ jobs:
           fi
         env:
           GH_TOKEN: ${{ secrets.ORG_GH_TOKEN }}
+          PR_BODY: ${{ github.event.pull_request.body }}
 
   cleanup-mirror-pr:
     runs-on: ubuntu-latest

.github/workflows/validate-docs.yml

@@ -28,9 +28,14 @@ jobs:
       run: |
         echo "🔍 Running documentation validation..."
         python scripts/validate_mintlify_docs.py . --validate-redirects --verbose
-        
+
+    - name: Check anchor fragments
+      run: |
+        echo "⚓ Checking internal anchor fragments..."
+        python scripts/validate_mintlify_docs.py . --check-anchors --links-only
+
     - name: Validation complete
       if: success()
       run: |
         echo "✅ Documentation validation passed!"
-        echo "All links, images, navigation, and redirects are valid."
+        echo "All links, images, navigation, redirects, and anchors are valid."

lychee.toml

@@ -0,0 +1,42 @@
+# Lychee external link checker configuration
+# Docs: https://lychee.cli.rs/usage/configuration/
+
+# Only check http/https URLs — skips mailto:, file://, etc.
+scheme = ["https", "http"]
+
+# Resolve root-relative links (e.g. /img/..., /page/...) against the live docs
+# base URL. Combined with the tyk.io exclude rule below, these get silently
+# skipped rather than erroring as "cannot convert path to URI".
+base_url = "https://tyk.io"
+
+# Only check external URLs — internal links are validated by validate-docs.yml
+exclude = [
+  # Tyk-owned domains (covers root-relative paths resolved via base above)
+  "https?://(.*\\.)?tyk\\.io",
+  "https?://(.*\\.)?tyktech\\.net",
+
+  # Localhost / private addresses
+  "https?://localhost",
+  "https?://127\\.0\\.0\\.1",
+
+  # Placeholder / example URLs
+  "https?://example\\.com",
+  "https?://your-",
+  "https?://<",
+
+  # Known to block bots / return false 403s
+  "https?://(www\\.)?linkedin\\.com",
+  "https?://(www\\.)?facebook\\.com",
+]
+
+# Treat these status codes as success (429 = rate-limited but link exists)
+accept = [200, 201, 202, 203, 204, 206, 301, 302, 303, 307, 308, 429]
+
+# Retry up to 3 times before marking a link as broken
+max_retries = 3
+
+# Timeout per request in seconds
+timeout = 15
+
+# Number of concurrent requests
+max_concurrency = 8