Fix admin refresh failures by migrating scraper to paginated all-time API

[Disseminator]

Summary

This PR fixes POST /api/admin/refresh returning 500 when scraping skills data.

Root Cause

The scraper was parsing https://skills.sh HTML with a hardcoded allTimeSkills regex.
The site payload format changed, so that key is no longer reliably present in page HTML.

Changes

• Updated scraper to use the paginated API endpoint:
  • GET /api/skills/all-time/{page} (0-based)
• Added pagination handling until completion (hasMore / total).
• Added safety guards:
  • max page cap
  • response validation
  • deduplication by source + skillId
• Kept HTML parsing as fallback for resilience, supporting both:
  • allTimeSkills
  • initialSkills
• Added unit tests for:
  • successful paginated API scraping
  • fallback to HTML parsing when API path fails
• Added src/registry/scraped-skills.json to .gitignore (generated file).

Result

Manual refresh no longer depends on fragile HTML structure and is resilient to future frontend payload changes.