Exclude deprecated apps from catalog and add URL health check#226
Open
ugurkocde wants to merge 1 commit into
Open
Exclude deprecated apps from catalog and add URL health check#226ugurkocde wants to merge 1 commit into
ugurkocde wants to merge 1 commit into
Conversation
Uploads for apps like Skype or Twitch Studio were guaranteed to fail because their vendors discontinued them, and apps like jamovi broke silently when the vendor removed a versioned download file. - collect_app_info.py now honors the deprecated and disabled flags of Homebrew casks: affected app JSONs get a deprecated flag with the upstream reason, and the flag is cleared automatically if the cask becomes healthy again - supported_apps.json generation and app packaging skip deprecated apps - New twice-weekly url-health-check workflow probes every catalog download URL and maintains a GitHub issue listing broken ones - Flag the 14 apps that are currently discontinued or removed upstream and regenerate supported_apps.json (1271 to 1257 apps)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Three IntuneBrew website upload runs failed on 2026-07-02. Investigating them revealed a systematic issue in the catalog pipeline: an audit of all 1271 download URLs found 19 broken ones, in two categories.
Zombie entries (14 apps). The nightly collector syncs versions from Homebrew casks but ignores the
deprecatedanddisabledcask flags, and it silently swallows 404s for casks that were deleted from Homebrew. Discontinued apps like Skype, Facebook Messenger, Boxcryptor, or Twitch Studio therefore stayed insupported_apps.jsonforever, and any upload attempt for them was guaranteed to fail.Silent URL rot (5 apps). Nothing in the pipeline ever probes the download URLs. When a vendor deletes a versioned file (jamovi removed its 2.7.37.0 DMG and redirects to a 404 page served with HTTP 200), the catalog keeps republishing the dead link every night with a green workflow run. The jamovi upload failure surfaced as a cryptic
commitFileFailedin Intune after uploading a 32 KB HTML page.Changes
collect_app_info.pyhonors thedeprecated/disabledflags of Homebrew casks. Affected app JSONs get"deprecated": trueplus the upstream reason, and the flag is cleared automatically if the cask becomes healthy again (verified both directions against the live Homebrew API).supported_apps.jsongeneration and the packaging step inbuild-app-packages.ymlskip flagged apps. The JSON files stay inApps/for history.url-health-check.ymlworkflow (Mondays and Thursdays, plus manual dispatch) probes every non-deprecated catalog URL and creates or updates a GitHub issue listing broken ones, closing it when everything is healthy. The checker tries several client profiles per URL (HEAD, ranged GET, curl and browser user agents) because different CDNs block different clients; a URL counts as broken only when every attempt fails.supported_apps.jsonis regenerated (1271 to 1257 apps). The README app table will be rebuilt by the next nightly run.Verification
deprecatedflag gets the flag removed and the JSON round-trips byte-identical otherwise.py_compileand YAML validation pass for all touched files.