Skip to content

Commit 7269eb3

Browse files
mcherifclaude
andcommitted
Resolve Himalayas apply URLs at fetch time to bypass Cloudflare
Himalayas jobs stored as himalayas.app/... URLs trigger Cloudflare when Playwright navigates to them. Follow the redirect chain at fetch time with a HEAD request so the actual ATS URL (Greenhouse, Lever, etc.) is stored in the DB instead. No impact for already-external URLs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 70366d4 commit 7269eb3

1 file changed

Lines changed: 19 additions & 1 deletion

File tree

connectors/himalayas.py

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,24 @@ def fetch_jobs(self) -> List[Dict[str, Any]]:
7272
logger.info(f"Successfully fetched {len(all_jobs)} remote jobs from {self.source_name}")
7373
return all_jobs
7474

75+
@staticmethod
76+
def _resolve_apply_url(app_link: str) -> str:
77+
"""Follow redirects on Himalayas apply links to get the final ATS URL.
78+
79+
Himalayas often stores links as himalayas.app/... which triggers
80+
Cloudflare when Playwright tries to open them. A HEAD request at
81+
fetch time resolves the redirect chain once and stores the real URL.
82+
"""
83+
if not app_link or "himalayas.app" not in app_link:
84+
return app_link
85+
try:
86+
r = requests.head(app_link, allow_redirects=True, timeout=8,
87+
headers={"User-Agent": "Mozilla/5.0"})
88+
final = r.url
89+
return final if final and final != app_link else app_link
90+
except Exception:
91+
return app_link
92+
7593
def normalize(self, raw_job: Dict[str, Any]) -> Dict[str, Any]:
7694
pub = raw_job.get("pubDate")
7795
posted_date = None
@@ -82,7 +100,7 @@ def normalize(self, raw_job: Dict[str, Any]) -> Dict[str, Any]:
82100
pass
83101

84102
app_link = raw_job.get("applicationLink", "")
85-
url = app_link if app_link else ""
103+
url = self._resolve_apply_url(app_link) if app_link else ""
86104

87105
description = raw_job.get("description") or raw_job.get("excerpt") or ""
88106

0 commit comments

Comments
 (0)