forked from opencats/OpenCATS
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathrobots.txt
More file actions
24 lines (18 loc) · 722 Bytes
/
Copy pathrobots.txt
File metadata and controls
24 lines (18 loc) · 722 Bytes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# By default, block crawling of the main application backend and only expose the public careers portal.
User-agent: *
Disallow: /
# Let crawlers revisit only the root login page so they can observe its
# noindex meta tag and remove the page from search results. The rest of the
# private OpenCATS backend remains blocked.
Allow: /$
# Public careers portal (job listings, job details, applications)
# Canonical public entry point for job listings and job details
Allow: /careers/
# Static assets required by the public careers portal
# Common static asset directories (adjust to match your installation if needed)
Allow: /js/
Allow: /images/
Allow: /main.css
Allow: /careersPage.css
Allow: /ie.css
Allow: /not-ie.css