Skip to content

Commit 793e24d

Browse files
JOhnsonKC201profvjreddi
authored andcommitted
fix(link-check): count broken links from the failure sections only
The broken-link count grepped the whole lychee report for '* [<digits>]', which counts the wrong lines in both directions: every followed redirect is reported as '* [200] <url>' and got counted as broken, while a request that never produced an HTTP response is reported as '[ERROR]' or '[TIMEOUT]' and was skipped entirely. Those non-numeric ones are DNS failures, TLS failures, timeouts and missing local files, so a site whose links fail that way parsed to zero. Scope the scan to the 'Errors per input' and 'Timeouts per input' sections, accept any bracketed status, and strip the '(at line:col)' span before the autolink delimiters so the reported URLs are clean. Verified against real lychee 0.23.0 output: on a report with one [ERROR] and one followed redirect, the old expression returned 1 by counting the redirect and missing the error, while this returns the error. Closes #2080
1 parent 910a367 commit 793e24d

1 file changed

Lines changed: 64 additions & 14 deletions

File tree

.github/workflows/infra-link-check.yml

Lines changed: 64 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -163,24 +163,74 @@ jobs:
163163
fi
164164
165165
# ---- Parse the lychee markdown report for an exact count ----
166-
# The action writes a report to ./lychee/out.md by default. The
167-
# "Errors per input" section lists each broken URL on its own line
168-
# like: * [ERROR] https://example.com/foo (404 Not Found)
169-
# We count those for an honest broken_count, and grab the first
170-
# 10 URLs as a triage sample for the issue body.
166+
# The action writes a report to ./lychee/out.md by default. It is a
167+
# series of top-level sections, one per outcome:
168+
#
169+
# ## Errors per input <- a failure
170+
# ## Timeouts per input <- a failure
171+
# ## Redirects per input <- NOT a failure, the link resolved
172+
# ## Ignored per input <- NOT a failure, excluded on purpose
173+
# ## Suggestions per input <- NOT a failure
174+
#
175+
# and each entry inside them looks like
176+
# * [404] <https://example.com/foo> (at 12:3) | Not Found
177+
# where the "(at line:col)" span is present whenever lychee knows the
178+
# source position. Four properties of that shape drive the parsing.
179+
#
180+
# 1. Collect from Errors AND Timeouts, and only those. They are
181+
# siblings, not nested, so bounding the scan at the first "## "
182+
# after Errors silently drops every timeout, which is the same
183+
# under-count this step exists to fix.
184+
#
185+
# 2. Do not scan the whole file. Redirect entries have the identical
186+
# "* [<status>] <url>" shape, so a whole-file scan counts every
187+
# followed redirect as broken. Those are usually [200], and can
188+
# even be a status this workflow passed --accept for, which means
189+
# reporting links as broken that lychee was told are fine.
190+
#
191+
# 3. The bracketed status is not always numeric. A request that never
192+
# produced an HTTP response is reported as [ERROR] or [TIMEOUT]:
193+
# TLS failures, DNS failures, timeouts, and missing local files all
194+
# land there. Matching only digits drops exactly the failures that
195+
# are hardest to diagnose from a URL alone, and a site whose
196+
# failures are all of that kind parses to 0 and then gets reported
197+
# as "?" by the guard below.
198+
#
199+
# 4. The URL is not always last on the line. Strip the "(at line:col)"
200+
# span before the markdown autolink delimiters (<https://...>), or
201+
# the emitted URL keeps a trailing "> (at 12:3)" and every entry in
202+
# the triage list is corrupt.
171203
REPORT="./lychee/out.md"
172204
BROKEN_COUNT=0
173205
BROKEN_SAMPLE=""
174206
if [ -f "$REPORT" ]; then
175-
# Lines starting with "* [" are per-link error entries in
176-
# lychee's markdown output. Be tolerant of leading whitespace.
177-
BROKEN_COUNT=$(grep -cE '^[[:space:]]*\*[[:space:]]+\[[[:digit:]]+\]' "$REPORT" || true)
178-
# Pull the URL (second whitespace-separated field after the
179-
# status code bracket) for the first 10 entries.
180-
BROKEN_SAMPLE=$(grep -hE '^[[:space:]]*\*[[:space:]]+\[[[:digit:]]+\]' "$REPORT" \
181-
| head -n 10 \
182-
| sed -E 's/^[[:space:]]*\*[[:space:]]+\[[[:digit:]]+\][[:space:]]+([^[:space:]]+).*/\1/' \
183-
|| true)
207+
BROKEN_URLS=$(awk '
208+
# Collect from every section that means "this link failed".
209+
# lychee writes each as its own top-level section, so Timeouts is
210+
# a SIBLING of Errors, not nested inside it. Stopping at the first
211+
# "## " after Errors therefore dropped every timeout, which is the
212+
# same silent under-count this step exists to fix. Redirects,
213+
# Ignored and Suggestions are siblings too, and are not failures.
214+
/^##[[:space:]]+(Errors|Timeouts)[[:space:]]+per input[[:space:]]*$/ { collect = 1; next }
215+
/^##[[:space:]]/ { collect = 0 }
216+
collect && /^[[:space:]]*\*[[:space:]]+\[[^]]+\][[:space:]]+/ {
217+
url = $0
218+
sub(/^[[:space:]]*\*[[:space:]]+\[[^]]+\][[:space:]]+/, "", url)
219+
sub(/[[:space:]]*\|.*$/, "", url)
220+
# lychee appends the source position as "(at line:col)" whenever
221+
# the link carries a span, so the URL is not always last on the
222+
# line and a bare trailing ">" strip leaves "url> (at 1:1)".
223+
sub(/[[:space:]]*\(at[[:space:]][^)]*\)[[:space:]]*$/, "", url)
224+
sub(/[[:space:]]+$/, "", url)
225+
sub(/^</, "", url)
226+
sub(/>$/, "", url)
227+
if (url != "") print url
228+
}
229+
' "$REPORT" || true)
230+
if [ -n "$BROKEN_URLS" ]; then
231+
BROKEN_COUNT=$(printf '%s\n' "$BROKEN_URLS" | grep -c . || true)
232+
BROKEN_SAMPLE=$(printf '%s\n' "$BROKEN_URLS" | head -n 10)
233+
fi
184234
fi
185235
186236
# If lychee said failure but we couldn't parse a count (e.g. report

0 commit comments

Comments
 (0)