Skip to content

test: stabilize Maven-resolving tests + e2e CSV against Maven Central rate limits#747

Merged
KunalSin9h merged 3 commits into
mainfrom
fix/maven-pomxml-transient-fetch-retry
Jun 24, 2026
Merged

test: stabilize Maven-resolving tests + e2e CSV against Maven Central rate limits#747
KunalSin9h merged 3 commits into
mainfrom
fix/maven-pomxml-transient-fetch-retry

Conversation

@KunalSin9h

@KunalSin9h KunalSin9h commented Jun 24, 2026

Copy link
Copy Markdown
Member

Problem

Several CI checks fail non-deterministically due to Maven Central rate-limiting (HTTP 429), not code regressions:

  • run-testpkg/parser (Test_MavenPomXmlParser_*) and pkg/readers (TestDirectoryReaderEnumPackages) resolve Maven dependencies live from Maven Central. CI runners share egress IPs that Maven Central frequently throttles, and osv-scalibr's resolver does not retry on 429, so resolution fails. The pkg/parser RemoteParent fixture (spring-boot-starter-parent) made it worse by burst-fetching ~200 BOM POMs, but even light fixtures (a single junit dependency) get 429'd once the IP is hot.
  • run-e2e scenario-11 (scenario-11-code-csvreport.sh) diffs vet's live output against a hardcoded golden CSV. Upstream published a new langchain advisory (GHSA-gr75-jv2w-4656), so vet emitted a row the golden file lacked.

Fix (test/CI only — no production code, no fixture changes)

  1. pkg/parser — skip the Maven parser tests when resolution fails with a Maven fetch/registry error (429 / unreachable). Any other error still fails the test, preserving regression coverage.
  2. pkg/readers — gate the pom.xml-bearing enumeration cases on an error-based probe: resolve a fixture pom through the same parser the directory reader uses, and skip those cases when Maven Central is rate-limited. (A full resolution is used as the probe rather than a single HTTP ping, so the signal reflects what the cases actually do.)
  3. e2e CSV golden — add the new langchain advisory row to the expected report.

Why skip rather than retry / lighter fixture

  • A lighter fixture alone is insufficient: CI runner IPs arrive pre-throttled, so even a ~4-request resolution gets 429'd.
  • Retrying re-fires the same burst and re-trips the limit.
  • osv-scalibr fetches via http.DefaultClient internally, so there is no clean per-request retry injection without globally swapping the transport.

Skip-on-unavailable is the idiomatic pattern for tests that depend on an external service. The tests still validate fully when Maven Central is reachable.

Note

A deeper, separate issue remains: vet's Maven pom.xml resolution has no 429 resilience, which can also affect real users scanning large (e.g. Spring Boot) poms. That is a production change worth tracking independently.

🤖 Generated with Claude Code

Test_MavenPomXmlParser_RemoteParent used spring-boot-starter-parent as the
remote parent. Its dependencyManagement transitively imports hundreds of
BOM POMs, so osv-scalibr's resolver burst-fetches Maven Central and gets
rate-limited (HTTP 429); osv-scalibr does not retry on 429, so resolution
fails. The 429 also throttles the shared CI runner IP, cascading into the
pkg/readers directory enumeration tests.

Switch to a small remote parent (oss-parent) with an explicitly pinned
junit-jupiter-api version. This still exercises remote-parent resolution
but drops remote fetches from ~200 to ~6, removing the burst that trips
the rate limit. The 4-package assertion is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@safedep

safedep Bot commented Jun 24, 2026

Copy link
Copy Markdown

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

View complete scan results →

This report is generated by SafeDep Github App

@KunalSin9h KunalSin9h requested a review from abhisek June 24, 2026 07:36
@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 12.41%. Comparing base (ed5b09d) to head (c9a6321).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #747      +/-   ##
==========================================
- Coverage   12.41%   12.41%   -0.01%     
==========================================
  Files         372      372              
  Lines       49231    49231              
==========================================
- Hits         6114     6111       -3     
- Misses      42632    42634       +2     
- Partials      485      486       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

KunalSin9h and others added 2 commits June 24, 2026 13:19
The code CSV report e2e scenario diffs vet's live output against a
hardcoded expected CSV. Upstream published a new langchain 0.2.1 advisory
(GHSA-gr75-jv2w-4656, "LangChain: Path traversal and sandbox escape in
LangChain file-search middleware and loaders"), so vet now emits a row the
golden file lacked, failing the diff. Add the missing row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI runners share egress IPs that Maven Central frequently rate-limits
(HTTP 429). osv-scalibr's resolver does not retry on 429, so the pom.xml
parser and directory enumeration tests - which resolve dependencies live
from Maven Central - fail non-deterministically. Even light fixtures
(a single junit-jupiter-api dependency) get throttled, so reducing the
fixture's transitive footprint is not sufficient on its own.

Skip these tests when the registry is unreachable or rate-limited, instead
of failing:

- pkg/parser: when parseMavenPomXmlFile returns a Maven fetch/registry
  error, skip; any other error still fails so real regressions are caught.
- pkg/readers: probe by resolving a fixture pom through the same parser the
  directory reader uses (error-based, representative of what the cases do)
  and skip the pom.xml-bearing cases when resolution is rate-limited.

This also reverts the earlier remote-parent fixture swap: with the skip
guard in place the fixture change is unnecessary, and keeping the original
fixture means the PR introduces no changed manifest for vet-action to scan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@KunalSin9h KunalSin9h changed the title test: use lightweight remote parent in maven pom.xml test fixture test: stabilize Maven-resolving tests + e2e CSV against Maven Central rate limits Jun 24, 2026
@KunalSin9h KunalSin9h merged commit 0727106 into main Jun 24, 2026
15 of 16 checks passed
@KunalSin9h KunalSin9h deleted the fix/maven-pomxml-transient-fetch-retry branch June 24, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants