Skip to content

Safely publish caches, bound HTTP requests, and redact response metadata - #53

Open
alexandrutodor wants to merge 5 commits into
agroportal:masterfrom
alexandrutodor:fix/safe-cache-publication
Open

Safely publish caches, bound HTTP requests, and redact response metadata#53
alexandrutodor wants to merge 5 commits into
agroportal:masterfrom
alexandrutodor:fix/safe-cache-publication

Conversation

@alexandrutodor

Copy link
Copy Markdown

Summary

  • Generate complete cache candidates before publishing them, validate a non-empty JSON object with an ontology count equal to the source catalogue, force candidate bytes to disk, and use an atomic same-directory replacement without a non-atomic fallback.
  • Preserve an existing cache when evaluation, writing, validation, or atomic replacement fails. For an existing POSIX cache, copy and verify its owner, group, and permissions on the candidate before atomic replacement; if POSIX metadata cannot be read, set, or verified, refuse publication and leave the old cache untouched. For a new POSIX cache, retain the prior 0666 readable default filtered by the process umask, while leaving non-POSIX providers on their native defaults. Add CacheHealthCMD <portal> with exact 0 (healthy) / 1 (unhealthy or invalid invocation) status behavior.
  • Add configurable positive connect/read timeouts and deterministic connection cleanup for API and resolvability HTTP calls.
  • Stop returning API keys or query strings in status metadata and stop logging portal credentials. Public errors identify the portal without reflecting query-bearing metadata URLs.
  • Keep the documented sync request behavior unchanged.

Compatibility

ResultCache.flush(String) remains public for source compatibility. Cache generation remains all-or-nothing: a candidate publishes only when every source acronym has a result. Legitimate catalogue shrinkage still publishes when the complete candidate count matches the smaller source count.

Test evidence

Base: 24def6b4b01fb7690e938adfc8477d6436c3cc1f

Head: cbe511dfd7bde4c8ba579ed3a5878eaec8d18765

$ /home/ranma/.local/tools/apache-maven-3.9.11/bin/mvn -B clean test
PASS — 12 tests, 0 failures, 0 errors, 0 skipped; Maven 3.932 s (wall 4.783 s)

$ /home/ranma/.local/tools/apache-maven-3.9.11/bin/mvn -B clean package
PASS — 12 tests, 0 failures, 0 errors, 0 skipped; Maven 6.677 s (wall 7.667 s)
WAR: target/fairness-assessment.war (8,545,701 bytes)

$ git diff --check
PASS

$ git diff --check origin/master...HEAD
PASS

$ git grep -nE 'apikey=.*getApikey|getQueryString|[?&]apikey=.*getApikey' -- src/main/java
PASS — no matches

$ git diff --name-only origin/master...HEAD
PASS — only the 15 documented README, Java, and JUnit files changed

Focused assertions cover:

  • malformed, wrong-shape, empty, and 1/10 incomplete candidates preserving the old cache bytes and POSIX permissions exactly while leaving no temporary files;
  • atomic replacement preserving an existing 0640 mode, owner, and an available supplementary group for cross-account readability; the test safely assumes/skips when POSIX group changes cannot be exercised;
  • denied publication preserving old bytes with no temporary files left behind; the test safely assumes/skips where POSIX directory denial cannot be exercised;
  • complete replacement and legitimate 10 -> 4 source/candidate shrink publication;
  • missing/malformed/wrong-shape/empty/non-empty cache health and exact 0/1 command outcomes;
  • API key, endpoint-query, and request-query secrets absent from serialized status and captured servlet logs, with sync still bypassing cache;
  • fast local HTTP success, repeated success/error cleanup, and 500 ms delayed endpoints bounded by 100 ms test read timeouts (measured timeout cases: 0.096 s and 0.102 s, each below 2 s).

Residual risks

  • Filesystems without atomic-move support refuse refresh while preserving the old cache.
  • Structural/count validation cannot detect semantically incorrect results or a consistently truncated source catalogue.
  • ResultCache.FILE_SAVE_NAME remains mutable static state and may race during concurrent multi-portal work; changing that requires a separate compatibility refactor.
  • save() retains the existing logging-and-swallowing failure contract; CacheHealthCMD proves structural health, not freshness.
  • FileChannel.force(true) plus atomic rename does not portably fsync the parent directory.
  • Status metadata intentionally loses API-key and query detail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant