Commit c929386
authored
Record policy-run failures as durable, actionable events (Review Flow PR 1) (#7269)
# Description of Changes
PR 1 of the failure-notification work: a durable, team-scoped record of
**why a policy run failed**, surfaced in the portal with the triage
actions each failure allows.
Today a failed policy run is not quite invisible, but it is unusable:
the ledger marks the file `ERROR`, and the audit aspect keeps the
exception message and status code. Nothing classifies either one,
nothing surfaces them, and neither offers a next step. If the file came
from a folder, bucket or webhook there is also no user watching, so
nobody learns it never made it through. This adds the record and the
read surface; the remediation that acts on documents comes later (see
below).
## What this does
**A failure kind registry as data.** `FailureKind` describes what can go
wrong: a stable wire id, i18n keys, an English fallback, and four facets
the review surface needs (`Stage`, `Severity`, `Remedy`, `Scope`). It is
shaped like the existing `ExceptionUtils.ErrorCode` and *links* to that
vocabulary rather than replacing it.
**Classification off structured codes, not message matching.** Policy
steps dispatch over loopback HTTP, so a tool's 4xx arrives as a
`RestClientResponseException` whose body is the Problem Details document
carrying `errorCode`. `FailureClassifier` reads that. Anything
unrecognised becomes `UNKNOWN`, which is the point: every failed run
gets an addressable record from day one, and which kinds to promote next
is answered by production frequency rather than guesswork.
**Actions declared by a kind, implemented as beans.** A kind lists the
`FailureActionId`s it offers; behaviour lives in `FailureAction` beans
resolved by id — the idiom this codebase already uses for `InputSource`,
`PolicyOutputSink` and `PolicyTrigger`. A kind cannot be sent an action
it never declared (400), so an incoherent pairing is unreachable rather
than merely unrendered. A new kind ships as a registry entry plus copy:
no new endpoint, no UI change.
**Repeat folding.** Recording folds a genuine repeat into the existing
incident instead of inserting again, keyed on `(team_id, dedup_key)`.
That matters for a snapshot-mode source that re-lists every file on each
sweep: the same broken file is one incident, not one per sweep. Distinct
files keep distinct rows. The unique constraint is enforced by the
database, and a writer that loses the insert race folds into the
winner's row.
One granularity caveat worth naming: nothing populates `file_id` in this
PR, so every row has it NULL. A FILE-scoped kind therefore dedups on
`policy + run` rather than `policy + file`. That still yields one row
per document for the sources shipped here, because the folder, S3 and
webhook sources each start one run per file; it stops holding as soon as
a single run carries several documents, which is why editor-origin
reporting (item 3 below) populates `file_id`.
**No document identity is stored.** No file name, no content. `fileId`
is an opaque reference only the owner's own client can resolve locally.
`detail` keeps the raw message (the only diagnostic an `UNKNOWN` failure
has) with anything path- or filename-shaped stripped on the way in,
capped at 2,000 characters. `PolicyExecutor`'s type-mismatch message now
reports the *extension* rather than the filename, since that message
becomes the stored `detail`.
**Access.** Reads and triage are leader-only, gated exactly the way
`PolicyController` gates policy editing, with the single-user carve-out
when login is disabled. Every read and write is scoped to the caller's
own team from the authenticated principal — there is no team parameter
on the API.
Self-hosted needs no migration: the table is created from the entity by
`ddl-auto=update`, as with every other table.
## What this does not do yet
- **Actions are incident dispositions, not document dispositions.**
Acknowledge and Dismiss change how a failure is displayed and touch
nothing else — not the document, not the processed-file ledger, not the
run, not any output destination. That is what makes them safe to offer
against `UNKNOWN`, and why there is no Approve/Release yet.
- **Two kinds only.** `INPUT_PASSWORD_PROTECTED` and `UNKNOWN`.
Everything else classifies as `UNKNOWN` and shows its raw message.
- **Editor-origin failures are not reported.** Every row is `PROCESSOR`.
`FailureOrigin.EDITOR` and `API` exist in the enum but nothing writes
them.
- **The list is dev-only for now.** The section renders behind
`import.meta.env.DEV`, so it ships in no production bundle. The
endpoints are live and gated.
- **No retention or per-team cap** on `file_run_events`. Tracked
separately.
- **No suspend-and-prompt.** `PolicyInputRequiredException` and the
engine's `suspend()` exist but nothing throws it, so a run cannot pause
to ask for a password today.
- **SaaS needs a migration** in `Stirling-PDF-SaaS` (`CREATE TABLE IF
NOT EXISTS stirling_pdf.file_run_events`), per the convention documented
at `app/saas/src/main/resources/application-saas.properties:21`.
## What follows in later PRs
1. **Map the remaining error codes to specific kinds** — corrupted file,
OCR unavailable, output destination unreachable, entitlement refusals,
and so on — each with its own copy and its own action set, replacing
today's `UNKNOWN` catch-all with a named notification in the review UI.
2. **Real remediation actions** attached to those kinds: fix (supply a
password and resume), skip (drop this file, continue the batch), and
decline (reject an incoming file outright), acting on the held document
rather than only on the incident row. This is where the
suspend-and-prompt path gets wired.
3. **Editor-origin reporting**, so a failure a user hits in the editor
lands in the same queue as one from a bucket.
4. **The user-facing review surface**: notifications with a sticky
review section, per-file badges, and an export gate, with the dev-only
list here replaced by the real thing.
## How to test
Needs a SaaS or proprietary build with login enabled, and an account
that leads a team.
1. Create a policy in the Processor with any step (Auto-redact is fine)
and a source you can drop files into.
2. Upload two files that will fail it: **a password-protected PDF**, and
**a corrupted PDF** (truncate a valid one, or rename a `.csv` to
`.pdf`).
3. Let the policy run and fail on both.
4. Go to the portal's **Documents** view and scroll to **Failures** (dev
builds only).
Expect two rows:
- **Password-protected document** — classified from `E004`, with the
kind's own labels **"I'll unlock this"** and **"Skip this file"** rather
than generic wording.
- **Unrecognised failure** — the corrupted file, classified `UNKNOWN`
(`E001` is not claimed by a kind yet), showing its raw message with
generic **Acknowledge** / **Dismiss**.
Neither row contains a file name anywhere, including in the raw message.
Press **Show raw JSON** to read exactly what the server returned. Acting
on a row transitions it and comes back with both buttons disabled and a
reason.
Re-running the same batch increments the occurrence count on the
existing rows rather than adding new ones; two *different*
password-protected files produce two separate rows.
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [ ] I have run `task check` to verify linters, typechecks, and tests
pass1 parent 0ff4ef6 commit c929386
71 files changed
Lines changed: 6500 additions & 46 deletions
File tree
- app/proprietary/src
- main/java/stirling/software/proprietary
- failure
- policy
- engine
- input
- model
- network
- security/configuration
- test/java/stirling/software/proprietary
- failure
- policy
- controller
- engine
- frontend
- .storybook
- editor
- public/locales/en-US
- src
- core/ui
- portal
- api
- components
- documents
- failures
- pipelines
- policies
- sources
- mocks
- handlers
- queries
- views
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 39 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
Lines changed: 32 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
Lines changed: 23 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
Lines changed: 49 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
Lines changed: 11 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
Lines changed: 68 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
Lines changed: 109 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
0 commit comments