Run classification locally first and only escalate an unsure verdict to the AI (#7580)

Split out of #7574 — this is the classification half, which is
independent of the editor-source work and can land on its own.

## What this does

- **Runs the local heuristic first and only escalates an unsure verdict
to the AI.** A high-confidence local answer stands; anything less (or a
file the heuristic hasn't reached yet) goes to the engine. A wrong label
costs more than an engine call, so the bar is deliberately strict.
- **Makes `classify` an authorable pipeline task**, so it can be used as
a step like any other tool, and skips files that are already classified.
- **Leaves the seeded Classification policy unowned** rather than naming
a `system` placeholder that was never a real user; existing seeds are
repaired on boot.

## Review feedback applied

From @jbrunton96 on #7574:

- **The generic runner no longer names classification.** Everything
classification-specific moved into
`proprietary/data/classificationPolicy.ts`, and `usePolicyAutoRun` now
asks capability questions instead: `policyRewritesDocument`,
`policyDeliversOutputFiles`, `policyRequiresAiEngine`,
`shouldDispatchToAi`. There is no `id === "classification"` left in the
runner.
- **Ordering is no longer a name in the runner.**
`pinClassificationLast` is gone; the runner sorts annotating policies
after rewriting ones. The constraint is real: an annotating policy is
non-blocking, so a rewriting one running after it forks from the
pre-annotation version and drops the labels. To be straight about what
this is and isn't - see "Still open" below - `policyRewritesDocument` is
still keyed on the category id, not on a property each policy declares.
The check moved out of the runner; it did not stop being a check on one
id.
- **Confidence is typed.** New `ClassificationConfidence` union in
`core/types/fileContext.ts`, reused by `fileStorage`,
`HeuristicConfidence`, and the trusted-verdict constant instead of being
respelled at each site.
- **Comments trimmed** to the repo's 2-line guideline, and a stale
seeder javadoc that still claimed an internal-user owner was corrected.

## Still open, deliberately

`classificationPolicy.ts` answers its capability questions with
`categoryId === "classification"`. That is the same check relocated, not
removed, and the module doc now says so outright.

Deliberate, for two reasons:

- **The concept it would be declared against is going away.** Policies
are becoming pipelines with labels behind a separate enforcement layer,
which removes the category the flag would live on. A capability system
built on `categoryId` today gets migrated twice.
- **Classification is genuinely privileged, not accidentally special.**
It is the only policy with a browser-side implementation, so it can
answer without the server. That is a product decision, and a local-only
mode for set scenarios is planned - the flag for it should be designed
with that feature, not guessed at now.

The end state for the rest: an in-place output mode retires the ordering
rule and `policyDeliversOutputFiles`, and a run result that can carry
findings as well as files retires the remainder. Both touch the import
path, which is the most delicate code in `usePolicyAutoRun` - not
something to bolt on to a PR that has already been split once.

Nothing is broken by leaving it. A user-built classify pipeline still
gets its labels: the generic import path reads them off the returned
PDF. It versions the file instead of labelling in place, and it misses
the local-heuristic shortcut, so it always bills the engine.

## Testing

- `classificationPolicy.test.ts` — 12 cases covering each capability and
the escalation rule
- Full frontend `proprietary` project: 39 files / 442 tests
- `:proprietary:test` for `DefaultClassificationPolicySeederTest` +
`ClassifyLabelControllerTest`
- `tsc --noEmit` on core, proprietary, portal, saas, desktop, cloud

---------

Co-authored-by: James Brunton <jbrunton96@gmail.com>

Author: Frooodle

app/proprietary/src/main/java/stirling/software/proprietary/controller/api/ClassifyLabelController.java

@@ -9,6 +9,7 @@
 import java.util.Set;
 
 import org.apache.pdfbox.pdmodel.PDDocument;
+import org.apache.pdfbox.pdmodel.PDDocumentInformation;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.core.io.Resource;
 import org.springframework.http.MediaType;
@@ -20,12 +21,13 @@
 import org.springframework.web.multipart.MultipartFile;
 
 import io.github.pixee.security.Filenames;
-import io.swagger.v3.oas.annotations.Hidden;
 import io.swagger.v3.oas.annotations.Operation;
 import io.swagger.v3.oas.annotations.tags.Tag;
 
 import lombok.extern.slf4j.Slf4j;
 
+import stirling.software.common.model.tool.ToolFormat;
+import stirling.software.common.model.tool.ToolIO;
 import stirling.software.common.service.CustomPDFDocumentFactory;
 import stirling.software.common.service.PdfMetadataService;
 import stirling.software.common.service.UserServiceInterface;
@@ -48,11 +50,13 @@
  * <p>Runs as a Classification-policy pipeline step: it reads a bounded page window, asks the AI
  * engine to classify the document against the built-in label set, and stores the engine's JSON
  * answer — minus the transport-only {@code outcome} field — in the custom Info-dictionary key
- * {@link PdfMetadataService#CLASSIFICATION_KEY}. Returns the labelled PDF. Not intended for direct
- * client use.
+ * {@link PdfMetadataService#CLASSIFICATION_KEY}. Returns the labelled PDF.
+ *
+ * <p>Published in the API spec rather than hidden, so the tool-model generator emits it and a
+ * pipeline can name it as a step like any other tool. Classification is a thing a pipeline does,
+ * not a thing only the Classification policy may do.
  */
 @Slf4j
-@Hidden
 @RestController
 @RequestMapping("/api/v1/ai/tools")
 @Tag(name = "AI Tools", description = "Dispatchable AI-backed tools.")
@@ -99,19 +103,31 @@ public ClassifyLabelController(
     }
 
     @PostMapping(value = "/classify-and-label", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
+    // PDF in, the same PDF out with a verdict on it, so a chain can be checked across this step.
+    @ToolIO(accepts = ToolFormat.PDF, produces = ToolFormat.PDF)
     @Operation(
             summary = "Classify a PDF and label its metadata",
             description =
                     "Reads the first two and last two pages, classifies the document via the AI"
                             + " engine, and stores the result in the StirlingPDFClassification"
-                            + " metadata field. Dispatched by the Classification policy; not"
-                            + " intended for direct client use.")
+                            + " metadata field. A document that already carries a verdict is"
+                            + " passed through untouched unless reclassify=true.")
     public ResponseEntity<Resource> classifyAndLabel(
-            @RequestParam("fileInput") MultipartFile fileInput) throws IOException {
+            @RequestParam("fileInput") MultipartFile fileInput,
+            @RequestParam(value = "reclassify", defaultValue = "false") boolean reclassify)
+            throws IOException {
         aiFeatureGate.requireClassify();
         try (PDDocument document = pdfDocumentFactory.load(fileInput, true)) {
             String fileName = safeFileName(fileInput.getOriginalFilename());
 
+            if (!reclassify && isClassified(document)) {
+                // Classifying twice costs a second engine call and charges for it, and a document
+                // that already carries a verdict has nothing new to learn. A pipeline can run this
+                // step over a mixed batch without paying for the ones already done.
+                log.debug("[classify-and-label] {} already classified; passing through", fileName);
+                return WebResponseUtils.pdfDocToWebResponse(document, fileName, tempFileManager);
+            }
+
             List<EngineLabel> allowed = resolveAllowedLabels();
             if (allowed.isEmpty()) {
                 // No vocabulary to classify against: pass the file through unlabelled rather than
@@ -135,6 +151,25 @@ public ResponseEntity<Resource> classifyAndLabel(
         }
     }
 
+    /**
+     * Whether a verdict is already on the document.
+     *
+     * <p>This only reads back what a previous run of this step wrote. It is not a statement that
+     * the verdict is trustworthy: the key is ordinary PDF metadata that whoever supplied the file
+     * can set. Skipping the engine on the strength of it is safe because the cost of being wrong is
+     * a missing re-classification, not a wrong decision. Anything that makes a SECURITY decision
+     * from this field - routing a document somewhere on the strength of its label, say - must
+     * classify with {@code reclassify=true} rather than trust what arrived.
+     */
+    private static boolean isClassified(PDDocument document) {
+        PDDocumentInformation info = document.getDocumentInformation();
+        if (info == null) {
+            return false;
+        }
+        String existing = info.getCustomMetadataValue(PdfMetadataService.CLASSIFICATION_KEY);
+        return existing != null && !existing.isBlank();
+    }
+
     private List<AiPageText> extractWindow(PDDocument document) throws IOException {
         List<AiPageText> pages = new ArrayList<>();
         for (int pageNumber : windowPageNumbers(document.getNumberOfPages(), WINDOW_PAGES)) {

app/proprietary/src/main/java/stirling/software/proprietary/policy/model/Policy.java

@@ -85,6 +85,11 @@ public Policy withOutput(OutputSpec resolved) {
         return new Policy(id, name, owner, enabled, inputs, steps, resolved, outputIds, teamId);
     }
 
+    /** A copy under a different owner (e.g. moving a seed off a placeholder name). */
+    public Policy withOwner(String newOwner) {
+        return new Policy(id, name, newOwner, enabled, inputs, steps, output, outputIds, teamId);
+    }
+
     /** A copy referencing the given saved output destinations. */
     public Policy withOutputIds(List<String> newOutputIds) {
         return new Policy(id, name, owner, enabled, inputs, steps, output, newOutputIds, teamId);

app/proprietary/src/main/java/stirling/software/proprietary/policy/seed/DefaultClassificationPolicySeeder.java

@@ -22,8 +22,8 @@
 import stirling.software.proprietary.security.service.TeamService;
 
 /**
- * Seeds an enabled Classification policy per team so classification is on by default. Idempotent;
- * skips the internal team.
+ * Seeds an enabled Classification policy per team; idempotent, skips the internal team. Left
+ * unowned: nobody created it, and an owner here would have to name a real user.
  */
 @Slf4j
 @Component
@@ -34,6 +34,11 @@ public class DefaultClassificationPolicySeeder {
     private static final String CLASSIFY_ENDPOINT = "/api/v1/ai/tools/classify-and-label";
     private static final String POLICY_NAME = "Classification Policy";
 
+    /**
+     * Pre-existing seeds used this placeholder, which was never a user; see {@link #repairOwner}.
+     */
+    private static final String LEGACY_OWNER = "system";
+
     private final PolicyStore policyStore;
     private final TeamRepository teamRepository;
 
@@ -46,9 +51,8 @@ public void seedDefaultTeamOnStartup() {
                 .ifPresent(team -> seedIfMissing(team.getId(), team.getName()));
     }
 
-    // Any team created at runtime (admin-created, SaaS sign-ups). Seeds inside the team's own
-    // transaction: rollback still leaves no policy behind, and the store's pessimistic lock needs a
-    // live transaction, which AFTER_COMMIT cannot offer.
+    // Seeds inside the new team's own transaction: rollback leaves no policy behind, and the
+    // store's pessimistic lock needs a live transaction, which AFTER_COMMIT cannot offer.
     @TransactionalEventListener(phase = TransactionPhase.BEFORE_COMMIT)
     public void onTeamCreated(TeamCreatedEvent event) {
         seedIfMissing(event.teamId(), event.teamName());
@@ -58,16 +62,33 @@ private void seedIfMissing(Long teamId, String teamName) {
         if (teamId == null || TeamService.INTERNAL_TEAM_NAME.equals(teamName)) {
             return;
         }
-        boolean alreadySeeded =
+        Policy existing =
                 policyStore.findByTeam(teamId).stream()
-                        .anyMatch(DefaultClassificationPolicySeeder::isClassification);
-        if (alreadySeeded) {
+                        .filter(DefaultClassificationPolicySeeder::isClassification)
+                        .findFirst()
+                        .orElse(null);
+        if (existing != null) {
+            repairOwner(existing);
             return;
         }
         policyStore.save(defaultPolicy(teamId));
         log.info("Seeded default Classification policy for team {}", teamId);
     }
 
+    /**
+     * Clear an owner seeded as a placeholder name. An owner someone deliberately set is left alone.
+     */
+    private void repairOwner(Policy policy) {
+        if (!LEGACY_OWNER.equals(policy.owner())) {
+            return;
+        }
+        policyStore.save(policy.withOwner(null));
+        log.info(
+                "Cleared placeholder owner '{}' on Classification policy {}",
+                LEGACY_OWNER,
+                policy.id());
+    }
+
     private static boolean isClassification(Policy policy) {
         return policy.output() != null
                 && CATEGORY.equals(policy.output().options().get("categoryId"));
@@ -85,7 +106,9 @@ static Policy defaultPolicy(Long teamId) {
         return new Policy(
                 null,
                 POLICY_NAME,
-                "system",
+                // Nobody created this - it is seeded. A name here would have to be a real user, and
+                // every consumer of owner already handles its absence.
+                null,
                 true,
                 List.of(),
                 List.of(new PipelineStep(CLASSIFY_ENDPOINT, Map.of())),

app/proprietary/src/test/java/stirling/software/proprietary/controller/api/ClassifyLabelControllerTest.java

@@ -14,6 +14,7 @@
 import java.util.List;
 
 import org.apache.pdfbox.pdmodel.PDDocument;
+import org.apache.pdfbox.pdmodel.PDDocumentInformation;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.ExtendWith;
 import org.mockito.ArgumentCaptor;
@@ -76,7 +77,7 @@ private void stubSinglePageDocument() throws Exception {
                 .thenReturn("{\"outcome\":\"classification\",\"labels\":[\"invoice\"]}");
 
         try {
-            controller.classifyAndLabel(file);
+            controller.classifyAndLabel(file, false);
         } catch (Exception ignored) {
             // WebResponseUtils.pdfDocToWebResponse needs a real temp file; the engine call and
             // metadata write we assert on have already happened by the time it runs.
@@ -89,6 +90,52 @@ private JsonNode sentEngineRequest() throws Exception {
         return objectMapper.readTree(body.getValue());
     }
 
+    /** Stubs a document that already carries a verdict, as a second run over a batch would see. */
+    private MultipartFile alreadyClassifiedDocument() throws Exception {
+        PDDocument document = mock(PDDocument.class);
+        PDDocumentInformation info = mock(PDDocumentInformation.class);
+        when(document.getDocumentInformation()).thenReturn(info);
+        when(info.getCustomMetadataValue(PdfMetadataService.CLASSIFICATION_KEY))
+                .thenReturn("{\"labels\":[\"invoice\"]}");
+        MultipartFile file = mock(MultipartFile.class);
+        when(file.getOriginalFilename()).thenReturn("invoice.pdf");
+        when(pdfDocumentFactory.load(any(MultipartFile.class), eq(true))).thenReturn(document);
+        return file;
+    }
+
+    @Test
+    void classifyAndLabel_skipsADocumentThatAlreadyCarriesAVerdict() throws Exception {
+        withLabels(List.of(new ClassificationLabel("invoice", "Invoice", null)));
+        MultipartFile file = alreadyClassifiedDocument();
+
+        try {
+            controller.classifyAndLabel(file, false);
+        } catch (Exception ignored) {
+            // The response needs a real temp file; the decision under test happens before it.
+        }
+
+        // No second engine call, and no charge for one: re-classifying buys the same answer twice.
+        verify(aiEngineClient, never()).post(anyString(), anyString(), any());
+        verify(pdfMetadataService, never()).setClassificationMetadata(any(), anyString());
+    }
+
+    @Test
+    void classifyAndLabel_reclassifiesWhenAskedTo() throws Exception {
+        withLabels(List.of(new ClassificationLabel("invoice", "Invoice", null)));
+        MultipartFile file = alreadyClassifiedDocument();
+        when(pdfContentExtractor.extractPageTextRaw(any(), eq(1))).thenReturn("Invoice total");
+        when(aiEngineClient.post(eq("/api/v1/documents/classify"), anyString(), isNull()))
+                .thenReturn("{\"outcome\":\"classification\",\"labels\":[\"receipt\"]}");
+
+        try {
+            controller.classifyAndLabel(file, true);
+        } catch (Exception ignored) {
+            // As above.
+        }
+
+        verify(aiEngineClient).post(eq("/api/v1/documents/classify"), anyString(), isNull());
+    }
+
     @Test
     void classifyAndLabel_writesClassificationWithoutOutcome() throws Exception {
         withLabels(List.of(new ClassificationLabel("invoice", "Invoice", null)));

app/proprietary/src/test/java/stirling/software/proprietary/policy/seed/DefaultClassificationPolicySeederTest.java

@@ -36,10 +36,14 @@ private DefaultClassificationPolicySeeder seeder() {
     }
 
     private static Policy classificationPolicy(Long teamId) {
+        return classificationPolicy(teamId, null);
+    }
+
+    private static Policy classificationPolicy(Long teamId, String owner) {
         return new Policy(
                 "p1",
                 "Classification Policy",
-                "system",
+                owner,
                 true,
                 List.of(),
                 List.of(),
@@ -77,6 +81,29 @@ void doesNotSeedWhenAClassificationPolicyAlreadyExists() {
         verify(policyStore, never()).save(any());
     }
 
+    @Test
+    void clearsAPlaceholderOwnerSeededBeforeOwnersHadToBeReal() {
+        when(policyStore.findByTeam(7L)).thenReturn(List.of(classificationPolicy(7L, "system")));
+
+        seeder().onTeamCreated(new TeamCreatedEvent(7L, "Acme"));
+
+        // "system" was never a user row, and a step dispatch authenticates as the owner. Absence
+        // is handled everywhere; a placeholder name is not.
+        ArgumentCaptor<Policy> saved = ArgumentCaptor.forClass(Policy.class);
+        verify(policyStore).save(saved.capture());
+        assertThat(saved.getValue().owner()).isNull();
+        assertThat(saved.getValue().id()).isEqualTo("p1");
+    }
+
+    @Test
+    void leavesADeliberatelyChosenOwnerAlone() {
+        when(policyStore.findByTeam(7L)).thenReturn(List.of(classificationPolicy(7L, "alice")));
+
+        seeder().onTeamCreated(new TeamCreatedEvent(7L, "Acme"));
+
+        verify(policyStore, never()).save(any());
+    }
+
     @Test
     void doesNotSeedForTheInternalTeam() {
         seeder().onTeamCreated(new TeamCreatedEvent(2L, "Internal"));

frontend/editor/public/locales/en-US/translation.toml

@@ -4592,6 +4592,10 @@ desc = "Change document restrictions and permissions"
 tags = "permissions,restrictions,rights,access control,allow,deny,printing,copying,editing,modify permissions,security settings,user rights"
 title = "Change Permissions"
 
+[home.classify]
+desc = "Identify what kind of document this is and tag it."
+title = "Classify"
+
 [home.compare]
 desc = "Compares and shows the differences between 2 PDF Documents"
 tags = "difference,compare,diff,compare PDFs,compare documents,find differences,show differences,changes,what changed,track changes,revisions,version compare,side by side,contrast,delta"
@@ -10841,6 +10845,7 @@ searchPlaceholder = "Search tools..."
 
 [toolPicker.subcategories]
 advancedFormatting = "Advanced Formatting"
+ai = "AI"
 automation = "Automation"
 developerTools = "Developer Tools"
 documentReview = "Document Review"

frontend/editor/public/og-metadata.json

@@ -195,6 +195,11 @@
       "title": "Compress - Stirling PDF",
       "description": "Compress PDFs to reduce their file size."
     },
+    "classify": {
+      "image": "/og_images/home.png",
+      "title": "Classify - Stirling PDF",
+      "description": "Identify what kind of document this is and tag it."
+    },
     "extractPages": {
       "image": "/og_images/extract-pages.png",
       "title": "Extract Pages - Stirling PDF",
@@ -575,6 +580,7 @@
     "/remove-cert-sign": "removeCertSign",
     "/unlock-p-d-f-forms": "unlockPDFForms",
     "/compress": "compress",
+    "/classify": "classify",
     "/extract-pages": "extractPages",
     "/reorganize-pages": "reorganizePages",
     "/extract-images": "extractImages",

frontend/editor/public/og-metadata.saas.json

@@ -196,6 +196,11 @@
       "title": "Compress - Stirling PDF",
       "description": "Compress PDFs to reduce their file size."
     },
+    "classify": {
+      "image": "/og_images/home.png",
+      "title": "Classify - Stirling PDF",
+      "description": "Identify what kind of document this is and tag it."
+    },
     "extractPages": {
       "image": "/og_images/extract-pages.png",
       "title": "Extract Pages - Stirling PDF",
@@ -588,6 +593,7 @@
     "/remove-cert-sign": "removeCertSign",
     "/unlock-p-d-f-forms": "unlockPDFForms",
     "/compress": "compress",
+    "/classify": "classify",
     "/extract-pages": "extractPages",
     "/reorganize-pages": "reorganizePages",
     "/extract-images": "extractImages",

frontend/editor/scripts/generate-tool-api-types.mts

@@ -11,20 +11,16 @@ import { dirname, resolve } from "node:path";
 import { parseArgs } from "node:util";
 import { compile, type JSONSchema } from "json-schema-to-typescript";
 
-// The API namespaces whose endpoints a pipeline can reference. `/api/v1/ai/tools/`
-// is absent from the spec, so it cannot appear here. Extend this list when other
-// namespaces become tools.
-//
-// `/api/v1/filter/` and `/api/v1/integration/` are included even though neither is a
-// user-facing tool: a stored pipeline can contain one, and ToolEndpoint keys the I/O
-// table, so leaving them out would stop a chain being checked past such a step.
+// Endpoints a pipeline can reference. filter/integration are not user-facing tools but a stored
+// pipeline can contain one; the AI namespace is admitted one endpoint at a time, not wholesale.
 const ALLOWED_PATH_PREFIXES = [
   "/api/v1/general/",
   "/api/v1/misc/",
   "/api/v1/security/",
   "/api/v1/convert/",
   "/api/v1/filter/",
   "/api/v1/integration/",
+  "/api/v1/ai/tools/classify-and-label",
 ];
 
 // File plumbing, not user parameters: `fileInput` and `file` are the uploaded primary document