Get segment anything working

[cholmes]

Right now I'm not seeing either of the segment anything models working:

If we can get them going that's great, as it shows it's a wider ecosystem. If not let's just take them out for the first release.

[m-mohr]

I'm not sure this is an issue of the app, more likely of the server or ftw-baselines?!

Potentially related to #47

[cholmes]

I'm not sure this is an issue of the app, more likely of the server or ftw-baselines?!

Yeah, most likely the server - just figured I'd start with the issue here.

Potentially related to #47

FTW v2 single window works fine, so it seems to just be segment anything.

[asmithml]

As far as I can tell the ftw-tools wasn't updated to resolve the issue. The problem is/was that the DelineateAnything models use a different call process using .pt files. The logic that converts the model registry to actionable inference doesn't make the right request as it is generic to all models in the registry.

Here are the logs when in the API when trying to use the two variants of DA.

---

LOG 1 - INFO: Starting ML inference

{
  "timestamp": "2025-11-25T18:15:07.%fZ",
  "level": "INFO",
  "message": "Starting ML inference",
  "logger": "app.ml.pipeline",
  "request_id": "c914e37f-c1c0-48ce-b906-d91faf988b30",
  "endpoint": "PUT /v1/example",
  "ml_metrics": {
    "processing_stage": "pipeline_start",
    "bounding_box": {
      "min_lon": 13.0,
      "min_lat": 48.0,
      "max_lon": 13.05,
      "max_lat": 48.05
    },
    "bbox_area_km2": 20.67,
    "image_urls": [
      "https://planetarycomputer.microsoft.com/.../S2B_MSIL2A_20210617T100559_R022_T33UUP_20210624T063729"
    ],
    "model_path": "DelineateAnything",
    "gpu_enabled": true
  }
}

---

LOG 2 - ERROR: Command failed

{
  "timestamp": "2025-11-25T18:15:13.%fZ",
  "level": "ERROR",
  "message": "Command failed: KeyError: 'hyper_parameters'",
  "logger": "app.core.utils",
  "lineno": 30,
  "request_id": "c914e37f-c1c0-48ce-b906-d91faf988b30"
}

---

LOG 3 - ERROR: Inference pipeline failed

{
  "timestamp": "2025-11-25T18:15:13.%fZ",
  "level": "ERROR",
  "message": "Inference pipeline failed",
  "logger": "app.ml.pipeline",
  "ml_context": {
    "processing_stage": "pipeline_start",
    "total_time_ms": 11943.64,
    "error_type": "ValueError",
    "model_path": "DelineateAnything",
    "gpu_enabled": true
  }
}

Traceback:
File "app/ml/pipeline.py", line 110, in execute_inference_pipeline
    inference_result = await run_inference(...)
File "app/ml/pipeline.py", line 63, in run_inference
    await run_async(cmd)
File "app/core/utils.py", line 34, in run_async
    raise ValueError(error_message)
ValueError: KeyError: 'hyper_parameters'

---

LOG 4 - ERROR: Example_Workflow failed

{
  "timestamp": "2025-11-25T18:15:13.%fZ",
  "level": "ERROR",
  "message": "Example_Workflow failed",
  "logger": "app.services.inference_service",
  "ml_metrics": {
    "processing_stage": "example_workflow_failed",
    "error_type": "ValueError"
  }
}

---

LOG 5 - Request completed
Status: 500 Internal Server Error
Duration: 11949.4ms

---

Root Cause: The DelineateAnything model checkpoint is missing a hyper_parameters key that the inference code expects. This is likely a mismatch between how this model was saved/structured vs. the ftw models.

[isaaccorley]

@asmithml is correct here. This is a simple fix actually. We simply need to use a different CLI command pipeline if the model is DelineateAnything. The below command will return a fiboa parquet file which can be immediately displayed in the webapp.

ftw inference run-instance-segmentation ...