Summary
When a user deletes a template via DELETE /templates/:templateId, the API only soft-deletes the database record. The associated build artifacts in GCS/S3 are never removed.
Steps to Reproduce
- Build a template and verify its build artifacts exist in storage.
- Delete the template via
DELETE /templates/:templateId.
- Observe that the build objects remain in GCS/S3 indefinitely.
Expected Behavior
Template deletion should clean up build artifacts that are no longer referenced by any other template.
Actual Behavior
softDeleteTemplate() in api/internal/handlers/template_delete.go only performs DB operations:
SoftDeleteTemplate
ReleaseTemplateAliases
DeleteActiveTemplateBuilds
No storage cleanup (DeleteObjectsWithPrefix / DeleteObjects) is triggered at any point in the user-facing delete flow.
Evidence of Planned but Incomplete Implementation
The query db/queries/builds/get_exclusive_builds_for_template_deletion.sql already exists with the comment:
"Returns builds that are ONLY assigned to this template (safe to delete)"
However, this query has no Go callers — the storage cleanup logic that was meant to consume it was never implemented.
The only code path that calls DeleteObjectsWithPrefix on template builds is the internal TemplateBuildDelete gRPC handler, which is invoked only by admin build-cancel and a deprecated build-start flow — not by user template deletion.
Impact
- Storage leak: Deleted template artifacts accumulate indefinitely in GCS/S3, incurring ongoing storage costs.
- Runtime I/O errors (potential): If an external GCS lifecycle policy or manual GC later removes these objects, any sandboxes still using that build will encounter
object does not exist errors propagating as NBD read failures and Firecracker disk I/O errors (EIO), crashing the sandbox filesystem.
Suggested Fix
On DELETE /templates/:templateId, after marking the template as deleted:
- Call
GetExclusiveBuildsForTemplateDeletion to find build IDs solely owned by this template.
- For each such build, invoke
TemplateBuildDelete (or call DeleteObjectsWithPrefix directly) to remove the GCS/S3 artifacts.
Summary
When a user deletes a template via
DELETE /templates/:templateId, the API only soft-deletes the database record. The associated build artifacts in GCS/S3 are never removed.Steps to Reproduce
DELETE /templates/:templateId.Expected Behavior
Template deletion should clean up build artifacts that are no longer referenced by any other template.
Actual Behavior
softDeleteTemplate()inapi/internal/handlers/template_delete.goonly performs DB operations:SoftDeleteTemplateReleaseTemplateAliasesDeleteActiveTemplateBuildsNo storage cleanup (
DeleteObjectsWithPrefix/DeleteObjects) is triggered at any point in the user-facing delete flow.Evidence of Planned but Incomplete Implementation
The query
db/queries/builds/get_exclusive_builds_for_template_deletion.sqlalready exists with the comment:However, this query has no Go callers — the storage cleanup logic that was meant to consume it was never implemented.
The only code path that calls
DeleteObjectsWithPrefixon template builds is the internalTemplateBuildDeletegRPC handler, which is invoked only by admin build-cancel and a deprecated build-start flow — not by user template deletion.Impact
object does not existerrors propagating as NBD read failures and Firecracker disk I/O errors (EIO), crashing the sandbox filesystem.Suggested Fix
On
DELETE /templates/:templateId, after marking the template as deleted:GetExclusiveBuildsForTemplateDeletionto find build IDs solely owned by this template.TemplateBuildDelete(or callDeleteObjectsWithPrefixdirectly) to remove the GCS/S3 artifacts.