Periodic dataset deletion job in calibration to canary createDataSet flow

[BigLep]

Context

Action item from the 2026-05-28 FWSS v1.2.0 / PDPVerifier v3.4.0 post-mortem.

During the PDPVerifier v3.4.0 rollout, createDataSet broke on both calibration and mainnet. Dealbot didn't detect the ~40h calibration outage because it had already reached its 100-dataset-per-provider cap — no new datasets were being created, so nothing went red. The last successful calibnet create was 2026-05-25, two days before the outage even started.

Problem

Dealbot's createDataset job ensures enough datasets exist per provider but never deletes them. Once the cap is reached, the createDataSet on-chain flow is never exercised again, meaning dealbot can't act as a canary for regressions in dataset creation (contract upgrades, ABI changes, deposit requirements, etc.).

Proposal

Add a periodic job (calibration only) that deletes one or more datasets per provider so that the existing createDataset job naturally re-creates them. This gives us a continuous canary for the full createDataSet → deleteDataSet lifecycle on-chain.

Scope

• Calibration only for now — dataset creation costs tFIL (free) rather than real FIL/USDC
• The job should delete datasets at a cadence that keeps dealbot regularly exercising createDataSet without being so aggressive that it disrupts ongoing deal/retrieval checks
• Should integrate with existing alerting so that a createDataSet failure after deletion surfaces as a dealbot failure
• Metrics visualization in BetterStack by augmenting the existing Dealbot internal dashboard(s)

Design considerations

• Coordinate with the existing createDataset job so deleted datasets get replenished promptly
• Consider whether to delete the oldest dataset, a random one, or rotate through them
• Ensure in-flight deals/retrievals against a dataset aren't disrupted by deletion (or handle gracefully)
• The deleteDataSet flow itself is also worth canary-testing (it exercises PDPVerifier.deleteDataSet which has its own deposit/cleanup mechanics as of v3.4.0)
• Ensure it has the ability not to run in mainnet.
• Give a way to specify how frequently deletes will happen, so we can also reason about how frequently creates will happen.

Notes

• Before doing any implementation, a design document should be created and reviewed with the team. That could be in Notion or given we have good design docs in https://github.com/FilOzone/dealbot/blob/main/docs, maybe we add here. Ideally it would live alongside the design for the "createDataSets", but I see we don't have that there.

Related

• Post-mortem: 2026-05-28 Hotfix FWSS v1.2.0 / PDPVerifier v3.4.0
• Post-mortem action item: Dealbot deletes and creates datasets in calibration
• new DataSet creation on regular intervals for each SP #64 — original dataset creation on intervals (closed, implemented creation but not deletion)
• Define cleanup policy for blocked providers and terminated datasets #462 — cleanup policy for blocked providers and terminated datasets (related but different scope)
• add monitoring to dealbot ops dashboard for cleanup job #500 — monitoring for cleanup job