Inspection: Wallet mismatch in deal creation job

[silent-cipher]

Problem Surfaced

The piece_cleanup job began throwing the following error for 32 deals:

Error: StorageContext resolveByDataSetId failed: Data set <data_set_id> is not owned by
0x305025D07c1DEe47F25a4990179eFf2becddCA0B (owned by 0xa5F90bc2AA73a2E0Bad4D7092a932644d5dD5d71)

The Synapse SDK's resolveByDataSetId enforces wallet ownership when performing piece deletion. The current process wallet (0x305025D) does not own those data sets on-chain. They were created by the old wallet (0xa5F90bc). Cleanup therefore fails permanently for all 32 affected deals under the current process.

---

What We Confirmed from the Database

Step 1 — on-chain check: The new wallet (0x305025D) legitimately owns data sets with IDs greater than 721 on-chain. Data set IDs below 721 in dealbot db belong to the old wallet (0xa5F90bc).

Step 2 — DB query: Querying deals tagged with the new wallet but referencing a data set ID below the ownership boundary surfaced exactly 32 mismatched deals:

SELECT id, sp_address, data_set_id, piece_id, created_at
FROM deals
WHERE wallet_address = '0x305025D07c1DEe47F25a4990179eFf2becddCA0B'
  AND data_set_id::bigint < 721
ORDER BY created_at;

Result: 32 deals, all created between 2026-04-02 19:06:17.713129+00 and 2026-04-02 20:49:04.935386+00.

Field	Value
wallet_address (DB)	0x305025D07c1DEe47F25a4990179eFf2becddCA0B (new wallet)
data_set_id range (DB)	< 721 — on-chain owned by 0xa5F90bc2AA73a2E0Bad4D7092a932644d5dD5d71 (old wallet)
created_at window	2026-04-02 19:06:17+00 → 2026-04-02 20:49:04+00
Count	32 deals

The drift window aligns precisely with two infrastructure commits:

• b549bc8 — 2026-04-02 18:31 GMT+00 — env vars updated: WALLET_ADDRESS changed to new wallet, WALLET_PRIVATE_KEY removed, SESSION_KEY_PRIVATE_KEY added
• 5da1cac — 2026-04-02 20:58 GMT+00 — full production deployment

---

Proposed Fix

Immediate: patch the 32 affected rows in the database

Because piece_cleanup filters candidates by wallet_address, these 32 deals will never be selected for cleanup by the current process. The recommended remediation is to mark them as cleaned up:

UPDATE deals
SET wallet_address = '0xa5F90bc2AA73a2E0Bad4D7092a932644d5dD5d71', cleaned_up = true, cleaned_up_at = NOW()
WHERE wallet_address = '0x305025D07c1DEe47F25a4990179eFf2becddCA0B'
  AND data_set_id::bigint < 721
  AND cleaned_up = false;

Important: The pieces from old wallet are already deleted on-chain.

---

Further Investigation Required

The commit at 18:31 migrated the system from direct-key mode (WALLET_PRIVATE_KEY) to session-key mode (SESSION_KEY_PRIVATE_KEY), and updated WALLET_ADDRESS to the new wallet (0x305025D).

In session-key mode, WALLET_ADDRESS is the single source of truth for both the SDK's on-chain identity and the DB stamp:

// synapse-factory.ts — session-key mode
const readClient = createClient({
  account: { ...sessionKeyAccount, address: walletAddress }, // from blockchainConfig.walletAddress
});

// deal.service.ts:316
deal.walletAddress = this.blockchainConfig.walletAddress;   // same source

A clean pod restart with both changes applied atomically would be fully consistent — readClient.account.address and deal.walletAddress always agree. There is no obvious code path in a clean restart that produces the drift.

[BigLep]

@silent-cipher : lets do the immediate patch for now (please post in slack the SQL you're going to execute before you run it and get a ✅ before executing. Also confirm that it will only touch the affected rows.).

After that, I think we can defer more investigation until post GA.

[silent-cipher]

I've successfully run the remediation query, which should resolve the ongoing issues for now.

The root cause of the drift is still unknown. I'll be prioritizing other work for now and will return to this investigation later if no one else uncovers the cause in meantime.

[SgtPooki]

those rows are all old right? I think it happened during rollover to a new wallet, which we shouldn't run into again..

[BigLep]

For visibility: cleanup queries were verified/approved in https://filecoinproject.slack.com/archives/C08TVNKJV7C/p1780413374649079?thread_ts=1780371887.906609&cid=C08TVNKJV7C

[silent-cipher]

those rows are all old right?

yes, those rows were all from old dealbot wallet.