fix: retry for database transaction errors. Fixes #16101

[isubasinghe]

Fixes #16101

Motivation

Workflows using database-backed mutexes or semaphores fail permanently on PostgreSQL SQLSTATE 40001 during lock acquisition. 40001 is a normal serializable-isolation
conflict; the retry loop in TryAcquire is built for it but never fires.

databaseSemaphore.acquire drops the SQL error and returns plain false. tryAcquireImpl wraps the bare failure as bug: failed to acquire something that should have been checked: (empty payload). The retry detector substring-matches serialization/dependencies/deadlock/rollback against that string, finds nothing, and gives
up after one attempt.

Modifications

• acquire returns (bool, error); tryAcquire returns (bool, string, error). SQL error propagates.
• tryAcquireImpl returns the underlying error directly. The bug: wrap stays for its original case: (false, nil) after checkAcquire approved.
• Retry loop rewritten with retry.OnError and wait.Backoff{Duration: 10ms, Factor: 2, Steps: 5, Jitter: 0.5, Cap: 600ms}. Old loop had zero delay; jitter avoids
  replica lockstep.
• Substring check extracted into isRetryableSyncError.
• prioritySemaphore and the semaphore interface updated to match. In-memory acquire returns nil error. Initialize callers still drop the error — pre-existing,
  separate change.

Verification

Test pass

Documentation

None needed.

AI

Claude Code (Claude Opus 4.7) used for investigation, tests, fix, and this description. Reviewed by hand; Cap raised from 200ms to 600ms during review.