bitcoind container runs with default Docker ulimit of 1024 file descriptors, causing LevelDB corruption during IBD

[invite-frey]

Repeated crashes during IBD with increasing frequency, always during a UTXO flush:

[warning] Flushing large (N MiB) UTXO set to disk, it may take several minutes
[error] Fatal LevelDB error: Corruption: block checksum mismatch: /home/bitcoin/.bitcoin/chainstate/XXXXXX.ldb

Many different possible causes were explored and ruled out. The only change that resolved the issue was increasing the container's file descriptor limit.

The error presents as hardware failure but is not. LevelDB exhausts available file descriptors mid-compaction and cannot finalize the write cleanly, producing a file with an invalid checksum rather than a clear "too many open files" error.

Fix

Add to the bitcoind/bitcoinknots fragment(s):

yaml
services:
  bitcoind:
    ulimits:
      nofile:
        soft: 65536
        hard: 65536