Possible blob inconsistency after IMAP APPEND causing indexing failure (occurs with filesystem and S3 storage)

[gdewey]

Issue Description

During testing of IMAP APPEND operations, the message is successfully accepted and appended to the mailbox, but the asynchronous indexing task fails because the message blob cannot be retrieved.

The IMAP command completes successfully and the message metadata is created, but the indexing worker fails with Blob not found.

This behavior occurs regardless of the blob storage backend being used:

•	filesystem storage
•	S3 storage

Because the issue reproduces with both backends, it appears unlikely to be specific to a single storage implementation.

Observed behavior

1. An IMAP client issues an APPEND command.
2. The server successfully accepts the message:

OK [APPENDUID 2211562292 29] APPEND completed

3. The server logs the message ingestion:

INFO Message appended via IMAP
documentId = 29
blobId = "214e213dc66be12eab834d505beaadc5f482babd626f97b857d1b1f400d08587"
size = 151295

4. Shortly after, the indexing worker attempts to process the message.
5. The worker fails to retrieve the blob:

TRACE Blob read operation ... size = 0
DEBUG Record not found in database ... "Blob not found"
WARN Task failed during processing ... "Indexing task failed"

Relevant log excerpt

INFO Message appended via IMAP (message-ingest.imap-append)
listenerId = "imaptls"
accountId = 141
documentId = 29
mailboxId = [7]
blobId = "214e213dc66be12eab834d505beaadc5f482babd626f97b857d1b1f400d08587"
size = 151295

DEBUG IMAP APPEND command (imap.append)
mailboxName = "Archive"

TRACE Raw IMAP output sent
"50 OK [APPENDUID 2211562292 29] APPEND completed"

TRACE Blob read operation (store.blob-read)
size = 0

DEBUG Record not found in database
details = "Blob not found"
collection = "email"
details = "Failed to build document for indexing"

WARN Task failed during processing
details = "Indexing task failed"

Expected Behavior

After a successful IMAP APPEND:

1.	Message metadata should be stored.
2.	The message blob should be persisted.
3.	The indexing task should retrieve the blob and complete successfully.

Actual Behavior

Actual behavior

1.	IMAP APPEND succeeds.
2.	Message metadata is created.
3.	The indexing task cannot retrieve the blob.
4.	Indexing fails with Blob not found.

Reproduction Steps

1- configure stalwart with blob storage set to filesystem or s3, leave all other storage to default rocksdb
2- use imapsync to sync email from server 1 to stalwart
3- IMAP append will be used by default to migrate email.
4- headers on rocksdb will be save, but body of each email will be empty. For some reason in a random way some, but very few emails get the body.

Note:
I made many corrections via imapsync to enforce some headers to be properly formated, omit message size etc etc.. but was not able to make IMAP append to really work under this config.

Relevant Log Output

INFO Message appended via IMAP (message-ingest.imap-append)
listenerId = "imaptls"
accountId = 141
documentId = 29
mailboxId = [7]
blobId = "214e213dc66be12eab834d505beaadc5f482babd626f97b857d1b1f400d08587"
size = 151295

DEBUG IMAP APPEND command (imap.append)
mailboxName = "Archive"

TRACE Raw IMAP output sent
"50 OK [APPENDUID 2211562292 29] APPEND completed"

TRACE Blob read operation (store.blob-read)
size = 0

DEBUG Record not found in database
details = "Blob not found"
collection = "email"
details = "Failed to build document for indexing"

WARN Task failed during processing
details = "Indexing task failed"

Stalwart Version

v0.15.x

Installation Method

Binary (Linux)

Database Backend

RocksDB

Blob Storage

Filesystem

Search Engine

Internal

Directory Backend

Internal

Additional Context

No response

I acknowledge that:

• I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.
• I have searched the Stalwart repository (both open and closed Discussions and Issues) and confirm this is not a duplicate.
• I have set the logging level to trace and included relevant log output if applicable.
• I agree to follow the project's Code of Conduct.

[mdecimus]

This looks like an issue with your blob store. Stalwart uses a two-phase commit mechanism to write blobs. The message won't be accepted if writing to the blob store fails (i.e. your blob store returns an error during insertion). Once it is inserted, Stalwart won't delete the blob until the purge task runs (usually at night) and there are no records referencing it. So, if you're seeing the Blob does not exist error, probably something external to Stalwart is deleting it.

[gdewey]

after weeks of doing test my only conclusion is that migrations that come from Axigen Mail server fail to be imported to Stalwart. The only way arround this was to migrate to Dovecot, then to Stalwart in order for mail to get into S3 blob storage type.

I made some more migrations from other, more used mailservers we had arround and imapsync and isync works with out an issue over stalwart. love the s3 storage implementation so far :-D

thanks for the reply and hope this helps if someone come across migrating from axigen mail server.

Guillermo