Skip to content

Multipart upload crashes persist on 2.53.3 (thread-safety in StorageMultipartUploadSession) #4138

Description

@guillaumeramey

Describe the bug

We are experiencing multiple crashes in AWSS3StoragePlugin's multipart upload state machine on version 2.53.3. These appear related to #3991 which was fixed, but the crashes persist across different code paths in StorageMultipartUploadSession.

The crashes affect approximately 5-10% of our users uploading video files.

Stack Traces

Crash 1 — retryPartUpload (assertion failure "Invalid state")

Crashed: com.apple.NSURLSession-delegate
0  libswiftCore.dylib             _assertionFailure(_:_:file:line:flags:)
1  Rematch                        StorageMultipartUploadSession.retryPartUpload(partNumber:error:) + 350 (StorageMultipartUploadSession.swift:350)
2  Rematch                        StorageMultipartUploadSession.handle(uploadPartEvent:) + 327 (StorageMultipartUploadSession.swift:327)
3  Rematch                        StorageServiceSessionDelegate.urlSession(_:task:didCompleteWithError:) + 128 (StorageServiceSessionDelegate.swift:128)

crash_info_entry_0: AWSS3StoragePlugin/StorageMultipartUploadSession.swift:350: Fatal error: Invalid state

Crash 2 — swift_retain in StorageMultipartUpload.uploadId getter

Crashed: com.apple.root.user-initiated-qos.cooperative
0  libswiftCore.dylib             swift_retain
1  libswiftCore.dylib             swift_cvw_initWithCopyImpl
2  Rematch                        initializeWithCopy for StorageMultipartUpload + 220
3  Rematch                        StorageMultipartUpload.uploadId.getter + 45 (StorageMultipartUpload.swift:45)
4  Rematch                        StorageMultipartUploadSession.fail(error:) + 313 (StorageMultipartUpload.swift:313)
5  Rematch                        closure #1 in closure #2 in DefaultStorageMultipartUploadClient.uploadPart(partNumber:multipartUpload:subTask:) + 167 (StorageMultipartUploadClient.swift:167)

Crash 3 — assertion failure in FileSystem.getFileSize

Crashed: com.apple.root.default-qos.cooperative
0  libswiftCore.dylib             _assertionFailure(_:_:file:line:flags:)
1  Rematch                        Fatal.require(_:file:line:) + 81 (Fatal.swift:81)
2  Rematch                        FileSystem.getFileSize(fileURL:) + 171 (FileSystem.swift:171)
3  Rematch                        UploadSource.getFile(fileSystem:) + 21 (UploadSource.swift:21)
4  Rematch                        closure #2 in AWSS3StorageService.upload(serviceKey:uploadSource:contentType:metadata:accelerate:onEvent:) + 14 (UploadSource.swift:14)

Analysis

These crashes occur on different threads (NSURLSession-delegate, user-initiated-qos.cooperative, default-qos.cooperative), suggesting the multipart upload state machine is being accessed concurrently without proper synchronization. Crash 2 (swift_retain on uploadId getter) is a strong indicator of a data race.

Expected behavior

The multipart upload state machine should handle concurrent access safely, throwing recoverable errors instead of crashing via assertions or memory corruption.

Amplify Framework Version

2.53.3

Amplify Categories

Storage

Dependency manager

Swift PM

Swift version

5

Xcode version

16

Relevant log output

AWSS3StoragePlugin/StorageMultipartUploadSession.swift:350: Fatal error: Invalid state

Is this a regression?

Related to #3991 which was fixed, but these crashes persist on the latest version (2.53.3) through different code paths in the same state machine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstorageIssues related to the Storage category

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions