Skip to content

Commit c91cad8

Browse files
committed
Prepare v1.4.3 release
1 parent 0ac096e commit c91cad8

1 file changed

Lines changed: 25 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [1.4.3] - 2025-11-07
11+
12+
### Fixed
13+
- **Multipart Upload Caching:**
14+
- Fixed issue where multipart uploads (files >100MB) would always be re-processed even when cached metadata was available
15+
- Added proper multipart ETag caching and comparison to avoid re-extracting and re-uploading files that haven't changed
16+
- Previously, multipart uploads would bypass cache checks because S3 ETag format for multipart uploads differs from simple MD5 hashes
17+
18+
### Technical Details
19+
- Added `MultipartETag` field to `PartitionCacheEntry` struct
20+
- Created `calculateMultipartETagFromFile()` to compute S3 multipart ETag from temp files using the same 5MB part size as s3manager
21+
- Updated cache storage to calculate and save multipart ETag after upload
22+
- Updated cache retrieval to return multipart ETag along with MD5
23+
- Modified `shouldSkipPartition` logic to compare multipart ETags when appropriate
24+
- S3 multipart ETag format: "MD5_of_MD5s-partCount" (e.g., "abc123-56")
25+
- Part size: 5MB (matches s3manager.NewUploader default)
26+
- Multipart threshold: 100MB (matches uploadTempFileToS3 logic)
27+
- Backward compatible: Old cache entries without multipartETag still work
28+
29+
### Impact
30+
- Eliminates unnecessary re-extraction and re-upload of large files
31+
- Reduces database load from redundant queries
32+
- Improves performance for incremental archival runs
33+
- Maintains data integrity through proper ETag validation
34+
1035
## [1.4.2] - 2025-11-07
1136

1237
### Added

0 commit comments

Comments
 (0)