Description
S3 endpoints contain duplicated patterns for download/upload operations that should be extracted into reusable utilities.
Current Duplication
Similar code patterns repeated across:
/translate/pptx_s3
/translate/text_s3
/transcribe/audio_s3
/translate/course_s3
Common Patterns
- Download files from S3 to local temp directory
- Process files with different tools
- Upload results back to S3
- Progress tracking and error handling
- Task status management
Proposed Utilities
async def process_s3_files(
input_keys: List[str],
processor_func: Callable,
output_prefix: Optional[str],
task_id: str,
**processor_kwargs
) -> List[str]:
"""Generic S3 file processing pipeline"""
async def download_s3_batch(keys: List[str], temp_dir: Path) -> List[Path]:
"""Download multiple files from S3"""
async def upload_s3_batch(files: List[Path], prefix: str) -> List[str]:
"""Upload multiple files to S3"""
Requirements
Benefits
- Eliminates code duplication
- Consistent error handling across endpoints
- Easier to add new S3 operations
- Better testability of S3 operations
- Simplified endpoint implementations
Acceptance Criteria
- Common S3 patterns extracted to utilities
- All S3 endpoints use shared functions
- No loss of functionality during refactor
- Progress tracking preserved
- Error handling consistent
Related
Priority
Medium - Code quality and maintainability
Description
S3 endpoints contain duplicated patterns for download/upload operations that should be extracted into reusable utilities.
Current Duplication
Similar code patterns repeated across:
/translate/pptx_s3/translate/text_s3/transcribe/audio_s3/translate/course_s3Common Patterns
Proposed Utilities
Requirements
Benefits
Acceptance Criteria
Related
Priority
Medium - Code quality and maintainability