Commit b4167f2
committed
Add US-wide data processing pipeline scripts
- build_us_zarr.py: Download and process state-by-state forest data
- Checkpoint/resume capability for long-running jobs
- Progress tracking with rich UI
- Support for priority states (high forest coverage)
- Optimal chunking (512x512) for cloud streaming
- create_manifest.py: Generate state data manifest
- Indexes all available state Zarr stores
- Supports local and cloud URL paths
- Enables API discovery of state data
- upload_states_to_cloud.py: Upload states to cloud storage
- Supports both Cloudflare R2 and Backblaze B2
- Batch upload with progress tracking
- Updates manifest with public URLs1 parent d3f9116 commit b4167f2
3 files changed
Lines changed: 970 additions & 0 deletions
0 commit comments