Skip to content

Upload to R2 (but keep S3 as the source-of-truth)#474

Open
fredrikekre wants to merge 1 commit intomainfrom
fe/r2
Open

Upload to R2 (but keep S3 as the source-of-truth)#474
fredrikekre wants to merge 1 commit intomainfrom
fe/r2

Conversation

@fredrikekre
Copy link
Copy Markdown
Member

No description provided.

@fredrikekre fredrikekre force-pushed the fe/r2 branch 4 times, most recently from 5ddf901 to 6508408 Compare August 14, 2025 14:13
@fredrikekre fredrikekre marked this pull request as ready for review August 14, 2025 14:13
@DilumAluthge
Copy link
Copy Markdown
Member

Just to clarify, for the time being, we're going to upload to both S3 and R2, right?

@fredrikekre
Copy link
Copy Markdown
Member Author

Yes, and I think we will keep S3 no matter what as the "source of truth" and use R2 as a caching layer in front. R2 has functionality to automatically cache an S3 bucket, but once a resource is cached it won't be updated. This is why we have to explicitly upload to R2 from here too since for -latest uploads and such we overwrite existing files.

@DilumAluthge
Copy link
Copy Markdown
Member

Yes, and I think we will keep S3 no matter what as the "source of truth" and use R2 as a caching layer in front.

That sounds good to me.

@fredrikekre
Copy link
Copy Markdown
Member Author

I need to update the aws_uploader rootfs, it has a very old aws cli version.

@DilumAluthge DilumAluthge changed the title Upload to R2 Upload to R2 (but keep S3 as the source-of-truth) Mar 10, 2026
This patch adds the `julialangnightlies` bucket on Cloudflare R2 as an
additional upload target mirroring the layout of the
`julialangnightlies` bucket on AWS S3.
@DilumAluthge
Copy link
Copy Markdown
Member

@fredrikekre Should we move forward with this?

@fredrikekre
Copy link
Copy Markdown
Member Author

Yea, I will get back to this sooner rather than later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants