Unable to read S3 Tables due to incorrect URL scheme

[rchowell]

Apache Iceberg Rust version

None

Describe the bug

The opendal configuration for S3 Tables was setup with "s3a" as the default scheme, but S3 Tables file URIs use the "s3://" scheme. I was unable to read files and got the error:

 Error: DataInvalid => Invalid s3 url:
     s3://....metadata.json, should start with
     s3a://...

To Reproduce

Use the s3tables catalog then try to describe a table.

Expected behavior

I believe we should use "s3://" as the scheme for S3Tables.

Willingness to contribute

I can contribute a fix for this bug independently

[dannycjones]

I agree that the scheme is wrong here and the issue was introduced in iceberg-rust 0.9.

I've taken a quick look at the code and believe that it wouldn't be possible to load tables from S3 Tables catalog nor write out to those tables, so I don't believe there's a risk of breaking anyone on 0.9 who has been using this catalog the past 3 weeks.

• We can't read/load that table because I expect we'll fail to read the manifest with a s3:// URI from the service).
• We can't write out to those tables since we'll try to metadata layer files and data files with the wrong scheme.

I see that we previously used s3:// before this PR: https://github.com/apache/iceberg-rust/pull/2116/changes#diff-451c025a4aa08feaa36ea69026645beeb15ddbe8128f899b29f61c6ddfa93392.

Xander also notes that the Java implementation uses s3:// in S3FileIO: #2313 (comment)

Note, there's a proposal to add integration tests that would have caught this issue: #2211