How to restore data backups of Ammor data lakes on object storage of different cloud vendors?

[FourSpaces]

Scenario：
We use Ammor to manage an Iceberg data lake, and we have made a synchronous backup of the Iceberg data on S3 of other cloud vendors. How can we quickly switch to different vendor's object storage S3 to ensure that the data lake can provide services normally on different cloud vendor's object storage?

Note:
1、We will synchronize data on the object storage.
2、We may switch between different cloud vendors multiple times.

[czy006]

@FourSpaces Maybe alluxio can help u ?

[FourSpaces]

We can test it out

[baiyangtx]

The point of this issue is the metadata organization of Iceberg.
Iceberg uses manifest files to index all data files. If the files are located in different cloud S3 locations, the file paths may not be the same. This results in the table being unreadable when the data is backed up to another cloud.

[huyuanfeng2018]

Maybe you can consider extending the FileIo interface in iceberg and replacing the path according to the rules to obtain the read and write io of different object storage.