Skip to content

[KYUUBI #7251] Use RustFS instead of MinIO in Playground #7252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
aajisaka wants to merge 2 commits into from
Closed

[KYUUBI #7251] Use RustFS instead of MinIO in Playground #7252

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e5f1899
[KYUUBI #7251] Use RustFS instead of MinIO in Playground
aajisaka Nov 19, 2025
9dca056
Add credentials info for RustFS Web UI
aajisaka Nov 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker/playground/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Kyuubi supply some built-in dataset, after Kyuubi started, you can run the follo

### Access Service

- MinIO: http://localhost:9001
- RustFS: http://localhost:9001

@pan3793 pan3793 Nov 20, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since it requires authN, let's leave the credentials info here

@aajisaka aajisaka Nov 20, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I'll update

@aajisaka aajisaka Nov 20, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. Thank you!

- PostgreSQL localhost:5432 (username: postgres, password: postgres)
- Spark UI: http://localhost:4040 (available after Spark application launching by Kyuubi, port may be 4041, 4042... if you launch more than one Spark applications)
- Prometheus: http://localhost:9090
Expand Down
34 changes: 26 additions & 8 deletions docker/playground/compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,17 +16,35 @@
#

services:
minio:
image: bitnami/minio:2023-debian-11
rustfs:
image: rustfs/rustfs:latest
environment:
MINIO_ROOT_USER: minio
MINIO_ROOT_PASSWORD: minio_minio
MINIO_DEFAULT_BUCKETS: spark-bucket
container_name: minio
hostname: minio
RUSTFS_ACCESS_KEY: rustfsadmin
RUSTFS_SECRET_KEY: rustfsadmin
RUSTFS_ADDRESS: 0.0.0.0:9000
RUSTFS_CONSOLE_ACCESS: 0.0.0.0:9001
hostname: rustfs
ports:
- 9000
- 9001:9001
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 20s

rustfs-init:
image: amazon/aws-cli:latest
depends_on:
rustfs:
condition: service_healthy
environment:
AWS_ACCESS_KEY_ID: rustfsadmin
AWS_SECRET_ACCESS_KEY: rustfsadmin
AWS_REGION: us-east-1
entrypoint: /bin/sh
command: -c "aws --endpoint-url http://rustfs:9000 s3api create-bucket --bucket spark-bucket"

postgres:
image: postgres:${POSTGRES_VERSION}
Expand Down Expand Up @@ -79,7 +97,7 @@ services:
- ./script/load-dataset-tpch-tiny.sql:/opt/load_data/load-dataset-tpch-tiny.sql
depends_on:
- metastore
- minio
- rustfs
- zookeeper

prometheus:
Expand Down
6 changes: 3 additions & 3 deletions docker/playground/conf/core-site.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@
<configuration>
<property>
<name>fs.s3a.access.key</name>
<value>minio</value>
<value>rustfsadmin</value>
</property>
<property>
<name>fs.s3a.secret.key</name>
<value>minio_minio</value>
<value>rustfsadmin</value>
</property>
<property>
<name>fs.s3a.connection.ssl.enabled</name>
Expand All @@ -35,7 +35,7 @@
</property>
<property>
<name>fs.s3a.endpoint</name>
<value>http://minio:9000</value>
<value>http://rustfs:9000</value>
</property>
<property>
<name>fs.s3a.committer.name</name>
Expand Down
Loading