Merge pull request #526 from linagora/feat/ray-external-cluster-address

feat(api): connect to an external Ray cluster via RAY_ADDRESS

Author: EnjoyBacon7

.env.example

@@ -74,6 +74,14 @@ RAY_task_retry_delay_ms=3000
 RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 # critical with the newest version of UV
 # # To disable worker killing
 # RAY_memory_monitor_refresh_ms=0
+# Connect to an external Ray cluster instead of starting an embedded one.
+# When set, the app attaches to this cluster and does NOT start a local dashboard
+# (the head node owns it). See docs/documentation/deploy_ray_cluster.
+# RAY_ADDRESS=ray://X.X.X.X:10001
+# Interface the embedded Ray dashboard binds to. Defaults to 127.0.0.1 (loopback)
+# because the dashboard/job API is unauthenticated (CVE-2023-48022). Set to 0.0.0.0
+# only when the port is firewalled or behind an auth proxy. Ignored when RAY_ADDRESS is set.
+# RAY_DASHBOARD_HOST=127.0.0.1
 
 # Indexer UI 
 ## 1. replace X.X.X.X with localhost if launching local or with your server IP

docs/assets/env_example.env

@@ -43,6 +43,11 @@ RAY_DEDUP_LOGS=0 # turns off ray log deduplication that appear across multiple p
 RAY_ENABLE_RECORD_ACTOR_TASK_LOGGING=1 # # to enable logs at task level in ray dashboard
 RAY_task_retry_delay_ms=3000
 RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 # critical with the newest version of UV
+# Attach to an external Ray cluster instead of starting an embedded one (disables the local dashboard).
+# RAY_ADDRESS=ray://X.X.X.X:10001
+# Interface the embedded Ray dashboard binds to. Defaults to 127.0.0.1 (loopback) because the
+# dashboard/job API is unauthenticated (CVE-2023-48022). Set 0.0.0.0 only behind a firewall/auth proxy.
+# RAY_DASHBOARD_HOST=127.0.0.1
 
 # Indexer UI 
 ## 1. replace X.X.X.X with localhost if launching local or with your server IP

docs/assets/env_linux_gpu.env

@@ -39,6 +39,11 @@ RAY_DEDUP_LOGS=0 # turns off ray log deduplication that appear across multiple p
 RAY_ENABLE_RECORD_ACTOR_TASK_LOGGING=1 # # to enable logs at task level in ray dashboard
 RAY_task_retry_delay_ms=3000
 RAY_ENABLE_UV_RUN_RUNTIME_ENV=0 # critical with the newest version of UV
+# Attach to an external Ray cluster instead of starting an embedded one (disables the local dashboard).
+# RAY_ADDRESS=ray://X.X.X.X:10001
+# Interface the embedded Ray dashboard binds to. Defaults to 127.0.0.1 (loopback) because the
+# dashboard/job API is unauthenticated (CVE-2023-48022). Set 0.0.0.0 only behind a firewall/auth proxy.
+# RAY_DASHBOARD_HOST=127.0.0.1
 
 # Indexer UI 
 ## 1. replace X.X.X.X with localhost if launching local or with your server IP

docs/content/docs/documentation/deploy_ray_cluster.md

@@ -136,6 +136,10 @@ docker compose up -d
 
 Once running, **OpenRAG will auto-connect** to the Ray cluster using `RAY_ADDRESS` from `.env`.
 
+:::note
+When `RAY_ADDRESS` is set, the app **attaches** to the external cluster and does **not** start its own embedded Ray dashboard — the head node owns it (started above via `--dashboard-host 0.0.0.0 --dashboard-port ${RAY_DASHBOARD_PORT:-8265}`). The app-side `RAY_DASHBOARD_HOST` setting is only used in embedded (single-node) mode, where it defaults to `127.0.0.1` because the dashboard API is unauthenticated ([CVE-2023-48022](https://nvd.nist.gov/vuln/detail/CVE-2023-48022)).
+:::
+
 ---
 
 With this setup, your app is now fully distributed and ready to handle concurrent tasks across your Ray cluster.

docs/content/docs/documentation/env_vars.md

@@ -350,6 +350,8 @@ Ray is used for distributed task processing and parallel execution in the RAG pi
 | `RAY_POOL_SIZE` | `int` | 1 | Number of serializer actor instances (typically 1 actor per cluster node) |
 | `RAY_MAX_TASKS_PER_WORKER` | `int` | 8 | Maximum number of concurrent tasks (serialization tasks) per serializer actor instance |
 | `RAY_DASHBOARD_PORT` | `int` | 8265 | Ray Dashboard port used for monitoring. In production, [comment out this line](https://github.com/linagora/openrag/blob/ee732ea8e080dcde0107d62d12703a7525f810cd/docker-compose.yaml#L21C1-L22C1) to avoid exposing the port, as it may introduce security vulnerabilities. |
+| `RAY_DASHBOARD_HOST` | `str` | `127.0.0.1` | Interface the **embedded** Ray dashboard binds to. Defaults to loopback because the Ray dashboard/job-submission API is **unauthenticated** ([CVE-2023-48022](https://nvd.nist.gov/vuln/detail/CVE-2023-48022)). Set to `0.0.0.0` only when the dashboard port is firewalled or sits behind an authenticating proxy. Ignored when `RAY_ADDRESS` is set. |
+| `RAY_ADDRESS` | `str` | (unset) | When set, attach to an **external** Ray cluster at this address (e.g. `ray://HEAD_IP:10001`) instead of starting an embedded cluster in-process. In this mode the app does not start a local dashboard — the head node owns it. See [Ray Cluster deployment](/openrag/documentation/deploy_ray_cluster/). |
 
 :::danger[Attention]
 The following environment variables control Ray's logging behavior, task retry settings. These are not set by default and must be supplied [as suggested in the .env](/openrag/getting_started/quickstart#2-create-a-env-file)

openrag/api.py

@@ -15,9 +15,15 @@
 from fastapi.openapi.utils import get_openapi
 from fastapi.responses import JSONResponse, RedirectResponse
 
-# Bind the Ray dashboard to localhost; it's unauthenticated (CVE-2023-48022).
-# Override with RAY_DASHBOARD_HOST (e.g. behind an auth proxy).
-ray.init(dashboard_host=os.environ.get("RAY_DASHBOARD_HOST", "127.0.0.1"))
+_ray_address = os.environ.get("RAY_ADDRESS")
+if _ray_address:
+    # Connect to an external Ray cluster (e.g. a dedicated ray-head container).
+    ray.init(address=_ray_address)
+else:
+    # Embedded mode: start a local Ray cluster inside this process.
+    # Bind the Ray dashboard to localhost; it's unauthenticated (CVE-2023-48022).
+    # Override with RAY_DASHBOARD_HOST (e.g. behind an auth proxy).
+    ray.init(dashboard_host=os.environ.get("RAY_DASHBOARD_HOST", "127.0.0.1"))
 
 # Apply noqa: E402 to ignore "module level import not at top of file" cause ray.init has to be called first