Documentation updates for RAPIDS for Apache Spark -> NVIDIA cuDF plugin for Apache Spark rename

README.md

@@ -1,15 +1,19 @@
-# RAPIDS Accelerator For Apache Spark
+# NVIDIA cuDF for Apache Spark
 NOTE: For the latest stable [README.md](https://github.com/nvidia/spark-rapids/blob/main/README.md) ensure you are on the main branch.
 
-The RAPIDS Accelerator for Apache Spark provides a set of plugins for
-[Apache Spark](https://spark.apache.org) that leverage GPUs to accelerate processing 
-via the [RAPIDS](https://rapids.ai) libraries.
+NVIDIA cuDF for Apache Spark provides a set of plugins for
+[Apache Spark](https://spark.apache.org) that leverage GPUs to accelerate processing
+via the [cuDF](https://rapids.ai) (CUDA DataFrame) libraries.
 
 Documentation on the current release can be found [here](https://nvidia.github.io/cudf-spark/).
 
 To get started and try the plugin out use the [getting started guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/getting-started/overview.html).
 
-[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/spark-rapids)
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/cudf-spark)
+
+Note: cuDF for Spark was formerly known as the RAPIDS Accelerator for Apache Spark.  Going forward,
+the project name is cuDF for Spark.  The RAPIDS name will be sunset over time.  Github links from
+`spark-rapids` will redirect to `cudf-spark`.  Artifact names will remain the same for now.
 
 ## Compatibility
 
@@ -29,8 +33,8 @@ The plugin has a set of Spark configs that control its behavior and are document
 ## Issues & Questions
 
 We use github to track bugs, feature requests, and answer questions. File an
-[issue](https://github.com/NVIDIA/spark-rapids/issues/new/choose) for a bug or feature request. Ask
-or answer a question on the [discussion board](https://github.com/NVIDIA/spark-rapids/discussions).
+[issue](https://github.com/NVIDIA/cudf-spark/issues/new/choose) for a bug or feature request. Ask
+or answer a question on the [discussion board](https://github.com/NVIDIA/cudf-spark/discussions).
 
 ## Download
 
@@ -46,7 +50,7 @@ See the [build instructions in the contributing guide](CONTRIBUTING.md#building-
 Tests are described [here](tests/README.md).
 
 ## Integration
-The RAPIDS Accelerator For Apache Spark does provide some APIs for doing zero copy data
+cuDF for Apache Spark provides some APIs for doing zero copy data
 transfer into other GPU enabled applications.  It is described
 [here](https://docs.nvidia.com/spark-rapids/user-guide/latest/additional-functionality/ml-integration.html).
 
@@ -67,7 +71,7 @@ for more details on how to use the tools.
 
 ## Dependency for External Projects
 
-If you need to develop some functionality on top of RAPIDS Accelerator For Apache Spark (we currently
+If you need to develop some functionality on top of cuDF for Apache Spark (we currently
 limit support to GPU-accelerated UDFs) we recommend you declare our distribution artifact
 as a `provided` dependency.
 

docs/_config.yml

@@ -1,5 +1,5 @@
 #
-# Copyright (c) 2020, NVIDIA CORPORATION.
+# Copyright (c) 2020-2026, NVIDIA CORPORATION.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -18,10 +18,10 @@
 remote_theme: pmarsceill/just-the-docs
 
 aux_links:
-  "RAPIDS Accelerator for Apache Spark User Guide":
-    - "//docs.nvidia.com/spark-rapids/user-guide/latest/index.html"
-  "RAPIDS Accelerator for Apache Spark Plugin on Github":
-    - "//github.com/nvidia/spark-rapids"
+  "cuDF for Apache Spark User Guide":
+    - "//docs.nvidia.com/cudf-spark/user-guide/latest/index.html"
+  "cuDF for Apache Spark Plugin on Github":
+    - "//github.com/nvidia/cudf-spark"
 
 plugins:
   - jekyll-optional-front-matter # GitHub Pages

docs/additional-functionality/advanced_configs.md

@@ -11,14 +11,14 @@ nav_order: 10
  > variations
 
 
-# RAPIDS Accelerator for Apache Spark Advanced Configuration
+# cuDF for Apache Spark Advanced Configuration
 Most users will not need to modify the configuration options listed below.
 They are documented here for completeness and advanced usage.
 
-The following configuration options are supported by the RAPIDS Accelerator for Apache Spark.
+The following configuration options are supported by cuDF for Apache Spark.
 
 For commonly used configurations and examples of setting options, please refer to the
-[RAPIDS Accelerator for Configuration](../configs.md) page.
+[cuDF for Spark Configuration](../configs.md) page.
 
 
 ## Advanced Configuration
@@ -33,23 +33,24 @@ Name | Description | Default Value | Applicable at
 <a name="filecache.maxBytes"></a>spark.rapids.filecache.maxBytes|Controls the maximum amount of data that will be cached locally. If left unspecified, it will use half of the available disk space detected on startup for the configured Spark local disks.|None|Startup
 <a name="filecache.minimumFreeSpace.bytes"></a>spark.rapids.filecache.minimumFreeSpace.bytes|Specify the minimum amount of free space in the Spark local disks. When the amount of free space on the Spark local disks drops below this value, cache data will be removed automatically to free disk space. A zero or negative value will disable this feature. Note if multiple Spark applications running on the same node, or there are other applications running with heavy disk writing, the filecache may not drop caches in time and may cause full disk errors. Please increase this value for this case.|32212254720|Startup
 <a name="filecache.useChecksums"></a>spark.rapids.filecache.useChecksums|Whether to write out and verify checksums for the cached local files.|false|Startup
-<a name="flameGraph.asyncProfiler.options"></a>spark.rapids.flameGraph.asyncProfiler.options|Spark-RAPIDS plugin uses the async profiler to generate flame graphs. You can specify profiler options via this property. The plugin supports all options except for the 'file' option listed in https://github.com/async-profiler/async-profiler/blob/b3f58429f5c0252e9ced3f0fcb444fed17671321/docs/ProfilerOptions.md#options-applicable-to-any-output-format .The default values is 'jfr,event=cpu,wall=10ms'.|jfr,event=cpu,wall=10ms|Runtime
+<a name="flameGraph.asyncProfiler.options"></a>spark.rapids.flameGraph.asyncProfiler.options|cuDF for Spark uses the async profiler to generate flame graphs. You can specify profiler options via this property. The plugin supports all options except for the 'file' option listed in https://github.com/async-profiler/async-profiler/blob/b3f58429f5c0252e9ced3f0fcb444fed17671321/docs/ProfilerOptions.md#options-applicable-to-any-output-format .The default values is 'jfr,event=cpu,wall=10ms'.|jfr,event=cpu,wall=10ms|Runtime
 <a name="flameGraph.executors"></a>spark.rapids.flameGraph.executors|Comma-separated list of executors IDs and hyphenated ranges of executor IDs to profile when async-profiler (for flame graph) is enabled. The default value '*' means all executors|*|Runtime
 <a name="flameGraph.jfr.compression"></a>spark.rapids.flameGraph.jfr.compression|Enable compression for JFR files generated by async profiler. When enabled, JFR files will be compressed after generation to save disk space.|false|Runtime
 <a name="flameGraph.pathPrefix"></a>spark.rapids.flameGraph.pathPrefix|Enables collecting flame graph (with async profiler) and specifies a file prefix to use when writing the JFR file by async-profiler. The async-profiler will write a flame graph file for each stage. It is strongly recommended to set 'spark.scheduler.mode' to 'FIFO' so that there is a clean boundary between stages, and then we can better understand each stage.|None|Runtime
 <a name="flameGraph.stageEpochInterval"></a>spark.rapids.flameGraph.stageEpochInterval|Interval in seconds to determine the current stage epoch based on running task counts. The profiler will check which stage has the most running tasks and profile that stage during each epoch. This allows profiling when multiple stages run concurrently even if FIFO scheduling is already chosen.|5|Runtime
 <a name="gpu.resourceName"></a>spark.rapids.gpu.resourceName|The name of the Spark resource that represents a GPU that you want the plugin to use if using custom resources with Spark.|gpu|Startup
-<a name="iceberg.s3.async.connection-max-idle-time-ms"></a>spark.rapids.iceberg.s3.async.connection-max-idle-time-ms|Connection-max-idle-time (ms) for the AwsCrtAsyncHttpClient used by the spark-rapids Iceberg S3 byte-range reader. No equivalent Iceberg property.|300000|Startup
-<a name="iceberg.s3.async.max-concurrency"></a>spark.rapids.iceberg.s3.async.max-concurrency|Max concurrent connections for the AwsCrtAsyncHttpClient used by the spark-rapids Iceberg S3 byte-range reader. Used only when the Iceberg FileIO property `s3.crt.max-concurrency` is not set.|200|Startup
-<a name="iceberg.s3.async.tcp-keepalive-interval-ms"></a>spark.rapids.iceberg.s3.async.tcp-keepalive-interval-ms|TCP keep-alive probe interval (ms) for the AwsCrtAsyncHttpClient used by the spark-rapids Iceberg S3 byte-range reader. No equivalent Iceberg property.|60000|Startup
-<a name="iceberg.s3.async.tcp-keepalive-timeout-ms"></a>spark.rapids.iceberg.s3.async.tcp-keepalive-timeout-ms|TCP keep-alive probe timeout (ms) for the AwsCrtAsyncHttpClient used by the spark-rapids Iceberg S3 byte-range reader. No equivalent Iceberg property.|30000|Startup
+<a name="iceberg.s3.async.connection-max-idle-time-ms"></a>spark.rapids.iceberg.s3.async.connection-max-idle-time-ms|Connection-max-idle-time (ms) for the AwsCrtAsyncHttpClient used by the cuDF for Spark Iceberg S3 byte-range reader. No equivalent Iceberg property.|300000|Startup
+<a name="iceberg.s3.async.max-concurrency"></a>spark.rapids.iceberg.s3.async.max-concurrency|Max concurrent connections for the AwsCrtAsyncHttpClient used by the cuDF for Spark Iceberg S3 byte-range reader. Used only when the Iceberg FileIO property `s3.crt.max-concurrency` is not set.|200|Startup
+<a name="iceberg.s3.async.tcp-keepalive-interval-ms"></a>spark.rapids.iceberg.s3.async.tcp-keepalive-interval-ms|TCP keep-alive probe interval (ms) for the AwsCrtAsyncHttpClient used by the cuDF for Spark Iceberg S3 byte-range reader. No equivalent Iceberg property.|60000|Startup
+<a name="iceberg.s3.async.tcp-keepalive-timeout-ms"></a>spark.rapids.iceberg.s3.async.tcp-keepalive-timeout-ms|TCP keep-alive probe timeout (ms) for the AwsCrtAsyncHttpClient used by the cuDF for Spark Iceberg S3 byte-range reader. No equivalent Iceberg property.|30000|Startup
 <a name="memory.gpu.allocFraction"></a>spark.rapids.memory.gpu.allocFraction|The fraction of available (free) GPU memory that should be allocated for pooled memory. This must be less than or equal to the maximum limit configured via spark.rapids.memory.gpu.maxAllocFraction, and greater than or equal to the minimum limit configured via spark.rapids.memory.gpu.minAllocFraction.|1.0|Startup
 <a name="memory.gpu.debug"></a>spark.rapids.memory.gpu.debug|Provides a log of GPU memory allocations and frees. If set to STDOUT or STDERR the logging will go there. Setting it to NONE disables logging. All other values are reserved for possible future expansion and in the mean time will disable logging.|NONE|Startup
 <a name="memory.gpu.oomDumpDir"></a>spark.rapids.memory.gpu.oomDumpDir|The path to a local directory where a heap dump will be created if the GPU encounters an unrecoverable out-of-memory (OOM) error. The filename will be of the form: "gpu-oom-<pid>-<dumpId>.hprof" where <pid> is the process ID, and the dumpId is a sequence number to disambiguate multiple heap dumps per process lifecycle|None|Startup
 <a name="memory.gpu.pool"></a>spark.rapids.memory.gpu.pool|Select the RMM pooling allocator to use. Valid values are "DEFAULT", "ARENA", "ASYNC", and "NONE". With "DEFAULT", the RMM pool allocator is used; with "ARENA", the RMM arena allocator is used; with "ASYNC", the new CUDA stream-ordered memory allocator in CUDA 11.2+ is used. If set to "NONE", pooling is disabled and RMM just passes through to CUDA memory allocation directly.|ASYNC|Startup
 <a name="memory.gpu.reserve"></a>spark.rapids.memory.gpu.reserve|The amount of GPU memory that should remain unallocated by RMM and left for system use such as memory needed for kernels and kernel launches.|671088640|Startup
 <a name="memory.gpu.state.debug"></a>spark.rapids.memory.gpu.state.debug|To better recover from out of memory errors, RMM will track several states for the threads that interact with the GPU. This provides a log of those state transitions to aid in debugging it. STDOUT or STDERR will have the logging go there empty string will disable logging and anything else will be treated as a file to write the logs to.||Startup
 <a name="memory.gpu.unspill.enabled"></a>spark.rapids.memory.gpu.unspill.enabled|When a spilled GPU buffer is needed again, should it be unspilled, or only copied back into GPU memory temporarily. Unspilling may be useful for GPU buffers that are needed frequently, for example, broadcast variables; however, it may also increase GPU memory usage|false|Startup
+<a name="perfio.gcs.enabled"></a>spark.rapids.perfio.gcs.enabled|Controls the Google Cloud Storage reader for improved performance in certain queries. When true, enables it and throws at startup if google-cloud-storage classes are not on the classpath. When false, disables it unconditionally. When unset (default), enables it opportunistically if google-cloud-storage classes are found, otherwise falls back to the configured GCS connector with a warning. The presence of com.google.cloud:google-cloud-storage on the executor classpath is required.|None|Startup
 <a name="perfio.s3.enabled"></a>spark.rapids.perfio.s3.enabled|Controls the AWS S3 reader for improved performance in certain queries. When true, enables it and throws at startup if no compatible HTTP client is on the classpath. When false, disables it unconditionally. When unset (default), enables it opportunistically if a compatible HTTP client is found, otherwise falls back to S3A with a warning. The presence of AWS SDK packages for Netty and/or CRT HTTP clients on the classpath is required. You can use Spark submit option `--packages software.amazon.awssdk:s3:2.22.12,software.amazon.awssdk:aws-crt-client:2.22.12` to achieve this. See https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/crt-based-s3-client.html#crt-based-s3-client-depend|None|Startup
 <a name="python.concurrentPythonWorkers"></a>spark.rapids.python.concurrentPythonWorkers|Set the number of Python worker processes that can execute concurrently per GPU. Python worker processes may temporarily block when the number of concurrent Python worker processes started by the same executor exceeds this amount. Allowing too many concurrent tasks on the same GPU may lead to GPU out of memory errors. >0 means enabled, while <=0 means unlimited|0|Runtime
 <a name="python.memory.gpu.allocFraction"></a>spark.rapids.python.memory.gpu.allocFraction|The fraction of total GPU memory that should be initially allocated for pooled memory for all the Python workers. It supposes to be less than (1 - $(spark.rapids.memory.gpu.allocFraction)), since the executor will share the GPU with its owning Python workers. Half of the rest will be used if not specified|None|Runtime
@@ -150,7 +151,7 @@ Name | Description | Default Value | Applicable at
 <a name="sql.lore.idsToDump"></a>spark.rapids.sql.lore.idsToDump|Specify the LORE ids of operators to dump. The format is a comma separated list of LORE ids. For example: "1[0]" will dump partition 0 of input of gpu operator with lore id 1. For more details, please refer to [the LORE documentation](../dev/lore.md). If this is not set, no data will be dumped.|None|Runtime
 <a name="sql.lore.nonStrictMode.enabled"></a>spark.rapids.sql.lore.nonStrictMode.enabled|Allow LoRE dumping to continue when a selected lore id fails. When enabled, failing lore ids are skipped with a warning, previously dumped data under the dump path is kept, and the rest of the query continues executing.|false|Runtime
 <a name="sql.lore.parquet.useOriginalSchemaNames"></a>spark.rapids.sql.lore.parquet.useOriginalSchemaNames|When enabled, LORE writes Parquet files using the original Spark schema names instead of auto-generated type-based names. This makes the dumped Parquet data easier to consume directly via Spark/other tools.|true|Runtime
-<a name="sql.mode"></a>spark.rapids.sql.mode|Set the mode for the Rapids Accelerator. The supported modes are explainOnly and executeOnGPU. This config can not be changed at runtime, you must restart the application for it to take affect. The default mode is executeOnGPU, which means the RAPIDS Accelerator plugin convert the Spark operations and execute them on the GPU when possible. The explainOnly mode allows running queries on the CPU and the RAPIDS Accelerator will evaluate the queries as if it was going to run on the GPU. The explanations of what would have run on the GPU and why are output in log messages. When using explainOnly mode, the default explain output is ALL, this can be changed by setting spark.rapids.sql.explain. See that config for more details.|executeongpu|Startup
+<a name="sql.mode"></a>spark.rapids.sql.mode|Set the mode for cuDF for Spark. The supported modes are explainOnly and executeOnGPU. This config can not be changed at runtime, you must restart the application for it to take affect. The default mode is executeOnGPU, which means the cuDF for Spark plugin converts Spark operations and executes them on the GPU when possible. The explainOnly mode allows running queries on the CPU and cuDF for Spark will evaluate the queries as if it was going to run on the GPU. The explanations of what would have run on the GPU and why are output in log messages. When using explainOnly mode, the default explain output is ALL, this can be changed by setting spark.rapids.sql.explain. See that config for more details.|executeongpu|Startup
 <a name="sql.optimizer.joinReorder.enabled"></a>spark.rapids.sql.optimizer.joinReorder.enabled|When enabled, joins may be reordered for improved query performance|true|Runtime
 <a name="sql.python.gpu.enabled"></a>spark.rapids.sql.python.gpu.enabled|This is an experimental feature and is likely to change in the future. Enable (true) or disable (false) support for scheduling Python Pandas UDFs with GPU resources. When enabled, pandas UDFs are assumed to share the same GPU that the RAPIDs accelerator uses and will honor the python GPU configs|false|Runtime
 <a name="sql.reader.chunked"></a>spark.rapids.sql.reader.chunked|Enable a chunked reader where possible. A chunked reader allows reading highly compressed data that could not be read otherwise, but at the expense of more GPU memory, and in some cases more GPU computation. Currently this only supports ORC and Parquet formats.|true|Runtime
@@ -177,7 +178,7 @@ Name | Description | Default Value | Applicable at
 <a name="sql.window.range.short.enabled"></a>spark.rapids.sql.window.range.short.enabled|When the order-by column of a range based window is short type and the range boundary calculated for a value has overflow, CPU and GPU will get the different results. When set to false disables the range window acceleration for the short type order-by column|false|Runtime
 
 ## Supported GPU Operators and Fine Tuning
-_The RAPIDS Accelerator for Apache Spark_ can be configured to enable or disable specific
+cuDF for Apache Spark can be configured to enable or disable specific
 GPU accelerated expressions.  Enabled expressions are candidates for GPU execution. If the
 expression is configured as disabled, the accelerator plugin will not attempt replacement,
 and it will run on the CPU.