Skip to content

Commit f85c6ad

Browse files
author
Sameer Raheja
committed
Updated based on nvliyuan's feedback
Signed-off-by: Sameer Raheja <sraheja@.nvidia.com>
1 parent b6676f1 commit f85c6ad

6 files changed

Lines changed: 15 additions & 16 deletions

File tree

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# NVIDIA cuDF plugin for Apache Spark
22
NOTE: For the latest stable [README.md](https://github.com/nvidia/cudf-spark/blob/main/README.md) ensure you are on the main branch.
33

4-
The NVIDIA cuDF plugin for Apache Spark provides a plugin library [Apache
5-
Spark](https://spark.apache.org) that leverages GPUs to accelerate processing via the
6-
[cuDF](https://rapids.ai) (CUDA DataFrame) libraries.
4+
The NVIDIA cuDF plugin for [Apache Spark](https://spark.apache.org) provides a plugin library that
5+
leverages GPUs to accelerate processing via the [cuDF](https://github.com/rapidsai/cudf) (CUDA
6+
DataFrame) libraries.
77

88
Documentation on the current release can be found [here](https://nvidia.github.io/cudf-spark/).
99

docs/additional-functionality/cache-serializer.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
layout: page
3-
title: NVIDIA cuDF pluign for Apache Spark Cache Serializer
3+
title: NVIDIA cuDF plugin for Apache Spark Cache Serializer
44
parent: Additional Functionality
55
nav_order: 2
66
---

docs/additional-functionality/iceberg-per-table-scan-options.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ wants to tune scan behavior across an existing Spark session — for example
1515
before a benchmark run, or to try different split sizes against existing tables
1616
without modifying them.
1717

18-
The NVIDIA cuDF plugin for Apachde Spark ships a thin Iceberg session-catalog wrapper that lets
18+
The NVIDIA cuDF plugin for Apache Spark ships a thin Iceberg session-catalog wrapper that lets
1919
these options be set via session conf at three scopes — per-table, per-catalog,
2020
and global. The wrapper is opt-in and a pure pass-through for any table whose
2121
conf is not set at any scope, so enabling it does not change behavior for

docs/dev/README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ nav_order: 11
55
has_children: true
66
permalink: /developer-overview/
77
---
8-
# cuDF for Apache Spark Developer Overview
8+
# NVIDIA cuDF plugin for Apache Spark Developer Overview
99
This document provides a developer overview of the project and covers the
1010
following topics:
1111
* [Spark SQL and Query Plans](#spark-sql-and-query-plans)
@@ -79,7 +79,7 @@ turn will need to pull values from the input nodes, chaining all the way down
7979
the tree until eventually the iterator of the leaf nodes is pulled and causes
8080
the reading of rows from the raw input data.
8181

82-
## How the cuDF for Spark Plugin Works
82+
## How the cuDF Plugin Works
8383
The plugin leverages two main features in Spark. The first is a
8484
plugin interface in [Catalyst](https://databricks.com/glossary/catalyst-optimizer)
8585
that allows the optimizer to be extended. The plugin is a Catalyst extension
@@ -90,7 +90,7 @@ which allows extensions to operate on Spark SQL data in a `ColumnarBatch` form.
9090
Processing columnar data is much more GPU friendly than row-by-row processing.
9191

9292
For example, the same query plan shown above becomes the following plan after
93-
being processed by the cuDF for Spark plugin:
93+
being processed by the cuDF plugin:
9494
```
9595
*(5) Sort [o_orderpriority#5 ASC NULLS FIRST], true, 0
9696
+- Exchange rangepartitioning(o_orderpriority#5 ASC NULLS FIRST, 200), true, [id=#611]
@@ -176,7 +176,7 @@ scala> import com.nvidia.spark.rapids.RapidsConf
176176
import com.nvidia.spark.rapids.RapidsConf
177177
178178
scala> RapidsConf.help(true)
179-
# cuDF for Spark Plugin Configuration
179+
# cuDF Plugin Configuration
180180
The following is the list of options that `rapids-plugin-4-spark` supports.
181181
182182
On startup use: `--conf [conf key]=[conf value]`. For example:

docs/dev/shims.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,11 @@ parent: Developer Overview
77

88
# Shim Development
99

10-
cuDF for Apache Spark supports multiple feature version lines of
11-
Apache Spark such as 3.3.x, 3.4.x, 3.5.x, 4.x and a number of vendor releases that contain
12-
a mix of patches from different upstream releases. These artifacts are generally
13-
incompatible between each other, at both source code level and even more often
14-
at the binary level. The role of the Shim layer is to hide these issues from the
15-
common code, maximize reuse, and minimize logic duplication.
10+
The NVIDIA cuDF plugin for Apache Spark supports multiple feature version lines of Apache Spark such
11+
as 3.3.x, 3.4.x, 3.5.x, 4.x and a number of vendor releases that contain a mix of patches from
12+
different upstream releases. These artifacts are generally incompatible between each other, at both
13+
source code level and even more often at the binary level. The role of the Shim layer is to hide
14+
these issues from the common code, maximize reuse, and minimize logic duplication.
1615

1716
This is achieved by using a ServiceProvider pattern. All Shims implement the same API,
1817
the suitable Shim implementation is loaded after detecting the current Spark build version

docs/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ permalink: /
66
description: This site serves as a collection of documentation about the NVIDIA cuDF plugin for Apache Spark
77
---
88
# Overview
9-
**If you are a customer looking for information on how to adopt cuDF for Apache Spark
9+
**If you are a customer looking for information on how to adopt the cuDF plugin
1010
for your Spark workloads, please go to our User Guide for more information: [link](https://docs.nvidia.com/spark-rapids/user-guide/latest/index.html).**
1111

1212
The cuDF plugin leverages GPUs to accelerate processing via the

0 commit comments

Comments
 (0)