Skip to content

Commit 4324a25

Browse files
authored
Deduplicate Java/Scala template projects in skills [skip ci] (#15111)
Fixes #15094. ### Description This removes the Java template project provided with the skills, making the Scala project the ground truth for any input JVM UDF. The instructions are adjusted accordingly. Jacoco and Hive support is moved over to the Scala templates. ### Checklists Documentation - [X] Updated for new or modified user-facing features or behaviors - [ ] No user-facing change Testing - [ ] Added or modified tests to cover new code paths - [ ] Covered by existing tests (Please provide the names of the existing tests in the PR description.) - [X] Not required Performance - [ ] Tests ran and results are added in the PR description - [ ] Issue filed with a link in the PR description - [X] Not required --------- Signed-off-by: Rishi Chandra <rishic@nvidia.com>
1 parent 66a33f3 commit 4324a25

29 files changed

Lines changed: 74 additions & 1797 deletions

File tree

skills/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ Once you've installed the skills, try the workflow with one of the provided exam
110110
Results
111111

112112
- Project directory: FormatPhoneUDF/
113-
- Unit test: FormatPhoneUDF/src/test/java/com/udf/UnitTest.java
113+
- Unit test: FormatPhoneUDF/src/test/scala/com/udf/UnitTest.scala
114114
- Status: All tests passing
115115
- Test coverage: 12 cases
116116
```

skills/docs/dev/VERSIONS.md

Lines changed: 1 addition & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,7 @@
22

33
## Files To Update
44

5-
### Java udf-gen-test Maven template
6-
7-
File: `skills/udf-gen-test/templates/java/pom.xml`
8-
9-
Update these properties together:
10-
11-
- `<scala.binary.version>`
12-
- `<spark.version>`
13-
- `<rapids4spark.version>`
14-
- `<cuda.version>` if the RAPIDS artifact classifier changes
15-
- `<cudf.git.branch>`
16-
- `<rapids.cmake.branch>`
17-
18-
### Scala udf-gen-test Maven template
5+
### udf-gen-test Maven template
196

207
File: `skills/udf-gen-test/templates/scala/pom.xml`
218

skills/udf-benchmark/SKILL.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Derive `<CamelName>` and `<snake_name>` from the UDF class name.
2828
2929
## Step 1: Implement BenchUtils
3030

31-
Read `src/main/<java|scala>/com/udf/bench/BenchUtils.<java|scala>`. Replace placeholders with the actual camel/snake UDF name.
31+
Read `src/main/scala/com/udf/bench/BenchUtils.scala`. Replace placeholders with the actual camel/snake UDF name.
3232

3333
Fill in the TODO methods following the docstrings. For variable-length inputs, generate sizable rows representative of enterprise-scale data. Refer to the unit test for schema and example data.
3434

@@ -76,7 +76,7 @@ Follow [CUDF_MICROBENCHMARKS.md](CUDF_MICROBENCHMARKS.md) to implement and run i
7676
## Output
7777

7878
Upon successful completion:
79-
- Benchmark utilities: `src/main/<java|scala>/com/udf/bench/BenchUtils.<java|scala>`
80-
- Microbenchmarks (cuDF): `src/main/<java|scala>/com/udf/bench/MicroBenchRunner.<java|scala>`
79+
- Benchmark utilities: `src/main/scala/com/udf/bench/BenchUtils.scala`
80+
- Microbenchmarks (cuDF): `src/main/scala/com/udf/bench/MicroBenchRunner.scala`
8181
- Generated data: `data/`
8282
- Benchmark results: `results/`

skills/udf-convert-to-cuda/SKILL.md

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ Use `src/main/java/com/udf/PlaceholderUDFNameNativeRapidsUDF.java` as a starting
5454
4. Implement `evaluateColumnar` to validate column count/types and call the native method.
5555
5. Rename the native method to a descriptive operation name, e.g. `cosineSimilarityNative`.
5656

57-
For Scala projects, keep this Java wrapper under `src/main/java/com/udf/` and register it from the Scala test/project. JNI can be used from Scala, but the Java wrapper keeps native symbol names and examples simpler.
57+
The project joint-compiles Java, so keep this Java wrapper under `src/main/java/com/udf/` and register it from the Scala test. JNI can be used from Scala, but the Java wrapper keeps native symbol names and examples simpler.
5858
If the Java wrapper's CPU fallback needs to call a Scala object, direct references can fail before `scala-maven-plugin` compiles the Scala classes; use reflection in the row-by-row fallback only, and keep `evaluateColumnar` on the normal JNI path.
5959

6060
Read [JNI_CUDA_GUIDE.md](references/JNI_CUDA_GUIDE.md) for the `evaluateColumnar` contract, type mapping, pointer ownership, `NativeDepsLoader`, and native memory rules.
@@ -104,16 +104,12 @@ If the build fails while resolving cuDF headers or RAPIDS CMake, check network a
104104

105105
## Step 5: Build and Test
106106

107-
Fill in the target-specific TODOs in `src/test/<java|scala>/com/udf/CudfComparisonTest.<java|scala>`:
107+
Fill in the target-specific TODOs in `src/test/scala/com/udf/CudfComparisonTest.scala`:
108108
- Register `<CamelName>NativeRapidsUDF` as the GPU implementation
109109
- Replace placeholder UDF names
110110

111111
Run:
112112
```bash
113-
# Java
114-
mvn test -Dtest=CudfComparisonTest -Pcuda-native-udf
115-
116-
# Scala project using a Java native RapidsUDF wrapper
117113
mvn test -Dsuites=com.udf.CudfComparisonTest -Pcuda-native-udf
118114
```
119115

@@ -129,7 +125,7 @@ docker run --rm --gpus all \
129125
-v /etc/group:/etc/group:ro \
130126
-w /workspace \
131127
cuda-udf-build \
132-
-c "mvn -B -Dmaven.repo.local=/workspace/.m2/repository test -Dtest=CudfComparisonTest -Pcuda-native-udf -Dnative.build.path=/workspace/target/native-build-docker -DskipCudfExtraction=true"
128+
-c "mvn -B -Dmaven.repo.local=/workspace/.m2/repository test -Dsuites=com.udf.CudfComparisonTest -Pcuda-native-udf -Dnative.build.path=/workspace/target/native-build-docker -DskipCudfExtraction=true"
133129
```
134130

135131
If tests fail, iterate on the Java bridge or native implementation.

skills/udf-convert-to-cudf/SKILL.md

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -64,16 +64,12 @@ Create a copy of the original UDF file in the same source directory (`src/main/<
6464

6565
## Step 3: Build and Test
6666

67-
Fill in the target-specific TODOs in `src/test/<java|scala>/com/udf/CudfComparisonTest.<java|scala>`:
67+
Fill in the target-specific TODOs in `src/test/scala/com/udf/CudfComparisonTest.scala`:
6868
- Implement `registerRapidsUDF` to register the new RapidsUDF class.
6969
- Replace placeholders with the actual camel/snake UDF name
7070

7171
Then run the test:
7272
```bash
73-
# Java
74-
mvn test -Dtest=CudfComparisonTest
75-
76-
# Scala
7773
mvn test -Dsuites=com.udf.CudfComparisonTest
7874
```
7975

@@ -91,10 +87,6 @@ Treat the unit test as the CPU behavior specification. Do not weaken or remove t
9187

9288
Re-run with memory leak detection:
9389
```bash
94-
# Java
95-
mvn test -Dtest=CudfComparisonTest -Ddebug.memory.leaks=true > /tmp/memleak.log 2>&1
96-
97-
# Scala
9890
mvn test -Dsuites=com.udf.CudfComparisonTest -Ddebug.memory.leaks=true > /tmp/memleak.log 2>&1
9991

10092
# Check for leaks

skills/udf-convert-to-sql/SKILL.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -41,16 +41,12 @@ Implement the SQL expression in a file at `src/main/resources/<snake_name>.sql`.
4141

4242
## Step 2: Fill in test and iterate
4343

44-
Update `src/test/<java|scala>/com/udf/SqlComparisonTest.<java|scala>`:
44+
Update `src/test/scala/com/udf/SqlComparisonTest.scala`:
4545
- Update the SQL file path to point to your `src/main/resources/<snake_name>.sql` file
4646
- Replace placeholders with the actual camel/snake UDF name
4747

4848
Then run the test:
4949
```bash
50-
# Java
51-
mvn test -Dtest=SqlComparisonTest
52-
53-
# Scala
5450
mvn test -Dsuites=com.udf.SqlComparisonTest
5551
```
5652

skills/udf-gen-test/SKILL.md

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Derive `<CamelName>` and `<snake_name>` from the UDF class name.
3333

3434
The project can be found under this skill's templates directory.
3535
```bash
36-
cp -r templates/<java|scala> <project_root>/<CamelName>/
36+
cp -r templates/scala <project_root>/<CamelName>/
3737
```
3838

3939
This provides a complete Maven project with all test and benchmark infrastructure.
@@ -56,7 +56,7 @@ Set the package declaration to `com.udf`:
5656

5757
## Step 2: Implement the Unit Test
5858

59-
Read `src/test/<java|scala>/com/udf/UnitTest.<java|scala>`. Replace placeholders with the actual camel/snake UDF name.
59+
Read `src/test/scala/com/udf/UnitTest.scala`. Replace placeholders with the actual camel/snake UDF name.
6060

6161
Fill in the TODO methods following the docstrings. Include diverse edge cases in `createTestData` (nulls, empty strings, malformed inputs, varying lengths).
6262

@@ -86,10 +86,6 @@ Assertions should verify schema, row count, deterministic ordering, output value
8686
## Step 3: Compile and Test
8787

8888
```bash
89-
# Java
90-
mvn test -Dtest=UnitTest
91-
92-
# Scala
9389
mvn test -Dsuites=com.udf.UnitTest
9490
```
9591

@@ -100,16 +96,16 @@ If it fails, analyze the error output (stdout/stderr) and fix the test code. Con
10096
The template projects use JaCoCo (Java) / scoverage (Scala) code coverage tools.
10197

10298
```bash
103-
# Java
104-
mvn -Pcoverage test jacoco:report -Dtest=UnitTest
99+
# Java UDF
100+
mvn -Pcoverage test jacoco:report -Dsuites=com.udf.UnitTest
105101

106-
# Scala
102+
# Scala UDF
107103
mvn -Pcoverage scoverage:report -Dsuites=com.udf.UnitTest
108104
```
109105

110-
For Java, read `target/site/jacoco/jacoco.csv` and inspect LINE, BRANCH, and METHOD counters for the target CPU UDF class and local helper classes. In `jacoco.xml`, counters appear as `<counter type="...">` elements, and source-line misses appear under `<sourcefile><line nr="..." mi="..." ci="..." mb="..." cb="...">`.
106+
For a Java UDF, read `target/site/jacoco/jacoco.csv` and inspect LINE, BRANCH, and METHOD counters for the target CPU UDF class and local helper classes. In `jacoco.xml`, counters appear as `<counter type="...">` elements, and source-line misses appear under `<sourcefile><line nr="..." mi="..." ci="..." mb="..." cb="...">`.
111107

112-
For Scala, read `target/scoverage.xml` and inspect statement, branch, and method-level coverage for the target CPU UDF class/object and local helper classes/objects. scoverage XML stores package/class/method `statement-rate` and `branch-rate` attributes, and each executable statement has `line`, `branch`, and `invocation-count` attributes.
108+
For a Scala UDF, read `target/scoverage.xml` and inspect statement, branch, and method-level coverage for the target CPU UDF class/object and local helper classes/objects. scoverage XML stores package/class/method `statement-rate` and `branch-rate` attributes, and each executable statement has `line`, `branch`, and `invocation-count` attributes.
113109

114110
Use the coverage report as actionable feedback:
115111
1. Inspect missed Java line, branch, and method coverage, or missed Scala statement, branch, and method-level coverage.
@@ -143,6 +139,6 @@ If any quality checks fail, revise the test code and re-run.
143139

144140
Upon successful completion:
145141
- Project directory: `<project_root>/<CamelName>/`
146-
- Unit test: `src/test/<java|scala>/com/udf/UnitTest.<java|scala>`
142+
- Unit test: `src/test/scala/com/udf/UnitTest.scala`
147143

148144
These outputs are required for **Step 2: Convert UDF**.

skills/udf-gen-test/templates/java/.mvn/jvm.config

Lines changed: 0 additions & 16 deletions
This file was deleted.

0 commit comments

Comments
 (0)