You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Deduplicate Java/Scala template projects in skills [skip ci] (#15111)
Fixes#15094.
### Description
This removes the Java template project provided with the skills, making
the Scala project the ground truth for any input JVM UDF. The
instructions are adjusted accordingly. Jacoco and Hive support is moved
over to the Scala templates.
### Checklists
Documentation
- [X] Updated for new or modified user-facing features or behaviors
- [ ] No user-facing change
Testing
- [ ] Added or modified tests to cover new code paths
- [ ] Covered by existing tests
(Please provide the names of the existing tests in the PR description.)
- [X] Not required
Performance
- [ ] Tests ran and results are added in the PR description
- [ ] Issue filed with a link in the PR description
- [X] Not required
---------
Signed-off-by: Rishi Chandra <rishic@nvidia.com>
Copy file name to clipboardExpand all lines: skills/udf-benchmark/SKILL.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ Derive `<CamelName>` and `<snake_name>` from the UDF class name.
28
28
29
29
## Step 1: Implement BenchUtils
30
30
31
-
Read `src/main/<java|scala>/com/udf/bench/BenchUtils.<java|scala>`. Replace placeholders with the actual camel/snake UDF name.
31
+
Read `src/main/scala/com/udf/bench/BenchUtils.scala`. Replace placeholders with the actual camel/snake UDF name.
32
32
33
33
Fill in the TODO methods following the docstrings. For variable-length inputs, generate sizable rows representative of enterprise-scale data. Refer to the unit test for schema and example data.
34
34
@@ -76,7 +76,7 @@ Follow [CUDF_MICROBENCHMARKS.md](CUDF_MICROBENCHMARKS.md) to implement and run i
Copy file name to clipboardExpand all lines: skills/udf-convert-to-cuda/SKILL.md
+3-7Lines changed: 3 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -54,7 +54,7 @@ Use `src/main/java/com/udf/PlaceholderUDFNameNativeRapidsUDF.java` as a starting
54
54
4. Implement `evaluateColumnar` to validate column count/types and call the native method.
55
55
5. Rename the native method to a descriptive operation name, e.g. `cosineSimilarityNative`.
56
56
57
-
For Scala projects, keep this Java wrapper under `src/main/java/com/udf/` and register it from the Scala test/project. JNI can be used from Scala, but the Java wrapper keeps native symbol names and examples simpler.
57
+
The project joint-compiles Java, so keep this Java wrapper under `src/main/java/com/udf/` and register it from the Scala test. JNI can be used from Scala, but the Java wrapper keeps native symbol names and examples simpler.
58
58
If the Java wrapper's CPU fallback needs to call a Scala object, direct references can fail before `scala-maven-plugin` compiles the Scala classes; use reflection in the row-by-row fallback only, and keep `evaluateColumnar` on the normal JNI path.
59
59
60
60
Read [JNI_CUDA_GUIDE.md](references/JNI_CUDA_GUIDE.md) for the `evaluateColumnar` contract, type mapping, pointer ownership, `NativeDepsLoader`, and native memory rules.
@@ -104,16 +104,12 @@ If the build fails while resolving cuDF headers or RAPIDS CMake, check network a
104
104
105
105
## Step 5: Build and Test
106
106
107
-
Fill in the target-specific TODOs in `src/test/<java|scala>/com/udf/CudfComparisonTest.<java|scala>`:
107
+
Fill in the target-specific TODOs in `src/test/scala/com/udf/CudfComparisonTest.scala`:
108
108
- Register `<CamelName>NativeRapidsUDF` as the GPU implementation
109
109
- Replace placeholder UDF names
110
110
111
111
Run:
112
112
```bash
113
-
# Java
114
-
mvn test -Dtest=CudfComparisonTest -Pcuda-native-udf
115
-
116
-
# Scala project using a Java native RapidsUDF wrapper
117
113
mvn test -Dsuites=com.udf.CudfComparisonTest -Pcuda-native-udf
118
114
```
119
115
@@ -129,7 +125,7 @@ docker run --rm --gpus all \
129
125
-v /etc/group:/etc/group:ro \
130
126
-w /workspace \
131
127
cuda-udf-build \
132
-
-c "mvn -B -Dmaven.repo.local=/workspace/.m2/repository test -Dtest=CudfComparisonTest -Pcuda-native-udf -Dnative.build.path=/workspace/target/native-build-docker -DskipCudfExtraction=true"
128
+
-c "mvn -B -Dmaven.repo.local=/workspace/.m2/repository test -Dsuites=com.udf.CudfComparisonTest -Pcuda-native-udf -Dnative.build.path=/workspace/target/native-build-docker -DskipCudfExtraction=true"
133
129
```
134
130
135
131
If tests fail, iterate on the Java bridge or native implementation.
This provides a complete Maven project with all test and benchmark infrastructure.
@@ -56,7 +56,7 @@ Set the package declaration to `com.udf`:
56
56
57
57
## Step 2: Implement the Unit Test
58
58
59
-
Read `src/test/<java|scala>/com/udf/UnitTest.<java|scala>`. Replace placeholders with the actual camel/snake UDF name.
59
+
Read `src/test/scala/com/udf/UnitTest.scala`. Replace placeholders with the actual camel/snake UDF name.
60
60
61
61
Fill in the TODO methods following the docstrings. Include diverse edge cases in `createTestData` (nulls, empty strings, malformed inputs, varying lengths).
62
62
@@ -86,10 +86,6 @@ Assertions should verify schema, row count, deterministic ordering, output value
86
86
## Step 3: Compile and Test
87
87
88
88
```bash
89
-
# Java
90
-
mvn test -Dtest=UnitTest
91
-
92
-
# Scala
93
89
mvn test -Dsuites=com.udf.UnitTest
94
90
```
95
91
@@ -100,16 +96,16 @@ If it fails, analyze the error output (stdout/stderr) and fix the test code. Con
100
96
The template projects use JaCoCo (Java) / scoverage (Scala) code coverage tools.
101
97
102
98
```bash
103
-
# Java
104
-
mvn -Pcoverage test jacoco:report -Dtest=UnitTest
99
+
# Java UDF
100
+
mvn -Pcoverage test jacoco:report -Dsuites=com.udf.UnitTest
For Java, read `target/site/jacoco/jacoco.csv` and inspect LINE, BRANCH, and METHOD counters for the target CPU UDF class and local helper classes. In `jacoco.xml`, counters appear as `<counter type="...">` elements, and source-line misses appear under `<sourcefile><line nr="..." mi="..." ci="..." mb="..." cb="...">`.
106
+
For a Java UDF, read `target/site/jacoco/jacoco.csv` and inspect LINE, BRANCH, and METHOD counters for the target CPU UDF class and local helper classes. In `jacoco.xml`, counters appear as `<counter type="...">` elements, and source-line misses appear under `<sourcefile><line nr="..." mi="..." ci="..." mb="..." cb="...">`.
111
107
112
-
For Scala, read `target/scoverage.xml` and inspect statement, branch, and method-level coverage for the target CPU UDF class/object and local helper classes/objects. scoverage XML stores package/class/method `statement-rate` and `branch-rate` attributes, and each executable statement has `line`, `branch`, and `invocation-count` attributes.
108
+
For a Scala UDF, read `target/scoverage.xml` and inspect statement, branch, and method-level coverage for the target CPU UDF class/object and local helper classes/objects. scoverage XML stores package/class/method `statement-rate` and `branch-rate` attributes, and each executable statement has `line`, `branch`, and `invocation-count` attributes.
113
109
114
110
Use the coverage report as actionable feedback:
115
111
1. Inspect missed Java line, branch, and method coverage, or missed Scala statement, branch, and method-level coverage.
@@ -143,6 +139,6 @@ If any quality checks fail, revise the test code and re-run.
0 commit comments