Problem
In our repo, several data-format converters currently run on the CPU, one row at a time, and also forces a GPU->host->GPU roundtrip. That is very inefficient and slow.
Such converters can be categorized into two main groups:
TypedImperativeAggregate buffer transitions (CpuToGpuBufferTransition/GpuToCpuBufferTransition, both CodegenFallback) for percentile and collect_list/collect_set.
- Row-columnar conversion for variable-width and nested types (
string, binary, decimal128, array, map, struct).
Other lower-priority host-side per-row converters include:
ColumnarCopyHelper (ColumnarCopyHelper.java, called from HostColumnarToGpu.scala): copies each host columnar batch to the GPU one row at a time for non-Arrow CPU columnar scans.
GpuRand (GpuRandomExpressions.scala): builds rand()/randn() row by row on the CPU.
ParquetCachedBatchSerializer (ParquetCachedBatchSerializer.scala): processes nested array/map/struct elements on the host, for CPU cache fallback.
CudfMergeM2 (aggregateFunctions.scala): merges stddev_*/var_* partial output on the host in the reduction context.
- Text ingest builds string columns per line on the host (
HostStringColBufferer in GpuTextBasedPartitionReader.scala).
Solution
Move such functions onto the GPU by implementing dedicated data conversion kernels. By doing so, we will have all data processing running on the GPU.
Problem
In our repo, several data-format converters currently run on the CPU, one row at a time, and also forces a GPU->host->GPU roundtrip. That is very inefficient and slow.
Such converters can be categorized into two main groups:
TypedImperativeAggregatebuffer transitions (CpuToGpuBufferTransition/GpuToCpuBufferTransition, bothCodegenFallback) forpercentileandcollect_list/collect_set.string,binary,decimal128,array,map,struct).Other lower-priority host-side per-row converters include:
ColumnarCopyHelper(ColumnarCopyHelper.java, called fromHostColumnarToGpu.scala): copies each host columnar batch to the GPU one row at a time for non-Arrow CPU columnar scans.GpuRand(GpuRandomExpressions.scala): buildsrand()/randn()row by row on the CPU.ParquetCachedBatchSerializer(ParquetCachedBatchSerializer.scala): processes nestedarray/map/structelements on the host, for CPU cache fallback.CudfMergeM2(aggregateFunctions.scala): mergesstddev_*/var_*partial output on the host in the reduction context.HostStringColBuffererinGpuTextBasedPartitionReader.scala).Solution
Move such functions onto the GPU by implementing dedicated data conversion kernels. By doing so, we will have all data processing running on the GPU.