|
| 1 | +# OpenCL Acceleration with UMat |
| 2 | + |
| 3 | +OpenCvSharp can use OpenCV's Transparent API (T-API) through `UMat`. When OpenCL is available and an operation has an OpenCL implementation, passing `UMat` inputs and outputs allows OpenCV to select that implementation without changing the `Cv2` method being called. |
| 4 | + |
| 5 | +OpenCL acceleration is not the same as CUDA support. OpenCvSharp does not expose OpenCV's CUDA modules, and `UMat` uses OpenCL rather than CUDA. |
| 6 | + |
| 7 | +## Treat acceleration as an execution choice, not a guarantee |
| 8 | + |
| 9 | +Using `UMat` does not guarantee that an operation runs on a GPU or that it is faster than `Mat`: |
| 10 | + |
| 11 | +- The operating system must provide a working OpenCL runtime and device driver. |
| 12 | +- OpenCV must have an OpenCL implementation for the operation, image type, and parameters. |
| 13 | +- OpenCV may reject an OpenCL path through its runtime performance checks and fall back to the CPU. |
| 14 | +- Kernel compilation, host-device transfers, and synchronization can outweigh the benefit for small images or short pipelines. |
| 15 | +- Performance varies by operation, device, and driver. An optimization that performs well on one vendor's device may behave differently on another. |
| 16 | + |
| 17 | +Measure the complete production-shaped pipeline instead of assuming that replacing `Mat` with `UMat` will improve it. |
| 18 | + |
| 19 | +## Check availability and the current state |
| 20 | + |
| 21 | +Use `Cv2.Ocl` to inspect and control OpenCL use: |
| 22 | + |
| 23 | +```csharp |
| 24 | +Console.WriteLine($"OpenCL available: {Cv2.Ocl.HaveOpenCL()}"); |
| 25 | +Console.WriteLine($"OpenCL enabled: {Cv2.Ocl.UseOpenCL()}"); |
| 26 | + |
| 27 | +Cv2.Ocl.SetUseOpenCL(true); |
| 28 | +Console.WriteLine($"OpenCL enabled: {Cv2.Ocl.UseOpenCL()}"); |
| 29 | +``` |
| 30 | + |
| 31 | +These methods answer different questions: |
| 32 | + |
| 33 | +- `HaveOpenCL()` reports whether OpenCV can find an OpenCL runtime with at least one platform. |
| 34 | +- `UseOpenCL()` reports whether OpenCL is currently enabled for the calling thread. |
| 35 | +- `SetUseOpenCL(bool)` enables or disables OpenCL use for the calling thread. |
| 36 | + |
| 37 | +An available and enabled runtime still does not prove that a particular operation used OpenCL. OpenCV can fall back to another implementation for that call. |
| 38 | + |
| 39 | +## Inspect platforms and devices |
| 40 | + |
| 41 | +`GetPlatformsInfo()` returns read-only snapshots of the OpenCL platforms and devices visible to OpenCV. It returns an empty list when no runtime is available. |
| 42 | + |
| 43 | +```csharp |
| 44 | +foreach (var platform in Cv2.Ocl.GetPlatformsInfo()) |
| 45 | +{ |
| 46 | + Console.WriteLine( |
| 47 | + $"Platform: {platform.Name} ({platform.Vendor}, {platform.Version})"); |
| 48 | + |
| 49 | + foreach (var device in platform.Devices) |
| 50 | + { |
| 51 | + Console.WriteLine($" Device: {device.Name}"); |
| 52 | + Console.WriteLine($" Type: {device.Type}"); |
| 53 | + Console.WriteLine($" OpenCL: {device.OpenCLVersion}"); |
| 54 | + Console.WriteLine($" Driver: {device.DriverVersion}"); |
| 55 | + Console.WriteLine($" Memory: {device.GlobalMemorySize} bytes"); |
| 56 | + } |
| 57 | +} |
| 58 | +``` |
| 59 | + |
| 60 | +The returned objects contain diagnostic information. They do not expose or own native OpenCL platform, device, context, or queue handles, and they cannot be used to select a device. |
| 61 | + |
| 62 | +## Keep a pipeline in UMat |
| 63 | + |
| 64 | +OpenCL is most useful when several supported operations can run while the data remains in `UMat` storage: |
| 65 | + |
| 66 | +```csharp |
| 67 | +using var source = new UMat(1080, 1920, MatType.CV_8UC3); |
| 68 | +using var blurred = new UMat(); |
| 69 | +using var hsv = new UMat(); |
| 70 | + |
| 71 | +Cv2.GaussianBlur(source, blurred, new Size(5, 5), 0); |
| 72 | +Cv2.CvtColor(blurred, hsv, ColorConversionCodes.BGR2HSV); |
| 73 | +``` |
| 74 | + |
| 75 | +Converting between `Mat` and `UMat`, accessing pixels on the CPU, or calling `UMat.GetMat()` can introduce synchronization and host-device transfers. Frequent transitions can cost more than the accelerated operations save. |
| 76 | + |
| 77 | +Use `GetMat(AccessFlag.READ)` when the CPU genuinely needs the result, not merely to make an asynchronous benchmark wait. |
| 78 | + |
| 79 | +## Benchmark queued work correctly |
| 80 | + |
| 81 | +OpenCL commands may be queued asynchronously. Measuring only the calls that enqueue work can produce unrealistically short results. Warm up the operation to exclude one-time kernel compilation, then call `Finish()` at the measurement boundaries: |
| 82 | + |
| 83 | +```csharp |
| 84 | +static TimeSpan MeasureOpenCL(Action operation, int iterations) |
| 85 | +{ |
| 86 | + operation(); |
| 87 | + Cv2.Ocl.Finish(); // Complete warm-up and kernel compilation. |
| 88 | +
|
| 89 | + var stopwatch = Stopwatch.StartNew(); |
| 90 | + for (var i = 0; i < iterations; i++) |
| 91 | + { |
| 92 | + operation(); |
| 93 | + } |
| 94 | + |
| 95 | + Cv2.Ocl.Finish(); // Include completion of all queued operations. |
| 96 | + stopwatch.Stop(); |
| 97 | + return stopwatch.Elapsed; |
| 98 | +} |
| 99 | +``` |
| 100 | + |
| 101 | +Run the operation and `Finish()` on the same thread because OpenCL enablement and the default execution context are thread-specific. |
| 102 | + |
| 103 | +Do not call `Finish()` after every operation in a normal pipeline. It blocks the calling thread and prevents OpenCV from overlapping or batching queued work. It is primarily useful for benchmarks and genuine synchronization boundaries. |
| 104 | + |
| 105 | +Compare equivalent `Mat` and `UMat` pipelines with the same source data, output consumption, dimensions, types, warm-up, and build configuration. Use Release builds, and report both latency and throughput. |
| 106 | + |
| 107 | +## Interpret OpenCL build information |
| 108 | + |
| 109 | +`Cv2.GetBuildInformation()` reports whether OpenCV was built with OpenCL support. An include path containing a version such as `opencl/1.2` identifies the OpenCL headers used to compile OpenCV; it does not cap the version reported by the installed runtime or device. |
| 110 | + |
| 111 | +OpenCV loads the OpenCL runtime at execution time. A device can therefore report OpenCL 3.0 through `GetPlatformsInfo()` even when the build information mentions 1.2 headers. Rebuilding OpenCvSharp with newer headers is not, by itself, expected to make an image-processing operation faster. |
| 112 | + |
| 113 | +## Diagnose unexpected performance |
| 114 | + |
| 115 | +When reporting an OpenCL performance problem, include: |
| 116 | + |
| 117 | +- OpenCvSharp managed and runtime package versions. |
| 118 | +- Operating system and architecture. |
| 119 | +- Release or Debug build configuration. |
| 120 | +- `HaveOpenCL()` and `UseOpenCL()` results. |
| 121 | +- Platform, device, OpenCL, and driver values from `GetPlatformsInfo()`. |
| 122 | +- The OpenCL section of `Cv2.GetBuildInformation()`. |
| 123 | +- Equivalent, warmed-up `Mat` and synchronized `UMat` measurements. |
| 124 | +- Image dimensions, `MatType`, parameters, and iteration count. |
| 125 | + |
| 126 | +OpenCvSharp forwards operations such as `Cv2.GaussianBlur` to OpenCV. Device-specific OpenCL kernels, performance guards, and CPU fallbacks are implemented by upstream OpenCV. Once synchronization and transfer costs have been accounted for, an operation-specific regression will usually need to be reproduced and investigated upstream. |
| 127 | + |
| 128 | +## Related guides |
| 129 | + |
| 130 | +- [Copies, Native Memory, and Performance](memory-copy-and-performance.md) |
| 131 | +- [InputArray, OutputArray, and In-place Processing](input-output-arrays-and-in-place.md) |
| 132 | +- [Resource Management](resource-management.md) |
| 133 | + |
| 134 | +## Official OpenCV reference |
| 135 | + |
| 136 | +- [OpenCV configuration options: OpenCL support](https://docs.opencv.org/5.0/tutorials/introduction/config_reference/config_reference.html#opencl-support) |
| 137 | + |
| 138 | +## Related OpenCvSharp API |
| 139 | + |
| 140 | +- [UMat](xref:OpenCvSharp.UMat) |
| 141 | +- [Cv2.Ocl](xref:OpenCvSharp.Cv2.Ocl) |
| 142 | +- [OclPlatformInfo](xref:OpenCvSharp.OclPlatformInfo) |
| 143 | +- [OclDeviceInfo](xref:OpenCvSharp.OclDeviceInfo) |
0 commit comments