[RFC] vo_libmpv/gpu-next: add Vulkan render API by Superborec22 · Pull Request #18258 · mpv-player/mpv

Superborec22 · 2026-07-13T21:19:50Z

Summary

This RFC adds a Vulkan backend to the libmpv render API, allowing embedding applications to render directly into externally owned VkImage targets.

The implementation is based on Andrew Rabert's experimental Vulkan libmpv work, forward-ported to current mpv master. It also adds client-provided wait and signal semaphores so rendering and presentation can remain GPU-synchronized instead of blocking the CPU after every frame.

Existing OpenGL and software render paths remain unchanged.

Motivation

This was developed for mpvpaper, which owns its Wayland surfaces and presentation loop.

Standalone mpv can select Vulkan through normal GPU options, but those options cannot replace the render API and target supplied by an embedding application. A Vulkan-capable client needs to provide its own instance, device, queue, image target, layouts and synchronization objects.

The immediate use case is rendering through a selected GPU in a multi-GPU Wayland setup.

API

The RFC introduces a Vulkan render API and public render_vk.h header. The client supplies:

Vulkan instance, physical device, logical device and graphics queue
an externally owned VkImage and VkImageView
current and final image layouts
optional wait and signal semaphores

The client retains ownership of all Vulkan resources.

The backend imports the device into libplacebo, renders into the supplied image and transitions it to the requested final layout. With a client signal semaphore, the image can be passed directly to vkQueuePresentKHR() without a per-frame CPU wait.

Origin

The initial backend is based on work from the andrewrabert/mpv fork. This PR forward-ports it to:

e5486b96d7d06dd148337899bfdc46bf25101663

The synchronization, final-layout handling and client integration were adjusted while testing it with mpvpaper. I am not claiming sole authorship of the original backend.

Testing

Built successfully with GCC 16.1.1, Meson 1.11.1, libplacebo 7.360.1 and Vulkan 1.4.350.

A patched mpvpaper was built against the resulting libmpv.so.2.5.0 and tested on Wayland/Hyprland with an RTX 4070 driving the compositor and a GTX 1660 SUPER selected for rendering.

4K H.264 videos played and looped correctly using Vulkan hardware decoding. nvidia-smi confirmed that mpvpaper was running on the selected GPU.

Status

This is an RFC because the public API, ownership model and semaphore interface need upstream review. The corresponding client-api-changes.rst entry can be added once the API shape is agreed upon.

AI assistance disclosure

AI was used to prepare parts of the patch, comments in the code and description.

Add Vulkan image targets to the gpu-next libmpv rendering API. Support externally owned Vulkan images, caller-provided wait and signal semaphores, target layout transitions, and asynchronous GPU-to-GPU synchronization suitable for Wayland swapchain presentation. This implementation is based on Andrew Rabert's experimental Vulkan libmpv backend and has been forward-ported and extended for mpvpaper.

CounterPillow · 2026-07-13T22:24:11Z

@@ -0,0 +1,213 @@
+/* Copyright (C) 2018 the mpv developers


2018? oh this is just gonna be straight up slop isn't it

CounterPillow · 2026-07-13T22:25:12Z

 */

-#include <libplacebo/config.h>
+#include <libplacebo/config.h>                   // for PL_HAVE_OPENGL, PL_API_VER


CounterPillow · 2026-07-13T22:25:35Z

+#include "mpv/render_gl.h"                       // for mpv_opengl_init_params
+#include <libplacebo/opengl.h>                   // for pl_opengl_destroy
+#include "video/out/gpu_next/libmpv_gpu_next.h"  // for libmpv_gpu_next_context
+#include "video/out/gpu_next/ra.h"               // for ra_pl_create, ra_pl_...


stop with the comments, god

CounterPillow · 2026-07-13T22:26:46Z

+#if HAVE_GL
+// Store Libplacebo OpenGL context information.
+struct priv {
+    pl_log pl_log;
+    pl_opengl gl;
+    pl_gpu gpu;
+    struct ra_next *ra;
+
+    // Store a persistent copy of the init params to avoid a dangling pointer.
+    mpv_opengl_init_params gl_params;
+};


random opengl changes, sure, whatever, fuck me

CounterPillow · 2026-07-13T22:30:43Z

+    // Setup libplacebo logging
+    struct pl_log_params log_params = {
+        .log_level = PL_LOG_DEBUG
+    };
+
+    // Enable verbose logging if trace is enabled
+    if (mp_msg_test(ctx->log, MSGL_TRACE)) {
+        log_params.log_cb = pl_log_cb_vk;
+        log_params.log_priv = ctx->log;
+    }
+
+    p->pl_log = pl_log_create(PL_API_VER, &log_params);
+    if (!p->pl_log) {
+        MP_ERR(ctx, "Failed to create libplacebo log\n");
+        return MPV_ERROR_GENERIC;
+    }


yeah better duplicate all of this

kasper93 · 2026-07-13T22:30:52Z

+ *
+ * Required device extensions (user must enable these):
+ * - VK_KHR_timeline_semaphore (core in 1.2+)
+ * - VK_KHR_external_memory (for hwdec interop)


Why is this required?

kasper93 · 2026-07-13T22:31:10Z

+ * - User must enable required extensions when creating the device (see below)
+ *
+ * Required device extensions (user must enable these):
+ * - VK_KHR_timeline_semaphore (core in 1.2+)


Why is this required if this is core in 1.3+?

kasper93 · 2026-07-13T22:31:36Z

+ * Recommended device extensions:
+ * - VK_KHR_video_decode_queue (for Vulkan video decode hwdec)
+ * - VK_KHR_video_decode_h264
+ * - VK_KHR_video_decode_h265


Only h264 and h265 is supported?

CounterPillow · 2026-07-13T22:32:44Z

+        .get_proc_addr = get_proc,
+        .queue_graphics = {
+            .index = vk_params->graphics_queue_family,
+            .count = 1,


so vulkan-queue-count option is just not used, ok cool

kasper93 · 2026-07-13T22:33:13Z

+ * -----------------
+ *
+ * For Vulkan hwdec to work, the user must:
+ * - Provide MPV_RENDER_PARAM_WL_DISPLAY (Wayland) for dmabuf import


Why? What's wrong with existing primitives for importing vaapi or drm images?

kasper93 · 2026-07-13T22:33:52Z

+ *
+ * For Vulkan hwdec to work, the user must:
+ * - Provide MPV_RENDER_PARAM_WL_DISPLAY (Wayland) for dmabuf import
+ * - Enable the appropriate video decode extensions on the device


I think we should export recommended and required extentions as static arrays, same as libplacebo does.

kasper93 · 2026-07-13T22:35:01Z

+     * The device MUST have been created with at least the features required
+     * by libplacebo, including:
+     * - hostQueryReset (Vulkan 1.2 / VK_EXT_host_query_reset)
+     * - timelineSemaphore (Vulkan 1.2 / VK_KHR_timeline_semaphore)


Don't we support only vulkan 1.3?

CounterPillow · 2026-07-13T22:36:39Z

+    if (p->ra) {
+        ra_pl_destroy(&p->ra);
+    }
+
+    if (p->frame_tex)
+        pl_tex_destroy(p->gpu, &p->frame_tex);
+    if (p->completion)
+        vkDestroySemaphore(p->vulkan->device, p->completion, NULL);
+    if (p->vulkan) {
+        pl_vulkan_destroy(&p->vulkan);
+    }


sure just use braces however you want

kasper93 · 2026-07-13T22:37:04Z

+    // context current inside its get_proc_address callback. We can trigger
+    // this by calling it with a harmless, common function name.
+    if (gl_params && gl_params->get_proc_address) {
+        gl_params->get_proc_address(gl_params->get_proc_address_ctx, "glGetString");


Could you explain why this is needed?

kasper93 · 2026-07-13T22:38:04Z

+static void pl_log_cb(void *log_priv, enum pl_log_level level, const char *msg)
+{
+    struct mp_log *log = log_priv;
+    mp_msg(log, MSGL_WARN, "[gpu-next:pl] %s\n", msg);


You don't have to use manual prefixes, create child logger.

kasper93 · 2026-07-13T22:39:01Z

+static void pl_log_cb_vk(void *log_priv, enum pl_log_level level, const char *msg)
+{
+    struct mp_log *log = log_priv;
+    mp_msg(log, MSGL_WARN, "[gpu-next:pl] %s\n", msg);


This looks very similar to previous code.

kasper93 · 2026-07-13T22:39:27Z

 #pragma once

-#include <libplacebo/renderer.h>
+#include "libplacebo/gpu.h"        // for pl_gpu


I think <> is proper way for includes.

kasper93 · 2026-07-13T22:39:40Z

+#include "libplacebo/gpu.h"        // for pl_gpu
+#include "libplacebo/log.h"        // for pl_log
+#include "libplacebo/swapchain.h"  // for pl_swapchain
+#include "stdbool.h"               // for bool


<> and put includes at the top

kasper93 · 2026-07-13T22:40:04Z

@@ -0,0 +1,314 @@
+#include "libmpv_gpu_next.h"


license header missing

kasper93 · 2026-07-13T22:41:25Z

+static struct mp_image *get_image(struct render_backend *ctx, int imgfmt,
+                                  int w, int h, int stride_align, int flags)
+{
+    return NULL;


What if user wants to get image?

CounterPillow · 2026-07-13T22:41:37Z

+    }
+
+    // Create hardware decoder devices.
+    ctx->hwdec_devs = hwdec_devices_create();


I guess ra_hwdec_ctx_init is not cool enough to be called

kasper93 · 2026-07-13T22:41:51Z

+ */
+static void perfdata(struct render_backend *ctx,
+                     struct voctrl_performance_data *out)
+{


Could you explain how this implementation collects performance data?

kasper93 · 2026-07-13T22:42:11Z

@@ -0,0 +1,409 @@
+#include "video/out/gpu_next/ra.h"


missing license header

CounterPillow · 2026-07-13T22:42:26Z

+ra_queue ra_next_queue_create(struct ra_next *ra)
+{
+    return pl_queue_create(ra->gpu);
+}
+
+void ra_next_queue_destroy(ra_queue *queue)
+{
+    pl_queue_destroy(queue);
+}
+
+void ra_next_queue_push(ra_queue queue, const struct pl_source_frame *frame)
+{
+    pl_queue_push(queue, frame);
+}
+
+void ra_next_queue_update(ra_queue queue, struct pl_frame_mix *mix, const struct pl_queue_params *params)
+{
+    pl_queue_update(queue, mix, params);
+}
+
+void ra_next_queue_reset(ra_queue queue)
+{
+    pl_queue_reset(queue);
+}
+
+pl_tex ra_next_tex_create(struct ra_next *ra, const struct pl_tex_params *params)
+{
+    return pl_tex_create(ra->gpu, params);
+}
+
+void ra_next_tex_destroy(struct ra_next *ra, pl_tex *tex)
+{
+    pl_tex_destroy(ra->gpu, tex);
+}
+
+bool ra_next_tex_recreate(struct ra_next *ra, pl_tex *tex, const struct pl_tex_params *params)
+{
+    return pl_tex_recreate(ra->gpu, tex, params);
+}


very useful, thank you

kasper93 · 2026-07-13T22:43:08Z

+/* Cleanup any textures/resources attached to a pl_frame created by upload. */
+void ra_cleanup_pl_frame(struct ra_next *ra, struct pl_frame *frame);
+
+/* --- New Texture Management Wrappers (prefixed to avoid conflicts) --- */


Will they always stay New?

CounterPillow · 2026-07-13T22:43:18Z

@@ -0,0 +1,63 @@
+#pragma once
+
+#include "stdbool.h"         // for bool


could've never guessed

CounterPillow · 2026-07-13T22:44:21Z

+    char *backend_name = get_mpv_render_param(params, MPV_RENDER_PARAM_BACKEND, "");
+    const struct render_backend_fns **render_backends;
+
+    if (backend_name && strcmp(backend_name, "gpu-next") == 0) {
+        MP_VERBOSE(ctx, "Using gpu-next backend.\n");
+        static const struct render_backend_fns* backends[] = {
+            &render_backend_gpu_next,
+            &render_backend_sw,
+            NULL
+        };
+        render_backends = backends;
+    } else {
+        MP_VERBOSE(ctx, "Using default gpu backend.\n");
+        static const struct render_backend_fns* backends[] = {
+            &render_backend_gpu,
+            &render_backend_sw,
+            NULL
+        };
+        render_backends = backends;
+    }
+


???????????????????????????

kasper93 · 2026-07-13T22:44:28Z

+ */
+bool pl_video_check_format(struct pl_video *p, int imgfmt) {
+    // For simplicity, we assume libplacebo can handle it.
+    // A more robust implementation might query libplacebo's capabilities.


I think we need to be robust unfortunately.

kasper93 · 2026-07-13T22:52:07Z

I am not claiming sole authorship of the original backend.

Then you should preserve history and commits of prior work. Also spliting work into smaller commits will be easier to review. Additionally, some things should be tested in isolation. For example we probably should first port the vo_gpu_next. to ra primitives, validate this and then expose this in libmpv api. Additionally it would be nice if you could implement the meson test for libmpv rendering api or a standalone app that can be used at least for build test.

CounterPillow · 2026-07-13T22:53:11Z

Dudemanguy · 2026-07-13T22:53:50Z

The initial backend is based on work from the andrewrabert/mpv fork.

The synchronization, final-layout handling and client integration were adjusted while testing it with mpvpaper. I am not claiming sole authorship of the original backend.

Uh in that case don't strip authorship and make one big commit that pretends you did all of it?
https://github.com/andrewrabert/mpv/commits/feat/libplacebo-swapchain/

Not really salvageable. I'll just close.

Superborec22 mentioned this pull request Jul 13, 2026

Add experimental Vulkan render backend GhostNaN/mpvpaper#133

Closed

Superborec22 force-pushed the feat/libmpv-vulkan-port branch from e2262b0 to fd96c86 Compare July 13, 2026 22:00

CounterPillow reviewed Jul 13, 2026

View reviewed changes