vo: add vrr functionality for non-display sync modes

The intention is for vrr mode to be a change to base behaviour, and thus be compatible with every selection of video-sync modes. However, currently it is only compatible for non-display sync ones. For display sync modes in the future, the expectation is to continue using this vrr structure, but just have vo immediatelly output frames one after the other like display sync does, and then use api functions like presentAt to present the frame based on the data from voframe.

The expectation would be for display-sync vrr to exclude some of the estimations that display-sync uses, since they are not required and for cleaner design.

One other expectation is for the driver to immediatelly drop frames when the swapchain is fully filled out, so that vo can then decide whether to move to a newer frame or recalculate the dropped frame position before trying to send it again, instead of the driver dropping older valid frames, or delaying the newer frame to later.

Additional changes:
#. merged the repeat functionality from display-sync modes into a new variable that can be used by vrr. it is not expected but it is possible for display-sync behaviour to have changed because of that.
#. moved in->wakeup_pts set up inside render_frame instead of vo_queue_frame. This is because it doesn't appear to have a pre-render_frame purpose inside vo_queue_frame, and to allow decision based set up in render_frame.
#. we now directly update frame-> information when pts and duration changes (i.e like when "now" has moved past our pts), to give the driver accurate data.
#. after doing the initial duration < 0 check, we then clamp duration to 0, as non-valid negative duration input messes up the logic.
#. display-sync now also uses a simple pts + duration for its in->wakeup_pts instead of an "= 0" special case. Since it already modifies pts to 0 and duration to -1 -> going to 0, end time is expected to also result in 0. Trying to re-use existing logic than making a lot of special cases.
#. The "strict drop threshold" has been changed to include the case where end_time == now (as per duration <= 0), instead of just end_time < now. The vrr logic is cleaner when we drop frames at duration = 0, and i believe makes sense.
#. Added a in->current_frame check for VO_CAP_NORETAIN, since stuff may randomly set current_frame to null (like vo_reconfig?), and just generally safer.
#. Clarified the VO_CAP_FRAMEOWNER condition to be compatible with vrr, and also put up that the addition of the driver_dropped_frame condition (the logic of which is exactly the same as before this change) doesn't make much sense and should be removed, but i'll leave it to somebody else.

Author: AnataBakka

DOCS/man/input.rst

@@ -2311,11 +2311,12 @@ Property list
 
 ``mistimed-frame-count``
     Number of video frames that were not timed correctly in display-sync mode
-    for the sake of keeping A/V sync. This does not include external
+    for the sake of keeping A/V sync (i.e using the ``display-desync`` mode should
+    not change this value from 0), or in ``vrr-adjust`` mode for the sake of keeping
+    within the defined refresh rate range. This does not include external
     circumstances, such as video rendering being too slow or the graphics
     driver somehow skipping a vsync. It does not include rounding errors either
-    (which can happen especially with bad source timestamps). For example,
-    using the ``display-desync`` mode should never change this value from 0.
+    (which can happen especially with bad source timestamps).
 
 ``vsync-ratio``
     For how many vsyncs a frame is displayed on average. This is available if

DOCS/man/options.rst

@@ -1320,6 +1320,10 @@ Video
     Set this option only if you have reason to believe the automatically
     determined value is wrong.
 
+``--minimum-display-fps=<fps>``
+    Set the minimum display FPS used with the ``--vrr-adjust=true`` mode. By
+    default, it uses the maximum display fps.
+
 ``--hwdec=<api1,api2,...|no|auto|auto-copy>``
     Specify the hardware video decoding API that should be used if possible.
     Whether hardware decoding is actually done depends on the video codec. If
@@ -8224,6 +8228,56 @@ Video Sync
     frame dropping due to the audio "overshooting" and skipping multiple video
     frames before the sync logic can react.
 
+``--vrr-adjust=<yes|no>``
+    This option tries to increase the chance of the display being ready for
+    frames the moment they are sent (default: no), for cases where the
+    display is unreliable, primarily useful for VRR displays. This uses
+    ``--minimum-display-fps`` and display-fps (maximum fps).
+    
+    This is done by observing each individual frame and dynamically:
+
+    1. Not doing anything if they are perfectly between the minimum refresh time
+       and the middle refresh time.
+    2. Repeating frames at a middle refresh time if possible, else repeating
+       tending towards the middle refresh time (the middle refresh time gives
+       us the best error leeway in case there are random application or OS delays
+       that would put us in a position that exceeds either the minimum or
+       the maximum refresh time).
+    3. If we can't reliably repeat it, then we send it even if it's between the 
+       middle refresh time and the maximum refresh time. Note that because of the
+       above mentioned potential delays, this may cause us to exceed our target
+       even if we initially appear in valid range.
+    4. If the next frame exceeds our minimum refresh time, whether because our
+       repetition failed or that's how the frames exist, we will move it to the
+       closest valid position that's within the refresh range. If two frames are
+       put on the same position, we send the newest one only.
+
+    Note: This is not compatible with ``--video-sync=display-...`` modes, yet,
+    which would be the only ones capable of getting rid of the last displaying
+    weirdnesses.
+
+``--vrr-adjust-max-refresh-variance=<0.0-1.0>``
+    How much percentage wise the refresh time can vary from one frame to the next,
+    where 1.0 means the refresh time can immediatelly move between the minimum and
+    maximum refresh time, while 0 means the refresh time is static. (default: 1.0)
+
+    The default does not take into account refresh rate flicker (which may happen
+    in specific cases, eg. when time between frames is constantly higher than
+    minimum refresh time + middle refresh time), and optimizes just for reaching
+    target position. If that's a problem, then this option can limit sudden changes in refresh
+    time. The lower the value set, the more refresh time stability, the higher chance
+    of missing our target. 
+
+``--vrr-adjust-target-refresh-rate=<fps>``
+    Set the refresh rate we should tend towards, instead of the default "middle refresh
+    time". Setting it at a lower refresh rate will reduce the amount of repetitions,
+    which can help if computational burden is a problem. However, increasing it or
+    reducing it will come at a cost of potential chance increase of missing our target.
+
+    When needed, we will still use the entire range of ``--minimum-display-fps`` and
+    ``--display-fps-override``, so this option only increases the chance towards
+    the refresh rate we tend towards.
+
 Miscellaneous
 -------------
 

options/options.c

@@ -189,6 +189,8 @@ static const m_option_t mp_vo_opt_list[] = {
     {"show-in-taskbar", OPT_BOOL(show_in_taskbar)},
     {"display-fps-override", OPT_DOUBLE(display_fps_override),
         M_RANGE(0, DBL_MAX)},
+    {"minimum-display-fps", OPT_DOUBLE(minimum_display_fps),
+        M_RANGE(0, DBL_MAX)},
     {"video-timing-offset", OPT_DOUBLE(timing_offset), M_RANGE(0.0, 1.0)},
     {"video-sync", OPT_CHOICE(video_sync,
         {"audio", VS_DEFAULT},
@@ -200,6 +202,9 @@ static const m_option_t mp_vo_opt_list[] = {
         {"display-vdrop", VS_DISP_VDROP},
         {"display-desync", VS_DISP_NONE},
         {"desync", VS_NONE})},
+    {"vrr-adjust", OPT_BOOL(vrr_adjust)},
+    {"vrr-adjust-max-refresh-variance", OPT_DOUBLE(vrr_max_refresh_variance), M_RANGE(0.0, 1.0)},
+    {"vrr-adjust-target-refresh-rate", OPT_DOUBLE(vrr_target_refresh_rate)},
 #if HAVE_X11
     {"x11-netwm", OPT_CHOICE(x11_netwm, {"auto", 0}, {"no", -1}, {"yes", 1})},
     {"x11-bypass-compositor", OPT_CHOICE(x11_bypass_compositor,
@@ -292,7 +297,10 @@ const struct m_sub_options vo_sub_opts = {
         .wl_present = true,
         .mmcss_profile = "Playback",
         .ontop_level = -1,
+        .minimum_display_fps = -1,
         .timing_offset = 0.050,
+        .vrr_max_refresh_variance = 1,
+        .vrr_target_refresh_rate = -1,
         .swapchain_depth = 2,
         .focus_on = 1,
     },

options/options.h

@@ -82,8 +82,12 @@ typedef struct mp_vo_opts {
     int window_corners;
 
     double display_fps_override;
+    double minimum_display_fps;
     double timing_offset;
     int video_sync;
+    bool vrr_adjust;
+    double vrr_max_refresh_variance;
+    double vrr_target_refresh_rate;
 
     struct m_geometry android_surface_size;
 

player/command.c

@@ -765,7 +765,7 @@ static int mp_property_mistimed_frame_count(void *ctx, struct m_property *prop,
                                             int action, void *arg)
 {
     MPContext *mpctx = ctx;
-    if (!mpctx->vo_chain || !mpctx->display_sync_active)
+    if (!mpctx->vo_chain || !(mpctx->display_sync_active || mpctx->video_out->opts->vrr_adjust))
         return M_PROPERTY_UNAVAILABLE;
 
     return m_property_int_ro(action, arg, mpctx->mistimed_frames_total);

player/video.c

@@ -808,6 +808,11 @@ static void handle_display_sync_frame(struct MPContext *mpctx,
 
     mpctx->display_sync_active = false;
 
+    if (vo->opts->vrr_adjust && vo_get_pts_offset(vo) != 0) {
+        mpctx->mistimed_frames_total += 1;
+        MP_STATS(mpctx, "mistimed");
+    }
+
     if (!VS_IS_DISP(mode) || !vo_is_visible(vo))
         return;
 

video/out/gpu/video.c

@@ -3596,7 +3596,7 @@ void gl_video_render_frame(struct gl_video *p, struct vo_frame *frame,
 
                 // For the non-interpolation case, we draw to a single "cache"
                 // texture to speed up subsequent re-draws (if any exist)
-                bool repeats = frame->num_vsyncs > 1 && frame->display_synced;
+                bool repeats = frame->request_repeat;
                 bool r = false;
                 if ((repeats || frame->still) && !p->dumb_mode &&
                     (p->ra->caps & RA_CAP_BLIT) && fbo->tex->params.blit_dst)