[spec] Detection of page and/or media visbility

[meyerweb]

I’m not sure if this is really the extensions API or part of the specification itself, but it feels more like browser extension API. Feel free to reclassify/retitle if I’m wrong.

---

For continuous monetization, as might be applied to a podcast or video stream, there needs to be a way to detect whether the page is visible and whether the media is still perceivable by the user, and possibly stop/start monetization in response.

As an example, if a podcast page is set up to charge 5 cents per 5 minutes, that charge should not happen if the audio has been paused, or if the page has been frozen. Conversely, the charge should happen if the page is hidden but the audio is still audible.

It is possible to tell if a page is visible or hidden via the Page Visibility API, which operates at the page level and is visible if even one pixel of the page is displayed on screen. If it is completely obscured by other windows, or is in a background tab, then the API yields hidden instead. It is not possible to do this at the element level, and when applied to iframes, the iframe will still be regarded as visible as long as any part of the parent page is visible, even if the iframe itself is outside the viewport or hidden with CSS or similar.

Google has implemented a Page Lifecycle API, but only they support it and, again, it operates at the page level, not the element level.

So the extension needs to be able to keep track of what monetized resources are still “valid” for monetization. It seems most likely to me this would be done through some sort of DOM watcher functions, imposed by the extension on monetized things. This would look something like:

• Given monetized resource, if resource is audible media, attach a listener for play/pause and volume to the monetized resource.
  • If the media is paused or muted, stop monetization. Muting can happen with a volume slider or a mute/unmute button.
  • If the media is started or unmuted, start monetization. Unmuting as in previous.
• Further, monitor for changes to the value of Document.visibilityState where available (see Page Lifecycle API)
  • If the page is Terminated or Discarded, stop monetization.
  • It is possible that Frozen states should also stop monetization, depending on how the audio is managed. It may be necessary to impose forced-pause on audible media if a mage is Frozen in order to not permit the media to continue while in a non-monetized state.

Similar concerns apply to resources like online games, though “pause” has a different connotation in a game than it does for audible media. (A game might be paused for play but still emitting music, for example.) It is not clear how to handle this in a way the protects users from being unduly charged.

Text content might seem unlikely to be monetized via a stream rather than a one-off payment, but it possible to imagine “metered” access to text resources such as online LLM services, adult chatrooms, or premium technical support. Thus, the extension should be able to handle what to do when a text resource is backgrounded or otherwise made inaccessible to the user. This might look something like:

• Given monetized resource, if resource is not audio/video media or similar, create listeners for Document.visibilitychange (see HTML 6.2) and Document.visibilityState (where available; see Page Lifecycle API).
  • If the page becomes visible, start monetization.
  • If the page becomes hidden, stop monetization.
  • If the page becomes Frozen, Terminated, or Discarded, stop monetization.
  • If the page becomes unfrozen but not Terminated or Discarded, start monetization. (This point may need further refinement.)
• Further, if resource is confined a part of a page (an Element, iframe, or other sub-part), add listeners to that part of the page that check for the visibility and interactivity of the element.
  • If resource’s container is hidden (through JS, CSS, or visible obscuration), stop monetization.
  • If resource’s container is unhidden (through JS, CSS, or removed obscuration), start monetization.
  • If resource’s container becomes inaccessible (through JS, CSS, or input blocking), stop monetization.
  • If resource’s container becomes accessible (through JS, CSS, or input restoration), start monetization.

It is not clear if there are ways to hide a page but still capture its text content or otherwise interact with it in a way that would otherwise trigger a monetization start. If there are, then devising ways to detect this and override the monetization stop would present a risk for abuse by stealth-charging users even though the page is backgrounded or otherwise not in use.

[sidvishnoi]

Thank you @meyerweb for going deeper in this.

While the spec has an example for video/audio, it doesn't specify the behavior.
I think it'll be best if the behavior is defined in the WM specification and implemented in core, not as an extension API ¹. The core spec defines the page visibility, so we can extend it to video/audio too.

In the Web Monetization extension, we've explored in past: Tab.muted/Tab.audible, pictureinpicture and related audio/video APIs/events (like AudioContext), but that's not the whole story. MediaSession API could be something worth looking into as well.

This whole audio/video monetization idea is something we plan to discuss in our upcoming work week meeting next week in Romania. Last time we had a related discussion, we felt like only the monetization links under <audio>, <video> should be monetized while page is not in view, but that assumption can change.

Footnotes

1. The Web Monetization extension is "core" until native implementations are there, but that's different from "extension API" in context of this repo. ↩

[meyerweb]

I think it'll be best if the behavior is defined in the WM specification and implemented in core, not as an extension API 1. The core spec defines the page visibility, so we can extend it to video/audio too.

Sounds good to me. I’ll update this issue subject to match.

Last time we had a related discussion, we felt like only the monetization links under ,

Speaking of which, I poked around the HTML spec a little to see if link elements under audio/video was already a supported thing, or a monkey patch from WM. Still not sure, to be honest. If it is in fact already supported by HTML, can you point me to the bit that makes it legal?