Support out-of-tree plugins autoloading

[dvrogozh]

This commit adds support of out-of-tree plugins autoloading using python package entry point specification:

• https://packaging.python.org/en/latest/specifications/entry-points/#entry-points
• https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/#using-package-metadata

Out-of-tree plugins must register entry points as torchcodec.backends:

[project.entry-points.'torchcodec.backends']
device_backend = 'torchcodec_plugin:load_plugin'

Torchcodec will automatically load plugins if discovered. Loading can be explicitly suppressed with TORCHCODEC_DEVICE_BACKEND_AUTOLOAD=1 environment variable.

The same approach is being used to load PyTorch device backends. See:

• [RFC] Add support for device extension autoloading pytorch/pytorch#127074

Here is a PR for XPU plugin which will enable autoloading if torchcodec PR will get merged:

• Enable plugin autoloading dvrogozh/torchcodec-xpu#8

CC: @scotts @NicolasHug

[NicolasHug]

Thanks for the PR @dvrogozh . Just to make sure I understand correctly, the main goal of this PR is to go from:

import torchcodec
import some_torchcodec_plugin

to:

import torchcodec  # automatically loads some_torchcodec_plugin if it was installed

Is my understanding correct?

[dvrogozh]

@NicolasHug, yes, your understanding is correct.

[NicolasHug]

Thanks for confirming @dvrogozh .

I'm still in the process to form an opinion on this, so please bear with me. My first reaction was pretty much the same as Ed's (pytorch/pytorch#122468 (comment)): an extra import isn't the end of the world.

I get the argument that some scripts are device agnostic and that the device is passed as a CLI parameter, in which case having to do an extra import is a problem. Whether that's really an issue in practice for torchcodec, I don't know yet.

Quick Qs which will help me form a better opinion:

1. what's the current way for users to use the intel TorchCodec extension? They'd have to pass VideoDecoder(... device="xpu") or e.g. device=torch.xpu.device(0), right?
2. correct me if I'm wrong: if a user is leaving the device to its default, i.e. VideoDecoder(..., device=None), then the existence of the XPU TorchCodec extension has zero effect, right (whether it's imported manually or automatically through the plugin mechanism)?

Basically, I just want to make sure that users will be using a new out-of-tree backend only when they explicitly request it, either by passing device=SomeXPUDevice explicitly, or by passing a CLI arg like device=xpu:0. What I'm trying to avoid are non-explicit behaviors.

[dvrogozh]

what's the current way for users to use the intel TorchCodec extension? They'd have to pass VideoDecoder(... device="xpu") or e.g. device=torch.xpu.device(0), right?

Yes, that is right. Users have to pass the device explicitly. For example, that's what Huggingface Transformers are doing:
https://github.com/huggingface/transformers/blob/de306e8e14672dd8392b4bd344054a6a18de8613/src/transformers/video_utils.py#L601

correct me if I'm wrong: if a user is leaving the device to its default, i.e. VideoDecoder(..., device=None), then the existence of the XPU TorchCodec extension has zero effect, right (whether it's imported manually or automatically through the plugin mechanism)?

As of today TorchCodec handles device=None by calling torch.get_default_device() which by default is set to cpu. Thus, by default registering xpu with explicit import or implicitly will be unnoticed and have zero effect.

torchcodec/src/torchcodec/decoders/_video_decoder.py

Lines 223 to 224 in 48844c8

	if device is None:
	device = str(torch.get_default_device())

User can explicitly override the default device by calling torch.get_default_device("xpu") - after that TorchCodec will start to use xpu registered device (if it was registered implicitly or explicitly with TorchCodec).

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1151

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 8fc6410 with merge base 04a1e34 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[NicolasHug]

LGTM, I was a bit on the fence with this but the argument about passing --device from the shell CLI makes sense.

I made a few simplification to the tests.

[NicolasHug]

@dvrogozh can you confirm we can remove this if this PR is merged?

[dvrogozh]

The import torchcodec_xpu corresponds to the current released versions of torchcodec and torchcodec xpu plugin. Merging just this PR won't make autoloading work for XPU as respective change needs to be done on xpu plugin side as well (to register it for autoloading). Overall we will need to make new torchcodec and tochcodec xpu plugin releases with these changes to make everything work and formally remove this line.

I think that this should be OK to drop the line right now considering that this is development repository. In this way we will be heading to make instruction correct on the next approaching releases of torchcodec and xpu plugin. From my side I can commit for us doing the xpu plugin release in a short follow up after torchcodec one - 1-2 days timeframe.