Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/changelog/3897.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add ``virtualenv-pep-723`` runner that reads dependencies and Python version from :PEP:`723` inline script metadata — no
need to duplicate them in tox config - by :user:`gaborbernat`.
40 changes: 40 additions & 0 deletions docs/explanation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,46 @@ invocation. No configuration changes will be needed.
**Change detection** works the same as for ``deps``: tox caches the resolved requirements. When packages are added, only
the new ones are installed. When packages are removed, the environment is recreated.

.. _pep723-explanation:

Inline script metadata (PEP 723)
================================

.. versionadded:: 4.52

:PEP:`723` lets Python scripts declare their dependencies and required Python version directly in comments at the top of
the file:

.. code-block:: python

# /// script
# requires-python = ">=3.12"
# dependencies = ["requests>=2.31", "rich"]
# ///

import requests
from rich import print

print(requests.get("https://httpbin.org/get").json())

The ``virtualenv-pep-723`` runner reads this metadata so that a tox environment needs only a ``runner`` and ``script``
key — no ``deps`` or ``base_python`` duplication. Both ``requires-python`` and ``dependencies`` are optional per the PEP
723 spec; omitting ``requires-python`` uses the host Python, and omitting ``dependencies`` installs nothing.

**How it differs from the default runner:** the ``virtualenv-pep-723`` runner skips ``PythonRun`` in the class
hierarchy. Config keys like ``deps``, ``extras``, ``dependency_groups``, and ``pylock`` are structurally absent — they
do not exist for this runner. Packaging is unconditionally disabled since scripts are not packages. Setting
``base_python`` explicitly is rejected to prevent conflicts with the script's ``requires-python`` specifier.

**Python version resolution:** the runner uses the normal Python discovery (env name factors, ``.python-version`` files,
or the host Python running tox). When ``requires-python`` is present, tox validates that the discovered Python satisfies
the constraint and fails with a clear error if it does not. This works correctly with free-threaded interpreters and
avoids forcing an unnecessarily old Python version.

**Plugin support:** the ``Pep723Mixin`` class at ``tox.tox_env.python.pep723`` contains all the PEP 723 logic
independent of the venv backend. Third-party plugins (e.g. tox-uv) can compose it with their own venv implementation
without duplicating code.

Open-ended range bounds
=======================

Expand Down
64 changes: 64 additions & 0 deletions docs/how-to/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -607,6 +607,70 @@ subcommand, use the ``run`` subcommand explicitly:

.. ------------------------------------------------------------------------------------------

.. _howto_pep723:

**********************
Run a PEP 723 script
**********************

.. versionadded:: 4.52

If you have a standalone Python script with :PEP:`723` inline metadata:

.. code-block:: python

# /// script
# requires-python = ">=3.12"
# dependencies = ["requests>=2.31", "rich"]
# ///

import requests
from rich import print

print(requests.get("https://httpbin.org/get").json())

You can run it through tox without duplicating the dependency list:

.. tab:: TOML

.. code-block:: toml

[env.fetch]
runner = "virtualenv-pep-723"
script = "tools/fetch.py"

.. tab:: INI

.. code-block:: ini

[testenv:fetch]
runner = virtualenv-pep-723
script = tools/fetch.py

Run it with ``tox r -e fetch``. Positional arguments are forwarded: ``tox r -e fetch -- --verbose``.

To override the default command (which runs the script), set ``commands`` as usual:

.. tab:: TOML

.. code-block:: toml

[env.fetch]
runner = "virtualenv-pep-723"
script = "tools/fetch.py"
commands = [["python", "-m", "pytest", "tests/"]]

.. tab:: INI

.. code-block:: ini

[testenv:fetch]
runner = virtualenv-pep-723
script = tools/fetch.py
commands = python -m pytest tests/

See :ref:`pep723-explanation` for how Python version resolution and dependency installation work.

.. _faq_custom_pypi_server:

.. _howto_custom_pypi_server:
Expand Down
28 changes: 27 additions & 1 deletion docs/reference/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1102,7 +1102,33 @@ Run
:version_added: 4.0.0

The tox execute used to evaluate this environment. Defaults to Python virtual environments, however may be
overwritten by plugins.
overwritten by plugins. Set to ``virtualenv-pep-723`` to use inline script metadata (see :ref:`script`).

.. conf::
:keys: script
:default: <empty string>
:version_added: 4.52

Path to a Python script with :PEP:`723` inline metadata, relative to :ref:`tox_root`. Only available when
``runner = virtualenv-pep-723``. The script's ``requires-python`` and ``dependencies`` fields drive the environment's
Python version and installed packages. ``commands`` defaults to running the script with ``{posargs}`` forwarded, but
can be overridden. See :ref:`pep723-explanation` for details.

.. tab:: TOML

.. code-block:: toml

[env.check]
runner = "virtualenv-pep-723"
script = "tools/check.py"

.. tab:: INI

.. code-block:: ini

[testenv:check]
runner = virtualenv-pep-723
script = tools/check.py

.. conf::
:keys: description
Expand Down
1 change: 1 addition & 0 deletions docs/tutorial/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -332,3 +332,4 @@ Now that you have a working tox setup, explore these topics:
with :ref:`virtualenv_spec`)
- :ref:`configuration` -- full configuration reference
- :ref:`cli` -- complete CLI reference
- :ref:`howto_pep723` -- run standalone scripts with inline dependencies (:PEP:`723`)
3 changes: 2 additions & 1 deletion src/tox/plugin/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from tox.config.loader import api as loader_api
from tox.session.cmd.run import parallel, sequential
from tox.tox_env import package as package_api
from tox.tox_env.python.virtual_env import runner
from tox.tox_env.python.virtual_env import pep723_runner, runner
from tox.tox_env.python.virtual_env.package import cmd_builder, pyproject
from tox.tox_env.register import REGISTER, ToxEnvRegister

Expand Down Expand Up @@ -59,6 +59,7 @@ def _register_plugins(self, inline: ModuleType | None) -> None:
internal_plugins = (
loader_api,
provision,
pep723_runner,
runner,
pyproject,
cmd_builder,
Expand Down
151 changes: 151 additions & 0 deletions src/tox/tox_env/python/pep723.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
"""PEP 723 inline script metadata support — shared logic for any venv backend."""

from __future__ import annotations

import logging
import re
import sys
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Final, cast

from packaging.specifiers import SpecifierSet
from packaging.version import Version

from tox.config.types import Command
from tox.tox_env.errors import Fail
from tox.tox_env.python.pip.req_file import PythonDeps
from tox.tox_env.python.runner import add_skip_missing_interpreters_to_core, add_skip_missing_interpreters_to_env
from tox.tox_env.runner import RunToxEnv

from .api import Python

if sys.version_info >= (3, 11): # pragma: >=3.11 cover
import tomllib
else: # pragma: <3.11 cover
import tomli as tomllib

if TYPE_CHECKING:
from pathlib import Path

from tox.config.main import Config
from tox.config.of_type import ConfigDynamicDefinition
from tox.tox_env.api import ToxEnvCreateArgs

_SCRIPT_METADATA_RE: Final = re.compile(
r"""
(?m)
^[#][ ]///[ ](?P<type>[a-zA-Z0-9-]+)$ # opening: # /// <type>
\s # blank line or whitespace
(?P<content> # TOML content lines:
(?:^[#](?:| .*)$\s)+ # each line starts with # (optionally followed by space + text)
)
^[#][ ]///$ # closing: # ///
""",
re.VERBOSE,
)


@dataclass(frozen=True)
class ScriptMetadata:
requires_python: str | None = None
dependencies: list[str] = field(default_factory=list)


class Pep723Mixin(Python, RunToxEnv):
"""Mixin providing PEP 723 script metadata support for any venv-backed runner.

Concrete runners compose this with a venv backend (VirtualEnv, UvVenv, etc.) and RunToxEnv.

"""

_script_metadata: ScriptMetadata | None

def __init__(self, create_args: ToxEnvCreateArgs) -> None:
self._script_metadata = None
super().__init__(create_args)

def register_config(self) -> None:
super().register_config()
self.conf.add_config(
keys=["script"],
of_type=str,
default="",
desc="path to Python script with PEP 723 inline metadata (relative to tox_root)",
)

def default_commands(conf: Config, env_name: str | None) -> list[Command]: # noqa: ARG001
if script := self.conf["script"]:
tox_root: Path = self.core["tox_root"]
args = ["python", str(tox_root / script)]
if (pos_args := conf.pos_args(None)) is not None:
args.extend(pos_args)
return [Command(args)]
return []

commands_def = cast("ConfigDynamicDefinition[list[Command]]", self.conf._defined["commands"]) # noqa: SLF001
commands_def.default = default_commands
add_skip_missing_interpreters_to_core(self.core, self.options)
add_skip_missing_interpreters_to_env(self.conf, self.core, self.options)

def _setup_env(self) -> None:
super()._setup_env()
if self._base_python_explicitly_set:
msg = "cannot set base_python with virtualenv-pep-723 runner; use requires-python in the script"
raise Fail(msg)
if script := self.conf["script"]:
tox_root: Path = self.core["tox_root"]
if not (tox_root / script).is_file():
msg = f"script file not found: {tox_root / script}"
raise Fail(msg)
metadata = self._get_script_metadata()
if metadata.requires_python:
info = self.base_python
py_version = Version(f"{info.version_info.major}.{info.version_info.minor}.{info.version_info.micro}")
if py_version not in SpecifierSet(metadata.requires_python):
msg = f"python {py_version} does not satisfy requires-python {metadata.requires_python!r}"
raise Fail(msg)
if getattr(self.options, "skip_env_install", False):
logging.warning("skip installing dependencies")
return
if metadata.dependencies:
root: Path = self.core["tox_root"]
requirements = PythonDeps(metadata.dependencies, root)
self._install(requirements, type(self).__name__, "deps")

def _get_script_metadata(self) -> ScriptMetadata:
if self._script_metadata is None:
if not (script := self.conf["script"]):
self._script_metadata = ScriptMetadata()
return self._script_metadata
tox_root: Path = self.core["tox_root"]
full_path = tox_root / script
if not full_path.is_file():
self._script_metadata = ScriptMetadata()
return self._script_metadata
self._script_metadata = _parse_script_metadata(full_path.read_text(encoding="utf-8"))
return self._script_metadata


def _parse_script_metadata(script: str) -> ScriptMetadata:
blocks = [(m.group("type"), m.group("content")) for m in _SCRIPT_METADATA_RE.finditer(script)]
script_blocks = [(t, c) for t, c in blocks if t == "script"]
if len(script_blocks) > 1:
msg = "multiple [script] metadata blocks found in script"
raise ValueError(msg)
if not script_blocks:
return ScriptMetadata()
content = script_blocks[0][1]
stripped = "".join(
line[2:] if len(line) > 1 and line[1] == " " else line[1:] for line in content.splitlines(keepends=True)
)
metadata = tomllib.loads(stripped)
return ScriptMetadata(
requires_python=metadata.get("requires-python"),
dependencies=metadata.get("dependencies", []),
)


__all__ = [
"Pep723Mixin",
"ScriptMetadata",
]
45 changes: 45 additions & 0 deletions src/tox/tox_env/python/virtual_env/pep723_runner.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
"""Concrete virtualenv-backed PEP 723 runner."""

from __future__ import annotations

from typing import TYPE_CHECKING

from tox.plugin import impl
from tox.tox_env.python.pep723 import Pep723Mixin
from tox.tox_env.runner import RunToxEnv

from .api import VirtualEnv

if TYPE_CHECKING:
from tox.tox_env.package import Package
from tox.tox_env.register import ToxEnvRegister


class Pep723Runner(Pep723Mixin, VirtualEnv, RunToxEnv):
@staticmethod
def id() -> str:
return "virtualenv-pep-723"

def _register_package_conf(self) -> bool: # noqa: PLR6301
return False

@property
def _package_tox_env_type(self) -> str:
raise NotImplementedError

@property
def _external_pkg_tox_env_type(self) -> str:
raise NotImplementedError

def _build_packages(self) -> list[Package]: # noqa: PLR6301
return []


@impl
def tox_register_tox_env(register: ToxEnvRegister) -> None:
register.add_run_env(Pep723Runner)


__all__ = [
"Pep723Runner",
]
Loading