Proposal: scripts and hooks improvements

[noirbizarre]

Hi 👋🏼

I have been thinking a lot about this one, but never had the time to share and implement it.
I contributed a lot to scripts handling that I have been extensively using since, and I'd like to share my feedback and an evolution proposal from this experience.

Current state

As of today pre and post scripts and hooks are defined like this:

• name convention:
  • pre_<script> and post_<script> for scripts
  • pre_<signal> and post_<script> for hooks
• they are full tasks with special meaning/treatment

They come with documented drawbacks:

• they can't receive arguments
• they are subject to name conflicts

As well as non-documented ones, but obvious when you use them:

• cannot be grouped or sorted properly, with dot-syntax or table syntax
• are invisible in the schema (no discovery, no completion and no validation possible)
• reusing an existing task is very verbose

At the same time, composite tasks are great, but they suffer (in my opinion) a few issues:

• each task in a composite is considered a cmd unless a task matching the name exists
• you need to use an array even when referencing a single task

Proposal

My idea is to make a few changes, not radical ones, to make this both more powerful and flexible.

pre and post as attributes

The first change I'd like to make is to formalize pre and post tasks or hooks as attribute instead of name convention.

This would allow having dedicated schema entries with completion, validation and discoverability.

But it would also make syntax more clean and logical. As an example, here are the same scripts definitions in the current syntax and the one I propose, each time as object syntax, dot syntax or table syntax

Object syntax

Current:

pre_my_task = "do something before"
my_task = {shell = "do something", help = "My Task"}
post_my_task = "do something after"

Proposal:

my_task = {shell = "do something", help = "My Task", pre = "do something before", post = "do something after"}

I think this is the form where it does not shine, but I don't use much this form because it already has the same drawback on other attributes (help, env, env_file...):

my_task = {shell = "do something", help = "A sentence as description will spread", env = {VAR = "You just reached max line length"}}

Dot syntax

This is the syntax I use the more.

Current:

my_task.help = "My Task"
my_task.shell = "do something"
pre_my_task = "do something before"
post_my_task = "do something after"

Proposal:

my_task.help = "My Task"
my_task.shell = "do something"
my_task.pre = "do something before"
my_task.post = "do something after"

As you can see, grouping is more obvious.

Table syntax

Current:

[tool.pdm.scripts]
pre_my_task = "do something before"
post_my_task = "do something after"

[tool.pdm.scripts.my_task]
help = "My Task"
shell = "do something"

Proposal:

[tool.pdm.scripts.my_task]
help = "My Task"
shell = "do something"
pre = "do something before"
post = "do something after"

With this one, the grouping drawback is even more obvious

pre and post as composites

pre and post hooks being tasks is a bit overkill because:

• they don't benefit from associated tooling (they are hidden from listing, their help is ignored ...)
• it requires some extra round trips in the code to handle them

Making them composite would provide the following benefit:

• same code to run them
• could benefit from arguments and interpolation

inline composite tasks

Currently, composite tasks are very practical, but they are ambiguous.
The algorithm is the following:

• look for a task matching the first word
• execute the task if found
• treat it as a command if not

You can't define a shell script or call script, they need to be defined as separate plain task.

My proposal is to allow an inline task object syntax:

my_task.composite = [
    "existing:task {args}",
    { shell = "something with $ENV" },
    { call = "my.package:method"},
]

To be consistent and explicit, I would:

• add a use attribute for explicit task reuse
• explicitly says that string syntax is a shortcut for use

This would make this syntax equivalent:

my_task.composite = [
    { use = "existing:task {args}" },
    { shell = "something with $ENV" },
    { call = "my.package:method"},
]

Combined with the previous proposition, it would allow:

my_task.pre = [
     "pre-task {args}"
]
my_task.composite = [
    "existing:task {args}",
    { shell = "something with $ENV" },
    { call = "my.package:method"},
]

Bonus: Syntactic sugar for single element composite

I often use composite with a single entry just to benefit from reuse:

cover.composite = ["test --with --extra args"]

As there is currently no case where composite accept a single entry (string or object as defined above),
I would allow single entry composite to remove the array:

cover.composite = "test --with --extra args"

or with the previous proposition:

cover.composite = { use = "test --with --extra args" }

The hooks ?

Same as above for everything, meaning you would write:

build.pre = "existing:task"
build.post = [
    { call = "my.module:function" },
    { shell = "something --with $ENV" },
]

It would give the following benefits:

• they could also benefit from string interpolation
• they can reuse tasks
• it removes the naming constraints while only having collision on the event name (that we can explicitly disallow)
• it is possible to explicitly have them in the schema, lint them and prevent overriding them (e.g. you can't define <signal>.cmd, <signal>.shell, <signal>.call or <signal>.composite, only .pre and .post)

Proposal migration path

If this proposal is accepted, I suggest naming the new section tasks instead of scripts to avoid keeping naming constraint on pre_/post_ as well as being able to properly detect syntax version and also provide a dedicated schema.
It would be consistent with the fact they are already called tasks in the code base and I think the wording task is more representative as scripts might not be scripts. From the user point of view, they are tasks to execute.

The way I see it, there are 2 possible paths:

Deprecation period

• warn for deprecation if a tool.pdm.scripts is found
• provide a pdm fix which can migrate tool.pdm.scripts to tool.pdm.tasks
• remove tool.pdm.scripts support after the deprecation period

This is the smooth migration path, but I think it's harder to maintain as during the deprecation period, it needs to support both syntax so most probably it will be more complex to test, and keep a lot more code. Plus, it means people might report bugs for both syntax during this period.

Forced migration

• Stop execution if a tool.pdm.scripts is found and ask for migration
• provide a pdm fix which can migrate tool.pdm.scripts to tool.pdm.tasks

This is the one I prefer as it is simpler to handle, represent way less maintenance, we can update the schema without having support for both format...

---

There it is ! What do you think about it ? Is this proposition worth it ? What would you choose or do differently ? Should I submit a pull request for it ?