-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Establish Automation/AI policy #15984
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -68,6 +68,7 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy). | |
|
|
||
| * Make sure to have [a clean history of commits on your branch by using rebase](https://www.digitalocean.com/community/tutorials/how-to-rebase-and-update-a-pull-request). | ||
| * [Mark the pull request as draft](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request) if you're not done with the changes. | ||
| * Review the **Automation/AI Policy** below. | ||
|
|
||
| 6. Do not expect your pull request to be reviewed immediately. | ||
| Nix maintainers follow a [structured process for reviews and design decisions](https://github.com/NixOS/nix/tree/master/maintainers#project-board-protocol), which may or may not prioritise your work. | ||
|
|
@@ -87,6 +88,69 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy). | |
|
|
||
| 7. If you need additional feedback or help to getting pull request into shape, ask other contributors using [@mentions](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams). | ||
|
|
||
| ## Automation/AI policy | ||
|
|
||
| Every contribution to Nix and related development venues, including code, documentation, and communication on GitHub and Matrix, must have a responsible person in the loop who is accountable for that contribution and reviews it before submission, and must transparently disclose any non‐trivial use of automation to produce it, including but not limited to LLM‐based AI tools. | ||
|
|
||
| Human communication must remain human. Pull request / issue descriptions and comments, documentation, commit messages, and code comments must be human-authored. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might be worth clarifying that typo/grammar checking is of course automatable (it's large Language models after all).
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Amended the exemptions section with a mention of this. |
||
|
|
||
| The following sections give more detail. | ||
|
|
||
| ### Scope | ||
|
|
||
| Any use of automated tools to generate non‐trivial amounts of output as part of a contribution, in whole or in part, verbatim or edited, is covered by this policy, except as listed in the Exemptions section. | ||
| Both LLM‐based AI tools and hand‐written automation are covered. | ||
| Contributions include code and documentation in commits, commit messages, pull request summaries and reviews, issue and vulnerability reports, GitHub comments, Matrix messages, and Discourse posts. | ||
| The covered venues are the GitHub repositories for Nix and related projects under the jurisdiction of the Nix team, Matrix rooms that are focused on development of those projects, and Discourse topics about Nix development. | ||
|
|
||
| PRs that seek to address issues that appear easier to fix, such as those marked with [good first issue](https://github.com/NixOS/nix/labels/good%20first%20issue), are held to the same standard as other issues. | ||
| Just because the problem seems "easy" and more likely to be successfully fixed by an unsupervised agent does not mean bending the rules is permissible. | ||
| Even the most trivial changes still must have a responsible person in the loop. | ||
|
|
||
| ### Accountability | ||
|
|
||
| Everyone who submits a contribution to Nix is responsible for it, regardless of the use of automated tooling. | ||
| Before submission, they must establish a reasonable level of understanding of the contribution and expectation of its correctness. | ||
| A contributor submitting a contribution intended for inclusion in Nix is also responsible for ensuring that it is [appropriately licensed](https://github.com/NixOS/nix/blob/master/COPYING) and credited, and not encumbered by any incompatible copyright. | ||
|
|
||
| When output from automated tooling is used in contributions, a contributor must establish confidence in that output. | ||
| This can be achieved by establishing confidence in the correctness of the tooling’s logic, manual review of the included output, or using further automation to verify the output (e.g. regression tests that surface the issue before a bugfix commit). | ||
| As the inner workings of LLM‐based AI tools cannot be sufficiently understood at present, only the latter two options are available when those are used; vibe coding without review is not permitted. | ||
|
|
||
| This policy applies equally to any further discussion of a contribution. | ||
| Comments and reviews must separately satisfy the same requirements of understanding, review, and disclosure. | ||
| Contributors are expected to be able to answer questions about their contribution and respond to feedback appropriately, **without simply forwarding messages back and forth to automated tools**. | ||
|
|
||
| It is not permitted to submit automated contributions without any manual review or intervention, outside of standard community automation. | ||
| Automation without any manual review must not be used as the sole arbiter of whether to merge a change. | ||
|
|
||
| ### Transparency | ||
|
|
||
| All covered use of automated tooling for a contribution must be disclosed as part of that contribution. | ||
|
|
||
| In the case of LLM‐based AI tooling used for commits, this **must** be in the form of an `Assisted-by:` Git commit trailer, including at least the tool name and the primary model name and version used for the contribution. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since we also have a few Anthropic contributors in our community, they might be not always allowed to disclose what exact model they are using. But I suppose they could also just provide the name of the latest released model?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good question whether using the released version for the final implementation to be submitted is a strong constraint or not really anymore (in the sense of using an undisclosed model to find relevant places and check that the change is doable, then ask the released model to do the final patch by doing this and that change in that place — this would only ask for the released model version under the policy, as far as I can tell).
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think for that it's simply okay to specify that they're using an unreleased model, I can clarify this.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So Anthropic policy is to put in something like this: |
||
| A `Co-authored-by:` trailer does not satisfy this policy. | ||
|
|
||
| Any adequate form of disclosure is permitted for other kinds of tooling and contribution. | ||
| Pull request summaries and review comments must be disclosed separately to commits. | ||
|
|
||
| ### Exemptions | ||
|
|
||
| The following situations are fully or partially exempt: | ||
|
|
||
| * Use of standard deterministic editor/IDE/formatter/text transformation tooling to produce changes that the author manually reviews and understands is exempt, including inline "auto‐completion" (even if LLM‐based) of short, rote snippets of text that do not contribute anything beyond boilerplate the author would have written anyway, and spelling and grammar checkers. | ||
|
|
||
| * Use of standard community automation is exempt (e.g. dependabot). | ||
|
|
||
| * Use of AI tools for research, testing, debugging, or private review is out of scope, if no substantial amount of their output is included in the resulting contribution. | ||
| However, if these tools had a significant technical influence on your contribution, you are still responsible for it per the Accountability section, and are expected to disclose this where relevant. | ||
|
|
||
| * Use of machine translation is exempt from the requirement to understand the translated output. | ||
| However, the requirements of appropriate confidence in the original text, responsibility, and disclosure still apply, and you are encouraged to additionally include the original untranslated contribution. | ||
|
|
||
| * Use of automation in a contribution clearly marked as not being ready for merge (e.g. a draft pull request) is exempt from the requirement for full self‐review, as long as some amount of review has been done and it is expected that the requirements will be met by the time it is marked as ready. | ||
| This does not waive any other requirement. | ||
|
|
||
| ## Making changes to the Nix manual | ||
|
|
||
| The Nix reference manual is hosted on https://nix.dev/manual/nix. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this lead to the AI policy instead? It might be a bit confusing to ask people to look at contributing docs when filing issues/bugreports
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think so, the first sections in the contributing guide are actually about reporting bugs and security vulnerabilities, so it seems relevant.