On allowing AI generated code.

[hperrin]

I'd like to continue the conversation I joined at this point:

#1509 (comment)

If the projects want to disallow AI generated contributions, may I suggest this policy:

https://sciactive.com/human-contribution-policy/

It's released as CC BY-SA, so it can be adapted to fit a team's specific needs.

I tried to make it very clear in the definitions of "project", "contribution", "AI generated", etc. to make it as unambiguous as possible.

There is also an extensive Reasoning section that should help resolve any arguments or disputes in pull requests. There are links to supporting evidence and relevant quotes from the US Copyrighy Office's guidance on AI in the More Info section.

[hperrin]

I'd like to link to this conversation on the Lutris platform:

lutris/lutris#6530 (reply in thread)

I think this bug, and the bug disclosed in a later comment, are good illustrations of how AI generated code can have very difficult to spot bugs that make it through code review.

These bugs would probably have been either completely avoided or found right away by a human author, since the human author would be familiar with the systems being used. The AI doesn't truly understand these systems, so it writes code that looks correct, but is actually wrong. In these cases, the specific circumstances of hitting the bug were never actually tested, and they were not caught in the review process.

I caught these two bugs in just the four commits I looked at. It would seem that the AI generated code is quite prone to introducing bugs, and those bugs are very difficult to spot in the review process.

[hperrin]

In the case of Lutris, the maintainer has decided that the introduction of these bugs is an acceptable tradeoff, and gives reasoning that the project has shipped worse bugs in the past.

That might be fine for a project like Lutris, that is a hobby project meant for gamers, but is that an acceptable risk for a project as vitally important to many production and mission critical applications as Node.js and the other projects under the OpenJS Foundation?

My comment here is particularly relevant:

One of those bugs would have caused an infinite loop of copying a file over and over again, eventually eating up all of the user's disk space and crashing the app, possibly their OS if left unattended. Describing this as a "subtle and not so critical bug" seems disingenuous to me. These are just the two I've found, because a. I've only looked at four commits, and b. I'm not even a Python or Wine dev. Who knows how many more there are.

Before I tried to look, I honestly didn't even know if I would find any. I was kind of just taking your word for it that the AI tools were really good quality, and I was more concerned with the legal aspects of AI generated code. But after seeing those commits, it's apparent to me that AI code generators have not improved nearly enough to be considered safe to use in my book.

If you genuinely maintain that you were asking for these examples because you were legitimately concerned about the quality, then I'll have to take your word for it. I feel like I've shown that the quality of the code Claude produces is poor. Poor enough that I wouldn't consider using it, and I wouldn't trust the code from someone who does. Edge cases matter, especially when the install base is as big as Lutris' is. I hope you'll reconsider whether you will continue to use AI.

[hperrin]

Not to mention the legal issues surrounding the non-copyrightability and potential for copyright infringement of AI generated code. I've outlined those reasons here:

https://sciactive.com/human-contribution-policy/#Reasoning

These were also discussed in the thread linked above:

#1509