Open source has a growing problem with LLM generated issues
We've seen a slight uptick in pull-requests and bug reports which appear to be LLM-generated, so it's probably about time to come to a decision on what we should and should not accept and document this somewhere (presumably in CONTRIBUTING.md).
My personal opinion is we shouldn't accept anything LLM-generated, but this is probably not the common position of most @opencontainers/runc-maintainers, so we should probably consider LLM-generated code and issues separately.
IMHO, we should close all LLM-generated issues as spam, because even if they are describing real issues the entire issue description contains so much unneeded (and probably incorrect) information that it'd be better if they just provided their LLM prompt as an issue instead. More importantly, when triaging bugs we have to assume that what the user has written did actually happen to them, but with LLM-generated issues -- who knows whether the description is actually describing something real? (See #4982 and #4972 as possible examples of LLM-generated bug reports.)
For LLM-generated code, I think the minimum bar should be that the submitter needs to be able to respond to review requests in their own words (i.e., they understand what their patch does and was able to write the code themselves). (#4940 and #4939 was the most recent example of this I can think of, and I'm not convinced the submitter would've cleared this bar.)
(FWIW, my view is that LLM-generated code cannot fulfil the requirements of the DCO and so we shouldn't accept it for the legal reasons alone, but I appreciate this is a minority view.)
For reference, Incus added add a note to their CONTRIBUTING.md earlier this year, banning all LLM usage.
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0