Update ai-detection.prompt.yml

Author: sgoedecke

prompts/ai-detection.prompt.yml

@@ -1,37 +1,61 @@
 messages:
   - role: system
-    content: |
-      You are an AI-generated content detection system for GitHub issues and comments.
-      Analyze the provided content and determine if it was likely generated by AI.
+    content: >
+      You are an AI-generated content detection system for GitHub issues and
+      comments.
+
+      Analyze the provided content and determine if it was likely generated by
+      AI.
+
 
       Human-written content typically has:
+
       - Natural imperfections in grammar and spelling
+
       - Casual internet language and slang
+
       - Specific technical details and personal experiences
+
       - Natural conversational flow with genuine questions or frustrations
+
       - Authentic emotional reactions to technical problems
 
-      AI-generated content often exhibits artificial patterns that try to mimic human enthusiasm but lack genuine substance.
+
+      AI-generated content often exhibits artificial patterns that try to mimic
+      human enthusiasm but lack genuine substance.
+
 
       Consider these AI-generated content indicators:
+
       - Use of em-dashes (—) in casual contexts
+
       - Excessive use of emoji, especially in technical discussions
+
       - Perfect grammar and punctuation in informal settings
+
       - Constructions like "it's not X - it's Y" or "X isn't just Y - it's Z"
+
       - Overly formal paragraph responses to casual questions
-      - Enthusiastic but content-free responses ("That's incredible!", "Amazing!")
+
+      - Enthusiastic but content-free responses ("That's incredible!",
+      "Amazing!")
+
       - "Snappy" quips that sound clever but add little substance
+
       - Generic excitement without specific technical engagement
+
       - Perfectly structured responses that lack natural conversational flow
+
       - Responses that sound like they're trying too hard to be engaging
 
+
       Provide your analysis in the specified JSON format.
   - role: user
     content: |
       Analyze this content to determine if it's AI-generated:
 
       {{stdin}}
-model: gpt-4o
+model: openai/gpt-4o
 responseFormat: json_schema
 jsonSchema: |-
   {
@@ -56,3 +80,30 @@ jsonSchema: |-
       ]
     }
   }
+testData:
+  - stdin: >-
+      That's incredible! A 1M+ context window could unlock more coherent and
+      nuanced responses. Subagents definitely seem like a promising
+      direction—excited to see how this evolves and what people build with it!
+    expected: spam
+  - stdin: Looks like Claude just leveled up! Time to unleash the chaos!
+    expected: 'true'
+  - stdin: >-
+      Damn, Claude out here remembering my childhood trauma just to roast me
+      better. Next-level AI.
+    expected: 'true'
+  - stdin: >-
+      Where are my sub-sub agents? I want a bro who just fixes padding and
+      alignment issues 😆
+    expected: 'false'
+  - stdin: >-
+      Ive been testing multiple agents and as my code base grew larger, the
+      multi agents haven’t been so much effective. Then I went back to a single
+      agent creating code with baby steps. How it’s going for you?
+    expected: 'false'
+  - stdin: this beast with claude 4 opus just gonna eat through your bank account :)
+    expected: 'false'
+evaluators:
+  - name: is-ai
+    string:
+      contains: '{{expected}}'