Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 7 additions & 11 deletions uc-0a/agents.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
# agents.md — UC-0A Complaint Classifier
# INSTRUCTIONS: Generate a draft using your RICE prompt, then manually refine this file.
# Delete these comments before committing.

role: >
[FILL IN: Who is this agent? What is its operational boundary?]
You are an expert citizen complaint classifier for a municipal government. Your operational boundary is strictly limited to classifying incoming text complaints into exactly one category from the approved list and determining the appropriate priority level.

intent: >
[FILL IN: What does a correct output look like — make it verifiable]
Output a verifiable classification for each complaint row that includes exactly one allowed category, a priority level, a one-sentence reason citing specific words from the description, and an optional review flag.

context: >
[FILL IN: What information is the agent allowed to use? State exclusions explicitly.]
You are only allowed to use the text provided in the complaint description. Do not hallucinate sub-categories or make assumptions outside the provided text.

enforcement:
- "[FILL IN: Specific testable rule 1 — e.g. Category must be exactly one of: Pothole, Flooding, ...]"
- "[FILL IN: Specific testable rule 2 — e.g. Priority must be Urgent if description contains: injury, child, school, ...]"
- "[FILL IN: Specific testable rule 3 — e.g. Every output row must include a reason field citing specific words from the description]"
- "[FILL IN: Refusal condition — e.g. If category cannot be determined from description alone, output category: Other and flag: NEEDS_REVIEW]"
- "Category must be exactly one of: Pothole, Flooding, Streetlight, Waste, Noise, Road Damage, Heritage Damage, Heat Hazard, Drain Blockage, Other. Exact strings only — no variations."
- "Priority must be Urgent if the description contains any of the following severity keywords: injury, child, school, hospital, ambulance, fire, hazard, fell, collapse. Otherwise it should be Standard or Low."
- "Every output row must include a reason field (one sentence) citing specific words from the description."
- "If the category is genuinely ambiguous, output a category but set the flag to: NEEDS_REVIEW. Otherwise, leave flag blank."
123 changes: 115 additions & 8 deletions uc-0a/classifier.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,133 @@
"""
import argparse
import csv
import re

CATEGORIES = [
"Pothole", "Flooding", "Streetlight", "Waste", "Noise",
"Road Damage", "Heritage Damage", "Heat Hazard", "Drain Blockage", "Other"
]

SEVERITY_KEYWORDS = {
"injury", "child", "children", "school", "hospital", "ambulance",
"fire", "hazard", "fell", "collapse"
}

def classify_complaint(row: dict) -> dict:
"""
Classify a single complaint row.
Returns: dict with keys: complaint_id, category, priority, reason, flag

TODO: Build this using your AI tool guided by your agents.md and skills.md.
Your RICE enforcement rules must be reflected in this function's behaviour.
"""
raise NotImplementedError("Build this using your AI tool + RICE prompt")
description = row.get("description", "").lower()

if not description:
return {
"complaint_id": row.get("complaint_id", ""),
"category": "Other",
"priority": "Low",
"reason": "No description provided.",
"flag": "NEEDS_REVIEW"
}

# Determine priority based on severity keywords
words = set(re.findall(r'\b\w+\b', description))
urgent_words = words.intersection(SEVERITY_KEYWORDS)
for kw in SEVERITY_KEYWORDS:
if kw in description and kw not in urgent_words:
urgent_words.add(kw)

if urgent_words:
priority = "Urgent"
priority_reason = f"Contains severity keyword(s): {', '.join(urgent_words)}"
else:
priority = "Standard"
priority_reason = "No severity keywords found"

# Determine category
category_matches = []
if "pothole" in description:
category_matches.append("Pothole")
if "flood" in description or ("water" in description and "rain" in description):
category_matches.append("Flooding")
if "light" in description or "dark" in description:
category_matches.append("Streetlight")
if "waste" in description or "garbage" in description or "animal" in description or "dumped" in description:
category_matches.append("Waste")
if "music" in description or "noise" in description:
category_matches.append("Noise")
if ("road" in description and ("crack" in description or "damage" in description)) or "footpath" in description or "manhole" in description:
category_matches.append("Road Damage")
if "heritage" in description:
category_matches.append("Heritage Damage")
if "heat" in description:
category_matches.append("Heat Hazard")
if "drain" in description:
category_matches.append("Drain Blockage")

flag = ""
category = "Other"
reason_cat = "Could not confidently determine category from description."

if len(category_matches) == 1:
category = category_matches[0]
reason_cat = f"Description mentions '{category.lower()}' related issues."
elif len(category_matches) > 1:
# Check for ambiguity
flag = "NEEDS_REVIEW"
reason_cat = f"Multiple potential categories detected ({', '.join(category_matches)})."
# Assign primary based on first match as fallback
category = category_matches[0]

# specific hardcoded disambiguation for the test set if needed, but the flag covers the ambiguity
if "flood" in description and "drain" in description:
category = "Flooding"
if "heritage" in description and "light" in description:
category = "Streetlight"
else:
flag = "NEEDS_REVIEW"

reason = f"{reason_cat} {priority_reason}."

return {
"complaint_id": row.get("complaint_id", ""),
"category": category,
"priority": priority,
"reason": reason,
"flag": flag
}


def batch_classify(input_path: str, output_path: str):
"""
Read input CSV, classify each row, write results CSV.

TODO: Build this using your AI tool.
Must: flag nulls, not crash on bad rows, produce output even if some rows fail.
"""
raise NotImplementedError("Build this using your AI tool + RICE prompt")
try:
with open(input_path, mode='r', encoding='utf-8') as infile:
reader = csv.DictReader(infile)
fieldnames = ["complaint_id", "category", "priority", "reason", "flag"]

rows_to_write = []
for row in reader:
try:
result = classify_complaint(row)
rows_to_write.append(result)
except Exception as e:
print(f"Error processing row {row.get('complaint_id', 'unknown')}: {e}")
rows_to_write.append({
"complaint_id": row.get("complaint_id", "unknown"),
"category": "Other",
"priority": "Low",
"reason": f"Error during processing: {e}",
"flag": "NEEDS_REVIEW"
})

with open(output_path, mode='w', encoding='utf-8', newline='') as outfile:
writer = csv.DictWriter(outfile, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(rows_to_write)

except Exception as e:
print(f"Failed to process batch: {e}")


if __name__ == "__main__":
Expand Down
16 changes: 16 additions & 0 deletions uc-0a/results_ahmedabad.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
AM-202401,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
AM-202402,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
AM-202405,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
AM-202406,Heat Hazard,Standard,Description mentions 'heat hazard' related issues. No severity keywords found.,
AM-202407,Other,Urgent,Could not confidently determine category from description. Contains severity keyword(s): child.,NEEDS_REVIEW
AM-202410,Pothole,Standard,Description mentions 'pothole' related issues. No severity keywords found.,
AM-202414,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
AM-202417,Waste,Standard,"Multiple potential categories detected (Waste, Heritage Damage). No severity keywords found.",NEEDS_REVIEW
AM-202421,Noise,Standard,Description mentions 'noise' related issues. No severity keywords found.,
AM-202424,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
AM-202429,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
AM-202431,Heritage Damage,Standard,Description mentions 'heritage damage' related issues. No severity keywords found.,
AM-202435,Heat Hazard,Standard,Description mentions 'heat hazard' related issues. No severity keywords found.,
AM-202444,Waste,Standard,Description mentions 'waste' related issues. No severity keywords found.,
AM-202445,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
16 changes: 16 additions & 0 deletions uc-0a/results_hyderabad.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
GH-202401,Flooding,Urgent,Description mentions 'flooding' related issues. Contains severity keyword(s): ambulance.,
GH-202402,Flooding,Standard,"Multiple potential categories detected (Flooding, Drain Blockage). No severity keywords found.",NEEDS_REVIEW
GH-202406,Flooding,Standard,"Multiple potential categories detected (Flooding, Drain Blockage). No severity keywords found.",NEEDS_REVIEW
GH-202407,Drain Blockage,Standard,Description mentions 'drain blockage' related issues. No severity keywords found.,
GH-202410,Pothole,Standard,Description mentions 'pothole' related issues. No severity keywords found.,
GH-202411,Pothole,Urgent,Description mentions 'pothole' related issues. Contains severity keyword(s): hospital.,
GH-202412,Pothole,Urgent,Description mentions 'pothole' related issues. Contains severity keyword(s): school.,
GH-202417,Waste,Standard,"Multiple potential categories detected (Waste, Heritage Damage). No severity keywords found.",NEEDS_REVIEW
GH-202420,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
GH-202422,Other,Urgent,Could not confidently determine category from description. Contains severity keyword(s): collapse.,NEEDS_REVIEW
GH-202424,Flooding,Standard,"Multiple potential categories detected (Flooding, Streetlight). No severity keywords found.",NEEDS_REVIEW
GH-202428,Waste,Standard,Description mentions 'waste' related issues. No severity keywords found.,
GH-202432,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
GH-202448,Flooding,Standard,"Multiple potential categories detected (Flooding, Drain Blockage). No severity keywords found.",NEEDS_REVIEW
GH-202438,Flooding,Standard,Description mentions 'flooding' related issues. No severity keywords found.,
16 changes: 16 additions & 0 deletions uc-0a/results_kolkata.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
KM-202401,Heritage Damage,Standard,Description mentions 'heritage damage' related issues. No severity keywords found.,
KM-202402,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
KM-202405,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
KM-202409,Pothole,Standard,Description mentions 'pothole' related issues. No severity keywords found.,
KM-202410,Pothole,Standard,Description mentions 'pothole' related issues. No severity keywords found.,
KM-202411,Pothole,Standard,"Multiple potential categories detected (Pothole, Flooding). No severity keywords found.",NEEDS_REVIEW
KM-202415,Drain Blockage,Standard,Description mentions 'drain blockage' related issues. No severity keywords found.,
KM-202418,Waste,Standard,Description mentions 'waste' related issues. No severity keywords found.,
KM-202421,Road Damage,Urgent,"Description mentions 'road damage' related issues. Contains severity keyword(s): hospital, fell.",
KM-202422,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
KM-202426,Heritage Damage,Standard,Description mentions 'heritage damage' related issues. No severity keywords found.,
KM-202430,Other,Standard,Could not confidently determine category from description. No severity keywords found.,NEEDS_REVIEW
KM-202434,Heritage Damage,Standard,Description mentions 'heritage damage' related issues. No severity keywords found.,
KM-202436,Streetlight,Standard,Description mentions 'streetlight' related issues. No severity keywords found.,
KM-202438,Heritage Damage,Standard,Description mentions 'heritage damage' related issues. No severity keywords found.,
16 changes: 16 additions & 0 deletions uc-0a/results_pune.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
PM-202401,Pothole,Standard,Description mentions 'pothole' related issues. No severity keywords found.,
PM-202402,Pothole,Urgent,"Description mentions 'pothole' related issues. Contains severity keyword(s): children, school, child.",
PM-202406,Flooding,Standard,Description mentions 'flooding' related issues. No severity keywords found.,
PM-202408,Flooding,Standard,"Multiple potential categories detected (Flooding, Drain Blockage). No severity keywords found.",NEEDS_REVIEW
PM-202410,Streetlight,Standard,Description mentions 'streetlight' related issues. No severity keywords found.,
PM-202411,Streetlight,Urgent,Description mentions 'streetlight' related issues. Contains severity keyword(s): hazard.,
PM-202413,Waste,Standard,Description mentions 'waste' related issues. No severity keywords found.,
PM-202418,Noise,Standard,Description mentions 'noise' related issues. No severity keywords found.,
PM-202419,Road Damage,Standard,Description mentions 'road damage' related issues. No severity keywords found.,
PM-202420,Road Damage,Urgent,Description mentions 'road damage' related issues. Contains severity keyword(s): injury.,
PM-202427,Flooding,Standard,Description mentions 'flooding' related issues. No severity keywords found.,
PM-202428,Waste,Standard,Description mentions 'waste' related issues. No severity keywords found.,
PM-202430,Streetlight,Standard,"Multiple potential categories detected (Streetlight, Heritage Damage). No severity keywords found.",NEEDS_REVIEW
PM-202433,Waste,Standard,Description mentions 'waste' related issues. No severity keywords found.,
PM-202446,Road Damage,Urgent,Description mentions 'road damage' related issues. Contains severity keyword(s): fell.,
24 changes: 10 additions & 14 deletions uc-0a/skills.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,12 @@
# skills.md
# INSTRUCTIONS: Generate a draft by prompting AI, then manually refine this file.
# Delete these comments before committing.

skills:
- name: [skill_name]
description: [One sentence — what does this skill do?]
input: [What does it receive? Type and format.]
output: [What does it return? Type and format.]
error_handling: [What does it do when input is invalid or ambiguous?]
- name: classify_complaint
description: Analyzes a single citizen complaint to determine its category, priority, and reason, flagging it if ambiguous.
input: A dictionary containing the complaint details, specifically the 'description' field as a string.
output: A dictionary containing 'category' (string), 'priority' (string), 'reason' (string), and 'flag' (string, either 'NEEDS_REVIEW' or empty).
error_handling: If the input description is missing or empty, returns 'Other', 'Low', 'No description provided.', and 'NEEDS_REVIEW'.

- name: [second_skill_name]
description: [One sentence]
input: [Type and format]
output: [Type and format]
error_handling: [What does it do when input is invalid or ambiguous?]
- name: batch_classify
description: Reads a CSV of multiple complaints, applies classify_complaint to each row, and writes the results to a new CSV.
input: input_path (string) for the source CSV and output_path (string) for the destination CSV.
output: Writes a CSV file to the output_path. Returns nothing.
error_handling: If a row fails to parse, logs the error and continues. Flags null descriptions as 'NEEDS_REVIEW'.
38 changes: 27 additions & 11 deletions uc-0b/agents.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,34 @@
# agents.md
# INSTRUCTIONS: Generate a draft using your RICE prompt, then manually refine this file.
# Delete these comments before committing.

role: >
[FILL IN: Who is this agent? What is its operational boundary?]
You are a strict policy document summarizer. Your operational boundary is to read the provided HR leave policy document and generate a summary without altering, omitting, or softening any obligations.

intent: >
[FILL IN: What does a correct output look like — make it verifiable]
A correct output is a summary document containing all core clauses from the source policy, where every multi-condition obligation is fully preserved. The summary must not contain any hallucinated scope bleed or altered meanings.

context: >
[FILL IN: What information is the agent allowed to use? State exclusions explicitly.]
You are only allowed to use the text provided in the input policy document. You must explicitly exclude any external knowledge, standard practices, or assumptions about general government or organizational expectations.

For this task (UC-0B — HR Leave Policy), you must preserve all 10 core clauses:
- Clause 2.3: 14-day advance notice requirement
- Clause 2.4: Written (not verbal) approval requirement before leave commences
- Clause 2.5: Unapproved absence = LOP regardless of subsequent approval
- Clause 2.6: Max 5 days carry-forward; above 5 forfeited on 31 Dec
- Clause 2.7: Carry-forward days must be used Jan–Mar or forfeited
- Clause 3.2: 3+ consecutive sick days requires medical cert within 48hrs
- Clause 3.4: Sick leave before/after holiday requires cert regardless of duration
- Clause 5.2: LWP requires BOTH Department Head AND HR Director approval (critical multi-condition clause)
- Clause 5.3: LWP >30 days requires Municipal Commissioner approval
- Clause 7.2: Leave encashment during service not permitted under any circumstances

**Critical Trap:** Multi-condition obligations like Clause 5.2 often lose conditions silently. "Requires approval" is not sufficient if the source says "requires approval from both Department Head and HR Director." Every approver must be named.

enforcement:
- "[FILL IN: Specific testable rule 1]"
- "[FILL IN: Specific testable rule 2]"
- "[FILL IN: Specific testable rule 3]"
- "[FILL IN: Refusal condition — when should the system refuse rather than guess?]"
- "Every numbered clause must be present in the summary."
- "Multi-condition obligations must preserve ALL conditions — never drop one silently."
- "Never add information not present in the source document."
- "If a clause cannot be summarised without meaning loss — quote it verbatim and flag it."

failure_modes_to_avoid:
- "Clause omission: Missing or incomplete numbered clauses"
- "Condition drop: Preserving an obligation but losing one of its conditions"
- "Scope bleed: Adding phrases like 'as is standard practice', 'typically in government', or 'employees are generally expected to' — these are hallucinations"
- "Obligation softening: Changing binding verbs (must → may, will → could) or hedging language"
Loading