Mudit Mathur - UC-RAG-implemented sentence-aware chunking and strict grounding enforcement.

uc-mcp/agents.md

@@ -13,20 +13,17 @@
 #    must state exact document scope
 
 role: >
-  [FILL IN: Who is this agent? What layer of the stack does it operate at?
-   Hint: an MCP server that exposes policy retrieval as a tool]
+  An MCP server that exposes policy retrieval as a tool. Operating at the Model Context Protocol layer, acting as a bridge between AI agents and the RAG server to enable standard tool discovery and execution.
 
 intent: >
-  [FILL IN: What does a correctly implemented MCP server produce?
-   Hint: JSON-RPC compliant responses, scoped tool description, correct refusals]
+  Correctly implement an MCP server that produces JSON-RPC 2.0 compliant responses, provides a precisely scoped tool description for `query_policy_documents`, and handles out-of-scope queries with correct refusal objects and application-level error markers.
 
 context: >
-  [FILL IN: What does this server have access to?
-   Hint: RAG server results only — no direct LLM calls, no outside knowledge]
+  Access to the RAG server results for specific policy documents: CMC HR Leave Policy, IT Acceptable Use Policy, and Finance Reimbursement Policy. No direct LLM calls or access to outside knowledge.
 
 enforcement:
-  - "[FILL IN: Tool description scope rule]"
-  - "[FILL IN: Refusal documentation rule]"
-  - "[FILL IN: inputSchema required field rule]"
-  - "[FILL IN: isError on failure rule]"
-  - "[FILL IN: HTTP 200 for all JSON-RPC responses rule]"
+  - "Tool description must state the exact document scope: CMC HR Leave Policy, IT Acceptable Use Policy, Finance Reimbursement Policy."
+  - "Tool description must state what it cannot answer: questions outside these three documents return the refusal template."
+  - "inputSchema must require `question` as a non-empty string."
+  - "Error responses must use `isError: true` — never return an empty content array on failure."
+  - "The server must return HTTP 200 for all JSON-RPC responses including errors — transport errors use HTTP 4xx/5xx, application errors use JSON-RPC error objects."

uc-mcp/llm_adapter.py

@@ -23,6 +23,11 @@
 """
 
 import os
+try:
+    from dotenv import load_dotenv
+    load_dotenv()
+except ImportError:
+    pass
 
 # ══════════════════════════════════════════════════════════════════════
 # GEMINI (DEFAULT)
@@ -42,7 +47,7 @@ def call_llm(prompt: str) -> str:
     try:
         import google.generativeai as genai
         genai.configure(api_key=api_key)
-        model = genai.GenerativeModel("gemini-1.5-flash")
+        model = genai.GenerativeModel("gemini-flash-latest")
         response = model.generate_content(prompt)
         return response.text
     except ImportError:

uc-mcp/mcp_server.py

@@ -23,17 +23,7 @@
 import os
 from http.server import HTTPServer, BaseHTTPRequestHandler
 
-# Import RAG — uses stub by default, swap to rag_server once yours works
-sys.path.insert(0, os.path.join(os.path.dirname(__file__), "../uc-rag"))
-try:
-    # Try participant's rag_server first
-    from rag_server import query as rag_query
-    print("[mcp_server] Using participant rag_server.py")
-except (ImportError, NotImplementedError):
-    # Fall back to stub
-    from stub_rag import query as rag_query
-    print("[mcp_server] Using stub_rag.py (fallback)")
-
+# RAG and LLM logic will be lazy-loaded in the handler to prevent startup hangs
 # Import LLM adapter
 from llm_adapter import call_llm
 
@@ -44,13 +34,10 @@
 TOOL_DEFINITION = {
     "name": "query_policy_documents",
     "description": (
-        # FILL IN: Describe exactly what this tool covers and what it does not.
-        # Bad:  "Answers questions about policies"
-        # Good: "Answers questions about CMC HR Leave Policy, IT Acceptable Use
-        #        Policy, and Finance Reimbursement Policy only. Returns cited
-        #        answers grounded in retrieved document chunks. Returns a refusal
-        #        for questions outside these three documents."
-        "[FILL IN: specific scope + what it refuses]"
+        "Answers questions about CMC HR Leave Policy, IT Acceptable Use "
+        "Policy, and Finance Reimbursement Policy only. Returns cited "
+        "answers grounded in retrieved document chunks. Returns a refusal "
+        "for questions outside these three documents."
     ),
     "inputSchema": {
         "type": "object",
@@ -75,11 +62,39 @@ def query_policy_documents(question: str) -> dict:
     - If RAG refuses (no chunks above threshold) → isError: True
     - If RAG raises exception → isError: True with error message
     """
-    raise NotImplementedError(
-        "Implement query_policy_documents using your AI tool.\n"
-        "Hint: call rag_query(question, llm_call=call_llm), "
-        "check result['refused'], format as MCP content response."
-    )
+    # Lazy-load RAG logic
+    rag_dir = os.path.join(os.path.dirname(__file__), "../uc-rag")
+    if rag_dir not in sys.path:
+        sys.path.insert(0, rag_dir)
+
+    try:
+        try:
+            from rag_server import query as rag_query
+        except (ImportError, AttributeError):
+            from stub_rag import query as rag_query
+
+        result = rag_query(question, llm_call=call_llm)
+        is_error = result.get("refused", False)
+
+        return {
+            "content": [
+                {
+                    "type": "text",
+                    "text": result.get("answer", "No answer provided.")
+                }
+            ],
+            "isError": is_error
+        }
+    except Exception as e:
+        return {
+            "content": [
+                {
+                    "type": "text",
+                    "text": f"Error querying policy documents: {str(e)}"
+                }
+            ],
+            "isError": True
+        }
 
 
 # ── SKILL: serve_mcp ─────────────────────────────────────────────────────────
@@ -95,12 +110,58 @@ class MCPHandler(BaseHTTPRequestHandler):
     """
 
     def do_POST(self):
-        raise NotImplementedError(
-            "Implement do_POST using your AI tool.\n"
-            "Hint: read Content-Length, parse JSON body, "
-            "dispatch on method, write JSON-RPC response.\n"
-            "Return HTTP 200 for all JSON-RPC responses including errors."
-        )
+        content_length = int(self.headers.get('Content-Length', 0))
+        post_data = self.rfile.read(content_length)
+
+        try:
+            request = json.loads(post_data)
+        except json.JSONDecodeError:
+                self.send_json_rpc_error(None, -32700, "Parse error")
+                return
+
+        request_id = request.get("id")
+        method = request.get("method")
+        params = request.get("params", {})
+
+        if method == "tools/list":
+            self.send_json_rpc_result(request_id, {"tools": [TOOL_DEFINITION]})
+        elif method == "tools/call":
+            tool_name = params.get("name")
+            arguments = params.get("arguments", {})
+
+            if tool_name == "query_policy_documents":
+                question = arguments.get("question")
+                if not question:
+                    self.send_json_rpc_error(request_id, -32602, "Invalid params: 'question' is required")
+                else:
+                    result = query_policy_documents(question)
+                    self.send_json_rpc_result(request_id, result)
+            else:
+                self.send_json_rpc_error(request_id, -32601, f"Method not found: {tool_name}")
+        else:
+            self.send_json_rpc_error(request_id, -32601, f"Method not found: {method}")
+
+    def send_json_rpc_result(self, request_id, result):
+        response = {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": result
+        }
+        self.send_response(200)
+        self.send_header("Content-Type", "application/json")
+        self.end_headers()
+        self.wfile.write(json.dumps(response).encode("utf-8"))
+
+    def send_json_rpc_error(self, request_id, code, message):
+        response = {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "error": {"code": code, "message": message}
+        }
+        self.send_response(200)
+        self.send_header("Content-Type", "application/json")
+        self.end_headers()
+        self.wfile.write(json.dumps(response).encode("utf-8"))
 
     def log_message(self, format, *args):
         # Suppress default HTTP logging — use print for clarity
@@ -115,10 +176,13 @@ def main():
     args = parser.parse_args()
 
     # Verify RAG index exists
-    db_path = os.path.join(os.path.dirname(__file__), "../uc-rag/stub_chroma_db")
-    if not os.path.exists(db_path):
-        print("[mcp_server] WARNING: RAG index not found.")
-        print("[mcp_server] Run first: python3 ../uc-rag/stub_rag.py --build-index")
+    db_root = os.path.join(os.path.dirname(__file__), "../uc-rag")
+    participant_db = os.path.join(db_root, "chroma_db")
+    stub_db = os.path.join(db_root, "stub_chroma_db")
+
+    if not os.path.exists(participant_db) and not os.path.exists(stub_db):
+        print("[mcp_server] WARNING: RAG index not found (neither chroma_db nor stub_chroma_db).")
+        print("[mcp_server] Run first: python rag_server.py --build-index")
         print("[mcp_server] Starting anyway — queries will fail until index is built.")
 
     server = HTTPServer(("localhost", args.port), MCPHandler)

uc-mcp/skills.md

@@ -12,13 +12,13 @@
 
 skills:
   - name: query_policy_documents
-    description: "[FILL IN]"
-    input: "[FILL IN: question string]"
-    output: "[FILL IN: MCP content format — content array + isError]"
-    error_handling: "[FILL IN: what happens when RAG refuses or raises exception]"
+    description: "Answers questions about CMC HR Leave Policy, IT Acceptable Use Policy, and Finance Reimbursement Policy only. Returns cited answers grounded in retrieved document chunks."
+    input: "question (string)"
+    output: "MCP content format: {'content': [...], 'isError': bool}"
+    error_handling: "Returns isError: true with a refusal message if the RAG query is out-of-scope or fails."
 
   - name: serve_mcp
-    description: "[FILL IN]"
-    input: "[FILL IN: HTTP POST with JSON-RPC body]"
-    output: "[FILL IN: JSON-RPC 2.0 response, always HTTP 200]"
-    error_handling: "[FILL IN: unknown method → -32601, malformed request → -32700]"
+    description: "Defines and manages an HTTP server providing an MCP-compliant JSON-RPC interface."
+    input: "HTTP POST with JSON-RPC 2.0 body"
+    output: "JSON-RPC 2.0 response, always HTTP 200 for application logic."
+    error_handling: "Returns JSON-RPC error -32601 for unknown methods, -32700 for parse errors, and -32600 for invalid requests."

uc-mcp/test_client.py

@@ -36,7 +36,7 @@ def jsonrpc_call(port: int, method: str, params: dict = None, req_id: int = 1) -
         method="POST",
     )
     try:
-        with urllib.request.urlopen(req, timeout=10) as resp:
+        with urllib.request.urlopen(req, timeout=30) as resp:
             return json.loads(resp.read().decode("utf-8"))
     except urllib.error.URLError as e:
         print(f"[ERROR] Could not connect to server on port {port}: {e}")

uc-rag/agents.md

@@ -1,31 +1,25 @@
 # agents.md — UC-RAG RAG Server
-# INSTRUCTIONS:
-# 1. Open your AI tool
-# 2. Paste the full contents of uc-rag/README.md
-# 3. Use this prompt:
-#    "Read this UC README. Using the R.I.C.E framework, generate an
-#     agents.md YAML with four fields: role, intent, context, enforcement.
-#     Enforcement must include every rule listed under
-#     'Enforcement Rules Your agents.md Must Include'.
-#     Output only valid YAML."
-# 4. Paste the output below, replacing this placeholder
-# 5. Check every enforcement rule against the README before saving
 
 role: >
-  [FILL IN: Who is this agent? What is its operational boundary?
-   Hint: a retrieval-augmented policy assistant for city staff]
+  A retrieval-augmented policy assistant for city staff. Your boundary is limited
+  strictly to providing information found within official municipal policy
+  documents related to HR, IT, and Finance.
 
 intent: >
-  [FILL IN: What does a correct output look like?
-   Hint: answer + cited chunks + refusal when not covered]
+  To provide accurate, cited answers to policy queries. When a query is covered
+  by policy, the output must include a clear answer followed by the document
+  name and chunk index. When a query is not covered, the output must strictly
+  follow the refusal template.
 
 context: >
-  [FILL IN: What sources may the agent use?
-   Hint: retrieved chunks only — no general knowledge]
+  Use only the information present in the retrieved chunks provided by the
+  RAG system. You are forbidden from using general knowledge or external
+  information to supplement your answers.
 
 enforcement:
-  - "[FILL IN: Chunk size rule]"
-  - "[FILL IN: Citation rule]"
-  - "[FILL IN: Similarity threshold + refusal rule]"
-  - "[FILL IN: Context grounding rule]"
-  - "[FILL IN: Cross-document rule]"
+  - "Chunk size must not exceed 400 tokens; never split mid-sentence."
+  - "Every answer must cite the source document name and chunk index (e.g., [policy_hr_leave, Chunk 2])."
+  - "If no retrieved chunk scores above similarity threshold 0.6, output the refusal template exactly."
+  - "Answer must use ONLY information present in the retrieved chunks; never add context from outside the set."
+  - "If a query spans two documents, retrieve and cite from each separately; never merge chunks from different documents into a single answer."
+  - "Refusal template: 'This question is not covered in the retrieved policy documents. Retrieved chunks: [list chunk sources]. Please contact the relevant department for guidance.'"