Skip to content

06-home-assignments.md

Latest commit

 

History

History
300 lines (234 loc) · 30.8 KB

06-home-assignments.md

File metadata and controls

300 lines (234 loc) · 30.8 KB
Raw

Home Assignments

Take-home assignments, paid work trials, and asynchronous assessments for AI/ML engineering roles. Based on 1,765 job descriptions, 100+ GitHub repos of actual candidate submissions, and practitioner reports.

Of the 51 companies with disclosed interview processes, 17 (33%) include a take-home or asynchronous assignment. An additional 5 companies use paid work trials instead. Analysis of 100+ GitHub repos (Q4 2025 / Q1 2026) shows what companies actually ask:

  • RAG systems (40%+) - document upload, vector databases, citation support
  • Agentic systems (30%+) - tool-calling, multi-step reasoning, multi-agent orchestration
  • Conversational AI (20%+) - chatbots, live chat agents, voice assistants
  • Document processing (15%) - PDF parsing, data extraction, marksheet extraction
  • LLM-as-judge evaluation (10%+) - build a system then evaluate it with another LLM

Format

Asynchronous assignments completed on your own time, typically with a deadline of 2-7 days. You submit code, a writeup, or a working prototype, then discuss your solution in a follow-up interview. 1

Typical assignment formats:

  • Take-home coding project (most common) - build something from scratch, typically 2-4 hours of actual work. "Not code golf" - companies emphasize decision-making and clarity over cleverness
  • Defence round - present and defend your solution in a 45-90 minute walkthrough interview

Only 1 company explicitly allows AI tools in take-homes ("AI tools are explicitly allowed and encouraged"). No company explicitly bans AI for take-home assignments (bans only apply to live interviews). Most don't mention AI tool policy at all.

Assignment Examples

Actual take-home assignments from candidate reports, job listings, and 100+ GitHub repos of real submissions.

RAG and Document Q&A

The most common assignment type (40%+ of GitHub submissions).

  • Build a RAG chatbot that ingests PDFs/documents, creates embeddings in a vector DB, and answers questions with citations. Must respond "I don't have that information" when answer is unavailable. Answers must come strictly from retrieved context (10+ candidate submissions across 5+ companies). 2 3 4 5 6 7
  • Build a policy document RAG assistant with mandatory source citations for every answer. Return safe fallbacks for out-of-scope questions. Comes with a 7-question evaluation set across 3 categories (answerable, partially answerable, unanswerable). 8
  • Build a document Q&A system with citation tracking that handles multi-hop questions (questions requiring information from multiple documents or sections to answer). 9
  • Build a live chat agent grounded in FAQ knowledge base. Model must answer only from known FAQ data. 10 11
  • Design a customer support chatbot using RAG with open-source models. Requirements: 100+ concurrent users, <2 second latency, grounded in company docs, analytics tracking. 12
  • Build a CLI tool for summarizing long PDFs with configurable models and chunking strategies. 1
  • Refactor an existing messy RAG application into a clean architecture. Preserve all external behaviors (exact API endpoints), eliminate global mutable state, ensure testability without requiring running services (5+ candidate submissions for one company). 13 14 15 16 17
  • Build an agentic RAG system for government documents. 100% open-source required (Ollama + CrewAI + pgvector). Must integrate with OpenWebUI. Evaluated using RAGAS metrics (faithfulness, answer relevancy, context precision, context recall). 18

Agents and Tool-Calling

Second most common (30%+).

  • Build an assistant agent handling database queries, document search, and bash commands. Bash commands require explicit user approval. 19
  • Build an AI agent that transforms Monday.com project management data into conversational business insights using dual-LLM architecture. 20
  • Build an AI agent demonstrating natural interaction, agentic behavior, and clear reasoning steps. 21
  • Build a customer support agent. 22
  • Build an autonomous agent using an open-source LLM with observability/eval layer. 22
  • Build an agent that reads customer CSV data and generates personalized email campaigns with evaluation metrics. 9
  • Build a code review agent that analyzes Python files and provides actionable feedback. 9
  • Build a Singapore public transport query agent that fetches live data from 7 LTA APIs about buses, trains, traffic, and station conditions. 23
  • Build a sales insights agent that answers questions about subscription/revenue data. Must detect and refuse PII requests (emails, phone numbers, credit card tokens). No raw rows passed to the LLM - aggregates only. Evaluated on 3 dimensions: accuracy, safety/refusal correctness, reasoning quality. 24
  • Build an evaluation tool for LLM hallucination detection. 1

Multi-Agent Systems

  • Build a multi-agent content generation system: 5 core agents (research, writing, editing, SEO, publishing). Takes product JSON input, generates FAQ document, product page, and comparison page. All outputs must follow strict JSON formats. LangChain + Groq. 25
  • Implement a minimal workflow engine with graph-based nodes, state management, branching/looping, and tool-based logic. Max 50 steps. Built-in infinite-cycle protection required. Unit tests mandatory (6+ candidate submissions). 26 27 28 29 30 31
  • Build a 5-agent CBT therapy system: agents autonomously design, critique, and refine therapy exercises. Human-in-the-loop approval required before finalization. 32
  • Build a 4-stage bedtime story pipeline: Spec Builder, Storyteller, LLM Judge, Rewriter. Must use gpt-3.5-turbo. Up to 2 revision cycles. LLM judge evaluates stories against the spec. 33 34 35 36

Document Extraction and Processing

  • Build a marksheet extraction API: parse complex table layouts and handwriting from academic marksheets into structured JSON. 37
  • Build a physician notetaker: transform physician-patient conversations into structured clinical documentation.. 38 39
  • Build a legal document analysis tool for contracts: extract key information, identify risks (auto-renewal traps, liability, IP ownership, non-competes), generate structured summaries. 40
  • Build a CBT assistant combining RAG with safety mechanisms. Crisis detection mandatory. PII redaction and pseudonymization required. No secrets in logs. Educational only, not clinical advice. 41
  • Take a blood test report as PDF, understand medical issues, generate suggestions by fetching content from online blog articles with source links. 42
  • Build a question deduplication and clustering pipeline: exact dedup, semantic dedup, LLM-based cluster discovery, classification. Output evaluated with ARI, NMI, homogeneity, completeness metrics. 43
  • Build a data pipeline that processes 1,000 messy products from 4 vendors, normalizes them into a unified schema, fetches supplementary data via rate-limited async API calls (vendor-specific token-bucket rate limits), enriches products through AI-powered duplicate detection. Must support both CLI and API access. 2
  • Build a transaction-to-user matching system: identify users whose names appear in transaction descriptions, find similar transactions via text matching, propose improvements (semantic embeddings, database integration). Spring Boot + Java. 44
  • Build real-time earnings call transcription and insight streaming: streaming audio-to-text via Whisper, real-time extraction of financial signals (revenue, guidance, risks, outlook), SSE output. 45

Full-Stack AI Applications

  • Build a Telegram bot for investment coaching with safety filtering. Educational content only - no personalized financial advice. 46
  • Build a multi-agent D&D dungeon simulation: Game Master agent + Player agents. Must address at least 3 of 6 challenges (long campaigns, secrets, rulings, self-aware dungeon, living world, ambiguity). LangGraph required. 47
  • Build a memory extraction and personality transformation system: extract structured long-term memory from chat history as JSON, transform responses based on personas (calm mentor, witty friend, therapist). Open-source LLMs only, no proprietary APIs. 48
  • Build an AI judge for a Rock-Paper-Scissors variant: classify player inputs as VALID/INVALID/UNCLEAR, handle typos and edge cases, tool-based state management workflow (2 submissions). 49 50
  • Build a web app that converts markdown to slide deck presentations by splitting content into logical sections based on a target slide count. Document size limited to 150K tokens, single API call. Next.js + OpenAI. 51
  • Build an LLM processing pipeline with intelligent routing, multi-level caching (exact + semantic), provider health monitoring with failover, and distributed tracing. Targets: 100+ req/s, p95 latency under 2s, >40% cache hit rate. 52
  • Build an NPC system for a job simulation platform: three AI co-workers with distinct personalities, a "Director Agent" that detects conversation loops via semantic similarity (0.85 threshold), RAG-based knowledge retrieval. FastAPI + Claude API + FAISS. 53
  • Build an LLM-based rating prediction and prompt evaluation system with user and admin dashboards. Node.js + MongoDB + Google Generative AI. 54
  • Build an AI-first CRM module: React/Redux frontend, FastAPI backend, LangGraph with 5+ tools. Deliverable: GitHub repo + 10-15 minute demo video. 1

Evaluation Criteria Found in Assignments

Many assignments include explicit evaluation criteria. Patterns across repos:

  • Functional correctness - does the system work end-to-end, handle edge cases, produce correct outputs
  • Code quality and architecture - modular design, clean code, extensibility, proper error handling
  • Evaluation methodology - whether candidates build eval harnesses, define metrics, measure quality systematically
  • Production readiness - scalability considerations, caching, monitoring, cost optimization, security (PII handling, input sanitization, rate limiting)
  • Performance targets - response time (e.g., <2s p95 latency, ~0.5s per response), throughput (100+ req/s), cache hit rates (>40%), cost reduction (>30%)
  • Testing - unit tests (sometimes mandatory), test coverage targets (e.g., 80%), edge case handling
  • Documentation - README quality, design decision explanations, trade-off analysis
  • Weighted rubrics - some assignments provide explicit scoring: e.g., 30% functionality, 30% challenge completion, 25% context engineering, 15% code quality 47
  • Quantitative metrics - ARI/NMI for clustering 43, RAGAS metrics for RAG (faithfulness, answer relevancy, context precision) 18, confidence scores for extraction 37

How to Prepare

The best submissions share these traits: 1 55 22

  • Ask clarifying questions before starting. Double your time estimate. 56
  • Start with evaluation - build an eval harness before writing the main logic. YC startups report this as the top signal: "Red flag if candidate doesn't start with evals" 22
  • Document design decisions and trade-offs - why you chose this approach over alternatives
  • Include a Loom/video walkthrough of your submission 1
  • Make it configurable - one engineer built a PDF summarizer CLI with a config file for different models and chunking strategies, got two competing offers within 72 hours 1
  • Test edge cases - even if testing isn't explicitly required, include basic tests 56
  • Show production awareness - error handling, monitoring hooks, cost estimates
  • Connect technical metrics to business outcomes 55
  • Prepare for the defence round - practice explaining your architecture decisions, trade-offs, and what you'd do differently with more time

Common mistakes:

  • Not asking clarifying questions before starting
  • Putting too little effort in - "most engineers put way too little effort into take-homes" 1
  • Rushing without documenting design decisions and trade-offs
  • Over-engineering beyond what was asked without justification
  • Ignoring evaluation and testing of AI outputs - this is the single biggest differentiator 22
  • Not preparing for the walkthrough/defence round - the follow-up interview is often more important than the code itself

June 2026 Additions

New assignments discovered in a GitHub search pass covering Q2 2026. These extend the patterns above and introduce several new assignment archetypes.

Legal Document AI

Legal document processing emerged as a distinct category in 2026, with a repeating pattern: ingest messy legal documents, retrieve grounded evidence, generate cited drafts, and learn from operator edits.

  • Build a legal document AI workflow with a three-tier extraction cascade and an improvement-from-edits loop. The system must ingest, extract, ground, and iteratively improve based on operator corrections. 57 58 59 60
  • Pearson Specter Litt: Take-home requiring ingestion of messy legal documents, grounded retrieval, cited draft generation, and learning from operator edits. Delivered as a scrum sprint plan. 59

RAG and Document Q&A (New Companies)

  • Quorium: Build a RAG Q&A chatbot for an AI Engineer Trainee role. Standard RAG pipeline with document ingestion and question answering. 61
  • ITJ: RAG-based document QA system as a take-home challenge. 62
  • NTT DATA: RAG system over sustainability reports. 63
  • NeoStats: Chatbot assignment for an AI Engineer case study. 64
  • Trinamix: Supply chain RAG chatbot over a 2,000-PO supplier register and governance policy. Built with Flowise, Pinecone, and GPT. 65
  • GoTyme Bank: Full-stack document extraction system combining OCR and LLM, with a React frontend. 66

Agents and Multi-Agent Systems (New)

  • Go Fig AI: Build an inbox-triage agent skill with a human-in-the-loop approval gate. 2-hour cap. 67
  • Yuno: Multi-agent orchestration platform with Ollama, Streamlit, and Telegram. 68
  • RefundPilot: Containerized internal support workspace that evaluates e-commerce refund requests, applies refund policy, resists prompt-injection attempts, and records structured decision logs. 69
  • AgentCollect: Full-Stack AI Engineer (AI-Native) hiring challenge. 70
  • Neon Health: AI Agent with OpenAI integration. 71
  • Future Research: LangGraph multi-agent fitness coach with hub routing to sub-agents (coach, workout-generator, workout-logger) via Claude structured output. 72
  • KarthikTools: Full-stack web app where a backend agent executes predefined tools with a clear execution trace. 73

LLM Applications and Infrastructure

  • GenAI Labs: Production-ready LLM-driven SQL analytics pipeline with token counting, SQL validation, observability, and benchmarking. Multiple submissions. 74 75 76
  • VantageScore: GenAI-powered credit risk scoring with ML ensemble, explainability, and LLM enrichment. 77
  • AEGIS: AEO content scoring, LLM query fan-out, and embedding-based semantic gap analysis. 78
  • SHL: GenAI assessment recommendation system. 79
  • Camplight: LLM interview task. 80
  • VijaySaravanaPandi: NL-to-app "compiler" — turns a natural-language app description into a working application via structured, validated intermediate representations (not just prompt to code). 81
  • Jpower3145: Local LLM pre-interview task. 82
  • Cerebras: AI Engineer Model Quality and Performance hiring challenge (perf UI, eval pruning). 83
  • nickusevich: Novelty detection for football news using hybrid retrieval (pgvector + tsvector + RRF), LLM reranking, and LLM-based publish/skip/review decisions. 84

Embeddable and Conversational AI

  • EloquentAI: Build an embeddable AI agent chat widget. 85
  • Fleetio: Hybrid deterministic + LLM weekly fleet digest. 86
  • Zap: AI-powered client onboarding automation. 87
  • Adobe (FDE): GenAI-powered creative automation pipeline for social ad campaigns. 88

Official Company Challenges

Official repos published by companies as part of their hiring process:

  • ML6 (laine) - Coding challenge for evaluating AI Engineer candidates
  • Jaseci Labs - Take-home for AI Software Engineer candidates
  • Jitera - Take-home task for GenAI roles
  • AuxoAI - AI Engineering Take-Home
  • Coginis Research - AI Product Engineer Hiring Challenge
  • Future Research - AI Engineer take-home assessment + exercise dataset
  • Go Fig AI - Inbox-triage agent with human-in-the-loop
  • Cerebras - Model Quality and Performance challenge
  • AI:AT - Multi-criteria location evaluation agent
  • Bloom (radialreview) - Pre-built coffee ordering app candidates extend with an AI order taker

Sources

Footnotes

  1. Medium - Fonzi AI ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8

  2. GitHub - RokomariTask - Rokomari.com AI Engineer ↩ ↩2

  3. GitHub - Streamkar-Chatbot - StreamKar RAG chatbot ↩

  4. GitHub - DGE-assignment - DGE agentic RAG ↩

  5. GitHub - DGE_RAG_APP - DGE RAG application ↩

  6. GitHub - bmw-ai-engineer-case-study - BMW Group AI Engineer Intern ↩

  7. GitHub - ai_engineer_interview_2025 - RAG chatbot with MongoDB Atlas ↩

  8. GitHub - Company-Policy-Assistant - Neura Dynamics AI Engineer Intern ↩

  9. PromptLayer - The Agentic System Design Interview ↩ ↩2 ↩3

  10. GitHub - spur-live-chat-agent - Spur live chat agent ↩

  11. GitHub - spur-ai-chat - Spur AI chat with Prisma/Svelte ↩

  12. Dev.to - Mai Chi Bao, RAG Chatbot Interview ↩

  13. GitHub - bithealth-crfc - Bithealth code refactoring ↩

  14. GitHub - bithealth_home_assesment - Bithealth assessment ↩

  15. GitHub - bithealth-assesment - Bithealth assessment ↩

  16. GitHub - bithealthtest - Bithealth intern ↩

  17. GitHub - TechnicalTest-Bithealth - Bithealth RAG service ↩

  18. GitHub - govgpt-agentic-rag - GovGPT agentic RAG ↩ ↩2

  19. GitHub - hiring-challenge-alpha - assistant agent: database, documents, bash ↩

  20. GitHub - skylark-bi-insight-agent - Skylark Drones AI Engineer ↩

  21. Medium - Tushar Bhardwaj, Eightfold.ai ↩

  22. Reddit - What Is Your Interview Assignment for AI Engineers? (r/ycombinator) ↩ ↩2 ↩3 ↩4 ↩5

  23. GitHub - Transport-Query-Agent - Hrytos Singapore transport agent ↩

  24. GitHub - cohere_sales_agent - Cohere sales insights agent ↩

  25. GitHub - kasparro-ai-agentic-content-generation - Kasparro multi-agent content generation ↩

  26. GitHub - Minimal-Workflow-Agent-Enigne-Tredence - Tredence workflow engine ↩

  27. GitHub - tredence submission 2 - Tredence AI Engineer Intern ↩

  28. GitHub - tredence submission 3 - Tredence AI Engineer Intern ↩

  29. GitHub - tredence submission 4 - Tredence AI Engineer Intern ↩

  30. GitHub - tredence submission 5 - Tredence AI Engineer Intern ↩

  31. GitHub - tredence submission 6 - Tredence AI Engineer Intern ↩

  32. GitHub - Cerina-Health-AI-Engineer-Role-Task - Cerina Health 5-agent CBT system ↩

  33. GitHub - hippocratic-ai-bedtime-stories - Hippocratic AI bedtime story generator ↩

  34. GitHub - agent_deployment_bedtime_stories - Hippocratic AI with guardrails ↩

  35. GitHub - hippocratic-ai - Hippocratic AI bedtime stories ↩

  36. GitHub - AI-Agent-Deployment-Engineer-Takehome - Hippocratic AI story service ↩

  37. GitHub - Trestle_AI_Engineer_Intern_Assignment - Trestle marksheet extraction ↩ ↩2

  38. GitHub - Physician-Notetaker - Emitrr medical transcription ↩

  39. GitHub - Emitrr submission 2 - Emitrr AI Engineer Intern ↩

  40. GitHub - Files.Invis - AI-powered legal document analysis ↩

  41. GitHub - mindwell-assignment-2026 - Mindwell AI Engineer case study ↩

  42. Medium - Khushal Kumar ↩

  43. GitHub - krisp_ai_engineer_role_task - Krisp dedup and clustering ↩ ↩2

  44. GitHub - deel-assignment - Deel transaction matching ↩

  45. GitHub - voice-ai-assignment - real-time concall transcription ↩

  46. GitHub - investment_coach_bot - PineOS.ai investment coaching bot ↩

  47. GitHub - context-engineering-takehome - D&D simulation with context engineering ↩ ↩2

  48. GitHub - GUPPSHUPP_Founding_AI_Engineer_Assignment - memory and personality engine ↩

  49. GitHub - RPS-Plus-Al-Judge - upliance.ai conversational agents ↩

  50. GitHub - upliance.ai_assignment - upliance.ai conversational agents ↩

  51. GitHub - gamma-project - Gamma markdown-to-slides ↩

  52. GitHub - Smart-LLM-Router-Observability-Platform - Zuneko Labs LLM router ↩

  53. GitHub - AI-Coworker-Engine - Edtronaut NPC system ↩

  54. GitHub - fynd-ai-feedback-system - Fynd AI feedback system ↩

  55. InterviewNode - Cracking ML Take-Home Assignments ↩ ↩2

  56. dev.to - Learn From My Mistakes: My First Take-Home Code Challenge ↩ ↩2

  57. GitHub - Legal-Document-AI-Workflow - Legal document AI workflow ↩

  58. GitHub - Legal-AI-Assessment - Legal document understanding, grounded retrieval ↩

  59. GitHub - legal-ai-assessment - Pearson Specter Litt legal docs ↩ ↩2

  60. GitHub - grounded-legal-drafting - Grounded drafting, improvement from edits ↩

  61. GitHub - quorium-rag-chatbot - Quorium RAG Q&A chatbot ↩

  62. GitHub - itj-rag-challenge - ITJ RAG document QA ↩

  63. GitHub - ntt-rag - NTT DATA sustainability reports RAG ↩

  64. GitHub - riviera-paradise - NeoStats chatbot ↩

  65. GitHub - scm-assistant-bot - Trinamix supply chain RAG chatbot ↩

  66. GitHub - tymebank-document-extraction - GoTyme Bank document extraction ↩

  67. GitHub - take-home-inbox-triage - Go Fig AI inbox-triage agent ↩

  68. GitHub - yuno-agent-platform - Yuno multi-agent platform ↩

  69. GitHub - refundpilot-ai-agent - RefundPilot AI agent ↩

  70. GitHub - AgentCollect-Challenge - AgentCollect Full-Stack AI Engineer challenge ↩

  71. GitHub - Neon_assesment - Neon Health AI agent ↩

  72. GitHub - spotter - Future Research multi-agent fitness coach ↩

  73. GitHub - ai-engineer-coding-challenge - Agent execution with tool traces ↩

  74. GitHub - gal-assignment - GenAI Labs LLM analytics pipeline ↩

  75. GitHub - genai-labs - GenAI Labs LLM analytics pipeline ↩

  76. GitHub - SQLToText-project - GenAI Labs SQL-to-text ↩

  77. GitHub - vantagescore-credit-intelligence-platform - VantageScore GenAI credit scoring ↩

  78. GitHub - aegis-ai-engineer-assignment - AEGIS AEO content scoring ↩

  79. GitHub - shl-assessment-recommendation - SHL GenAI assessment recommendation ↩

  80. GitHub - camplight-llm-task - Camplight LLM interview task ↩

  81. GitHub - ai-app-compiler - NL-to-app compiler ↩

  82. GitHub - CQL-Translator - Local LLM pre-interview task ↩

  83. GitHub - ai-model-quality-challenge - Cerebras Model Quality challenge ↩

  84. GitHub - ai-engineering-assignment - Novelty detection with hybrid retrieval ↩

  85. GitHub - Embeddable-Chat-Widget-EloquentAI - EloquentAI embeddable chat widget ↩

  86. GitHub - fleet-weekly-digest - Fleetio hybrid LLM fleet digest ↩

  87. GitHub - zap-onboarding-ai - Zap AI client onboarding ↩

  88. GitHub - creative-automation-pipeline - Adobe FDE GenAI creative automation ↩