Skip to content

Pdo notebook - #54

Merged
heyjustinai merged 7 commits into
mainfrom
pdo-notebook
Nov 12, 2025
Merged

Pdo notebook#54
heyjustinai merged 7 commits into
mainfrom
pdo-notebook

Conversation

@heyjustinai

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR adds comprehensive documentation and examples for PDO (Prompt Duel Optimizer) functionality, including a tutorial notebook, use case examples, and enhancements to support both open-ended and close-ended task evaluation.

Key Changes:

  • Tutorial: Added interactive PDO tutorial notebook with theory, installation guide, and practical examples
  • Core Enhancements: Improved meta-prompt templates for open/close-ended tasks and LLM-as-judge evaluation
  • Examples: Added MS-MARCO (close-ended QA) and enhanced Web-of-Lies (open-ended reasoning) examples
  • Documentation: Added visualizations, README files, and improved code documentation

Test Plan

Tested all components:

  • PDO tutorial notebook runs end-to-end without errors
  • MS-MARCO example successfully optimizes prompts on 100 QA pairs
  • Web-of-Lies example correctly evaluates prompts using LLM-as-judge

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 12, 2025
@heyjustinai
heyjustinai merged commit 4ead714 into main Nov 12, 2025
9 checks passed
@heyjustinai
heyjustinai deleted the pdo-notebook branch November 26, 2025 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants