Merge pull request #58 from connortreacy/frontend

Frontend

Author: heyjustinai

.coveragerc

@@ -1,5 +1,5 @@
 [run]
-source = src/llama_prompt_ops
+source = src/prompt_ops
 omit =
     */tests/*
     */site-packages/*

.github/ISSUE_TEMPLATE/bug.yml

@@ -1,77 +1,61 @@
 name: 🐛 Bug Report
-description: Create a report to help us reproduce and fix the bug
+description: Report a bug to help us improve prompt-ops
 labels: ["bug"]
 body:
   - type: markdown
     attributes:
       value: >
-        #### Before submitting a bug, please make sure the issue hasn't been already addressed by searching through [the
-        existing and past issues](https://github.com/meta-llama/llama-stack/issues).
+        Thanks for taking the time to report a bug! Please search [existing issues](https://github.com/meta-llama/prompt-ops/issues) first to avoid duplicates.
 
   - type: textarea
-    id: system-info
+    id: bug-description
     attributes:
-      label: System Info
-      description: |
-        Please share your system info with us. You can use the following command to capture your environment information
-        python -m "torch.utils.collect_env"
-
+      label: What happened?
+      description: A clear description of the bug and what you expected to happen instead.
       placeholder: |
-        PyTorch version, CUDA version, GPU type, #num of GPUs...
+        I was trying to optimize prompts using the facility dataset, but...
+
+        Expected: The optimization should complete successfully
+        Actual: Got an error about missing configuration
     validations:
       required: true
 
-  - type: checkboxes
-    id: information-scripts-examples
-    attributes:
-      label: Information
-      description: 'The problem arises when using:'
-      options:
-        - label: "The official example scripts"
-        - label: "My own modified scripts"
-
   - type: textarea
-    id: bug-description
+    id: reproduction
     attributes:
-      label: 🐛 Describe the bug
-      description: |
-        Please provide a clear and concise description of what the bug is.
-
-        Please also paste or describe the results you observe instead of the expected results.
+      label: How to reproduce
+      description: Steps to reproduce the issue, including any commands or code.
       placeholder: |
-        A clear and concise description of what the bug is.
+        1. Run `prompt-ops optimize --config configs/facility.yaml`
+        2. See error...
 
-        ```llama stack
-        # Command that you used for running the examples
+        ```bash
+        # Your command here
         ```
-        Description of the results
     validations:
       required: true
 
   - type: textarea
+    id: environment
     attributes:
-      label: Error logs
-      description: |
-       If you observe an error, please paste the error message including the **full** traceback of the exception. It may be relevant to wrap error messages in ```` ```triple quotes blocks``` ````.
-
+      label: Environment
+      description: Your Python version and any relevant system info.
       placeholder: |
-        ```
-        The error message you got, with the full traceback.
-        ```
-
+        - Python version: 3.9.0
+        - prompt-ops version: 0.0.8
+        - OS: macOS 14.0
     validations:
-      required: true
-
+      required: false
 
   - type: textarea
-    id: expected-behavior
-    validations:
-      required: true
-    attributes:
-      label: Expected behavior
-      description: "A clear and concise description of what you would expect to happen."
-
-  - type: markdown
+    id: error-logs
     attributes:
-      value: >
-        Thanks for contributing 🎉!
+      label: Error logs (if any)
+      description: Paste any error messages or stack traces.
+      placeholder: |
+        ```
+        Traceback (most recent call last):
+          ...
+        ```
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/config.yml

@@ -2,11 +2,5 @@ blank_issues_enabled: false
 
 contact_links:
   - name: Have you read the docs?
-    url: https://llama-stack.readthedocs.io/en/latest/index.html
-    about: Much help can be found in the docs
-  - name: Start a discussion
-    url: https://github.com/meta-llama/llama-stack/discussions/new
-    about: Start a discussion on a topic
-  - name: Chat on Discord
-    url: https://discord.gg/llama-stack
-    about: Maybe chatting with the community can help
+    url: https://github.com/meta-llama/prompt-ops/blob/main/README.md
+    about: Much help can be found in the README and documentation

.github/ISSUE_TEMPLATE/feature-request.yml

@@ -1,28 +1,35 @@
-name: 🚀 Feature request
-description: Request a new llama-stack feature
+name: 🚀 Feature Request
+description: Suggest a new feature for prompt-ops
 labels: ["enhancement"]
 body:
 - type: textarea
-  id: feature-pitch
+  id: feature-description
   attributes:
-    label: 🚀 Describe the new functionality needed
-    description: >
-      A clear and concise description of _what_ needs to be built.
+    label: What feature would you like?
+    description: A clear description of the feature you'd like to see.
+    placeholder: |
+      I'd like to be able to...
+
+      This would help with...
   validations:
     required: true
 
 - type: textarea
-  id: feature-motivation
+  id: use-case
   attributes:
-    label: 💡 Why is this needed? What if we don't build it?
-    description: >
-      A clear and concise description of _why_ this functionality is needed.
+    label: What's your use case?
+    description: How would you use this feature? What problem does it solve?
+    placeholder: |
+      I'm working on optimizing prompts for customer support and need...
   validations:
     required: true
 
 - type: textarea
-  id: other-thoughts
+  id: additional-context
   attributes:
-    label: Other thoughts
-    description: >
-      Any thoughts about how this may result in complexity in the codebase, or other trade-offs.
+    label: Additional context
+    description: Any other details, examples, or screenshots that might help.
+    placeholder: |
+      Here's an example of what I'm trying to achieve...
+  validations:
+    required: false

.github/workflows/ci-with-coverage.yml

@@ -34,7 +34,7 @@ jobs:
 
     - name: Run tests with coverage
       run: |
-        pytest --cov=src/llama_prompt_ops --cov-report=xml tests/
+        pytest --cov=src/prompt_ops --cov-report=xml tests/
 
     - name: Upload coverage to Codecov
       uses: codecov/codecov-action@v3

.github/workflows/publish.yml

@@ -40,7 +40,7 @@ jobs:
         echo "Package version: $VERSION"
 
         # Check if this version already exists on PyPI
-        if pip index versions llama-prompt-ops 2>/dev/null | grep -q "$VERSION"; then
+        if pip index versions prompt-ops 2>/dev/null | grep -q "$VERSION"; then
           echo "Error: Version $VERSION already exists on PyPI"
           exit 1
         fi

.gitignore

@@ -1,3 +1,6 @@
+local.env
+
+
 # Python cache files
 __pycache__/
 *.py[cod]

CONTRIBUTING.md

@@ -1,4 +1,4 @@
-# Contributing to llama-prompt-ops
+# Contributing to prompt-ops
 We want to make contributing to this project as easy and transparent as
 possible.
 
@@ -23,4 +23,4 @@ We use GitHub issues to track public bugs. Please ensure your description is cle
 Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe disclosure of security bugs. In those cases, please go through the process outlined on that page and do not file a public issue.
 
 ## License
-By contributing to llama-prompt-ops, you agree that your contributions will be licensed under the LICENSE file in the root directory of this source tree.
+By contributing to prompt-ops, you agree that your contributions will be licensed under the LICENSE file in the root directory of this source tree.

MANIFEST.in

@@ -1,4 +1,4 @@
-recursive-include src/llama_prompt_ops *.py
-recursive-include src/llama_prompt_ops/interfaces *.py
-recursive-include src/llama_prompt_ops/core *.py
-recursive-include src/llama_prompt_ops/templates *
+recursive-include src/prompt_ops *.py
+recursive-include src/prompt_ops/interfaces *.py
+recursive-include src/prompt_ops/core *.py
+recursive-include src/prompt_ops/templates *

README.md

@@ -1,40 +1,52 @@
-<h1 align="center"> Llama Prompt Ops </h1>
+<h1 align="center"> Prompt Ops </h1>
 
-## What is llama-prompt-ops?
+### 🎉 New: Prompt Duel Optimizer (PDO) Published!
+
+We've published a new paper on **PDO (Prompt Duel Optimizer)** - an efficient label-free prompt optimization method using dueling bandits and Thompson sampling. PDO achieves state-of-the-art results on BIG-bench Hard and MS MARCO benchmarks.
+
+📄 **Read the paper:** [LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization](https://www.arxiv.org/abs/2510.13907) (arXiv:2510.13907)
+
+🧪 **Try it yourself:** Check out the [Web of Lies use case](use-cases/web-of-lies-pdo/) demonstrating PDO on logical reasoning tasks
+
+⭐ **Star this repo** and follow along - we'll be publishing a detailed tutorial notebook soon!
+
+---
+
+## What is prompt-ops?
 <p align="center">
-  <a href="https://pypi.org/project/llama-prompt-ops/"><img src="https://img.shields.io/pypi/v/llama-prompt-ops.svg" /></a>
+  <a href="https://pypi.org/project/prompt-ops/"><img src="https://img.shields.io/pypi/v/prompt-ops.svg" /></a>
 </p>
 <p align="center">
-  <a href="https://llama.developer.meta.com/?utm_source=llama-prompt-ops&utm_medium=readme&utm_campaign=main"><img src="https://img.shields.io/badge/Llama_API-Join_Waitlist-brightgreen?logo=meta" /></a>
-  <a href="https://llama.developer.meta.com/docs?utm_source=llama-prompt-ops&utm_medium=readme&utm_campaign=main"><img src="https://img.shields.io/badge/Llama_API-Documentation-4BA9FE?logo=meta" /></a>
+  <a href="https://llama.developer.meta.com/?utm_source=prompt-ops&utm_medium=readme&utm_campaign=main"><img src="https://img.shields.io/badge/Llama_API-Join_Waitlist-brightgreen?logo=meta" /></a>
+  <a href="https://llama.developer.meta.com/docs?utm_source=prompt-ops&utm_medium=readme&utm_campaign=main"><img src="https://img.shields.io/badge/Llama_API-Documentation-4BA9FE?logo=meta" /></a>
 
 </p>
 
 <p align="center">
-  <a href="https://github.com/meta-llama/llama-models/blob/main/models/?utm_source=llama-prompt-ops&utm_medium=readme&utm_campaign=main"><img alt="Llama Model cards" src="https://img.shields.io/badge/Llama_OSS-Model_cards-green?logo=meta" /></a>
-  <a href="https://www.llama.com/docs/overview/?utm_source=llama-prompt-ops&utm_medium=readme&utm_campaign=main"><img alt="Llama Documentation" src="https://img.shields.io/badge/Llama_OSS-Documentation-4BA9FE?logo=meta" /></a>
+  <a href="https://github.com/meta-llama/llama-models/blob/main/models/?utm_source=prompt-ops&utm_medium=readme&utm_campaign=main"><img alt="Llama Model cards" src="https://img.shields.io/badge/Llama_OSS-Model_cards-green?logo=meta" /></a>
+  <a href="https://www.llama.com/docs/overview/?utm_source=prompt-ops&utm_medium=readme&utm_campaign=main"><img alt="Llama Documentation" src="https://img.shields.io/badge/Llama_OSS-Documentation-4BA9FE?logo=meta" /></a>
   <a href="https://huggingface.co/meta-llama"><img alt="Hugging Face meta-llama" src="https://img.shields.io/badge/Hugging_Face-meta--llama-yellow?logo=huggingface" /></a>
 
 </p>
 <p align="center">
   <a href="https://github.com/meta-llama/synthetic-data-kit"><img alt="Llama Tools Syntethic Data Kit" src="https://img.shields.io/badge/Llama_Tools-synthetic--data--kit-orange?logo=meta" /></a>
-  <a href="https://github.com/meta-llama/llama-prompt-ops"><img alt="Llama Tools Syntethic Data Kit" src="https://img.shields.io/badge/Llama_Tools-llama--prompt--ops-orange?logo=meta" /></a>
+  <a href="https://github.com/meta-llama/prompt-ops"><img alt="Llama Tools Syntethic Data Kit" src="https://img.shields.io/badge/Llama_Tools-prompt--ops-orange?logo=meta" /></a>
     <a href="https://github.com/meta-llama/llama-cookbook"><img alt="Llama Cookbook" src="https://img.shields.io/badge/Llama_Cookbook-llama--cookbook-orange?logo=meta" /></a>
 </p>
 
 
 
-llama-prompt-ops is a Python package that **automatically optimizes prompts** for Llama models. It transforms prompts that work well with other LLMs into prompts that are optimized for Llama models, improving performance and reliability.
+prompt-ops is a Python package that **automatically optimizes prompts** for Llama models. It transforms prompts that work well with other LLMs into prompts that are optimized for LLM models, improving performance and reliability.
 
 **Key Benefits:**
 - **No More Trial and Error**: Stop manually tweaking prompts to get better results
-- **Fast Optimization**: Get Llama-optimized prompts in minutes with template-based optimization
+- **Fast Optimization**: Get model-optimized prompts in minutes with template-based optimization
 - **Data-Driven Improvements**: Use your own examples to create prompts that work for your specific use case
 - **Measurable Results**: Evaluate prompt performance with customizable metrics
 
 ## Requirements
 
-To get started with llama-prompt-ops, you'll need:
+To get started with prompt-ops, you'll need:
 
 - Existing System Prompt: Your existing system prompt that you want to optimize
 - Existing Query-Response Dataset: A JSON file containing query-response pairs (as few as 50 examples) for evaluation and optimization (see [prepare your dataset](#preparing-your-data) below)
@@ -50,7 +62,7 @@ To get started with llama-prompt-ops, you'll need:
              │                              │                           │
              ▼                              ▼                           ▼
          ┌────────────────────────────────────────────────────────────────────┐
-         │                     llama-prompt-ops migrate                       │
+         │                     prompt-ops migrate                       │
          └────────────────────────────────────────────────────────────────────┘
                                             │
                                             │
@@ -66,7 +78,7 @@ To get started with llama-prompt-ops, you'll need:
 2. [**Prepare your dataset**](#preparing-your-data): Create a JSON file with query-response pairs for evaluation and optimization
 3. **Configure optimization**: Set up a simple YAML file with your dataset and preferences (see [example configuration](configs/facility-simple.yaml))
 4. [**Run optimization**](#step-4-run-optimization): Execute a single command to transform your prompt
-5. [**Get results**](#prompt-transformation-example): Receive a Llama-optimized prompt with performance metrics
+5. [**Get results**](#prompt-transformation-example): Receive a model-optimized prompt with performance metrics
 
 
 ## Real-world Results
@@ -85,27 +97,29 @@ These results were measured on the [HotpotQA multi-hop reasoning benchmark](http
 
 ### Step 1: Installation
 
+> **Note:** We recommend installing from source as we are currently transitioning package names on PyPI. This ensures you get the latest stable version without any naming conflicts.
+
 ```bash
 # Create a virtual environment
 conda create -n prompt-ops python=3.10
 conda activate prompt-ops
 
-# Install from PyPI
-pip install llama-prompt-ops
-
-# OR install from source
-git clone https://github.com/meta-llama/llama-prompt-ops.git
-cd llama-prompt-ops
+# Recommended: Install from source
+git clone https://github.com/meta-llama/prompt-ops.git
+cd prompt-ops
 pip install -e .
 
+# Alternative: Install from PyPI (may have naming transition issues, still on version 0.0.7)
+# pip install llama-prompt-ops
+
 ```
 
 ### Step 2: Create a sample project
 
 This will create a directory called my-project with a sample configuration and dataset in the current folder.
 
 ```bash
-llama-prompt-ops create my-project
+prompt-ops create my-project
 cd my-project
 ```
 
@@ -116,13 +130,14 @@ Add your API key to the `.env` file:
 ```bash
 OPENROUTER_API_KEY=your_key_here
 ```
-You can get an OpenRouter API key by creating an account at [OpenRouter](https://openrouter.ai/). For more inference provider options, see [Inference Providers](./docs/inference_providers.md).
+
+prompt-ops uses LiteLLM as a unified API client. LiteLLM automatically detects the provider from your model name (e.g., `openrouter/model`, `groq/model`) and looks for the corresponding provider-specific environment variable (`OPENROUTER_API_KEY`, `GROQ_API_KEY`, etc.). For more inference provider options, see [Inference Providers](./docs/inference_providers.md).
 
 ### Step 4: Run Optimization
 The optimization will take about 5 minutes.
 
 ```bash
-llama-prompt-ops migrate # defaults to config.yaml if --config not specified
+prompt-ops migrate # defaults to config.yaml if --config not specified
 ```
 
 Done! The optimized prompt will be saved to the `results` directory with performance metrics comparing the original and optimized versions.
@@ -141,7 +156,7 @@ Below is an example of a transformed system prompt from proprietary LM to Llama:
 
 ## Preparing Your Data
 
-To use llama-prompt-ops for prompt optimization, you'll need to prepare a dataset with your prompts and expected responses. The standard format is a JSON file structured like this:
+To use prompt-ops for prompt optimization, you'll need to prepare a dataset with your prompts and expected responses. The standard format is a JSON file structured like this:
 
 ```json
 [
@@ -156,15 +171,15 @@ To use llama-prompt-ops for prompt optimization, you'll need to prepare a datase
 ]
 ```
 
-If your data matches this format, you can use the built-in [`StandardJSONAdapter`](src/llama_prompt_ops/core/datasets.py) which will handle it automatically.
+If your data matches this format, you can use the built-in [`StandardJSONAdapter`](src/prompt_ops/core/datasets.py) which will handle it automatically.
 
 ### Custom Data Formats
 
 If your data is formatted differently, and there isn't a built-in dataset adapter, you can create a custom dataset adapter by extending the `DatasetAdapter` class. See the [Dataset Adapter Selection Guide](docs/dataset_adapter_selection_guide.md) for more details.
 
 ## Multiple Inference Provider Support
 
-llama-prompt-ops supports various inference providers and endpoints to fit your infrastructure needs. See our [detailed guide on inference providers](./docs/inference_providers.md) for configuration examples with:
+prompt-ops supports various inference providers and endpoints to fit your infrastructure needs. See our [detailed guide on inference providers](./docs/inference_providers.md) for configuration examples with:
 
 - OpenRouter (cloud-based API)
 - vLLM (local deployment)
@@ -174,7 +189,7 @@ llama-prompt-ops supports various inference providers and endpoints to fit your
 
 For more detailed information, check out these resources:
 
-- [Quick Start Guide](docs/basic/readme.md): Get up and running with llama-prompt-ops in 5 minutes
+- [Quick Start Guide](docs/basic/readme.md): Get up and running with prompt-ops in 5 minutes
 - [Intermediate Configuration Guide](docs/intermediate/readme.md): Learn how to configure datasets, metrics, and optimization strategies
 - [Dataset Adapter Selection Guide](docs/dataset_adapter_selection_guide.md): Choose the right adapter for your dataset format
 - [Metric Selection Guide](docs/metric_selection_guide.md): Select appropriate evaluation metrics for your use case