What happened?
Description
When explicitly configuring strategy: "pdo" in the optimization config, the system ignores this setting and auto-detects/forces a different strategy based on the model name, preventing PDO from running.
How to reproduce
config.yaml
optimization:
strategy: "pdo"
total_rounds: 10
num_duels_per_round: 3
num_eval_examples_per_duel: 5
num_initial_instructions: 2
thompson_alpha: 2.0
num_top_prompts_to_combine: 2
gen_new_prompt_round_frequency: 2
num_threads: 5
verbose: true
max_errors: 5
max_concurrent_threads: 5
model:
name: "meta/llama-4-scout-17b-16e-instruct"
api_base: "https://integrate.api.nvidia.com/v1"
temperature: 0.5
Environment
Package: llama-prompt-ops / prompt-ops
Model: meta/llama-4-scout-17b-16e-instruct
Python Version: 3.10+
Installation: pip install from source
Error logs (if any)
Auto-detected LlamaStrategy for model: llama-4-scout-17b-16e-instruct
2026-01-05 13:13:07,428 - root - WARNING - Model does not appear to be a Llama model
Applying LlamaStrategy to optimize prompt
What happened?
Description
When explicitly configuring strategy: "pdo" in the optimization config, the system ignores this setting and auto-detects/forces a different strategy based on the model name, preventing PDO from running.
How to reproduce
config.yaml
optimization:
strategy: "pdo"
total_rounds: 10
num_duels_per_round: 3
num_eval_examples_per_duel: 5
num_initial_instructions: 2
thompson_alpha: 2.0
num_top_prompts_to_combine: 2
gen_new_prompt_round_frequency: 2
num_threads: 5
verbose: true
max_errors: 5
max_concurrent_threads: 5
model:
name: "meta/llama-4-scout-17b-16e-instruct"
api_base: "https://integrate.api.nvidia.com/v1"
temperature: 0.5
Environment
Package: llama-prompt-ops / prompt-ops
Model: meta/llama-4-scout-17b-16e-instruct
Python Version: 3.10+
Installation: pip install from source
Error logs (if any)
Auto-detected LlamaStrategy for model: llama-4-scout-17b-16e-instruct
2026-01-05 13:13:07,428 - root - WARNING - Model does not appear to be a Llama model
Applying LlamaStrategy to optimize prompt