Proposal: gpu-aware-execution skill for hardware-aware planning

[efecnc]

Summary

Proposing a new always: true skill (gpu-aware-execution) that teaches the agent to read the x_hw_snapshot extension from capabilities_summary() and use it to make hardware-aware decisions before launching GPU-heavy or memory-heavy work.

This depends on the hw_discovery module landing first (GPU/hardware auto-detection at startup).

What the skill does

Instructs the agent to:

1. Check hardware before planning -- read gpu_summary.count, min_vram_bytes, homogeneous, and detection_backend from x_hw_snapshot.
2. Branch on GPU count:
   • count == 0: Don't plan CUDA/ROCm workflows; suggest CPU alternatives or GPU-enabled providers.
   • count == 1: Use min_vram_bytes as the memory budget, estimate model fit (~2 bytes/param fp16/bf16, ~4 bytes/param fp32), suggest mitigations if it doesn't fit.
   • count > 1: Prefer data parallelism if homogeneous; warn about load imbalance if mixed GPUs.
   • detection_backend: none: Detection may have failed -- suggest a runtime probe rather than assuming no GPU.
3. Guardrails: Never launch a run that will obviously OOM, never silently downgrade training objectives (defer to oom-recovery-playbook), never assume GPU presence when hardware says otherwise.

Proposed skill content

---
name: gpu-aware-execution
description: Read x_hw_snapshot from capabilities_summary before GPU/memory-heavy work.
always: true
---

# GPU-aware execution planning

Read `x_hw_snapshot` from `capabilities_summary()` **before** planning any GPU-heavy or memory-heavy work. Key fields: `gpu_summary.count`, `gpu_summary.min_vram_bytes`, `gpu_summary.homogeneous`, `detection_backend`.

If `x_hw_snapshot` is absent, fall back to `torch.cuda.is_available()` and tell the user.

## Decision rules

**No GPU** (`count == 0`) -- do not plan CUDA/ROCm workflows. Suggest CPU-only alternatives (GGUF/llama.cpp) or a GPU-enabled provider.

**Single GPU** (`count == 1`) -- `min_vram_bytes` is your budget. Model memory ~ 2 bytes/param (fp16/bf16) or 4 bytes/param (fp32), plus optimizer states. If it does not fit: lower precision, quantize, gradient checkpointing. Do not silently split across CPU+GPU.

**Multi-GPU** (`count > 1`) -- if `homogeneous`, use data parallelism. If not, warn about load imbalance. Suggest tensor/pipeline parallelism only when the framework supports it.

**`detection_backend: none`** -- detection may have failed, not necessarily "no GPU". Suggest a runtime probe.

## Guardrails

- Never ignore `x_hw_snapshot` and launch a run that will OOM.
- Never silently downgrade the training objective to fit hardware -- use `load_skill_instructions` for `oom-recovery-playbook`.
- Never assume GPU presence when `count == 0` just because the user mentioned "GPU".

Design notes

• ~196 words -- on par with other always: true skills (ml-execution-preflight is ~190 words).
• No overlap with oom-recovery-playbook (reactive recovery) or ml-execution-preflight (pre-launch checklist). This skill covers proactive hardware-aware planning only.
• Depends on the hw_discovery module populating x_hw_snapshot in capabilities_summary().
• Companion implementation instruction: docs/instructions/gpu-auto-discovery.md

Questions for maintainers

1. Is this the right granularity for an always: true skill, or should it be always: false and loaded on demand when the agent detects ML/GPU work?
2. Should the skill reference specific field paths (e.g., gpu_summary.min_vram_bytes) or stay more abstract to avoid coupling to the data model?
3. Any concerns about the token cost of injecting this into every LLM call on CPU-only machines where it adds no value?

[monatis]

we already have the execution_env_info tool to provide the agent with insight into capabilities, and the prompt tells the agent to discover capabilities with that. I think it's the right place to fix it if we observe repeated failures for this specific piece of logic.

Skılls should be rather preferred for hard-to-find information or instructions to discourage LLM laziness. I'll revisit the built-in skills after #35 is merged