GPU Watt Watcher (GPU Egy)

GPU Watt Watcher is a small app you run on your own NVIDIA GPU.

It shows:

• How fast your GPU runs simple benchmarks (GFLOPs/s, tokens/s)
• How much energy those benchmarks use (Joules)
• Simple hints about what to change (batch size, sequence length, precision) to use less energy for the same work

---

1. What you need

Before installing:

• NVIDIA GPU with recent drivers
  • nvidia-smi should work in a terminal
• CUDA-enabled PyTorch
  • python -c "import torch; print(torch.cuda.is_available())" → should print True
• Python 3.9 or newer
• Linux or Windows (macOS is not supported for CUDA)

---

3. Run it on your GPU

Requirements:

• NVIDIA GPU with drivers installed (nvidia-smi works)
• CUDA-enabled PyTorch
• Python 3.9+

Steps:

git clone https://github.com/raffrant/gpu-watt-watcher.git
cd gpu-watt-watcher
python -m venv .venv
source .venv/bin/activate      # .venv\Scripts\Activate.ps1 on Windows
pip install -r requirements.txt
cd gpu_energy_bench
streamlit run streamlit_app.py

Then open http://localhost:8501 in your browser.

4. What you get

In the app you will find:

• GPU info – name, driver/CUDA version, current power, temperature, memory
• Matmul benchmark – GFLOPs/s and Joules per GFLOP
• Memory benchmark – GB/s and Joules per GB of data moved
• Data-movement experiments – contiguous/strided bandwidth and fused versus unfused pointwise kernels, with stride penalties and avoided-memory-traffic gains
• AI presets – tokens/s and Joules per token for a tiny Transformer block
• Test suite – a few predefined tests with pass/fail thresholds on energy and performance
• History – a CSV log of your runs so you can compare changes over time

---

5. License & contributions

(Choose a license you like, e.g. MIT, Apache-2.0.)

Contributions are welcome:

• new benchmarks (kernels, models)
• better energy advice rules
• deeper history analysis or visualization

---

6. Website: gpuegy.dev

Visit https://gpuegy.dev/ for:

• a quick overview of GPU Egy
• screenshots and examples
• direct links to this repo and usage instructions

The app always runs on your GPU, from your machine. Nothing is sent to a server unless you choose to share results yourself.

Memory bottleneck experiment

From the repository root:

python -m gpu_energy_bench.cli memory --size-mb 512 --repetitions 50

The generated gpu_memory_output/memory_report.json shows whether performance falls with non-coalesced access and whether kernel fusion reduces runtime and joules by avoiding intermediate tensor reads/writes. Use Nsight Compute afterward to confirm physical DRAM traffic; the built-in byte counts are algorithmic minimum/useful traffic.

The simplestart ALU can also define a repeatable GPU workload:

python -m gpu_energy_bench.cli rtl-gpu --size-mb 512 --trace-repetitions 100

This decodes the add/subtract/and/xor sequence from simplestart/wave.vcd, replays it over CUDA uint8 tensors, and writes RTL switching plus GPU bandwidth/energy metrics to gpu_rtl_output/rtl_gpu_report.json. Pass --regenerate to rebuild the waveform from alu.sv with Verilator first.

GPU learning lab

The progressive education and diagnostic suite lives in gpu_learning_lab/. It pairs PyTorch operations with explicit CUDA C++ kernels for launch overhead, copy bandwidth, strided access, fusion, reduction, and tiled matrix multiplication:

python -m gpu_learning_lab.pytorch_gpu_lab --experiment all
cd gpu_learning_lab/cuda
make
./gpu_lab --experiment all

Read docs/GOLDEN_GPU_CRASH_BOOK.md for the GPU hardware mental model, measurement protocol, crash lookup table, diagnostic commands, and an energy-efficiency product roadmap.

Run the complete product-style workflow with:

python -m gpu_learning_lab.ship doctor --build
python -m gpu_learning_lab.ship run --suite both --trials 10

This produces a versioned JSON report with golden correctness, repeated-trial statistics, CUDA-event latency, NVML-derived energy estimates, and regression gate inputs. See gpu_learning_lab/SHIP.md.

Tensor-network compression and decomposition sweeps are also available:

python -m gpu_learning_lab.tensor_network_lab --mode all \
  --max-bonds 8 16 32 64 128 --objective energy

This searches QR/SVD shapes, controlled singular-value spectra, randomized low-rank SVD, and TEBD-style MPS truncation. The reported sweet spot is the lowest-energy feasible configuration under an explicit discarded-weight bound.

Optimize and benchmark tensor-contraction ordering with:

python -m gpu_learning_lab.contraction_path_lab \
  --bonds 4 8 16 32 64 --strategies left greedy dp optimal \
  --reuse-count 1000 --objective energy

This demonstrates why contracting connected legs first can avoid enormous outer-product intermediates. Paths are planned once, compiled into reusable opt_einsum expressions, and selected using measured latency/energy after golden final-state validation.

Run Trotterized multiqubit XYZ dynamics with arbitrary tensor legs:

python -m gpu_learning_lab.quantum_dynamics_lab \
  --backend both --build-native \
  --qubits 12 16 20 --steps 4 8 16 32 \
  --order 2 --random-middle --objective energy

This compares PyTorch tensor-leg operations with the native quantum_xyz.cu state-vector kernels and selects the cheapest converged Trotter step count.

7. Used Lovable to buy a domain and help with the web development.