Cost side is real and fixed (GCP pricing). Throughput (TPS) side is the unknown that decides everything and must be benchmarked - it is not invented here.
- Why isn't TPU cost-competitive? - the FLOPs-vs-FLOPs/$ distinction, MFU, and idle time
- Methodology - what is measured vs assumed, and why
- Break-even model - the TPS each device must hit
- Token economics and margin - $0.02/1M tokens applied
- Benchmark plan - how to get the missing TPS numbers
- cost-model.csv - the parametric table
revenue_per_token = $0.02 / 1,000,000 = $2e-8
revenue_per_hour(T) = T (tokens/s) * 3600 * $2e-8 = $7.2e-5 * T
gross_margin(T) = 1 - cost_per_hour / revenue_per_hour(T)
breakeven_TPS = cost_per_hour / $7.2e-5
| Device | $/hr | Break-even TPS (tokens/s to cover cost) |
|---|---|---|
| L4 (g2-standard-4) | $0.7092 | 9,850 |
| TPU v5e | $1.20 | 16,667 |
| TPU v6e | $2.70 | 37,500 |
| TPU v5p | $4.20 | 58,333 |
| Ironwood | $12.00 | 166,667 |
At equal throughput, the cheaper device always yields more tokens per dollar, so L4 wins by construction. TPU only wins if it runs faster by at least its price ratio to L4:
| TPU | Required speedup vs one L4 to be cost-competitive |
|---|---|
| v5e | >= 1.7x |
| v6e | >= 3.8x |
For 0.2-1.7B embedding models, a single TPU chip clearing these bars is plausible only for v5e and only if the model saturates the chip - which small models often do not. This must be measured.
A spec-grounded answer to "could TPU even hit these speedups?" lives in 02-hardware/throughput-ceiling-analysis.md: v5e's raw 1.6x speed is exactly cancelled by its 1.7x price (compute-per-dollar 0.96x = parity, no gain); only v6e has a >1x compute-per-dollar (1.99x ceiling, ~1.3-1.5x realistic), and only on a saturated lane.
Throughput for v5 on TPU cannot currently be measured (no software path, see 04-feasibility). The CSV shows margin across a range of assumed TPS to illustrate the shape of the economics. Treat every throughput cell as "to be replaced with a benchmark," per benchmark-plan.md. Do not cite these margins as results.