Simulating performance, sustainability, and efficiency of LLM Ecosystems under inference.
This repository is the home of Kavier, the first scientific instrument for predicting performance, sustainability, and efficiency of LLM ecosystems under inference, through discrete-event, cache-aware simulation.
Kavier helps operators, researchers, and engineers predict:
- Performance — pre-fill & decode latencies, throughput, GPU utilization
- Sustainability — energy, Wh/Mtoken/s, carbon emissions (kgCO2/Mtoken/s)
- Financial efficiency — €/Mtoken/s given GPU-hour prices
We divide the documentation into the following sections:
- Getting Started
- Using the Kavier CLI
- Thesis
- Contributing guide
Questions, suggestions and contributions are welcome and appreciated! Please refer to the contributing guidelines for more details.
Kavier is distributed under the MIT license. See LICENSE.txt.