Skip to content

README.md

Latest commit

 

History

History
50 lines (39 loc) · 2.23 KB

README.md

File metadata and controls

50 lines (39 loc) · 2.23 KB
Raw

llm-d-benchmark Analysis

For an overview of the full analysis pipeline (in-container analysis, --analyze flag, per-request plots, cross-treatment comparison, and Prometheus metric visualization), see docs/analysis.md.

Jupyter Analysis Notebook

Data analysis can be performed using the Jupyter notebook analysis.ipynb using Jupyter Lab, an interactive development environment. This notebook is written in Python, and will import all benchmark report files found within a provided list of directories and populate a Pandas DataFrame. You may then execute pre-built plotting functions, modify these functions, or add your own custom analysis.

Creating a Python virtual environment

To get started, you must first have a Python ≥3.12 environment installed. If you are running Mac or Linux you likely already satisfy this requirement. For Windows you may download a Python distribution like Anaconda.

Next you will need to create a virtual environment, where we will install the requisite Python packages for analysis.

  • Create a new Python 3 virtual environment.
    Linux/Mac: 
    python -m venv /path/to/new/virtual/environment
    Windows: 
    PS> python -m venv C:\path\to\new\virtual\environment
  • Activate the virtual environment
    Linux/Mac: 
    source <venv_path>/bin/activate
    Windows: 
    C:\> <venv_path>\Scripts\activate.bat
  • Install packages from build/requirements-analysis.txt 
    pip install -r build/requirements-analysis.txt
  • Install Jupyter Lab 
    pip install jupyterlab

Running analysis.ipynb

After activating the virtual environment, launch Jupyter Lab, optionally adding the path to analysis.ipynb as an argument to open it immediately.

jupyter lab analysis.ipynb

This should open Jupyter Lab in your web browser. With the analysis notebook open, click to select the first cell, then press Shift + Enter to execute the cell. Any printouts or error messages will be shown immediately after the cell.