graph LR
Model_Frontend_IR_Management["Model Frontend & IR Management"]
Compiler_Optimization_Scheduling["Compiler Optimization & Scheduling"]
Code_Generation_Runtime["Code Generation & Runtime"]
Hardware_Abstraction_Deployment["Hardware Abstraction & Deployment"]
Python_Control_Plane_Extensions["Python Control Plane & Extensions"]
Model_Frontend_IR_Management -- "passes ingested models as IRs for optimization to" --> Compiler_Optimization_Scheduling
Compiler_Optimization_Scheduling -- "provides optimized IRs for target-specific code generation to" --> Code_Generation_Runtime
Code_Generation_Runtime -- "deploys and executes compiled modules on" --> Hardware_Abstraction_Deployment
Hardware_Abstraction_Deployment -- "exchanges hardware configurations and device contexts with" --> Code_Generation_Runtime
Code_Generation_Runtime -- "facilitates remote execution and deployment via RPC with" --> Hardware_Abstraction_Deployment
Python_Control_Plane_Extensions -- "uses APIs to load and define models in" --> Model_Frontend_IR_Management
Python_Control_Plane_Extensions -- "uses APIs to apply optimization passes and configure tuning in" --> Compiler_Optimization_Scheduling
click Model_Frontend_IR_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/tvm/Model_Frontend_IR_Management.md" "Details"
click Compiler_Optimization_Scheduling href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/tvm/Compiler_Optimization_Scheduling.md" "Details"
click Code_Generation_Runtime href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/tvm/Code_Generation_Runtime.md" "Details"
click Hardware_Abstraction_Deployment href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/tvm/Hardware_Abstraction_Deployment.md" "Details"
click Python_Control_Plane_Extensions href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/tvm/Python_Control_Plane_Extensions.md" "Details"
The TVM architecture is structured as a modular compilation stack, enabling efficient deployment of deep learning models across diverse hardware. The Python Control Plane & Extensions acts as the orchestrator, guiding models from ingestion through optimization to final execution. Models are initially processed by the Model Frontend & IR Management component, which converts them into TVM's internal intermediate representations (IRs). These IRs then flow to the Compiler Optimization & Scheduling component for extensive transformations and performance tuning. Subsequently, the optimized IRs are passed to the Code Generation & Runtime component, which compiles them into target-specific executables. Finally, the Hardware Abstraction & Deployment component provides the necessary interfaces and runtime environment for executing these compiled modules on various devices, including remote execution capabilities. This clear, unidirectional flow from frontend to backend, controlled by the Python layer, ensures a robust and extensible compilation framework.
Model Frontend & IR Management [Expand]
The initial stage for ingesting deep learning models from various frameworks (e.g., PyTorch, ONNX) and converting them into TVM's internal Relax and TensorIR. It also provides tools for building and manipulating these IRs.
Related Classes/Methods:
tvm.relax.frontend.onnx.onnx_frontend.from_onnx:3897-3985tvm.relax.frontend.torch.fx_translator.from_fx:1121-1228tvm.ir.module.IRModule:33-312tvm.relax.block_builder.function:207-262tvm.relax.op.base.call_tir:93-132
Compiler Optimization & Scheduling [Expand]
Applies a wide array of transformations and optimizations to the IRs, including graph-level optimizations, hardware-specific scheduling, and automated performance tuning (MetaSchedule, Dlight) to prepare models for efficient execution.
Related Classes/Methods:
tvm.relax.pipeline._pipeline:158-189tvm.tir.schedule.schedule.__init__:125-181tvm.relax.backend.cuda.pipeline._pipeline:76-86tvm.meta_schedule.relax_integration.tune_relax:160-269tvm.dlight.gpu.matmul.apply:964-1124
Code Generation & Runtime [Expand]
Responsible for converting the optimized IR into executable code for specific hardware targets and providing the runtime environment for executing these compiled modules, managing memory, and device contexts.
Related Classes/Methods:
tvm.driver.build_module.compile:71-111tvm.runtime.module.export_library:137-303tvm.runtime.vm.__init__:45-91
Hardware Abstraction & Deployment [Expand]
Provides a unified interface for defining and managing diverse hardware targets (CPUs, GPUs, DSPs) and their features. It also encompasses the RPC system for remote execution and deployment of TVM programs.
Related Classes/Methods:
tvm.target.Targettvm.runtime.device.Device:29-334tvm.rpc.client.connect:481-549tvm.rpc.server._listen_loop:179-279
Python Control Plane & Extensions [Expand]
Serves as the primary user-facing interface and orchestration layer, enabling Python to control the entire TVM compilation process via Foreign Function Interface (FFI). It also includes advanced model scaling optimizations (MSC) and a library of optimized tensor operators (TOPI).
Related Classes/Methods: