graph LR
Metric_Definition_API["Metric Definition & API"]
Core_Metric_Computation_Engine["Core Metric Computation Engine"]
Distributed_Metric_Orchestration["Distributed Metric Orchestration"]
Metric_Primitives_State_Management["Metric Primitives & State Management"]
Metric_Definition_API -- "delegates requests to" --> Core_Metric_Computation_Engine
Metric_Definition_API -- "orchestrates handling with" --> Distributed_Metric_Orchestration
Core_Metric_Computation_Engine -- "utilizes" --> Metric_Primitives_State_Management
Distributed_Metric_Orchestration -- "manages state with" --> Metric_Primitives_State_Management
Distributed_Metric_Orchestration -- "leverages" --> Core_Metric_Computation_Engine
The metric subsystem in EasyRec is structured around a clear separation of concerns, facilitating both ease of use and robust distributed computation. The Metric Definition & API acts as the primary entry point, allowing users to define and configure various evaluation metrics. This component then delegates the actual computation to the Core Metric Computation Engine, which houses the fundamental algorithms for individual metrics like AUC and F1-score. For distributed environments, the Distributed Metric Orchestration component takes over, coordinating metric calculations and aggregations across multiple workers or replicas, leveraging the core computation engine as needed. Both the core computation and distributed orchestration rely heavily on the Metric Primitives & State Management component, which provides the necessary low-level utilities for managing metric-related state and TensorFlow variables, ensuring accurate and consistent metric accumulation throughout the training and evaluation processes. This layered architecture ensures modularity, scalability, and maintainability of the metric evaluation framework.
This component serves as the high-level public interface for users to define, configure, and access various evaluation metrics (e.g., AUC, GAUC, Session AUC). It acts as a facade, abstracting the underlying metric implementations and providing a clean entry point for metric configuration.
Related Classes/Methods:
easy_rec.python.core.metrics.max_f1:25-56easy_rec.python.core.metrics.gauc:260-277easy_rec.python.core.metrics.session_auc:280-297
This is the central hub containing the fundamental logic and implementations for a wide range of individual metrics (e.g., AUC, Precision, Recall, Mean). It performs the actual calculations based on model predictions and ground truth labels. This component is primarily accessed through the Metric Definition & API.
Related Classes/Methods:
easy_rec.python.core.metrics.fast_auc:111-155easy_rec.python.core.metrics._separated_auc_impl:59-108
This component manages the orchestration, execution, and aggregation of metric computations specifically tailored for distributed environments, including Alibaba Cloud PAI and generic TensorFlow distributed setups. It ensures metrics are correctly calculated and combined across multiple workers or replicas.
Related Classes/Methods:
easy_rec.python.core.easyrec_metrics.distribute_metrics_impl_paieasy_rec.python.core.easyrec_metrics.distribute_metrics_impl_tfeasy_rec.python.core.metrics._distribute_separated_auc_impl:158-257
This component provides foundational utilities for accumulating metric-related state (e.g., confusion matrix components, running sums) and managing TensorFlow variables used to maintain this state across batches and steps during the training and evaluation phases. These primitives are utilized by the Core Metric Computation Engine and Distributed Metric Orchestration for their calculations.
Related Classes/Methods: