graph LR
Classification_Evaluation_Module["Classification Evaluation Module"]
MSRP_Evaluation_Module["MSRP Evaluation Module"]
SICK_Evaluation_Module["SICK Evaluation Module"]
TREC_Evaluation_Module["TREC Evaluation Module"]
Ranking_Evaluation_Module["Ranking Evaluation Module"]
The Evaluation System subsystem is a comprehensive suite designed to assess the Skip-Thoughts model's performance across various NLP downstream tasks. Its boundaries are defined by the distinct evaluation workflows for classification, paraphrase detection (MSRP), compositional knowledge (SICK), question classification (TREC), and ranking tasks.
Orchestrates the complete evaluation workflow for classification tasks, including robust nested k-fold cross-validation. It handles the loading of classification-specific datasets and the execution of the evaluation process.
Related Classes/Methods:
Manages the entire evaluation process for paraphrase detection on the MSRP dataset. This includes loading the MSRP-specific data and performing k-fold statistical evaluation to assess model performance.
Related Classes/Methods:
Directs the evaluation of compositional knowledge using the SICK dataset. This module is responsible for preparing the model for SICK-specific tasks and overseeing its training process before evaluation.
Related Classes/Methods:
Oversees the evaluation of question classification on the TREC dataset. It handles the loading of TREC-specific data and executes k-fold cross-validation to measure the model's accuracy in classifying questions.
Related Classes/Methods:
Coordinates the evaluation of ranking tasks, which involves building the encoder, performing text-to-image (t2i) and image-to-text (i2t) retrieval operations, and managing model parameters.
Related Classes/Methods: