graph LR
PreTrainedModel["PreTrainedModel"]
PretrainedConfig["PretrainedConfig"]
BertModel["BertModel"]
PreTrainedModel -- "uses" --> PretrainedConfig
BertModel -- "inherits from" --> PreTrainedModel
The Model Core & Architectures subsystem is defined by its role as the central hub for deep learning models within the transformers library. Its boundaries encompass foundational classes for model definition, loading, saving, and manipulation, as well as specific neural network implementations. Key source files include src/transformers/modeling_utils.py, src/transformers/configuration_utils.py, and architecture-specific files like src/transformers/models/bert/modeling_bert.py.
This is the abstract base class for all deep learning models. It provides the fundamental interface and common functionalities for model interaction, including loading pre-trained weights, saving model states, and defining the standard forward pass. It serves as the primary abstraction for model extensibility and reusability across various architectures.
Related Classes/Methods:
This component encapsulates all architectural hyperparameters and configuration details necessary to instantiate and reconstruct a specific model. It ensures consistent model definition and sharing by storing parameters like layer counts, hidden dimensions, and vocabulary sizes. It acts as the blueprint for a model's structure.
Related Classes/Methods:
This represents concrete implementations of specific neural network architectures (e.g., BERT, GPT-2, T5). These classes inherit from PreTrainedModel and define the unique layers, weights, and forward pass logic specific to their respective architectures. BertModel is a representative example of how various models are concretely implemented within the framework.
Related Classes/Methods: