Gemma is Google's original lightweight open model family. Megatron Bridge supports Gemma causal language models through the GemmaBridge implementation for the Hugging Face GemmaForCausalLM architecture.
Megatron Bridge supports Hugging Face Gemma checkpoints that use the gemma model type, including:
- Gemma 2B: https://huggingface.co/google/gemma-2b
- Gemma 7B: https://huggingface.co/google/gemma-7b
- Gemma release collection: https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b
- RMSNorm with zero-centered gamma.
- GeGLU-style gated MLPs.
- RoPE positional embeddings and flash attention backend.
- Shared input/output embedding weights.
Gemma uses the common conversion and generation entry points:
uv run python examples/conversion/convert_checkpoints.py import \
--hf-model google/gemma-2b \
--megatron-path /checkpoints/gemma_2b_megatronuv run python examples/conversion/hf_to_megatron_generate_text.py \
--hf_model_path google/gemma-2b \
--megatron_model_path /checkpoints/gemma_2b_megatron \
--prompt "What is artificial intelligence?"- Bridge implementation:
src/megatron/bridge/models/gemma/gemma_bridge.py - Conversion examples:
examples/conversion