Skip to content

Configure FL backends before TP overlap initialization - #1263

Open
lihongyang1990 wants to merge 1 commit into
flagos-ai:mainfrom
lihongyang1990:fix/te-backend-before-tp-overlap
Open

Configure FL backends before TP overlap initialization#1263
lihongyang1990 wants to merge 1 commit into
flagos-ai:mainfrom
lihongyang1990:fix/te-backend-before-tp-overlap

Conversation

@lihongyang1990

Copy link
Copy Markdown
Contributor

Problem

When tp_comm_overlap is enabled, initialize_megatron() creates TransformerEngine UserBuffers before returning to pretrain(). The existing TE_FL_* and MG_FL_PREFER environment variables were only set afterward, so calls made by initialize_ub() could resolve and cache a backend before te_fl_prefer took effect.

Changes

  • Apply MG-FL and TE-FL backend-selection arguments before distributed setup and UserBuffers initialization.
  • Reset the cached TransformerEngine-FL selection policy when it was already loaded, without importing TransformerEngine during configuration.
  • Remove the late environment writes from pretrain() while keeping FlagGems enablement in its existing post-initialization position.

With the configuration applied before UserBuffers are created, no post-initialization buffer rebuild is needed. The existing distributed_backend: flagcx flow remains unchanged.

@lihongyang1990
lihongyang1990 force-pushed the fix/te-backend-before-tp-overlap branch from 53bba4c to 35860fc Compare August 7, 2026 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant