11title : activation_recompute
22validated_on : " 2026-08-12"
3- guidance_updated_on : " 2026-08-25"
43summary : >
54 Selective activation recompute trades replayed forward work for lower retained
65 activation memory. The correct checkpoint boundary is architecture- and
@@ -17,8 +16,6 @@ validation_status:
1716 - code_verified # transformer_config.py and transformer_block.py
1817 recompute_modules_enum :
1918 - code_verified # transformer_config.py
20- qwen35_gdn_boundary :
21- - code_verified # pinned MCore gdn_norm_out boundary and current Bridge recipe use
2219 llama3_70b_sft_fp8_cs_experiment :
2320 - measured # historical PR #3107; contextual, not a cross-module ranking
2421 moonlight_16b_bf16_experiment :
@@ -121,7 +118,6 @@ recommended_path:
121118 standard_attention : test core_attn, but compare it with an empty list under fused or Flash Attention
122119 mla : test mla_up_proj first when expanded Q/K/V tensors dominate; add core_attn only with evidence
123120 grouped_moe : test moe_act and then layernorm when their outputs drive the peak
124- qwen35_gdn : test gdn_norm_out on the exact revision; core_attn alone does not cover the GDN normalization output
125121 dense_ffn : test mlp only when its broader replay cost is acceptable
126122 broader_moe : reserve whole moe for cases where narrower MoE boundaries do not fit
127123 last_resort : use full recompute with recompute_method and recompute_num_layers
@@ -339,3 +335,10 @@ follow_up_validation:
339335 - Add matched GDN measurements.
340336 - Measure context-parallel communication replay for attention and GDN boundaries.
341337 - Validate representative selective boundaries under the supported CUDA-graph scopes and FP8 recipes.
338+
339+ guidance_updated_on : " 2026-08-25"
340+ operational_guidance :
341+ qwen35_gdn : >
342+ Test gdn_norm_out on the exact revision; core_attn alone does not cover the
343+ GDN normalization output. Use full-layer recompute when targeted boundaries
344+ still do not make the complete optimizer step fit.
0 commit comments