minimax-m1.json

{
  "machine_name": "MiniMax-M1",
  "assembly_instructions": "1. Bolt input_plate to the floor datum with its underside at y=0, then mount embedding_table above it as the first data deck. 2. Erect the four brushed-steel frame posts, lower side rails, upper service gantry, and front/back cross braces to form an open frame with no opaque outer walls. 3. Install residual_pipe through the centre of the frame and attach ten open cartridge trays to it at the specified heights. 4. Populate each tray left-to-right with seven copper lightning layer plates and one black softmax layer plate; repeat for all ten trays so the full 70 lightning plus 10 softmax stack is visible. 5. Mount the front attention_bus_lightning rail and rear attention_bus_softmax rail, then connect lightning plates to the lightning detail rack and softmax plates to the softmax GQA/KV rack. 6. Build the lightning detail rack on the left side in order: pre-RMSNorm tile, QKV feature projection, right-product kernel, fixed S state cylinder, z normalizer, and output projection. 7. Build the softmax detail rack on the right side in order: query heads, KV heads, RoPE drum, FlashAttention tile, KV cache bank, and output projection. 8. Build the MoE platform on the right: router disk, dispatch manifold, thirty-two routed expert blocks, shared expert, combine manifold, and two copper all-to-all rings, then connect the combine manifold back to residual_pipe. 9. Mount the tied prediction head, cross-entropy loss node, rollout buffer, CISPO trainer, reward model slab, AdamW block, H800 training rail, and output spout on the top service gantry. 10. Add the context ruler, sequence-parallel chunker, parallelism panel, and training data pipe; verify every part has a connection path back to input_plate through the frame, residual_pipe, or a named bus.",
  "metadata": {
    "domain": "ml-architecture",
    "reference": "https://huggingface.co/MiniMaxAI/MiniMax-M1-80k",
    "thumbnail_camera": [
      13,
      9,
      14
    ],
    "info": {
      "english_name": "MiniMax-M1 (MiniMax, 2025) — 456B-A45.9B hybrid-attention reasoning MoE",
      "summary": "456B-parameter Mixture-of-Experts reasoning model from MiniMax (45.9B activated per token). The signature feature is a HYBRID attention stack: 7 LIGHTNING-attention layers (linear-time, no KV cache) for every 1 SOFTMAX-attention layer (standard GQA with KV cache). Built on MiniMax-Text-01, then post-trained with the new CISPO RL algorithm over 41 days on H800s. 1 M-token native input context, 80 K output budget for very long chains-of-thought.",
      "description": "MiniMax-M1 is the reasoning-extension of MiniMax-Text-01. The two carry the same architecture: 80 transformer layers, hidden 6144, 32 routed experts + 1 shared with top-2 routing, but the attention is HYBRID — lightning-attention (linear) layers replace standard softmax attention 7-out-of-8 times. Lightning attention is the right-product / kernel form of linear attention: it maintains a fixed-size hidden state S ∈ R^{d_k × d_v} that is updated per token, so the per-token cost is O(d_k · d_v) regardless of context. Only the 1-in-8 softmax layers see the full O(T) attention, and they use GQA. The result: at 1M context the attention compute and KV memory are dominated by the 10 softmax layers, not the 70 lightning layers — which makes 1M context economical.\n\nM1 specifically adds reasoning via CISPO (Clipped IS-weight Policy Optimisation): a sequence-level importance-sampling RL with token-level clipping designed to handle very long rollouts (up to 80K output tokens) without the variance blow-up that kills naïve PPO/GRPO at this length.\n\nDesign rationale, in one line each:\n• Lightning attention: linear-time attention with fixed-size state — 1M context becomes affordable.\n• 7:1 hybrid: pure linear attention loses associative recall; the 1-in-8 softmax layer restores it.\n• Sparse MoE (32 experts top-2): cheap per-token capacity at large total scale.\n• CISPO RL over 41 days: the training time itself is a feature — long-horizon reasoning needs many rollout steps.",
      "facts": [
        {
          "label": "total parameters",
          "value": "456 B total, with 45.9 B activated per token"
        },
        {
          "label": "transformer depth",
          "value": "80 layers represented as 80 individual plates in ten open trays"
        },
        {
          "label": "attention ratio",
          "value": "70 lightning-attention layers and 10 softmax GQA layers, exactly 7:1"
        },
        {
          "label": "context length",
          "value": "1,000,000 input tokens with 80,000 output-token reasoning budget"
        },
        {
          "label": "hidden dimension",
          "value": "6144 channels in the residual stream"
        },
        {
          "label": "softmax GQA heads",
          "value": "64 query heads share 8 KV heads"
        },
        {
          "label": "MoE layout",
          "value": "1 shared expert plus 32 individually modelled routed experts with top-2 routing"
        },
        {
          "label": "expert width",
          "value": "d_ff=9216 per routed SwiGLU expert"
        },
        {
          "label": "RL training",
          "value": "CISPO for about 41 days on 512 H800 GPUs"
        },
        {
          "label": "scene scale",
          "value": "Open frame footprint is about 8.6 m wide, 8.6 m tall, and 4.2 m deep in descriptor units"
        }
      ],
      "comparisons": [
        {
          "versus": "vs MiniMax-Text-01 (Jan 2025)",
          "delta": "MiniMax-Text-01 introduced the 7:1 lightning/softmax hybrid and the 32-expert MoE. M1 keeps the architecture EXACTLY but adds a long, stable RL-for-reasoning phase using CISPO. M1-80k is the same base weights with a much larger output budget (80K tokens vs 8K) — the architecture had to support 1M input from day one for this to work."
        },
        {
          "versus": "vs DeepSeek-V3 and DeepSeek-R1",
          "delta": "Both are MoE reasoning models, but DeepSeek-V3/R1 use all-softmax attention with compression while M1 uses 7/8 linear attention. At 1M context, the KV footprint is driven by only the 10 softmax layers."
        },
        {
          "versus": "vs full-softmax frontier chat models",
          "delta": "M1 trades some short-context full-attention capacity for economical 1M-token context and long-output reasoning."
        }
      ],
      "sources": [
        {
          "title": "MiniMax-M1 technical report",
          "url": "https://arxiv.org/abs/2506.13585"
        },
        {
          "title": "MiniMax-M1 model card (HF)",
          "url": "https://huggingface.co/MiniMaxAI/MiniMax-M1-80k"
        },
        {
          "title": "MiniMax-Text-01 technical report",
          "url": "https://arxiv.org/abs/2501.08313"
        },
        {
          "title": "Lightning Attention / TransNormer-2",
          "url": "https://arxiv.org/abs/2401.04658"
        }
      ]
    }
  },
  "parts": [
    {
      "id": "input_plate",
      "name": "Input Token Floor Plate",
      "shape": "box",
      "position": [
        0,
        0.08,
        0
      ],
      "size": [
        4.2,
        0.16,
        2.2
      ],
      "material": "dark anodized aluminum",
      "role": "Grounded base plate carrying token ids from the tokenizer into the model; its top surface is the assembly datum.",
      "compute_profile": "host-cpu",
      "parallelism": "embarrassingly-parallel",
      "connections": [
        "embedding_table",
        "frame_post_fl",
        "frame_post_fr",
        "frame_post_bl",
        "frame_post_br",
        "training_data_pipe"
      ]
    },
    {
      "id": "embedding_table",
      "name": "Tied Embedding Table",
      "shape": "box",
      "position": [
        0,
        0.42,
        0
      ],
      "size": [
        3.7,
        0.28,
        1.75
      ],
      "material": "glass display",
      "role": "The approximately 200K by 6144 embedding slab converts token ids into the residual stream and is tied to the final LM head.",
      "compute_profile": "gpu-bandwidth-bound",
      "memory_footprint": "200K x 6144 x BF16 is about 2.46 GiB.",
      "parallelism": "tensor-parallel",
      "connections": [
        "input_plate",
        "residual_pipe"
      ]
    },
    {
      "id": "frame_post_fl",
      "name": "Front Left Open-Frame Post",
      "shape": "box",
      "position": [
        -2.35,
        4.25,
        -1.2
      ],
      "size": [
        0.12,
        8.2,
        0.12
      ],
      "material": "brushed steel",
      "role": "Left-front vertical post keeps the architecture as an open cutaway instead of an opaque chassis.",
      "connections": [
        "input_plate",
        "upper_service_gantry",
        "cross_brace_front",
        "lower_side_rail_left"
      ]
    },
    {
      "id": "frame_post_fr",
      "name": "Front Right Open-Frame Post",
      "shape": "box",
      "position": [
        2.35,
        4.25,
        -1.2
      ],
      "size": [
        0.12,
        8.2,
        0.12
      ],
      "material": "brushed steel",
      "role": "Right-front vertical post supports the exposed layer trays and leaves their interiors visible.",
      "connections": [
        "input_plate",
        "upper_service_gantry",
        "cross_brace_front",
        "lower_side_rail_right"
      ]
    },
    {
      "id": "frame_post_bl",
      "name": "Back Left Open-Frame Post",
      "shape": "box",
      "position": [
        -2.35,
        4.25,
        1.2
      ],
      "size": [
        0.12,
        8.2,
        0.12
      ],
      "material": "brushed steel",
      "role": "Left-back post gives the stack depth without hiding the attention and MoE subassemblies.",
      "connections": [
        "input_plate",
        "upper_service_gantry",
        "cross_brace_back",
        "lower_side_rail_left"
      ]
    },
    {
      "id": "frame_post_br",
      "name": "Back Right Open-Frame Post",
      "shape": "box",
      "position": [
        2.35,
        4.25,
        1.2
      ],
      "size": [
        0.12,
        8.2,
        0.12
      ],
      "material": "brushed steel",
      "role": "Right-back post completes the root frame for the transformer factory.",
      "connections": [
        "input_plate",
        "upper_service_gantry",
        "cross_brace_back",
        "lower_side_rail_right"
      ]
    },
    {
      "id": "lower_side_rail_left",
      "name": "Lower Left Side Rail",
      "shape": "box",
      "position": [
        -2.35,
        0.72,
        0
      ],
      "size": [
        0.12,
        0.1,
        2.55
      ],
      "material": "forged steel",
      "role": "Lower rail ties the left frame posts together and sets the service rack datum.",
      "connections": [
        "frame_post_fl",
        "frame_post_bl",
        "input_plate"
      ]
    },
    {
      "id": "lower_side_rail_right",
      "name": "Lower Right Side Rail",
      "shape": "box",
      "position": [
        2.35,
        0.72,
        0
      ],
      "size": [
        0.12,
        0.1,
        2.55
      ],
      "material": "forged steel",
      "role": "Lower rail ties the right frame posts together and supports the softmax service rack.",
      "connections": [
        "frame_post_fr",
        "frame_post_br",
        "input_plate"
      ]
    },
    {
      "id": "cross_brace_front",
      "name": "Front Cross Brace",
      "shape": "box",
      "position": [
        0,
        4.55,
        -1.2
      ],
      "size": [
        4.8,
        0.08,
        0.1
      ],
      "material": "forged steel",
      "role": "Horizontal brace tying the front frame posts together while staying thin enough not to occlude the layer plates.",
      "connections": [
        "frame_post_fl",
        "frame_post_fr",
        "residual_pipe",
        "moe_platform"
      ]
    },
    {
      "id": "cross_brace_back",
      "name": "Back Cross Brace",
      "shape": "box",
      "position": [
        0,
        4.55,
        1.2
      ],
      "size": [
        4.8,
        0.08,
        0.1
      ],
      "material": "forged steel",
      "role": "Back brace stabilizes the open frame and gives the top view a readable envelope.",
      "connections": [
        "frame_post_bl",
        "frame_post_br",
        "residual_pipe"
      ]
    },
    {
      "id": "upper_service_gantry",
      "name": "Upper Service Gantry",
      "shape": "box",
      "position": [
        0,
        8.42,
        0
      ],
      "size": [
        5,
        0.14,
        2.55
      ],
      "material": "brushed steel",
      "role": "Thin top rail carrying prediction, loss, reward, and CISPO training components above the transformer stack.",
      "connections": [
        "frame_post_fl",
        "frame_post_fr",
        "frame_post_bl",
        "frame_post_br",
        "prediction_head"
      ]
    },
    {
      "id": "residual_pipe",
      "name": "Residual Stream Mast d=6144",
      "shape": "cylinder",
      "position": [
        0,
        4.35,
        0
      ],
      "size": [
        0.14,
        7.65
      ],
      "material": "brass",
      "role": "Central vertical residual stream carrying hidden dimension 6144 through all 80 layers.",
      "compute_profile": "gpu-memory-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "embedding_table",
        "tray_01",
        "tray_02",
        "tray_03",
        "tray_04",
        "tray_05",
        "tray_06",
        "tray_07",
        "tray_08",
        "tray_09",
        "tray_10",
        "prediction_head"
      ]
    },
    {
      "id": "context_ruler_1m",
      "name": "1M Context Ruler",
      "shape": "box",
      "position": [
        -2.85,
        4.35,
        0
      ],
      "size": [
        0.07,
        6.75,
        0.5
      ],
      "material": "aluminum",
      "role": "Scale marker showing the million-token context that the hybrid attention system is built to support.",
      "connections": [
        "input_plate",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "attention_bus_lightning",
      "name": "Lightning Plate Bus Rail",
      "shape": "box",
      "position": [
        0,
        4.25,
        -0.72
      ],
      "size": [
        3.65,
        7,
        0.06
      ],
      "material": "copper",
      "role": "Front vertical bus visibly ties all seventy lightning plates to the linear-attention service rack.",
      "compute_profile": "gpu-bandwidth-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "lightning_kernel",
        "residual_pipe"
      ]
    },
    {
      "id": "attention_bus_softmax",
      "name": "Softmax Plate Bus Rail",
      "shape": "box",
      "position": [
        1.65,
        4.25,
        0.72
      ],
      "size": [
        0.08,
        7,
        0.06
      ],
      "material": "black composite",
      "role": "Rear vertical bus visibly ties the ten softmax plates to the GQA and KV-cache service rack.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "softmax_attn_tile",
        "softmax_kv_cache",
        "residual_pipe"
      ]
    },
    {
      "id": "tray_01",
      "name": "Open Tray for Layers 01-08",
      "shape": "box",
      "position": [
        0,
        1.02,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 1-8 without hiding the seven lightning plates or the softmax plate.",
      "connections": [
        "residual_pipe",
        "layer_01_lightning",
        "layer_08_softmax"
      ]
    },
    {
      "id": "layer_01_lightning",
      "name": "Layer 01 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        1.08,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 1 uses fixed-size recurrent state instead of a per-token KV cache.",
      "compute_profile": "gpu-bandwidth-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_01",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_02_lightning",
      "name": "Layer 02 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        1.08,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 2 continues the linear-time attention path for long context.",
      "connections": [
        "tray_01",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_03_lightning",
      "name": "Layer 03 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        1.08,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 3 is one of seven no-KV-cache layers before the first softmax recall layer.",
      "connections": [
        "tray_01",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_04_lightning",
      "name": "Layer 04 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        1.08,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 4 applies the right-product kernel to the residual stream.",
      "connections": [
        "tray_01",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_05_lightning",
      "name": "Layer 05 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        1.08,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 5 adds capacity without growing KV memory with sequence length.",
      "connections": [
        "tray_01",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_06_lightning",
      "name": "Layer 06 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        1.08,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 6 carries compact S and z state across sequence chunks.",
      "connections": [
        "tray_01",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_07_lightning",
      "name": "Layer 07 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        1.08,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 7 completes the seven-layer linear-attention run before a softmax layer.",
      "connections": [
        "tray_01",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_08_softmax",
      "name": "Layer 08 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        1.08,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 8 provides full associative recall using GQA and writes to the KV cache bank.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_01",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_02",
      "name": "Open Tray for Layers 09-16",
      "shape": "box",
      "position": [
        0,
        1.76,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 9-16 with the 7:1 cadence exposed.",
      "connections": [
        "residual_pipe",
        "layer_09_lightning",
        "layer_16_softmax"
      ]
    },
    {
      "id": "layer_09_lightning",
      "name": "Layer 09 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        1.82,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 9 begins the second eight-layer cadence after softmax layer 8.",
      "connections": [
        "tray_02",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_10_lightning",
      "name": "Layer 10 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        1.82,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 10 processes long context with fixed recurrent state.",
      "connections": [
        "tray_02",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_11_lightning",
      "name": "Layer 11 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        1.82,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 11 contributes to the no-KV-cache majority of the stack.",
      "connections": [
        "tray_02",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_12_lightning",
      "name": "Layer 12 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        1.82,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 12 routes through the shared lightning QKV feature projection.",
      "connections": [
        "tray_02",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_13_lightning",
      "name": "Layer 13 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        1.82,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 13 keeps per-token attention cost independent of the 1M-token context length.",
      "connections": [
        "tray_02",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_14_lightning",
      "name": "Layer 14 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        1.82,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 14 forwards compact state through sequence-parallel chunks.",
      "connections": [
        "tray_02",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_15_lightning",
      "name": "Layer 15 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        1.82,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 15 is the seventh linear-attention layer in this tray.",
      "connections": [
        "tray_02",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_16_softmax",
      "name": "Layer 16 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        1.82,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 16 is the second full-attention recall stage and connects to the KV cache.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_02",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_03",
      "name": "Open Tray for Layers 17-24",
      "shape": "box",
      "position": [
        0,
        2.5,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 17-24 with all plates visible from ISO.",
      "connections": [
        "residual_pipe",
        "layer_17_lightning",
        "layer_24_softmax"
      ]
    },
    {
      "id": "layer_17_lightning",
      "name": "Layer 17 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        2.56,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 17 begins the third visible 7:1 attention cadence.",
      "connections": [
        "tray_03",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_18_lightning",
      "name": "Layer 18 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        2.56,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 18 uses the linear kernel path for economical context processing.",
      "connections": [
        "tray_03",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_19_lightning",
      "name": "Layer 19 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        2.56,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 19 is a countable copper plate in the seventy-layer linear majority.",
      "connections": [
        "tray_03",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_20_lightning",
      "name": "Layer 20 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        2.56,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 20 applies QKV feature projection before the right-product update.",
      "connections": [
        "tray_03",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_21_lightning",
      "name": "Layer 21 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        2.56,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 21 stores only compact S and z state rather than token-wise keys and values.",
      "connections": [
        "tray_03",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_22_lightning",
      "name": "Layer 22 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        2.56,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 22 passes recurrent state between sequence shards.",
      "connections": [
        "tray_03",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_23_lightning",
      "name": "Layer 23 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        2.56,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 23 is the last linear layer before softmax layer 24.",
      "connections": [
        "tray_03",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_24_softmax",
      "name": "Layer 24 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        2.56,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 24 is the third GQA recall layer with full causal attention.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_03",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_04",
      "name": "Open Tray for Layers 25-32",
      "shape": "box",
      "position": [
        0,
        3.24,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 25-32 and continuing the exact hybrid cadence.",
      "connections": [
        "residual_pipe",
        "layer_25_lightning",
        "layer_32_softmax"
      ]
    },
    {
      "id": "layer_25_lightning",
      "name": "Layer 25 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        3.3,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 25 begins the fourth eight-layer module in the transformer stack.",
      "connections": [
        "tray_04",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_26_lightning",
      "name": "Layer 26 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        3.3,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 26 uses fixed-size state to reduce long-context memory pressure.",
      "connections": [
        "tray_04",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_27_lightning",
      "name": "Layer 27 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        3.3,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 27 contributes one visible unit to the 70-layer lightning total.",
      "connections": [
        "tray_04",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_28_lightning",
      "name": "Layer 28 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        3.3,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 28 receives projected Q, K, and V features from the lightning rack.",
      "connections": [
        "tray_04",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_29_lightning",
      "name": "Layer 29 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        3.3,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 29 accumulates recurrent state rather than growing a token cache.",
      "connections": [
        "tray_04",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_30_lightning",
      "name": "Layer 30 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        3.3,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 30 links to sequence parallelism for million-token prefill.",
      "connections": [
        "tray_04",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_31_lightning",
      "name": "Layer 31 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        3.3,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 31 completes the seven-layer linear run before layer 32.",
      "connections": [
        "tray_04",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_32_softmax",
      "name": "Layer 32 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        3.3,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 32 is the fourth full-attention layer and contributes KV entries to the cache bank.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_04",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_05",
      "name": "Open Tray for Layers 33-40",
      "shape": "box",
      "position": [
        0,
        3.98,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 33-40 at the middle of the tower.",
      "connections": [
        "residual_pipe",
        "layer_33_lightning",
        "layer_40_softmax"
      ]
    },
    {
      "id": "layer_33_lightning",
      "name": "Layer 33 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        4.04,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 33 starts the fifth 7:1 tray at mid-depth in the model.",
      "connections": [
        "tray_05",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_34_lightning",
      "name": "Layer 34 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        4.04,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 34 keeps attention memory fixed with respect to sequence length.",
      "connections": [
        "tray_05",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_35_lightning",
      "name": "Layer 35 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        4.04,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 35 is a visible copper plate in the linear-attention majority.",
      "connections": [
        "tray_05",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_36_lightning",
      "name": "Layer 36 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        4.04,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 36 consumes QKV features from the projection tile.",
      "connections": [
        "tray_05",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_37_lightning",
      "name": "Layer 37 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        4.04,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 37 stores recurrent S and z summaries instead of token-wise cache rows.",
      "connections": [
        "tray_05",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_38_lightning",
      "name": "Layer 38 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        4.04,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 38 communicates compact recurrent state across long sequence shards.",
      "connections": [
        "tray_05",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_39_lightning",
      "name": "Layer 39 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        4.04,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 39 precedes the fifth softmax recall layer.",
      "connections": [
        "tray_05",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_40_softmax",
      "name": "Layer 40 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        4.04,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 40 is the fifth GQA recall layer and sits at the centre of the stack.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_05",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_06",
      "name": "Open Tray for Layers 41-48",
      "shape": "box",
      "position": [
        0,
        4.72,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 41-48 above the midpoint.",
      "connections": [
        "residual_pipe",
        "layer_41_lightning",
        "layer_48_softmax"
      ]
    },
    {
      "id": "layer_41_lightning",
      "name": "Layer 41 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        4.78,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 41 starts the sixth exposed hybrid tray.",
      "connections": [
        "tray_06",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_42_lightning",
      "name": "Layer 42 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        4.78,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 42 provides linear-time attention in the upper half of the model.",
      "connections": [
        "tray_06",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_43_lightning",
      "name": "Layer 43 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        4.78,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 43 avoids per-token KV memory growth.",
      "connections": [
        "tray_06",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_44_lightning",
      "name": "Layer 44 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        4.78,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 44 receives projected features and emits to the residual mast through the output projection.",
      "connections": [
        "tray_06",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_45_lightning",
      "name": "Layer 45 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        4.78,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 45 carries a compact recurrent summary across the million-token context.",
      "connections": [
        "tray_06",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_46_lightning",
      "name": "Layer 46 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        4.78,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 46 is chunked during long prefill for sequence parallelism.",
      "connections": [
        "tray_06",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_47_lightning",
      "name": "Layer 47 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        4.78,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 47 is the seventh lightning layer before softmax layer 48.",
      "connections": [
        "tray_06",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_48_softmax",
      "name": "Layer 48 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        4.78,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 48 is the sixth full-attention layer and restores associative recall.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_06",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_07",
      "name": "Open Tray for Layers 49-56",
      "shape": "box",
      "position": [
        0,
        5.46,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 49-56 with no enclosing side wall.",
      "connections": [
        "residual_pipe",
        "layer_49_lightning",
        "layer_56_softmax"
      ]
    },
    {
      "id": "layer_49_lightning",
      "name": "Layer 49 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        5.52,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 49 begins the seventh eight-layer hybrid tray.",
      "connections": [
        "tray_07",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_50_lightning",
      "name": "Layer 50 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        5.52,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 50 keeps the majority of attention compute linear in context.",
      "connections": [
        "tray_07",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_51_lightning",
      "name": "Layer 51 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        5.52,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 51 is one visible element of the upper copper stack.",
      "connections": [
        "tray_07",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_52_lightning",
      "name": "Layer 52 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        5.52,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 52 uses the same QKV feature projection as the other lightning layers.",
      "connections": [
        "tray_07",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_53_lightning",
      "name": "Layer 53 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        5.52,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 53 stores fixed recurrent state rather than a growing KV buffer.",
      "connections": [
        "tray_07",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_54_lightning",
      "name": "Layer 54 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        5.52,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 54 participates in chunked million-token prefill.",
      "connections": [
        "tray_07",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_55_lightning",
      "name": "Layer 55 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        5.52,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 55 completes the linear run before softmax layer 56.",
      "connections": [
        "tray_07",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_56_softmax",
      "name": "Layer 56 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        5.52,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 56 is the seventh GQA recall point in the eighty-layer stack.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_07",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_08",
      "name": "Open Tray for Layers 57-64",
      "shape": "box",
      "position": [
        0,
        6.2,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 57-64 near the top of the tower.",
      "connections": [
        "residual_pipe",
        "layer_57_lightning",
        "layer_64_softmax"
      ]
    },
    {
      "id": "layer_57_lightning",
      "name": "Layer 57 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        6.26,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 57 begins the eighth visible 7:1 tray.",
      "connections": [
        "tray_08",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_58_lightning",
      "name": "Layer 58 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        6.26,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 58 continues fixed-state attention above the model midpoint.",
      "connections": [
        "tray_08",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_59_lightning",
      "name": "Layer 59 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        6.26,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 59 adds another visible count to the seventy lightning layers.",
      "connections": [
        "tray_08",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_60_lightning",
      "name": "Layer 60 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        6.26,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 60 receives QKV features before recurrent state update.",
      "connections": [
        "tray_08",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_61_lightning",
      "name": "Layer 61 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        6.26,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 61 carries compact recurrent summaries for long-context reasoning.",
      "connections": [
        "tray_08",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_62_lightning",
      "name": "Layer 62 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        6.26,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 62 participates in sequence-parallel state transfer.",
      "connections": [
        "tray_08",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_63_lightning",
      "name": "Layer 63 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        6.26,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 63 is the seventh linear layer before softmax layer 64.",
      "connections": [
        "tray_08",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_64_softmax",
      "name": "Layer 64 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        6.26,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 64 is the eighth full-attention recall stage.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_08",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_09",
      "name": "Open Tray for Layers 65-72",
      "shape": "box",
      "position": [
        0,
        6.94,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Thin open rail tray holding layers 65-72 just below the prediction head.",
      "connections": [
        "residual_pipe",
        "layer_65_lightning",
        "layer_72_softmax"
      ]
    },
    {
      "id": "layer_65_lightning",
      "name": "Layer 65 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        7,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 65 starts the ninth eight-layer tray.",
      "connections": [
        "tray_09",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_66_lightning",
      "name": "Layer 66 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        7,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 66 continues the economical long-context attention path.",
      "connections": [
        "tray_09",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_67_lightning",
      "name": "Layer 67 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        7,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 67 is a visible copper unit near the model output end.",
      "connections": [
        "tray_09",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_68_lightning",
      "name": "Layer 68 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        7,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 68 uses feature projection and right-product attention.",
      "connections": [
        "tray_09",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_69_lightning",
      "name": "Layer 69 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        7,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 69 stores recurrent state independent of context length.",
      "connections": [
        "tray_09",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_70_lightning",
      "name": "Layer 70 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        7,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 70 is the final lightning layer in the 70-layer total.",
      "connections": [
        "tray_09",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_71_lightning",
      "name": "Layer 71 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        7,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 71 is the seventh lightning plate in tray nine before softmax layer 72.",
      "connections": [
        "tray_09",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_72_softmax",
      "name": "Layer 72 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        7,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 72 is the ninth GQA recall stage before the final tray.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_09",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache"
      ]
    },
    {
      "id": "tray_10",
      "name": "Open Tray for Layers 73-80",
      "shape": "box",
      "position": [
        0,
        7.68,
        0
      ],
      "size": [
        3.5,
        0.08,
        1.55
      ],
      "material": "brushed steel",
      "role": "Final open rail tray holding layers 73-80 before the tied LM head.",
      "connections": [
        "residual_pipe",
        "layer_73_lightning",
        "layer_80_softmax",
        "prediction_head"
      ]
    },
    {
      "id": "layer_73_lightning",
      "name": "Layer 73 Lightning Attention",
      "shape": "box",
      "position": [
        -1.4,
        7.74,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 73 starts the final hybrid cadence before the output head.",
      "connections": [
        "tray_10",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_74_lightning",
      "name": "Layer 74 Lightning Attention",
      "shape": "box",
      "position": [
        -1,
        7.74,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 74 keeps the final tray's attention mostly linear-time.",
      "connections": [
        "tray_10",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_75_lightning",
      "name": "Layer 75 Lightning Attention",
      "shape": "box",
      "position": [
        -0.6,
        7.74,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 75 remains visible as a separate copper plate, not hidden in a cartridge block.",
      "connections": [
        "tray_10",
        "attention_bus_lightning",
        "lightning_kernel"
      ]
    },
    {
      "id": "layer_76_lightning",
      "name": "Layer 76 Lightning Attention",
      "shape": "box",
      "position": [
        -0.2,
        7.74,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 76 receives QKV feature projections in the final tray.",
      "connections": [
        "tray_10",
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "layer_77_lightning",
      "name": "Layer 77 Lightning Attention",
      "shape": "box",
      "position": [
        0.2,
        7.74,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 77 carries fixed S and z state near the model output.",
      "connections": [
        "tray_10",
        "attention_bus_lightning",
        "lightning_state_s"
      ]
    },
    {
      "id": "layer_78_lightning",
      "name": "Layer 78 Lightning Attention",
      "shape": "box",
      "position": [
        0.6,
        7.74,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 78 forwards compact recurrence through the final sequence chunk.",
      "connections": [
        "tray_10",
        "attention_bus_lightning",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "layer_79_lightning",
      "name": "Layer 79 Lightning Attention",
      "shape": "box",
      "position": [
        1,
        7.74,
        -0.58
      ],
      "size": [
        0.24,
        0.09,
        0.3
      ],
      "material": "copper",
      "role": "Lightning layer 79 is the final lightning plate before the last full softmax recall layer.",
      "connections": [
        "tray_10",
        "attention_bus_lightning",
        "lightning_o_proj"
      ]
    },
    {
      "id": "layer_80_softmax",
      "name": "Layer 80 Softmax GQA",
      "shape": "box",
      "position": [
        1.4,
        7.74,
        0.58
      ],
      "size": [
        0.36,
        0.12,
        0.38
      ],
      "material": "black composite",
      "role": "Softmax layer 80 is the tenth and final full-attention recall stage before the tied LM head.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "tray_10",
        "attention_bus_softmax",
        "softmax_attn_tile",
        "softmax_kv_cache",
        "prediction_head"
      ]
    },
    {
      "id": "lightning_pre_norm",
      "name": "Lightning Pre-RMSNorm Tile",
      "shape": "box",
      "position": [
        -3.15,
        2.05,
        -0.72
      ],
      "size": [
        0.72,
        0.14,
        0.46
      ],
      "material": "white composite",
      "role": "Pre-normalization tile feeding lightning attention sublayers before QKV projection.",
      "compute_profile": "gpu-bandwidth-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "attention_bus_lightning",
        "lightning_qkv_proj"
      ]
    },
    {
      "id": "lightning_qkv_proj",
      "name": "Lightning QKV and Feature Projection",
      "shape": "box",
      "position": [
        -3.15,
        2.5,
        -0.72
      ],
      "size": [
        0.88,
        0.22,
        0.58
      ],
      "material": "aluminum",
      "role": "Projects hidden states into Q, K, and V and applies the non-negative feature map used by right-product linear attention.",
      "compute_profile": "gpu-compute-bound",
      "flops_per_token": "Approximately 200 MFLOP per token for the projection stage.",
      "parallelism": "tensor-parallel",
      "algorithm": "Q = phi(x @ W_Q); K = phi(x @ W_K); V = x @ W_V",
      "connections": [
        "lightning_pre_norm",
        "lightning_kernel"
      ]
    },
    {
      "id": "lightning_kernel",
      "name": "Right-Product Lightning Kernel",
      "shape": "box",
      "position": [
        -3.15,
        3,
        -0.72
      ],
      "size": [
        1,
        0.28,
        0.68
      ],
      "material": "glass display",
      "role": "Maintains S += K outer V and emits Q times S divided by Q times z, making attention cost independent of context length.",
      "compute_profile": "gpu-bandwidth-bound",
      "flops_per_token": "About 4*d_k*d_v for the recurrent update and readout.",
      "memory_footprint": "State size is fixed per layer and sequence rather than proportional to token count.",
      "parallelism": "sequential",
      "algorithm": "S += outer(K_t,V_t); z += K_t; out_t = (Q_t @ S) / (Q_t @ z + eps)",
      "bottleneck": "Decode is sequential along time; chunked prefill restores throughput on long inputs.",
      "connections": [
        "lightning_qkv_proj",
        "lightning_state_s",
        "lightning_state_z",
        "lightning_o_proj"
      ]
    },
    {
      "id": "lightning_state_s",
      "name": "Lightning Fixed State S",
      "shape": "cylinder",
      "position": [
        -4.05,
        3,
        -0.72
      ],
      "size": [
        0.34,
        0.9
      ],
      "material": "copper",
      "role": "Fixed-size recurrent matrix S for lightning layers; it replaces per-token KV storage for 70 of the 80 layers.",
      "compute_profile": "gpu-memory-bound",
      "memory_footprint": "Roughly 1-2 MB per sequence per layer, independent of T.",
      "parallelism": "data-parallel",
      "connections": [
        "lightning_kernel",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "lightning_state_z",
      "name": "Lightning Normalizer z",
      "shape": "cylinder",
      "position": [
        -4.05,
        2.28,
        -0.72
      ],
      "size": [
        0.16,
        0.7
      ],
      "material": "brass",
      "role": "Accumulator z normalizes the linear attention readout and is carried alongside S across chunks.",
      "compute_profile": "gpu-memory-bound",
      "parallelism": "data-parallel",
      "connections": [
        "lightning_kernel",
        "sequence_parallel_chunker"
      ]
    },
    {
      "id": "lightning_o_proj",
      "name": "Lightning Output Projection",
      "shape": "box",
      "position": [
        -3.15,
        3.48,
        -0.72
      ],
      "size": [
        0.82,
        0.2,
        0.54
      ],
      "material": "brass",
      "role": "Projects lightning attention output back to hidden dimension 6144 and returns it to the residual stream.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "lightning_kernel",
        "residual_pipe",
        "attention_bus_lightning"
      ]
    },
    {
      "id": "softmax_q_heads",
      "name": "Softmax 64 Query Heads",
      "shape": "box",
      "position": [
        3.05,
        2.7,
        -0.7
      ],
      "size": [
        0.96,
        0.2,
        0.54
      ],
      "material": "aluminum",
      "role": "Q projection bank for the ten GQA softmax layers with 64 query heads.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "attention_bus_softmax",
        "rope_drum",
        "softmax_attn_tile"
      ]
    },
    {
      "id": "softmax_kv_heads",
      "name": "Softmax 8 KV Heads",
      "shape": "box",
      "position": [
        3.05,
        3.12,
        -0.7
      ],
      "size": [
        0.74,
        0.2,
        0.54
      ],
      "material": "brushed steel",
      "role": "Grouped-query K and V projection bank shared by 64 query heads through 8 KV heads.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "rope_drum",
        "softmax_kv_cache",
        "softmax_attn_tile"
      ]
    },
    {
      "id": "rope_drum",
      "name": "RoPE Position Drum",
      "shape": "cylinder",
      "position": [
        3.9,
        2.9,
        -0.7
      ],
      "rotation": [
        0,
        0,
        1.5708
      ],
      "size": [
        0.22,
        0.72
      ],
      "material": "copper",
      "role": "Rotary position embedding drum applies long-context position phase to Q and K before softmax attention.",
      "compute_profile": "gpu-bandwidth-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "softmax_q_heads",
        "softmax_kv_heads"
      ]
    },
    {
      "id": "softmax_attn_tile",
      "name": "FlashAttention Softmax Tile",
      "shape": "box",
      "position": [
        3.05,
        3.62,
        -0.7
      ],
      "size": [
        1.05,
        0.28,
        0.68
      ],
      "material": "black composite",
      "role": "Full causal softmax attention over the long context; it exists in only ten layers but dominates attention compute.",
      "compute_profile": "gpu-compute-bound",
      "flops_per_token": "Scales with T for training and prefill, unlike lightning attention.",
      "memory_footprint": "Uses on-chip SRAM tiles during FlashAttention and writes KV to the cache bank.",
      "parallelism": "embarrassingly-parallel",
      "algorithm": "out = flash_attn(Q, K, V, mask=causal)",
      "bottleneck": "At 1M context these ten layers dominate KV memory and attention compute.",
      "connections": [
        "softmax_q_heads",
        "softmax_kv_heads",
        "softmax_kv_cache",
        "softmax_o_proj"
      ]
    },
    {
      "id": "softmax_kv_cache",
      "name": "Softmax KV Cache Bank",
      "shape": "cylinder",
      "position": [
        4.1,
        4.55,
        -0.7
      ],
      "size": [
        0.4,
        3.5
      ],
      "material": "glass display",
      "role": "Tall cache bank stores GQA K and V only for the ten softmax layers, not for the seventy lightning layers.",
      "compute_profile": "memory-capacity-bound",
      "memory_footprint": "About 38 GiB per 1M-token sequence for 10 softmax layers under the stated GQA estimate.",
      "parallelism": "tensor-parallel",
      "connections": [
        "softmax_attn_tile",
        "context_ruler_1m",
        "parallelism_panel",
        "attention_bus_softmax"
      ]
    },
    {
      "id": "softmax_o_proj",
      "name": "Softmax Output Projection",
      "shape": "box",
      "position": [
        3.05,
        4.08,
        -0.7
      ],
      "size": [
        0.82,
        0.2,
        0.54
      ],
      "material": "brass",
      "role": "Projects the softmax attention result back to hidden dimension and returns it to the residual stream.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "softmax_attn_tile",
        "residual_pipe",
        "attention_bus_softmax"
      ]
    },
    {
      "id": "moe_platform",
      "name": "MoE Service Platform",
      "shape": "box",
      "position": [
        5.65,
        3.55,
        0.62
      ],
      "size": [
        2.75,
        0.14,
        1.85
      ],
      "material": "dark anodized aluminum",
      "role": "Right-side platform that carries the router, shared expert, routed experts, and all-to-all communication hardware.",
      "connections": [
        "cross_brace_front",
        "moe_router",
        "expert_01",
        "expert_32"
      ]
    },
    {
      "id": "moe_router",
      "name": "Top-2 Router Disk",
      "shape": "cylinder",
      "position": [
        5.65,
        3.86,
        0.62
      ],
      "size": [
        0.38,
        0.14
      ],
      "material": "glass display",
      "role": "Softmax router scores 32 experts and selects top-2 routed experts per token with a capacity gate.",
      "compute_profile": "gpu-bandwidth-bound",
      "flops_per_token": "About 2*d*32, roughly 0.4 MFLOP per token.",
      "parallelism": "tensor-parallel",
      "algorithm": "scores = softmax(x @ W_router); topk_idx, topk_w = topk(scores, 2)",
      "connections": [
        "moe_platform",
        "dispatch_manifold",
        "moe_shared_expert"
      ]
    },
    {
      "id": "dispatch_manifold",
      "name": "Expert Dispatch Manifold",
      "shape": "cylinder",
      "position": [
        4.6,
        4.42,
        0.62
      ],
      "rotation": [
        0,
        0,
        1.5708
      ],
      "size": [
        0.1,
        1.25
      ],
      "material": "copper",
      "role": "Dispatch manifold buckets tokens by selected expert before expert-parallel all-to-all exchange.",
      "compute_profile": "communication-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "moe_router",
        "alltoall_ring_a",
        "expert_01",
        "expert_08",
        "expert_16",
        "expert_24",
        "expert_32"
      ]
    },
    {
      "id": "combine_manifold",
      "name": "Expert Combine Manifold",
      "shape": "cylinder",
      "position": [
        6.7,
        4.42,
        0.62
      ],
      "rotation": [
        0,
        0,
        1.5708
      ],
      "size": [
        0.1,
        1.25
      ],
      "material": "copper",
      "role": "Combine manifold returns weighted top-2 expert outputs to the residual stream after expert execution.",
      "compute_profile": "communication-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "alltoall_ring_b",
        "expert_01",
        "expert_08",
        "expert_16",
        "expert_24",
        "expert_32",
        "residual_pipe"
      ]
    },
    {
      "id": "alltoall_ring_a",
      "name": "NCCL All-to-All Dispatch Ring",
      "shape": "torus",
      "position": [
        5.1,
        4.9,
        0.62
      ],
      "rotation": [
        1.5708,
        0,
        0
      ],
      "size": [
        0.46,
        0.05
      ],
      "material": "copper",
      "role": "Copper ring represents the first all-to-all exchange that sends token batches to owning expert ranks.",
      "compute_profile": "communication-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "alltoall_ring_b",
      "name": "NCCL All-to-All Combine Ring",
      "shape": "torus",
      "position": [
        6.2,
        4.9,
        0.62
      ],
      "rotation": [
        1.5708,
        0,
        0
      ],
      "size": [
        0.46,
        0.05
      ],
      "material": "copper",
      "role": "Second all-to-all ring returns expert outputs to the original token order for residual addition.",
      "compute_profile": "communication-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "combine_manifold",
        "residual_pipe"
      ]
    },
    {
      "id": "moe_shared_expert",
      "name": "Always-On Shared Expert",
      "shape": "box",
      "position": [
        5.65,
        5.28,
        0.62
      ],
      "size": [
        0.58,
        0.45,
        0.58
      ],
      "material": "brushed steel",
      "role": "Shared SwiGLU expert runs for every token and absorbs common computation or tokens dropped by capacity limits.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "tensor-parallel",
      "connections": [
        "moe_router",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_01",
      "name": "Routed Expert 01",
      "shape": "box",
      "position": [
        4.55,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 1 with intermediate width 9216, selected only when top-2 routing assigns tokens.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_02",
      "name": "Routed Expert 02",
      "shape": "box",
      "position": [
        4.9,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 2 provides sparse feed-forward capacity for assigned tokens.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_03",
      "name": "Routed Expert 03",
      "shape": "box",
      "position": [
        5.25,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 3 is one of the countable thirty-two expert modules.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_04",
      "name": "Routed Expert 04",
      "shape": "box",
      "position": [
        5.6,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 4 executes its feed-forward path only for routed token batches.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_05",
      "name": "Routed Expert 05",
      "shape": "box",
      "position": [
        5.95,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 5 adds sparse model capacity without activating all parameters.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_06",
      "name": "Routed Expert 06",
      "shape": "box",
      "position": [
        6.3,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 6 receives token buckets from the dispatch manifold.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_07",
      "name": "Routed Expert 07",
      "shape": "box",
      "position": [
        6.65,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 7 returns weighted output through the combine manifold.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_08",
      "name": "Routed Expert 08",
      "shape": "box",
      "position": [
        7,
        3.92,
        0.08
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 8 completes the first visible row of expert modules.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_09",
      "name": "Routed Expert 09",
      "shape": "box",
      "position": [
        4.55,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 9 starts the second row in the expert bank.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_10",
      "name": "Routed Expert 10",
      "shape": "box",
      "position": [
        4.9,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 10 provides one sparse FFN path among thirty-two choices.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_11",
      "name": "Routed Expert 11",
      "shape": "box",
      "position": [
        5.25,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 11 is addressed by the top-2 router when its score is high.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_12",
      "name": "Routed Expert 12",
      "shape": "box",
      "position": [
        5.6,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 12 contributes to the activated 45.9B-parameter path only when selected.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_13",
      "name": "Routed Expert 13",
      "shape": "box",
      "position": [
        5.95,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 13 stores its own feed-forward weights.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_14",
      "name": "Routed Expert 14",
      "shape": "box",
      "position": [
        6.3,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 14 is evaluated only for its routed token bucket.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_15",
      "name": "Routed Expert 15",
      "shape": "box",
      "position": [
        6.65,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 15 returns its weighted result to the combine manifold.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_16",
      "name": "Routed Expert 16",
      "shape": "box",
      "position": [
        7,
        4.28,
        0.42
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 16 completes the first half of the routed expert set.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_17",
      "name": "Routed Expert 17",
      "shape": "box",
      "position": [
        4.55,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 17 begins the rear bank visible in top view.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_18",
      "name": "Routed Expert 18",
      "shape": "box",
      "position": [
        4.9,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 18 receives token buckets through expert-parallel dispatch.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_19",
      "name": "Routed Expert 19",
      "shape": "box",
      "position": [
        5.25,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 19 is one of the individually buildable MoE blocks.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_20",
      "name": "Routed Expert 20",
      "shape": "box",
      "position": [
        5.6,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 20 contributes sparse capacity for routed tokens.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_21",
      "name": "Routed Expert 21",
      "shape": "box",
      "position": [
        5.95,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 21 runs only when chosen by top-2 routing.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_22",
      "name": "Routed Expert 22",
      "shape": "box",
      "position": [
        6.3,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 22 returns weighted activations to the combine manifold.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_23",
      "name": "Routed Expert 23",
      "shape": "box",
      "position": [
        6.65,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 23 is a separate buildable module rather than part of a group block.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_24",
      "name": "Routed Expert 24",
      "shape": "box",
      "position": [
        7,
        3.92,
        0.82
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 24 completes the third visible expert row.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_25",
      "name": "Routed Expert 25",
      "shape": "box",
      "position": [
        4.55,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 25 begins the final visible row of routed experts.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_26",
      "name": "Routed Expert 26",
      "shape": "box",
      "position": [
        4.9,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 26 executes the d_ff 9216 feed-forward branch for assigned tokens.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_27",
      "name": "Routed Expert 27",
      "shape": "box",
      "position": [
        5.25,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 27 is selected sparsely by the router.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_28",
      "name": "Routed Expert 28",
      "shape": "box",
      "position": [
        5.6,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 28 helps represent the large total parameter count without dense activation.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_29",
      "name": "Routed Expert 29",
      "shape": "box",
      "position": [
        5.95,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 29 is connected to both dispatch and combine paths.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_30",
      "name": "Routed Expert 30",
      "shape": "box",
      "position": [
        6.3,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 30 contributes to the full thirty-two-expert bank.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_31",
      "name": "Routed Expert 31",
      "shape": "box",
      "position": [
        6.65,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 31 is one selectable sparse expert branch.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "expert_32",
      "name": "Routed Expert 32",
      "shape": "box",
      "position": [
        7,
        4.28,
        1.16
      ],
      "size": [
        0.28,
        0.3,
        0.26
      ],
      "material": "white composite",
      "role": "Individual routed SwiGLU expert 32 completes the countable routed expert bank.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "expert-parallel",
      "connections": [
        "dispatch_manifold",
        "combine_manifold"
      ]
    },
    {
      "id": "prediction_head",
      "name": "Tied LM Head",
      "shape": "box",
      "position": [
        0,
        8.12,
        0
      ],
      "size": [
        3.55,
        0.32,
        1.65
      ],
      "material": "white composite",
      "role": "Final projection from hidden dimension 6144 back to the approximately 200K vocabulary using weights tied to the embedding table.",
      "compute_profile": "gpu-compute-bound",
      "flops_per_token": "About 2*d*V, approximately 2.5 GFLOP per token.",
      "parallelism": "tensor-parallel",
      "connections": [
        "residual_pipe",
        "embedding_table",
        "lm_loss",
        "cispo_trainer",
        "output_token_spout"
      ]
    },
    {
      "id": "lm_loss",
      "name": "Cross-Entropy Loss Node",
      "shape": "sphere",
      "position": [
        -1.75,
        8.48,
        -0.58
      ],
      "size": [
        0.22
      ],
      "material": "glass display",
      "role": "Pre-training next-token loss node connected to the tied LM head.",
      "compute_profile": "gpu-bandwidth-bound",
      "parallelism": "data-parallel",
      "connections": [
        "prediction_head",
        "optimizer_adamw"
      ]
    },
    {
      "id": "rollout_buffer",
      "name": "80K-Token Rollout Buffer",
      "shape": "box",
      "position": [
        1.85,
        8.52,
        -0.58
      ],
      "size": [
        1.1,
        0.3,
        0.7
      ],
      "material": "glass display",
      "role": "Stores very long reasoning trajectories up to the 80K output-token budget before CISPO updates.",
      "compute_profile": "storage-bound",
      "parallelism": "data-parallel",
      "connections": [
        "prediction_head",
        "cispo_trainer",
        "reward_models"
      ]
    },
    {
      "id": "cispo_trainer",
      "name": "CISPO RL Trainer",
      "shape": "sphere",
      "position": [
        2.95,
        8.78,
        -0.58
      ],
      "size": [
        0.36
      ],
      "material": "copper",
      "role": "Clipped sequence-level importance-sampling trainer stabilizes RL on long reasoning rollouts.",
      "compute_profile": "communication-bound",
      "parallelism": "data-parallel",
      "algorithm": "rho = clip(pi_new(tau)/pi_old(tau)); loss = -mean(clip_token(rho) * advantage)",
      "connections": [
        "rollout_buffer",
        "reward_models",
        "optimizer_adamw",
        "h800_training_rail"
      ]
    },
    {
      "id": "reward_models",
      "name": "Reward Model Slab",
      "shape": "box",
      "position": [
        4.05,
        8.78,
        -0.58
      ],
      "size": [
        1.1,
        0.36,
        0.7
      ],
      "material": "black composite",
      "role": "Aggregates math, code, agentic tool-use, and software-engineering reward signals for CISPO.",
      "compute_profile": "gpu-compute-bound",
      "parallelism": "data-parallel",
      "connections": [
        "rollout_buffer",
        "cispo_trainer"
      ]
    },
    {
      "id": "optimizer_adamw",
      "name": "AdamW Optimizer Block",
      "shape": "box",
      "position": [
        -4.2,
        5.15,
        0.82
      ],
      "size": [
        1,
        0.78,
        0.72
      ],
      "material": "dark anodized aluminum",
      "role": "AdamW optimizer stores first and second moments for pre-training and RL updates with beta1 0.9 and beta2 0.95.",
      "compute_profile": "gpu-bandwidth-bound",
      "memory_footprint": "Optimizer state is roughly twice parameter size before sharding.",
      "parallelism": "data-parallel",
      "connections": [
        "lm_loss",
        "cispo_trainer",
        "parallelism_panel"
      ]
    },
    {
      "id": "training_data_pipe",
      "name": "Pretraining and RL Data Pipe",
      "shape": "box",
      "position": [
        6.65,
        0.28,
        0.65
      ],
      "size": [
        1.5,
        0.28,
        0.72
      ],
      "material": "black composite",
      "role": "Grounded data inlet feeding MiniMax-Text-01 pretraining data and the M1 long-CoT RL rollout stream.",
      "compute_profile": "storage-bound",
      "parallelism": "data-parallel",
      "connections": [
        "input_plate",
        "rollout_buffer"
      ]
    },
    {
      "id": "sequence_parallel_chunker",
      "name": "Sequence-Parallel Chunker",
      "shape": "box",
      "position": [
        -4.25,
        3.9,
        -0.72
      ],
      "size": [
        0.9,
        0.24,
        0.58
      ],
      "material": "aluminum",
      "role": "Chunks million-token prefill so lightning recurrence can pass compact S and z state across sequence shards.",
      "compute_profile": "communication-bound",
      "parallelism": "sequence-parallel",
      "connections": [
        "lightning_state_s",
        "lightning_state_z",
        "parallelism_panel"
      ]
    },
    {
      "id": "parallelism_panel",
      "name": "TP EP PP DP Parallelism Panel",
      "shape": "box",
      "position": [
        7.15,
        5.4,
        0.65
      ],
      "size": [
        0.9,
        1.85,
        0.58
      ],
      "material": "glass display",
      "role": "Service panel summarizing tensor, expert, pipeline, and data parallelism used to train the hybrid MoE model.",
      "compute_profile": "communication-bound",
      "parallelism": "pipeline-parallel",
      "connections": [
        "softmax_kv_cache",
        "sequence_parallel_chunker",
        "optimizer_adamw",
        "h800_training_rail"
      ]
    },
    {
      "id": "h800_training_rail",
      "name": "512 H800 41-Day Training Rail",
      "shape": "box",
      "position": [
        3.05,
        8.28,
        0.82
      ],
      "size": [
        2.45,
        0.14,
        0.3
      ],
      "material": "brushed steel",
      "role": "Upper rail marks the reported 41-day RL training run on 512 H800 GPUs.",
      "compute_profile": "gpu-cluster",
      "parallelism": "data-parallel",
      "connections": [
        "cispo_trainer",
        "parallelism_panel",
        "upper_service_gantry"
      ]
    },
    {
      "id": "output_token_spout",
      "name": "Output Token Spout",
      "shape": "cylinder",
      "position": [
        0,
        8.62,
        1.1
      ],
      "rotation": [
        1.5708,
        0,
        0
      ],
      "size": [
        0.16,
        0.95
      ],
      "material": "brass",
      "role": "Short top spout represents decoded output tokens leaving the tied LM head after reasoning.",
      "compute_profile": "gpu-bandwidth-bound",
      "parallelism": "data-parallel",
      "connections": [
        "prediction_head"
      ]
    }
  ]
}