1360 lines (1310 loc) · 156 KB

Paper List for Machine Learning Systems (arXiv preprints)

A curated list of ML systems papers posted on arXiv.

Paper List for Machine Learning Systems (arXiv preprints)

Data Processing

Data pipeline optimization

[arxiv'26] Lakestream: A Consistent and Brokerless Data Plane for Large Foundation Model Training
[arxiv'25] Scalable and Performant Data Loading
[arxiv'25] OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training
[arxiv'25] The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution
[arxiv'25] In-Network Preprocessing of Recommender Systems on Multi-Tenant SmartNICs
[arxiv'24] TensorSocket: Shared Data Loading for Deep Learning Training
[arxiv'24] Efficient Tabular Data Preprocessing of ML Pipelines

Preprocessing stalls

Specific workloads (GNN, DLRM)

[arxiv'23] Towards Data-centric Graph Machine Learning: Review and Outlook
[arxiv'23] FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models
[arxiv'23] MTrainS: Improving DLRM training efficiency using heterogeneous memories

LLM data plane

Training System

ML job analysis on GPU clusters

Resource scheduling

[arxiv'26] CvxCluster: Solving Large, Complex, Granular Resource Allocation Problems 100-1000x Faster
[arxiv'26] PromptTuner: SLO-Aware Elastic System for LLM Prompt Tuning
[arxiv'26] BOA Constrictor: Squeezing Performance out of GPUs in the Cloud via Budget-Optimal Allocation
[arxiv'26] SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost
[arxiv'25] Semantic-Aware Scheduling for GPU Clusters with Large Language Models
[arxiv'25] Holistic Heterogeneous Scheduling for Autonomous Applications using Fine-grained, Multi-XPU Abstraction
[arxiv'25] Tesserae: Scalable Placement Policies for Deep Learning Workloads
[arxiv'25] LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systems
[arxiv'25] TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms
[arxiv'24] Zeal: Rethinking Large-Scale Resource Allocation with "Decouple and Decompose"
[arxiv'24] Rubick: Exploiting Job Reconfigurability for Deep Learning Cluster Scheduling
[arxiv'23] Energy-Efficient GPU Clusters Scheduling for Deep Learning
[arxiv'22] Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads

Distributed training

[arxiv'26] Tangram: Hiding GPU Heterogeneity for Efficient LLM Parallelization
[arxiv'26] Breaking the Bubble: Asynchronous Pipeline Parallel Training with Bounded Weight Inconsistency
[arxiv'26] Piper: A Programmable Distributed Training System
[arxiv'26] DynaTrain: Fast Online Parallelism Switching for Elastic LLM Training
[arxiv'26] JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials
[arxiv'26] A Readiness-Driven Runtime for Pipeline-Parallel Training under Runtime Variability
[arxiv'26] Accelerating Compound LLM Training Workloads with Maestro
[arxiv'26] AutoScout: Structured Optimization for Automating ML System Configuration
[arxiv'26] NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning
[arxiv'26] veScale-FSDP: Flexible and High-Performance FSDP at Scale
[arxiv'26] DHP: Efficient Scaling of MLLM Training with Dynamic Hybrid Parallelism
[arxiv'26] Joint Training on AMD and NVIDIA GPUs
[Survey 🔍] [arxiv'26] Distributed Hybrid Parallelism for Large Language Models: Comparative Study and System Design Guide
[arxiv'26] tLoRA: Efficient Multi-LoRA Training with Elastic Shared Super-Models
[arxiv'26] TimelyFreeze: Adaptive Parameter Freezing Mechanism for Pipeline Parallelism
[arxiv'26] Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation
[arxiv'25] Diving into 3D Parallelism with Heterogeneous Spot Instance GPUs: Design and Implications
[arxiv'25] SIGMA: An AI-Empowered Training Stack on Early-Life Hardware
[arxiv'25] BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models
[arxiv'25] AsyncHZP: Hierarchical ZeRO Parallelism with Asynchronous Scheduling for Scalable LLM Training
[arxiv'25] PRISM: Probabilistic Runtime Insights and Scalable Performance Modeling for Large-Scale Distributed Training
[arxiv'25] A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training
[arxiv'25] SlimPack: Fine-Grained Asymmetric Packing for Balanced and Efficient Variable-Length LLM Training
[arxiv'25] AdaPtis: Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models
[arxiv'25] HAPT: Heterogeneity-Aware Automated Parallel Training on Heterogeneous Clusters
[arxiv'25] Scaling Up Data Parallelism in Decentralized Deep Learning
[arxiv'25] Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
[arxiv'25] TrainVerify: Equivalence-Based Verification for Distributed LLM Training
[arxiv'25] Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage
[arxiv'25] ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates
[arxiv'25] Rethinking Dynamic Networks and Heterogeneous Computing with Automatic Parallelization
[arxiv'25] H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
[arxiv'25] Balanced and Elastic End-to-end Training of Dynamic LLMs
[arxiv'25] ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates
[arxiv'25] SpanTrain: Highly Efficient Cross-domain Model Distributed Training System under Heterogeneous GPUs and Networks in CEE Environment
[arxiv'25] Parallel Scaling Law for Language Models
[arxiv'25] Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations
[arxiv'25] Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
[arxiv'25] PipeWeaver: Addressing Data Dynamicity in Large Multimodal Model Training with Dynamic Interleaved Pipeline
[arxiv'25] You Don't Need All Attentions: Distributed Dynamic Fine-Tuning for Foundation Models
[arxiv'25] WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training
[arxiv'25] Nonuniform-Tensor-Parallelism: Mitigating GPU failure impact for Scaled-up LLM Training
[arxiv'25] CFP: Low-overhead Profiling-based Intra-operator Parallelism Generation by Preserving Communication-Free Structures
[arxiv'25] OrchMLLM: Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training
[arxiv'25] Cornstarch: Distributed Multimodal Training Must Be Multimodality-Aware
[arxiv'25] PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
[arxiv'25] AutoHete: An Automatic and Efficient Heterogeneous Training System for LLMs
[arxiv'25] Astra: Efficient and Money-saving Automatic Parallel Strategies Search on Heterogeneous GPUs
[arxiv'25] Scaling Inference-Efficient Language Models
[arxiv'25] MiniMax-01: Scaling Foundation Models with Lightning Attention
[arxiv'24] Automatically Planning Optimal Parallel Strategy for Large Language Models
[arxiv'24] Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
[arxiv'24] Frenzy: A Memory-Aware Serverless LLM Training System for Heterogeneous GPU Clusters
[arxiv'24] Echo: Simulating Distributed Training At Scale
[arxiv'24] Scaling Deep Learning Training with MPMD Pipeline Parallelism
[arxiv'24] Demystifying Workload Imbalances in Large Transformer Model Training over Variable-length Sequences
[arxiv'24] HETHUB: A Distributed Training System with Heterogeneous Cluster for Large-Scale Models
[arxiv'24] Data-Centric and Heterogeneity-Adaptive Sequence Parallelism for Efficient LLM Training
[arxiv'24] Accelerating Large Language Model Training with 4D Parallelism and Memory Consumption Estimator
[arxiv'24] BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
[arxiv'24] Cephalo: Harnessing Heterogeneous GPU Clusters for Training Transformer Models
[arxiv'24] SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile
[arxiv'24] FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression
[arxiv'24] PipeFill: Using GPUs During Bubbles in Pipeline-parallel LLM Training
[arxiv'24] Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
[arxiv'24] DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
[arxiv'24] Efficient Multi-Task Large Model Training via Data Heterogeneity-aware Model Management
[arxiv'24] FlashFlex: Accommodating Large Language Model Training over Heterogeneous Environment
[arxiv'24] PARALLELGPUOS: A Concurrent OS-level GPU Checkpoint and Restore System using Validated Speculation
[arxiv'24] Unicron: Economizing Self-Healing LLM Training at Scale
[arxiv'24] TBA: Faster Large Language Model Training Using SSD-Based Activation Offloading
[arxiv'24] Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
[Survey 🔍] [arxiv'24] Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
[arxiv'24] LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
[arxiv'24] PAFT: A Parallel Training Paradigm for Effective LLM Fine-Tuning
[arxiv'24] BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
[arxiv'24] Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
[arxiv'24] Accelerating Heterogeneous Tensor Parallelism via Flexible Workload Control
[arxiv'24] GRAWA: Gradient-based Weighted Averaging for Distributed Training of Deep Learning Models
[arxiv'24] BitDelta: Your Fine-Tune May Only Be Worth One Bit
[arxiv'24] NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models
[arxiv'24] Accelerating Parallel Sampling of Diffusion Models
[arxiv'24] Training DNN Models over Heterogeneous Clusters with Optimal Performance
[arxiv'24] Breaking MLPerf Training: A Case Study on Optimizing BERT
[arxiv'24] LocMoE: A Low-overhead MoE for Large Language Model Training
[arxiv'24] Re-evaluating the Memory-balanced Pipeline Parallelism: BPipe
[arxiv'24] InternEvo: Efficient Long-sequence Large Language Model Training via Hybrid Parallelism and Redundant Sharding
[arxiv'23] SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
[arxiv'23] vTrain: A Simulation Framework for Evaluating Cost-effective and Compute-optimal Large Language Model Training
[arxiv'23] ASPEN: High-Throughput LoRA Fine-Tuning of Large Language Models with a Single GPU
[arxiv'23] FlexModel: A Framework for Interpretability of Distributed Large Language Models
[arxiv'23] Holmes: Towards Distributed Training Across Clusters with Heterogeneous NIC Environment
[arxiv'23] RTP: Rethinking Tensor Parallelism with Memory Deduplication
[arxiv'23] FP8-LM: Training FP8 Large Language Models
[arxiv'23] Redco: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs
[arxiv'23] DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
[arxiv'23] A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
[arxiv'23] FLM-101B: An Open LLM and How to Train It with $100K Budget
[arxiv'23] UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming
[arxiv'23] Modeling Parallel Programs using Large Language Models
[arxiv'23] Proteus: Simulating the Performance of Distributed DNN Training
[arxiv'23] Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training
[arxiv'23] Decoupled Model Schedule for Deep Learning Training
[arxiv'23] RAF: Holistic Compilation for Deep Learning Model Training
[arxiv'23] Ada-Grouper: Accelerating Pipeline Parallelism in Preempted Network by Adaptive Group-Scheduling for Micro-Batches
[arxiv'23] Does compressing activations help model parallel training?
[arxiv'23] Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models
[arxiv'23] Scaling Vision Transformers to 22 Billion Parameters
[arxiv'23] Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform
[arxiv'23] TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation
[arxiv'23] SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
[arxiv'23] ATP: Adaptive Tensor Parallelism for Foundation Models
[arxiv'22] Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training
[arxiv'22] Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
[arxiv'21] Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training
[arxiv'21] GSPMD: General and Scalable Parallelization for ML Computation Graphs
[arxiv'20] Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
[arxiv'20] Scaling Laws for Neural Language Models

RL post-training

[arxiv'26] AReaL-DTE: Sparse Policy-Weight Transfer for Online Agentic Reinforcement Learning
[arxiv'26] JoyNexus: Service-Oriented Multi-Tenant Post-Training for VLA Models
[arxiv'26] LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
[arxiv'26] Next-Generation Agentic Reinforcement Learning Systems Enable Self-Evolving Agents
[arxiv'26] Accelerating Disaggregated RL for Visual Generative LLMs with Diffusion-Based Parallelism and Trainer-Assisted Generation
[arxiv'26] Spotlight: Synergizing Seed Exploration and Spot GPUs for DiT RL Post-Training
[arxiv'26] Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models
[arxiv'26] Harnessing Routing Foresight for Micro-step-level MoE load balancing in RL Post-training
[arxiv'26] Libra: Efficient Resource Management for Agentic RL Post-Training
[arxiv'26] Schedule-Level Shared-Prefix Reuse for LLM RL Training
[arxiv'26] Polar: Agentic RL on Any Harness at Scale
[arxiv'26] AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs
[arxiv'26] ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning
[arxiv'26] ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RL
[arxiv'26] Heddle: A Distributed Orchestration System for Agentic RL Rollout
[arxiv'26] Role-Based Fault Tolerance System for LLM RL Post-Training
[arxiv'26] ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning
[arxiv'26] RLHFless: Serverless Computing for Efficient RLHF
[arxiv'26] TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents
[arxiv'26] Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
[arxiv'26] WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL
[arxiv'26] RL over Commodity Networks: Overcoming the Bandwidth Barrier with Lossless Sparse Deltas
[arxiv'26] Dr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generations
[arxiv'26] Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow
[arxiv'26] Unleashing Efficient Asynchronous RL Post-Training via Staleness-Constrained Rollout Coordination
[arxiv'26] OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RL
[arxiv'25] HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments
[arxiv'25] ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
[arxiv'25] RLHFSpec: Breaking the Efficiency Bottleneck in RLHF Training via Adaptive Drafting
[arxiv'25] Fast LLM Post-training via Decoupled and Best-of-N Speculation
[arxiv'25] Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter
[arxiv'25] Beat the long tail: Distribution-Aware Speculative Decoding for RL Training
[arxiv'25] WeChat-YATT: A Scalable, Simple, Efficient, and Production Ready Training Library
[arxiv'25] The Path Not Taken: RLVR Provably Learns Off the Principals
[arxiv'25] AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
[arxiv'25] Ask a Strong LLM Judge when Your Reward Model is Uncertain
[arxiv'25] RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMs
[arxiv'25] Laminar: A Scalable Asynchronous RL Post-Training Framework
[arxiv'25] The Art of Scaling Reinforcement Learning Compute for LLMs
[arxiv'25] xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning
[arxiv'25] Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense
[arxiv'25] Learning from Failures: Understanding LLM Alignment through Failure-Aware Inverse RL
[arxiv'25] Spurious Rewards: Rethinking Training Signals in RLVR
[arxiv'25] Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
[arxiv'25] RL in the Wild: Characterizing RLVR Training in LLM Deployment
[arxiv'25] APRIL: Active Partial Rollouts in Reinforcement Learning to tame long-tail generation
[arxiv'25] ToRL: Scaling Tool-Integrated RL
[arxiv'25] VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use
[arxiv'25] Parallel-R1: Towards Parallel Thinking via Reinforcement Learning
[Survey 🔍] [arxiv'25] A Survey of Reinforcement Learning for Large Reasoning Models
[arxiv'25] RewardDance: Reward Scaling in Visual Generation
[arxiv'25] floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL
[arxiv'25] ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Compute
[arxiv'25] History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRL
[arxiv'25] SeamlessFlow: A Trainer Agent Isolation RL Framework Achieving Bubble-Free Pipelines via Tag Scheduling
[arxiv'25] SPECS: Faster Test-Time Scaling through Speculative Drafts
[arxiv'25] Balanced Actor Initialization: Stable RLHF Training of Distillation-Based Reasoning Models
[arxiv'25] ReTool: Reinforcement Learning for Strategic Tool Use in LLMs
[arxiv'25] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
[arxiv'25] Multi-module GRPO: Composing Policy Gradients and Prompt Optimization for Language Model Programs
[arxiv'25] Scaling RL to Long Videos
[arxiv'25] Test-Time Training Done Right
[arxiv'25] LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training
[arxiv'25] On-Policy RL with Optimal Reward Baseline
[arxiv'25] StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation
[arxiv'25] DAPO: An Open-Source LLM Reinforcement Learning System at Scale
[arxiv'25] Reward Reasoning Model
[arxiv'24] Optimizing RLHF Training for Large Language Models with Stage Fusion

AutoML

GNN training

For comprehensive list of GNN systems papers, refer to https://github.com/chwan1016/awesome-gnn-systems.

Inference System

Attention Optimization

[arxiv'26] AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
[arxiv'26] FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
[arxiv'26] SageBwd: A Trainable Low-bit Attention
[arxiv'26] Test-Time Training with KV Binding Is Secretly Linear Attention
[arxiv'26] MonarchRT: Efficient Attention for Real-Time Video Generation
[arxiv'26] PackInfer: Compute- and I/O-Efficient Attention for Batched LLM Inference
[arxiv'25] BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
[arxiv'25] SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention [Code]

Mixture of Experts (MoE)

Communication Optimization & Network Infrastructure for Distributed ML

[arxiv'26] NUNA: Characterizing and Mitigating Non-Uniform Network Access in Multi-Die GPU Scale-Up Systems
[arxiv'26] StrataCL: Fabric-Native Communication Library for Production Supernodes
[arxiv'26] X-Stage: An Overlooked Pipeline Stage for Communication-Computation Overlap in DiT Inference
[arxiv'26] Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives
[arxiv'26] Scalable Optimal Transport Algorithm for Network Alignment
[arxiv'26] SiFAR: Synchronization-Free All-Reduce for Low-Latency LLM Inference
[arxiv'26] CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems
[arxiv'26] Toward a Unified GPU-Aware OpenSHMEM Specification
[arxiv'26] The Multipath Reliable Connection (MRC) Transport
[arxiv'26] Eidola: Modeling Multi-GPU Network Communication Traffic in Distributed AI Workloads
[arxiv'26] PCCL: Process Group-Aware Scalable and Generic Collective Algorithm Synthesizer
[arxiv'26] Demystifying NVSHMEM: A System-Level Analysis on Symmetric Memory and Device-Initiated Operations in GPU Communication
[arxiv'26] HetCCL: Enabling Collective Communication For Mixed-Vendor Heterogeneous Clusters
[arxiv'26] ScaleAcross Explorer: Exploring Communication Optimization for Scale-Across AI Model Training
[arxiv'26] Extreme-Scale Interconnection Networks
[arxiv'26] Revisiting Bruck: Phase-Efficient All-to-All Communication in Reconfigurable Networks
[arxiv'26] Exploiting Multicast for Accelerating Collective Communication
[arxiv'26] Rethinking Network Topologies for Cost-Effective Mixture-of-Experts LLM Serving
[arxiv'26] Harvest: Adaptive Photonic Switching Schedules for Collective Communication in Scale-up Domains
[arxiv'26] UCCL-Zip: Lossless Compression Supercharged GPU Communication
[arxiv'26] From Skew to Symmetry: Node-Interconnect Multi-Path Balancing with Execution-time Planning for Modern GPU Clusters
[arxiv'26] To Reconfigure or Not to Reconfigure: Optimizing All-to-All Collectives in Circuit-Switched Photonic Interconnects
[arxiv'26] NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL
[arxiv'26] Network Design for Wafer-Scale Systems with Wafer-on-Wafer Hybrid Bonding
[arxiv'26] Spritz: Path-Aware Load Balancing in Low-Diameter Networks
[arxiv'26] ACOS: Arrays of Cheap Optical Switches
[arxiv'26] Bring Your Own Objective: Inter-operability of Network Objectives in Datacenters
[arxiv'26] RL over Commodity Networks: Overcoming the Bandwidth Barrier with Lossless Sparse Deltas
[arxiv'26] DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reduce
[arxiv'26] MonkeyTree: Near-Minimal Congestion for Multi-tenant Training via Migration
[arxiv'26] HetCCL: Accelerating LLM Training with Heterogeneous GPUs
[arxiv'26] AutoOverlap: Enabling Fine-Grained Overlap of Computation and Communication with Chunk-Based Scheduling
[arxiv'26] Heterogeneous Low-Bandwidth Pre-Training of LLMs
[arxiv'25] Analyzing Communication Predictability in LLM Training
[arxiv'25] UCCL-EP: Portable Expert-Parallel Communication
[arxiv'25] Design Space Exploration of DMA based Finer-Grain Compute Communication Overlap
[arxiv'25] Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design
[arxiv'25] FarSkip-Collective: Unhobbling Blocking Communication in Mixture of Experts Models
[arxiv'25] GPU-Initiated Networking for NCCL
[arxiv'25] DMA Collectives for Efficient ML Communication Offloads
[arxiv'25] Collective Communication for 100k+ GPUs
[arxiv'25] Uno: A One-Stop Solution for Inter- and Intra-Datacenter Congestion Control and Reliable Connectivity
[arxiv'25] MSCCL++: Rethinking GPU Communication Abstractions for Cutting-edge AI Applications
[arxiv'25] Toward Co-adapting Machine Learning Job Shape and Cluster Topology
[arxiv'25] Efficient AllReduce with Stragglers
[arxiv'25] TASP: Topology-aware Sequence Parallelism
[arxiv'25] Bine Trees: Enhancing Collective Operations by Optimizing Communication Locality
[arxiv'25] RoCE BALBOA: Service-enhanced Data Center RDMA for SmartNICs
[arxiv'25] RailX: A Flexible, Scalable, and Low-Cost Network Architecture for Hyper-Scale LLM Training Systems
[arxiv'25] Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms
[arxiv'25] NoLoCo: No-all-reduce Low Communication Training Method for Large Models
[arxiv'25] TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inference
[arxiv'25] FLASH: Fast All-to-All Communication in GPU Clusters
[arxiv'25] MCMComm: Hardware-Software Co-Optimization for End-to-End Communication in Multi-Chip-Modules
[arxiv'25] GenTorrent: Scaling Large Language Model Serving with An Overley Network
[arxiv'25] Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
[arxiv'25] FlashOverlap: A Lightweight Design for Efficiently Overlapping Communication and Computation
[arxiv'25] An Extensible Software Transport Layer for GPU Networking (UCCL) [Code]
[arxiv'25] HeteroPod: XPU-Accelerated Infrastructure Offloading for Commodity Cloud-Native Applications
[Survey 🔍] [arxiv'25] GPU-centric Communication Schemes for HPC and ML Applications
[arxiv'25] UB-Mesh: a Hierarchically Localized nD-FullMesh Datacenter Network Architecture
[arxiv'25] Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
[arxiv'25] InfinitePOD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers
[arxiv'25] In-Network Preprocessing of Recommender Systems on Multi-Tenant SmartNICs
[arxiv'25] Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning
[arxiv'25] The Power of Negative Zero: Datatype Customization for Quantized Large Language Models
[arxiv'25] mFabric: An Efficient and Scalable Fabric for Mixture-of-Experts Training
[arxiv'24] TokenRing: An Efficient Parallelism Framework for Infinite-Context LLMs via Bidirectional Communication
[arxiv'24] The Landscape of GPU-Centric Communication
[arxiv'24] Revisiting the Time Cost Model of AllReduce
[arxiv'24] LuWu: An End-to-End In-Network Out-of-Core Optimizer for 100B-Scale Model-in-Network Data-Parallel Training on Distributed GPUs
[arxiv'24] LumosCore: Highly Scalable LLM Clusters with Optical Interconnect
[arxiv'24] HiCCL: A Hierarchical Collective Communication Library
[arxiv'24] CSPS: A Communication-Efficient Sequence-Parallelism based Serving System for Transformer based Models with Long Prompts
[arxiv'24] Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping
[arxiv'24] Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects
[arxiv'24] Demystifying the Communication Characteristics for Distributed Transformer Models
[arxiv'24] MLTCP: Congestion Control for DNN Training
[arxiv'24] ForestColl: Efficient Collective Communications on Heterogeneous Network Fabrics
[Survey 🔍] [arxiv'23] Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
[arxiv'23] Optimized Network Architectures for Large Language Model Training with Billions of Parameters
[arxiv'23] FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models
[arxiv'23] Rethinking Memory and Communication Cost for Efficient Large Language Model Training
[arxiv'23] Zen: Near-Optimal Sparse Tensor Synchronization for Distributed DNN Training
[arxiv'23] Optimized Network Architectures for Large Language Model Training with Billions of Parameters
[arxiv'23] TACOS: Topology-Aware Collective Algorithm Synthesizer for Distributed Training

Fault tolerance & Straggler mitigation

[arxiv'26] Concordia: JIT-Compiled Persistent-Kernel Checkpointing for Fault-Tolerant LLM Inference
[arxiv'26] ARGUS: Production-Scale Tracing and Performance Diagnosis for over 10,000-GPU Clusters
[arxiv'26] LUMEN: Coordinated Failure Recovery for Distributed LLM Serving
[arxiv'26] Don't Let a Few Network Failures Slow the Entire AllReduce
[arxiv'26] Characterization-Guided GPU Fault Resilience in NVIDIA MPS
[arxiv'26] Orbax: Distributed Checkpointing with JAX
[arxiv'26] LiveR: Fine-Grained Elasticity via Live Reconfiguration for Model Training
[arxiv'26] TierCheck: Tiered Checkpointing for Fault Tolerance in Large Language Model Training
[arxiv'26] From Detection to Recovery: Operational Analysis on LLM Pre-training with 504 GPUs
[arxiv'26] ResiHP: Taming LLM Training Failures with Dynamic Hybrid
[arxiv'26] Resilient AI Supercomputer Networking using MRC and SRv6
[arxiv'26] Role-Based Fault Tolerance System for LLM RL Post-Training
[arxiv'26] ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
[arxiv'26] Towards Resiliency in Large Language Model Serving with KevlarFlow
[arxiv'26] Training LLMs with Fault Tolerant HSDP on 100,000 GPUs
[arxiv'26] Making MoE-based LLM Inference Resilient with Tarragon
[arxiv'25] TTrace: Lightweight Error Checking and Diagnosis for Distributed Training
[arxiv'25] Reliable and Resilient Collective Communication Library for LLM Training and Serving
[arxiv'25] SHIFT: An RDMA Failure-Resilient Layer for Distributed Training
[arxiv'25] FFTrainer: Fast Failover in Large-Language Model Training with Almost-Free State Management
[arxiv'25] FailSafe: High-performance Resilient Serving
[arxiv'25] GoCkpt: Gradient-Assisted Multi-Step overlapped Checkpointing for Efficient LLM Training
[arxiv'25] MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
[arxiv'25] ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training
[arxiv'25] Efficient AllReduce with Stragglers
[arxiv'25] Adaptra: Straggler-Resilient Hybrid-Parallel Training with Pipeline Adaptation
[arxiv'25] Nonuniform-Tensor-Parallelism: Mitigating GPU failure impact for Scaled-up LLM Training
[arxiv'25] Enhancing reliability in AI inference services: An empirical study on real production incidents
[arxiv'25] Characterizing GPU Resilience and Impact on AI/HPC Systems
[arxiv'24] FALCON: Pinpointing and Mitigating Stragglers for Large-Scale Hybrid-Parallel Training
[arxiv'24] TrainMover: Efficient ML Training Live Migration with No Memory Overhead
[arxiv'24] Cloud Atlas: Efficient Fault Localization for Cloud Systems using Language Models and Causal Insight
[arxiv'24] ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development
[arxiv'24] Universal Checkpointing: Efficient and Flexible Checkpointing for Large Scale Distributed Training
[arxiv'24] Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models with Adaptive Expert Placement
[arxiv'24] PARALLELGPUOS: A Concurrent OS-level GPU Checkpoint and Restore System using Validated Speculation
[arxiv'23] Unicron: Economizing Self-Healing LLM Training at Scale

GPU Memory Management & Optimization

[arxiv'26] Xema: Efficient Diffusion Serving through Fine-Grained Memory Management and Auto-Configuration
[arxiv'26] FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training
[arxiv'25] CARMA: Collocation-Aware Resource Manager with GPU Memory Estimator
[arxiv'25] Reducing GPU Memory Fragmentation via Spatio-Temporal Planning for Efficient Large-Scale Model Training
[arxiv'25] Memory Analysis on the Training Course of DeepSeek Models
[arxiv'24] Accelerating Large Language Model Training with 4D Parallelism and Memory Consumption Estimator
[arxiv'23] Rethinking Memory and Communication Cost for Efficient Large Language Model Training
[arxiv'23] Quantized Distributed Training of Large Models with Convergence Guarantees (QSDP)
[arxiv'23] Does compressing activations help model parallel training?
[arxiv'16] Training Deep Nets with Sublinear Memory Cost

GPU Sharing

[arxiv'26] Gleam: Adaptive Network-Efficient CUDA API Remoting for Cross-Device GPU Sharing over LANs
[arxiv'26] MuxTune: Efficient Multi-Task LLM Fine-Tuning in Multi-Tenant Datacenters via Spatial-Temporal Backbone Multiplexing
[arxiv'25] MSched: GPU Multitasking via Proactive Memory Scheduling
[arxiv'25] Towards Efficient and Practical GPU Multitasking in the Era of LLM
[arxiv'25] Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
[arxiv'24] PREBA: A Hardware/Software Co-Design for Multi-Instance GPU based AI Inference Servers
[arxiv'24] Tally: Non-Intrusive Performance Isolation for Concurrent Deep Learning Workloads
[arxiv'23] GACER: Granularity-Aware ConcurrEncy Regulation for Multi-Tenant Deep Learning
[arxiv'23] MuxFlow: Efficient and Safe GPU Sharing in Large-Scale Production Deep Learning Clusters

Compiler

GPU Kernel Optimization

LLM Long Context

Model Compression

For comprehensive list of quantization papers, refer to https://github.com/Efficient-ML/Awesome-Model-Quantization.

Federated Learning

Privacy-Preserving ML

[arxiv'26] Scaling up Privacy-Preserving ML: A CKKS Implementation of Llama-2-7B

ML APIs & Application-Side Optimization

[arxiv'26] SimpleTool: Parallel Decoding for Real-Time LLM Function Calling
[arxiv'24] APIServe: Efficient API Support for Large-Language Model Inferencing

ML for Systems

[arxiv'26] CommBench: Can LLMs Write Correct and Efficient GPU Communication Code?
[arxiv'26] SparseDitto: Customizing GPU Kernels for Different Sparsity Patterns with LLM-Based Agentic System
[arxiv'26] KernelBrain: Coarse-to-Fine, Budget-Aware Search for Agentic GPU Kernel Optimization
[arxiv'26] Harness Engineering for LLM-Driven GPU Kernel Generation
[arxiv'26] KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
[arxiv'26] Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
[arxiv'26] SOLAR: AI-Powered Speed-of-Light Performance Analysis
[arxiv'26] SpecGen: Accelerating Agentic Kernel Optimization with Speculative Generation
[arxiv'26] Inductive Deductive Synthesis: Enabling AI to Generate Formally Verified Systems
[arxiv'26] PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization
[arxiv'26] VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
[arxiv'26] KEET: Explaining Performance of GPU Kernels Using LLM Agents
[arxiv'26] Assistants, Not Architects: The Role of LLMs in Networked Systems Design
[arxiv'26] Improving Efficiency of GPU Kernel Optimization Agents using a Domain-Specific Language and Speed-of-Light Guidance
[arxiv'26] Kernel-Smith: A Unified Recipe for Evolutionary Kernel Optimization
[arxiv'26] AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search
[arxiv'26] Improving Coherence and Persistence in Agentic AI for System Optimization
[arxiv'26] CUCo: An Agentic Framework for Compute and Communication Co-design
[arxiv'26] StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning
[arxiv'26] K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
[arxiv'26] AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
[arxiv'25] AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization
[arxiv'25] ASAP: an Agentic Solution to Auto-optimize Performance of Large-Scale LLM Training
[arxiv'25] Barbarians at the Gate: How AI is Upending Systems Research [Code]
[arxiv'25] SuperCoder: Assembly Program Superoptimization with Large Language Models
[arxiv'24] Cloud Atlas: Efficient Fault Localization for Cloud Systems using Language Models and Causal Insight
[arxiv'24] LLMTune: Accelerate Database Knob Tuning with Large Language Models
[arxiv'24] LLM-Enhanced Data Management
[arxiv'24] MPIrigen: MPI Code Generation through Domain-Specific Language Models
[arxiv'24] Can Large Language Models Write Parallel Code?
[arxiv'23] LLM-Assisted Code Cleaning For Training Accurate Code Generators
[arxiv'23] Large Language Models for Compiler Optimization

Energy Efficiency

Retrieval-Augmented Generation (RAG)

[arxiv'26] RAG-Stack: Co-Optimizing RAG Serving Performance and Quality
[arxiv'26] QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
[arxiv'26] PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving
[arxiv'25] Patchwork: A Unified Framework for RAG Serving
[arxiv'25] Accelerating Retrieval-Augmented Language Model Serving with Speculation
[arxiv'25] RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
[arxiv'25] Long-Context Inference with Retrieval-Augmented Speculative Decoding
[arxiv'24] Towards Understanding Systems Trade-offs in Retrieval-Augmented Generation Model Inference
[arxiv'24] RAGServe: Fast Quality-Aware RAG Systems with Configuration Adaptation
[arxiv'24] Dehallucinating Parallel Context Extension for Retrieval-Augmented Generation
[arxiv'24] Accelerating Retrieval-Augmented Language Model Serving with Speculation

Simulation

Systems for Agentic AI

[arxiv'26] When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference
[arxiv'26] AtumAI: A Principled Framework for Agentic Generation of Datacenter Control-Plane Policies
[arxiv'26] Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework
[arxiv'26] TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
[arxiv'26] SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving
[arxiv'26] Scalable LLM Agent Tool Access in the Cloud
[arxiv'26] A Workflow-Aware Serving Layer for Agentic Applications
[arxiv'26] SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference
[arxiv'26] SwarmX: Agentic Scheduling for Low-Latency Agentic Systems
[arxiv'26] CoAgent: Concurrency Control for Multi-Agent Systems
[arxiv'26] Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents
[arxiv'26] Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads
[arxiv'26] Idleness is Relative: Exploiting Tool-Call Idle Windows for Offloading in Agentic Systems with MORI
[arxiv'26] Agentic AI Workload Characteristics
[arxiv'26] VineLM: Trie-Based Fine-Grained Control for Agentic Workflows
[arxiv'26] HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling
[arxiv'26] xGoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources
[arxiv'26] MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems
[arxiv'26] Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines
[arxiv'26] KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
[arxiv'26] Heddle: A Distributed Orchestration System for Agentic RL Rollout
[arxiv'26] Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective
[arxiv'26] Orla: A Library for Serving LLM-Based Multi-Agent Systems
[arxiv'26] SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
[arxiv'26] CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation
[arxiv'26] ArchAgent: Agentic AI-driven Computer Architecture Discovery
[arxiv'26] Pancake: Hierarchical Memory System for Multi-Agent LLM Serving
[arxiv'26] DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
[arxiv'26] ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System
[arxiv'26] REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents
[arxiv'26] LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents
[arxiv'26] VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents
[arxiv'26] ToolCaching: Towards Efficient Caching for LLM Tool-calling
[arxiv'26] Toward Efficient Agents: Memory, Tool learning, and Planning
[arxiv'26] Sutradhara: An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference
[arxiv'26] Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents
[arxiv'26] XGrammar 2: Dynamic and Efficient Structured Generation Engine for Agentic LLMs
[arxiv'26] Nalar: An agent serving framework
[arxiv'26] Software-Defined Agentic Serving
[arxiv'25] Speculative Actions: A Lossless Framework for Faster Agentic Systems
[arxiv'25] ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
[arxiv'25] Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
[arxiv'25] Towards Efficient Agents: A Co-Design of Inference Architecture and System
[arxiv'25] Beyond Training: Enabling Self-Evolution of Agents with MOBIMEM
[arxiv'25] Optimizing Agentic Language Model Inference via Speculative Tool Calls
[arxiv'25] Astraea: A State-Aware Scheduling Engine for LLM-Powered Agents
[arxiv'25] Measuring Agents in Production
[arxiv'25] Matrix: Peer-to-Peer Multi-Agent Synthetic Data Generation Framework
[arxiv'25] Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflows
[arxiv'25] AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization
[arxiv'25] Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
[arxiv'25] Sherlock: Reliable and Efficient Agentic Workflow Execution
[arxiv'25] A CPU-Centric Perspective on Agentic AI
[arxiv'25] Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution
[arxiv'25] MobiAgent: A Systematic Framework for Customizable Mobile Agents
[arxiv'25] rStar2-Agent: Agentic Reasoning Technical Report
[arxiv'25] Efficient and Scalable Agentic AI with Heterogeneous Systems
[arxiv'25] Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC
[arxiv'25] GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
[arxiv'25] The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks
[arxiv'24] AI Metropolis: Scaling Large Language Model-based Multi-Agent Simulation with Out-of-order Execution

Multimodal

https://github.com/friedrichor/Awesome-Multimodal-Papers

Hybrid LLMs

Others

[arxiv'26] TensorCast: The Missing Tensor Management Layer in Large Language Model Infrastructure
[arxiv'26] Application-Driven Architecture Exploration for Cross-Layer Heterogeneous Systems
[arxiv'26] Enabling Multi-Dimensional Distributed Trace Comparison with Contrast
[arxiv'26] LLM-as-a-Verifier: A General-Purpose Verification Framework
[arxiv'26] Eiger: An Efficient Library for GPU-based Data Analytics
[arxiv'26] ROSA: A Robotics Foundation Model Serving System for Robot Factories
[arxiv'26] Instant GPU Efficiency Visibility at Fleet Scale
[arxiv'26] The Price Reversal Phenomenon: When Cheaper Reasoning Models End Up Costing More
[arxiv'26] Challenges and Design Considerations for Finding CUDA Bugs Through GPU-Native Fuzzing
[arxiv'26] AI+HW 2035: Shaping the Next Decade
[arxiv'26] SKYLIGHT: A Scalable Hundred-Channel 3D Photonic In-Memory Tensor Core Architecture for Real-time AI Inference
[arxiv'26] Semantic Search At LinkedIn
[arxiv'26] Long-term Monitoring of Kernel and Hardware Events to Understand Latency Variance
[arxiv'25] Cyclotron: Compilation of Recurrences to Distributed and Systolic Architectures
[arxiv'25] Streaming Tensor Program: A streaming abstraction for dynamic parallelism
[arxiv'25] OckBench: Measuring the Efficiency of LLM Reasoning
[arxiv'25] vAttention: Verified Sparse Attention
[arxiv'25] Training Large Language Models To Reason In Parallel With Global Forking Tokens
[arxiv'25] How to Train Your Advisor: Steering Black-Box LLMs with Advisor Models
[arxiv'25] Slm-mux: Orchestrating small language models for reasoning
[arxiv'25] Hybrid Architectures for Language Models: Systematic Analysis and Design Insights
[arxiv'25] Less is More: Recursive Reasoning with Tiny Networks
[arxiv'25] ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models
[arxiv'25] Rethinking Thinking Tokens: LLMs as Improvement Operators
[arxiv'25] Generalized Parallel Scaling with Interdependent Generations
[arxiv'25] Composer: A Search Framework for Hybrid Neural Architecture Design
[arxiv'25] dParallel: Learnable Parallel Decoding for dLLMs
[arxiv'25] AI Factories: It's time to rethink the Cloud-HPC divide
[arxiv'25] Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
[arxiv'25] SharedRep-RLHF: A Shared Representation Approach to RLHF with Diverse Preferences
[arxiv'25] Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs
[arxiv'25] LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model
[arxiv'25] DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
[arxiv'25] Less Is More: Training-Free Sparse Attention with Global Locality for Efficient Reasoning
[arxiv'25] Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
[arxiv'25] LobRA: Multi-tenant Fine-tuning over Heterogeneous Data
[arxiv'25] Copilot Arena: A Platform for Code LLM Evaluation in the Wild
[arxiv'25] ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism
[arxiv'25] Libra: Synergizing CUDA and Tensor Cores for High-Performance Sparse Matrix Multiplication
[arxiv'25] Prompt-to-Leaderboard: Prompt-Adaptive LLM Evaluations [Code]
[arxiv'25] SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
[arxiv'25] Reinforcement Pre-Training
[arxiv'25] MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models
[arxiv'25] Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately
[arxiv'25] Faster Video Diffusion with Trainable Sparse Attention
[arxiv'25] SSR: Speculative Parallel Scaling Reasoning in Test-time
[arxiv'25] Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought
[arxiv'25] Think Only When You Need with Large Hybrid-Reasoning Models
[arxiv'25] Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
[arxiv'25] Understanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloads
[arxiv'25] Process Reward Models That Think
[arxiv'25] Seed-Thinking-v1.5: Advancing Superb Reasoning Models with Reinforcement Learning
[arxiv'25] Sleep-time Compute: Beyond Inference Scaling at Test-time
[arxiv'25] SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
[arxiv'25] Scaling Laws for Native Multimodal Models Scaling Laws for Native Multimodal Models
[arxiv'25] OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens
[arxiv'25] NotebookOS: A Notebook Operating System for Interactive Training with On-Demand GPUs
[arxiv'25] Alchemist: Towards the Design of Efficient Online Continual Learning System
[arxiv'25] Linear Attention for Efficient Bidirectional Sequence Modeling
[arxiv'25] S*: Test Time Scaling for Code Generation
[arxiv'25] Optimizing Model Selection for Compound AI Systems
[arxiv'25] Copilot Arena: A Platform for Code LLM Evaluation in the Wild
[arxiv'25] Efficient-vDiT: Efficient Video Diffusion Transformers with Attention Tile
[arxiv'25] BARE: Combining Base and Instruction-Tuned Language Models for Better Synthetic Data Generation
[arxiv'25] Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
[arxiv'25] Adaptive Semantic Prompt Caching with VectorQ
[arxiv'25] Measuring GPU utilization one level deeper
[arxiv'24] Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
[arxiv'24] Debunking the CUDA Myth Towards GPU-based AI Systems
[arxiv'24] XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
[arxiv'24] Scorch: A Library for Sparse Deep Learning
[arxiv'24] Drowning in Documents: Consequences of Scaling Reranker Inference
[arxiv'24] Crafting Interpretable Embeddings for Language Neuroscience by Asking LLMs Questions
[arxiv'24] Computational Bottlenecks of Training Small-scale Large Language Models
[Survey 🔍] [arxiv'24] A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness
[arxiv'24] Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignment
[arxiv'24] DroidSpeak: Enhancing Cross-LLM Communication
[arxiv'24] Disaggregating Embedding Recommendation Systems with FlexEMR
[arxiv'24] JudgeBench: A Benchmark for Evaluating LLM-based Judges
[arxiv'24] You Only Need One Step: Fast Super-Resolution with Stable Diffusion via Scale Distillation
[arxiv'24] Computing in the Era of Large Generative Models: From Cloud-Native to AI-Native
[arxiv'23] Efficiently Programming Large Language Models using SGLang
[arxiv'23] Direct Preference Optimization: Your Language Model is Secretly a Reward Model
[arxiv'22] Training language models to follow instructions with human feedback