LLM Efficient RL: Taming Extreme Actor-Policy Mismatch
Making reinforcement learning practical when efficient rollout actors differ substantially from the policy being trained.
Yang Zhou 周旸
I am a PhD student in Electrical and Computer Engineering at Carnegie Mellon University, advised by Prof. Beidi Chen. Previously, I worked with Prof. Diana Marculescu on edge AI and Prof. Kurt Keutzer on model quantization. I received my bachelor's degree in Electrical and Computer Engineering from The University of Texas at Austin.
My research focuses on efficient post-training and inference for large language models, with an emphasis on reasoning, long-context generation, and reinforcement learning. I welcome conversations and collaborations in these areas.
News
Jackpot was accepted to ICLR 2026.
Act Only When It Pays was accepted to NeurIPS 2025.
GSM-Infinite was accepted to ICML 2025.
Sirius was accepted to NeurIPS 2024.
Gave a talk on GSM-Infinite at the University of Maryland. Talk page
Gave an invited talk on GSM-Infinite at NVIDIA, hosted by Dr. Boris Ginsberg.
GSM-Infinite was featured in the ASAP Seminar. Watch the talk
LLM Inference Unveiled was released on arXiv.
Began graduate studies at Carnegie Mellon University.
Graduated from The University of Texas at Austin.
Research
Efficient training, evaluation, and inference for reasoning models.
LLM Long-context Reasoning Evaluation
Building controllable evaluations that isolate how context length and reasoning complexity affect language models.
LLM Reasoning Inference Efficiency
Reducing the computation and memory required for long reasoning traces while preserving model capability.
Selected Publications
* denotes equal contribution.
Jackpot: Optimal Budgeted Rejection Sampling for Extreme Actor-Policy Mismatch RL
ICLR, 2026
Budgeted rejection sampling enables efficient RL training under extreme actor-policy mismatch.
Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
NeurIPS, 2025
GRESO predicts and skips uninformative prompts before rollout, reducing reinforcement learning overhead.
GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?
ICML, 2025
A controllable benchmark for evaluating language models under increasing context length and reasoning complexity.
Sirius: Contextual Sparsity with Correction for Efficient LLM
NeurIPS, 2024
A lightweight correction method recovers reasoning accuracy in contextually sparse language models while retaining efficiency.
LLM Inference Unveiled: Survey and Roofline Model Insights
Preprint, 2024
A survey of algorithm, system, and hardware techniques for LLM inference, accompanied by a roofline modeling tool for analyzing deployment bottlenecks.
HexGen: Generative Inference of Large-Scale Foundation Model over Heterogeneous Decentralized Environment
ICML, 2024
A distributed inference engine for deploying foundation models across heterogeneous GPUs and network connections.
DQRM: Deep Quantized Recommendation Models
Preprint, 2023
A 4-bit quantized recommendation model and sparse-gradient training system that reduce model size and communication overhead without sacrificing accuracy.
Play It Cool: Dynamic Shifting Prevents Thermal Throttling
DyNN at ICML (Oral), 2022
Dynamic switching between efficient and powerful networks prevents thermal throttling during sustained inference on edge devices.
ANT: Adapt Network Across Time for Efficient Video Processing
CVPRW ECV, 2022
ANT exploits semantic redundancy between video frames by adapting a purpose-fit network over time.
Selected Honors
- ECE Department Recognition Award for Exemplary Qualifying Exam Performance, 2025 (Top 10%)
- Carnegie Institute of Technology Dean's Fellowship, 2023-2024
- Distinguished College Scholar, UT Austin, 2021 and 2022 (Top 4%)
Service
Reviewer for ECCV 2023, MLSys 2024, and NeurIPS 2025. Selected as a Top Reviewer at NeurIPS 2025, free trip ✈️ yayyy!