Yang Sui | 隋阳

I am a Member of Technical Staff on the Microsoft AI (MAI) Superintelligence Team, where I work on AI infrastructure and machine learning systems (MLSys) for large-scale multimodal models, including LLMs and diffusion models, across training, reinforcement learning (RL), and inference.

I was a Postdoctoral Associate in the Department of Computer Science at Rice University, where I worked with Dr. Xia "Ben" Hu and the DATA Lab on systems and algorithms for large language models, diffusion models, and multimodal LLMs.

Before completing my Ph.D. at Rutgers University, where I was advised by Prof. Bo Yuan, I gained research and engineering experience across industry. In spring 2024, I was a Research Intern on the Creative Vision team at Snap Research, where I developed BitsFusion, a 1.99-bit weight-quantization method for text-to-image diffusion models. In 2022, I was a Research Intern at Media Lab, Tencent America, studying the efficiency and robustness of learned image compression and transformer models. In 2019, I was an Algorithm Engineer at JD, working on face verification and recognition. In 2018, as an R&D Intern at Baidu, I helped initiate Paddle-Lite, the mobile and edge inference framework for PaddlePaddle. The project was presented at the NeurIPS Expo, Baidu Create, and Wave Summit+.

Outside work, I enjoy basketball, soccer, DOTA/DOTA2, and World of Warcraft. I am a fan of Tracy McGrady, Stephen Curry, Lionel Messi, and PIS (YaphetS).

Email  /  Google Scholar  /  GitHub   

1. AI Infrastructure & Machine Learning Systems:

   • Large-scale training: Multidimensional parallelism across data, tensor, pipeline, sequence/context, and MoE experts; FSDP/ZeRO and distributed optimizers; mixed-precision training; activation checkpointing and offloading; fused kernels; and communication-computation overlap.

   • Reinforcement learning: PPO/GRPO-style post-training; distributed actor, rollout, reference, critic, and reward-model orchestration; colocated and disaggregated execution; asynchronous rollouts; dynamic batching and sequence packing; weight synchronization; CPU/GPU offloading; and resource-aware scheduling and load balancing.

   • Inference and serving: Prefill-decode (PD) disaggregation; continuous and chunked batching; paged KV caches and prefix caching; KV-cache and expert offloading; CUDA graphs; kernel fusion; speculative and parallel decoding; request scheduling; and distributed model serving.

   • System co-design: Algorithm-system and algorithm-hardware co-design for scalable, high-performance AI.

2. Efficient AI:

   • Model compression: Quantization, pruning, low-rank approximation, knowledge distillation, and structured sparsity.

   • Token and reasoning efficiency: Token compression, efficient reasoning, dynamic computation, sparse computation, and memory-efficient attention.

2026

2025

2024

2023

Best Paper Runner-Up Award

Oral

Oral

2022

2021

2018

Paddle-Lite, the successor to Paddle-Mobile, is an open-source deep learning inference framework for mobile, embedded, and IoT devices. It supports PaddlePaddle models and models converted from other frameworks. The project was presented at the NeurIPS Expo, Baidu Create, and Wave Summit+.

Previous "Efficient Deep Learning Reading Group" Sessions:

• EDLRG, Spring 2023. Feb 2023 to May 2023. (EDLRG Website)

• Workshop Organization:
  • Publicity Chair, International Workshop on Resource-Efficient Learning for Knowledge Discovery (RelKD 2023) @ KDD 2023
  • Publicity Chair, International Workshop on DL-Hardware Co-Design for Generative AI Acceleration (DCgAA 2024) @ DAC 2024

• Program Committee and Reviewing:
  • NeurIPS'22, 23, 24
  • ICLR'24
  • ICML'22, 23, 24
  • CVPR'22, 23, 24
  • ICCV'23
  • ECCV'22, 24
  • KDD'23
  • AAAI'22, 23, 24, 25
  • IROS'23
  • TNNLS

• Teaching Assistant at Rutgers University

  • 14:332:351 - Programming Methodology II, Fall 2020
    Instructor: Prof. Saman Zonouz   

  • 14:332:351 - Programming Methodology II, Fall 2023
    Instructor: Prof. Yao Liu   

• Daniel Gu, undergraduate student at Rice University
  Topic: LLM quantization.
  

• Eric Chien, master's student at Rice University
  Topic: Multimodal LLMs.
  

• Cheng Yang, Ph.D. student at Rutgers University
  Topic: Token pruning for multimodal LLMs.
  [CVPR 2025] TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model.
  

• Wenjin Zhang, Ph.D. at Rutgers University
  Topic: Quantization for model compression.

• Justin Ding, master's student at Rutgers University
  Topic: Model pruning for a Graduate Special Problems project.
  

• Linqi Xiao, master's student at Rutgers University
  Topic: Error-correcting codes for a Graduate Special Problems project.
  

• Srinihar Bondalapati, master's student at Rutgers University
  Topic: Quantization for model compression.
  

• Yue Wang, master's student at Rutgers University
  Topic: Dataset distillation and model pruning for a Graduate Special Problems project.
  

• Veena Vrushi, undergraduate student at Rutgers University
  Topic: Deep learning through the Project SUPER research program.
  

• Vijay Maddila, graduate student at Rutgers University
  Topic: Large language models.
  

• Ayan Patel, high school student at High Technology High School
  Topic: Deep learning.
  

• Zhiyu Chen, master's student at Rutgers University
  Topic: Large language models for a Graduate Special Problems project.
  

• Efficient Diffusion Models and Large Language Models: Quantization, Pruning, and LoRA. (Video)
  July, 2023.
  

• Model Compression: Pruning, Quantization, and Recent Advances.
  Texas A&M University, CSCE 689 Special Topics: Generative AI, Instructor: Dr. Zhengzhong Tu, October, 2024.
  

• Efficient Multimodal LLMs and Efficient Large Reasoning Models.
  Rice University, COMP 652 001: Natural Language Processing, Instructor: Dr. Hanjie Chen, April, 2025.
  

• Efficient MLLMs and LRMs: Token Pruning in Multimodal Large Language Models and Efficient Reasoning in Large Reasoning Models.
  Texas A&M University, CSCE 753 Computer Vision and Robot Perception, Instructor: Dr. Zhengzhong Tu, April, 2025.
  

• Graduate Academic Achievement Award, Rutgers University
• Paul Panayotatos Scholarship, Rutgers University, 2024
• Best Paper Runner-Up Award, DCAA Workshop at AAAI, 2023
• First Place, Classification Track, Fair and Intelligent Embedded Systems Challenge, Tiny and Fair ML Design Competition at ESWEEK 2023
• Sixth Place, 2023 ACM/IEEE TinyML Design Contest at ICCAD
• SGS Conference Travel and Research Award, 2023.

Outside work, I follow basketball, soccer, Formula 1, and snooker. I am a fan of Tracy McGrady, Stephen Curry, Lionel Messi, and PIS (YaphetS).

I have played DOTA/DOTA2, World of Warcraft, and Warcraft III for many years. My favorite classes and races include Balance Druid and Havoc Demon Hunter in World of Warcraft, and Night Elf in Warcraft III. A few memorable gaming milestones:

• World of Warcraft
  • Ranked among the world's top 10 Balance Druid DPS players for Heroic Flamebender Ka'graz in Blackrock Foundry, as reported by Warcraft Logs in 2015.
  • Led a 15-player team that completed the Heroic Highmaul raid in Warlords of Draenor in 2014.
• DOTA
  
  • Member of the five-player DOTA team at Jilin City No. 1 High School, 2011.
• Hearthstone Legend
  
  • Reached Legend rank and a ladder ranking of 147 in 2016.
• Diablo III
  
  • Reached ladder rank 698 as a Witch Doctor in Season 9, 2017.

I enjoy R&B and classical music, especially works by Chopin, Bach, and Paganini.

Kim Tae-yeon was an important role model for me in high school and a source of encouragement during a difficult period of my life.

Xiaolan—whose name was inspired by "Detective Conan"—is my playful and affectionate grey-and-white cat. She even appears in my NeurIPS 2021 paper, "CHIP." Say hi, Xiaolan!

*Last updated on 08/2026*
Inspired by Jon Barron