Publications

RLHFuse: Efficient RLHF Training for Large Language Models with Inter- and Intra-Stage Fusion.
Yinmin Zhong, Zili Zhang, Bingyang Wu, Shengyu Liu, Yukun Chen, Changyi Wan, Hanpeng Hu, Lei Xia, Ranchen Ming, Yibo Zhu, Xin Jin

In Preprint. PDF Cite

DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models.
Zili Zhang, Yinmin Zhong, Ranchen Ming, Hanpeng Hu, Jianjian Sun, Zheng Ge, Yibo Zhu, Xin Jin

In Preprint. PDF Cite

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving.
Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang

In OSDI 2024. PDF Cite Code Slides Video

LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism.
Bingyang Wu, Shengyu Liu, Yinmin Zhong, Peng Sun, Xuanzhe Liu, Xin Jin

In SOSP 2024. PDF Cite

Aquifer: Transparent Microsecond-scale Scheduling for vRAN Workloads.
Yunshan Jia, Yinmin Zhong, Meng Wang, Jiaqi Gao, Pengyu Zhang, Xuanzhe Liu, Xin Jin

In TSC 2024. PDF Cite

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs.
Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, Yulu Jia, Sun He, Hongmin Chen, Zhihao Bai, Qi Hou, Shipeng Yan, Ding Zhou, Yiyao Sheng, Zhuo Jiang, Haohan Xu, Haoran Wei, Zhang Zhang, Pengfei Nie, Leqi Zou, Sida Zhao, Liang Xiang, Zherui Liu, Zhe Li, Xiaoying Jia, Jianxi Ye, Xin Jin, Xin Liu

In NSDI 2024. PDF Cite Code Slides Video

DistMind: Efficient Resource Disaggregation for Deep Learning Workloads.
Xin Jin, Zhihao Bai, Zhen Zhang, Yibo Zhu, Yinmin Zhong, Xuanzhe Liu

In TON 2024. PDF Cite

Fast Distributed Inference Serving for Large Language Models.
Bingyang Wu, Yinmin Zhong, Zili Zhang, Gang Huang, Xuanzhe Liu, Xin Jin

In Preprint. PDF Cite

AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving.
Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E. Gonzalez, Ion Stoica

In OSDI 2023. PDF Cite Code Video

ElasticFlow: An Elastic Serverless Training Platform for Distributed Deep Learning.
Diandian Gu, Yihao Zhao, Yinmin Zhong, Yifan Xiong, Zhenhua Han, Peng Cheng, Fan Yang, Gang Huang, Xin Jin, Xuanzhe Liu

In ASPLOS 2023. PDF Cite Code Video