Publications

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving.
Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang

In OSDI 2024. PDF Cite Code

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs.
Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, Yulu Jia, Sun He, Hongmin Chen, Zhihao Bai, Qi Hou, Shipeng Yan, Ding Zhou, Yiyao Sheng, Zhuo Jiang, Haohan Xu, Haoran Wei, Zhang Zhang, Pengfei Nie, Leqi Zou, Sida Zhao, Liang Xiang, Zherui Liu, Zhe Li, Xiaoying Jia, Jianxi Ye, Xin Jin, Xin Liu

In NSDI 2024. PDF Cite Code Slides Video

DistMind: Efficient Resource Disaggregation for Deep Learning Workloads.
Xin Jin, Zhihao Bai, Zhen Zhang, Yibo Zhu, Yinmin Zhong, Xuanzhe Liu

In TON 2024. PDF Cite

AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving.
Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E. Gonzalez, Ion Stoica

In OSDI 2023. PDF Cite Code Video

ElasticFlow: An Elastic Serverless Training Platform for Distributed Deep Learning.
Diandian Gu, Yihao Zhao, Yinmin Zhong, Yifan Xiong, Zhenhua Han, Peng Cheng, Fan Yang, Gang Huang, Xin Jin, Xuanzhe Liu

In ASPLOS 2023. PDF Cite Code Video

LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism.
Bingyang Wu, Shengyu Liu, Yinmin Zhong, Peng Sun, Xuanzhe Liu, Xin Jin

In Preprint. PDF Cite

FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion.
Liwen Chang, Wenlei Bao, Qi Hou, Chengquan Jiang, Ningxin Zheng, Yinmin Zhong, Xuanrun Zhang, Zuquan Song, Ziheng Jiang, Haibin Lin, Xin Jin, Xin Liu

In Preprint. PDF Cite

Fast Distributed Inference Serving for Large Language Models.
Bingyang Wu, Yinmin Zhong, Zili Zhang, Gang Huang, Xuanzhe Liu, Xin Jin

In Preprint. PDF Cite