Yibo Zhu

Co-founder of an AGI startup. LLM and VLM ranked top 10 on Lmsys Chatbot Arena (Jan. 2025) Open-weights SOTA video generation model (https://huggingface.co/stepfun-ai/stepvideo-t2v) Open-weights SOTA audio LLM model (https://huggingface.co/stepfun-ai/Step-Audio-Chat)

Briefly led the efforts of next-gen AI Infra for trillion-parameter models on cloud. I reported to Mack McCauley, VP of GCE, who treated me well. I am sorry for leaving so soon for co-founding a startup.

Started as a staff-level research scientist and got multiple promotions to a director in 2021. I led a group of 100+ engineers and researchers. We were responsible for ByteDance's infrastructure for deep learning (CV/NLP/Speech/RL) training and inference. We also contributed to in-house recommendation and ads systems. We work on the following directions: 1. Compiler stack (MLIR, TVM, etc.) and OP optimization, for GPU and ASIC 2. Inference framework supporting multiple model formats, batching and different hardware 3. Distributed training (data & tensor & pipeline parallelism, optimizer, RDMA network, parameter server) 4. High speed I/O and data preprocessing for ML 5. Cluster resource scheduling (multi-tenant, maximizing GPU/ASIC utilization) 6. MLOps platforms for ML engineers In addition to day-to-day work that aims at immediate production deployment, I also actively bootstrap forward-looking projects with long-term goals.

My work was focused on designing hyperscale and reliable systems/network for cloud and AI. Many of my past projects had been shipped into Microsoft Azure and Microsoft AI. I led the following projects. • DCQCN: the de facto standard RoCEv2 congestion control protocol. It is adopted by all Mellanox CX3 Pro and later NICs, and deployed in Azure data centers. • Enabled RDMA and accelerated TCP/IP stack over container overlay network, for running AI workloads with Kubernetes. https://github.com/Microsoft/Freeflow • RDMA network monitoring and scheduling for AI infrastructure. • RoCE network deadlock avoidance. • Everflow: Large-scale in-network telemetry for network troubleshooting. • CrystalNet: Large-scale network emulator for verifying network operations. Regarding academic research, please check my publication list at http://yibozhu.com/

I spent my last three PhD years interning at Microsoft Research & Azure (combining three full-time terms and two part-time terms due to visa limit, with some breaks at school for PhD defense). Mentors: Dr. Jitu Padhye, Dr. Ming Zhang and Dr. Ratul Mahajan.

Worked on networking and distributed systems. Published 10+ top tier conference papers, e.g. three SIGCOMM's and three MobiCom's. I appreciate that my advisors allowed me to spend most of my time at Microsoft working on collaborative projects. Advisors: Prof. Ben Y. Zhao and Prof. Heather Zheng

Designed and implemented a reliable group data delivery system in datacenters. Mentor: Dr. Chuanxiong Guo