Fei Wu

1. Optimize large model training MFU and memory efficiency. Focusing on model sharding techniques TP/PP/CP/EP/FSDP etc on both GPU and AWS Trainium 2. Optimize large scale training goodput, fault tolerance and resilience. - Optimized the fault handling overhead including NCCL communicator bootstrapping, dataloading and checkpoint save/load. - Implemented a novel checkpoint-less training method with RDMA peer-to-peer model states recovery 3. RDMA implementation and optimization for AWS EFA, building libraries like DeepEP on AWS 4. Building RL-based post-training platform

Distributed training for very large deep learning(DL) models on cloud. Model parallel, data parallel and all other DL related optimizations. Projects: - Sagemaker model parallel: A cross-framework(Tensorflow, Pytorch) library that allows user to train very large models in a model parallel way on AWS sagemaker when a single GPU's memory could not fit the entire model. - Various DL model optimization, including Mask-RCNN, Bert. Optimizing the models for best performance, scaling to large cluster(data parallel), fp16, gradient accumulation, etc.

Research interests include parallel big data scheduling, machine learning in virtualization and resource-sharing systems.

Develop Algorithms for A/B testing 1. Using Beyesian A/B testing to replace the traditional A/B testing; 2. Enhance the A/B testing by AI method including reinforcement learning and neural network 3. Implement Q-learning for multi-channel A/B testing