Dongyue Li

My research focuses on multitask learning, fine-tuning, and their applications to post-training foundation models. Specifically, I have designed scalable multitask learning algorithms for foundation models. Use cases include supervised fine-tuning of large language models, chain-of-thought fine-tuning, and multi-step reasoning. I have also worked on designing robust fine-tuning with generalization guarantees based on Hessian measures.

Low-precision training and quantization methods for large language models

Pre-training transformer-based foundation models for large-scale tabular time-series data

Designed deep neural network compression algorithms, such as pruning and quantization, for convolutional neural networks, graph neural networks, and transformers.