Zhenyu Wang

@Siri Speech Foundation model post-training for audio modality Production-oriented LLM-based streaming ASR Multilingual ASR Modeling R&D for MoE-based knowledge factorization and compute-aware dynamic execution

@ Electrical and Computer Engineering Department Voice Conversion, Speech synthesis

@ Computer Science & Software Engineering Department Teaching Courses: CS1430-Introduction to Programming CS3520-Programming Language Structure

Speaker Recognition, Speech Recognition, Model Post-training, Audio Anti-spoofing, Speech Signal Processing, and Deep Learning @ CRSS The Ph.D. thesis was successfully defended on July 17th, 2024. The title is "Robust Text-independent Forensic Speaker Recognition Against Multiple Acoustic Domain Mismatches and Spoofing Attacks." Major publications: * Z. Wang, and John H. L. Hansen. Multi-source Domain Adaptation for Text-independent Forensic Speaker Recognition [J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 60-75, 2021. * Z. Wang and J. H. L. Hansen, Toward Improving Synthetic Audio Spoofing Detection Robustness via Meta-Learning and Disentangled Training With Adversarial Examples [J]. IEEE ACCESS, vol. 12, pp. 99894-99911, 2024.

Audio Generation R&D with diffusion model and LLM. Responsibilities include data engineering/analysis, research, model design, implementation, data parallel distributed training, optimization, evaluation, results & demo presentations. The internship work titled as "Textually-Guided Audio Generation Using A Dual-Conditioned Latent Diffusion Model" was accepted by NeurIPS 2024 Workshop.

Customized keyword spotting system R&D. Responsibilities cover implementing the entire pipeline (data I/O, model design, implementation, parallel distributed training, evaluation), committing code to the FB codebase, and presentations. The internship work titled "Hardware-efficient Customized Keyword Spotting with Spectral-Temporal Graph Attentive Pooling and a hybrid Loss" accepted by INTERSPEECH 2024.

Robust small-footprint KWS system R&D @ AI Speech Team based on disentangled training with adversarial examples. Responsibilities include model architecture design, improvement and implementation, research, model training, and testing. This internship work, titled "Disentangled Training with Adversarial Examples For Robust Small-footprint Keyword Spotting", was accepted by ICASSP 2023.

@ Speech Interactive Technology Team Online decoding platform development based on the Kaldi Nnet3 recipe for the real-time acoustic model assessment. Conducting research on ASR, including DNN pruning based speech recognition acoustic model iterative training and testing, revising the acoustic dictionary and language model for addressing issues introduced by English liaison and reduction in the mixed Chinese-English scenario, feature engineering, model architecture improvement, and training acceleration.

Data preprocessing, GMM-HMM, DNN acoustic model training and testing, applied in far-field speech recognition scenarios. Keyword spotting R&D for the voice assistant XiaoYa