James Z.

Designed and scaled the agentic RL post-training pipeline for Gemini 3/3.1 Pro—enabling frontier tool calling, multi-step planning, and autonomous web browsing (Project Mariner); contributed to Gemini leading 13/16 benchmarks including SWE-Bench Verified (80.6%), GPQA Diamond (94.3%), and ARC-AGI-2 (77.1%). Built asynchronous agentic RL training infrastructure with decoupled rollout/training stages, windowed scheduling, and train-rollout GPU multiplexing—solving the throughput-stability trilemma for long-horizon agent tasks. Scaled to 200K+ multi-step tool-use trajectories across 3000+ GPU training runs. Identified and resolved three critical agentic RL challenges: (1) train-serve distribution mismatch—built token-level consistency framework across rollout/training engines; (2) heavy-tailed training instability—designed adaptive mask-and-filter with positive-trajectory curriculum; (3) exploration diversity collapse—implemented diversity-aware RL objectives preserving distinct tool-call strategies. Built chunk-level advantage estimation for scalable credit assignment—each tool-call interaction as the natural credit unit, reducing gradient variance by 3x on long-horizon trajectories. Developed fine-grained behavior monitoring detecting subtle reward hacking patterns with early-warning signals.

Led a team of 20+ to build Microsoft 365 Copilot—AI virtual collaborator adopted by 90%+ of Fortune 500 with 33M+ monthly active users; designed agentic RL training pipelines and tool-use reward models for enterprise agents navigating organizational knowledge and executing multi-step workflows across finance, legal, and healthcare. Built Copilot Tuning—an RL system fine-tuning LLMs on organizational data to learn enterprise tool preferences and domain expertise; adopted by 50K+ organizations. Designed multi-agent orchestration where specialized agents delegate tasks and collaborate under human oversight, trained on 100K+ tool-use trajectories (92% tool call success rate). Architected human-in-the-loop agentic training platform coordinating 500+ domain experts, generating 2M+ annotated agent trajectories monthly with multi-stage quality control; deployed at Barclays, Pfizer, and Eli Lilly for domain-specific agent fine-tuning.