Balaji Lakshminarayanan

I lead a research team within the Gemini core team focused on Gemini post-training and evals. Specific areas of focus include: self-improvement, automating evaluation of LLMs (autoraters), generative reward models, self-critic capability, agents, post-training methods (SFT/RLHF).

Tech Lead/Manager for the "Reliable Deep Learning" team in Google Brain. Built and managed a team of research scientists and engineers in Google Brain focused on improving uncertainty and out-of-distribution robustness in deep learning. – Principal investigator for multiple research projects on uncertainty & robustness in deep learning which were published in NeurIPS, ICML, ICLR, AISTATS, JMLR, TMLR, Medical Imaging, etc. – Led Brain moonshot on "reliable deep learning" focused on compute-efficient research solutions for uncertainty (improving single model uncertainty quantification, efficient ensembles). - Developed scalable research solutions for problems faced by product teams such as improving robustness to dataset shift, noisy labels and leveraging uncertainty for adaptive computation (defer to expert model or human, ask for clarifications, etc), active learning & exploration. Our research led to several first-of-its-kind launches in Google products like YouTube Home/WatchNext/Trust&Safety, Assistant, Ads, Maps, etc and is being used across scientific efforts at Google (health, drug discovery, material discovery, etc). Wrote a practical cookbook distilling best practices for product teams with accompanying code libraries and demo colabs. – Mentored interns, AI residents and junior research scientists and research engineers. - Received “Google Research Tech Impact Award” for outstanding contributions to research, product and infrastructure. See http://www.gatsby.ucl.ac.uk/~balaji/ for more information and links to papers/talks/etc.

Worked on research topics in deep learning and artificial intelligence, and applying this research to challenging real-world problems. – Principal investigator for multiple research projects on uncertainty & robustness in deep learning which were published in NeurIPS, ICML, ICLR, Nature Medicine. – Developed scalable research solutions for problems faced by product teams (improving robustness to dataset shift, better decision making through uncertainty) and successfully launched them in production. – Mentored interns, Google AI residents and junior research scientists and research engineers. - Received “pioneer” award from DeepMind for pioneering research on uncertainty and robustness. – I was in DeepMind London from Sep 2015 to Aug 2017 and moved to Mountain View after that. See http://www.gatsby.ucl.ac.uk/~balaji/ for more information.

My PhD thesis was focused on exploring (and exploiting :)) connections between neat mathematical ideas in (non-parametric) Bayesian land and computationally efficient tricks in decision tree land, to get the best of both worlds. Specifically, I developed a variant of random forests called "Mondrian Forests" that are as scalable as popular random forests, but can be trained online (incrementally) and give much better uncertainty estimates. See my talk for details: https://project.inria.fr/bnpsi/files/2015/07/balaji.pdf PhD thesis: Decision trees and forests: a probabilistic perspective https://www.gatsby.ucl.ac.uk/~balaji/balaji-phd-thesis.pdf Advisor: Prof. Yee Whye Teh Selected Papers: Mondrian forests for large-scale regression when uncertainty matters Balaji Lakshminarayanan, Daniel M. Roy and Yee Whye Teh AISTATS, 2016 Particle Gibbs for Bayesian additive regression trees Balaji Lakshminarayanan, Daniel M. Roy and Yee Whye Teh AISTATS, 2015 Mondrian forests: Efficient online random forests Balaji Lakshminarayanan, Daniel M. Roy and Yee Whye Teh NeurIPS, 2014 Top-down particle filtering for Bayesian decision trees Balaji Lakshminarayanan, Daniel M. Roy and Yee Whye Teh ICML, 2013

Worked on ranking problems and probabilistic models for multi-annotator (crowd-sourced) data

Developed robust probabilistic models for collaborative filtering, to handle heavy-tailed behavior in discrete as well as continuous-valued ratings

Implemented algorithms for detecting fraud, waste and abuse in prescription drug claims

I worked with Prof. Yee Whye Teh, and was part of the Gatsby Unit. My PhD research was focused on combining the scalability of decision trees/forests with ideas in probabilistic/Bayesian machine learning. I worked on "Mondrian forests", which improve uncertainty quantification in random forests and enable efficient online updates for streaming data by leveraging ideas from Bayesian non-parametrics. PhD thesis: "Decision trees and forests: a probabilistic perspective" http://www.gatsby.ucl.ac.uk/~balaji/balaji-phd-thesis.pdf See http://www.gatsby.ucl.ac.uk/~balaji/ for full list of publications.

I worked on probabilistic machine learning with Prof. Raviv Raich, and was part of the Bioacoustics research group. See my webpage http://www.gatsby.ucl.ac.uk/~balaji/ for link to publications.

I was an Undergraduate Research Assistant at Integrated Systems Laboratory, where I worked on the development of communication subsystems for the Anna university micro-satellite project (https://en.wikipedia.org/wiki/ANUSAT) in collaboration with Indian Space Research Organization (ISRO).