Patrick Toulme

Software Engineer @ Google

About

Latest blog post: https://patricktoulme.substack.com/p/frontier-pretraining-infrastructureWorking on XLA compiler at Google TPU.Previously at Meta MTIA compiler and performance and AWS Neuron working on the XLA compiler and kernel development.I contribute actively to open source - https://github.com/patrick-toulmeInterests: Large language models, XLA, MLIR, HLO, MHLO, LLVM, Pallas, Triton, machine learning compilers, Jax, PyTorch, distributed training, PJRT, machine learning scaling, high performance computing, machine learning accelerators, pretraining, fine tuning, reinforcement learning, large accelerator clusters (Slurm)

Country

United States

City

New York

Industry

Computer Software

Skill

Python (Programming Language), Communication, Information Extraction, Question Answering, Amazon Web Services (AWS), Pattern Recognition, Modeling, Machine Learning, Neural Networks, Computer Science, Language Modeling, Deep Learning, Artificial Intelligence (AI), Generative AI, Large Language Models (LLM), PyTorch Lightning, Kubernetes, Docker, Nvidia NeMo, CUDA

Experience

Software Engineer

Google

2026-4 - Present · 6 mos

New York, NY

TPU XLA team https://cloud.google.com/tpu https://openxla.org/

Software Engineer - MTIA Compiler and Performance Team

Machine Learning Engineer II | AWS Neuron - Annapurna Labs

Amazon Web Services (AWS)

2024-4 - 2025-1 · 10 mos

Cupertino, California, United States

Promoted to Machine Learning Engineer II - Currently working on the XLA compiler for the Neuron backend - Middle end compiler loop optimizations - High performance kernels targeting Neuron hardware

Machine Learning Engineer | AWS Neuron - Annapurna Labs

Amazon Web Services (AWS)

2023-6 - 2024-4 · 11 mos

Cupertino, California, United States

Machine Learning Engineer on AWS Neuron Training Team - Annapurna Laboratories - High performance deep learning on Trn1 and Inf2 instances - OpenXLA compiler passes - PJRT support for Neuron backend - Training on multi thousand Neuron device clusters. - GSPMD with PyTorch/Jax for automatic parallelization of machine learning computations. - Distributed training for Bedrock - Pretraining and fine tuning multi hundred billion parameter large language models on Neuron cores - Training using XLA and PyTorch and Jax https://aws.amazon.com/machine-learning/neuron/

Software Engineer | AWS Bedrock

Amazon Web Services (AWS)

2022-5 - 2023-6 · 1 yr 2 mos

Arlington, Virginia, United States

Software Engineer on AWS Bedrock Training Team - Pretraining and fine tuning large language models (Titan models) - Distributed multi node training using Deepspeed and NeMo https://aws.amazon.com/bedrock/

Researcher

UVA Engineering

2020-8 - 2022-6 · 1 yr 11 mos

Charlottesville, Virginia, United States

Researching with Dr. Panagiotis Apostolellis on the development of using chatbots for depression screening. Currently using Google DialogFlow, Kommunicate and Swift to develop Marcus: Depression Screening.

Software Engineer | Institutional Trading

Coinbase

2022-5 - 2022-5 · 1 mo

San Francisco, California, United States

Software Engineer on the Institutional Trading team. Layoff due to economic conditions

Software Engineer

Living Popups Inc.

2019-8 - 2020-2 · 7 mos

Beverly Hills, California, United States

Worked as a software engineer working under the Chief Technology Officer and Lead Developer. I worked on a variety of projects, such as Boxcar AR, Boxcar ES and Orkney Origins. I worked extensively on all three applications. I also added Firebase database systems to the applications and developed the in app purchase and billing system for the applications. I also managed the applications on the App Store and Google Play Store

Software Engineer

Nimbly

2019-7 - 2019-8 · 2 mos

New York City Metropolitan Area

Served as lead software engineer in the development of a full 3D store front simulation built in Unity. I reported to the CEO, and I built the project in a crunch deadline of only 2.5 weeks from contract signing to delivery by Nimbly to the client.