Om Rajyaguru

Post-training, synthetic data generation, and end-to-end evaluation for Microsoft and OpenAI models used across the Copilot suite (also agentic evaluation for M365 Copilot) Currently supporting ~46,000 people across the Experiences + Devices (E+D) org by developing a library of reusable agent skills, subagents, MCPs, hooks, and tutorials to enhance agentic coding adoption across Microsoft. I spearheaded a massive overhaul of Microsoft’s Core AI documentation in collaboration with the Office of the CTO, establishing the central and authoritative AI resource for all 300,000+ worldwide employees and contractors. I expanded the platform by authoring comprehensive playbooks, tutorials, integrations, metrics tracking, and community platform enhancements.

Invented and patented CITADEL agents - Constructed at Inference Time Agent with Dynamically Executable Library. Invented as a replacement for many Multi-Agent systems when supporting a high cardinality of tools and minimizing cost and latency is the primary concern. This solution dynamically loads tools and their implementations from a vector index based on the user's query, re-written and enhanced by an orchestration agent (which can optionally be fine tuned for better control over query rewriting and instructions). The loaded tools and corresponding "chunks" of a system prompt are used to create a specialist agent on the spot loaded with the tools most likely needed to answer the users query. Invented and patented the Retriever architecture - an "all-in-one" semantic search solution that generates retrievers for each dataset/use case within an application. This is done upon application startup via a generator factory that persists for the lifecycle of the application. Retrievers are highly configurable objects that perform different kinds of semantic search per dataset/use case. Fits perfectly into RAG / semantic search use cases against an enterprise knowledge base (including structured data), especially when invoked as a tool via an agentic system. Invented and patented the orchestrator/subLLM framework. This is the first abstraction of what are known today as "multi agent systems", invented in mid 2023, shortly after models capable of tool calling released from OpenAI. The orchestrator agent acts as a "head" and is provided with details of the subLLM agents in its system prompt, and the subLLMs are provided as tools the orchestrator can invoke. All agents are stateless and all routers/workflows could be stateful as needed. Unlike other solutions (i.e., LangGraph, AutoGen, Swarm, CrewAI) this makes no assumptions about the structure and order of your agents and workflows. This solution is the backbone of the Splunk AI Assistant for Observability.

We found a method for intelligent non-uniform segmentation of time series data, followed by adaptive thresholding, for use in Splunk IT Service Intelligence. This reduces the time taken to set up time policies and adaptive thresholding from a few days to a few seconds. Did some work on drift detection and improving seasonality detection with multiple layers (weekly, daily, hourly). I developed several methods to evaluate large language models for code generation tasks for languages where public benchmarks are not available (e.g. SPL, SignalFlow, etc.). This evaluation suite uses a mix of lexical and semantic comparison methods, along with syntax validation tools and LLM based evaluations (only for comparing generated output against a ground truth). I fine tuned several LLMs for code generation tasks using multi-gpu distributed training (including a custom implementation of Fully Shared Data Parallel because PyTorch was bugged at the time), followed by quantizing and deploying these models on a custom ML platform built on AWS. From this, I created full fine-tuning and LoRA fine-tuning recipes for different parameter variants of Llama 2, CodeLlama, T5, and Mistral. Recommended multi-gpu compute configurations were also provided for full hardware+algorithmic reproducibility. In reality at the time, I had to train on GPUs with 16gb-40gb VRAM because higher memory GPUs were extremely scarce at the time. I designed and developed algorithms for the Search Assistant App for Splunk to help new users find insights from their log data with minimal effort. The app also acts as a query builder that generates SPL queries to find outliers or rare patterns in their data, which helps users learn SPL syntax in the process.

My research project was on distance-based clustering methods for semi-structured text data (log parsing/clustering). I experimented with several open-source log parsers in an effort to find an algorithm that was good at intelligently grouping logs together. I ended up modifying Splunk's custom implementation of log clustering, as it produced better results than any open source or competing log clustering solution. The integration into SPL 2 allowed log clustering to service a variety of use cases such as summarization, log anomaly detection, error searching, etc. all from the same SPL command, depending on how it was invoked. https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Cluster I also won Intern of the Month from over 250 interns company-wide!

I began tutoring mathematics in group, individual, and even virtual settings in middle school. In high school, I pivoted to tutoring for the SAT after receiving a perfect score, which involved teaching the reading and writing sections as well. In 11th grade, I began taking math classes at NCSU and volunteering at the math multimedia center for Calculus I, II, and III, Differential Equations, and Linear Algebra. After I started as a student at NCSU and the COVID-19 pandemic hit, I continued tutoring virtually and additionally taught material for introductory statistics classes.

Our project for the Game2Learn lab was to detect "wheel spinning" in an educational online game (ST Math). This is when students don't think about a question before answering and simply continue to enter answers until one of them turns out to be correct. This behavior, when learned, has been shown to carry on throughout adulthood and ideally should be detected and unlearned. My role involved plotting the performance of students, partitioned by county, school, grade, and classroom to identify patterns. To help identify wheel spinning behavior, I extracted features with significant variance from a massive quantity of semi-structured telemetry data.

Graduated Summa Cum Laude with a minor in Statistics Concentration was on Machine Learning and AI, and over half of the courses I took towards the Applied Math major requirements were at the Masters or PhD level. My two research areas were multimodal extensions of the transformer architecture and reducing the parameter count of large transformer models.