Saket Harsh

Senior Software Engineer @ Coram AI

About

Driven by a passion for building high-performance systems, I specialize in optimizing on-device AI execution to bring machine learning closer to the edge. As a Systems Engineering Lead at NimbleEdge, I focus on designing and implementing efficient systems that enable machine learning models to run seamlessly on mobile devices. My work involves deep interaction with ML runtimes, ensuring faster inference times and optimal resource utilization. Previously, I was a Software Engineer 2 at Microsoft Azure, where I contributed to large-scale distributed systems, managing mission-critical production workloads. This experience strengthened my expertise in building resilient, scalable architectures capable of handling immense computational demands. My technical proficiency spans across C++, Golang, and C#, with a strong foundation in distributed systems and on-device machine learning. Additionally, I have a keen interest in algorithms and mathematics, which helps me approach problem-solving with a rigorous and analytical mindset. I thrive on solving complex engineering challenges at the intersection of systems architecture, AI acceleration, and performance optimization. Always eager to explore cutting-edge advancements in systems engineering and AI, I welcome opportunities for collaboration and discussion.

Country

India

City

Bengaluru

Industry

Computer Software

Skill

gdb, GNU Debugger, Go (Programming Language), Cloud-Native Applications, Back-End Web Development, Computer Science, Cloud Computing, Microservices, Distributed Systems, Algorithms, C++, Functional Programming, C, Git, Github, Python, Linux, Software Development

Experience

Senior Software Engineer

Coram AI

2025-4 - Present · 1 yr 6 mos

Bengaluru, Karnataka, India

Systems Engineering Lead

NimbleEdge

2022-12 - 2025-2 · 2 yrs 3 mos

Bengaluru, Karnataka, India

• Bootstrapped the engineering team from scratch, built out platform for on-device machine learning. • Built custom light weight Python interpreter in C++ that can be executed on mobile devices. • Worked with ONNXRuntime and TFLite C/C++ APIs to provide seamless APIs on Android via JNI and NDK Toolchain. • Integrated ONNXRuntime for GenAI to provide support for structured streaming LLM responses.

Software Engineer II

Microsoft

2021-9 - 2022-11 · 1 yr 3 mos

Hyderabad Area, India

Enabled Azure Service Fabric clusters to be hosted on Spot instances ,leading to lower evictions rate, and 10x fewer interruptions on clusters for thousands of Azure customers. Streamlined the placement of stateless microservices on Service Fabric clusters by creating specific Stateless only host node types, which perform 10x faster in scale-in/out and can scale up to 1000 instances.

Software Engineer

Microsoft

2019-6 - 2021-8 · 2 yrs 3 mos

Hyderabad, Telangana, India

• Simplified OS upgrade process on large clusters, by rewritine a service which considered all production service health, traffic patterns, and timely maintenance window constraints. The service got instantly adopted by large scale teams like Office and Microsoft Teams. • Established service-to-service communication between Compute and Cluster resource services of Azure, during host nodes allocation, resulting in reduced Node up time from 5 mins to 2 mins.

Software Development Intern

Samsung India

2018-5 - 2018-7 · 3 mos

Bengaluru Area, India

- Built a C++ tool to construct skeleton from a 3D Point Cloud frame using Principal Component Analysis and Iterative Closest Point Algorithms - Devised an O(n) clustering algorithm to classify points on the basis of their normal distance from the skeleton to identify body parts