Lakshya lnu

Leading the development of the next version of Adobe's text-video foundational model.

I am a Machine Learning Engineer at Adobe Firefly, where I lead the development of the Generative Extend feature, the first AI-powered tool for editors in Premiere Pro.

Worked in Adobe's Document Cloud team, to extract meaningful constructs out of documents, called asides. {Asides are non-contextual pieces of text or quotes or images in the flow of the page that needs to be identified for better page understanding}. Fine-tuned LayoutLM-v3 model on 40k pages with segment-level bounding boxes which are extracted from existing adobe pipeline to achieve 87.78\% word level f1-score, compared to 77.2\% achieved by heuristics being used at that time. Used multi-class classification(by also predicting other constructs like tables) instead of binary classification which led to better performance. Moreover, tested the model after its integration into the pipeline with additional heuristics. After integration, it was able to correct ~85\% of the existing problem pages. Also trained Unidoc, which is the commercially available model for Adobe, to get equivalent performance (89.9\% segment level f1 scores) to LayoutLMv3, by improving data quality itself.

I worked as a Machine Learning Engineer in Samsung's IoT group. Hence, I was actively involved in the research and development of Machine Learning Techniques. Moreover, I was also responsible for their deployment on various platforms in different languages. I've deployed Tensorflow and Pytorch models in nodejs, java, Halide-C++, and on Cloud, Microsoft Azure-Edge, and Android. Please refer to the CV for a brief description of my projects at Samsung.

A prism program is a symbiosis between Samsung and Indian Universities. It can be seen as a remote internship opportunity for a group of university students. I was in-charge of two such group for the tasks of Motion Detection, Human Detection and Video Summarization.

Worked on deep Neural Network based "Intelligent Filter Advisor" - Provide ranking to the image filters conditioned on the input image. An Android app was also built for the same. I was able to achieve 78% accuracy on our custom data set by utilizing transfer learning.

I worked on ”OPTIVE : Optimal Configuration of Virtual Sensor”. In Mobile Sensor Clouds, the mobility of the sensors can cause the coverage of a sensing region to drop, necessitating the reconfiguration of the corresponding virtual sensor. I modelled the problem as a Markov Decision Process (MDP) with an infinite horizon, which could be solved by the Value Iteration Method found in RL. I also formulated the theorem that showed my approach guaranteed optimal solution.

Responsibilities: SWG website management Managed and taught in SWG initiative of Extra-classes for the students by the students. Seminars. Managed student consultation program.