Hemanth Nidamanuru

As a Data Engineer specializing in scalable data infrastructure, I build high-performance, production-grade platforms using Apache Spark, Kafka, and Databricks to power analytics and business intelligence. My experience spans designing real-time streaming systems, optimizing distributed pipelines, and implementing data quality frameworks that transform unreliable data into trusted assets.At Cognizant, I engineered end-to-end ETL workflows for enterprise clients across healthcare, retail, and financial services. I architected Kafka-based streaming pipelines for real-time ingestion, optimized batch workloads using Spark techniques such as partition pruning and broadcast joins, and implemented data validation frameworks with Great Expectations to reduce downstream incidents. My solutions leveraged both AWS (S3, Glue, Redshift) and Azure (Data Factory, Synapse), delivering cloud-native platforms with high production reliability.My expertise covers the full data engineering lifecycle:- Stream processing with Kafka and PySpark- Building ACID-compliant data lakes with Delta Lake- Designing dimensional models for analytics- Creating monitoring systems to detect schema drift and anomaliesI've debugged memory-intensive Spark jobs on large datasets, integrated inconsistent REST APIs and CDC streams, and documented architectures to ensure long-term maintainability.I'm passionate about solving complex data challenges at scale, building resilient streaming systems, implementing robust data quality controls, and optimizing distributed workloads. I thrive in collaborative environments where engineering and analytics teams work together to turn raw data into actionable insights.Beyond project work, I actively share knowledge through technical documentation and mentoring junior engineers on distributed systems and best practices. I value strong computer science fundamentals combined with practical, production-ready implementations.- Databricks Certified Data Engineer | MS in Computer Science from Purdue University Fort Wayne- Published researcher: Traffic sign detection using deep learning, Arduino-based robotics automationI'm excited to contribute to organizations that prioritize reliability, performance, and data quality. Whether designing Lakehouse architectures, scaling real-time platforms, or strengthening governance, I'm ready for the next challenge.Let's connect if you're building data platforms where quality, scale, and reliability truly matter.