Chandini Nekkanti

Senior Big Data Engineer @ Cardinal Health

About

Senior Big Data Engineer with 11+ years of proven expertise delivering enterprise-scale data solutions across healthcare, financial services, and e-commerce sectors. Specialized in architecting and optimizing high-performance data pipelines using Apache Spark 3.x, with a track record of achieving measurable results, including 45% performance improvements through Hadoop-to-Spark migrations and 60% query acceleration using advanced optimization techniques like Adaptive Query Execution and Dynamic Partition Pruning. Deep technical proficiency spans the complete big data ecosystem, including Kafka, Airflow, and distributed processing frameworks, with hands-on experience across AWS, Azure, and GCP cloud platforms. Currently driving digital transformation at Cardinal Health, where I've modernized legacy data architectures to support real-time healthcare analytics at petabyte scale while maintaining strict HIPAA compliance. Previously engineered mission-critical fraud detection systems at BNY Mellon processing billions in daily financial transactions, and built scalable inventory management platforms at Amazon serving millions of customers globally. Expert in translating complex business requirements into robust technical solutions, with demonstrated ability to collaborate effectively with cross-functional teams and infrastructure stakeholders. Core competencies include Spark application development (Scala/PySpark), real-time streaming architectures (Kafka + Spark Streaming), cloud-native data platform design (Databricks, EMR, Synapse), and advanced ETL orchestration (Airflow, ADF, AWS Glue). Combine strong software engineering fundamentals with data engineering best practices to build scalable, cost-efficient solutions that drive measurable business impact and accelerate time-to-insight for analytics teams.

Country

City

United States

Industry

Information Technology & Services

Skill

Data Visualization, Dagster, Apache Atlas, Alation, Collibra, Soda Core, Deequ, Great Expectations, Data Mesh Concepts, Data Build Tool (DBT), Medallion Architecture, Apache Hudi, Apache Iceberg, Delta Lake, Custom ETL Job Scheduler & Monitoring Tools, In-house Data Warehouse, AWS Lambda (first use case), AWS S3, AWS Glue (first use case in ETL context), AWS Redshift

Experience

Senior Big Data Engineer

Cardinal Health

2024-8 - Present · 2 yrs 2 mos

Dublin, OH

Healthcare Supply Chain & Analytics Platform • Led end-to-end design of real-time and batch data pipelines for healthcare supply chain and inventory analytics using Kafka, Spark, Databricks, ADF, and Synapse. • Built scalable ingestion frameworks integrating POS, supplier, and hospital systems into Azure Data Lake for enterprise analytics. • Implemented real-time alerting and monitoring for critical medical supply levels using Kafka Streams and Spark Streaming. • Optimized large-scale healthcare datasets with PySpark/Spark SQL, improving processing performance and data quality. • Delivered executive dashboards in Power BI enabling real-time visibility into inventory, logistics, and operational KPIs. • Ensured HIPAA compliance through data masking, encryption, and secure access controls.

Sr Data Engineer

BNY

2021-10 - 2024-7 · 2 yrs 10 mos

New York, United States

Financial Data Engineering & Fraud Analytics • Designed and implemented high-volume financial data pipelines for fraud detection, market analytics, and regulatory reporting. • Built real-time and batch workflows using Kafka, Spark, AWS Glue, EMR, Airflow, and Redshift. • Implemented Semarchy xDM for master data management to ensure consistent client and financial instrument data. • Developed distributed Spark applications in Scala and Python for large-scale transaction and portfolio analytics. • Optimized data warehouse performance using star/snowflake schemas, indexing, and partitioning. • Enabled near real-time risk monitoring and fraud alerts using Kafka + Spark Streaming.

Data Analyst / Engineer

Amazon

2018-12 - 2021-9 · 2 yrs 10 mos

Seattle, WA

E-Commerce Analytics & Inventory Systems • Built scalable inventory and order processing pipelines handling terabytes of data across global marketplaces. • Implemented real-time ingestion using Kinesis, Lambda, and AWS Glue, supporting low-latency analytics. • Designed analytics workflows on EMR, Redshift, Athena, and S3 for sales, demand forecasting, and KPIs. • Deployed ML models on SageMaker to support product recommendations and demand forecasting. • Created interactive dashboards using QuickSight for operations and leadership teams.

Data Engineer / Analyst

State Farm

2016-2 - 2018-11 · 2 yrs 10 mos

Bloomington, Illinois, United States

Insurance Data Migration & Analytics Platform • Led large-scale insurance data migration from legacy systems to Azure-based platforms. • Built ingestion and orchestration pipelines using ADF, Talend, Airflow, and Databricks. • Designed data lakes and warehouses supporting claims, policy, and risk analytics. • Implemented predictive models for risk assessment and fraud detection. • Delivered self-service dashboards in Power BI for underwriting and claims teams.

ETL Developer

Zoho

2014-6 - 2015-12 · 1 yr 7 mos

Chennai, Tamil Nadu, India

SaaS Data Integration & Analytics • Built ETL pipelines integrating CRM and product data using Informatica and Talend. • Automated large-scale data ingestion, transformation, and quality checks across SaaS platforms. • Supported analytics and reporting by designing optimized warehouse structures and BI integrations.