Divya V

Senior Data Engineer @ Capital One

About

Tech to me, is more than just lines of code--its about solving problems, automating the boring stuff, and occasionally wondering why it worked on my machine but nowhere else.Senior Data Engineer with the experience in designing, developing, and deploying scalable, high-performance cloud-based data platforms and big data pipelines for enterprise and fi nancial services environments.Expert in AWS, Snowflake, Apache Spark (PySpark/Scala), and Apache Kafka, with hands-on experience buildingbatch and real-time ETL/ELT pipelines, data lakes, and streaming architectures using serverless and distributed systems.Strong background in SQL and NoSQL databases, data modeling, validation, reconciliation, and performance tuning, ensuring reliable, low-latency, and high-throughput data processing. Experienced in CI/CD pipelines, Docker, Kubernetes, and Agile/Scrum, delivering production-ready and secure data solutions.Known for collaborating with cross-functional teams, mentoring engineers, and translating business requirements into scalable, maintainable, and enterprise-grade data platforms that enable analytics, reporting, and data-driven decision making.🔹Core Skills: AWS, Snowfl ake, Spark, Kafka, Python, SQL, Azure.🔹Certifi cations: AWS | Google Cloud (GCP) | Microsoft Azure.📌 Open to Senior Data Engineer / Cloud Data Engineer opportunities.At Wichita State University I mentored students in Python, helping them navigate real-world coding challenges and discover the creative side of debugging.Programming Languages:  Python, JavaScript, Node.js.Frameworks: Spring Boot, REST Services, React.js, REST, Bootstrap, JPA, Angular, GroovyDatabase: MySQL, NoSQL, Cassandra, MongoDB, Oracle DB, Redis, MariaDBCloud & DevOps: AWS (Lambda, RDS, S3, EC2, ELB, Redshift, CloudWatch), Azure, Docker, OpenShift (OCP)Tools & Platforms: Kafka, Power BI, Postman, IntelliJ, SAP CPI, Chef, Snowflake, GIT, SVN, Jenkins Testing: Junit, Mockito, Selenium, Cucumber, Gatling, JMeterMonitoring and Performance: AppDynamics, Dynatrace, Instana, Data Dog, SplunkSoftware Methodologies: Agile, Scrum, TDD, Watertall.IDE's: Eclipse, Visual Studio, IntelliJ, Spring Tool Suite Other Tools: Taleo, PoweBl, Six SigmaStronger Leadership: Jira, Scrum.

Country

United States

City

Plano

Industry

Computer Software

Skill

Postman API, PySpark, Amazon Relational Database Service (RDS), Databricks, Databricks Products, iceberg, MySQL, Python (Programming Language), SQL, Snowflake, SharePoint, Windows Server, Microsoft Azure, Agile Methodologies, Linux, Amazon Web Services (AWS)

Experience

Senior Data Engineer

Capital One

2024-7 - Present · 2 yrs 3 mos

Plano, TX

Designed and deployed highly available, cloud-native AWS data platforms using S3, EC2, EMR, Glue, Lambda, Step Functions, Redshift, DynamoDB, SNS/SQS, and CloudFormation. Built and managed enterprise-scale data lakes with security, governance, lifecycle management, and cost optimization. Developed real-time streaming pipelines using Apache Kafka and Spark Structured Streaming to meet strict SLA requirements. Migrated legacy MapReduce workloads to Apache Spark (PySpark & Scala) , significantly improving performance and scalability. Built optimized batch and streaming ETL frameworks, reducing processing time by ~40% Implemented robust data validation and reconciliation frameworks in Snowflake. Created optimized Snowfl ake schemas and views for analytics and reporting. Built CI/CD pipelines using Jenkins and Git, improving deployment reliability and speed. Collaborated with architects, product owners, and stakeholders to deliver enterprise-grade data solutions

Data Engineer

BMO Capital Markets

2021-9 - 2024-6 · 2 yrs 10 mos

United States

Engineered high-volume ETL pipelines processing millions of daily transactions. Built Kafka-based real-time streaming pipelines for operational insights and anomaly detection. Developed batch and streaming jobs using Apache Spark, improving throughput and stability. Designed optimized data models for risk, credit, and regulatory reporting. Automated ETL workflows across SQL, NoSQL, APIs, and flat-file sources. Deployed containerized workloads using Docker and Kubernetes. Performed Spark and Kafka performance tuning to reduce processing latency. Implemented data quality checks, monitoring, and alerting frameworks.