Sai Chaitanya Guntupally

Data Engineer @ Intact

About

Data Engineer with over 6 years of experience designing and optimizing cloud and big data platforms, building scalable data pipelines, and enabling data‑driven decision‑making. I specialize in handling large, complex datasets, integrating diverse data sources, and delivering secure, high‑performance systems on AWS and Azure. I have led the development of enterprise‑scale data platforms using AWS, Azure, Hadoop, Spark, Databricks, and Kafka to improve scalability and performance. I design and automate ETL pipelines with Glue, Airflow, SSIS, and Azure Data Factory, and work extensively with SQL Server, Oracle, PostgreSQL, Snowflake, and MongoDB to support analytics, reporting, and business intelligence. My experience includes implementing CI/CD pipelines, Terraform automation, and Kubernetes clusters, as well as collaborating with cross‑functional stakeholders to align data solutions with business goals. I am recognized for bridging engineering and business teams to implement efficient, cost‑effective solutions that improve data quality, governance, and decision‑making.

Country

Canada

City

Toronto

Industry

Information Technology & Services

Skill

Python (Programming Language), Data Engineering, Data Architects

Experience

Data Engineer

Intact

2022-9 - Present · 4 yrs 1 mo

Toronto, Ontario, Canada

Created Kinesis Data Streams, Kinesis Data Firehose, and Kinesis Data Analytics to capture and process streaming data, delivering outputs into S3, DynamoDB, and Redshift for storage and analysis. Used AWS services including S3, EC2, Glue, Athena, Redshift, EMR, SNS, SQS, and DMS to build scalable, secure data solutions. Extracted data from multiple source systems (S3, Redshift, RDS) and created tables/databases in Glue Data Catalog using Glue Crawlers. Built application and database servers using EC2, created AMIs, and used RDS for PostgreSQL to support analytics workloads. Handled heterogeneous data sources and databases (Oracle, IBM DB2, XML) using SSIS with robust error handling and logging. Developed Spark applications in Scala and Python (PySpark) and machine learning models to convert unstructured data into structured representations. Integrated Apache Airflow with S3, Redshift, and EMR to orchestrate complex batch and streaming data workflows. Implemented Terraform state management with remote backends (S3, DynamoDB) and integrated Terraform with CI/CD (Jenkins, GitLab) for automated infrastructure deployments. Guided development teams on PySpark‑based ETL best practices and data engineering standards. Collaborated with UX/UI designers and front‑end teams (React, Angular, Vue.js) to integrate analytics into interactive dashboards. Utilized Power BI visualizations (charts, graphs, maps, slicers) to present insights and support intuitive data exploration. Worked with orchestration and workflow technologies including Apache Airflow and Dagster to manage and automate complex workflows.

Data Engineer

Citi

2019-10 - 2022-8 · 2 yrs 11 mos

Mississauga, Ontario, Canada

Analyzed, designed, built, and deployed modern data solutions using Microsoft Azure PaaS/SaaS services to support data visualization and analytics. Built scalable distributed data solutions using Azure Data Lake, Azure Databricks, Azure HDInsight, and Azure Cosmos DB. Created PySpark frameworks to move data from RDBMS systems to Amazon S3 and Azure Data Lake. Designed and developed SSIS packages, stored procedures, configuration files, tables, views, and functions following performance best practices. Implemented robust error handling and retry mechanisms in Apache Airflow DAGs to ensure fault‑tolerant ETL operations. Worked extensively with Azure Data Factory activities (Lookups, Stored Procedures, If Condition, For Each, Set/Append Variable, Get Metadata, Filter, Wait) and configured triggers and monitoring. Integrated Apache NiFi with SQL/NoSQL databases, message queues, APIs, and IoT devices for seamless data integration. Utilized Dagster to build scalable, maintainable data workflows with diverse sources and sinks. Configured Azure Logic Apps for automated notifications, dynamic pipelines, and secure connections using Azure Key Vault. Implemented data governance and security policies in Tableau and Power BI to meet privacy and regulatory requirements. Performed one‑time data migration of multi‑state data from SQL Server to Snowflake using Python and SnowSQL, and developed ETL pipelines for regulatory and financial reporting in Snowflake. Coordinated with business teams (Membership, Retail, Finance, Claims) to process and store data in the Data Lake for future analytics use.

Data Engineer

Infor

2018-5 - 2019-7 · 1 yr 3 mos

Hyderabad, Telangana, India

Implemented scalable distributed data solutions using the Hadoop ecosystem with focus on efficient resource allocation. Created MapReduce streaming jobs in Java and implemented them with Hive and Pig, applying optimization techniques to reduce HDFS storage requirements. Migrated databases from SQLite to MySQL to PostgreSQL while maintaining complete data integrity. Designed, implemented, and maintained data pipelines with Apache Airflow for ingestion, transformation, and loading. Designed and maintained databases using Python and developed RESTful APIs with Flask, SQLAlchemy, and PostgreSQL. Built and managed CI/CD pipelines using tools such as Jenkins, Docker, Kubernetes, GoCD, and Autosys to streamline development and deployment workflows. Integrated multiple data sources into Power BI (databases, APIs, flat files) and developed advanced SQL queries, procedures, and triggers in Oracle and MySQL. Addressed data consistency challenges in distributed systems using Kafka message ordering and delivery semantics. Configured and administered Hadoop clusters with Apache Hadoop and Cloudera, following Agile methodologies (story grooming, sprint planning, daily stand‑ups).