Dhavalkumar Patel

- Uplifting and modernisation of the web application from on-premise to cloud infrastructure using Amazon Web Services and Open source tools such as Kubernetes, Docker for container orchestration; Terraform and Cloud Formation for cloud resource; Prometheus for alerting and monitoring; with AWS as the middleware - Designed and Developed the CI/CD pipeline architecture for Ad Team Comcast using opensource tools such as Jenkins for writing the Jobs and providing automated flows - Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Athena - Designed and developed a Validation Framework on AWS. executes as a spark job and uses AWS Aurora Serverless. It is used by all the applications managed by the team and introduced automation which reduced the job processing time by ~50%. - Developed ETL data pipelines using AWS Lambda triggers Glue jobs and Databricks jobs to share relevant data with Programmers and Measurement partners. - Orchestrated ingestion pipelines using S3 event notifications, AWS Lambda triggers and AWS Glue jobs which introduced automation and provided alerts using Cloud Watch onto various metrics. - Worked on data migrations, fixed production issues and bugs blocking critical report deliveries. - Implemented Event based AWS Lambda functions to achieve serverless architectural needs. - Involved in working with and ensuring consistency between various data formats like Avro, parquet etc. - Supervised the design, setup and maintenance of the ELK stack for fast data science, performance engineering and exploratory analysis requirements. Designed job trigger mechanisms using aws DataPipeline. - Configured auto-scalable and highly available microservices set with monitoring and logging using AWS, Docker, and Jenkins. The architecture included Docker as the container technology with Kubernetes and worked with REST.

- Led the design and deployment of micro-backend architecture using Django framework, ensuring consistent delivery of web components. - Deployed AWS services such as EC2, RDS, and Lambda for resource provisioning, data management, and serverless computations, thus ensuring scalability and efficiency of the system built with Django. - Employed open-source tools like Docker and Kubernetes for containerization and orchestration of the Django and Flask applications, scaling, and management across clusters of hosts. - Implemented a CI/CD pipeline using Jenkins to maintain a seamless workflow from development to deployment. Leveraged Django's robustness and flexibility to develop a high-performance order domain orchestration layer, which interfaces with over 20 disparate systems. - Managed peak traffic of around 450 transactions per second (TPS) with write latency maintained under 200 milliseconds, and a 94th percentile read response time of 100 milliseconds. - Utilized Python's Elasticsearch library in conjunction with Django to build an efficient Order Search Cluster capable of indexing multiple terabytes of data. - Managed the performance of the Order Search Cluster to handle an estimated peak load of 400 TPS, ensuring write latency within 7 seconds and maintaining a 95th percentile read response time of 120 milliseconds. - Utilized Python's data analysis libraries such as NumPy and Pandas in combination with Django's ORM for proactive system metrics analysis. - Involved in deploying web applications through Application Load Balancer and API Gateway for high durability of the application along with managing communication to other AWS resources, Environment. - Wrote Terraform templates for the required automation in AWS services and created database objects in SnowFlake. - Integrated Snowflake with AWS Lambda functions to automate data processing tasks, such as data enrichment, data validation, or data aggregation, improving data pipeline efficiency and accuracy.