Sai Arun Kumar Katherashala

• Collaborated with cross-functional teams within the data science and analytics team to design, develop, and execute solutions for deriving business insights and solving clients' operational and strategic problems. • Built multiple notebooks and piped all the jobs together, built an ETL pipeline to making wear algorithm predictions of the model, and writing the outputs to Azure Cosmos DB. • Can work parallelly in both GCP and Azure Clouds coherently. • Analyzed the SQL scripts and designed the solution to implement using Spark. • Designed and developed an automated framework to create and automate the development process in the Data Lake. • Created on-demand tables on S3 files using Lambda Functions and AWS Glue with Python and PySpark. Extensive experience in Java SE, including object-oriented programming, data structures, multithreading, and exception handling. • Migrated an existing on-premises application to AWS, utilizing services like EC2 and S3 for small dataset processing and storage, while also maintaining a Hadoop cluster on AWS EMR. • Utilized AWS Data Pipeline to configure data loads from S3 into Redshift. • Leveraged AWS Glue catalog with crawler to extract data from S3 and perform SQL query operations. Hands-on experience with popular Java frameworks like Spring, Hibernate, and JavaFX for building robust and scalable applications. • Implemented Spark using Python, utilizing DataFrames and Spark SQL API for faster data processing. • Developed Spark applications using PySpark and SparkSQL for data extraction, transformation, and aggregation from multiple file formats, aiming to analyze and transform the data to uncover insights into customer usage patterns. • Developed Terraform scripts to automate AWS services such as Lambda, Glue, EventBridge, ELB, CloudFront distribution, RDS, EC2, database security groups, and S3 buckets. • Conducted data migration from On-Premises systems into a Snowflake Cloud data warehouse, automating workloads in the process.

• Worked with cross functional teams within the data science and analytics team to design, develop, and execute solutions to derive business insights and solve clients' operational and strategic problems. • Developed and consumed RESTful and SOAP web services using Java, enabling seamless integration between distributed systems. • Designed and implemented microservices-based applications using Spring Boot, enhancing modularity, scalability, and maintainability. • Proficient in using JDBC and JPA for database connectivity and ORM, with hands-on experience in writing complex SQL queries and optimizing database performance. • Skilled in identifying performance bottlenecks and optimizing Java applications using profiling tools and best practices for efficient memory and CPU utilization. • Responsible for designing and developing an automated framework which creates and automates the development process in Data Lake. • Created on-demand tables on S3 files using Lambda Functions and AWS Glue using Python and PySpark. • Migrated an existing on-premises application to AWS. Used AWS services like EC2 and S3 for small data sets processing and storage, experienced in maintaining the Hadoop cluster on AWS EMR. • Worked on AWS Data Pipeline to configure data loads from S3 to into Redshift. • Install and configure Apache Airflow for S3 bucket and Snowflake data warehouse and created DAGs to run the Airflow.

• Interacting with dev team, business team, data analyst, and data architects. • Work with stakeholders to assist in the data-related technical issues and support their data infrastructure needs. • Experience in developing and deploying serverless applications using AWS Lambda with Java, enhancing scalability and reducing operational overhead. • Automate manual ingest processes and optimize data delivery subject to service level agreements, work with infrastructure on re-design for greater scalability. • Developed data pipeline using SQOOP, HQL, Spark, AWS Glue and other Hadoop technologies to ingest data from RDBMS to Hadoop. • Automate cloud infrastructure using Terraform scripts and develop python scripts for data pipelines. • Developed custom ETL solutions, batch processing and real-time data ingestion pipeline to move data in and out of Hadoop using Pyspark and shell scripting. • Handle huge datasets with Partitions, Spark in Memory features, and Broadcasts in Spark with Scala and Python. • Developed Python and Pyspark scripts to transfer data from on-premises storage to AWS S3. • Migrated data from on-premises to AWS storage buckets

Client : - Currency Holdings limited • Developed multiple MapReduce jobs in Python for data cleaning and preprocessing and assisted with data capacity planning and node forecasting. • Involved in design and ongoing operation of several Hadoop clusters and Configured and deployed Hive Meta store using MySQL and thrift server • Implemented and operated on-premises Hadoop clusters from the hardware to the application layer including compute and storage. • Uploaded and processed more than 30 terabytes of data from various structured and unstructured sources into HDFS (AWS cloud) using Sqoop and Flume. • Designed custom deployment and configuration automation systems to allow for hands-off management of clusters via Cobbler, FUNC, and Puppet. • Prepared complete description documentation as per the Knowledge Transferred about the Phase-II Talend Job Design and goal and prepared documentation about the Support and Maintenance work to be followed in Talend. • Deployed the company's first Hadoop cluster running Cloudera's CDH2 to a 44-node cluster storing 160TB and connecting via 1 GB Ethernet. • Debug and solve the major issues with Cloudera manager by interacting with the Cloudera team. • Modified reports and Talend ETL jobs based on the feedback from QA testers and Users in development and staging environments.