Sourav Debthakur

Senior Data Engineer with 18+ years in tech and 4+ years of hands-on experience in designing scalable, cloud-native data solutions using PySpark, Snowflake, AWS, and Azure. Proven ability to architect and deploy robust data pipelines across large volumes of structured and unstructured data. Expertise in building ETL workflows with Matillion, implementing real-time ingestion using Kafka, and enabling machine learning with tools like scikit-learn and PySpark MLlib. Delivered performance gains of up to 40% through Spark tuning and Snowflake optimization. Strong track record in stakeholder collaboration, Agile delivery, and working across global, cross-functional teams. Industry experience includes banking, retail, insurance, and utilities. Passionate about solving complex data challenges, enabling data-driven decision-making, and bridging the gap between engineering and analytics. Prior to my transition into data engineering, I spent over a decade managing large-scale QA initiatives and Agile delivery programs across the UK, Finland, and India. As a certified Test Manager and Scrum Master, I led cross-functional teams of 50+ people, implemented CI/CD testing frameworks, and coached product teams on Agile best practices. This background gives me a unique edge in bridging the gap between technical engineering and product delivery, especially in enterprise environments. 📌 Core Tech: Python, PySpark, Snowflake, Databricks, EMR, ADF, Matillion, Kafka, Airflow, scikit-learn, AWS, Azure