Omer khan Mohammed

Configured and maintained Alation Data Catalog to enhance metadata governance and compliance. Built and optimized data ingestion pipelines using ADF, Databricks, and Snowflake. Developed PySpark and Python workflows for real-time and batch transformations. Improved distributed query performance using Starburst Enterprise. Automated ETL workflows with Airflow and dbt, reducing manual intervention by 60%. Worked with Starburst Enterprise to enable data federation and optimize distributed queries across data lakes and cloud data warehouses like Snowflake and AWS S3. Assisted in Starburst configuration and query performance tuning to improve data accessibility for analytics teams. Integrated data pipelines in Databricks and Snowflake with downstream visualization tools like Tableau and data prep solutions like Alteryx.

Led Alation implementation and configuration, enabling enterprise-wide metadata management. Designed and developed metadata pipelines for seamless data cataloging and governance. Created custom Alation workflows for metadata validation, improving catalog accuracy by 40%. Developed data security policies and access control mechanisms within Alation and Azure Data Lake. Implemented infrastructure provisioning and automation with Terraform, ensuring consistency and repeatability across environments. Developed and tuned Elasticsearch indexes to support low-latency search and analytics Collaborated with BI teams to prepare and model data optimized for Tableau dashboards, ensuring data pipelines aligned with visualization requirements. Supported Alteryx workflows by enabling seamless data integration and ensuring clean, structured datasets for advanced data prep and analytics tasks. Built real-time Kafka pipelines to integrate metadata with Snowflake for better data traceability. Managed technical lineage and metadata onboarding across multiple cloud and on-prem data sources. Developed and deployed Python-based automation scripts to streamline metadata ingestion.

Designed and implemented Hadoop-based ETL pipelines for processing multi-terabyte datasets. Developed PySpark-based transformations for large-scale structured and semi-structured data. Configured Hive, Spark, and MapReduce for high-performance data processing and analytics. Built automation scripts using Python and SQL to improve data ingestion efficiency. Designed and implemented PySpark-based ETL pipelines in Databricks for transforming large-scale datasets. Created CI/CD workflows using GitHub and Jenkins for automated deployment of ETL and data pipeline code. Applied data governance and security best practices to ensure compliance with organizational standards. Provided troubleshooting support for Starburst, Tableau, and Alteryx users, ensuring data reliability and performance across tools. Utilized Python extensively for building reusable data transformation modules, data validations, and pipeline orchestration. Collaborated on a project utilizing Palantir Foundry for data workflow design and analytics dash boarding, contributing to module development and integration efforts.

Designed data governance frameworks leveraging Collibra and Alation. Developed data validation scripts to ensure data quality before onboarding into catalogs. Built Python-based automation solutions for metadata tagging and transformation. Integrated data lineage tools with Alation to enhance data cataloging. Provided technical support and troubleshooting for metadata ingestion issues. Managed API integrations between Alation and enterprise data sources. Assisted in migration projects, moving legacy data pipelines to AWS. Conducted knowledge-sharing sessions to train teams on data governance best practices. Created CI/CD workflows using GitHub and Jenkins for automated deployment of ETL and data pipeline code. Applied data governance and security best practices to ensure compliance with organizational standards. Provided troubleshooting support for Starburst, Tableau, and Alteryx users, ensuring data reliability and performance across tools. Utilized Python extensively for building reusable data transformation modules, data validations, and pipeline orchestration. Collaborated on a project utilizing Palantir Foundry for data workflow design and analytics dash boarding, contributing to module development and integration efforts.