Caroline G.

Project: Data Engineering consultancy via Fcamara, allocated to a financial sector client. Key Achievements & Responsibilities: • Implemented Data Mesh architecture with Medallion Model layers and high‑performance ETL processes. • Leveraged GCP services: BigQuery, Composer, Cloud Storage, Cloud Run, Pub/Sub, Dataform, Dataflow, Dataplex, Datastream. • Developed pipelines with Python, SQL, Apache Beam and IaC tools (GitHub, Docker). • Connected diverse data sources: Oracle, PostgreSQL, Firestore, SFTP servers, CSV files, and APIs. • Created CI/CD automations using GitHub Actions. • Designed and supported ingestion models from transactional databases to Bigquery: Full, Incremental, and CDC. • Built observability solutions with Apache Airflow (VM and cloud) integrated with Google Chat for automatic alerts. • Enhanced performance/readability of Airflow pipelines (OOP, optimized libraries) and SQL queries (CTEs, platform‑specific functions, clean structure). • Developed AI‑powered automated data cataloging integrated into Dataplex for governance. • Maintained frequent contact with stakeholders to align on data usage, update frequency, and Lake health. • Documented processes in Confluence; applied Agile (Kanban–Jira) methodology. • Troubleshot legacy architecture failures; created single source of truth tables for critical data. • Extensive use of Git commands, shell scripting, and bash files

• Cloud Infrastructure: Experience in multi‑cloud environments with a focus on AWS (S3, Glue, Athena, DynamoDB, ECS, Fargate, Lambda, EC2, EFS), GCP (BigQuery), and Azure (Users and Groups management for Power BI access control). • Data Pipelines (ELT/ETL): Design and maintenance of data pipelines using Apache Airflow and DAGs, applying layered architecture (Bronze, Silver, Gold) with data ingestion via APIs and integration with Data Lake and Data Warehouse tools. • Python Programming: Development of scripts, libraries, and object‑oriented solutions for automation and data ingestion; extensive use of pandas, boto3, and awswrangler for dataframe manipulation. • APIs: Consumption of third‑party APIs and development of an internal REST API with Flask for data sharing between teams. • Version Control & Automation: Source control with Git and GitHub; CI/CD integration via GitHub Actions and AWS CodePipeline for automated DAG and infrastructure deployments. • Data Modeling: Creation of datasets using star schema or snowflake schema standards, with entity relationship design (ERD). • Data Products: Development, control, and maintenance of datasets, dashboards (Power BI, Looker Studio), spreadsheets (Google Sheets), and data marts; managing data lifecycle and access permissions (GCP IAM, Alerts). • Documentation & Governance: Preparation of flowcharts, policies (NDA and other standards), technical documents, and knowledge base articles (ServiceNow); access control in line with governance policies. • Observability & Monitoring: Implementation of Data Lineage solutions and monitoring of data product usage. • Communication & Collaboration: Supporting internal clients via tickets; requirements gathering with stakeholders; delivering SQL training for business teams; participating in Agile sprints; and liaising with vendors for technical alignment.

• Analytical Layer Modeling: Design of analytical layers (data marts) in BigQuery. • Data Warehouse Management: Administration of the BigQuery Data Warehouse and control of Data Swamp scenarios. • Documentation & Policies: Creation of documentation and policies for data capture and access to data products. • Access Control: Management and review of permissions for GCP services (IAM). • Business Communication: Engagement with internal business areas to understand data needs and usage requirements. • GCP Resources: Utilization of various GCP tools and features (Alerts, Logs, BigQuery). Other Activities: • Delivery of data training sessions covering BigQuery, SQL, Google Sheets, and Looker Studio.

Data Analysis: Backlog management, creation of risk and comparison dashboards, database querying, and presentation of results. Communication: Delivery of onboarding sessions for new employees; administrative support to project managers (acting as a focal point for technical support); proposals for Customer Experience initiatives; creation of knowledge base articles (ServiceNow); development of flowcharts. Development: Automation of routines through scripts, dashboards, and reports.

Support for customers with enterprise-level support plans or support plans combined with a subscription package of products/services. Assessment of the customer’s issue or inquiry and offering the best available support option. Case creation: gathering contact and issue details, registering and routing the case, and informing the customer of the SLA. Case follow-up: liaising with engineers (in Portuguese or English) or support queue managers, and aligning expectations for callbacks with the customer.