Ming ZHAO

As part of the XPA (eXtended Planning & Analysis, Finspark train of the IT Department) project, as the leader of the data engineering team, I collaborated with the Product Owners to design and implement a new AWS architecture from scratch, focusing on serverless and event-driven principles. We ran the OnPrem system in parallel and gradually migrated it to AWS. I fostered and promoted an IT culture within the team, embracing diverse mindsets and tools. New technologies were tested and applied to avoid reinventing the wheel and to enhance the platform's stability and robustness. AWS Cloud Architecture: Design and implementation of a new AWS architecture, integrating collaboration with OnPrem systems and a progressive migration strategy. Security and Compliance: Strengthening of security standards and banking compliance for a resilient platform. Technological Innovation: Introduction of new technologies to replace custom solutions and enhance system robustness. Team Empowerment: Pioneering team in implementing continuous deployment (CI/CD) and automated alert systems. Technical Documentation: Creation of architecture diagrams using MindMapping, Drawio, and Ilograph. DevOps Practices: Maintenance of Git repositories, contribution to enterprise Terraform modules, and setup of CI/CD pipelines with GitLab, Jenkins, and Atlantis.

Led a team in a Big Data Visualization project focused on developing a pipeline to provide daily data access to internal and external clients. Data and ETL Architecture: Stabilization and optimization of ETL pipelines on Spark for daily data exposure to internal and external clients. Cloud Migration: Migration of the OnPrem infrastructure to AWS/Azure. DevOps Collaboration: Integration of Airflow for pipeline orchestration and development of internal tools to streamline daily development. Technical Leadership: Task prioritization according to the roadmap and team management during the manager's absence.

As a Big Data engineer in Squad team, I participated in an internal project where I maintained an ETL pipeline for data visualization. ETL Pipeline Optimization: Reduced processing time from 3 hours to 25 minutes using Spark and Apache Oozie. Spark Performance Improvement: Analyzed Spark DAGs to optimize performance from over 90 minutes to 25 minutes. CI/CD: Maintained CI/CD process automation using Jenkins, Ansible, and Git for seamless deployment.

As a member of the data engineering team, I collaborated to understand and utilize the internal application, Data Loader, in support of IoT data processing with business stakeholders. Data Streaming Pipeline: Design of a Kafka/Spark pipeline to process real-time messages and store them in the Data Lake (HDFS). Spark Optimization: Collaboration with a Spark expert to improve the performance of ETL jobs.

Developed a PAAS (Platform as a Service) using Apache Kafka, Spring Boot, and Apache Cassandra to manage the lifecycle (creation, modification, suspension, deletion) of contracts between the postal service and the general public. Contract Management via a PAAS: Development of a Spring Boot and Cassandra solution for contract lifecycle management with Apache Kafka. Documentation and Training: Setup of the Spring project structure and training of the team. User Story Mapping: Use of MindMapping to convert User Stories into a clear map, facilitating understanding and detecting missing cases.

For 3 years, I worked as a Java EE developer and participated in multiple web application projects. I have gained strong expertise in Java 8, object-oriented programming (OOP), data structures and algorithms, web application architecture and libraries, and have a good understanding of SQL.