Fouad B.

Context: Integrated into the Transverse IT unit, I contributed to the implementation of a Datahub by leading the implementation of application scheduling. My role was to provide expertise in workflow orchestration to support the migration of Control M jobs to Apache Airflow as part of the data processing infrastructure modernization.

• Analyze the existing Control-M setup: inventory of jobs, dependencies, and schedules.

• Design the target architecture under Apache Airflow, leveraging the existing setup.

• Migrate workflows from Control-M to Airflow.

• Ensure service continuity during the transition phase.

• Write project documentation (functional specifications, installation, operation).

• Conduct a complete audit of the existing system and define a detailed migration plan.

• Develop and validate functional Airflow DAGs.

• Develop and execute a testing and acceptance plan.

• Ensure knowledge transfer and handover to internal teams.

Context: Assisting managers of three projects within the Engineering, Quality, and Finance entities. By proposing modern data architecture solutions, help data engineers adapt to complexity. Collaborating with data governance teams, the objective is to align the company with the group's policy.

Responsibilities:

• Facilitate workshops with VPs and FDMs to define operational processes for the domains (Engineering, Quality, Finance) and data flow.

• Assist the Functional Data Manager in defining Business Objects and the roadmap for data ingestion.

• Establish specifications for data flows from source systems to the Data Warehouse (DWH).

• Manage the Backlog in collaboration with the Scrum Master to develop user stories and plan sprints, in terms of requirements and deliverables.

• Collaborate with the tech leads of each squad to resolve issues and ensure milestone delivery.

• Implement and oversee the data quality process in cooperation with various stakeholders (data governance, architects, Business managers).

• Write test scenarios in Jira to validate the functionality of each sprint.

• Provide support to end-users for delivered increments.

• Document dashboards functionally with a HowTo approach to clarify their usage.

Deliverables:

• Specifications, documentation, test cases, code, code review

• Writing user stories, backlog refinement, and roadmap prioritization

Technical Environment:

• Data Modeling, databases, DataMarts, Data Governance, Data Security, DBT, BI tools, ETL Solutions, Data Science, DataOps, MLOps, Figma, MagicDraw UML

• Talend, DataGalaxy, SSAS, Microsoft Fabric, Power BI, Ms SQL Server, MinIO, Kafka, Trino, ELK

Context 1 (SGSS entity): Migration of valuation flows from dollar Universe to Airflow.

• Design and development of a core orchestration framework based on Airflow and Java Spring layers.

• Adaptation and migration of current processes.

• Implementation and development of functional business rules.

• Integration of various internal SG frameworks (authentication, authorization management, monitoring, workflow...).

• Implementation of CI/CD pipelines across all features.

• Setting up automated tests and adhering to Craft development practices (TDD, BDD, Clean code...) during implementations.

Context 2 (ITIM entity): RAF for the Migration of BDH collections (Hortonworks platform (old platform) + ELK cluster) to LUCID / Evolutions (Cloudera platform (new platform)).

• Take over scripts for accessing AWS S3 buckets or APIs from BDH to implement them on Lucid.

• Implementation of the solution on LUCID: Development and delivery of the solution.

• Take over Streaming treatments developed by other services to integrate them into our standardized Streaming chain / Finalize the migration of other streaming chains.

• Contacting issuers for synchronization on the management switchovers to Kafka LUCID and other components. Maintain a regular progress update with the CLK entity manager and Big Data architectures.

Deliverables:

Description of deliverables: code, documentation, code review

Collection components (scripting, ...)

Operating procedures for collection services

Acceptance certificates for collection services

Progress reports for services migrated to Lucid

Technical Environment:

Java, JPA, Spring

Hive, Jenkins, Kafka, Scala, Scripting, DBT, Microsoft Fabric

AirFlow/Python/FastAPI – JSON - POSTMAN - Jenkins/git/sonar - Postgre/Oracle/SQL - MongoDB

Docker/Kubernetes - Elastic Kebana - SOAP and REST - Swagger