Youssef Chaabani

Mission: Design, Implementation, and deployment of jobs for ingesting, processing, and loading data into datalakes from multiple sources and via different data storage methods.

Activities performed:

• Provide, process, and analyze (VIA SPARK) data streams for major clients and internal use.

• Normalize data from golden sources to create models (or business views) ready for reporting.

• Implement specific datasets (or usage views) for reference data.

• Monitor data ingestion and integration jobs and processes.

• Gather requirements, analyze, create design documents, and perform needs and impact analysis.

Tasks performed:

• Implementation using Scala of Spark jobs for transforming and loading large volumes of data into target datalakes.

• Participation in the implementation and configuration of Spark cluster architecture and environment.

• Orchestration and scheduling of Spark data integration jobs.

• Adaptation of an "ingestion as a service" data tool (IGD) for internal needs.

• Implementation of an SSAS cube for data aggregation, exposing a semantic model for Power BI, enabling on-demand queries on large data volumes.

• Implementation of monitoring and auditing scripts for data integration jobs.

• Configuration of DevOps pipelines for implemented Spark projects.

• Participation in BA testing for Data projects.

Technical environment: Scala, Apache Spark Framework, HIVE, Oozie, Data Lakes, Maven, Cloud, Cloudera, REST API, Kafka, Kibana, Microsoft SSAS, Microsoft Power BI, Jenkins, AWX, Jira.

Mission: Design, Implementation, and deployment of data frameworks for ingesting, processing, and analyzing data from multiple sources and via different data storage methods.

Activities performed:

• Design and implementation of batch/real-time Declarative-Data Pipelines (DAG) and ETLs.

• Provide, process, and analyze (VIA SPARK) metric data for major clients and internal use.

• Extraction, transformation, and integration of data from various sources (relational and non-relational databases, Data Lakes, files).

• Data anonymization.

• Gather requirements, analyze, create design documents, and perform needs and impact analysis.

Tasks performed:

• Implementation using JAVA of data ingestion pipeline steps.

• Loading data from multiple sources into Data Lakes built on AWS S3 buckets.

• Use of the Amazon JAVA SDK for data collection via S3 and exposing this data via REST APIs (data queried by SQL).

• Design, implementation, and deployment of an automatic placement Machine Learning model (using Spark's MLlib) based on placement history, for the iObeya platform's Visual Management needs.

• Creation of business intelligence dashboards for data reporting.

• Configuration of DevOps pipelines for internal ETL projects.

• Creation of CI/CD pipelines based on Gitlab CI usage.

• Development and testing of REST APIs.

• Participation in QA testing for Data projects.

Technical environment: JAVA, Apache Spark Framework, SQL RDBMS (MySQL, Oracle), NoSQL (MongoDB, Redis), Python Jupyter, Pandas, Scikit-Learn, Machine Learning Stack, Data Lakes, AWS (S3, RDS, Glue, EC2, EKS), Microsoft Power BI, Docker, Kubernetes, Lens, Helm, Gitlab CI, Jenkins, Jira.

Mission: Development of a Business Intelligence module for managing the semi-annual liquidity balance.

Activities performed:

• Creation of ETL JOBS.

• Selection, collection, and cleaning of financial liquidity contract data.

• Creation of a task script to automate the process of formulating, generating, and emailing balance sheets in PDF format.

• Creation of business intelligence dashboards for reporting the generated balance sheet data.

Technical environment: Business Intelligence, MS BI, Sql Server, IBM Cognos suite, C#, SVN.