Ibrahima Diattara

Project: Migration of a Data platform to Google Cloud Platform (GCP) Participation in a project to migrate an existing data architecture to GCP, with the aim of modernizing, increasing performance, and simplifying the maintenance of data processing. Achievements: ✓ Migration of data from Amazon S3 to Google Cloud Storage (GCS) ✓ Transformation of Apache NiFi pipelines into Airflow DAGs developed in PySpark ✓ Migration of external SQL Server tables to external BigQuery tables ✓ Rewriting and optimization of SQL Server queries to BigQuery's SQL language ✓ Implementation of validation tests (functional and technical) ✓ Writing technical documentation for new workflows

Technical environment: Nifi, SQL Server, BigQyery, Google Cloud Storage, Airflow, PySpark

Project: TEGG – Piezoelectric data processing Project aimed at collecting, processing, and analyzing data from sensors measuring the level of groundwater (piezoelectric data), in order to guarantee their quality and produce reliable analyses. Achievements: ✓ Optimization of Nifi flows ✓ Developing Custom Processors Java in Apache NiFi ✓ Automatic retrieval of files from Amazon S3 ✓ Developing Neo4j stored procedures in Java ✓ Integration and orchestration of data flows via Apache NiFi ✓ Data quality control with application of specific business rules with Nifi ✓ Implementation of an advanced processing layer with Spark in Scala (aggregations, anomaly detection) ✓ Insertion of enriched Neo4J data ✓ Indexing application logs in elasticsearch ✓ Setting up Data Table type visualizations in kibana to monitor the evolution of processing

Technical environment: Nifi, Spark/Scala, Neo4j, Amazon S3, Java and Groovy, Python and Bash

Data Engineering Professor (Databricks, Google Cloud, Azure, Spark/Scala, Nifi, KAFKA, ELK,

Airflow/PySpark)

Metrology API Project: Development of a Python/Flask application with Swagger interface for flow creation.

Achievements: Automation of NiFi flow creation, Kafka topics, Ranger policies, HDFS directories, and complete Elasticsearch index management (Kibana index patterns, ILM, field security).

Metrology Monitoring Project: Supervision of NiFi/Kafka clusters via dashboards to anticipate data loss.

Achievements: Configuration of PrometheusReporting (NiFi), metric collection via Telegraf and Jolokia, Kafka Lag Exporter deployment, Grafana dashboard creation, NiFi processor and Kafka partition tuning.

HDF Upgrade Project: Impact study and post-upgrade optimization of the HDF platform.

Achievements: Analysis of NiFi flows, Kafka behavior (producers/consumers), and performance optimization after update.

DHR Usage Layer Project: Valorization of the technical HR Business View.

Achievements: Automation of Hive table management, partition updates, indexing of application logs in Elasticsearch.

Technical stack: NiFi, Kafka, Python, Flask, Ranger, Elasticsearch/Kibana, Prometheus, Telegraf, Jolokia, Grafana, HDFS, Spark/Scala, Hive/HUE, Jenkins, SonarQube, Nexus