Freelancer profile translated to English.

Description

I am a Data Engineer with 15 years of development experience, including 8 years in Big Data & Cloud.

I have worked on several big data projects for various clients, most recently as a Tech Lead Azure Data Engineer at SNCF VOYAGEURS since January 2022.

Languages

English
Fluent
French
Native or bilingual

Workplace preferences

Can work on-site

Paris (up to 50km)

SNCF Voyageurs
Tech Lead Big Data/Azure/Scala/Python/Spark
TRANSPORTATION
January 2022 - Today (4 years and 5 months)
Saint-Denis, France
As a Tech Lead Big Data, I lead several strategic projects, including a flagship project for SNCF Voyage NAOMI. My job is to process and optimize large volumes of customer data (140 GB of raw JSON data daily), with reconciliation on 16 TB of historical data covering 4 years. I ensure the complete processing of data via a high-performance chain on the Azure cloud, up to its provision in Synapse.

Collaboration and coordination with business teams:
Close collaboration with Product Owners (PPO) to define and formalize functional/technical specifications based on business needs.
Data analysis and reverse engineering to optimize existing processes and ensure the adequacy of technical solutions to expectations.
Processing of large volumes of data:
Design and implementation of solutions for processing 150 GB of daily data in JSON format, with reconciliation on 10 TB of historical data.
Technical skills:
Development and optimization of the data processing chain on the Azure cloud, from ingestion to provision in Azure Synapse.
Use of Python, Scala and Apache Spark to ensure high performance and optimal scalability.
Optimization and tuning:
Continuous improvement of performance and optimization of code to meet the key objective of complete processing in less than 4 hours, in accordance with business requirements.
Incident management and production support:
Resolution of critical incidents in production, identification and correction of bottlenecks, and optimization of performance in real time.
Quality standards and best practices:
Contribution to the development of code quality standards and implementation of best development practices within teams.
Scala Python 3 Spark PySpark Databricks Azure Synapse Analytics Pandas Ansible
Société Générale
Tech Lead Azure Data Engineer
BANKING AND INSURANCE
November 2019 - Today (6 years and 7 months)
Paris, France
Redesign of the backend part of the C-profit application initially written in pl/sql. It is a reporting application that covers one of the most critical needs of SG, namely making the right decisions to increase the bank's revenues in view of the profitability of customers and the performance of sales.

▪ Setting up the architecture of the redesign project, configuration of the various bricks used.
▪ Setting up the initial load. Snowflake schema, spark jdbc reading of the data mart on exadata + sqoop jobs for integrating dimensions from exadata, and processing everything via spark.
▪ Spark jobs for processing files in XML/CSV/JSON format.
▪ Jobs for feeding the SSAS cube via an intermediate step based on postgresql
▪ Integration of spark with cucumber for BDD.
▪ Management of the customer tree and the structure of the departments (SG Business Unit) via GraphFrame.
▪ Scheduling job flows via Oozie.
▪ Migration of the application from the on-premise lake to Azure HDInsight.
▪ Setting up the Azure Data Factory pipeline for job synchronization.
▪ Setting up spark standalone on Azure VMs via Ansible/Terraform, for daily job testing.
Azure Data Factory Azure HDInsight Databricks Spark Scala Hadoop Terraform Ansible
Natixis
Tech Lead Big Data/Scala/Spark
BANKING AND INSURANCE
August 2018 - November 2019 (1 year and 3 months)
Within the IT department, a number of Hadoop applications have been developed in Scala/Spark. They perform ETL and calculation processing for the Risk department. These applications evolve over the months according to regulatory changes and business needs.

This is a consulting, development and development support mission around the tools of the Hadoop ecosystem in order to evolve the applications developed in Scala/Spark towards real-time streaming.
Scala Spark Streaming Hive HBase spark kafka Jenkins