Freelancer profile translated to English.

Description

Data Engineer with +10 years of experience in creating robust, reliable and scalable applications, designing and deploying data pipelines in a datalake both on premise and in the cloud. Excellent knowledge of architecture and performance, my versatility allows me to intervene effectively and quickly in quite varied contexts.

Active, autonomous with an excellent sense of communication and teamwork, always in search of new data challenges in order to meet the unique needs of different clients.

Industry field of expertise

Languages

English
Fluent
French
Native or bilingual

Workplace preferences

Remote only

Primarily works remotely

Allianz IARD
Senior Data Engineer
BANKING AND INSURANCE
February 2020 - Today (6 years and 4 months)
Puteaux, France
"Swiss Army knife" within the data BU, my roles are, on the one hand, to provide rapid expertise on various subjects directly related to the Data Engineer (DE) manager, and on the other hand, to be the technical right-hand man of the Data Client unit manager.

Data Engineering / Data architecture
• Setting up data ingestion pipelines in the datalake: orchestration with Luigi, parquet format storage.
• Support for DataScientists (DS): debugging and optimization of PySpark applications
• Proposal and challenge of data ingestion and processing architectures

Data Exposure / Restitution / API
• Exporting data to PostgreSQL/ Kafka (full, delta, upsert)
• Design and deployment of business-oriented REST APIs

DataOps / Cloud Engineer
• Design and maintenance of application deployment templates (Nomad, AKS)
• Deployment and maintenance of a fleet of Jupyter Servers for DS
• Evolution, provisioning and maintenance of the infrastructure on Azure / GitOps Terraform

Production / Run
• Daily check
• Application maintenance, debugging, hot fix.
• Building Grafana dashboards for monitoring

Other
• Onboarding new DE / DS to the datalake, configuring their work tools
• Conducting PoCs / Tests of new technologies in the datalake
• Proposing technical solutions and developing utilities to facilitate the work of collaborators
• Migrating applications to Cloud Azure

Example projects
• Know Your Customer RBE: Ingestion of RBE data from the INPI API to determine the Beneficial Owners linked to Allianz clients
• Digital Assessment: Dev of a Flask API for analyzing images of buildings to assess their quality for insurance contracts. Application of AI model in real time. (Response time challenge)
• Dev of a Kafka / KafkaStream app for real-time calculation of email processing and classification KPIs
• Process mining in Celonis, data routed via a kafka pipeline
Data Engineering Kubernetes PySpark Microsoft Azure Java Apache Kafka Docker Github Actions Gitlab CI/CD Prometheus Grafana luigi Helm AKS Big Data
Societe Generale Corporate and Investment Banking
Confirmed Date Engineer
BANKING AND INSURANCE
April 2018 - February 2020 (1 year and 10 months)
Colombes, France
Ingestion and processing of data in a Hadoop cluster with Spark and the Hortonworks stack, daily exposure of data in a SQL database.

Ingestion

• Ingestion of financial transaction data in csv format, from NAS to HDFS
• Cleaning and validation of data (null values, recovery, Etc.)
• Storage in HDFS, partitioning

Processing, consolidation

• Design and development of spark applications for formatting and transforming data according to the business model
• Development of Spark applications to enrich financial data (Order, AUM, Position) by cross-referencing it with reference data (Funds, Investors, ...). The challenge is to choose and use the best Spark APIs according to the situations.

Data Exposure / Restitution

• Exporting data to MS SQL Server databases using Sqoop
• UPSERT management

Data pipeline / scheduling

• Design and development of ingestion and processing workflows with Oozie
• Setting up schedulers via Oozie coordinators

Performance / job optimization

• Improved performance of an ingestion and validation job, processing time reduced from 2h (OOM sometimes) to 10 min.
• Optimizing the performance of a job exporting hundreds of millions of records from Hive to MS SQL Server. The challenge here is to find the optimal configuration of the Sqoop job so that it runs in the best possible time

• Proposal of solutions to implement business constraints
Spark Java Hive Sqoop Oozie SQL Git Scrum Maven
ATOS
FullStack Java / CLOUD Developer
CONSULTING AND AUDITS
March 2014 - March 2018 (4 years)
Issy-les-Moulineaux, France
Alien4Cloud project, DevOps platform for application lifecycle management.

• Agile (SCRUM), continuous delivery, opensource, TOSCA
• Active product developer: Back (J2EE, Spring, ElasticSearch) + Front (HTML5, AngularJS)
• Deployment on Clouds (Aws EC2, Openstack, Docker)
• Participation in establishing the backlog, proposing and designing features
• GitHub management: pull-requests, Code-review, merge, communication
Java AngularJS Elasticsearch Spring boot Spring MVC Spring Security Git Maven Kubernetes Amazon Web Services Docker